Method and device for extracting main blood vessel regions based on blood vessel images

Abstract

A method for extracting a major vessel region from a vessel image executed by a processor may include the following steps: extracting an entire vessel region from the vessel image; extracting the major vessel region from the vessel image based on a machine learning model for extracting the major vessel region; and connecting separated portions of the vessel based on the entire vessel region, thereby correcting the major vessel region.

Description

Technical Field

[0001] The following description relates to a method and apparatus for extracting major blood vessel regions based on a blood vessel image. Background Art

[0002] Interventional procedures using catheters to insert stents and other devices for treating cardiovascular, cerebral, and peripheral blood vessels have become widespread. As a tool for routing stents through guidewires or catheters for delivery into blood vessels, visual information based on medical images such as angiography and tactile information based on fine manual senses are utilized to guide the guidewire to the distal end of the diseased vessel.

[0003] Recently, remote robots and other devices are being developed to reduce the physical burden on surgeons, such as radiation exposure, and to precisely control surgical tools. Although surgical robots have been certified by the U.S. Food and Drug Administration (FDA) and are being commercialized, research is still needed to adapt new tools to perform simple minor surgeries. Currently, operations such as moving a guide wire backward or rotating it at a predetermined angle are not performed directly by the surgeon but by a robot, but this is not a high percentage of minor surgeries. Summary of the Invention

[0004] Technical methods to solve problems

[0005] A method for extracting a major vessel region from a vessel image executed by a processor according to an embodiment may include the following steps: extracting an entire vessel region from the vessel image; extracting a major vessel region from the vessel image based on a machine learning model for extracting the major vessel region; and connecting separated portions of the vessel based on the entire vessel region, thereby correcting the major vessel region.

[0006] According to one aspect, the machine learning model may be a machine learning model trained for predetermined shapes of major blood vessels. Specifically, the machine learning model may be a machine learning model trained for the shape of at least one of the right coronary artery (RCA), the left anterior descending artery (LAD), and the left circumflex coronary artery (LCX).

[0007] In addition, the method for extracting a major blood vessel region may further include the step of detecting a portion of the major blood vessel region where the blood vessel is separated. The step of detecting the portion of the major blood vessel region may include the step of determining a region between blobs corresponding to the major blood vessel region where the shortest distance between the blobs is less than a threshold distance as the portion of the major blood vessel region.

[0008] Furthermore, the step of correcting the main blood vessel region may include the following steps: generating a connecting line connecting regions between blobs corresponding to the main blood vessel region whose shortest distance between blobs is less than a threshold distance; and connecting the separated blood vessel portions in the main blood vessel region based on regions corresponding to the connecting line in the entire blood vessel region, thereby correcting the main blood vessel region. Specifically, when there are multiple regions that can connect the separated blood vessel portions via the connecting line corresponding to the entire blood vessel region, correcting the main blood vessel region based on the region with the shortest distance that can connect the separated blood vessel portions among the multiple regions.

[0009] According to an embodiment, the method for extracting a main blood vessel region may include the following steps: converting RGB values ​​of the blood vessel image into grayscale levels; and normalizing the blood vessel image converted into the grayscale levels.

[0010] In addition, the step of extracting the entire blood vessel region may include the step of extracting the entire blood vessel region based on a partial blood vessel image generated from the entire blood vessel image, and the step of extracting the main blood vessel region may include the step of extracting the main blood vessel region based on a partial blood vessel image generated from the entire blood vessel image.

[0011] According to one embodiment, the step of correcting the main blood vessel region may include the following steps: determining the separated portion of the blood vessel in response to a user's input of a positioning of the main blood vessel region; and correcting the main blood vessel region at the target position based on surrounding main blood vessel regions adjacent to the separated portion of the blood vessel.

[0012] Effects of the Invention

[0013] The method for extracting major blood vessel regions according to an embodiment corrects the major blood vessel regions extracted from a blood vessel image using a machine learning model, and can provide uninterrupted images of the major blood vessel regions, thereby enabling surgeons to perform accurate diagnosis and minor surgeries.

[0014] Furthermore, the method of extracting a main blood vessel region according to an embodiment enables surgeons to accurately identify blocked or narrowed portions in main blood vessels by removing images such as microvessels from a blood vessel image and isolating only the blood vessel region of interest and displaying it as an image. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 1 and 2 are drawings illustrating operations of a medical tool insertion device and a medical tool according to an embodiment.

[0016] Figure 2 FIG. 1 is a flowchart illustrating a method for extracting and correcting a main blood vessel region from a blood vessel image according to an embodiment.

[0017] Figure 3 FIG. 1 is a diagram illustrating cardiac vessels to be extracted as main vessel regions according to an embodiment.

[0018] Figure 4 FIG. 4 is a flowchart illustrating a preprocessing method for extracting main blood vessel regions and entire blood vessel regions according to an embodiment.

[0019] Figure 5 FIG. 1 is a diagram illustrating correction of a main blood vessel region based on an entire blood vessel region according to an embodiment.

[0020] Figure 6 FIG. 1 is a diagram illustrating extracting a corrected main blood vessel region from a blood vessel image according to an embodiment.

[0021] Figure 7 FIG. 1 is a block diagram showing a schematic structure of an apparatus for extracting major blood vessel regions according to an embodiment. DETAILED DESCRIPTION

[0022] The specific structure or function description of the disclosed embodiments is for illustrative purposes only and can be changed and implemented in various forms. Therefore, the embodiments are not limited to the specific disclosed forms, and the scope of this specification includes changes, equivalents or substitutes included in the technical spirit.

[0023] The terms "first" or "second" can be used to describe multiple components, however, these terms are only used to distinguish one component from other components. For example, the first component can be named the second component, and similarly, the second component can also be named the first component.

[0024] When it is stated that a component is “connected” to another component, it can be directly connected to or attached to the other component; however, it can also be understood that there are other components between them.

[0025] Unless otherwise specified in the content, the singular includes the plural. In this specification, the terms "including" or "having" are used to express the presence of the features, numbers, steps, operations, components, accessories, or combinations thereof described in the specification, and do not exclude the presence of one or more other features, numbers, steps, operations, components, accessories, or combinations thereof, or additional functions.

[0026] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the ordinary meanings understood by those skilled in the art. Commonly used terms that are the same as dictionary definitions should be understood to have meanings consistent with the general context of the relevant technology and should not be overly idealized or interpreted as formal unless explicitly mentioned in this application.

[0027] Figure 1 1 and 2 are drawings illustrating operations of the medical tool insertion device 110 and the medical tool 120 according to one embodiment.

[0028] According to one embodiment, the medical tool insertion device 110 can move the medical tool 120 to a target region of a blood vessel in response to a drive command from a processor. For example, the medical tool insertion device 110 can move the tip of the medical tool 120 to the target region of the blood vessel. The medical tool insertion device 110 can be implemented by a surgical robot, for example, a robot that controls medical tools used in cardiovascular interventional surgery.

[0029] The medical tool 120 is a member inserted into a blood vessel and may include a medical tool disposed at the tip of the medical tool 120 and a medical wire connecting the medical tool to a driving unit. For example, the medical wire includes a catheter or a guidewire. A guidewire is a medical wire used to insert and guide the medical tool to the intended location in the blood vessel. The medical tool may be a surgical tool operated under the control of a physician, such as an introducer kit.

[0030] The medical tool insertion device 110 can use the guidance data to determine the aforementioned drive commands. For example, the medical tool insertion device 110 can operate according to a machine learning model to output drive commands based on the guidance data. A machine learning model is a model designed and trained to receive and output guidance data, and can be implemented, for example, as a neural network model.

[0031] The driving command may be a command indicating operation of a driving portion connected to the medical tool 120 to move and rotate the medical tool 120. The driving command may be a forward command, a backward command, a clockwise rotation command, and a counterclockwise rotation command, but is not limited thereto.

[0032] The guidance data refers to the data that maps the guidance information to the blood vessel image or blood vessel structure image. The blood vessel structure image can be an image obtained by extracting specific blood vessels from the blood vessel image and pre-processing the blood vessel image. Figure 3 Describes a vascular structure image. A vascular image can be an image generated using coronary angiography (CAG) or magnetic resonance imaging (MRI). In a vascular image, not only the blood vessels but also the medical tool 120 can be captured.

[0033] Guidance information is used to guide the movement and rotation of the medical tool 120. For example, it may include information regarding the starting point, transit points, and destination of the medical tool 120 within the blood vessel. The information regarding each point may include, but is not limited to, the image coordinates of the corresponding point in the vascular structure image. According to one embodiment, the guidance information can be intuitively mapped to the vascular structure image. For example, a graphical object corresponding to each target area can be visualized in the vascular structure image. The vascular structure image that visualizes the target area can be referred to as a guidance image.

[0034] For example, the medical tool insertion device 110 may receive guidance data from an external device (e.g., a guidance data provider). The guidance data provider receives and analyzes a vascular image from the vascular imaging device 130 and may generate guidance data from the vascular image. As another example, the medical tool insertion device 110 may be integrated with the guidance data provider. In this case, the medical tool insertion device 110 receives the vascular image from the vascular imaging device 130 and analyzes the received vascular image to generate guidance data.

[0035] The processor of the medical tool insertion device 110 can determine whether to drive the medical tool 120 based on the analysis results of the blood vessel image. The medical tool insertion device 110 analyzes the received blood vessel image to generate guidance data, and can determine a drive command based on the generated guidance data. For example, the medical tool insertion device 110 can select one of a forward command, a backward command, a clockwise rotation command, and a counterclockwise rotation command from the guidance data as an operation command. The drive unit of the medical tool insertion device 110 can be driven based on the selected operation command. For example, the drive unit can move the medical tool 120 forward in response to the forward command. The drive unit can move the medical tool 120 backward in response to the backward command. The drive unit can rotate the guidewire clockwise relative to the longitudinal axis of the guidewire in response to the clockwise rotation command. The drive unit can rotate the guidewire counterclockwise relative to the longitudinal axis of the guidewire in response to the counterclockwise rotation command.

[0036] Therefore, the medical tool insertion device 110 uses the guidance data generated by analyzing the blood vessel image to determine a series of operation commands, thereby moving the tip of the medical tool 120 to the point guided by the guidance data. The medical tool insertion device 110 repeats the operation determination using the guidance data, thereby moving the tip of the medical tool 120 to the final destination area. At the tip of the medical tool 120, for example, after the medical tool reaches the destination area, the medical tool can perform a surgical operation according to the control of the doctor. Figures 2 to 7 The described embodiments relate to a method for extracting a major blood vessel region so that a medical tool operating device can move a medical tool to a target region. However, the embodiments are not limited thereto and may also be a method for extracting a major blood vessel region to allow a surgeon to identify a blood vessel region requiring minor surgery from a major blood vessel.

[0037] Figure 2 FIG. 1 is a flowchart illustrating a method for extracting and correcting a main blood vessel region from a blood vessel image according to an embodiment.

[0038] In step 210, a processor configured to extract a major vascular region may extract an entire vascular region from a vascular image. According to one embodiment, the processor may extract the entire vascular region based on a machine learning model for extracting the entire vascular region. The machine learning model for extracting the entire vascular region may be a model learned using python (anaconda version 2.7.14). The deep learning library used to learn the machine learning model may use the Theano backend of keras, but is not limited thereto. Tensorflow or pytorch may also be used, and the machine learning model may use U-net. U-net is a model developed for patch-based medical image segmentation that can show acceptable performance with a small amount of data.

[0039] According to another embodiment, the processor may detect boundaries based on the grayscale difference between a pixel and adjacent pixels in the blood vessel image, thereby extracting the entire blood vessel region. For example, the processor may detect as a boundary an area where the grayscale of any pixel and its adjacent pixels changes dramatically, or may detect as a boundary a pixel where the grayscale gradient value is greater than a threshold gradient value.

[0040] In step 220, the processor may extract major vessel regions from the vessel image based on a machine learning model for extracting major vessel regions. The machine learning model for extracting major vessel regions may be a machine learning model trained for a specific shape of major vessels. For example, the major vessels may be three cardiac vessels, including one right coronary artery and two left coronary arteries. In this case, the machine learning model may be trained for the shape of at least one of the right coronary artery and the two left coronary arteries.

[0041] For example, the processor can extract the major blood vessel regions based on a machine learning model learned for the shapes of three major blood vessels. The major blood vessels can be learned using Python (Anaconda version 3.5.2). The deep learning library used for learning the major blood vessels is TensorFlow (1.4.0).

[0042] The processor can use PSPNET as a machine learning model for extracting major vascular regions. PSPNET helps learn the context of the entire image and is well-suited for segmenting multiple layers simultaneously. In addition to PSPNET, any deep learning model that uses the entire image size as input can also be used.

[0043] 512x512 images are used as learning data, but when learning is performed in the model, the images are resized to 720x720 and used.

[0044] In step 230, the processor may correct the main vessel region based on the entire vessel region connecting the separated portion of the vessel. Figure 5 and Figure 6 Describes the correction of the extracted main blood vessel regions.

[0045] Figure 3 FIG. 1 is a diagram illustrating cardiac vessels to be extracted as main vessel regions according to an embodiment.

[0046] According to one embodiment, the main blood vessels may be heart vessels, Figure 3 Shown are the aorta that supplies blood to the heart and three coronary arteries that receive blood from the aorta, namely, one right coronary artery 310 and two left coronary arteries 320. The right coronary artery 310 and the left coronary artery 320 supply blood to a large number of microvessels.

[0047] Since there are a large number of microvessels, even if one or two microvessels are narrowed or blocked, in most cases, there is no problem with blood supply. Therefore, the diagnosis of the surgeon usually focuses on determining whether the right coronary artery 310 and the left coronary artery 320 are blocked or narrowed.

[0048] However, because there are many microvessels surrounding the coronary arteries, even experienced surgeons find it difficult to separate the coronary arteries from the microvessels to determine whether the blood vessels are narrowed or blocked.

[0049] Therefore, according to one embodiment, the processor can extract the right coronary artery and the left coronary artery as the main vascular regions based on the machine learning model trained on the shapes of the right coronary artery and the left coronary artery. Therefore, the surgeon can determine whether minor surgery is required on the main blood vessels based on the main vascular regions generated by the processor.

[0050] Figure 4 FIG. 4 is a flowchart illustrating a preprocessing method for extracting main blood vessel regions and entire blood vessel regions according to an embodiment.

[0051] According to one embodiment, after receiving the vascular image from the imaging device in step 410, the processor may preprocess the entire vascular image. By default, the vascular image format used for image processing may be a 512x512 bmp image extracted in DICOM (Digital Imaging and Communications in Medicine) format, but the present invention is not limited thereto. Specifically, the image size may be large enough to distinguish blood vessels. In addition to bitmap (bmp) format, various image formats, such as portable network graphics (png) format, may also be used. The processor may crop all edges of the image and mask file by 20 pixels to eliminate surrounding noise.

[0052] In step 420, the processor may preprocess the vascular image to extract the entire vascular region. According to one embodiment, the processor may generate a partial vascular image from the entire vascular image. In this case, the partial vascular image may be in the bmp format. However, the format of the partial vascular image is not limited thereto. The image size may be large enough to distinguish the vascular regions, and various formats other than bmp may be used.

[0053] The processor converts the RGB values ​​of the partial blood vessel image into grayscale values ​​using an in-house program and normalizes the grayscale image obtained thereby.

[0054] The partial blood vessel image of the entire blood vessel region can have the value of the area not masked as a black and white image be 0 and the value of the masked area be 255, and the image size can be adjusted to 256x256 and used.

[0055] The processor may extract the entire blood vessel region in step 210 from the image pre-processed in step 420 .

[0056] In step 430, the processor may preprocess the vascular image to extract the major vascular regions. According to one embodiment, the processor may generate a partial vascular image from the entire vascular image. In this case, the vascular image and mask of the partial vascular image may be in PNG format, but are not limited to this. The mask PNG file may have values ​​of 0 for unmasked areas and 1 for masked areas. Furthermore, the image size may be resized to 512x512. The processor may extract the major vascular regions in step 220 from the image preprocessed in step 430.

[0057] Figure 5 FIG. 1 is a diagram illustrating correction of a main blood vessel region based on an entire blood vessel region according to an embodiment.

[0058] According to an embodiment, a processor can extract a main vessel region 520 and an entire vessel region 530 from a vessel image 510. When the processor extracts the main vessel region 520 based on a machine learning model, a portion 522 from which the vessel is separated may exist, even though the vessel is actually connected. The entire vessel region 530 includes the main vessel region. Therefore, the processor can correct the main vessel region 522 by connecting the portion 522 from which the vessel is separated in the main vessel region 520 based on the entire vessel region 630.

[0059] The processor may detect a portion 522 in the main blood vessel region 520 where the blood vessel is separated. According to one embodiment, the region between the blobs 521 and 523 corresponding to the main blood vessel region 152, where the shortest distance between the blobs is less than a threshold distance, may be determined as the portion 522 in which the blood vessel is separated. A binary large object (BLOB) is a block having a certain size or larger in an image (e.g., a binary image), and may be a group of pixels connected by a number equal to or greater than a threshold ratio of the image size. For example, pixels indicating a blood vessel region may be extracted from the blood vessel image 510, and among the extracted pixels, an object indicated by a group of pixels connected by a number equal to or greater than a threshold ratio may be Figure 5. For example, the processor may measure the size of blobs 521 and 523 in pixels and, among the objects, may set objects whose number of connected pixels is at least 10% of the image size as blobs 521 and 523 corresponding to at least a portion of a major blood vessel. When the size of the portion represented as the major blood vessel region 520 is smaller than a certain size, the processor may classify the object smaller than the certain size as noise. The processor may exclude the object classified as noise from the major blood vessel region 520.

[0060] After determining that objects larger than a certain size in the main vessel region 520 are blobs 521 and 523, the processor can determine whether the region between blobs 521 and 521 corresponding to the main vessel region 520 is a separated vessel portion 522 by comparing the shortest distance between blobs 521 and 523 with a threshold distance. For example, if the shortest distance between blobs 521 and 523 is short, the blobs 521 and 523 may appear separated due to noise in the image capture and extraction, even though they are actually connected. In this case, the processor, according to one embodiment, can determine that blobs 521 and 523 correspond to the same vessel when the shortest distance between blobs 521 and 523 is less than a threshold distance relative to the image size.

[0061] According to an embodiment, the processor may generate a connecting line to connect the blobs 521, 523 in the region between the separated blobs 521, 523 in response to a condition where the shortest distance between the blobs 521, 523 corresponding to the main blood vessel region 520 is less than a threshold distance. The connecting line is a line connecting the separated blobs 521, 523 in the main blood vessel region 520.

[0062] The processor may match the image size of the entire vessel region 530 with the image size of the main vessel region 520 to correct the main vessel region 520 based on the entire vessel region 53. For example, the image size of the entire vessel region 530 is 256×256 pixels, and the image size of the main vessel region 520 is 768×768 pixels, which are different from each other. The processor may match the image sizes of the entire vessel region 530 and the image size of the main vessel region 520 by scaling at least one of the image size of the entire vessel region 530 and the image size of the main vessel region 520.

[0063] After matching the image sizes, the processor can connect the separated blood vessel portion 522 in the main blood vessel region 520 based on the region corresponding to the connecting line of the entire blood vessel region 530, thereby outputting a corrected main blood vessel region 540. The processor can determine a region 531 corresponding to the connecting line of the main blood vessel region 520 from the entire blood vessel region 630. For example, the processor can determine a region 531 corresponding to the pixel position indicated by the connecting line of the main blood vessel region 520 from the entire blood vessel region 530. The processor can connect the separated blood vessel portion 522 by applying the determined region 532 to the main blood vessel region 522. For example, the processor can perform region growing on the separated portion 522 between the blobs 521 and 523 based on the determined region 531, thereby connecting the blobs 521 and 523.

[0064] When there are multiple regions in the entire blood vessel region 530 corresponding to the connecting line connecting the separated blood vessel portion 522 , the processor may correct the main blood vessel region 520 based on the region with the shortest distance connecting the separated blood vessel portion 521 among the multiple regions.

[0065] According to an embodiment, the processor can compare the thickness of corresponding blood vessels in the entire blood vessel region 530 and the main blood vessel region 520, and correct the main blood vessel region 152 based on the main blood vessel region 520 extracted by enlarging the entire blood vessel region 630 to be thinner than the actual blood vessel thickness. For example, at any point in the blood vessel, when the blood vessel thickness of the main blood vessel region 520 is a first thickness and the blood vessel thickness of the entire blood vessel region 530 is a second thickness greater than the first thickness, the processor can increase the blood vessel thickness of the main blood vessel region 520 from the first thickness to the second thickness. Therefore, by correcting the main blood vessel region 520 that is output as thin or separated from the machine learning model for extracting the main blood vessel region, the processor can accurately extract blood vessel regions that were not extracted by the machine learning model.

[0066] Figure 6 FIG. 6 is a diagram illustrating extracting corrected main blood vessel regions 620 and 640 from blood vessel images 610 and 630 according to an embodiment.

[0067] exist Figure 6 In this example, if a machine learning model extracts major vessel regions 620 and 640 and generates images based solely on vessel images 610 and 630 shown in the left cardiovascular X-ray image, without performing correction based on the entire vessel region, even if a portion of the major vessel is actually narrowed, the processor may still identify the narrowed portion as noise such as microvessels and remove it. However, as part of the major vessel, the narrowed state of this narrowed portion is the most representative of the current condition, so removing this narrowed portion may interfere with a correct diagnosis.

[0068] In contrast, the entire blood vessel region image is an image generated for the entire blood vessel, regardless of the size of the blood vessel width. Therefore, even if a portion of a major blood vessel is narrowed, the processor does not determine it as noise and remove the narrowed portion, but outputs it as is.

[0069] Therefore, after extracting the major blood vessel regions 620 and 640 from the blood vessel images 610 and 630, the processor replaces the identified portion of the entire blood vessel region with the separated blood vessel portion 621 and displays it. This allows accurate extraction of only the major blood vessels from among the blood vessels shown in the blood vessel images 610 and 630 and displaying them on the screen.

[0070] In this case, the processor can emphasize to the operator that the corresponding portion of the main blood vessel is a stenotic portion by displaying the blood vessel portion replaced with the portion extracted by separating the blood vessel from the main blood vessel regions 620 and 640 in a different color.

[0071] Furthermore, the processor combines the images obtained from the main vessel regions 620 and 640 with the image obtained from the entire vessel region and outputs the resulting image, thereby providing the operator with information on how narrow the stenotic portion is compared to the size and thickness of the main vessel. The main vessel region extraction device can calculate and output the ratio of the thickness of the stenotic portion of the vessel to the thickness of the main vessel as the degree of stenosis. For example, by comparing the entire vessel region with the main vessel regions 620 and 640, the processor can calculate the percentage by which the stenotic portion has shrunk compared to the thickness of the main vessel and display this on the screen.

[0072] In addition, the processor can suggest the most appropriate treatment for the patient on the screen by comparing the entire vessel area and the main vessel areas 620 and 640. For example, the processor can display one of the following information on the screen.

[0073] “No specific treatment required”

[0074] “Need to take medication”

[0075] "Angioplasty is required"

[0076] According to the present invention described above, the processor removes the image of microvessels from the main blood vessel images 610, 630 obtained by angiography, and separates only the main blood vessels as the objects of interest and displays them as images, which allows the operator to accurately identify the blocked portion or stenosis portion in the main blood vessels and make a more accurate diagnosis.

[0077] According to one embodiment, the main vessel region extraction device utilizes artificial intelligence technology to isolate only the main vessels and display them as images, thereby improving the degree of separation of the main vessels. Furthermore, the main vessel region extraction device can use artificial intelligence technology to separate and display the blood vessels included in the vessel images 610 and 630 from non-vessel elements (e.g., plastic tubes or organs used to inject contrast agents), thereby enabling surgeons to more accurately locate blocked or stenotic portions of the main vessels. Furthermore, when the main vessel region extraction device utilizes artificial intelligence technology, the processor uses different paths to separate and display the main vessels and the entire vessel separately within the vessel images 610 and 630, and combines the resulting learning data. This improves extraction accuracy compared to extracting only the main vessels through learning. Furthermore, the main vessel region extraction device displays the degree of blockage or stenosis of the main vessels as a numerical value based on the output obtained as described above, enabling more objective judgments.

[0078] Figure 7 FIG. 1 is a block diagram showing a schematic structure of an apparatus for extracting major blood vessel regions according to an embodiment.

[0079] According to an embodiment, a system 700 for extracting a major blood vessel region may include at least one of a major blood vessel region extraction device 710, a medical tool minor surgery device 720, and a blood vessel imaging device 730. The major blood vessel region extraction device 710 may include a processor 711, a memory 712, and an input / output interface 713.

[0080] The processor 711 extracts the entire vessel region from the vessel image, extracts the major vessel region from the vessel image based on the machine learning model for extracting the major vessel region, and connects the separated portions of the vessel based on the entire vessel region to correct the major vessel region. Figures 2 to 6 The process of processor 711 extracting and correcting the major vascular regions has been described, so a detailed description will be omitted here. Memory 712 can temporarily store at least one of vascular images, entire vascular regions, major vascular regions, and machine learning models. Processor 711 can extract and correct the major vascular regions by loading the data stored in memory 712. Input / output interface 713 connects to at least one of vascular imaging device 730 and medical minor surgery device 720 to transmit and receive data.

[0081] The embodiments described above can be implemented using hardware components, software components, and / or a combination of hardware components and software components. For example, the devices and components described in the embodiments can be implemented using, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor (DSP), a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, and can be implemented using one or more general-purpose computers or special-purpose computers. The processing device can execute an operating system (OS) and one or more application software executed in the operating system. Furthermore, the processing device responds to the execution of the software to access, store, operate, process, and generate data. For ease of understanding, the description is given as having only one processing device, but those skilled in the art should understand that the processing device can include multiple processing elements and / or multiple types of processing elements. For example, the processing device can include multiple processors or one processor and one controller. Furthermore, other processing configurations such as parallel processors can also be included.

[0082] Software can include a computer program, code, instructions, or a combination of more than one of these, that causes a processing device to operate in a desired manner or, individually or collectively, to command a processing device. Software and / or data can be permanently or temporarily embodied in any type of equipment, component, physical device, virtual equipment, computer storage medium or device, or transmitted signal wave for interpretation by the processing device or to provide commands or data to the processing device. Software is distributed across computer systems connected via a network and can be stored or executed in a distributed manner. Software and data can be stored in one or more computer read-write storage media.

[0083] The method according to the embodiment is embodied in the form of program commands that can be executed by various computer means and recorded in a computer-readable and writable medium. The computer-readable and writable medium can include program commands, data files, data structures, etc., either individually or in combination. The program instructions recorded on the medium can be instructions specially designed and constructed to implement the embodiment, or instructions that can be used based on known knowledge by a person skilled in the art of computer software. Computer-readable and writable recording media can include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical media similar to CD-ROMs and DVDs; magneto-optical media similar to floppy disks; and hardware devices specially constructed for storing and executing program commands, such as read-only memories (ROMs), random access memories (RAMs), and flash memories. Examples of program instructions include not only machine language codes generated by compilers, but also high-level language codes that can be executed by computers using interpreters, etc. To perform the operations of the embodiment, the hardware device can be configured to implement the operations using one or more software modules, or vice versa.

[0084] In summary, the embodiments are described with limited drawings, and persons skilled in the art will be able to make various modifications and variations based on the description. For example, the described techniques may be performed in a different order than the illustrated method, and / or the described systems, structures, devices, circuits, and other components may be combined or combined in a different manner than the illustrated method, or replaced or substituted with other components or equivalents, and appropriate results may still be achieved.

Claims

1. A method for extracting a main blood vessel region from a blood vessel image executed by a processor, It is characterized in that The following steps are involved: extracting the entire blood vessel region from the blood vessel image; extracting the main blood vessel region from the blood vessel image based on a machine learning model for extracting the main blood vessel region; and Connecting the separated parts of the blood vessels based on the extracted entire blood vessel region, thereby correcting the extracted main blood vessel region, The step of correcting the extracted main blood vessel region further includes: The thicknesses of corresponding blood vessels in the extracted entire blood vessel region and the extracted main blood vessel region are compared, and the thicknesses of blood vessels in the extracted main blood vessel region that are thinner than the actual blood vessel thickness are expanded based on the comparison result.

2. The method for extracting main blood vessel regions according to claim 1, characterized in that: The machine learning model is a machine learning model trained on predetermined shapes of major blood vessels.

3. The method for extracting main blood vessel regions according to claim 2, characterized in that: The machine learning model is a machine learning model trained on at least one blood vessel shape among the right coronary artery, the left anterior descending coronary artery, and the left circumflex coronary artery.

4. The method for extracting main blood vessel regions according to claim 1, characterized in that: The following steps are also included: A portion where blood vessels are separated in the extracted main blood vessel region is detected.

5. The method for extracting main blood vessel regions according to claim 4, characterized in that: The step of detecting the separated portion of the blood vessel comprises the following steps: An area between blobs corresponding to the extracted main blood vessel area, where the shortest distance between the blobs is less than a threshold distance, is determined as a portion where the blood vessel is separated.

6. The method for extracting main blood vessel regions according to claim 4, characterized in that: The step of correcting the extracted main blood vessel regions, The following steps are involved: generating a connecting line connecting regions between blobs corresponding to the extracted main blood vessel regions, wherein the shortest distance between the blobs is less than a threshold distance; and The extracted main blood vessel region is corrected by connecting the separated portions of the blood vessels in the extracted main blood vessel region based on the region corresponding to the connection line in the extracted entire blood vessel region.

7. The method for extracting main blood vessel regions according to claim 6, characterized in that: The step of correcting the extracted main blood vessel region comprises the following steps: When there are multiple regions that can connect the separated blood vessel parts by the connecting line corresponding to the entire extracted blood vessel region, the extracted main blood vessel region is corrected based on the region with the shortest distance that can connect the separated blood vessel parts among the multiple regions.

8. The method for extracting main blood vessel regions according to claim 1, characterized in that: The following steps are also included: Converting the RGB value of the blood vessel image into grayscale; and The blood vessel image converted to the grayscale is normalized.

9. The method for extracting main blood vessel regions according to claim 1, characterized in that: The step of extracting the entire blood vessel region comprises the following steps: extracting the entire blood vessel region based on a partial blood vessel image generated from the entire blood vessel image, The step of extracting the main blood vessel region comprises the following steps: The main blood vessel region is extracted based on a partial blood vessel image generated from the entire blood vessel image.

10. The method for extracting main blood vessel regions according to claim 1, characterized in that: The step of correcting the extracted main blood vessel regions, The following steps are involved: In response to a user's input of a location of the main blood vessel region, determining a portion of the blood vessel to be separated; The main blood vessel region of the target position is corrected based on surrounding main blood vessel regions adjacent to the separated portion of the blood vessel.

11. A computer-readable recording medium storing one or more computer programs, comprising instructions for executing the method according to any one of claims 1 to 10.

12. A device for extracting a main blood vessel region, characterized in that: include: a processor that extracts an entire blood vessel region from the blood vessel image, extracts a main blood vessel region from the blood vessel image based on a machine learning model for extracting the main blood vessel region, and connects separated portions of the blood vessel based on the extracted entire blood vessel region, thereby correcting the extracted main blood vessel region; and a memory storing at least one of the vascular image, the entire vascular region, the main vascular region, and the machine learning model; The processor is configured to: The thicknesses of corresponding blood vessels in the extracted entire blood vessel region and the extracted main blood vessel region are compared, and the thicknesses of blood vessels in the extracted main blood vessel region that are thinner than the actual blood vessel thickness are expanded based on the comparison result.

13. The main blood vessel region extraction device according to claim 12, characterized in that: The machine learning model is a machine learning model trained on predetermined shapes of major blood vessels.

14. The main blood vessel region extraction device according to claim 13, characterized in that: The machine learning model is a machine learning model trained on at least one blood vessel shape among the right coronary artery, the left anterior descending coronary artery, and the left circumflex coronary artery.

15. The main blood vessel region extraction device according to claim 12, characterized in that: The processor detects a portion in the extracted main blood vessel region where the blood vessel is separated.

16. The main blood vessel region extraction device according to claim 15, characterized in that: The processor determines a region between blobs corresponding to the extracted main blood vessel region, where the shortest distance between blobs is less than a threshold distance, as a separated portion of the blood vessel.

17. The main blood vessel region extraction device according to claim 15, characterized in that: The processor generates a connecting line connecting regions between blobs corresponding to the extracted main blood vessel region whose shortest distance between blobs is less than a threshold distance, and connects separated portions of the blood vessel in the extracted main blood vessel region based on regions corresponding to the connecting line in the entire extracted blood vessel region, thereby correcting the extracted main blood vessel region.

18. The main blood vessel region extraction device according to claim 17, characterized in that: When there are a plurality of regions to which the separated blood vessel portions can be connected by the connecting line corresponding to the entire extracted blood vessel region, the processor corrects the extracted main blood vessel region based on a region of the plurality of regions with a shortest distance capable of connecting the separated blood vessel portions.

19. The main blood vessel region extraction device according to claim 12, characterized in that: The processor converts the RGB value of the blood vessel image into a grayscale and normalizes the blood vessel image converted into the grayscale.

20. The main blood vessel region extraction device according to claim 12, characterized in that: The processor extracts the entire blood vessel region and the main blood vessel region based on a partial blood vessel image generated from the entire blood vessel image.

21. The main blood vessel region extraction device according to claim 12, characterized in that: The processor determines a separated blood vessel portion in response to a user's input of a position of the extracted main blood vessel region, and corrects the main blood vessel region of the target position based on surrounding main blood vessel regions adjacent to the separated blood vessel portion.

Images