Method for generating coil shape and electronic device performing same

Abstract

A method for generating a coil shape according to one embodiment comprises the steps of: acquiring three-dimensional data including an aneurysm region; acquiring a first image obtained by imaging a coil in a blood vessel at a first position by using a first camera; acquiring a second image obtained by imaging the coil at a second position by using a second camera; acquiring a third image obtained by imaging the coil at a third position by using the first camera; acquiring a fourth image obtained by imaging the coil at a fourth position by using the second camera; and determining a three-dimensional coil shape on the basis of the three-dimensional data, the first image, the second image, the third image, and the fourth image.

Description

Method for generating a coil shape and electronic device for performing the same

[0001] The disclosure relates to a method for generating a coil shape and an electronic device for performing the same.

[0002] An aneurysm is a condition in which a portion of an artery wall weakens and bulges out like a balloon; examples include cerebral aneurysms, aortic aneurysms, and renal aneurysms. In particular, cerebral aneurysms can cause life-threatening cerebral hemorrhage if they rupture, making early detection and treatment crucial. One of the representative methods for treating such aneurysms is coil embolization, a treatment that inhibits the growth of the aneurysm or prevents rupture by inserting coils into the aneurysm to block blood flow.

[0003] For the successful execution of coil embolization, it is essential to determine accurate three-dimensional shape, volume, length, and other size information of the aneurysm. This is because it directly influences the selection of coils and the determination of the insertion volume.

[0004] Previously, medical staff estimated the shape, size, and location of coils inserted into aneurysms based on 2D images, but this method had limitations in that it could not perfectly represent the actual 3D structure of the coil. Furthermore, when medical staff acquired 3D data of the coil, there were problems such as the patient being exposed to a large amount of radiation, the administration of large quantities of contrast agent, and the time required. Accordingly, attempts are being made to resolve the issues of contrast agent, time, and radiation by generating a 3D coil shape based on 2D images.

[0005] One embodiment aims to provide a method and apparatus for generating a three-dimensional coil shape based on a two-dimensional image.

[0006] One embodiment aims to provide a method and apparatus for rapidly generating a coil shape while irradiating a patient with relatively low radio waves and administering a contrast agent.

[0007] However, the problems that the present invention aims to solve are not limited to those mentioned above, and may include problems that are not mentioned but can be clearly understood by those skilled in the art from the description below.

[0008] A method for generating a coil shape according to one embodiment for solving such technical problems comprises: acquiring three-dimensional data including an aneurysm region; acquiring a first image of a coil inside a blood vessel at a first position using a first camera; acquiring a second image of the coil at a second position using a second camera; acquiring a third image of the coil at a third position using the first camera; acquiring a fourth image of the coil at a fourth position using the second camera; and determining a three-dimensional coil shape based on the three-dimensional data, the first image, the second image, the third image, and the fourth image.

[0009] An electronic device according to one embodiment includes a processor and a memory connected to the processor, the memory is configured to store a program, the processor is configured to execute the program, and when the program is executed, steps of a method for generating a coil shape are implemented.

[0010] FIG. 1 is a schematic block diagram of a computing system according to one embodiment.

[0011] FIG. 2 is a flowchart illustrating a method for creating a coil shape according to one embodiment.

[0012] FIG. 3 shows an example of three-dimensional data according to one embodiment.

[0013] FIG. 4 shows an example of reference images according to one embodiment.

[0014] FIG. 5 shows an example of rotational images according to one embodiment.

[0015] FIG. 6 is a flowchart illustrating a method for determining a coil region according to one embodiment.

[0016] FIGS. 7 to 10 are drawings for explaining the configuration of an electronic device that determines a coil region according to one embodiment.

[0017] FIG. 11 is a flowchart illustrating a method for creating a coil shape according to one embodiment.

[0018] FIG. 12 is a flowchart illustrating a configuration for determining a camera matrix according to one embodiment.

[0019] FIGS. 13 and 14 are drawings illustrating a configuration for determining a camera matrix using reference images according to one embodiment.

[0020] FIG. 15 is a flowchart illustrating a configuration for determining a transformation matrix according to one embodiment.

[0021] FIGS. 16 to 18 are drawings illustrating a configuration for determining a transformation matrix according to one embodiment.

[0022] FIG. 19 is a flowchart illustrating a configuration for correcting a camera matrix and a transformation matrix according to one embodiment.

[0023] FIG. 20 is a flowchart illustrating a configuration for determining a camera matrix according to one embodiment.

[0024] FIGS. 21 to 23 are drawings illustrating a configuration for determining a camera matrix according to one embodiment.

[0025] FIG. 24 shows an example of three-dimensional data according to one embodiment.

[0026] The various embodiments described in this specification are illustrative for the purpose of clearly explaining the technical concept of this disclosure and are not intended to limit it to specific embodiments. The technical concept of this disclosure includes various modifications, equivalents, alternatives, and embodiments optionally combined from all or part of each embodiment described in this specification. Furthermore, the scope of the technical concept of this disclosure is not limited to the various embodiments presented below or the specific descriptions thereof.

[0027] Terms used in this specification, including technical or scientific terms, may have the meaning generally understood by those skilled in the art to which this disclosure pertains, unless otherwise defined.

[0028] Expressions used herein such as “comprising,” “may compose,” “possessing,” “possessing,” “having,” and “possessing” imply the existence of the subject feature (e.g., function, operation, or component, etc.) and do not exclude the existence of other additional features. That is, such expressions should be understood as open-ended terms implying the possibility of including a second embodiment.

[0029] In this specification, singular expressions include plural expressions unless the context clearly specifies them as singular. Additionally, plural expressions include singular expressions unless the context clearly specifies them as plural. Throughout the specification, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0030] Additionally, the terms 'module' or 'part' as used in the specification refer to software or hardware components, and the 'module' or 'part' performs certain roles. However, the meaning of 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, as an example, the 'module' or 'part' may include components such as software components, object-oriented software components, class components, and task components, and at least one of processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The components and the functions provided within the 'module' or 'part' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0031] According to one embodiment of the present disclosure, a ‘module’ or ‘part’ may be implemented as a processor and memory. The term ‘processor’ should be broadly interpreted to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some environments, the term ‘processor’ may refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. The term ‘processor’ may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors combined with a DSP core, or any other combination of such configurations. Additionally, the term ‘memory’ should be broadly interpreted to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as Random Access Memory (RAM), Read-Only Memory (ROM), Non-Volatile Random Access Memory (NVRAM), Programmable Read-Only Memory (PROM), Erasable-Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), Flash Memory, Magnetic or Optical Data Storage Devices, Registers, etc. If a processor can read information from memory and / or write information to memory, the memory is said to be in an electronic communication state with the processor. Memory integrated into a processor is in an electronic communication state with the processor.

[0032] Expressions such as "first," "second," or "first," "second" as used in this specification are used to distinguish one object from another when referring to a plurality of objects of the same kind, unless otherwise indicated in the context, and do not limit the order or importance of said objects.

[0033] Expressions used herein such as “A, B, and C,” “A, B, or C,” “A, B, and / or C,” or “at least one of A, B, and C,” “at least one of A, B, or C,” “at least one of A, B, and / or C,” “at least one selected from A, B, and C,” “at least one selected from A, B, or C,” “at least one selected from A, B, and / or C,” etc., may mean each of the listed items or all possible combinations of the listed items. For example, “at least one selected from A and B” may refer to (1) A, (2) at least one of A, (3) B, (4) at least one of B, (5) at least one of A and at least one of B, (6) at least one of A and B, (7) at least one of B and A, and (8) all of A and B.

[0034] As used herein, the expression “based on” is used to describe one or more factors affecting an act or action of a decision or judgment described in the phrase or sentence containing such expression, and such expression does not exclude additional factors affecting said act or action of a decision or judgment.

[0035] As used in this specification, the expression that a certain component (e.g., a first component) is "connected" or "connected" to another component (e.g., a second component) may mean that the said certain component is not only directly connected or connected to the said other component, but is also connected or connected through a new other component (e.g., a third component).

[0036] As used herein, the expression "configured to" may have meanings such as "set to," "capable of," "modified to," "made to," or "capable of." Such expression is not limited to the meaning of "specifically designed in hardware," and, for example, a processor configured to perform a specific operation may mean a generic-purpose processor capable of performing that specific operation by executing software.

[0037] Various embodiments of the present disclosure will be described below with reference to the accompanying drawings. In the accompanying drawings and the description thereof, identical or substantially equivalent components may be given the same reference numerals. Furthermore, in the description of the various embodiments below, the description of identical or corresponding components may be omitted, but this does not mean that such components are not included in the embodiments.

[0038]

[0039] FIG. 1 is a schematic block diagram of a computing system according to one embodiment, FIG. 2 is a flowchart showing a method for generating a coil shape according to one embodiment, FIG. 3 shows an example of three-dimensional data according to one embodiment, FIG. 4 shows an example of reference images according to one embodiment, and FIG. 5 shows an example of rotated images according to one embodiment.

[0040] Referring to FIG. 1, a computing system (10) according to one embodiment may acquire a blood vessel image of a first user, perform image processing on the acquired blood vessel image, and display the processed image. The computing system (10) may provide the processed image to a second user. For example, the first user may be a patient, and the second user may be a medical professional.

[0041] In one embodiment, the computing system (10) can photograph the brain blood vessels of the first user to obtain a brain blood vessel image and perform image processing on the brain blood vessel image. For example, the computing system can generate three-dimensional data representing the blood vessels through image processing. Additionally, the computing system can generate a three-dimensional coil shape on the three-dimensional data through image processing. However, the embodiment is not necessarily limited thereto, and the computing system (10) can photograph blood vessels that are subject to angiography, such as cardiovascular vessels and gastrointestinal blood vessels.

[0042] A computing system (10) according to one embodiment includes a shooting device (100) and an electronic device (200). The shooting device (100) may be a shooting device that captures a first user to acquire an image. For example, the shooting device (100) may be an angiography device (e.g., angio device), an optical coherence tomography (OCT) device, a computed tomography (CT) device, a magnetic resonance imaging (MRI) device, a magnetic resonance angiography (MRA) device, etc.

[0043] The shooting device (100) can photograph the first user at multiple shooting points and acquire multiple images. In one embodiment, the shooting device (100) can photograph the first user while rotating around the first user. In another embodiment, the shooting device (100) can rotate the first user and photograph the rotating first user. The shooting device (100) can transmit the acquired multiple images to an electronic device (200).

[0044] The electronic device (200) may be a computing device that performs image processing on an image received from the imaging device (100). For example, the electronic device (200) may generate three-dimensional data based on a plurality of first images. Additionally, the electronic device (200) may generate a coil shape based on a plurality of second images. The time at which the imaging device (100) captures the first user to acquire the first image and the time at which the second image is acquired may be different. For example, the first image may be acquired before the second user performs coil embolization to treat an aneurysm on the first user, and the second image may be acquired after the coil embolization is performed. That is, the second image may include an intravascular coil. The first image may or may not include an intravascular coil.

[0045] An electronic device (200) according to one embodiment can perform the coil shape generation method of FIG. 2. The electronic device (200) may include a memory that stores a program for executing the coil shape generation method and a processor configured to execute the program to generate a coil shape. The electronic device (200) may further include an input / output device (including an input device such as a mouse or keyboard, an output device such as a display panel, an input / output device such as a touchscreen panel, etc.), a communication device, etc.

[0046] Referring to FIGS. 1 and 2, an electronic device (200) according to one embodiment may acquire three-dimensional data including an aneurysm area (S110). A second user may acquire a plurality of initial images using a imaging device (100) to determine parameters (size, diameter, volume, etc.) regarding the aneurysm of the first user. The electronic device (200) may generate three-dimensional data using the plurality of initial images. According to an embodiment, the three-dimensional data may include data around the aneurysm area and may not include data at a location far from the aneurysm area.

[0047] The shooting device (100) may be a biplane shooting device comprising a first camera (CAM_1) and a second camera (CAM_2). The first camera (CAM_1) and the second camera (CAM_2) may photograph a first user while moving along a preset path. For example, the first camera (CAM_1) may photograph the first user while moving along a first path to obtain first initial images. The second camera (CAM_2) may photograph the first user while moving along a second path different from the first path to obtain second initial images. The electronic device (200) may have generated three-dimensional data using the first initial images and the second initial images. The electronic device (200) may obtain the generated and stored three-dimensional data.

[0048] Referring to FIG. 3, three-dimensional data (3DRA) according to one embodiment may include a blood vessel mesh (VMES) and an aneurysm region (ANRG). In FIG. 3, for convenience of explanation, the blood vessel mesh (VMES) and the aneurysm region (ANRG) are shown as separated, but the embodiment is not necessarily limited thereto. That is, the three-dimensional data (3DRA) may include a portion of the aneurysm region (ANRG) and may not necessarily have a boundary separated from the blood vessel mesh (VMES).

[0049] Referring again to FIGS. 1 and FIGS. 2, the electronic device (200) can acquire a first image (IMG_1) from a first camera (CAM_1) at a first location (PSTN_1) (S120). The electronic device (200) acquiring the first image (IMG_1) may be performed after coil embolization has been performed. That is, the first image (IMG_1) may include a coil within a blood vessel (e.g., an aneurysm). The first location (PSTN_1) may be a point on a first path.

[0050] The electronic device (200) can acquire a second image (IMG_2) from a second camera (CAM_2) at a second location (PSTN_2) (S130). The second location (PSTN_2) may be a point on a second path. The second image (IMG_2) may be an image of the first user taken at a different location from the first image (IMG_1). The second image (IMG_2) may include a coil inside the blood vessel, similar to the first image (IMG_1). The first image (IMG_1) and the second image (IMG_2) may also be referred to as reference images. Referring together with FIG. 4, the first image (IMG_1) and the second image (IMG_2) according to one embodiment are illustrated. The first image (IMG_1) may include a coil (1010) captured by a first camera (CAM_1) at a first position (PTSN_1), and the second image (IMG_2) may include a captured coil (1010) captured by a second camera (CAM_2) at a second position (PTSN_2). That is, the first image (IMG_1) and the second image (IMG_2) may have been captured from different angles.

[0051] Referring again to FIGS. 1 and FIGS. 2, the electronic device (200) can acquire a third image (IMG_3) from a first camera (CAM_1) at a third position (PSTN_3) (S150). The third position (PSTN_3) may be a position where the first camera (CAM_1) has moved along a first path. For example, the first camera (CAM_1) may be positioned at the third position (PTSN_3) by rotating by a predetermined angle from a first position (PTSN_1) on the first path. Rotation of the first camera (CAM_1) may mean moving around the axis of rotation of the first path.

[0052] The electronic device (200) can acquire a fourth image (IMG_4) from a second camera (CAM_2) at a fourth position (PSTN_4) (S150). The fourth position (PSTN_4) may be a position where the second camera (CAM_2) has moved along a second path. For example, the second camera (CAM_2) may be positioned at the fourth position (PTSN_4) by rotating by a predetermined angle from a second position (PTSN_2) on the second path. At this time, the angle at which the second camera (CAM_2) is rotated may be the same as the angle at which the first camera (CAM_1) is rotated, but the embodiment is not necessarily limited thereto and may be implemented by rotating by a different angle. Referring together to FIG. 5, a third image (IMG_3) and a fourth image (IMG_4) according to one embodiment are shown. The third image (IMG_4) may include a coil (1010) captured by the first camera (CAM_1) at the third position (PSTN_3) by rotating from the first position (PTSN_1). The second image (IMG_2) may include a coil (1010) captured by the second camera (CAM_2) at the fourth position (PSTN_4) by rotating from the second position (PTSN_2).

[0053] The electronic device (200) can determine a three-dimensional coil shape based on three-dimensional data and first to fourth images (IMG_1~IMG4) (S160). The electronic device (200) can determine a plurality of camera matrices and transformation matrices, and determine a three-dimensional coil shape based on the plurality of camera matrices and transformation matrices.

[0054] A plurality of camera matrices may include first to fourth camera matrices. The first camera matrix may be related to parameters when the first camera (CAM_1) captures the first image (IMG_1). The second camera matrix may be related to parameters when the second camera (CAM_2) captures the second image (IMG_2). The third camera matrix may be related to parameters when the first camera (CAM_1) captures the third image (IMG_3). The fourth camera matrix may be related to parameters when the second camera (CAM_2) captures the fourth image (IMG_4). The transformation matrix is ​​for transforming three-dimensional data and may include a translation matrix, a rotation matrix, etc.

[0055] The electronic device (200) can perform camera optimization by correcting the offsets of multiple camera matrices. For example, the acquired camera matrices may not be accurate due to the movement of the first camera (CAM_1) and the second camera (CAM_2), differences in the shooting time, or the movement of the subject (first user). Accordingly, the electronic device (200) can correct the offsets of multiple camera matrices.

[0056] The electronic device (200) can correct the offset of the transformation matrix. The electronic device (200) can correct the offset so that when 3D data is back-projected onto a 2D image, the back-projected image aligns with the 2D image. In this way, the electronic device (200) can determine the 3D coil shape using the camera matrix and the transformation matrix, which have been optimized with the offset corrected.

[0057] The electronic device (200) may be an artificial intelligence (AI) device, a personal computer (PC), a laptop computer, a mobile phone, a smartphone, a tablet PC, a wearable device, medical imaging equipment, a healthcare device, etc.

[0058] In some embodiments, the computing system (10) may include a server such as an AI server or a data center, and the electronic device (200) may be implemented to communicate with the server. That is, the electronic device (200) may use the artificial neural network of the server.

[0059]

[0060] FIG. 6 is a flowchart illustrating a method for determining a coil region according to one embodiment, and FIGS. 7 to 10 are drawings for explaining the configuration of an electronic device according to one embodiment that determines a coil region. An electronic device according to one embodiment (e.g., 200 in FIG. 1) can determine a coil region and determine a transformation matrix based on the coil region.

[0061] Referring to FIG. 6, an electronic device according to one embodiment can generate a Region of Interest (ROI) image from an input image (S610). The input image may be obtained from a camera (e.g., one of CAM_1 to CAM_4 of FIG. 2) and may be as shown in FIG. 7. The input image may include a shield area (1110, 1120) and a coil image (1200). To prevent a first user from being exposed to excessive radio waves (e.g., X-rays), the input image may be obtained only for the area around the coil by placing a shield and photographing the first user. The electronic device can generate the region of interest image by removing the shield area (1110, 1120) from the input image.

[0062] In one embodiment, the electronic device can generate a black border on an input image through padding processing. The electronic device selects a corner of the input image with the generated border (e.g., coordinates (0, 0)) as a starting point and can use a flood fill algorithm starting from the starting point. The electronic device can generate a region of interest image by detecting the boundary area of ​​the occlusion through the flood fill algorithm and removing the detected boundary area. The region of interest image generated by the electronic device may be as shown in FIG. 8. The region of interest image of FIG. 8 includes a coil image (1200) and the occlusion area may have been removed.

[0063] The electronic device can calculate pixel values ​​and edge values ​​of the region of interest image (S620). For example, the electronic device can generate a distance map and an edge map from the region of interest image. The distance map contains pixel values ​​and can be understood as a map indicating how far each pixel (or point) of the region of interest image is from a specific reference point. The edge map contains edge values ​​and can be understood as a map indicating the boundaries or outlines of objects in the region of interest image.

[0064] The electronic device can determine the coil area based on pixel values ​​and edge values ​​(S630). In one embodiment, the electronic device can generate a new map by multiplying a distance map and an edge map. Multiplying the distance map and the edge map can be understood as multiplying the pixel values ​​of corresponding locations. In the new map, parts with distinct boundaries and significant importance may be displayed brighter. The new map generated by the electronic device according to one embodiment may be as shown in FIG. 9. In the new map of FIG. 9, it can be seen that the coil image (1200) is displayed brightly.

[0065] The electronic device can separate the new map into two colors through pre-segmentation. For example, the electronic device can separate the new map into black and white using the Otsu thresholding method. Accordingly, the electronic device can distinguish the coil and the background in the new map.

[0066] The electronic device can determine connected components in the new map where pre-segmentation has been performed. A connected component may refer to a set of connected pixels in the new map. The electronic device can determine the largest connected component among the connected components. The electronic device can determine the largest connected component as the coil region.

[0067] In one embodiment, the electronic device determines a prompt point through point sampling in the coil region and can determine a more refined coil region based on the prompt point. The electronic device can calculate pixel values ​​and edge values ​​in the coil region, as in generating a region of interest image. The electronic device can output at least one pixel coordinate as a prompt point based on the pixel values ​​and edge values ​​within the coil region. For example, the electronic device can output the top n pixel coordinates having high brightness within the coil region (n is an integer greater than or equal to 1).

[0068] The electronic device can generate a coil mask by inputting prompt points into a prompt segmentation model. The prompt segmentation model may include a Segment Anything Model (SAM). A SAM can be an artificial intelligence model designed to accurately segment specific objects in an image. The electronic device can make coil regions more prominent using adaptive histogram equalization. For example, adaptive histogram equalization may include a Contrast Limited Adaptive Histogram Equalization (Clahe) operation. Through this method, the electronic device can acquire a coil mask accurately and efficiently from the input image. This offers higher generalization performance than existing algorithms and can achieve improved performance with less data and lower computational resources than deep learning-based methods.

[0069] Referring to FIG. 10, a coil mask (CMSK) and a prompt point (PRPT) generated by an electronic device according to one embodiment can be identified. The electronic device can determine the coil region in other images (e.g., the second to fourth images (IMG_2 to IMG_4) of FIG. 4 and FIG. 5) using the same method.

[0070]

[0071] FIG. 11 is a flowchart illustrating a method for creating a coil shape according to one embodiment.

[0072] Referring to FIG. 11, the electronic device may determine a plurality of camera matrices (S1110). Each of the plurality of camera matrices may correspond to each of the first to fourth images. For example, the plurality of camera matrices may include the first to fourth camera matrices. The first camera matrix may be associated with parameters when the first camera (e.g., CAM_1 in FIG. 2) captures the first image (IMG_1). The second camera matrix may be associated with parameters when the second camera (e.g., CAM_2 in FIG. 2) captures the second image (IMG_2). The third camera matrix may be associated with parameters when the first camera (CAM_1) captures the third image (IMG_3). The fourth camera matrix may be associated with parameters when the second camera (CAM_2) captures the fourth image (IMG_4).

[0073] The electronic device can determine a plurality of camera matrices based on three-dimensional data, a first image, a second image, a third image, and a fourth image. A configuration for the electronic device to determine a plurality of camera matrices will be described later with reference to FIG. 12.

[0074] An electronic device according to one embodiment can determine a transformation matrix (S1120). The electronic device can back-project three-dimensional data onto a two-dimensional image using the transformation matrix and the camera matrix. The electronic device can determine the transformation matrix based on the three-dimensional data, the first image, the second image, the third image, and the fourth image. The configuration for the electronic device to determine the transformation matrix will be described later with reference to FIG. 15.

[0075] The electronic device can determine a three-dimensional coil shape based on a plurality of camera matrices and transformation matrices (S1130).

[0076] In one embodiment, the electronic device can generate a point cloud based on an aneurysm region in three-dimensional data. For example, the electronic device can determine the center point of the aneurysm region. The center point of the aneurysm region may be determined based on user input or may be automatically determined by computation of the electronic device or a server. The electronic device may also determine the center point using an artificial neural network of the server. The electronic device can determine a polyhedron centered on the center point of the aneurysm region and having a volume larger than the volume of the aneurysm region. The electronic device can generate a point cloud within the polyhedron.

[0077] An electronic device can acquire first carving data based on a point cloud, a first image, and a second image. For example, the electronic device can back-project the point cloud onto the first image using a first camera matrix and a transformation matrix of the first image. Based on the back-projected data, the electronic device can acquire first cloud data by removing points in the point cloud that do not correspond to the first coil region of the first image. Similarly, the electronic device can acquire first carving data by removing points in the first cloud data that do not correspond to the second coil region of the second image using a second camera matrix and a transformation matrix of the second image.

[0078] The electronic device can acquire second carving data based on first carving data, a third image, and a fourth image. For example, the electronic device can back-project the first carving data onto the third image using the third camera matrix and transformation matrix of the third image. Based on the back-projected data, the electronic device can acquire second cloud data by removing points in the first carving data that do not correspond to the third coil region of the third image. Similarly, the electronic device can acquire second carving data by removing points in the second cloud data that do not correspond to the fourth coil region of the fourth image using the fourth camera matrix and transformation matrix of the fourth image.

[0079] The electronic device can determine a three-dimensional coil shape from the second carving data. For example, the electronic device can determine the coil shape by generating a surface mesh based on the second carving data. The electronic device can use a marching cube algorithm.

[0080]

[0081] FIG. 12 is a flowchart illustrating a configuration for determining a camera matrix according to one embodiment, and FIG. 13 and FIG. 14 are drawings illustrating a configuration for determining a camera matrix using reference images according to one embodiment. An electronic device according to one embodiment can determine a camera matrix of reference images. The reference images may include a first image of a first camera and a second image of a second camera.

[0082] Referring to FIG. 12, an electronic device according to one embodiment can obtain a first initial matrix (M_INIT1) of a first image from a first camera (S1210). The first initial matrix (M_INIT1) may be a camera matrix associated with parameters when the first camera obtains the first image.

[0083] The electronic device can determine the first initial matrix (M_INIT1) as the first camera matrix (M_CAM1) (S1220). That is, the electronic device can fix the first initial matrix (M_INIT1) as the first camera matrix (M_CAM1).

[0084] The electronic device can obtain a second initial matrix (M_INIT2) of the second image from the second camera (S1230). The second initial matrix (M_INIT2) may be a camera matrix associated with parameters when the second camera acquires the second image. The electronic device can determine the second camera matrix (M_CAM2) by correcting the offset of the second initial matrix (M_INIT2).

[0085] The electronic device can determine a common image point in the first image and the second image (S1240). Referring together to FIG. 13, a first image (IMG_1) and a second image (IMG_2) according to one embodiment are shown. The first image (IMG_1) and the second image (IMG_2) may be identical to the first image (IMG_1) and the second image (IMG_2) of FIG. 4. For example, the electronic device can determine a first common point (CIP1) in the first image (IMG_1). The electronic device can determine a second common point (CIP2) in the second image (IMG_2). The second common point (CIP2) may correspond to the first common point (CIP1).

[0086] The first common point (CIP1) and the second common point (CIP2) may be determined based on user input or automatically determined by the operation of an electronic device or server. For example, the first common point (CIP1) and the second common point (CIP2) may correspond to identifiers arranged at predetermined intervals in a coil. The electronic device or server may determine the first common point (CIP1) and the second common point (CIP2) by detecting identifiers in the first image (IMG_1) and the second image (IMG_2).

[0087] The electronic device can determine the second camera matrix (M_CAM2) by correcting the second initial matrix (M_INIT2) based on the common image points (CIP1, CIP2) (S1250). The electronic device can determine the epipolar lines of the first common point (CIP1) and the second common point (CIP2). The electronic device can display the epipolar lines in the first and second images (IMG_1, IMG_2). For example, the electronic device can display the first epipolar line (EPPL1) in the first image (IMG_1) and the second epipolar line (EPPL2) in the second image (IMG_2). Ideally, the first epipolar line (EPPL1) passes through the first common point (CIP1) in the first image (IMG_1), and the second epipolar line (EPPL2) passes through the second common point (CIP2) in the second image (IMG_2). However, due to noise from the first and second cameras, the elevation lines (EPPL1, EPPL2) may not pass through the common point (CIP1, CIP2).

[0088] The electronic device can determine a second camera matrix (M_CAM2) based on the distance between a first common point (CIP1) and a second common point (CIP2) and epipolar lines (EPPL1, EPPL2). For example, the electronic device can determine a first loss function based on a first distance between the first common point (CIP1) and the first epipolar line (EPPL1) and a second distance between the second common point (CIP2) and the second epipolar line (EPPL2). The first and second distances refer to epipolar line distances and can be calculated in two or three dimensions.

[0089] The electronic device can determine a second camera matrix (M_CAM2) using a first algorithm on a first loss function based on distance. For example, the first algorithm may include gradient descent. The second camera matrix (M_CAM2) can be optimized through gradient descent. For example, if the camera matrix is ​​in the form of 4*4, the ISO offset (x, y, z), CArm angles (α, β, γ), and focal length can be optimized.

[0090] Referring to FIG. 14, as a result of optimizing the second camera matrix (M_CAM2), it can be seen that the third elevation line (EPPL3) in the first image (IMG_1) passes through the first common point (CIP1), and the fourth elevation line (EPPL4) in the second image (IMG_2) passes through the second common point (CIP2). That is, noise in the camera matrices of the first camera and the second camera can be removed from the reference image.

[0091] In FIG. 12, for convenience of explanation, the first initial matrix (M_INIT1) is fixed as the first camera matrix (M_CAM1), and the second camera matrix (M_CAM2) is determined by correcting the offset of the second initial matrix (M_INIT2); however, the embodiment is not necessarily limited thereto. For example, the electronic device may fix the second initial matrix (M_INIT2) as the second camera matrix (M_CAM2) and determine the first camera matrix (M_CAM1) by correcting the offset of the first initial matrix (M_INIT1).

[0092]

[0093] FIG. 15 is a flowchart illustrating a configuration for determining a transformation matrix according to one embodiment, and FIGS. 16 to 18 are drawings illustrating a configuration for determining a transformation matrix according to one embodiment. An electronic device according to one embodiment can determine a transformation matrix of a reference image. The reference image may include a first image of a first camera and a second image of a second camera.

[0094] Referring to FIG. 15, an electronic device according to one embodiment can determine a transformation matrix (M_TRSF) based on a first camera matrix (M_CAM1), a second camera matrix (M_CAM2), three-dimensional data (3DRA), a first image (IMG_1), and a second image (IMG_2) (S1510).

[0095] In one embodiment, the electronic device may acquire a first background image corresponding to a first image (IMG_1). The first camera acquires a plurality of time-series images at a first location, and the first background image may represent the first frame among the plurality of time-series images. For example, the second user administers a contrast agent to the first user, and the first camera may acquire a second frame after acquiring the first frame. Since the second frame is acquired subsequently to the first frame, the vascular region may be prominently displayed due to the contrast agent. The electronic device may determine the first frame as the first background image. The electronic device may generate a first differential image by subtracting the first background image from the first image (IMG_1).

[0096] The electronic device can acquire a second background image corresponding to the second image (IMG_2). For example, the second camera can acquire a plurality of time-series images at a second location and determine the first frame among the plurality of time-series images as the second background image. The electronic device can generate a second image by subtracting the second background image from the second image (IMG_2).

[0097] Referring to FIG. 16, a first image (RDM_1) and a second image (RDM_2) generated by an electronic device according to one embodiment can be seen. The second image (RDM_2) may be generated based on a second image (IMG_2). Vascular regions may be brightly displayed in the first image (RDM_1) and the second image (RDM_2).

[0098] The electronic device can determine a transformation matrix (M_TRSF) based on a first camera matrix (M_CAM1), a second camera matrix (M_CAM2), three-dimensional data (3DRA), a first image (RDM_1), and a second image (RDM_2). For example, the electronic device can generate a first back-projected image and a second back-projected image from the three-dimensional data (3DRA) based on the first camera matrix (M_CAM1) and the second camera matrix (M_CAM2). Referring to FIG. 17, the first image (RDM_1), the second image (RDM_2), the first back-projected image (BPG_1), and the second back-projected image (BPG_2) generated by the electronic device according to one embodiment can be seen. The second back-projected image (BPG_2) can be generated based on the second camera matrix (M_CAM2).

[0099] The electronic device can determine a transformation matrix (M_TRSF) based on the pixel values ​​of the first image (RDM_1), the second image (RDM_2), the first back-projected image (BPG_1), and the second back-projected image (BPG_2). The electronic device can determine a transformation matrix (M_TRSF) that maximizes the sum of the pixel values.

[0100] The electronic device can calculate a first pixel value in the part of the first back-projected image (BPG_1) that overlaps with the first difference image (RDM_1). Additionally, the electronic device can calculate a second pixel value in the part of the second back-projected image (BPG_2) that overlaps with the second difference image (RDM_2). The electronic device can determine a transformation matrix (M_TRSF) based on the first pixel value and the second pixel value.

[0101] The electronic device may determine a second loss function based on a first pixel value and a second pixel value. The second loss function may be a function obtained by multiplying the score function of the pixel value by -1. The electronic device may determine a transformation matrix (M_TRSF) by using a second algorithm on the second loss function based on the pixel value. For example, the second algorithm may include a genetic algorithm. According to an embodiment, the electronic device may also use a first algorithm on the second loss function.

[0102] However, the embodiments are not necessarily limited to this and may be modified to include the average of pixel values, the product of pixel values, etc. In addition, they may be modified to include gradient values, mutual information, etc., instead of pixel values.

[0103] In one embodiment, the electronic device can determine a three-dimensional coil center point. For example, the electronic device can determine a first coil region from a first image (RDM_1) (or a first image (IMG_1)) and a second coil region from a second image (RDM_2) (or a second image (IMG_2)). The configuration in which the electronic device determines a coil region in an image can be applied in the same way as described with reference to FIGS. 6 to 10. Accordingly, redundant descriptions are omitted.

[0104] The electronic device can determine a first center point of a first coil region and a second center point of a second coil region. The electronic device can determine a three-dimensional coil center point based on the first center point, the second center point, the first position of the first camera, and the second position of the second camera. The electronic device can map the first and second center points into a three-dimensional space using the first and second camera matrices (M_CAM1, M_CAM2). The electronic device can generate first and second mapping points from the first and second center points.

[0105] The electronic device can generate a first straight line connecting a first mapping point and a first position, and generate a second straight line connecting a second mapping point and a second position. The electronic device can determine the intersection point of the first straight line and the second straight line as the coil center point. According to an embodiment, the electronic device can determine the shortest distance between the first straight line and the second straight line, and determine the midpoint of the two points corresponding to the shortest distance as the coil center point. In this embodiment, it has been described that the electronic device determines the coil center point based on the center point of the coil area, but the embodiment is not necessarily limited thereto, and the electronic device may determine the coil center point using a common point, etc.

[0106] The electronic device can translate the 3D data (3DRA) based on the distance between the center point of the aneurysm region (hereinafter referred to as the "aneurysm center point") and the coil center point of the 3D data (3DRA). The aneurysm center point may be a point representing the aneurysm region. According to the embodiments, the aneurysm center point may be located on the surface of the aneurysm region or inside the aneurysm region. In the embodiments, the electronic device may determine the aneurysm center point based on user input or automatically determine the aneurysm center point. For example, the electronic device may use an artificial neural network trained to output the aneurysm center point from the 3D data (3DRA). In this case, the artificial neural network may be located inside or outside the electronic device. However, the embodiments are not necessarily limited thereto, and the electronic device may determine the aneurysm center point through a series of methods.

[0107] The electronic device can move the 3D data (3DRA) so that the distance between the aneurysm center point and the coil center point is minimized. In some embodiments, the electronic device can move the 3D data (3DRA) so that the aneurysm center point and the coil center point coincide. The electronic device can obtain an initial matrix of the transformation matrix (M_TRSF) (e.g., an initial transformation matrix) by moving the 3D data (3DRA) based on the distance.

[0108] The electronic device can rotate the center point-matched (aligned) 3D data (3DRA). In this case, movement of the 3D data (3DRA) may not occur. The electronic device can determine an offset based on pixel values ​​that change as the 3D data (3DRA) is rotated. For example, the electronic device can obtain an offset for the rotation matrix by using a genetic algorithm in the second loss function.

[0109] In one embodiment, the electronic device can generate two-dimensional back-projected images (BPG_1, BPG_2) from three-dimensional data (3DRA) based on a transformation matrix (M_TRSF) and camera matrices (M_CAM1, M_CAM2).

[0110] The electronic device can determine a first registration score for a first back-projection image (BPG_1) and a first difference image (RDM_1). The electronic device can determine a second registration score for a second back-projection image (BPG_2) and a second difference image (RDM_2). The registration score represents the degree of registration of the vascular region, and may show a higher value as the vascular region of the back-projection images (BPG_1, BPG_2) and the vascular region of the difference images (RDM_1, RDM_2) are more aligned. For example, the registration score can be calculated based on pixel values.

[0111] The electronic device can determine an offset based on first and second alignment scores. For example, the electronic device can calculate a third alignment score by adding the first alignment score and the second alignment score. The electronic device can maintain the rotation matrix if the third alignment score is greater than or equal to a reference value. The electronic device can adjust the rotation matrix if the third alignment score is less than a reference value. For example, the electronic device can adjust the rotation matrix by rotating the three-dimensional data (3DRA). Offsetting can be determined due to the rotation matrix adjusted in this way. The electronic device can determine an optimal matrix based on the initial matrix and the offset.

[0112] Due to the rotation of the 3D data (3DRA), the center points may not coincide. That is, in the 3D data (3DRA) that is translated based on the initial transformation matrix and rotated based on the offset, the aneurysm center point and the coil center point may not coincide. Accordingly, the electronic device can translate the 3D data (3DRA) again based on the distance between the aneurysm center point and the coil center point. The electronic device can determine the optimal matrix updated due to the translation of the 3D data (3DRA) as the transformation matrix (M_TRSF).

[0113] Referring to FIG. 18, a first image (RDM_1), a second image (RDM_2), a first optimal back-projection image (BPG_R1), and a second optimal back-projection image (BPG_R2) according to one embodiment can be seen. The electronic device can generate the first optimal back-projection image (BPG_R1) based on a transformation matrix (M_TRSF) and a first camera matrix (M_CAM1). The electronic device can generate the second optimal back-projection image (BPG_R2) based on a transformation matrix (M_TRSF) and a second camera matrix (M_CAM2). It can be seen that the first optimal back-projection image (BPG_R1) aligns with the first image (RDM_1), and the second optimal back-projection image (BPG_R2) aligns with the second image (RDM_2).

[0114]

[0115] FIG. 19 is a flowchart illustrating a configuration for correcting a camera matrix and a transformation matrix according to one embodiment.

[0116] Referring to FIG. 19, an electronic device according to one embodiment can determine a third loss function (F_LOS3) based on a first loss function (F_LOS1) and a second loss function (F_LOS2) (S1910). The electronic device can determine the third loss function (F_LOS3) by combining the first loss function (F_LOS1) and the second loss function (F_LOS2). For example, the electronic device can add the first loss function (F_LOS1) and the second loss function (F_LOS2). With respect to the second loss function (F_LOS2) based on pixel values, bilinear interpolation can be used to make it a continuous function to solve the problem of discontinuous pixel values.

[0117] The electronic device can correct the second camera matrix (M_CAM2) and the transformation matrix (M_TRSF) using a first algorithm on the third loss function (F_LOS3) (S1920). The first algorithm may include gradient descent. By doing so, the electronic device can optimize seven parameters of the second camera matrix (M_CAM2) and six parameters of the transformation matrix (M_TRSF).

[0118] In this way, the electronic device according to one embodiment can match the geometric correlation between two cameras of a reference image and the relationship between each camera and 3D data by combining a first loss function (F_LOS1) and a second loss function (F_LOS2). In addition, the electronic device can obtain an optimal camera function and a transformation function even when there are few common points in the image.

[0119] In FIG. 19, for convenience of explanation, the configuration is described as fixing the first camera matrix and correcting the second camera matrix (M_CAM2), but the embodiment is not necessarily limited thereto. For example, the electronic device may fix the second camera matrix (M_CAM2) and correct the first camera matrix.

[0120]

[0121] FIG. 20 is a flowchart illustrating a configuration for determining a camera matrix according to one embodiment, and FIGS. 21 to 23 are drawings illustrating a configuration for determining a camera matrix according to one embodiment. An electronic device according to one embodiment can determine a camera matrix of a rotated image. The rotated image may include a third image of a first camera and a fourth image of a second camera.

[0122] Referring to FIG. 20, an electronic device according to one embodiment can obtain a third initial matrix (M_INIT3) of a third image (IMG_3) from a first camera (S2010). The third initial matrix (M_INIT3) may be a camera matrix associated with parameters when the first camera obtains the third image (IMG_3).

[0123] The electronic device can obtain a fourth initial matrix (M_INIT4) of a fourth image from a second camera (S2020). The fourth initial matrix (M_INIT4) may be a camera matrix associated with parameters when the second camera obtains the fourth image (IMG_4).

[0124] The electronic device can determine a first rotation offset (R_OFST1) and a second rotation offset (R_OFST2) based on three-dimensional data (3DRA), a third initial matrix (M_INIT3), a fourth initial matrix (M_INIT4), a third image (IMG_3), and a fourth image (IMG_4) (S2030).

[0125] For example, the electronic device can generate a point cloud based on the aneurysm region in three-dimensional data (3DRA). According to an embodiment, the electronic device may use a coil center point, an aneurysm center point, etc. The electronic device can acquire first carving data based on the point cloud, a transformation matrix (e.g., M_TRSF), a first camera matrix (e.g., M_CAM1), a second camera matrix (e.g., M_CAM2), a first image (e.g., IMG_1), and a second image (e.g., IMG_2). The configuration for the electronic device to acquire the first carving data may be applied in the same way as described with reference to FIG. 11. Accordingly, redundant descriptions are omitted.

[0126] The electronic device can determine rotation offsets (R_OFST1 and R_OFST2) based on the first carving data. Referring to FIG. 21, the electronic device can back-project the first carving data onto the third image (IMG_3) using the third initial matrix (M_INIT3) and the transformation matrix of the third image (IMG_3). The data (2110) back-projected onto the third image (IMG_3) may not be aligned with the coil (1010).

[0127] The electronic device can back-project the first carving data onto the fourth image (IMG_4) using the fourth initial matrix (M_INIT4) and the transformation matrix of the fourth image (IMG_4). The data (2120) back-projected onto the fourth image (IMG_4) can be verified.

[0128] The electronic device can generate a coil mask in the third and fourth images (IMG_3 and IMG_4). The electronic device can perform preprocessing on the third and fourth images (IMG_3 and IMG_4) based on the distance map of the third and fourth images (IMG_3 and IMG_4). For example, the electronic device can modify pixel values ​​so that they become brighter the closer they are to the coil mask. Referring to FIG. 22, the electronic device can generate a third preprocessed image (PMG_3) in the third image (IMG_3) that is brighter the closer it is to the first coil mask (2210) and darker the further it is from it. The electronic device can generate a fourth preprocessed image (PMG_4) in the fourth image (IMG_4) that is brighter the closer it is to the second coil mask (2220) and darker the further it is from it.

[0129] The electronic device can determine rotation offsets (R_OFST1 and R_OFST2) using a pixel value-based loss function. The first rotation offset (R_OFST1) may be an offset required to optimize the third initial matrix (M_INIT3) of the first camera, and the second rotation offset (R_OFST2) may be an offset required to optimize the fourth initial matrix (M_INIT4) of the second camera.

[0130] The electronic device may define a loss function based on pixel values ​​of a third image (IMG_3), a fourth image (IMG_4), a third preprocessed image (PMG_3), and a fourth preprocessed image (PMG_4). For example, the electronic device may define a third loss function for the third image (IMG_3) and the third preprocessed image (PMG_3). The electronic device may define a fourth loss function for the fourth image (IMG_4) and the fourth preprocessed image (PMG_4). The electronic device may define the loss function using bilinear interpolation. According to an embodiment, the electronic device may also determine rotation offsets (R_OFST1 and R_OFST2) using an isotropic distance-based loss function.

[0131] The electronic device may determine a first rotation offset (R_OFST1) using a first algorithm on a third loss function, and the electronic device may determine a second rotation offset (R_OFST2) using a first algorithm on a fourth loss function. The first algorithm may include gradient descent.

[0132] The electronic device can determine the third camera matrix (M_CAM3) of the first camera based on the third initial matrix (M_INIT3) and the first rotation offset (R_OFST1) (S2040). For example, the electronic device can determine the third camera matrix (M_CAM3) by adding the third initial matrix (M_INIT3) and the first rotation offset (R_OFST1). By doing so, the electronic device can optimize seven parameters of the third camera matrix (M_CAM3).

[0133] The electronic device can determine the fourth camera matrix (M_CAM4) of the second camera based on the fourth initial matrix (M_INIT4) and the second rotation offset (R_OFST2) (S2050). For example, the electronic device can determine the fourth camera matrix (M_CAM4) by adding the fourth initial matrix (M_INIT4) and the second rotation offset (R_OFST2). By doing so, the electronic device can optimize seven parameters of the fourth camera matrix (M_CAM4).

[0134] The electronic device can back-project the first carving data onto the third image (IMG_3) using the third camera matrix (M_CAM3) and the transformation matrix of the third image (IMG_3). Based on the data back-projected onto the third image (IMG_3), the electronic device can obtain cloud data by removing points among the first carving data that do not correspond to the coil (1010) of the third image (IMG_3).

[0135] The electronic device can back-project cloud data onto the fourth image (IMG_4) using the fourth camera matrix (M_CAM4) and the transformation matrix of the fourth image (IMG_4). Based on the data back-projected onto the fourth image (IMG_4), the electronic device can obtain second carving data by removing points in the cloud data that do not correspond to the coil region of the fourth image (IMG_4). The electronic device can determine a three-dimensional coil shape from the second carving data.

[0136] Referring to FIG. 23, a third image (IMG_3), a fourth image (IMG_4), and back-projection data (2310, 2320) according to one embodiment can be seen. It can be seen that the back-projection data (2310) generated by the electronic device based on the second carving data, the third camera matrix (M_CAM3), and the transformation matrix aligns with the coil region of the third image (IMG_3), and the back-projection data (2320) generated based on the second carving data, the fourth camera matrix (M_CAM4), and the transformation matrix aligns with the coil region of the fourth image (IMG_4).

[0137]

[0138] FIG. 24 shows an example of three-dimensional data according to one embodiment.

[0139] Referring to FIG. 24, an electronic device according to one embodiment can generate a three-dimensional coil shape based on second carving data, first to fourth camera matrices, and transformation matrices. The electronic device can perform point reconstruction using a carving method. The electronic device can generate first carving data based on a point cloud, transformation matrices, first and second camera matrices, and first and second images. The electronic device can generate second carving data based on first carving data, transformation matrices, third and fourth camera matrices, and third and fourth images. Carving data may refer to a set of three-dimensional points.

[0140] The electronic device can perform surface reconstruction on the second carving data. Surface reconstruction may mean turning three-dimensional points into a smooth surface. For example, the electronic device can generate a three-dimensional surface mesh using a marching cube algorithm on the second carving data.

[0141] According to an embodiment, the electronic device can control concavity by adjusting the ISO level. For example, the electronic device can back-project a 3D surface mesh onto a 2D image. The electronic device can define a score function based on the Intersection Over Union (IOU), which indicates the degree of correspondence between the back-projected data and the coil region. The electronic device can generate the most accurate 3D surface by selecting the ISO level with the highest IOU score. The generated 3D surface can be understood as a coil shape.

[0142] Referring to FIG. 24, three-dimensional data (3DRA) according to one embodiment may include a blood vessel mesh (VMES), an aneurysm region (ANRG), and a three-dimensional coil shape (CLSP). The blood vessel mesh (VMES) and the aneurysm region (ANRG) may be the same as the blood vessel mesh (VMES) and the aneurysm region (ANRG) illustrated in FIG. 3. That is, the electronic device generates a coil shape (CLSP), and the coil shape (CLSP) may be placed together with the blood vessel mesh (VMES) and the aneurysm region (ANRG) in the three-dimensional data (3DRA).

[0143] In this way, the electronic device can provide information necessary for the procedure and evaluation of coil embolization by utilizing two-dimensional images to reconstruct the accurate three-dimensional shape of the coil. The first user (patient) can minimize exposure to radio waves and contrast agents, and the second user (medical staff) can achieve efficient coil embolization by rapidly obtaining the three-dimensional coil shape.

[0144]

[0145] It is obvious that each step or operation of the method according to the embodiments of the present disclosure may be performed by a computer comprising one or more processors in accordance with the execution of a computer program stored in a computer-readable recording medium.

[0146] The computer-executable instructions stored on the aforementioned recording medium can be implemented through a computer program programmed to perform each corresponding step, and such a computer program can be stored on a computer-readable recording medium and executed by a processor. The computer-readable recording medium may be a non-transitory readable medium. In this case, a non-transitory readable medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short moment, such as a register, cache, or memory. Specifically, programs for performing the various methods described above may be provided by being stored on a non-transitory readable medium, such as semiconductor memory devices including erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices; magnetic disks such as internal hard disks and removable disks; optical-magnetic disks; and non-volatile memory including CD-ROMs and DVD-ROMs.

[0147] Methods according to the various examples disclosed in this document may be provided by being included in a computer program product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0148] As explained above, a person skilled in the art to which this disclosure pertains will understand that this disclosure may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of this disclosure is defined by the claims set forth below rather than by the detailed description, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts should be interpreted as being included within the scope of this disclosure.

[0149] The features and advantages described herein are not all included, and in particular, many additional features and advantages will become apparent to those skilled in the art by considering the drawings, the specification, and the claims. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and instructional purposes and may not be chosen to describe or limit the subject matter of this disclosure.

[0150] The foregoing description of the embodiments of the present disclosure is provided for illustrative purposes only. It is not intended to limit the present disclosure to the exact form disclosed or to make it incomplete. Those skilled in the art will understand that many modifications and variations are possible in light of the foregoing disclosure.

[0151] Therefore, the scope of the present disclosure is not limited by the detailed description but by any of the claims of the application based thereon. Accordingly, the disclosure of embodiments of the present disclosure is illustrative and does not limit the scope of the present disclosure as set forth in the following claims.

Claims

1. A step of acquiring three-dimensional data including the aneurysm region; A step of acquiring a first image of an intravascular coil at a first position using a first camera; A step of acquiring a second image of the coil at a second position using a second camera; A step of obtaining a third image of the coil at a third position using the first camera; A step of acquiring a fourth image of the coil at a fourth position using the second camera; and A step of determining a three-dimensional coil shape based on the above three-dimensional data, the first image, the second image, the third image, and the fourth image. A method for generating a coil shape including 2. In Paragraph 1, The step of determining the above three-dimensional coil shape is, A step of determining a transformation matrix based on the above three-dimensional data, the first image, the second image, the third image, and the fourth image; A step of determining a plurality of camera matrices based on the above three-dimensional data, the first image, the second image, the third image, and the fourth image - each of the plurality of camera matrices corresponds to each of the first to fourth images -; and A step of determining the three-dimensional coil shape based on the plurality of camera matrices and transformation matrices. A method for generating a coil shape including 3. In Paragraph 2, The step of determining the above three-dimensional coil shape is, A step of generating a point cloud based on the aneurysm region in the above three-dimensional data; A step of obtaining first carving data based on the point cloud, the first image, and the second image; A step of obtaining second carving data based on the first carving data, the third image, and the fourth image; and Step of determining the three-dimensional coil shape from the second carving data A method for generating a coil shape including 4. In Paragraph 3, The step of generating the above point cloud is, A step of determining the center point of the aneurysm region; A step of determining a polyhedron centered on the above-mentioned center point and having a volume larger than the volume of the aneurysm region; and Step of generating the point cloud within the polyhedron A method for generating a coil shape including 5. In Paragraph 3, The step of acquiring the above-mentioned first carving data is, A step of obtaining first cloud data by removing points that do not correspond to the first coil region of the first image from the point cloud using the first camera matrix of the first image; and A step of obtaining the first carving data by removing points from the first cloud data that do not correspond to the second coil region of the second image using the second camera matrix of the second image. A method for generating a coil shape including 6. In Paragraph 5, The step of acquiring the above second carving data is, A step of obtaining second cloud data by removing points that do not correspond to the third coil region of the third image from the first carving data using the third camera matrix of the third image; and A step of obtaining the second carving data by removing points that do not correspond to the fourth coil region of the fourth image from the second cloud data using the fourth camera matrix of the fourth image. A method for generating a coil shape including 7. In Paragraph 2, The step of determining the plurality of camera matrices above is, A step of obtaining a first initial matrix of the first image from the first camera; A step of determining the above first initial matrix as the first camera matrix; A step of obtaining a second initial matrix of the second image from the second camera; A step of determining a common image point in the first image and the second image; and A step of determining a second camera matrix by correcting the second initial matrix based on the common image point. A method for generating a coil shape including 8. In Paragraph 7, The step of determining a common image point in the first image and the second image is, A step of determining a first common point in the first image above; and Step of determining a second common point in the second image above Includes, The step of determining the second camera matrix above is, A step of determining the epipolar line of the first common point and the second common point; and A step of determining the second camera matrix based on the distance between the first common point and the second common point and the equal line. A method for generating a coil shape including 9. In Paragraph 8, The step of determining the second camera matrix based on the distance between the first common point and the second common point and the equal line is A step of determining the second camera matrix using a first algorithm on a first loss function based on the distance. A method for generating a coil shape including 10. In Paragraph 9, The step of determining the above transformation matrix is, A step of determining the transformation matrix based on the first camera matrix, the second camera matrix, the three-dimensional data, the first image, and the second image. A method for generating a coil shape including 11. In Paragraph 10, A step of obtaining a first background image corresponding to the first image above; A step of generating a first differential image by subtracting the first background image from the first image; A step of acquiring a second background image corresponding to the second image above; and A step of generating a second image by subtracting the second background image from the second image. Includes more, The step of determining the transformation matrix based on the first camera matrix, the second camera matrix, the three-dimensional data, the first image, and the second image is: A step of determining the transformation matrix based on the first camera matrix, the second camera matrix, the three-dimensional data, the first image, and the second image. A method for generating a coil shape including 12. In Paragraph 11, The step of determining the transformation matrix based on the first camera matrix, the second camera matrix, the three-dimensional data, the first image, and the second image is: A step of generating a first back-projection image from the three-dimensional data based on the first camera matrix; A step of generating a second back-projection image from the three-dimensional data based on the second camera matrix; and A step of determining the transformation matrix using a second algorithm on a second loss function based on the first backprojection image, the second backprojection image, the first difference image, and the second difference image. A method for generating a coil shape including 13. In Paragraph 12, A step of determining a third loss function by combining the first loss function and the second loss function; and A step of correcting the second camera matrix and the transformation matrix using the first algorithm on the third loss function. A method for generating a coil shape that further includes 14. In Paragraph 13, The step of determining the plurality of camera matrices above is, A step of obtaining a third initial matrix of the third image from the first camera; A step of obtaining a fourth initial matrix of the fourth image from the second camera; A step of determining a first rotation offset and a second rotation offset based on the above three-dimensional data, the above third initial matrix, the above fourth initial matrix, the above third image, and the above fourth image; A step of determining a third camera matrix of the first camera based on the third initial matrix and the first rotation offset; and A step of determining the fourth camera matrix of the second camera based on the fourth initial matrix and the second rotation offset. A method for generating a coil shape including 15. In Paragraph 14, The step of determining the rotation offset above is, A step of generating a point cloud based on the aneurysm region in the above three-dimensional data; A step of acquiring first carving data based on the point cloud, the first camera matrix, the second camera matrix, the first image, and the second image; A step of obtaining second carving data based on the first carving data, the third initial matrix, the fourth initial matrix, the third image, and the fourth image; and A step of determining a rotation offset based on the second carving data, the third image, and the fourth image. A method for generating a coil shape including 16. In Paragraph 11, The step of determining the transformation matrix based on the first camera matrix, the second camera matrix, the three-dimensional data, the first image, and the second image is, A step of determining a first coil region from the first image above; A step of determining a second coil region from the second image above; A step of determining the first center point of the first coil region; A step of determining the second center point of the second coil region; A step of determining a three-dimensional coil center point based on the first center point, the second center point, the first position, and the second position; and A step of determining the transformation matrix based on the first center point, the second center point, and the coil center point. A method for generating a coil shape including 17. In Paragraph 16, The step of determining a first coil region from the first image above is: A step of generating a region of interest image by removing a masking area from the first image above; A step of calculating pixel values ​​and edge values ​​in the above region of interest image; and A step of determining the first coil region based on the pixel value and the edge value. A method for generating a coil shape including 18. In Paragraph 16, The step of determining a three-dimensional coil center point based on the first center point, the second center point, the first position, and the second position is, A step of generating a first straight line connecting the first center point and the first position; A step of generating a second straight line connecting the second center point and the second position; and A step of determining the coil center point based on the shortest distance between the first straight line and the second straight line. A method for generating a coil shape including 19. In Paragraph 16, The step of determining the transformation matrix based on the first center point, the second center point, and the coil center point is, A step of obtaining a transformation initial matrix that moves the aneurysm center point of the above 3D data to the coil center point; A step of determining an offset of a rotation matrix based on pixel values ​​of a first back-projection image for the first image of the three-dimensional data and pixel values ​​of a second back-projection image for the second image of the three-dimensional data; and A step of determining the transformation matrix by moving the aneurysm center point to the coil center point in 3D data that is moved based on the initial transformation matrix and rotated based on the offset. A method for generating a coil shape including 20. A processor and a memory connected to the processor, and The above memory is configured to store a program, and The above processor is configured to execute the above program, and When the above program is executed, the steps of the method of any one of claims 1 to 19 are implemented, Electronic device.

Images