Method, electronic device, and computing system for generating three-dimensional data

Abstract

A method for generating three-dimensional data, according to one embodiment, comprises the steps of: acquiring medical images; generating three-dimensional data on the basis of the medical images; generating a user point on the three-dimensional data on the basis of a user input; and verifying the user point by rotating the three-dimensional data.

Description

Method for generating three-dimensional data, electronic device, and computing system

[0001] The disclosure relates to a method for generating three-dimensional data, an electronic device, and a computing system.

[0002] An aneurysm is a disease in which a portion of an artery expands like a balloon due to weakened artery walls; related conditions may include cerebral aneurysms, aortic aneurysms, renal artery aneurysms, and splenic artery aneurysms. Among the methods for treating such aneurysms, coil embolization is a representative treatment method that involves inserting a thin coil into the aneurysm to prevent blood from flowing into the cerebral aneurysm.

[0003] For the treatment of such coil embolization, the amount of coil is determined based on size information such as the volume and length of the aneurysm; therefore, precisely measuring the size information of the aneurysm can be considered the most important factor in aneurysm treatment. Previously, medical staff predicted the size information of the aneurysm by viewing images of the patient's aneurysm and using polygons such as octahedrons, but there is a need for a method to determine the size information more precisely to match the actual aneurysm.

[0004] One embodiment aims to provide a method, an electronic device, and a computing system for generating three-dimensional data to obtain precise size information of an aneurysm.

[0005] However, the problems that the present invention aims to solve are not limited to those mentioned above, and may include problems that are not mentioned but can be clearly understood by those skilled in the art from the description below.

[0006] A method for generating three-dimensional data according to one embodiment for solving such technical problems includes the steps of: acquiring medical images; generating three-dimensional data based on the medical images; generating user points on the three-dimensional data based on user input; and verifying the user points by rotating the three-dimensional data.

[0007] An electronic device according to one embodiment includes a processor and a memory connected to the processor, the memory is configured to store a program, the processor is configured to execute the program, and when the program is executed, the steps of the method according to the embodiment are implemented.

[0008] A computing system according to one embodiment includes: a shooting device configured to capture blood vessels and generate a plurality of images; and an electronic device configured to generate three-dimensional data based on the plurality of images, generate user points on the three-dimensional data based on user input, determine the ranking of rotation directions defined based on the user points, rotate the three-dimensional data based on the determined ranking to verify the user points, and if the verification of the user points is successful, generate a region of interest on the three-dimensional data based on the user points.

[0009] FIG. 1 is a schematic block diagram of a computing system according to one embodiment.

[0010] FIG. 2 is a block diagram of an electronic device according to one embodiment.

[0011] FIG. 3 is a flowchart illustrating a method for generating three-dimensional data according to one embodiment.

[0012] FIG. 4 is an example of a blood vessel mesh according to one embodiment.

[0013] FIG. 5 is an example of a drawing for explaining user input according to one embodiment.

[0014] FIG. 6 is a flowchart illustrating a method for rotating three-dimensional data according to one embodiment.

[0015] FIG. 7 is a diagram illustrating a method for rotating three-dimensional data according to one embodiment.

[0016] FIG. 8 is a diagram illustrating a method for rotating three-dimensional data according to one embodiment.

[0017] FIG. 9 is a diagram illustrating a method for rotating three-dimensional data according to one embodiment.

[0018] FIG. 10 is a flowchart illustrating a method for an electronic device to process three-dimensional data according to one embodiment.

[0019] FIGS. 11 and FIGS. 12 are drawings for explaining a method of separating a blood vessel to which an electronic device is attached according to one embodiment.

[0020] The various embodiments described in this specification are illustrative for the purpose of clearly explaining the technical concept of this disclosure and are not intended to limit it to specific embodiments. The technical concept of this disclosure includes various modifications, equivalents, alternatives, and embodiments optionally combined from all or part of each embodiment described in this specification. Furthermore, the scope of the technical concept of this disclosure is not limited to the various embodiments presented below or the specific descriptions thereof.

[0021] Terms used in this specification, including technical or scientific terms, may have the meaning generally understood by those skilled in the art to which this disclosure pertains, unless otherwise defined.

[0022] Expressions used herein such as “comprising,” “may compose,” “possessing,” “possessing,” “having,” and “possessing” imply the existence of the subject feature (e.g., function, operation, or component, etc.) and do not exclude the existence of other additional features. That is, such expressions should be understood as open-ended terms implying the possibility of including a second embodiment.

[0023] In this specification, singular expressions include plural expressions unless the context clearly specifies them as singular. Additionally, plural expressions include singular expressions unless the context clearly specifies them as plural. Throughout the specification, when a part is described as including a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0024] Additionally, the terms 'module' or 'part' as used in the specification refer to software or hardware components, and the 'module' or 'part' performs certain roles. However, the meaning of 'module' or 'part' is not limited to software or hardware. The 'module' or 'part' may be configured to reside in an addressable storage medium or configured to run on one or more processors. Thus, as an example, the 'module' or 'part' may include components such as software components, object-oriented software components, class components, and task components, and at least one of processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The components and the functions provided within the 'module' or 'part' may be combined into a smaller number of components and 'modules' or 'parts', or further separated into additional components and 'modules' or 'parts'.

[0025] According to one embodiment of the present disclosure, a ‘module’ or ‘part’ may be implemented as a processor and memory. The term ‘processor’ should be broadly interpreted to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some environments, the term ‘processor’ may refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), etc. The term ‘processor’ may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors combined with a DSP core, or any other combination of such configurations. Additionally, the term ‘memory’ should be broadly interpreted to include any electronic component capable of storing electronic information. 'Memory' may refer to various types of processor-readable media, such as Random Access Memory (RAM), Read-Only Memory (ROM), Non-Volatile Random Access Memory (NVRAM), Programmable Read-Only Memory (PROM), Erasable-Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), Flash Memory, Magnetic or Optical Data Storage Devices, Registers, etc. If a processor can read information from memory and / or write information to memory, the memory is said to be in an electronic communication state with the processor. Memory integrated into a processor is in an electronic communication state with the processor.

[0026] Expressions such as "first," "second," or "first," "second" as used in this specification are used to distinguish one object from another when referring to a plurality of objects of the same kind, unless otherwise indicated in the context, and do not limit the order or importance of said objects.

[0027] Expressions used herein such as “A, B, and C,” “A, B, or C,” “A, B, and / or C,” or “at least one of A, B, and C,” “at least one of A, B, or C,” “at least one of A, B, and / or C,” “at least one selected from A, B, and C,” “at least one selected from A, B, or C,” “at least one selected from A, B, and / or C,” etc., may mean each of the listed items or all possible combinations of the listed items. For example, “at least one selected from A and B” may refer to (1) A, (2) at least one of A, (3) B, (4) at least one of B, (5) at least one of A and at least one of B, (6) at least one of A and B, (7) at least one of B and A, and (8) all of A and B.

[0028] As used herein, the expression “based on” is used to describe one or more factors affecting an act or action of a decision or judgment described in the phrase or sentence containing such expression, and such expression does not exclude additional factors affecting said act or action of a decision or judgment.

[0029] As used in this specification, the expression that a certain component (e.g., a first component) is "connected" or "connected" to another component (e.g., a second component) may mean that the said certain component is not only directly connected or connected to the said other component, but is also connected or connected through a new other component (e.g., a third component).

[0030] As used herein, the expression "configured to" may have meanings such as "set to," "capable of," "modified to," "made to," or "capable of." Such expression is not limited to the meaning of "specifically designed in hardware," and, for example, a processor configured to perform a specific operation may mean a generic-purpose processor capable of performing that specific operation by executing software.

[0031] Various embodiments of the present disclosure will be described below with reference to the accompanying drawings. In the accompanying drawings and the description thereof, identical or substantially equivalent components may be given the same reference numerals. Furthermore, in the description of the various embodiments below, the description of identical or corresponding components may be omitted, but this does not mean that such components are not included in the embodiments.

[0032]

[0033] FIG. 1 is a schematic block diagram of a computing system according to one embodiment, and FIG. 2 is a block diagram of an electronic device according to one embodiment.

[0034] Referring to FIG. 1, a computing system (10) according to one embodiment may acquire a blood vessel image of a first user, perform image processing on the acquired blood vessel image, and display the processed image. The computing system (10) may provide the processed image to a second user. For example, the first user may be a patient, and the second user may be a medical professional.

[0035] In one embodiment, the computing system (10) may obtain a blood vessel image by photographing the blood vessel of the first user and generate a region of interest by performing image processing on the blood vessel image. The computing system (10) may photograph blood vessels that are subject to angiography, such as cerebral blood vessels, cardiovascular vessels, and gastrointestinal blood vessels, but the embodiment is not necessarily limited thereto.

[0036] A computing system (10) according to one embodiment includes a shooting device (100) and an electronic device (200). The shooting device (100) may be a device configured to capture a first user and acquire an image. For example, the shooting device (100) may be an X-ray imaging device (e.g., C-arm X-ray device), angiography device (e.g., angio device), optical coherence tomography (OCT) device, computed tomography (CT) device, magnetic resonance imaging (MRI), magnetic resonance angiography (MRA) device, etc., but the embodiment is not necessarily limited thereto and may be implemented as various devices configured to capture the blood vessels of the first user.

[0037] The imaging device (100) can photograph a first user at multiple shooting points and acquire multiple images. In one embodiment, the imaging device (100) can photograph the first user while rotating around the first user. In another embodiment, the imaging device (100) can rotate the first user and photograph the rotating first user. The imaging device (100) can transmit the acquired multiple images to an electronic device (200). Depending on the embodiment, the imaging device (100) may be implemented as a mono-plane device or a biplane device.

[0038] The electronic device (200) can generate a region of interest based on a plurality of images. For example, the region of interest may refer to an aneurysm region. The electronic device (200) can perform the method of generating three-dimensional data of FIG. 3. The three-dimensional data may include a region of interest. The electronic device (200) can generate three-dimensional data based on a plurality of images, generate user points on the three-dimensional data based on user input, determine the ranking of rotation directions defined based on the user points, and verify the user points by rotating the three-dimensional data based on the determined ranking. If the verification of the user points is successful, the electronic device (200) can generate a region of interest in the three-dimensional data based on the user points.

[0039] In this way, the electronic device (200) can minimize the error between the user's intention and the actual designated point by rotating the 3D data during the process of verifying user points generated based on user input. Through this, accurate positioning is possible even in complex 3D structures. In addition, the electronic device (200) can reduce unnecessary calculations and achieve efficient processing speed by determining the rotation order according to rank. That is, the electronic device (200) enables a rapid verification process regardless of the complexity and size of the 3D data.

[0040] The electronic device (200) may be a server, data center, artificial intelligence (AI) device, personal computer (PC), laptop computer, mobile phone, smartphone, tablet PC, wearable device, healthcare device, etc.

[0041] In some embodiments, where the electronic device (200) is implemented as a server, data center, etc., the computing system (10) may further include an electronic device for interacting with a second user. For example, such an electronic device may interact with the second user, such as displaying an image to the second user and receiving input from the second user. The electronic device of the second user may communicate with the shooting device (100) and / or the electronic device (200).

[0042] Referring to FIG. 2, an electronic device (200) according to one embodiment may include a processor (210) and a memory (220) connected to the processor (210). The memory (220) may be configured to store a program. The processor (210) may be configured to execute a program in the memory (220). When the program is executed, a method for generating three-dimensional data (or steps of the method) according to the embodiments may be implemented.

[0043] The memory (220) may be configured to store an image received from the imaging device (100). The image stored in the memory (220) may be loaded by the processor (210) and used to generate three-dimensional data. The processor (210) may perform calculations based on the image received from the imaging device (100) and generate three-dimensional data.

[0044] In FIG. 2, for convenience of explanation, the electronic device (200) is illustrated as including a processor (210) and a memory (220), but the embodiment is not necessarily limited thereto, and the electronic device (200) may be implemented to include at least one additional component. For example, the electronic device (200) may include additional components such as an input / output interface, a communication module, and a display. The input / output interface may be a component for an interface between an input / output device and the electronic device (200). The communication module may be a component for communicating with a device outside the electronic device (200) (e.g., a camera (100), a server, etc.). The display may be a component for displaying the output of the processor (210) (e.g., an input image (medical image), 3D data, etc.).

[0045]

[0046] FIG. 3 is a flowchart illustrating a method for generating three-dimensional data according to one embodiment, FIG. 4 is an example of a blood vessel mesh according to one embodiment, and FIG. 5 is an example of a drawing for explaining user input according to one embodiment.

[0047] Referring to FIGS. 1 and FIGS. 3, an electronic device (200) according to one embodiment can acquire medical images (S310). For example, the electronic device (200) can receive a plurality of images captured by a imaging device (100). The plurality of images received by the electronic device (200) may include blood vessels and may include X-ray images, ultrasound (or sonography) images, CT (Computed Tomography) images, PET (Positron Emission Tomography) images, MRI (Magnetic Resonance Imaging) images, fMRI (functional Magnetic Resonance Imaging) images, digital pathology WSI (WSI) images, DBT (Digital Breast Tomosynthesis) images, etc. The electronic device (200) may be a computing device that performs image processing on images input from the imaging device (100).

[0048] The electronic device (200) can generate three-dimensional data based on medical images (S320). For example, the electronic device (200) can generate a vessel mesh from an input image. The vessel mesh may be data representing a vessel in three dimensions. In one embodiment, the electronic device (200) may generate the vessel mesh using a marching cube algorithm, but the embodiment is not necessarily limited thereto.

[0049] Referring together with FIG. 4, an electronic device (200) according to one embodiment can generate a three-dimensional vascular mesh (400) from medical images. The vascular mesh (400) is composed of a three-dimensional surface, and the surface may include a plurality of cells. For example, a cell may be a two-dimensional plane formed using three points.

[0050] When the electronic device (200) generates a blood vessel mesh (400), it may use an initial threshold calculated by an algorithm from medical images. The electronic device (200) may classify the voxels of the medical images using the initial threshold. For example, the electronic device (200) may classify voxels greater than the initial threshold as blood vessels and voxels less than the threshold as background.

[0051] Referring again to FIGS. 1 and FIGS. 3, the electronic device (200) can determine a user point based on user input (S330). Referring together to FIG. 5, the electronic device (200) can display a blood vessel mesh (410) at a specific camera viewpoint (e.g., a first viewpoint) through a display. The electronic device (200) may change the camera viewpoint in response to a user's instruction to change the viewpoint and display the blood vessel mesh (410) at the changed camera viewpoint.

[0052] The electronic device (200) can receive user input at a second viewpoint (VP), which is a camera viewpoint. That is, the user can input user input to the electronic device (200) at the second viewpoint (VP). For example, the user can transmit user input through an input device such as a touchscreen panel (hand, stylus pen, etc.), a keyboard, a mouse, etc., of the electronic device (200). As another example, the user can transmit user input through a gesture near the display of the electronic device (200). For example, the display can receive the user's gesture without direct contact through a sensor that recognizes the user's movement. The electronic device (200) can determine a user point (UT) in response to the second viewpoint (VP) and user input.

[0053] The electronic device (200) can generate a straight line connecting a second viewpoint (VP) and user input in three dimensions. A vector from the second viewpoint (VP) to the user input can be displayed as an arrow. The electronic device (200) can determine the point where the generated straight line first touches the blood vessel mesh (410) as the user point (UT). The electronic device (200) can display the user point (UT) on a display.

[0054] Likewise, the electronic device (200) may further determine and display user points corresponding to additional user input. According to an embodiment, the electronic device (200) may further determine additional user points at the same camera viewpoint or at different camera viewpoints.

[0055] Referring again to FIGS. 1 and FIGS. 3, the electronic device (200) can verify the user point by rotating the three-dimensional data (S340). The electronic device (200) can rotate the three-dimensional data while changing the camera viewpoint to check whether the user point is designated according to the second user's intention. The electronic device (200) can determine the direction of rotation and rotate the three-dimensional data by a specified angle based on the determined direction of rotation.

[0056] For example, a second user may input user input into the electronic device (200) to specify an aneurysm area. However, it may not be easy to input a point in three-dimensional space through the display of the two-dimensional electronic device (200). Although the second user inputs user input into the electronic device (200) with the intention of specifying an aneurysm area, the electronic device (200) may determine and display the user point at a completely different location. Accordingly, the electronic device (200) may rotate the three-dimensional data to determine the user point and verify the user point. The configuration for the electronic device (200) to rotate the three-dimensional data will be described later with reference to FIG. 6.

[0057] In one embodiment, the electronic device (200) can determine an aneurysm region based on the user point if the verification of the user point is successful. If the verification of the user point fails, the electronic device (200) can determine a new user point based on the failed user point. For example, the electronic device (300) can predetermine a plurality of candidate points. The electronic device (300) can predetermine a plurality of candidate points based on an artificial neural network or an algorithm. The electronic device (300) can determine a candidate point among the plurality of candidate points that is located within a predetermined distance from the user point as a new user point.

[0058] According to an embodiment, the electronic device (200) may remove a failed user point and request a new user input from a second user.

[0059] By the electronic device (200) generating a vascular mesh from medical images and precisely separating the aneurysm area by determining it based on verified user points, the second user can precisely measure the parameters of the aneurysm included in the complex vascular structure. Therefore, the second user can observe the aneurysm area quickly and conveniently, thereby shortening the treatment time. In addition, the second user can make an accurate diagnosis and establish a treatment plan based on the measured parameters of the aneurysm area, thereby reducing the risk when performing a procedure or surgery and improving reliability.

[0060]

[0061] FIG. 6 is a flowchart illustrating a method for rotating three-dimensional data according to one embodiment, and FIG. 7 is a diagram for explaining a method for rotating three-dimensional data according to one embodiment. The method for rotating three-dimensional data according to one embodiment can be performed by an electronic device.

[0062] Referring to FIGS. 6 and 7, an electronic device according to one embodiment (e.g., 200 in FIG. 1) can determine a rank based on user points (UT) (S410). The rank may represent a priority for which direction to rotate the three-dimensional data. In FIG. 7, a mesh (MS) is shown as part of the three-dimensional data. The electronic device can determine user points (UT) on the mesh (MS) based on user input.

[0063] Multiple axes (AX1 to AX3) may be defined based on the camera viewpoint where the electronic device currently displays the mesh (MS). In one embodiment, the electronic device may define a first axis (AX1) in the up-down direction, a second axis (AX2) in the left-right direction, and a third axis (AX3) in the front-back direction based on the current camera viewpoint and the user point (UT). In this case, the first axis (AX1) and the second axis (AX2) may be orthogonal to a straight line (or vector) connecting the camera viewpoint and the user point (UT). The third axis (AX3) may coincide with a straight line connecting the camera viewpoint and the user point (UT).

[0064] In one embodiment, the electronic device can determine the rank of the first axis (AX1) and the second axis (AX2). The electronic device can determine the rank of the up, down, left, and right directions of the first axis (AX1) and the second axis (AX2). The electronic device can detect other meshes that intersect the first axis (AX1) and / or the second axis (AX2) in three-dimensional data. An axis and a mesh intersecting may mean that an axis passing through a user point (UT) is extended to pass through another point of the mesh. The electronic device can determine the rank based on the intersection points. The electronic device can determine the rank of the direction where no intersection point exists as the highest rank, 1st rank. If there are multiple directions where no intersection point exists, the electronic device may randomly select the direction to rotate first, or select the direction to rotate first based on a predetermined rank.

[0065] If only one intersection point is defined for the first axis (AX1) or the second axis (AX2), the electronic device may determine the direction in which the intersection point occurred as second priority. If multiple intersection points exist for the first axis (AX1) or the second axis (AX2), the electronic device may determine the priority based on the distance between the user point (UT) and the intersection point. The electronic device may assign a higher priority to directions that are further away. That is, the electronic device may assign the lowest priority to the direction that is closest in distance. For example, a first intersection point of a first distance may exist in the upward direction of the first axis (AX1), and a second intersection point of a second distance may exist in the right direction of the second axis (AX2). The first distance may be smaller than the second distance. The electronic device may determine the priority of the right direction to be higher than the priority of the upward direction (e.g., the right direction as second priority, the upward direction as third priority).

[0066] In one embodiment, the electronic device may determine the rank of the third axis (AX3) among a plurality of axes (AX1 to AX3) as the lowest. The electronic device may determine whether there is another mesh between the current camera viewpoint and the user point (UT). If there is no other mesh between the current camera viewpoint and the user point (UT), the electronic device may determine the rank of the third axis (AX3) as the lowest.

[0067] The electronic device can rotate three-dimensional data based on a ranking (S420). The electronic device can rotate the three-dimensional data sequentially from the first to the fourth ranking. For example, the electronic device can determine the upward direction as the first ranking, the right direction as the second ranking, the downward direction as the third ranking, and the left direction as the fourth ranking. The electronic device can rotate the three-dimensional data first in the upward direction, and then rotate it in the order of the right direction, the downward direction, and the left direction. However, the embodiment is not necessarily limited to this, and the electronic device can rotate the three-dimensional data in one or more of the plurality of rotation directions.

[0068] In this way, the electronic device helps the second user set an intended region of interest (e.g., an aneurysm region) through the rotation and verification process of the 3D data, and can improve input accuracy by providing visual feedback on the verified results. Additionally, the electronic device can reduce errors caused by the second user's incorrect input or unclear point designation and readjust the optimal position based on automated verification and intersection-based ranking. That is, the second user can conveniently determine whether the user input is well formed in the 3D data as a user point. The steps (S410 and S420) of FIG. 7 may be included in the step (S340) of FIG. 3.

[0069] In FIG. 7, for convenience of explanation, a configuration in which the electronic device sets three axes is described as an example, but the embodiment is not necessarily limited to this, and the electronic device can set various numbers of axes and determine the order to rotate three-dimensional data.

[0070]

[0071] FIG. 8 is a diagram illustrating a method for rotating three-dimensional data according to one embodiment.

[0072] Referring to FIG. 8, an electronic device according to one embodiment may define a first axis (AX1) in the up-down direction and a second axis (AX2) in the left-right direction based on the current camera viewpoint and user point (UR1). At this time, the first axis (AX1) and the second axis (AX2) may be orthogonal to a straight line (or vector) connecting the camera viewpoint and the user point (UR1).

[0073] In one embodiment, the electronic device can determine the order of the first axis (AX1) and the second axis (AX2). The electronic device can determine the order of the first axis (AX1) and the second axis (AX2) in the up, down, left, and right directions. The electronic device can detect other meshes that intersect the first axis (AX1) and / or the second axis (AX2) in three-dimensional data. For example, the electronic device can detect that the first axis (AX1) intersects the first intersection point (CX1) and the second intersection point (CX2) in the upward direction in three-dimensional data. Additionally, the electronic device can detect that the second axis (AX2) intersects the third intersection point (CX3) in the right direction.

[0074] The electronic device can determine that the first and second axes (AX1 and AX2) do not intersect with other meshes in the left and downward directions. The electronic device can determine the direction in which no intersection point exists as the first priority. That is, the electronic device can determine the left and downward directions as the first priority.

[0075] The electronic device can determine the order based on the point among the intersection points (CX1–CX3) that first touches another mesh. For example, the electronic device can determine the order of directions based on the first intersection point (CX1) and the third intersection point (CX3). Since the second intersection point (CX2) is not the first point where the first axis (AX1) intersects another mesh, the electronic device may not consider the second intersection point (CX2) when determining the order.

[0076] The electronic device can determine a first distance (L11) between a user point (UR1) and a first intersection point (CX1), and a second distance (L12) between a user point (UR1) and a third intersection point (CX3). The electronic device can compare the first distance (L11) and the second distance (L12). The electronic device can determine a ranking based on the comparison result. For example, the second distance (L12) may be longer than the first distance (L11). The electronic device may determine the right direction of the second distance (L12), which has a longer distance, as the second rank, and the upward direction of the first distance (L11), which has a relatively shorter distance, as the third rank. The ranking determined by the electronic device may be as shown in Table 1.

[0077] Rank / Direction 1 Left / Down 2 Right 3 Up

[0078]

[0079] An electronic device can rotate three-dimensional data based on rank. When the electronic device rotates the three-dimensional data in the left-right direction, it means rotating it around the first axis (AX1), and when it rotates it in the up-down direction, it means rotating it around the second axis (AX2). In one embodiment, the electronic device can select a direction with a higher rank in the up-down direction and a direction with a higher rank in the left-right direction. In the up-down direction, the electronic device can select the down direction after confirming that the rank of the down direction is 1st rank, which is higher than the up direction which is 3rd rank. Additionally, in the left-right direction, the electronic device can select the left direction after confirming that the rank of the left direction is 1st rank, which is higher than the right direction which is 2nd rank.

[0080] Accordingly, the electronic device can rotate the 3D data downwards based on the user point (UR1) and the second axis (AX2) at the current camera viewpoint. Then, the electronic device can rotate the 3D data to the left based on the user point (UR1) and the first axis (AX1) and return to the current camera viewpoint (initial camera viewpoint).

[0081] In another embodiment, the electronic device can rotate the three-dimensional data in order from a high rank to a low rank. The electronic device can rotate the three-dimensional data in at least one of the left and down directions from the current camera viewpoint. Then, the electronic device can rotate the three-dimensional data to the right, rotate it upward, and then return to the current camera viewpoint.

[0082] According to an embodiment, the electronic device may randomly select the direction to rotate first for directions having the same rank, or select the direction to rotate first based on a predetermined rank.

[0083] Additionally, according to an embodiment, the electronic device may rotate the three-dimensional data around a third axis (e.g., AX3 in FIG. 7). The third axis is an axis perpendicular to the first axis (AX2) and the second axis (AX2), and the electronic device may set the rank of the third axis to the lowest. The electronic device may rotate the three-dimensional data around the third axis just before returning to the current camera viewpoint.

[0084]

[0085] FIG. 9 is a diagram illustrating a method for rotating three-dimensional data according to one embodiment.

[0086] Referring to FIG. 9, an electronic device according to one embodiment may define a first axis (AX1) in the up-down direction and a second axis (AX2) in the left-right direction based on the current camera viewpoint and user point (UR2). At this time, the first axis (AX1) and the second axis (AX2) may be orthogonal to a straight line (or vector) connecting the camera viewpoint and the user point (UR2).

[0087] In one embodiment, the electronic device can determine the order of the first axis (AX1) and the second axis (AX2). The electronic device can determine the order of the first axis (AX1) and the second axis (AX2) in the up, down, left, and right directions. The electronic device can detect other meshes that intersect the first axis (AX1) and / or the second axis (AX2) in three-dimensional data. For example, the electronic device can detect that the first axis (AX1) intersects the first intersection point (CY1) and the second intersection point (CY2) in the upward direction and the third intersection point (CY3) in the downward direction in three-dimensional data. Additionally, the electronic device can detect that the second axis (AX2) intersects the fourth intersection point (CY4) and the fifth intersection point (CY5) in the right direction.

[0088] The electronic device can determine that the second axis (AX2) does not intersect with another mesh in the left direction. The electronic device can determine the direction where no intersection point exists as the first priority. That is, the electronic device can determine the left direction as the first priority.

[0089] The electronic device may determine the order based on the point among the intersection points (CY1–CY5) that first touches another mesh. For example, the electronic device may determine the order of directions based on the first intersection point (CY1), the third intersection point (CY3), and the fourth intersection point (CY4). Since the second and fifth intersection points (CY2 and CY5) are not the first points where the first and second axes (AX1 and AX2) intersect another mesh, the electronic device may not consider the second and fifth intersection points (CY2 and CY5) when determining the order.

[0090] The electronic device can determine a first distance (L21) between a user point (UR2) and a first intersection point (CY1), determine a second distance (L22) between a user point (UR2) and a third intersection point (CY3), and determine a third distance (L23) between a user point (UR2) and a fourth intersection point (CY4). The electronic device can compare the first to third distances (L21 to L23). The electronic device can determine a ranking based on the comparison results. For example, the third distance (L23) may be longer than the second distance (L22), and the second distance (L22) may be longer than the first distance (L21). The electronic device may determine the right direction corresponding to the third distance (L23) as the second ranking, the downward direction corresponding to the second distance (L22) as the third ranking, and the upward direction corresponding to the first distance (L21) as the fourth ranking. The ranking determined by the electronic device may be as shown in Table 2.

[0091] Rank Direction 1 Left 2 Right 3 Down 4 Up

[0092]

[0093] In one embodiment, the electronic device may determine the ranking of directions in which the difference between distances among the first to third distances (L21 to L23) is within a predetermined range. For example, the electronic device may determine that the difference between the first and second distances (L21 and L22) is within a first reference value. The electronic device may determine the ranking of the upper direction and the lower direction corresponding to the first and second distances (L21 and L22).

[0094] The electronic device can determine the ranking based on the intersection points through which the first axis (AX1) corresponding to the first and second distances (L21 and L22) additionally passes in the three-dimensional data. The electronic device can confirm that a second intersection point (CY2) exists in the upper direction and that there are no additional intersection points in the lower direction. The electronic device can determine the ranking of the lower direction, which has fewer intersection points than the upper direction, to be higher.

[0095] According to an embodiment, the electronic device can count the number of intersection points within a distance of a second reference value from a user point (UR2). The second reference value may be greater than the first reference value. The electronic device may determine a higher rank for directions with fewer intersection points. That is, the electronic device may determine a lower rank for directions with more intersection points.

[0096] The electronic device can rotate three-dimensional data based on rank. When the electronic device rotates the three-dimensional data in the left-right direction, it means rotating it around the first axis (AX1), and when it rotates it in the up-down direction, it means rotating it around the second axis (AX2). In one embodiment, the electronic device can select the direction with the higher rank in the up-down direction and the direction with the higher rank in the left-right direction. The electronic device can select the left direction by confirming that the left direction has the highest rank in the left-right direction, which is higher than the right direction, which has the second rank. Additionally, the electronic device can select the down direction by confirming that the down direction has the highest rank in the up-down direction, which is higher than the up direction, which has the fourth rank. Accordingly, the electronic device can rotate the three-dimensional data to the left direction based on the user point (UR2) and the first axis (AX1) from the current camera viewpoint, and rotate the three-dimensional data to the down direction based on the user point (UR2) and the second axis (AX2). After that, the electronic device can return to the current camera viewpoint.

[0097] In another embodiment, the electronic device can rotate the three-dimensional data in order from highest to lowest rank. The electronic device can rotate the three-dimensional data to the left, right, downward, and upward directions from the current camera viewpoint, and then return to the current camera viewpoint.

[0098] Additionally, according to an embodiment, the electronic device may rotate the three-dimensional data around a third axis (e.g., AX3 in FIG. 7). The third axis is an axis perpendicular to the first axis (AX2) and the second axis (AX2), and the electronic device may set the rank of the third axis to the lowest. The electronic device may rotate the three-dimensional data around the third axis just before returning to the current camera viewpoint.

[0099]

[0100] FIG. 10 is a flowchart illustrating a method of processing three-dimensional data using an electronic device according to one embodiment, and FIG. 11 and FIG. 12 are drawings for explaining a method of separating a blood vessel attached to an electronic device according to one embodiment.

[0101] Referring to FIG. 10, an electronic device according to one embodiment can acquire medical images and generate three-dimensional data based on the medical images (S320). The electronic device can generate a three-dimensional blood vessel mesh using an initial threshold calculated by an algorithm from the medical images. The electronic device can classify the voxels of the medical images using the initial threshold. For example, the electronic device can classify voxels larger than the initial threshold as blood vessels and voxels smaller than or equal to the threshold as background.

[0102] Referring to FIG. 11, the electronic device can generate a vascular mesh (MESH1) from medical images based on an initial threshold. The vascular mesh (MESH1) may include conjoined vascular regions (KVSR), noise regions (NVS1, NVS2), etc. Conjoined vascular regions (KVSR) are actually separated blood vessels, and noise regions (NVS1, NVS2) are actually regions that are not blood vessels, but may have been generated by a contrast agent or noise generated during the imaging process. The description of step (S320) can be applied in the same way as the description given with reference to FIG. 3. Accordingly, redundant content is omitted.

[0103] The electronic device can obtain a user threshold (S510). The user can input the user threshold by adjusting (increasing or decreasing) the threshold on the electronic device. For example, the user can adjust the threshold through the display. Accordingly, the threshold can be changed from an initial threshold to a user threshold.

[0104] The electronic device can process three-dimensional data based on a user threshold (S520). For example, the electronic device can perform image processing on the three-dimensional data to relax the attached blood vessels. The electronic device can process the attached blood vessels according to a user threshold that changes in real time. Accordingly, the user can adjust the threshold while observing changes in the blood vessel display according to the user threshold value.

[0105] With reference to Fig. 12, the electronic device can generate a vessel mesh (MESH2) based on a user threshold. It can be seen that the kissing vessel region (KVSR) of the vessel mesh (MESH1) is separated in the vessel mesh (MESH2). Additionally, it can be seen that noise regions are also removed in the vessel mesh (MESH2).

[0106] According to an embodiment, the electronic device can detect a threshold at which attached blood vessels begin to separate and suggest it to the user. In a configuration where the electronic device suggests a threshold, the process of obtaining a user threshold can be implemented without being omitted. The user may also fine-tune the user threshold based on the suggested threshold. In other words, the electronic device can suggest a minimum threshold at which the blood vessel mesh is output at its largest size without attached blood vessels or noise.

[0107] The electronic device can determine the aneurysm region from the generated three-dimensional data (S530). The generated three-dimensional data may refer to a mesh generated through threshold adjustment. The electronic device receives user input and can determine user points based on the user input. The contents of FIG. 3 may be applied equally to the user points. The electronic device verifies the user points, and if the verification is successful, can determine the aneurysm region based on the user points.

[0108] In one embodiment, the electronic device can remove unnecessary mesh portions in the aneurysm area. The unnecessary mesh portions may include artifacts such as attached blood vessels. The electronic device can retain the aneurysm area when there are no unnecessary mesh portions.

[0109] In one embodiment, the electronic device can separate blood vessels in an aneurysm region. For example, in the case of cerebral blood vessels, the electronic device can separate normal cerebral blood vessels and cerebral blood vessels to be analyzed in an aneurysm region. The cerebral blood vessels to be analyzed may be a region of interest (ROI). The electronic device can generate a three-dimensional mesh of the cerebral blood vessels to be analyzed. The electronic device can perform quantitative analysis on the three-dimensional mesh of the cerebral blood vessels to be analyzed.

[0110]

[0111] It is obvious that each step or operation of the method according to the embodiments of the present disclosure may be performed by a computer comprising one or more processors in accordance with the execution of a computer program stored in a computer-readable recording medium.

[0112] The computer-executable instructions stored on the aforementioned recording medium can be implemented through a computer program programmed to perform each corresponding step, and such a computer program can be stored on a computer-readable recording medium and executed by a processor. The computer-readable recording medium may be a non-transitory readable medium. In this case, a non-transitory readable medium refers to a medium that stores data semi-permanently and is readable by a device, rather than a medium that stores data for a short moment, such as a register, cache, or memory. Specifically, programs for performing the various methods described above may be provided by being stored on a non-transitory readable medium, such as semiconductor memory devices including erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices; magnetic disks such as internal hard disks and removable disks; optical-magnetic disks; and non-volatile memory including CD-ROM and DVD-ROM disks.

[0113] Methods according to the various examples disclosed in this document may be provided by being included in a computer program product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)) or online through an application store (e.g., Play Store™). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0114] As explained above, a person skilled in the art to which this disclosure pertains will understand that this disclosure may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. The scope of this disclosure is defined by the claims set forth below rather than by the detailed description, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts should be interpreted as being included within the scope of this disclosure.

[0115] The features and advantages described herein are not all included, and in particular, many additional features and advantages will become apparent to those skilled in the art by considering the drawings, the specification, and the claims. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and instructional purposes and may not be chosen to describe or limit the subject matter of this disclosure.

[0116] The foregoing description of the embodiments of the present disclosure is provided for illustrative purposes only. It is not intended to limit the present disclosure to the exact form disclosed or to make it incomplete. Those skilled in the art will understand that many modifications and variations are possible in light of the foregoing disclosure.

[0117] Therefore, the scope of the present disclosure is not limited by the detailed description but by any of the claims of the application based thereon. Accordingly, the disclosure of embodiments of the present disclosure is illustrative and does not limit the scope of the present disclosure as set forth in the following claims.

Claims

1. Step of acquiring medical images; A step of generating three-dimensional data based on the above medical images; A step of generating user points on the three-dimensional data based on user input; and A step of verifying the user point by rotating the above 3D data A method for generating three-dimensional data including 2. In Paragraph 1, The step of generating the above three-dimensional data is, A step of generating a first blood vessel mesh by applying an initial threshold to the medical images above; and A step of generating a second blood vessel mesh by modifying the initial threshold to a user threshold based on user input. A method for generating three-dimensional data including 3. In Paragraph 1, The step of generating the above user points is, A step of obtaining user input while displaying the above three-dimensional data; and A step of determining the point where the straight line defined by the current camera viewpoint and the user input first tangents the 3D data as the user point. A method for generating three-dimensional data including 4. In Paragraph 1, The step of verifying the user point by rotating the above three-dimensional data is, A step of determining at least one axis passing through the above user point; A step of determining the rank of rotation directions defined by at least one axis; and Step of rotating the 3D data based on the above ranking A method for generating three-dimensional data including 5. In Paragraph 4, The step of determining at least one axis passing through the above user point is, A step of calculating a plurality of axes parallel to or orthogonal to the straight line connecting the user point and the camera viewpoint. Includes, The step of determining the ranking of rotation directions defined by at least one axis is, Step of determining that the rank of the rotational direction of an axis orthogonal to the above line is higher than the rank of the rotational direction of an axis parallel to the above line A method for generating three-dimensional data including 6. In Paragraph 4, The step of determining the ranking of rotation directions defined by at least one axis is, A step of determining an intersection point where at least one axis is extended to first touch the three-dimensional data; and Step of determining the above ranking based on the above intersection point A method for generating three-dimensional data including 7. In Paragraph 6, The step of determining the ranking based on the above intersection is, A step of determining a first direction in which the above-mentioned intersection exists and a second direction in which the above-mentioned intersection does not exist; and Step of determining the ranking of the second direction as higher than the ranking of the first direction Includes, A method for generating three-dimensional data, wherein the first direction and the second direction are defined by the at least one axis.

8. In Paragraph 6, The above at least one axis includes a first axis and a second axis that are orthogonal to the straight line connecting the user point and the camera viewpoint, and The above intersection includes a first intersection on the first axis and a second intersection on the second axis, and The step of determining the ranking based on the above intersection is, A step of determining a first distance from the above user point to the above first intersection point; A step of determining a second distance from the above user point to the above second intersection point; Step of determining the ranking based on the first distance and the second distance A method for generating three-dimensional data including 9. In Paragraph 8, The step of determining the ranking based on the first distance and the second distance is, A step of determining the direction corresponding to the longer distance between the first distance and the second distance as having a higher rank. A method for generating three-dimensional data including 10. In Paragraph 8, The step of determining the ranking based on the first distance and the second distance is, A step of determining whether the difference between the first distance and the second distance is within a first reference value; If the difference is within the first reference value, a step of determining the number of additional intersection points existing within a distance of the second reference value on the first axis or the second axis based on the user point; and Step of determining the ranking based on the number of additional intersections mentioned above A method for generating three-dimensional data including 11. In Paragraph 4, The above at least one axis includes a first axis and a second axis that are orthogonal to the straight line connecting the user point and the camera viewpoint, and The above rotation directions include a first direction and a second direction defined by the first axis, and a third direction and a fourth direction defined by the second axis, and The step of determining the ranking of rotation directions defined by at least one axis is, A step of determining the direction of higher rank among the first direction and the second direction; and Step of determining the higher-ranking direction among the third direction and the fourth direction. Includes, The above first direction and the above second direction are opposite to each other, and The above third direction and the above fourth direction are opposite to each other, and The step of rotating the three-dimensional data based on the above ranking is, A step of rotating the three-dimensional data in the direction of higher priority among the first direction and the second direction, and in the direction of higher priority among the third direction and the fourth direction. A method for generating three-dimensional data including 12. In Paragraph 4, The step of verifying the user point by rotating the above three-dimensional data is, A step of rotating the 3D data based on the above ranking, and then returning the 3D data to the initial camera viewpoint. A method for generating 3D data that further includes 13. In Paragraph 1, A step of determining a region of interest in the 3D data based on the user points if the verification of the user points is successful. A method for generating 3D data that includes more.

14. A processor and a memory connected to the processor, and The above memory is configured to store a program, and The above processor is configured to execute the above program, and When the above program is executed, the steps of the method of any one of claims 1 to 13 are implemented, Electronic device.

15. A shooting device configured to photograph blood vessels and generate multiple images; and An electronic device configured to generate three-dimensional data based on the plurality of images, generate user points on the three-dimensional data based on user input, determine the ranking of rotation directions defined based on the user points, verify the user points by rotating the three-dimensional data based on the determined ranking, and generate a region of interest in the three-dimensional data based on the user points if the verification of the user points is successful. A computing system including

Images