System and method for diagnosing urinary stone

Abstract

The present invention relates to a system and a method for diagnosing a urinary stone. The system for diagnosing a urinary stone includes: a data receiving unit for receiving computed tomography image data of the urinary tract system; a three-dimensional (3D) urinary tract system data generation unit that generates 3D urinary tract system data by combining result data of a urinary tract system estimation module, a urinary tract precision prediction module, and a ureter classification module, generated on the basis of the image data; a stone candidate data generation unit that generates stone candidate data by searching for candidates similar to a stone by using a stone candidate search module on the basis of the image data; and a stone diagnosis unit that overlaps the 3D urinary tract system data and the stone candidate data to diagnose a stone, and diagnoses, from the overlapped region, a position determined as a urinary stone and a size of the stone.

Description

Urinary stone diagnosis system and method

[0001] The present invention relates to a system and method for diagnosing urinary stones using a plurality of modules, and more particularly, to a system and method for diagnosing urinary stones by estimating a urinary tract system based on image data to measure the size and location of urinary stones, generating stone candidates, and superimposing the estimated urinary tract system and stone candidates.

[0002] Urinary stones are a relatively common condition in which crystals form in the urinary tract due to various causes, such as genetic factors and anatomical abnormalities in the kidneys, ureters, and bladder. These crystals aggregate and grow larger to form stones, and as the stones pass through the ureters, they obstruct the flow of urine or cause acute pain.

[0003] If the stone is smaller than 5mm, natural discharge can be expected by drinking a large amount of water. However, if the stone is larger than 6mm, extracorporeal shock wave lithotripsy is generally used, which creates microscopic cracks in the stone using shock waves and then crushes it to naturally discharge it.

[0004] Additionally, stones are removed using ureteroscopy, which involves inserting an endoscope into the ureter and then using a laser to directly remove the stones.

[0005] However, urinary stones that do not respond to other treatments such as extracorporeal shock wave lithotripsy and ureteroscopic lithotripsy or are larger than 2.5 mm in the kidney are directly crushed and removed through percutaneous nephrolithotripsy.

[0006] If urinary stones occur, they can cause complications such as urinary tract infections, hydronephrosis, and renal failure, along with severe pain, frequent urination, vomiting, and abdominal distension. Accurate diagnosis of urinary stones is necessary because they must be differentiated from diseases such as acute cholecystitis, acute diverticulitis, bacterial cystitis, and acute appendicitis, which have similar symptoms to urinary stones that cause severe abdominal pain.

[0007] Currently used diagnostic tests for urinary stones mainly use abdominal X-rays and abdominal ultrasound to confirm the presence of stones, and to accurately determine the presence of urinary stones, CT (Computed Tomography) or intravenous pyelography is performed, and the results are interpreted to diagnose urinary stones.

[0008] However, during the interpretation process, it is difficult for the interpreter to interpret all of the images composed of a series of 500 to 800 images with a 1 mm standard in the CT image or to identify the location of the ureter with a small diameter of 2 mm.

[0009] In particular, it is difficult to distinguish when it overlaps with other parts such as the artery or psoas muscle of the blood vessel. The location of the stone is also likely to be confused with calcified material or high density material when it overlaps with an artery, and in the case of the bladder, it is difficult to estimate because the anatomical structure is different between men and women, and in particular, in the case of women, there may be calcified material (especially plaque) in the artery leading to the uterus, making the diagnosis of urinary stones difficult. Therefore, a technology that can accurately diagnose urinary stones is necessary.

[0010] As a conventional technique for diagnosing urinary stones, Patent Publication No. 10-2023-0094855 and Patent Registration No. 10-2209086 are known.

[0011] Patent Publication No. 10-2023-0094855 discloses a method of generating a segmented image by separating a urinary stone area by segmenting an image obtained by taking a CT scan of a patient's lesion area and then training it using a deep learning algorithm after a normalization process, and then quantifying data on the urinary stone area based on the segmented image to output the probability of natural discharge.

[0012] In addition, Patent Publication No. 10-2209086 discloses a method of detecting an area where stones exist using a machine learning model in multiple tomography images and automatically deriving information including the location and size of the stones, thereby providing information necessary for urinary stone surgery.

[0013] However, according to the prior art disclosed in Patent Publication No. 10-2023-0094855 and Patent Registration No. 10-2209086, there is a problem in that blood vessel plaques and bones can be confused with urinary stones in the images because only medical images are analyzed and learned using an artificial intelligence model to diagnose urinary stones.

[0014] Therefore, when constructing a urinary stone diagnosis system using multiple modules, if urinary stones are diagnosed using at least one module from the image data of the diagnosis target, a system can be constructed that can accurately diagnose the disease by distinguishing between blood vessel plaques and bones that can be confused with urinary stones in the image data.

[0015] [Patent Document]

[0016] Patent Publication No. 10-2023-0094855 (Published on June 28, 2023, Title: Deep Learning-Based Medical Image Automatic Urinary Stone Detection and Segmentation Method)

[0017] Patent Publication No. 10-2209086 (Registration date: January 22, 2021, Title: Urinary Stone Information Providing Method, Device, and Computer-Readable Medium)

[0018] The technical task of the present invention is to provide a urinary stone diagnosis system for making an accurate diagnosis of urinary stones using multiple modules from image data of a diagnosis target.

[0019] Furthermore, another object of the present invention is to provide a urinary stone diagnosis method capable of automatically diagnosing urinary stones from image data by estimating the urinary tract of a diagnosis target based on image data, generating a group of stone candidates, and superimposing the estimated urinary tract and stone candidates.

[0020] In order to solve such technical problems, the urinary stone diagnosis system according to the present invention includes a data receiving unit that receives image data obtained by taking a tomographic scan of the urinary tract, a urinary tract estimation module that estimates the shape and location of the urinary tract, a urinary tract precision prediction module, and a ureter classification module, and a 3D urinary tract data generation unit that generates 3D urinary tract data by fusing result data of the urinary tract estimation module, the urinary tract precision prediction module, and the ureter classification module generated based on the image data, a stone candidate data generation unit that searches for candidates similar to stones using a stone candidate search module based on the image data and generates stone candidate data, and a stone diagnosis unit that overlaps the 3D urinary tract data and the stone candidate data to diagnose stones, and diagnoses a location determined to be a urinary stone and the size of the stone from an overlapped area.

[0021] The above image data may include images of the kidneys, ureters, and bladder of a patient to be diagnosed, taken using at least one imaging device selected from among CT (Computed Tomography), MRI (Magnetic Resonance Imaging), X-ray, and ultrasound.

[0022] The above-described 3D urinary tract data generation unit includes the urinary tract estimation module, the urinary tract precision prediction module, and the ureter classification module for generating the 3D urinary tract data, wherein the urinary tract estimation module extracts a region of interest from the image data to estimate a rough location of the urinary tract, and the urinary tract precision prediction module analyzes the image data at a higher resolution than the urinary tract estimation module to perform segmentation to distinguish the anatomical locations of the kidney, ureter, and bladder corresponding to the coordinates of the region of interest, and the ureter classification module can classify the ureter into a proximal ureter, a middle ureter, and a distal ureter to predict a precise anatomical location of the ureter.

[0023] The above region of interest may include a three-dimensional boundary created from the left and right kidney ends to the bladder ends of the patient to be diagnosed based on the image data.

[0024] The above urinary tract estimation module and the urinary tract precision prediction module estimate the urinary tract of the patient to be diagnosed corresponding to the region of interest using a first algorithm, and the ureter classification module can classify the ureters of the patient to be diagnosed into the proximal ureter, the middle ureter, and the distal ureter using a second algorithm.

[0025] The above three-dimensional urinary tract data may include the location and shape of the left and right kidneys, the proximal and proximal ureters, the middle and proximal ureters, the distal and proximal ureters, and the bladder.

[0026] The above-mentioned stone candidate search module can search for candidates similar to the stone by deleting voxel blocks larger than 50 mm using 3D CCL (Connected Components Labeling) based on the image data.

[0027] The above absence candidate search module can learn candidates similar to the above absence using a third algorithm, and delete candidates similar to the above absence that are not determined to be absences through learning, thereby generating the above absence candidate data.

[0028] The above stone diagnosis unit maps the left and right kidneys, the proximal left and right ureters, the middle left and right ureters, the distal left and right ureters, and the bladder included in the three-dimensional urinary tract data into a three-dimensional space, and superimposes a candidate similar to the stone included in the stone candidate data onto the three-dimensional space mapped with the three-dimensional urinary tract data, and can diagnose the location and size of the stone determined to be the urinary tract stone from the overlapping area.

[0029] The method may further include a contrast agent determination module that can determine whether to administer contrast agent to a diagnosis subject using the image data received from the data receiving unit.

[0030] Meanwhile, a method for diagnosing urinary stones according to another embodiment of the present invention,

[0031] The method may include: receiving image data obtained by taking a CT scan of the urinary tract, including the kidney, ureter, and bladder of a patient to be diagnosed, from an imaging device; fusing result data for estimating the shape and location of the urinary tract generated using the urinary tract estimation module, the urinary tract precision prediction module, and the ureter classification module based on the image data to generate three-dimensional urinary tract data; searching for candidates similar to stones using a stone candidate search module based on the image data and generating stone candidate data; and overlapping the three-dimensional urinary tract data and the stone candidate data to diagnose stones, and diagnosing the location and size of stones determined to be urinary stones from the overlapped area.

[0032] The process of generating the three-dimensional urinary tract data using the urinary tract estimation module, the urinary tract precision module, and the ureter classification module may include a step of extracting a region of interest at a low resolution based on the image data using a first algorithm of the urinary tract estimation module to estimate a rough location of the urinary tract, a step of analyzing the image data at a higher resolution than the urinary tract estimation module using the first algorithm of the urinary tract precision prediction module and performing segmentation to distinguish the anatomical locations of the kidney, ureter, and bladder corresponding to the coordinates of the region of interest, a step of dividing the ureter into a proximal ureter, a middle ureter, and a distal ureter to predict a precise anatomical location of the ureter using a second algorithm of the ureter classification module, and a step of generating the three-dimensional urinary tract data by fusing the result data generated from the urinary tract estimation module, the urinary tract precision prediction module, and the ureter classification module.

[0033] The above region of interest may include a three-dimensional boundary created from the left and right kidney ends to the bladder ends of the patient to be diagnosed based on the image data.

[0034] The above three-dimensional urinary tract data may include the left and right kidneys, the bladder, the proximal left and right ureters, the middle left and right ureters, and the distal left and right ureters.

[0035] The process of generating stone candidate data using the above stone candidate search module may include a step of deleting a voxel block of 50 mm or more using 3D CCL (Connected Components Labeling) based on the image data and searching for a candidate similar to the stone, and a step of learning the stone-like candidate generated based on the image data using a third algorithm and deleting the stone-like candidate that is not determined to be a stone through the learning, thereby generating the stone candidate data.

[0036] The process of diagnosing the above stones is:

[0037] The method may include a step of mapping the left and right kidneys, the proximal left and right ureters, the middle left and right ureters, the distal left and right ureters, and the bladder included in the 3D urinary tract data in a 3D space, a step of finding an overlapping area by overlapping a candidate similar to the stone included in the stone candidate data in the 3D space where the 3D urinary tract data is mapped, and a step of diagnosing the location and size of the stone determined to be the urinary stone from the overlapped area.

[0038] Specific details of other embodiments are included in the detailed description and drawings.

[0039] According to the technical problem solving means described above, the urinary stone diagnosis system method according to the embodiment of the present invention can diagnose urinary stones using at least one module based on image data, and thus has the following advantages.

[0040] First, the urinary stone diagnosis system and method can improve the efficiency and accuracy of the shock wave procedure benefit review.

[0041] Second, the urinary stone diagnosis system and method can improve the diagnostic accuracy of urinary stones by excluding plaque and bone in blood vessels that can be confused with urinary stones in CT images.

[0042] Third, the urinary stone diagnosis system and method can measure the location and size of urinary stones in the kidney, ureter, and bladder.

[0043] Figure 1 illustrates a urinary stone diagnosis system according to an embodiment of the present invention.

[0044] Figure 2 illustrates a process for diagnosing urinary stones using multiple modules for diagnosing urinary stones in a patient to be diagnosed.

[0045] Figure 3 shows the structure of the algorithm of the urinary tract estimation module and the urinary tract precision prediction module.

[0046] Figure 4 is a diagram illustrating setting an area of ​​interest in image data.

[0047] Figure 5 illustrates the process of the urinary tract estimation module.

[0048] Figure 6 shows a drawing in which the remaining area except the area of ​​interest is removed from a two-dimensional space.

[0049] Figure 7 shows a drawing in which the remaining area except the area of ​​interest in a three-dimensional space is removed.

[0050] Figure 8 illustrates the process of the urinary tract precision prediction module.

[0051] Figure 9 is a drawing showing the collision between detailed classes of the ureter.

[0052] Figure 10 illustrates the process of the ureter classification module.

[0053] Figure 11 is a diagram illustrating a detailed classification of the ureter using the ureter classification module.

[0054] Figure 12 is a drawing showing a 3D map created based on 3D urinary tract data.

[0055] Figure 13 shows candidates similar to absences using the absence candidate search module.

[0056] Figure 14 is a diagram illustrating a diagnosis of urinary stones in a patient by overlapping three-dimensional urinary tract data and stone candidate data.

[0057] Figure 15 shows a flow chart of a method for diagnosing urinary stones.

[0058] Figure 16 shows the urinary stones of the patient diagnosed as a result of the urinary stone diagnosis.

[0059] The following detailed description of the present invention refers to the accompanying drawings, which illustrate specific embodiments in which the present invention may be practiced, in order to clarify the objects, techniques, solutions, and advantages of the present invention. These embodiments are described in sufficient detail to enable those skilled in the art to practice the present invention. Furthermore, throughout the detailed description and claims, the word "comprise" and variations thereof are not intended to exclude other technical features, additions, components, or steps. Other objects, advantages, and features of the present invention will become apparent to those skilled in the art, in part from this description, and in part from practice of the present invention. The examples and drawings below are provided by way of illustration and are not intended to limit the present invention. Moreover, the present invention encompasses all possible combinations of the embodiments set forth herein. It should be understood that the various embodiments of the present invention, while different from one another, are not necessarily mutually exclusive. It should also be understood that the positions or arrangements of individual components within each disclosed embodiment may be varied without departing from the spirit and scope of the present invention. Accordingly, the detailed description set forth below is not intended to be limiting, and the scope of the present invention is defined solely by the appended claims, along with the full scope equivalent to which such claims are entitled, if properly described. Furthermore, unless otherwise indicated herein or clearly contradicted by context, items referred to in the singular encompass the plural unless the context otherwise requires. Furthermore, in describing the present invention, if a detailed description of a related known structure or function is determined to obscure the gist of the present invention, the detailed description will be omitted.

[0060] FIG. 1 illustrates a urinary stone diagnosis system according to an embodiment of the present invention, and FIG. 2 illustrates a process of diagnosing urinary stones using multiple modules for diagnosing urinary stones in a patient to be diagnosed.

[0061] Before explaining Fig. 1 and Fig. 2, in order to diagnose urinary stones (10), it is generally possible to confirm the presence of stones through X-ray, but it may be difficult to distinguish urinary stones (10) because they are hidden by the pelvic bone or other organs.

[0062] Therefore, the urinary stone diagnosis system (100) can use image data taken of the kidney, ureter, and bladder of the diagnosis subject using CT (Computed Tomography) to confirm stones using accurate anatomical structures, but is not limited thereto, and according to an embodiment of the present invention, the urinary stone diagnosis system (100) can also use image data taken by at least one imaging device among MRI (Magnetic Resonance Imaging), X-ray, and ultrasound in addition to CT.

[0063] Referring to FIG. 1, the urinary stone diagnosis system (100) includes a data receiving unit (110), a 3D urinary tract data generation unit (120), a stone candidate data generation unit (130), a stone diagnosis unit (140), a data output unit (150), and a control unit (160).

[0064] The data receiving unit (110) can receive image data obtained by taking a cross-sectional photograph of the urinary tract.

[0065] The 3D urinary tract data generation unit (120) includes a urinary tract estimation module (121) that estimates the shape and location of the urinary tract, a urinary tract precision prediction module (122), and a ureter classification module (123), and can generate 3D urinary tract data (126) by fusing the result data of the urinary tract estimation module (121), the urinary tract precision prediction module (122), and the ureter classification module (123) based on image data.

[0066] Here, the urinary tract includes, but is not limited to, the kidneys, ureters, and bladder, and may also include various organs in the body involved in the process of producing and excreting urine, such as the urethra and prostate.

[0067] The absence candidate data generation unit (130) can search for candidates similar to absences using the absence candidate search module (131) based on image data and generate absence candidate data (133).

[0068] The stone diagnosis unit (140) maps the left and right kidneys, the proximal left and right ureters, the middle left and right ureters, the distal left and right ureters, and the bladder included in the three-dimensional urinary tract data (126) to a three-dimensional space in order to diagnose stones, and the stone diagnosis unit (140) overlaps a candidate similar to the stone included in the stone candidate data (133) on the three-dimensional space mapped to the three-dimensional urinary tract data (126) to diagnose the location and size of the stone determined to be a urinary stone (10) from the overlapped area.

[0069] The data output unit (150) can provide the user with the location and size of the stone diagnosed by the stone diagnosis unit (140).

[0070] According to an embodiment of the present invention, the data output unit (150) can display location information for at least one urinary stone (10) diagnosed by the stone diagnosis unit (140) and size information when the size of the urinary stone (10) is 4 mm or more based on 4 mm on a CT image and provide the information to the user. However, the present invention is not limited thereto, and when the size of the urinary stone (10) desired by the user is set and the size is greater than the set value, the information can be displayed on the screen and provided to the user as a notification.

[0071] The control unit (160) generally corresponds to a server, and the control unit (160) performs overall control of the data receiving unit (110), the 3D urinary data generating unit (120), the stone candidate data generating unit (130), the stone diagnosis unit (140), and the data output unit (150).

[0072] Referring to FIG. 2, the urinary stone diagnosis system (100) can diagnose urinary stones (10) from image data of a patient to be diagnosed using a contrast agent judgment module (161), a urinary tract estimation module (121), a urinary tract precision prediction module (122), a ureter classification module (123), and a stone candidate search module (131).

[0073] The contrast agent judgment module (161) can use image data received from the data receiving unit (110) to determine whether to administer contrast agent to a patient to be diagnosed.

[0074] According to an embodiment of the present invention, the contrast agent determination module (161) can also determine whether contrast agent extravasation may occur during contrast agent administration.

[0075] The 3D urinary tract data generation unit (120) may include a urinary tract estimation module (121), a urinary tract precision prediction module (122), and a ureter classification module (123) for generating 3D urinary tract data (126) based on received image data.

[0076] The urinary tract estimation module (121) can estimate the approximate location of the urinary tract by extracting a region of interest from image data.

[0077] The urinary tract precision prediction module (122) can perform segmentation by analyzing image data at a higher resolution than the urinary tract estimation module (121) to distinguish the anatomical locations of the kidney, ureter, and bladder corresponding to the coordinates of the region of interest.

[0078] Here, the urinary tract estimation module (121) and the urinary tract precision prediction module (122) can estimate the urinary tract of the patient to be diagnosed corresponding to the area of ​​interest using the first algorithm.

[0079] According to an embodiment of the present invention, the first algorithm uses URO-UNETR, which is slightly modified from UNETR and SWIN-UNETR, and a detailed description of URO-UNETR is described in FIG. 3.

[0080] The ureter classification module (123) can classify the ureter into the proximal ureter, the middle ureter, and the distal ureter using a second algorithm to predict the precise anatomical location of the ureter.

[0081] According to an embodiment of the present invention, the second algorithm can classify the ureter of a patient to be diagnosed using an ensemble technique including bagging, boosting, and stacking.

[0082] The three-dimensional urinary tract data (126) generated by fusing the result data of the urinary tract estimation module (121), the urinary tract precision prediction module (122), and the ureter classification module (123) may include the location and shape of the left and right kidneys, the proximal left and right ureters, the middle left and right ureters, the distal left and right ureters, and the bladder.

[0083] The stone candidate search module (131) can search for candidates similar to stones by deleting voxel blocks larger than 50 mm using 3D CCL (Connected Components Labeling) based on image data.

[0084] The absence candidate search module (131) learns candidates similar to absences using a third algorithm, and the absence candidate search module (131) can delete candidates that are not determined to be absences through learning, thereby generating absence candidate data (133).

[0085] Figure 3 shows the structure of the algorithm of the urinary tract estimation module and the urinary tract precision prediction module.

[0086] Referring to FIG. 3, the urinary tract estimation module (121) and the urinary tract precision prediction module (122) can be configured using URO-UNETR.

[0087] URO-UNETR is a slightly modified version of UNETR and SWIN-UNETR. It replaces the MLP of the backbone, Ttransformer, by using a channel-wise MLP to strongly connect voxel-level contexts, and some convolution layers can be replaced with the convolution layers of ResNet to prevent loss of operation information between blocks.

[0088] Figure 4 is a diagram showing a process of setting a region of interest in image data, and Figure 5 shows a process of a urinary tract estimation module.

[0089] Referring to FIGS. 4 and 5, the urinary tract estimation module (121) can estimate the approximate location of the urinary tract by extracting a region of interest from image data.

[0090] As can be seen in Fig. 4, the area of ​​interest of the urinary tract estimation module (121) may include a three-dimensional boundary created in the range from the left and right kidney ends of the diagnosis subject to the bladder end based on the image data of the diagnosis subject received from the data receiving unit (110).

[0091] The main purpose of the urinary tract estimation module (121) is to determine the end of the kidney and bladder, so it uses low resolution, and the urinary tract estimation module (121) can ignore information about the ureter.

[0092] According to an embodiment of the present invention, the urinary tract estimation module (121) extends the existing 2D algorithm to 3D so that it can consider the relationship between each slice through URO-UNETR.

[0093] As can be seen in Fig. 5, the urinary tract estimation module (121) can reconstruct the received image data to have a uniform voxel size of 4 mm to determine the end of the kidney and bladder.

[0094] Here, considering that the image data has an average space of 0.7 mm, all images can be organized as 3D tensors with the size of (128, 128, 128) with padding.

[0095] According to an embodiment of the present invention, the urinary tract estimation module (121) converts the start and end of the urinary tract into coordinates based on the result value to predict the end coordinates of the kidney and bladder. is set as the region of interest, and the urinary tract estimation module (121) provides information on the region of interest It can be transmitted to the urinary tract precision prediction module (122) so that it can be used as input to the urinary tract precision prediction module (122).

[0096] Figure 6 shows a drawing in which the remaining area except the area of ​​interest is removed in a two-dimensional space, Figure 7 shows a drawing in which the remaining area except the area of ​​interest is removed in a three-dimensional space, and Figure 8 shows the process of the urinary tract precision prediction module.

[0097] Referring to FIGS. 6 to 8, the urinary tract precision prediction module (122) can predict the urinary tract, including the kidneys, ureters, and bladder of a patient to be diagnosed.

[0098] As can be seen in Figure 8, the urinary tract precision prediction module (122) uses the information on the region of interest estimated by the urinary tract estimation module (121). Using this, the voxel size for the image constructed from the cropped area from the original image data can be reconstructed to (0.7 mm, 0.7 mm, 1 mm).

[0099] When the urinary tract precision prediction module (122) is learning, a random crop sampler is applied to the extracted region of interest, and when inference is performed, all regions are cropped to the size of (128, 128, 128) and a slide window ensemble is applied.

[0100] According to an embodiment of the present invention, Slide Window Ensemble is an algorithm that sets a Window of a certain size and solves a problem using the values ​​of elements inside the Window. The Slide Window Ensemble can also be a type of operation that uses Slide Window for convolution operations used in deep learning.

[0101] Figure 9 is a diagram showing a conflict between detailed classes of the ureter, Figure 10 is a diagram showing a process of the ureter classification module, and Figure 11 is a diagram showing a detailed classification of the ureter using the ureter classification module.

[0102] Referring to FIGS. 9 to 11, the ureter classification module (123) can classify the ureter in detail using the second algorithm.

[0103] As can be seen in Fig. 9, the ureter classification module (123) can classify the ureter of a patient to be diagnosed into the proximal ureter, the middle ureter, and the distal ureter using the second algorithm to prevent collisions between classes in the Logit Space.

[0104] As can be seen in Fig. 10, the ureter classification module (123) sets the area from the kidney to the SI Joint as the proximal part of the ureter, the area overlapping the SI Joint as the middle part of the ureter, and the area after the SI Joint as the distal part of the ureter based on the SI Joint.

[0105] According to an embodiment of the present invention, as can be seen in FIG. 11, the ureter classification module (123) is configured with a Swin-Transformer, and, unlike the ureter estimation module (121) and the urinary tract precision prediction module (122), it can use a 2D (Axial Slice) image aligned from Superior to Inferior.

[0106] Figure 12 is a drawing showing a 3D map created based on 3D urinary tract data.

[0107] Referring to FIG. 12, a three-dimensional map of the urinary tract of a patient to be diagnosed can be created based on three-dimensional urinary tract data (126).

[0108] As can be seen in Figure 12, by merging the result data of the ureter estimation module (121), the urinary tract precision prediction module (122), and the ureter classification module (123), a three-dimensional map of the left and right kidneys, the proximal left and right ureters, the middle left and right ureters, the distal left and right ureters, and the bladder can be created.

[0109] Figure 13 shows candidates similar to absences using the absence candidate search module.

[0110] Referring to FIG. 13, the stone candidate data generation unit (130) can search for stone candidates of a patient to be diagnosed from image data using the third algorithm of the stone candidate search module (131).

[0111] The absent candidate search module (131) performs binarization of image data using HU (Hounsfield Unit) Thresholding with the lower bound set to 120, and as a result, can have a voxel mass of thousands to tens of thousands.

[0112] Using 3D CCL (Connected Components Labeling) to exclude voxel lumps of tens of thousands to tens of thousands, voxel lumps larger than 50 mm and the spine and pelvis from the results, the stone candidate search module (131) can search for candidates similar to stones.

[0113] Next, the absence candidate search module (131) can learn candidates similar to absences using a third algorithm as an additional device to maximize diagnostic performance, and delete candidates similar to absences that are not determined to be absences through learning, thereby generating absence candidate data (133).

[0114] According to an embodiment of the present invention, the third algorithm can construct a simple classifier by using the backbone network and weights used in the first algorithm, using global average pooling and fully connected layers. Furthermore, to reduce the computational cost of the learning process, the encoder weights can be frozen and fine-tuned.

[0115] Next, during learning, candidates and answers similar to the absence based on HU are randomly sampled, and candidates and answers similar to the absence with a CCL (Connected Components Labeling) number can be additionally extracted in sizes (0 to 4 mm) in each direction based on the Bounding Cuboid.

[0116] Next, the extracted area can be resized and padded into a (96, 96, 96) Tensor, and one Series can be randomly sampled a total of 20 times, and the bounding cuboid of the answer sheet can be set to always be included regardless of the number.

[0117] Next, the classification result during inference is combined with the CCL number, and the candidate group that is not judged to be absent can be deleted to generate absent candidate data (133).

[0118] Figure 14 is a diagram illustrating a diagnosis of urinary stones in a patient by overlapping three-dimensional urinary tract data and stone candidate data.

[0119] Referring to FIG. 14, the stone diagnosis unit (140) can diagnose the location and size of a urinary stone (10) from the overlapped area by overlapping the 3D urinary tract data (126) and stone candidate data (133) in a 3D space.

[0120] The stone diagnosis unit (140) can map the prediction results of the kidney, proximal ureter, middle ureter, distal ureter, and bladder included in the three-dimensional urinary tract data (126) into a three-dimensional space.

[0121] At this time, the stone diagnosis unit (140) can restore the coordinate system transformed in the process of the urinary tract estimation module (121) and the urinary tract precision prediction module (122) to the coordinate system of the original image data.

[0122] Next, the stone diagnosis unit (140) finds an overlapping area by overlapping the stone candidate data (133) in a three-dimensional space mapped with the three-dimensional urinary tract data (126), and the overlapped area can be distinguished by the CCL number.

[0123] Through this, the stone diagnosis unit (140) can know the anatomical location of the urinary tract of the patient to be diagnosed corresponding to the coordinates in the image data and the three-dimensional urinary tract data (126) for each number, and can diagnose urinary stones (10) in the urinary tract of the patient to be diagnosed by combining the anatomical location of the urinary tract and the stone candidate data (133) through an XOR operation.

[0124] Figure 15 shows a flow chart of a method for diagnosing urinary stones, and Figure 16 shows urinary stones of a patient diagnosed as a result of urinary stones diagnosis.

[0125] Referring to FIGS. 15 and 16, a urinary stone diagnosis system (100) can diagnose urinary stones (10) of a patient to be diagnosed using image data.

[0126] A step is performed in which a data receiving unit (110) receives image data obtained by taking a cross-sectional image of the urinary tract, including the kidneys, ureters, and bladder of a patient to be diagnosed, from an imaging device.

[0127] Next, a step is performed to generate 3D urinary tract data (126) by fusing the result data for estimating the shape and location of the urinary tract generated by the 3D urinary tract data generation unit (120) using the urinary tract estimation module (121), the urinary tract precision prediction module (122), and the ureter classification module (123) based on the image data (S100).

[0128] Here, the process of generating 3D urinary tract data (126) using the urinary tract estimation module (121), the urinary tract precision prediction module (122), and the ureter classification module (123) by the 3D urinary tract data generation unit (120) is as follows.

[0129] Using the first algorithm (124), the urinary tract estimation module (121) extracts a region of interest at a low resolution based on image data and proceeds with a step of estimating the approximate location of the urinary tract.

[0130] According to an embodiment of the present invention, the region of interest may include a three-dimensional boundary created from the left and right kidney ends to the bladder ends of a patient to be diagnosed based on image data.

[0131] Using the first algorithm (124), the urinary tract precision prediction module (122) analyzes image data at a higher resolution than the urinary tract estimation module (121) and performs a segmentation step to distinguish the anatomical locations of the kidney, ureter, and bladder corresponding to the coordinates of the region of interest.

[0132] Using the second algorithm (125), the ureter classification module (123) proceeds with the step of dividing the ureter into the proximal ureter, the middle ureter, and the distal ureter to predict the precise anatomical location of the ureter.

[0133] The three-dimensional urinary tract data generation unit (120) generates three-dimensional urinary tract data (126) by fusing the result data generated from the urinary tract estimation module (121), the urinary tract precision prediction module (122), and the ureter classification module (123), and the three-dimensional urinary tract data generation unit (120) completes the process of generating three-dimensional urinary tract data (126) using the urinary tract estimation module (121), the urinary tract precision prediction module (122), and the ureter classification module (123).

[0134] Following the step of generating 3D urinary tract data (126), a stone candidate data generation unit (130) searches for candidates similar to stones using a stone candidate search module (131) based on image data and generates stone candidate data (133) (S120).

[0135] Here, the process in which the absence candidate data generation unit (130) generates absence candidate data (133) using the absence candidate search module (131) is as follows.

[0136] Based on the image data, the absence candidate search module (130) deletes voxel blocks larger than 50 mm using 3D CCL (Connected Components Labeling) and proceeds with the step of searching for candidates similar to the absence.

[0137] A step is performed in which the absence candidate search module (130) learns candidates similar to absences generated based on image data using a third algorithm, and deletes candidates similar to absences that are not determined to be absences through learning to generate absence candidate data (133), and the absence candidate data generation unit (130) completes the process of generating absence candidate data (133) using the absence candidate search module (131).

[0138] Following the step of generating stone candidate data (133), the stone diagnosis unit (140) overlaps the three-dimensional urinary tract data (126) and stone candidate data (133) to diagnose stones (S120). Thereafter, a step of diagnosing the location and size of a stone determined to be a urinary stone (10) from the overlapped area is performed (S130).

[0139] The process of diagnosing stones in more detail is as follows:

[0140] The stone diagnosis unit (140) performs a step of mapping the left and right kidneys, the proximal left and right ureters, the middle left and right ureters, the distal left and right ureters, and the bladder included in the three-dimensional urinary tract data (126) into a three-dimensional space.

[0141] The stone diagnosis unit (140) performs a step of finding an overlapping area by superimposing candidates similar to stones included in the stone candidate data (133) on the three-dimensional space where the three-dimensional urinary tract data (126) is mapped.

[0142] The step of diagnosing the location and size of the stone determined to be a urinary stone (10) from the overlapped area of ​​the stone diagnosis unit (140) is performed, thereby completing the urinary stone diagnosis method of the urinary stone diagnosis system (100).

[0143] While the present invention has been described with reference to the embodiments illustrated in the drawings, these are merely exemplary, and those skilled in the art will understand that various modifications and equivalent other embodiments are possible. For example, although the present invention has been described with the assumption of a system for diagnosing urinary stones, the present invention can be equally applied to all cases where disease can be diagnosed by analyzing image data without any particular modification. Therefore, the true technical protection scope of the present invention should be defined solely by the appended claims.

[0144] [Explanation of symbols]

[0145] 100: Urinary Stone Diagnosis System

[0146] 110: Data receiving unit

[0147] 120: 3D urinary tract data generation unit

[0148] 130: Absentee candidate data generation section

[0149] 140: Absence Diagnosis Department

[0150] 150: Data output section

[0151] 160: Control unit

Claims

1. A data receiving unit for receiving image data obtained by scanning the urinary tract; A 3D urinary system data generation unit including a urinary system estimation module for estimating the shape and location of the urinary system, a urinary system precision prediction module, and a ureter classification module, and generating 3D urinary system data by fusing result data of the urinary system estimation module, urinary system precision prediction module, and ureter classification module generated based on the image data; An absence candidate data generation unit that searches for candidates similar to absences and generates absence candidate data using an absence candidate search module based on the above image data; and A urinary stone diagnosis system, comprising: a stone diagnosis unit that overlaps the three-dimensional urinary tract data and the stone candidate data to diagnose stones, and diagnoses a location determined to be a urinary stone and the size of the stone from the overlapped area.

2. In paragraph 1, A urinary stone diagnosis system including an image data of a kidney, ureter, and bladder of a patient to be diagnosed using at least one imaging device selected from the group consisting of CT (Computed Tomography), MRI (Magnetic Resonance Imaging), X-ray, and ultrasound.

3. In paragraph 1, The above 3D urinary tract data generation unit includes the urinary tract estimation module, the urinary tract precision prediction module, and the ureter classification module for generating the 3D urinary tract data. The above urinary tract estimation module extracts a region of interest from the image data and estimates the approximate location of the urinary tract. The above urinary tract precision prediction module analyzes the image data at a higher resolution than the above urinary tract estimation module and performs segmentation to distinguish the anatomical locations of the kidney, ureter, and bladder corresponding to the coordinates of the region of interest. The above ureter classification module is a urinary stone diagnosis system that classifies the ureter into the proximal ureter, middle ureter, and distal ureter to predict the precise anatomical location of the ureter.

4. In paragraph 3, The above region of interest is a urinary stone diagnosis system including a three-dimensional boundary created from the left and right kidney ends to the bladder ends of the patient to be diagnosed based on the image data.

5. In paragraph 3, The above urinary tract estimation module and the urinary tract precision prediction module estimate the urinary tract of the patient to be diagnosed corresponding to the region of interest using the first algorithm, The above ureter classification module is a urinary stone diagnosis system that classifies the ureter of the patient to be diagnosed into the proximal ureter, the middle ureter, and the distal ureter using a second algorithm.

6. In paragraph 1, The above three-dimensional urinary tract data is a urinary stone diagnosis system including the location and shape of the left and right kidneys, the left and right proximal ureters, the left and right middle ureters, the left and right distal ureters, and the bladder.

7. In paragraph 1, The above-mentioned stone candidate search module is a urinary stone diagnosis system that deletes voxel lumps of 50 mm or more using 3D CCL (Connected Components Labeling) based on the image data and searches for candidates similar to the stone.

8. In paragraph 1, The above-mentioned stone candidate search module is a urinary stone diagnosis system that learns candidates similar to the above-mentioned stone using a third algorithm, and generates the above-mentioned stone candidate data by deleting candidates similar to the above-mentioned stone that are not determined to be stones through learning.

9. In paragraph 1, The above absence diagnosis section is, The left and right kidneys, the proximal left and right ureters, the middle left and right ureters, the distal left and right ureters, and the bladder included in the above three-dimensional urinary tract data are mapped into a three-dimensional space. A urinary stone diagnosis system that diagnoses a location and size of a urinary stone determined to be a urinary stone from an overlapped area by overlapping a candidate similar to the urinary stone included in the above-mentioned urinary stone candidate data on a three-dimensional space mapped to the three-dimensional urinary system data.

10. In paragraph 1, A urinary stone diagnosis system further comprising a contrast agent judgment module capable of judging whether to administer contrast agent to a diagnosis subject using the image data received from the data receiving unit.

11. A step of receiving image data obtained by scanning the urinary tract, including the kidneys, ureters, and bladder, of a patient to be diagnosed from an imaging device; A step of generating three-dimensional urinary system data by fusing result data for estimating the shape and location of the urinary system generated using the urinary system estimation module, the urinary system precision prediction module, and the ureter classification module based on the image data; A step of searching for candidates similar to absences and generating absence candidate data using an absence candidate search module based on image data; and A method for diagnosing urinary stones, comprising the steps of: overlapping the three-dimensional urinary tract data and the stone candidate data to diagnose stones, and diagnosing the location determined to be a urinary stone and the size of the stone from the overlapped area.

12. In paragraph 11, The process of generating the three-dimensional urinary tract data using the above urinary tract estimation module, urinary tract precision module, and ureter classification module is as follows. A step of extracting a region of interest at a low resolution based on the image data using the first algorithm of the urinary tract estimation module to estimate the approximate location of the urinary tract; A step of analyzing the image data at a higher resolution than the urinary tract estimation module using the first algorithm of the urinary tract precision prediction module, and performing segmentation to distinguish the anatomical locations of the kidney, ureter, and bladder corresponding to the coordinates of the region of interest; A step of dividing the ureter into the proximal ureter, the middle ureter, and the distal ureter to predict the precise anatomical location of the ureter using the second algorithm of the above ureter classification module; and A method for diagnosing urinary stones, comprising: a step of generating three-dimensional urinary data by fusing result data generated from the urinary tract estimation module, the urinary tract precision prediction module, and the ureter classification module.

13. In paragraph 12, The above region of interest is a method for diagnosing urinary stones, wherein the above region of interest includes a three-dimensional boundary created from the left and right kidney ends to the bladder ends of the patient to be diagnosed based on the image data.

14. In paragraph 11, The above three-dimensional urinary tract data is a method for diagnosing urinary stones including the left and right kidneys, bladder, proximal left and right ureters, middle left and right ureters, and distal left and right ureters.

15. In Article 11, The process of generating absence candidate data using the above absence candidate search module is as follows: A step of deleting a voxel mass of 50 mm or more using 3D CCL (Connected Components Labeling) based on the above image data and searching for a candidate similar to the stone; and A method for diagnosing urinary stones, comprising: a step of learning candidates similar to the stones generated based on the image data using a third algorithm, and deleting candidates similar to the stones that are not determined to be stones through learning to generate the stone candidate data; 16. In paragraph 11, The process of diagnosing the above absence is: A step of mapping the left and right kidneys, the proximal left and right ureters, the middle left and right ureters, the distal left and right ureters, and the bladder included in the three-dimensional urinary tract data in a three-dimensional space; A step of finding an overlapping area by superimposing a candidate similar to the stone included in the stone candidate data on a three-dimensional space where the three-dimensional urinary tract data is mapped; and A method for diagnosing urinary stones, comprising: a step of diagnosing a location and size of a urinary stone determined to be a urinary stone from the overlapped area;

Images