Method and apparatus for detecting and identifying calculus

Abstract

A method and an apparatus for detecting and identifying calculus are disclosed. According to one aspect of the present disclosure, a computer-implemented method for detecting and identifying calculus from an image is provided, the computer-implemented method comprising the steps of: detecting a calculus area indicating the position and the size of the calculus from the image; generating calculus features on the basis of features in an area corresponding to the calculus area in a feature map extracted from the image; generating peripheral portion features of the calculus on the basis of features in one or more divided areas generated by dividing, into a predetermined size, the feature map extracted from the image; and identifying the type of calculus on the basis of features obtained by combining the calculus features and the peripheral portion features of the calculus.

Description

Method and apparatus for detecting and identifying stones

[0001] The present disclosure relates to a method and apparatus for detecting and identifying calculi.

[0002] The following description merely provides background information related to the present embodiment and does not constitute prior art.

[0003] Stone problems manifest in various forms, including kidney stones, ureteral stones, and bladder stones, making accurate detection and treatment crucial. Since stones occur in various locations and sizes, doctors and veterinarians face the inconvenience of having to manually analyze images or perform additional tests to accurately identify the type of stone. This process is not only time-consuming but also presents challenges in quickly and accurately diagnosing the specific type of stone.

[0004] Object detection technology utilizing artificial intelligence (AI) is advancing in the field of medical image analysis. Existing AI-based object detection technology has the advantage of identifying the type of stone more quickly than doctors or veterinarians manually checking each one. However, since existing technology is based on the physical appearance of objects, it has a limitation in that it is difficult to accurately distinguish the types of stones that are similar in size and shape.

[0005] Therefore, a method and device are needed to accurately identify the type of stone by considering not only its appearance but also its location.

[0006] The present disclosure has the primary purpose of providing a method and apparatus for detecting and identifying stones. Specifically, the primary purpose is to provide a method and apparatus for more accurately detecting and identifying stones by detecting a stone region from an image, generating features of the stone based on the stone region, generating features of the periphery of the stone, and identifying the type of stone based on features combining the features of the stone and the features of the periphery of the stone.

[0007] The problems that the present invention aims to solve are not limited to those mentioned above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below.

[0008] According to one aspect of the present disclosure, a computer-implemented method for detecting and identifying a stone from an image comprises: a step of detecting a stone region in the image that indicates the location and size of the stone; a step of generating features of the stone based on features within a region corresponding to the stone region among a feature map extracted from the image; a step of generating features of the periphery of the stone based on features within one or more segmented regions generated by dividing the feature map extracted from the image into predetermined sizes; and a step of identifying the type of the stone based on a combined feature of the features of the stone and the features of the periphery of the stone.

[0009] According to another aspect of the present disclosure, an apparatus for detecting and identifying a stone from an image comprises: a structure detection module; and a stone identification module, wherein the structure detection module detects a stone region indicating the location and size of the stone from the image, and the stone identification module generates features of the stone based on features within a region corresponding to the stone region among feature maps extracted from the image, generates features of the periphery of the stone based on features within one or more segmented regions generated by dividing the feature map extracted from the image into predetermined sizes, and identifies the type of the stone based on a combined feature of the features of the stone and the features of the periphery of the stone.

[0010] According to one embodiment of the present disclosure, the type of stone can be accurately identified by identifying the type of stone based on a combined feature of the stone's characteristics and the characteristics of the stone's surrounding area, so as to reflect not only the characteristics of the stone but also the characteristics of the stone's surrounding area.

[0011] According to one embodiment of the present disclosure, the type of stone can be accurately identified by using a neural network based on weights updated in a direction that minimizes loss based on an operation between a feature vector formed by merging a feature vector for the stone and a feature vector for the periphery of the stone, and weights.

[0012] According to one embodiment of the present disclosure, the accuracy of detection and identification can be improved by excluding stones determined to be incorrectly detected from the identification results by comparing the location of the bone with the location of the stone.

[0013] The effects of the present disclosure are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below.

[0014] FIG. 1 is a schematic diagram showing the configuration of a device for detecting and identifying stones according to one embodiment of the present disclosure and the surrounding configuration thereof.

[0015] FIG. 2 is a diagram exemplarily illustrating the results of structure detection according to one embodiment of the present disclosure.

[0016] FIG. 3 is a diagram exemplarily illustrating the results of stone identification according to one embodiment of the present disclosure.

[0017] FIG. 4 is a drawing for explaining a post-processing method according to one embodiment of the present disclosure.

[0018] FIG. 5 is a diagram illustrating an exemplary post-processing result according to one embodiment of the present disclosure.

[0019] FIG. 6 is a diagram illustrating an exemplary visualization result according to one embodiment of the present disclosure.

[0020] FIG. 7 is a schematic diagram showing the structure of a neural network used to identify the type of stone in a stone identification module according to one embodiment of the present disclosure.

[0021] FIG. 8a is a drawing for illustrating a method for generating features for the periphery of a calculus according to one embodiment of the present disclosure.

[0022] FIG. 8b is a drawing for explaining a method for generating features of a calculus according to one embodiment of the present disclosure.

[0023] FIG. 9 is a flowchart schematically illustrating a method for detecting and identifying stones according to one embodiment of the present disclosure.

[0024] FIG. 10 is a schematic diagram showing the configuration of an exemplary computing device that can be used to implement the devices and methods described in the present disclosure.

[0025] Some embodiments of the present disclosure are described in detail below with reference to exemplary drawings. It should be noted that in assigning reference numerals to the components of each drawing, the same components are given the same reference numeral whenever possible, even if they are shown in different drawings. Furthermore, in describing the present disclosure, if it is determined that a detailed description of related known components or functions could obscure the essence of the present disclosure, such detailed description is omitted.

[0026] In describing the components of the embodiments according to the present disclosure, symbols such as first, second, i), ii), a), b), etc., may be used. These symbols are intended only to distinguish the components from other components, and the essence, order, or sequence of the components is not limited by the symbols. When a part in the specification is described as 'comprising' or 'having' a component, this means that, unless explicitly stated otherwise, it does not exclude other components but may include additional components.

[0027] The detailed description set forth below, together with the accompanying drawings, is intended to describe exemplary embodiments of the present disclosure and is not intended to represent the only embodiment in which the present disclosure can be practiced.

[0028] Although this disclosure describes examples of stones occurring inside the body of animals, this does not imply that the invention is limited to that field. This disclosure may also be applied to stones occurring inside the human body. In other words, the invention may be effectively applied to the detection and identification of stones occurring in the human kidneys, bladder, urethra, etc.

[0029] FIG. 1 is a schematic diagram showing the configuration of a device for detecting and identifying stones according to one embodiment of the present disclosure and the surrounding configuration thereof.

[0030] Referring to FIG. 1, a stone detection and identification device (11) according to one embodiment of the present disclosure acquires an image generated using an image capturing device (10).

[0031] The imaging device (10) may be an X-ray imaging device. The X-ray imaging device may obtain an image by photographing the inside of the animal's body. The image may include structures located inside the animal's body. There may be one or more structures. The structures may be anatomical structures and / or pathological structures. Anatomical structures may include bones. Bones may include ribs, pelvic bones, cervical vertebrae, thoracic vertebrae, and lumbar vertebrae. Pathological structures may include stones. There may be one or more stones. Stones may include urethral stones, bladder stones, and kidney stones.

[0032] The stone detection and identification device (11) may include a structure detection module (110), a stone identification module (120), and a post-processing module (130). Optionally, the stone detection and identification device (11) may further include a visualization module (140). The stone detection and identification device (11) may be implemented using one or more computing devices (100).

[0033] The structure detection module (110) can acquire at least one image from the image capturing device (10) and detect a structure from the acquired image. The process of detecting a structure may be a process based on an object detection technique. The object detection technique is typically performed in two stages: object localization and object classification. The process of detecting a structure may include a process of setting a bounding box for the structure. The process of setting a bounding box for the structure may be a process based on an object localization technique. The process of detecting a structure may include a process of determining which class the structure included in the bounding box belongs to based on the set bounding box. The process of determining which class the structure belongs to may be a process based on a classification technique. The classes may include ribs, pelvic bones, cervical vertebrae, thoracic vertebrae, lumbar vertebrae, and calculi. The process of detecting structures can be performed using one of the following algorithms: YOLO (You Only Look Once), SSD (Single Shot MultiBox Detector), R-CNN (Regions with Convolutional Neural Network), Fast R-CNN, or Faster R-CNN. As a result of the structure detection, a calculus region indicating the location and size of the calculus and a bone region indicating the location and size of the bone can be detected.

[0034] FIG. 2 is a diagram exemplarily illustrating the results of structure detection according to one embodiment of the present disclosure.

[0035] Referring to FIG. 2, a first bone region (210), a second bone region (220), a first stone region (230), a second stone region (240), a third stone region (250), a fourth stone region (260), and a fifth stone region (270) are illustrated. The first bone region (210) can be detected as a rib by the structure detection module (110). The second bone region (220) can be detected as a pelvic bone by the structure detection module (110). The first stone region (230), the second stone region (240), the third stone region (250), the fourth stone region (260), and the fifth stone region (270) can be detected as stones by the structure detection module (110).

[0036] The stone identification module (120) obtains a detection result from the structure detection module (110) and can identify the type of stone based on the obtained detection result. The detection result may include information regarding the coordinates of the area containing the structure and the class to which the structure included in the area belongs. For example, the detection result for the first stone area (230) may include the coordinates of the vertex of the first stone area (230) and information that the first stone area (230) is classified as a stone. The process of the stone identification module (120) identifying the type of stone may be implemented using one or more neural networks. Identifying the type of stone may mean determining what type of stone it is, that is, whether it is a urinary stone, a bladder stone, a kidney stone, etc. The process of determining what type of stone it is may include a process of determining which class the stone belongs to. The 'class' determined by the stone identification module (120) has the same basic meaning as the 'class' determined by the structure detection module (110), but its content may differ. The class determined by the structure detection module (110) may be a class for structures such as ribs, pelvic bones, and stones. Meanwhile, the class determined by the stone identification module (120) may be a class for specific types of stones such as urethral stones, bladder stones, and kidney stones.

[0037] The necrosis identification module (120) can determine the specific type of each necrosis by calculating weights and features that combine the characteristics of the necrosis and the features of the necrosis's periphery. The neural network implementing the necrosis identification module (120) may be trained, and the process by which the necrosis identification module (120) identifies the type of necrosis may be an inference process based on the trained neural network. The weights may be updated according to the training process so that the loss produced as a result of the neural network's operation is minimized. Since the training process and the inference process essentially follow the same procedure, the process of generating features of the necrosis, the process of generating features of the necrosis's periphery, and the process of combining the features of the necrosis and the periphery features will be explained in detail below with reference to FIGS. 7, FIGS. 8a, and FIGS. 8b.

[0038] FIG. 3 is a diagram exemplarily illustrating the results of stone identification according to one embodiment of the present disclosure.

[0039] Referring to FIG. 3, a first stone area (230), a second stone area (240), a third stone area (250), a fourth stone area (260), and a fifth stone area (270) are illustrated. The stone identification module (120) can receive as input information regarding the coordinates of the area containing the structure and the class to which the structure included in the area belongs. The stone identification module (120) can extract a feature map from an image using a neural network, generate features of the stone and features of the surrounding area of ​​the stone from the feature map, combine the features of the stone and the features of the surrounding area of ​​the stone, and perform calculations on the combined features with weights to determine what type of stone it is. For example, the stone identification module (120) obtains information from the structure detection module (110) regarding the coordinates of the vertices of the first stone region (230) and that the first stone region (230) is classified as a stone, extracts a feature map from the image (20) using a neural network, generates features of the stone based on features within the region corresponding to the stone region among the feature maps, generates features of the periphery of the stone based on features within one or more segmented regions generated by dividing the feature map into a predetermined size, combines the features of the stone and the features of the periphery of the stone, and determines the type of stone as a urinary stone based on the result of performing calculations on the combined features with weights. The weights may be weights updated based on learning. In summary, the stone contained in the first stone region (230) can be identified as a urinary stone by the stone identification module (120). The stone contained in the second stone region (240) can be identified as a kidney stone. The stone contained in the third stone area (250) can be identified as a bladder stone. The stone contained in the fourth stone area (260) can be identified as a kidney stone. The stone contained in the fifth stone area (270) can be identified as a urethral stone.Thus, the stone identification module (120) is implemented using a neural network based on weights updated in a direction that minimizes loss based on an operation between a feature vector formed by merging a feature vector for the stone and a feature vector for the surrounding area of ​​the stone, so as to reflect not only the features of the stone but also the features of the surrounding area of ​​the stone, thereby enabling the identification of specific types of stones (urinary stones, kidney stones, bladder stones, etc.).

[0040] The post-processing module (130) obtains identification results from the stone identification module (120) and can exclude stones determined to be incorrectly detected from the identification results by comparing the location of the stone area with the location of the bone area. The identification results may include information regarding the coordinates of the area containing the structure and the class to which the structure included in the area belongs. In the case of anatomical structures, e.g., bones, the information regarding the class is based on the class determined by the structure detection module (110). The class determined by the structure detection module (110) may include ribs and pelvic bones. In the case of pathological structures, e.g. stones, the information regarding the class is based on the class determined by the structure detection module (110) and the stone identification module (120). The class determined by the stone identification module (120) may include specific types of stones, namely urethral stones, bladder stones, and kidney stones. The post-processing module (130) can compare the location of the gallstone area with the location of the bone area to identify gallstones that are determined to be incorrectly detected among the gallstones included in the identification result. That is, the post-processing module (130) can identify gallstones that are determined to be incorrectly detected based on the relative location of the gallstone area. The process of the post-processing module (130) excluding gallstones determined to be incorrectly detected based on the relative location of the gallstone area from the identification result will be explained in detail below with reference to FIG. 4.

[0041] FIG. 4 is a drawing for explaining a post-processing method according to one embodiment of the present disclosure.

[0042] Referring to FIG. 4, a first bone region (210), a second bone region (220), a first calculus region (230), a second calculus region (240), a third calculus region (250), a fourth calculus region (260), and a fifth calculus region (270) are illustrated. The post-processing module (130) may receive as input information regarding the coordinates of the region containing the structure and the class to which the structure included in the region belongs. For example, regarding the first bone region (210), the post-processing module (130) may obtain information regarding the coordinates of the vertex of the first bone region (210) and that the structure included in the first bone region (210) is classified as a rib. For another example, regarding the first calculus region (230), the post-processing module (130) may obtain information regarding the coordinates of the vertex of the first calculus region (230) and that the structure included in the first calculus region (230) is classified as a urinary calculus. The post-processing module (130) can identify stones that are determined to be incorrectly detected by comparing the location of the stone area with the location of the bone area. For example, the post-processing module (130) can compare the location of each of the first stone area (230), the second stone area (240), the third stone area (250), the fourth stone area (260), and the fifth stone area (270) with the location of each of the first bone area (210) and the second bone area (220). The post-processing module (130) can compare the location of a line segment extending one side boundary of the bone area with the location of the center of the stone area. For example, the post-processing module (130) can compare the location of line segment L1 extending the right boundary of the first bone area (210) with the location of the center of each of the first stone area (230) to the fifth stone area (270). As another example, the post-processing module (130) can compare the location of L2 with the location of the center of each of the first to fifth bone regions (230) based on the line segment L2 extending the left boundary of the second bone region (220).The post-processing module (130) can exclude stones that are determined to be incorrectly detected from the identification results based on the comparison results. For example, since urinary stones cannot be located above the ribs, the post-processing module (130) can exclude from the identification results a first stone region (230) identified as a urinary stone, whose center is located to the left of L1, which represents the right boundary of the ribs in the image. For another example, since kidney stones cannot be located below the pelvic bone, the post-processing module (130) can exclude from the identification results a fourth stone region (260) identified as a kidney stone, whose center is located to the right of L2, which represents the left boundary of the pelvic bone in the image.

[0043] FIG. 5 is a diagram illustrating an exemplary post-processing result according to one embodiment of the present disclosure.

[0044] Referring to FIG. 5, a second stone area (240) identified as a kidney stone, a third stone area (250) identified as a bladder stone, and a fifth stone area (270) identified as a urethral stone are shown. Compared to FIG. 3, a first stone area (230) incorrectly identified as a urethral stone and a fourth stone area (260) incorrectly identified as a kidney stone have been excluded. In this way, the post-processing module (130) can exclude stones determined to be incorrectly detected from the stone identification results by comparing the location of the bone with the location of the stone. That is, the post-processing module (130) can improve the accuracy of detection and identification by verifying the detection and identification results based on the relative location of the stone to the bone and excluding data determined to be incorrectly detected from the final results.

[0045] FIG. 6 is a diagram illustrating an exemplary visualization result according to one embodiment of the present disclosure.

[0046] Referring to FIG. 6, a second missing area (240) and second missing information (640), a third missing area (250) and third missing information (650), a fifth missing area (270) and fifth missing information (670) are illustrated. The visualization module (140) can obtain an identification result and, based on the obtained identification result, display the missing area, the type of missing, and the probability of the identification result. The identification result may include the coordinates of the area containing the structure, information about the class to which the structure included in the area belongs, and the probability that the structure included in the area belongs to that class. According to an embodiment, the structure may include i) all stones detected by the structure detection module (110) and identified by the stone identification module (120), or ii) only the stones remaining after being selected by the post-processing module (130) among the stones detected by the structure detection module (110) and identified by the stone identification module (120), i.e., stones determined to be correctly detected. FIG. 6 illustrates the case of ii). For example, the visualization module (140) may obtain from the post-processing module (130) the coordinates of the vertices of the second stone area (240), the third stone area (250), and the fifth stone area (270), information that the stones included in each stone area are classified as kidney stones, bladder stones, and urethral stones, and information that the probability that the structures included in each area are the corresponding stones is 92%, 95%, and 88%, and output the second stone area (240) and second stone information (640), the third stone area (250) and third stone information (650), and the fifth stone area (270) and fifth stone information (670) based on the obtained information. The second stone information (640) may include information that the structures are classified as kidney stones and that the probability that the structures are kidney stones is 92%. The third stone information (650) may include information that the structure is classified as a bladder stone and that the probability that the structure is a bladder stone is 95%.The fifth stone information (670) may include information that the structure is classified as a urinary stone and that there is an 88% probability that the structure is a urinary stone. The visualization module (140) may perform visualization using one or more of a visualization library, such as OpenCV, Matplotlib, and JavaScript. According to an embodiment, the visualization module (140) may additionally include a hardware element, such as a display device or a graphics card.

[0047] FIG. 7 is a schematic diagram showing the structure of a neural network used to identify the type of stone in a stone identification module according to one embodiment of the present disclosure.

[0048] The process of the stone identification module (120) identifying the type of stone can be implemented using one or more neural networks. The neural network used by the stone identification module (120) to identify the type of stone may include a feature extractor (710) and a classifier (720), referring to FIG. 7. The neural network may be a Convolutional Neural Network (CNN). The convolutional neural network may include one or more convolutional layers. According to embodiments, to prevent overfitting, the convolutional neural network may further include dropout layers located between convolutional layers. Additionally, to reduce the computational burden by retaining only important information, the convolutional neural network may further include pooling layers located between convolutional layers. As the layers within the feature extractor (710) become deeper, the size of the feature map generated by each convolutional layer becomes smaller, and the number of filters in the convolutional layer can increase.

[0049] The neural network used by the stone identification module (120) to identify the type of stone may be pre-trained. The training process of the neural network may include one or more of the following processes. The training of the neural network may be based on supervised learning. The neural network may receive training data as input. The training data may include an image (20), coordinates of a stone region included in the image (20), and information about the class to which the stone belongs. The information about the class may be detailed information about the type of stone, such as kidney stone, bladder stone, or urethral stone. The neural network may extract a feature map from the image (20) using a feature extractor (710).

[0050] FIG. 8a is a drawing for illustrating a method for generating features for the periphery of a calculus according to one embodiment of the present disclosure.

[0051] A neural network can generate features for the periphery of a gallstone based on a feature map extracted from an image. The neural network can generate one or more segmented regions by dividing the feature map into predetermined sizes. For example, referring to FIG. 8a, the neural network can generate nine segmented regions by dividing the feature map into 3x3 sizes. For each segmented region, the neural network can generate one or more feature vectors by summarizing the features included in the segmented region. For example, the neural network can generate a ninth feature (801) for the periphery of the gallstone by applying a pooling technique to the features included in the ninth segmented region located at the bottom right of the feature map. Similarly, the neural network can apply a pooling technique to the features included in each of the remaining eight segmented regions. In this way, the neural network can generate first through ninth features for the periphery of the gallstone. Since the number of segmented regions generated varies depending on the size of the feature map, the number of feature vectors for the periphery of the gallstone may also vary. Therefore, when there are K partition regions (K is an arbitrary natural number), the feature vector generated by applying a pooling technique to the features included in the k-th (k is an arbitrary natural number) partition region, i.e., the k-th feature regarding the periphery of the necrosis, is given below as f Ak It can be expressed as.

[0052] FIG. 8b is a drawing for explaining a method for generating features of a calculus according to one embodiment of the present disclosure.

[0053] A neural network can generate features of a stone based on a feature map extracted from an image. The neural network can select an area containing features of a stone from the feature map. That is, the neural network can select a part of the feature map corresponding to a stone area from the entire area of ​​the feature map. Specifically, the neural network can calculate the coordinates of the vertices of the area corresponding to the stone area in the feature map based on the coordinates of the vertices of the stone area detected from the image (20). For example, referring to FIG. 8b, the neural network can select a first stone-corresponding area (81) and a second stone-corresponding area (82) from the feature map as areas corresponding to the stone area.

[0054] A neural network can generate a feature vector by summarizing the features included in the region corresponding to the missing area in the feature map. For example, the neural network can generate a feature (802) for the second missing stone by applying a pooling technique to the features included in the second missing stone corresponding region (82). If there is one or more regions corresponding to the missing area, the neural network can generate one or more feature vectors by summarizing the features included in each region. For example, the neural network can generate a feature (not shown) for the first missing stone by applying a pooling technique to the features included in the first missing stone corresponding region (81) in the same way. Since multiple missing stones may exist within the body of a single animal, there may also be multiple regions corresponding to the missing area in the feature map. Therefore, the feature vector generated by applying a pooling technique to the features included in the j-th (j is an arbitrary natural number) missing stone corresponding region, that is, the feature for the j-th missing stone, is f below. Bj It can be expressed as.

[0055] A neural network can generate features for a stone and features for a periphery of a stone such that features for a stone and features for a periphery of a stone have the same dimension. That is, the neural network can apply a pooling technique to features included in the region corresponding to the stone and to features included in the segmented region so that the feature vector for the stone and the feature vector for a periphery of the stone are reduced to the same dimension. For example, the second feature for the stone (802) and the ninth feature for a periphery of the stone (801) may have a dimension of 1x1xD. D may be the number of channels.

[0056] A neural network can combine features for a stone and features for a periphery of the stone. The process of combining features for a stone and features for a periphery of the stone may include merging a feature vector for one stone with a feature vector for one segmented region. Merging may be performed by concatenating the two vectors in a channel direction. For example, a feature (802) for a second stone may be concatenated in a channel direction with a ninth feature (801) for a periphery of the stone. As a result of concatenating the feature (802) for a second stone with the ninth feature (801) for a periphery of the stone in a channel direction, a vector of 1x1x2D dimensions may be generated.

[0057] A neural network can generate a feature vector that reflects the features of a single absence and the features of the entire segmented region by performing operations on the merged feature vector with weights and repeating the merging and operation process for the entire segmented region. Equation 1 is an equation for calculating a feature vector that reflects the features of a single absence and the features of the entire segmented region.

[0058]

[0059] Referring to mathematical equation 1, f jis a feature vector reflecting the features of the j-th abscess and the features of the entire partitioned region, K is the number of partitioned regions, W mix is the weight, f Bj is the characteristics of the j stone, f Ak is the k-th feature of the periphery of the stone. ∥ signifies the concatenation operation.

[0060] The neural network can generate a final feature vector for a single absence by performing operations with weights on a feature vector that reflects the features of a single absence and the features of the entire segmented region. The weights are W mix With other weights, it can be a classification weight for calculating the relationship between classes. Equation 2 is an equation for calculating the final feature vector for a single absence.

[0061]

[0062] Referring to mathematical equation 2, f j C is the final feature vector for the j-th absence, W C is a classification weight for calculating the relationship between classes. C can be the number of classes.

[0063] Neural networks are f j C Based on this, weights can be updated in a direction that minimizes loss. Cross-entropy can be used as the loss function. In this way, the neural network is trained to update weights in a direction that minimizes loss based on operations between the merged feature vector and weights, so that it can reflect not only the features of the stone but also the features of the surrounding area of ​​the stone, thereby enabling the identification of specific types of stones (urinary stones, kidney stones, bladder stones, etc.).

[0064] FIG. 9 is a flowchart schematically illustrating a method for detecting and identifying stones according to one embodiment of the present disclosure.

[0065] Referring to FIG. 9, the structure detection module (110) can detect a stone region indicating the location and size of the stone from the image (S910).

[0066] The stone identification module (120) can generate features of the stone based on features within the region corresponding to the stone region among the feature maps extracted from the image (S920).

[0067] The stone identification module (120) can generate features for the periphery of the stone based on features within one or more segmented regions generated by dividing a feature map extracted from an image into predetermined sizes (S930).

[0068] The stone identification module (120) can combine the characteristics of the stone and the characteristics of the surrounding area of ​​the stone (S940).

[0069] The stone identification module (120) can identify the type of stone based on a combination of the characteristics of the stone and the characteristics of the surrounding area of ​​the stone (S950). Identifying the type of stone may mean determining what kind of stone it is, that is, whether it is a urinary stone, a bladder stone, a kidney stone, etc.

[0070] FIG. 10 is a schematic diagram showing the configuration of an exemplary computing device that can be used to implement the devices and methods described in the present disclosure.

[0071] The computing device (100) may include some or all of memory (1000), a processor (1020), storage (1040), an input / output interface (1060), and a communication interface (1080). The computing device (100) may be a stationary computing device such as a desktop computer or server, as well as a mobile computing device such as a laptop computer or smartphone. The computing device (100) may include any specialized hardware accelerator capable of processing operations on an artificial intelligence model in an efficient manner. For example, the computing device (100) may include a graphic processing unit (GPU), a tensor processing unit (TPU), or a neural processing unit (NPU).

[0072] Memory (1000) may store a program that enables a processor (1020) to perform a method or operation according to various embodiments of the present disclosure. For example, the program may include a plurality of instructions executable by the processor (1020), and the aforementioned method or operation may be performed by executing the plurality of instructions by the processor (1020). Memory (1000) may be a single memory or a plurality of memories. In this case, information required to perform a method or operation according to various embodiments of the present disclosure may be stored in a single memory or divided and stored in a plurality of memories. If memory (1000) is composed of a plurality of memories, the plurality of memories may be physically separated. Memory (1000) may include at least one of volatile memory and non-volatile memory. Volatile memory includes Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM), etc., and non-volatile memory includes flash memory, etc.

[0073] The processor (1020) may include at least one core capable of executing at least one instruction. The processor (1020) may execute instructions stored in memory (1000). The processor (1020) may be a single processor or multiple processors.

[0074] Storage (1040) retains stored data even when power supplied to the computing device (100) is cut off. For example, storage (1040) may include non-volatile memory and may include storage media such as magnetic tape, optical disc, or magnetic disc. A program stored in storage (1040) may be loaded into memory (1000) before being executed by the processor (1020). Storage (1040) may store a file written in a programming language, and a program generated from the file by a compiler, etc., may be loaded into memory (1000). Storage (1040) may store data to be processed by the processor (1020) and / or data processed by the processor (1020).

[0075] The input / output interface (1060) may provide an interface with input devices such as a keyboard, mouse, etc. and / or output devices such as a display device, printer, etc. The user may trigger the execution of a program by the processor (1020) through the input device and / or check the processing results of the processor (1020) through the output device.

[0076] The communication interface (1080) can provide access to an external network. The computing device (100) can communicate with other devices through the communication interface (1080).

[0077] Each component of the device or method according to the present invention may be implemented in hardware or software, or in a combination of hardware and software. Additionally, the function of each component may be implemented in software, and a microprocessor may be implemented to execute the function of the software corresponding to each component.

[0078] Various embodiments of the systems and techniques described herein may be realized as digital electronic circuits, integrated circuits, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include being implemented as one or more computer programs executable on a programmable system. A programmable system comprises a storage system, at least one input device, and at least one programmable processor (which may be a special-purpose processor or a general-purpose processor) coupled to receive data and instructions from and transmit data and instructions to at least one output device. Computer programs (which are also known as programs, software, software applications, or code) include instructions for the programmable processor and are stored on a "computer-readable recording medium."

[0079] Computer-readable recording media include all types of recording devices in which data that can be read by a computer system is stored. Such computer-readable recording media may be non-volatile or non-transitory media such as ROM, CD-ROM, magnetic tape, floppy disk, memory card, hard disk, magneto-optical disk, and storage device, and may also include transitory media such as data transmission media. Additionally, computer-readable recording media may be distributed across networked computer systems, and computer-readable code may be stored and executed in a distributed manner.

[0080] Although the flowcharts and timing diagrams in this specification describe each process as being executed sequentially, this is merely an illustrative explanation of the technical concept of one embodiment of the present disclosure. In other words, a person skilled in the art to which one embodiment of the present disclosure belongs may modify and adapt the flowcharts and timing diagrams in various ways, such as changing the order described in the flowcharts and timing diagrams or executing one or more of the processes in parallel, without departing from the essential characteristics of one embodiment of the present disclosure; therefore, the flowcharts and timing diagrams are not limited to a chronological order.

[0081] The above description is merely an illustrative explanation of the technical concept of the present embodiment, and a person skilled in the art to which the present embodiment belongs would be able to make various modifications and variations within the scope of the essential characteristics of the present embodiment. Accordingly, the present embodiments are intended to explain, not limit, the technical concept of the present embodiment, and the scope of the technical concept of the present embodiment is not limited by these embodiments. The scope of protection of the present embodiment shall be interpreted by the claims below, and all technical concepts within an equivalent scope shall be interpreted as being included within the scope of rights of the present embodiment.

[0082] CROSS-REFERENCE TO RELATED APPLICATION

[0083] This patent application claims priority to Korean patent application No. 10-2024-0141636, filed on October 16, 2024, the entirety of which is incorporated herein by reference.

Claims

1. A computer-implemented method for detecting and identifying stones from an image, A process of detecting a stone region indicating the location and size of the stone from the above image; A process of generating features of the necrosis based on features within a region corresponding to the necrosis region among the feature maps extracted from the above image; A process of generating features for the periphery of the gallstone based on features within one or more segmented regions generated by dividing a feature map extracted from the above image into predetermined sizes; and A computer-implemented method comprising a process for identifying the type of stone based on a combined feature of the characteristics of the stone and the characteristics of the surrounding area of ​​the stone.

2. In Paragraph 1, A computer implementation method in which a feature combining the features of the stone and the features of the periphery of the stone is generated based on a process of merging a feature vector for the stone and a feature vector for the periphery of the stone.

3. In Paragraph 2, The process of identifying the type of stone based on the combined features described above includes the process of identifying the type of stone by calculating weights with a feature vector formed by merging a feature vector for the stone and a feature vector for the periphery of the stone, wherein A computer implementation method in which the above weights are updated in a direction that minimizes loss based on the result of an operation between the merged feature vector and the weights, so as to reflect both the features of the above stone and the features of the surrounding area of ​​the above stone.

4. In Paragraph 1, The above detection process further includes a process of detecting a bone region indicating the location and size of the bone from the image, and A computer-implemented method that, after the process of identifying the type of stone, further includes the process of comparing the location of the stone region with the location of the bone region to exclude stones determined to be incorrectly detected from the identification result.

5. In Paragraph 1, The process of generating the characteristics of the above stone is, A computer implementation method comprising the process of generating the features of the necrosis by summarizing the features included in the region corresponding to the necrosis region among the feature maps above.

6. In Paragraph 1, The process of generating features for the periphery of the above stone is, A computer-implemented method comprising the process of generating features for the periphery of the necrosis by summarizing the features included in the segmented region within the feature map.

7. In Paragraph 2, The process of generating the features of the above-mentioned necrosis includes the process of applying a pooling technique to features within a region corresponding to the necrosis region among the feature maps, and The process of generating features for the periphery of the stone includes the process of applying a pooling technique to features within the segmented region such that the features for the periphery of the stone have the same dimension as the features of the stone. A computer-implemented method comprising the process of merging the above-mentioned necrosis and the process of connecting the feature vector for the necrosis and the feature vector for the periphery of the above-mentioned necrosis in a channel direction.

8. A device for detecting and identifying stones from images, Includes a structure detection module; and an absence identification module, but, The above structure detection module is, Detecting a stone region indicating the location and size of the stone from the above image, and The above-mentioned stone identification module is, Based on features within the region corresponding to the stone region among the feature maps extracted from the above image, features of the stone are generated, and Features for the periphery of the gallstone are generated based on features within one or more segmented regions created by dividing a feature map extracted from the above image into predetermined sizes, and A device for identifying the type of stone based on a combined feature of the characteristics of the stone and the characteristics of the surrounding area of ​​the stone.

9. A computer program stored on a computer-readable recording medium to execute each process included in the method according to any one of paragraphs 1 through 7.

Images