Deep learning model-based ultrasound scanner system and method for extracting region of interest using same system
The ultrasound scanner system addresses segmentation inaccuracies in bladder scanners by employing a deep learning model with 3D convolution and edge-preserving padding, improving bladder volume measurement accuracy.
Patent Information
- Application Number
- PCT/KR2025/007790
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-05-09
- Filing Date
- 2025-06-06
- Publication Date
- 2026-11-12
AI Technical Summary
Conventional ultrasound bladder scanners suffer from inaccurate bladder segmentation due to limited image quality, low resolution, and noise intensity, leading to errors in urine volume estimation, especially when uterine fibroids or cysts are mistaken for the bladder, and mechanical scanning methods are slower than electronic scanning.
An ultrasound scanner system utilizing a deep learning model with a 3D convolution network and enhanced preprocessing by padding three-dimensional image data with adjacent frame data to improve bladder segmentation accuracy, particularly at image edges, using a circular rotational scanning method to acquire cross-sectional images.
The system enhances the accuracy of bladder segmentation and reduces errors in urine volume estimation by leveraging deep learning technology and appropriate padding techniques, ensuring precise measurement of bladder volume.
Smart Images

Figure KR2025007790_12112026_PF_FP_ABST
Abstract
Description
Ultrasound scanner system based on a deep learning model and method for extracting a region of interest in the said system
[0001] The present invention relates to an ultrasound scanner system based on a deep learning model, and more specifically, to an ultrasound scanner system configured to pad three-dimensional image data obtained through a circular rotational scanning method into one or more frames, apply the padded three-dimensional image data to a deep learning model to estimate and output a probability value in which each pixel of two-dimensional cross-sectional images included in the three-dimensional image data is included in a region of interest, and to a method for extracting a region of interest in said system.
[0002] In healthy adults, the maximum capacity of the bladder is approximately 400–500 ml, and on average, urination occurs when about 200–300 ml of urine is stored. Bladder size and function can vary from person to person and may differ depending on gender and age. Typically, the urge to urinate is felt when the bladder fills with about 150 ml of urine. This is because the bladder wall expands, sending a signal to the brain via the bladder nerves that the bladder is full. Upon feeling the urge, most people go to the restroom to urinate.
[0003] There are cases where the urge to urinate is not felt at all due to various physiological factors, even though the bladder is full. If the sensory nerves of the bladder are damaged due to neurological diseases such as stroke, Parkinson's disease, or dementia, one may not feel the urge to urinate even when the bladder is full. In men, if an enlarged prostate compresses the urethra, the urge to urinate may not be felt even when the bladder is full. Additionally, if the bladder mucosa is damaged or inflamed due to cystitis, bladder function deteriorates, making it difficult to feel the urge to urinate. Nerve damage caused by diabetes can also dull bladder sensation, making it difficult to sense the urge. Furthermore, as aging progresses and pelvic muscles weaken, they fail to adequately support the bladder, resulting in difficulty feeling the urge to urinate even when the bladder is full. Other causes include side effects of medication that dull bladder sensation. Finally, the urge to urinate may not be felt due to the temporary paralysis of the bladder nervous system following spinal surgery.
[0004] If normal urination does not occur even when the bladder is full, various symptoms may arise. For example, a sensation of residual urine, feeling as though urine remains in the bladder even after urinating; frequent urination, urinating more than 8 times a day; nocturia, feeling the urge to urinate frequently during sleep; and urinary incontinence, where urine leaks out due to an inability to hold it in. Additionally, cystitis or a decline in kidney function may occur.
[0005] In cases where a normal urge to urinate is not felt due to the various reasons described above, bladder urine volume is measured using various types of biometric devices to diagnose bladder function, or, if urination is necessary, forced urination is performed using a urinary catheter. Methods for measuring bladder urine volume include X-ray imaging, bioimpedance measurement, and ultrasound imaging. The most widely used method for measuring bladder urine volume is ultrasound imaging. Ultrasound imaging involves using an ultrasound scanner to acquire ultrasound cross-sectional images of the bladder area and using these images to estimate the bladder urine volume.
[0006] Since ultrasound cross-sectional images show the shape of the bladder exactly as it is, the amount of urine in the bladder can be accurately estimated from the cross-sectional image information of the bladder. Because the three-dimensional shape of the bladder varies from person to person, a three-dimensional image of the bladder must be obtained to accurately measure the amount of urine. A three-dimensional ultrasound image is constructed by obtaining ultrasound cross-sectional images passing through the bladder from multiple directions so that all parts of the bladder are included, and then the bladder area is segmented in the three-dimensional ultrasound image. The amount of urine in the bladder is then estimated by measuring various geometric indicator values of the bladder area, such as height or width.
[0007]
[0008] Meanwhile, an ultrasound scanner is a medical device that visualizes the internal structure of the human body using ultrasound waves. An ultrasound scanner converts electrical energy into ultrasound energy by applying voltage to a transducer that generates ultrasound signals in the several MHz band. The generated ultrasound propagates into the body through the skin, and these ultrasound waves are absorbed, scattered, or reflected by biological tissues. The degree to which ultrasound is scattered or reflected by tissues depends on physical properties such as tissue density and elasticity. The scattered or reflected ultrasound is received again by a transducer, which converts the ultrasound into an electrical signal. The received electrical signal is processed by a computer to form an image; in this process, a two-dimensional or three-dimensional image is generated based on the time the ultrasound was generated and the intensity of the reflected or scattered ultrasound signals. Non-invasive ultrasound scanners capable of real-time imaging are being used effectively in various medical situations.
[0009] The clinical application of ultrasound scanners dedicated to imaging the bladder—specifically, bladder ultrasound scanners—is on the rise. While general ultrasound scanners can image various parts of the human body and offer diverse functions, bladder ultrasound scanners are specialized for observing the shape of the bladder or measuring the volume of urine within it. Consequently, even those who are not ultrasound imaging specialists can easily and conveniently measure bladder volume using a bladder ultrasound scanner.
[0010] However, since ultrasound bladder scanners are generally manufactured as small, portable devices, they have a limited number of ultrasound transducers. Consequently, the image quality obtained from an ultrasound bladder scanner is significantly inferior to that of a standard ultrasound scanner, often leading to errors in estimating urine volume. Additionally, in female patients, uterine fibroids or cysts may be mistaken for the bladder.
[0011] Furthermore, conventional ultrasound bladder scanners utilized an image processing method that detects bladder wall boundaries (edge detection) by leveraging spatial gradient information of pixel brightness. However, this conventional method had a vulnerability in that it frequently caused errors in bladder wall detection due to the low resolution and high noise intensity of the images acquired by the ultrasound bladder scanner.
[0012] Recently, with the rapid advancement of deep learning technology, it is being widely used to segment organs or lesions in various medical images, such as X-ray images, magnetic resonance imaging (MRI), and ultrasound images. By using deep learning technology to segment regions of interest in medical images, the accuracy and reliability of the segmentation can be significantly improved compared to segmenting regions of interest in medical images using conventional image processing technology. Consequently, there is an increasing trend of utilizing deep learning technology in ultrasound bladder scanners to enhance the performance of conventional ultrasound bladder scanners. Ultrasound bladder scanners can also reduce measurement errors in urine volume by using deep learning technology to automatically segment the bladder area from ultrasound images and calculating the urine volume within the bladder using the segmentation information.
[0013] Supervised deep learning methods based on supervised learning are primarily used in ultrasound bladder scanners. Supervised learning is a method of training a deep learning model by using input data along with corresponding labels. For supervised learning, training data is first generated by experts segmenting bladder regions and assigning labels to ultrasound images of the bladder. Then, the deep learning model is supervised using this training data. The ultrasound bladder scanner predicts bladder regions from ultrasound images using the trained deep learning model.
[0014] Generally, since an ultrasound bladder scanner must measure the amount of urine in the bladder within a short period of time, the imaging time for acquiring ultrasound images is also short. Due to these limitations on imaging time, it is difficult for an ultrasound bladder scanner to acquire many 2D ultrasound cross-sectional images, and therefore, typically about 10 ultrasound cross-sectional images are captured.
[0015] Ultrasound bladder scanners were developed for imaging only the bladder area. Therefore, compared to general-purpose ultrasound scanners, ultrasound bladder scanners generally have a very simple structure, are compact, and lightweight.
[0016] Meanwhile, general-purpose ultrasound scanners utilize phased array transducers and adopt an electronic scanning method, resulting in fast imaging speeds and high image resolution. However, ultrasound bladder scanners generally use single-element transducers to keep costs down. To obtain a three-dimensional bladder image using a single-element transducer, ultrasound bladder scanners must mechanically scan the bladder in two directions. Ultrasound bladder scanners using this mechanical scanning method take several seconds to scan, which is significantly slower than electronic scanning methods.
[0017] The scanning methods primarily used in ultrasound bladder scanners include sector scanning and circular rotational scanning. Sector scanning is a scanning method used to acquire ultrasound images of a two-dimensional cross-section constituting a single sector plane using a single ultrasound transducer. Fig. 1 is a schematic diagram illustrating the sector scanning method of an ultrasound transducer. As shown in Fig. 1, ultrasound signals for each scan line are acquired while rotating the ultrasound transducer by an angle θ, and a two-dimensional ultrasound cross-sectional image in the shape of a fan is generated using the acquired ultrasound signals. In Fig. 1, Δθ is the separation angle between adjacent scan lines, and N θ N refers to the number of scan lines forming a cross-section. In ultrasound bladder scanners, the N value is approximately 50 to 200, and since the number of scan lines and the scanning speed are inversely proportional, an appropriate value is selected considering the clinical usage environment.
[0018] Meanwhile, the cyclic rotational scanning method is a scanning method for acquiring a three-dimensional image, wherein two-dimensional cross-sectional images of multiple sector planes are acquired by scanning while rotating the sector planes. FIG. 2 is a schematic diagram illustrating the cyclic rotational scanning method of an ultrasonic transducer. In FIG. 2, the dotted line (201) indicates sector planes, and Δφ represents the separation angle between adjacent cross-sections. The value of Δφ generally has a value of about 10 to 15 degrees. Images of M sector planes are acquired while rotating the ultrasonic transducer 180 degrees, and a three-dimensional image can be obtained using these.
[0019] Unlike X-ray or magnetic resonance imaging, the cross-sectional thickness in ultrasound imaging is difficult to define as a constant value. This is because the thickness of the ultrasound beam varies depending on the imaging depth, that is, as the ultrasound propagates. When imaging up to a depth of 20 cm using a single transducer with a diameter of 10 mm, the thickness of the ultrasound beam has a minimum value of approximately 5 mm at the focal distance and an average thickness of about 10 mm across the entire imaging area. Therefore, it can be said that the average cross-sectional thickness of the ultrasound cross-sectional image is also approximately 10 mm. While significantly increasing the number of cross-sections increases the imaging time required to obtain a 3D ultrasound image in proportion to the number of cross-sections, the spatial resolution of the ultrasound image does not improve significantly due to the influence of cross-sectional thickness. Therefore, it is common practice to limit the number of cross-sections in ultrasound bladder scanners to 15 or fewer.
[0020] After obtaining a three-dimensional ultrasound image of the bladder, the bladder region must be segmented in the cross-sectional image. The human body has a single bladder, and if the ureters connected to the bladder are ignored, the bladder consists of a closed surface. Therefore, the ideal result of segmenting the bladder in a two-dimensional cross-sectional image should always be in the form of a single closed area or nothing at all. FIG. 3 illustrates exemplary results of segmenting the bladder in a two-dimensional ultrasound cross-sectional image. As shown in FIG. 3 (a), if there is no bladder in the corresponding cross section, the segmentation result should appear as nothing. And, as shown in FIG. 3 (b), if there is a bladder in the corresponding cross section, it should appear as a single closed area (301).
[0021] However, the actual bladder segmentation result generally does not appear as a single closed plane due to various factors. Figure 4 illustrates exemplary actual bladder segmentation results. Figure 4(a) shows the result of a bladder region 401 being segmented normally. However, as shown in Figure 4(b), if the bladder is outside the scan angle (view angle) in the ultrasound image, the bladder region 402 may appear as an open segment. Additionally, as shown in Figure 4(c), the bladder may appear as multiple regions 403 due to ultrasound noise or the effect of a thick ultrasound beam. Furthermore, as shown in Figure 4(d), in the case of women, the uterus or uterine cyst adjacent to the bladder may be mistaken for the bladder, resulting in the segmentation into multiple regions 404. When such incorrect segmentation results occur, a post-processing step is performed by establishing various types of judgment rules, namely rules to determine whether the segmented bladder region is true or false. However, even if post-processing is performed, if there are errors, the bladder volume value calculated from the 3D image information of the bladder will also have errors.
[0022] Recently, there has been an increasing number of cases using deep learning technology to improve the accuracy of ultrasound bladder scanners. This is because it is expected that segmenting the bladder region in ultrasound images using deep learning technology can partially resolve the problems associated with the conventional general image processing methods described earlier. In particular, it has been reported that adopting a supervised deep learning method can significantly improve the accuracy of segmentation in ultrasound images because the results of bladder region segmentation performed by experts based on their prior background knowledge and experience can be utilized in training the deep learning model.
[0023] UNet, a variation of the Convolutional Neural Network (CNN), is widely used for the segmentation of medical images. UNet utilizes multiple convolution and pooling layers to extract feature maps inherent in the input image and upsamples these low-resolution feature maps to restore them to the original image resolution. The structure of UNet consists of an encoder and a decoder; the encoder converts the input image to a low resolution while extracting feature maps, while the decoder restores the extracted feature maps to the original input image resolution. UNet is particularly widely used in the field of medical imaging to segment organs or lesion regions. Because UNet has relatively fewer parameters compared to other deep learning models, it offers the advantages of fast training speeds and high accuracy. Furthermore, since UNet is suitable for transfer learning—which improves network model performance when new input data is added—it facilitates the easy reuse of previously trained models to enhance performance.
[0024] When segmenting specific organs or regions in an image using UNet or CNN, the image is processed via convolution; it is common practice to apply padding, which involves adding artificial data to the edges of the image, during this process. Without padding, the size of the output image decreases with each step of convolution, leading to a problem where the output image becomes significantly smaller than the input image after multiple steps. Furthermore, without padding, the convolution is calculated primarily around the center of the image, while edge pixels are relatively less reflected in the computation, resulting in the neglect of information at the edges of the image. To address these issues, padding is generally applied to the image when applying convolution.
[0025] In a 2D convolution operation, if the input data size is nxn and the convolution kernel size is fxf, the output data size is reduced to (n - f + 1) x (n - f + 1). Typically, the value of f is often used between 3 and 5, and deep learning models have multiple convolution layers. In such cases, as the number of layers in a deep learning model increases, the input data to be trained in the deep layers may become too small, potentially leading to a situation where convolution cannot be performed. To prevent the input data size from shrinking with each passing through a convolution layer, padding must be applied to the input data before convolution to adjust its size.
[0026] The commonly used padding method is zero padding. Figure 5 illustrates an exemplary result of zero padding on a 2D image. As shown in Figure 5, zero padding is a method of adding zeros to the edges of an image. In Figure 5, for convenience, zeros were padded to only one layer at the edges; however, if the kernel size is fxf, padding with zeros for f-2 layers can make the image size before and after convolution identical. However, when the edges of an image are repeatedly padded with artificial data such as zeros, a problem arises in which the image segmentation accuracy at the edges becomes relatively worse compared to the accuracy at the center of the image. This is because virtual artificial data, rather than actual data, was padded at the edges.
[0027] Meanwhile, when performing 3D convolution on a 3D ultrasound image, padding must be applied in the remaining 1D direction (φ direction in Fig. 2) in addition to the padding within the 2D image shown in Fig. 5. In a 3D convolution operation, if the size of the input image is nxnxm and the size of the convolution kernel is fxfxg, the size of the output image is reduced to (n - f + 1) x (n - f + 1) x (m-g+1). Typically, in the case of an ultrasound bladder scanner, the number of cross-sections (i.e., the g value) is very small, ranging from 10 to 20. Therefore, if padding is repeatedly applied to the images of an ultrasound bladder scanner using artificial data, the image segmentation performance in the cross-sectional direction may be significantly degraded.
[0028] To solve the aforementioned problems, the present invention aims to provide an ultrasonic scanner system capable of improving segmentation performance for a region of interest.
[0029] In addition, the present invention aims to provide a method for extracting a region of interest that enables accurate extraction and segmentation of the region of interest by more accurately estimating the probability that each pixel constituting two-dimensional cross-sectional images for each sector plane included in three-dimensional image data in an ultrasonic scanner system is included in the region of interest.
[0030] To achieve the aforementioned technical objectives, an ultrasonic scanner system according to the first aspect of the present invention comprises: an ultrasonic probe configured to transmit and receive ultrasonic signals for acquiring ultrasonic images using an ultrasonic transducer; and a control device that generates two-dimensional cross-sectional images of a plurality of sector planes using ultrasonic signals received from the ultrasonic probe, and estimates a probability value in which each pixel of the two-dimensional cross-sectional images is included in a region of interest using a probability value estimation model composed of a deep learning network; thereby extracting a region of interest from an ultrasonic image based on an artificial intelligence model.
[0031] In an ultrasonic scanner system according to the aforementioned features of the present invention, the control device comprises: a sector cross-sectional image acquisition unit that acquires two-dimensional cross-sectional images of a plurality of sector planes using ultrasonic signals received from the ultrasonic probe; a three-dimensional image data generation unit that generates three-dimensional image data composed of a plurality of frames that are reconstructed by sequentially arranging two-dimensional cross-sectional images of a plurality of sector planes; a data preprocessing unit that performs padding preprocessing on the three-dimensional image data; and a probability value estimation unit configured to estimate a probability value by applying the preprocessed three-dimensional image data as input data to the probability value estimation model, wherein the probability value estimation model has a probability value estimation model that is pre-trained and composed of a deep learning network.
[0032] In an ultrasound scanner system according to the aforementioned features of the present invention, the three-dimensional image data generating unit preferably arranges and rearranges two-dimensional cross-sectional images sequentially according to the positions of sector planes in which two-dimensional cross-sectional images are acquired, and generates three-dimensional image data consisting of a plurality of frames that reconstruct the rearranged plurality of two-dimensional cross-sectional images.
[0033] In an ultrasonic scanner system according to the aforementioned features of the present invention, the data preprocessing unit preferably pads the front of the first frame of the three-dimensional image data with one or more last frames of the three-dimensional image data, and pads the back of the last frame of the three-dimensional image data with one or more starting frames of the three-dimensional image data.
[0034] In an ultrasonic scanner system according to the aforementioned features of the present invention, the sector cross-sectional image acquisition unit preferably acquires two-dimensional cross-sectional images of a plurality of sector planes by driving the ultrasonic probe in a circular rotational scanning manner based on a center point where the ultrasonic probe is located to scan an object to be inspected.
[0035] In an ultrasonic scanner system according to the aforementioned features of the present invention, the control device preferably further comprises a region of interest volume measuring unit that calculates and provides the volume of the region of interest using probability values in which each pixel of two-dimensional cross-sectional images is included in the region of interest.
[0036] In an ultrasound scanner system according to the aforementioned features of the present invention, the region of interest is a bladder, and the ultrasound scanner system is preferably a system for measuring the amount of urine in the bladder.
[0037] A method for extracting a region of interest according to a second aspect of the present invention relates to a method for a control device of an ultrasound scanner system based on an artificial intelligence model to extract a region of interest from ultrasound images, comprising: (a) acquiring two-dimensional cross-sectional images of a plurality of sector planes using ultrasound signals received from an ultrasound probe; (b) generating three-dimensional image data composed of a plurality of frames reconstructed by sequentially arranging two-dimensional cross-sectional images of a plurality of sector planes; (c) preprocessing three-dimensional image data by padding the three-dimensional image data; and (d) estimating a probability value by applying the preprocessed three-dimensional image data as input data to the probability value estimation model using a probability value estimation model composed of a deep learning network and pre-trained; wherein the probability value estimation model estimates and outputs a probability value in which each pixel of the two-dimensional cross-sectional images of a plurality of sector planes corresponding to the three-dimensional image data is included in the region of interest.
[0038] In the method for extracting a region of interest according to the aforementioned features of the present invention, the step (b) preferably involves sequentially arranging and rearranging two-dimensional cross-sectional images according to the positions of sector planes in which the two-dimensional cross-sectional images are acquired, and generating three-dimensional image data composed of a plurality of frames that reconstruct the rearranged plurality of two-dimensional cross-sectional images.
[0039] In the method for extracting a region of interest according to the aforementioned features of the present invention, in step (c), it is preferable to pad the front of the first frame of the three-dimensional image data with one or more last frames of the three-dimensional image data, and pad the back of the last frame of the three-dimensional image data with one or more starting frames of the three-dimensional image data.
[0040] In the method for extracting a region of interest according to the aforementioned features of the present invention, the step (a) preferably involves scanning an object to be inspected by driving the ultrasonic probe in a circular rotational scanning manner based on a center point where the ultrasonic probe is located, thereby obtaining two-dimensional cross-sectional images of a plurality of sector planes.
[0041] The method for extracting a region of interest according to the aforementioned features of the present invention preferably further comprises the step of (e) calculating and providing the volume of the region of interest using probability values in which each pixel of the two-dimensional cross-sectional images is included in the region of interest.
[0042] An ultrasonic scanner system according to the present invention having the above-described configuration can drive an ultrasonic probe in a circular rotational scanning manner with respect to a region of interest to acquire two-dimensional cross-sectional ultrasonic images of a plurality of sector planes, and then use the plurality of two-dimensional cross-sectional ultrasonic images to generate three-dimensional image data composed of a plurality of frames arranged sequentially. Furthermore, the ultrasonic scanner system according to the present invention can apply the three-dimensional image data to an artificial intelligence model for calculating the probability that each pixel is included in the region of interest.
[0043] At this time, the ultrasound scanner system according to the present invention performs preprocessing on three-dimensional image data input to an artificial intelligence model by padding it with frame data corresponding to sector planes that are actually adjacent to each other on both sides of the three-dimensional image data. By applying this padding method, the accuracy of segmenting the bladder, which is the region of interest, can be improved. In particular, the bladder segmentation error in cross-sections corresponding to the first and last frames of the three-dimensional image data can be significantly reduced.
[0044] Figure 1 is a schematic diagram illustrating the sector scanning method of an ultrasonic transducer.
[0045] Figure 2 is a schematic diagram illustrating the cyclic rotational scanning method of an ultrasonic transducer.
[0046] Figure 3 illustrates exemplary results of segmenting the bladder in a two-dimensional ultrasound cross-sectional image.
[0047] Figure 4 illustrates exemplary results of actual bladder segmentation.
[0048] Figure 5 illustrates an exemplary result of zero padding on a two-dimensional image.
[0049] FIG. 6 is a configuration diagram illustrating an ultrasonic scanner system according to a preferred embodiment of the present invention.
[0050] FIG. 7 illustrates an ultrasound scanner system according to a preferred embodiment of the present invention, wherein (a) shows three-dimensional image data represented as a three-dimensional matrix, (b) shows a kernel having a size of 3 x 3 x 3 for convolution, and (c) shows the result of convolving the three-dimensional image data.
[0051] FIG. 8 illustrates an example of a 3 x 3 x 3 size convolution kernel in an ultrasonic scanner system according to a preferred embodiment of the present invention.
[0052] FIG. 9 is illustrated to explain an embodiment of a padding method for three-dimensional image data performed by a data preprocessing unit in an ultrasound scanner system according to a preferred embodiment of the present invention.
[0053] FIG. 10 is a graph for explaining the periodicity of frames constituting three-dimensional image data in an ultrasound scanner system according to a preferred embodiment of the present invention.
[0054] FIG. 11 illustrates another embodiment of padded three-dimensional image data by a data preprocessing unit in an ultrasonic scanner system according to a preferred embodiment of the present invention, wherein (a) is a frame of padded three-dimensional image data and (b) is a bar graph illustrating the frames constituting the padded three-dimensional image data.
[0055] FIG. 12 illustrates another embodiment of padded three-dimensional image data by a data preprocessing unit in an ultrasonic scanner system according to a preferred embodiment of the present invention, wherein (a) is a frame of padded three-dimensional image data and (b) is a bar graph illustrating the frames constituting the padded three-dimensional image data.
[0056] FIG. 13 is a flowchart sequentially illustrating a method for extracting a region of interest performed by a control device in an ultrasonic scanner system according to a preferred embodiment of the present invention.
[0057] FIGS. 14 and 15 are results of segmenting regions of interest from images acquired for two different patients by applying various methods to demonstrate the performance of an ultrasound scanner system according to a preferred embodiment of the present invention.
[0058] Figure 16 is a graph showing the results of comparing the average values of the die coefficients of Cases I, II, and III.
[0059] Hereinafter, an ultrasound scanner system according to a preferred embodiment of the present invention will be described in detail with reference to the attached drawings. The ultrasound scanner system according to the present invention is characterized by being configured to improve the accuracy of segmenting the bladder, which is a region of interest, by applying a deep learning technology based on a 3D convolution network and performing preprocessing by padding both sides of the 3D image data with frame data corresponding to sector planes that are actually adjacent to each other.
[0060] FIG. 6 is a configuration diagram illustrating an ultrasonic scanner system according to a preferred embodiment of the present invention. Referring to FIG. 6, the ultrasonic scanner system (1) according to the present invention is a system that extracts a region of interest from an ultrasonic image based on an artificial intelligence model, and comprises an ultrasonic probe (10) and a control device (20).
[0061] The above-described ultrasonic probe (10) is equipped with an ultrasonic transducer and a motor for driving the ultrasonic transducer. The above-described ultrasonic probe (10) is configured to transmit and receive ultrasonic signals for acquiring ultrasonic images using the ultrasonic transducer, and transmits the received ultrasonic signals to a control device (20). The control device (20) can generate two-dimensional cross-sectional images of multiple sector planes using the ultrasonic signals received from the ultrasonic probe, and can generate three-dimensional image data composed of multiple frames by reconstructing the two-dimensional cross-sectional images. Furthermore, the control device (20) can estimate the probability value of each pixel of the two-dimensional cross-sectional images being included in the region of interest by applying the three-dimensional image data to a probability value estimation model composed of a deep learning network. The configuration of the control device will be described in more detail below.
[0062] The above control device (20) may be composed of a processor, a central processing unit, etc., and is equipped with a sector cross-sectional image acquisition unit (200), a three-dimensional image data generation unit (210), a data preprocessing unit (220), and a probability value estimation unit (230). The above control device may further be equipped with a region of interest volume measurement unit (240).
[0063] The sector cross-sectional image acquisition unit (200) receives ultrasonic signals for a plurality of sector planes from the ultrasonic probe. Furthermore, the sector cross-sectional image acquisition unit (200) can generate two-dimensional cross-sectional images for a plurality of sector planes using the ultrasonic signals acquired by a circular rotational scanning method. As shown in FIG. 2, the ultrasonic probe of the ultrasonic scanner system according to the present invention scans the ultrasonic probe in a circular rotational scanning method based on the center point where the ultrasonic probe is located, and can receive ultrasonic signals for a plurality of sector planes according to this scanning. That is, as shown in FIG. 2, the ultrasonic probe receives ultrasonic signals for each sector plane while rotating and moving by a preset Δφ angle in a circular rotational method based on the center point. Then, the sector cross-sectional image acquisition unit generates two-dimensional cross-sectional images (S1, S2,…, S2) for each sector plane using the ultrasonic signals of the ultrasonic probe. Nφ It will generate ).
[0064] The above 3D image data generation unit (210) arranges and rearranges 2D cross-sectional images of a plurality of sector planes sequentially according to the positions of the corresponding sector planes, and generates 3D image data composed of a plurality of frames corresponding to the rearranged plurality of 2D cross-sectional images.
[0065] The above data preprocessing unit (220) preprocesses the three-dimensional image data by performing padding on the three-dimensional image data. In particular, the data preprocessing unit (220) pads the edges or boundary regions of the three-dimensional image data before performing convolution, thereby enabling the preservation of information regarding the edges of the three-dimensional image data. Below, the convolution process and padding methods for the three-dimensional image data will be explained in more detail.
[0066] FIG. 7 illustrates an ultrasound scanner system according to a preferred embodiment of the present invention, wherein (a) represents three-dimensional image data as a three-dimensional matrix, (b) illustrates a kernel having a size of 3 x 3 x 3 for convolution, and (c) illustrates the result of convolution of the three-dimensional image data. For convenience, in this specification, the x-axis and y-axis represent the horizontal and vertical axes of a frame that is a cross-sectional image, respectively. In the state prior to scan conversion of the ultrasound received signal by reflecting the sector scan of the ultrasound scanner, the x-axis of the ultrasound image represents a direction perpendicular to the direction of propagation of the ultrasound beam (i.e., a horizontal direction with constant depth), and the y-axis of the ultrasound image represents the direction of propagation of the ultrasound beam (i.e., the depth direction). In the state after scan conversion, the x-axis and y-axis of the ultrasound image represent two axes representing a plane in a Cartesian coordinate system. In the present invention, since there is no difference in the implementation method depending on whether the deep learning technology is applied before or after the scan conversion of the ultrasound image, the x-axis and y-axis in this specification represent the horizontal and vertical axes of the input image, respectively. The z-axis indicates the φ-direction, which is the rotation direction of the cross-section in FIG. 2, in the cross-sectional direction. N x , N y , N z represents the number of image pixels in the x-axis, y-axis, and z-axis directions, respectively. In an ultrasound scan, N x is the number of scan lines within one section, N y is the number of samples of the received ultrasound signal, and N zis determined by the number of cross-sections. In FIG. 7(a), for convenience, the size of the 3D pixel (601) is shown as uniform in three directions. However, in a real environment, the pixel size is largest in the cross-section direction (i.e., the z-direction), because the number of cross-sections is very small, ranging from 10 to 20, and the distance between cross-sections is large. FIG. 7(b) shows a 3D convolution kernel. In FIG. 7(b), the kernel size is 3 x 3 x 3, and the kernel lengths in all three directions are equal. However, the lengths of the kernel in all three directions do not need to be equal, and generally f x xf y xf z It has the size of and f x , f y , f zcan have natural number values. Since increasing the kernel size increases the amount of computation, the kernel size is generally 3 x 3 x 3 or 5 x 5 x 5; for convenience, this invention describes it based on 3 x 3 x 3. The elements of the convolution kernel are initialized with random numbers in the first stage of training and can be repeatedly changed during the model's training process. In a deep learning model, since there is a convolution kernel for each channel and each layer, multiple convolution kernels may exist within the deep learning model. During the training process, the element values of the convolution kernel are repeatedly changed through optimization operations, and a final value can be determined in the final stage of training. Figure 7(c) shows the convolution output image. The value of one pixel (602) in the output image is the result of convolving a portion of the 3 x 3 x 3 input image shown in block (603) of FIG. 7 (a) with a convolution kernel. The convolution operation is completed when the convolution result is obtained for all pixels of the output image by moving the position of the convolution kernel in the x-axis, y-axis, and z-axis directions, either one pixel at a time or by striding across several pixel units in the input image.
[0067] FIG. 8 illustrates an example of a 3 x 3 x 3 convolution kernel in an ultrasound scanner system according to a preferred embodiment of the present invention. In FIG. 8(a), h(i,j,k) within each rectangle is the kernel element value at the (i,j,k) position of the kernel tensor. FIG. 8(b) is an example of a 3D convolution kernel in a deep learning model obtained through actual training. To obtain the (l,m,n)th pixel value O(l,m,n) of a 3D output image, a convolution must be performed with a 3 x 3 x 3 sub-image I(i,j,k) (i=l-1, l, l+1, j=m-1, m, m+1, k=n-1, n, n+1) centered on the (l,m,n)th pixel of the 3D input image I. This can be expressed as Equation 1.
[0068]
[0069] To perform convolution at the edges of a 3D input image (e.g., at position (1,1,1)), pixel values must be specified at positions that are zero in the x-axis direction (i=0), zero in the y-axis direction (j=0), and zero in the z-axis direction (k=0). However, these pixels do not exist in the input image. Therefore, virtual data is padded in the x-axis and y-axis directions.
[0070] When the input image size is sufficiently large, even if padding is applied during each convolution, the effect of padding can be diluted as the number of deep learning layers increases and convolutions are performed multiple times. Furthermore, the padding effect appears only in pixels of a few layers around the edges. Therefore, when the input image size is sufficiently large, the padding effect can become negligible.
[0071] In convolution, padding effects can occur in several cross-sections around the edges. Therefore, when the total number of cross-sections is small, the number of cross-sections exhibiting padding effects becomes significant compared to the total number of cross-sections. In the case of an ultrasound bladder scanner system according to the present invention used for bladder measurement, the number of pixels in the z-axis direction is equal to the number of cross-sections and is typically a small number, usually around 10 to 20. Consequently, padding effects in the cross-sectional direction of an ultrasound bladder scanner can cause significant errors in estimating the amount of urine in the bladder. The data preprocessing unit according to the present invention proposes a method to reduce errors caused by such padding.
[0072] FIG. 9 illustrates an embodiment of a padding method for three-dimensional image data executed by a data preprocessing unit in an ultrasound scanner system according to a preferred embodiment of the present invention. Referring to FIG. 9, the data preprocessing unit may pad the front and back sides (i.e., both ends in the z-axis direction) of the three-dimensional image data with frames in which all pixels are zero. In FIG. 9, S Nz represents the number of cross-sections actually obtained by scanning, and in an ultrasound bladder scanner, S N It is the same value as.
[0073] FIG. 10 is a graph illustrating the periodicity of frames constituting three-dimensional image data in an ultrasound scanner system according to a preferred embodiment of the present invention. In FIG. 10, the frames constituting the three-dimensional image data (i.e., two-dimensional cross-sectional images) are shown as bar graphs for convenience. In a circular rotational scanning method, scanning 180 degrees in the φ-direction yields all two-dimensional cross-sectional image information as shown in FIG. 2. In actual imaging, the scan is stopped after scanning 180 degrees, but if scanning continues without stopping at 180 degrees, the same cross-sectional information may appear repeatedly unless there is movement of the patient.
[0074] Meanwhile, the data preprocessing unit (220) can pad the front and back of the 3D image data with one or more frames, respectively, by considering the periodicity of the 3D image data in the cyclic rotation scanning method. Accordingly, the data preprocessing unit can pad the front of the first frame of the 3D image data with one or more last frames of the 3D image data, and pad the back of the last frame of the 3D image data with one or more starting frames of the 3D image data. At this time, the number of frames to be padded can be determined according to the size of the convolution kernel, the size of the stride, the number of frames constituting the 3D image data, etc.
[0075] One embodiment of the above data preprocessing unit is characterized by padding one actual frame to the first frame and the last frame on both sides along the z-axis direction of the 3D image data. FIG. 11 illustrates one embodiment of 3D image data padded by the data preprocessing unit in an ultrasound scanner system according to a preferred embodiment of the present invention, wherein (a) is a frame of the padded 3D image data and (b) is a bar graph illustrating the frames constituting the padded 3D image data. Referring to FIG. 11, therefore, the 3D image data is S1, S2, … , S Nφ It consists of frames, S1 is the first frame, S2 is the second frame, and S Nφ represents the last frame.
[0076] The first frame (S1) and the last frame (S) of the 3D image data in the circular rotation scanning method Nφ ) are actually cross-sectional images of sector planes adjacent to each other. Therefore, in front of the first frame (S1) of the 3D image data, the last frame (S Nφ Padded with ), and the last frame of the 3D image data (S Nφ The back side of ) is padded with a start frame (S1). This padding method can be applied when the size of the convolution kernel is 3 x 3 x 3 and the stride is 1.
[0077] Another embodiment of the above-described data preprocessing unit is characterized by padding a plurality of actual frames to the first frame and the last frame on each side along the z-axis direction of the 3D image data. FIG. 12 illustrates another embodiment of 3D image data padded by the data preprocessing unit in an ultrasound scanner system according to a preferred embodiment of the present invention, wherein (a) is the frames of the padded 3D image data and (b) is a bar graph illustrating the frames constituting the padded 3D image data. Referring to FIG. 12, the 3D image data is S1, S2, … , S Nφ It can be composed of frames. The data preprocessing unit comprises two last frames (S) in front of the first frame (S1) of the 3D image data. Nφ-1 , S Nφ Padded with ), and the last frame of the 3D image data (S Nφ The back side of ) is padded with two start frames (S1, S2). This padding method can be applied when the size of the convolution kernel is 5 x 5 x 5 and the stride is 1, or when the size of the convolution kernel is 3 x 3 x 3 and the stride is 2.
[0078] The probability value estimation unit (230) is configured with a deep learning network and has a pre-trained probability value estimation model (232). The probability value estimation unit (230) applies the pre-processed 3D image data as input data to the probability value estimation model (232), and the probability value estimation model estimates and outputs a probability value in which each pixel of the 2D cross-sectional images of the plurality of sector planes included in the 3D image data is included in the region of interest.
[0079] The above-mentioned region of interest volume measurement unit (240) can calculate and provide the volume of the region of interest using probability values in which each pixel of the two-dimensional cross-sectional images obtained by the probability value estimation unit is included in the region of interest. The above-mentioned region of interest volume measurement unit can divide the region of interest from three-dimensional image data or two-dimensional cross-sectional images using probability values for each pixel, calculate the areas for the divided regions of interest, and calculate and provide the volume of the region of interest using the calculated areas.
[0080] The ultrasonic scanner system according to the present invention having the above-described configuration may be configured as an ultrasonic bladder scanner for measuring the amount of urine in the bladder, and in this case, the region of interest may be the bladder.
[0081] A method for a control device of an ultrasonic scanner system according to the present invention having the aforementioned configuration to extract a region of interest from ultrasonic images is described in detail. FIG. 13 is a flowchart sequentially illustrating a method for extracting a region of interest performed by a control device in an ultrasonic scanner system according to a preferred embodiment of the present invention.
[0082] Referring to FIG. 13, a control device according to the present invention first drives an ultrasonic probe of an ultrasonic scanner system in a circular rotational scanning manner to scan, and acquires ultrasonic signals for a plurality of sector planes according to this scanning (step 100). Then, using the ultrasonic signals acquired by the circular rotational scanning method, two-dimensional cross-sectional images of a plurality of sector planes are generated (step 110).
[0083] Next, 2D cross-sectional images of multiple sector planes are sequentially arranged and reconstructed according to the positions of the sector planes, and 3D image data is generated consisting of multiple frames arranged from the reconstructed multiple 2D cross-sectional images (step 120).
[0084] Next, the 3D image data is preprocessed by padding (step 130). At this time, the front of the first frame of the 3D image data may be padded with one or more last frames of the 3D image data, and the back of the last frame of the 3D image data may be padded with one or more starting frames of the 3D image data. In addition, the number of frames padded on the front and back of the 3D image data may be determined according to the size of the convolution kernel, the size of the stride, the total number of frames constituting the 3D image data, etc.
[0085] Next, a probability value is estimated and output using a probability value estimation model configured as a deep learning network and pre-trained (step 140). More specifically, the pre-processed 3D image data is applied as input data to the pre-trained probability value estimation model, and the probability value estimation model estimates and outputs a probability value in which each pixel of the 2D cross-sectional images of multiple sector planes included in the 3D image data is included in the region of interest.
[0086] Next, the volume of the region of interest can be calculated and provided using probability values in which each pixel of the 2D cross-sectional images is included in the region of interest (step 150).
[0087] To verify the performance of the ultrasound scanner system according to a preferred embodiment of the present invention, padding methods of the data preprocessing unit were applied to a 3D UNet model to perform training. For the training of the UNet model, images of approximately 1,000 patients were utilized, and measurements were taken at least 6 times per patient on average while varying measurement conditions. Twelve cross-sectional images were acquired for each measurement, with each cross-sectional image having a size of 65 x 230. The set of patient images used for model training was constructed to represent the entire patient group by considering the patient's sex, age, bladder capacity, weight, measurement location, operator proficiency, and subject's posture. In particular, for women, cases with uterine cysts or fibroids were included, and the representativeness of the female data was ensured by constructing the image set while considering various variables such as the size and number of cysts or fibroids. The independence of the patient dataset used for network performance verification was ensured by separating and managing it from the patient dataset used for model training. The labeling task required for UNet training—specifically, the manual segmentation of the bladder region—was performed by urologists and gynecologists. To enhance the reliability of the labeling, a labeling review committee was formed consisting of two specialists who did not participate in the labeling process, and this committee reviewed and finalized the labeling. The UNet model can be designed with various structures, such as the number of encoder and decoder layers, concatenation locations, the depth of convolution layers in each layer, and the number of channels in each layer. To compare the performance of the 2D UNet model and the 3D UNet model, the model structures were finalized through design optimization for each.
[0088] To compare performance, the accuracy of bladder region segmentation was calculated for three cases: when the bladder region was segmented by applying a 2D UNet model individually to each cross-sectional image (Case I); when the bladder region was segmented by applying a 3D UNet model with zero-filled cross-sectional images padded at both ends along the z-axis (Case II); and when the bladder region was segmented by applying a 3D UNet model with actual cross-sectional images padded at both ends along the z-axis (Case III). The accuracy of bladder region segmentation was evaluated by calculating the Dice coefficient, which represents the similarity between the label and the segmented result, for each cross-sectional image and averaging the Dice coefficients across all cross-sectional images of all patients. To evaluate the performance of the trained model, images of 400 patients (200 men and 200 women) that were not used for training were utilized.
[0089] FIGS. 14 and 15 are results of segmenting regions of interest from images acquired for two different patients by applying various methods to demonstrate the performance of an ultrasound scanner system according to a preferred embodiment of the present invention. FIGS. 14 and 15 (a) shows the result of a professional segmenting a region of interest from 2D cross-sectional images of 12 sector planes, i.e., the labeled result, and is the ground truth data. FIGS. 14 and 15 (b) is Case I, which shows the result of segmenting a region of interest using a trained 2D UNet model after zero padding data preprocessing of 2D cross-sectional images of 12 sector planes. Figures 14 and 15 (c) represent Case II, showing the result of segmenting a region of interest using a 3D UNet model trained after data preprocessing in which frames consisting of all pixels as zero are padded on the front and back sides, respectively, of 3D image data reconstructed from 2D cross-sectional images of 12 sector planes. Figures 14 and 15 (d) represent Case III, showing the result of segmenting a region of interest using a 3D UNet model trained after data preprocessing in which adjacent frames are padded on the front and back sides, respectively, of 3D image data reconstructed from 2D cross-sectional images of 12 sector planes. Figures 14 and 15 (d) are the results of applying a padding method according to a preferred embodiment of the present invention.
[0090] Referring to Figures 14 and 15 (a) and (b), it can be seen that in Case 1, where a 2D UNet is applied to a 2D cross-sectional image, there are many cases where the segmentation differs from the label. This phenomenon is attributed to the fact that the 2D UNet model does not fully reflect the continuous and smoothly changing 3D shape information of the bladder, which is the region of interest. Furthermore, it can be observed that such errors occur more frequently when there is significant noise or artifacts in the ultrasound image. Through Figures 14 and 15, it can be seen that when the 3D UNet models of Case II and Case III are applied, the accuracy of bladder segmentation is higher than in the case of the 2D UNet model of Case 1. However, in the case where the cross-sectional image consisting of zeros according to Case II, as shown in Figure 14 and 15 (c), is padded, it can be confirmed that there is a large error in bladder segmentation in the first and last frames.
[0091] Figure 16 is a graph showing the results of comparing the average Dice counts of Cases I, II, and III. The average Dice count is the value obtained by averaging the Dice counts from images of both cross-sections in the z-axis direction for all patients used in the test. Referring to Figure 16, it can be seen that the average Dice count of Case III was improved by 8.7% and 3.0% compared to those of Cases I and II, respectively.
[0092] Although the present invention has been described above with reference to preferred embodiments, this is merely illustrative and does not limit the invention. Those skilled in the art will understand that various modifications and applications not exemplified above are possible within the scope of the essential characteristics of the invention. Furthermore, differences related to such modifications and applications should be interpreted as being included within the scope of the invention as defined in the appended claims.
Claims
1. An ultrasonic probe configured to transmit and receive ultrasonic signals for acquiring an ultrasonic image using an ultrasonic transducer; and A control device that generates two-dimensional cross-sectional images of a plurality of sector planes using ultrasonic signals received from the ultrasonic probe, and estimates a probability value in which each pixel of the two-dimensional cross-sectional images is included in a region of interest using a probability value estimation model composed of a deep learning network; An ultrasound scanner system characterized by having an artificial intelligence model to extract a region of interest from an ultrasound image.
2. In paragraph 1, the control device is, A sector cross-sectional image acquisition unit that acquires two-dimensional cross-sectional images of a plurality of sector planes using ultrasonic signals received from the ultrasonic probe; A 3D image data generation unit that generates 3D image data consisting of multiple frames reconstructed by sequentially arranging 2D cross-sectional images of multiple sector planes; A data preprocessing unit that performs padding preprocessing on the above 3D image data; and A probability value estimation unit configured to estimate a probability value by applying the preprocessed 3D image data as input data to the probability value estimation model, and comprising a probability value estimation model pre-trained by a deep learning network. An ultrasound scanner system characterized by the above probability value estimation model using padded 3D image data to estimate and output a probability value in which each pixel of the 2D cross-sectional images included in the 3D image data is included in the region of interest.
3. In paragraph 2, the three-dimensional image data generating unit comprises: An ultrasound scanner system characterized by sequentially arranging and rearranging two-dimensional cross-sectional images according to the positions of sector planes in which the two-dimensional cross-sectional images are acquired, and generating three-dimensional image data consisting of multiple frames reconstructed from the multiple rearranged two-dimensional cross-sectional images.
4. In paragraph 2, the data preprocessing unit is, The front of the first frame of the 3D image data is padded with one or more of the last frames of the 3D image data, and An ultrasound scanner system characterized by padding the back of the last frame of the 3D image data with one or more starting frames of the 3D image data.
5. In paragraph 2, the sector cross-sectional image acquisition unit comprises: An ultrasonic scanner system characterized by scanning an object to be inspected by driving the ultrasonic probe in a circular rotational scanning manner based on a center point where the ultrasonic probe is located, and acquiring two-dimensional cross-sectional images of a plurality of sector planes.
6. In paragraph 2, the control device is, An ultrasound scanner system characterized by further comprising: a region of interest volume measuring unit that calculates and provides the volume of a region of interest using probability values in which each pixel of two-dimensional cross-sectional images is included in the region of interest.
7. In paragraph 1, the region of interest is the bladder, and The above-described ultrasound scanner system is characterized as being a system for measuring the amount of urine in the bladder.
8. A method for a control device of an ultrasound scanner system based on an artificial intelligence model to extract a region of interest from ultrasound images, wherein (a) acquiring two-dimensional cross-sectional images of a plurality of sector planes using ultrasonic signals received from an ultrasonic probe; (b) a step of generating 3D image data composed of multiple frames reconstructed by sequentially arranging 2D cross-sectional images of multiple sector planes; (c) a step of preprocessing the 3D image data by padding the 3D image data; and (d) a step of estimating probability values by applying the preprocessed 3D image data as input data to the probability value estimation model using a probability value estimation model pre-trained and configured as a deep learning network; and A method for extracting a region of interest characterized by the above-described probability value estimation model estimating and outputting a probability value in which each pixel of a 2D cross-sectional image corresponding to a plurality of sector planes is included in the region of interest.
9. In paragraph 8, the above step (b) is, A method for extracting a region of interest characterized by sequentially arranging and rearranging two-dimensional cross-sectional images according to the positions of sector planes in which the two-dimensional cross-sectional images are acquired, and generating three-dimensional image data composed of multiple frames reconstructed from the multiple rearranged two-dimensional cross-sectional images.
10. In paragraph 8, the above step (c) is, The front of the first frame of the 3D image data is padded with one or more of the last frames of the 3D image data, and A method for extracting a region of interest characterized by padding the back of the last frame of the 3D image data with one or more starting frames of the 3D image data.
11. In paragraph 8, the above step (a) is, A method for extracting a region of interest characterized by driving an ultrasonic probe in a circular rotational scanning manner based on a center point where the ultrasonic probe is located to scan an object to be inspected, and acquiring two-dimensional cross-sectional images of multiple sector planes.
12. In paragraph 8, the above method for extracting the region of interest is, (e) a step of calculating and providing the volume of the region of interest using probability values in which each pixel of the two-dimensional cross-sectional images is included in the region of interest; a method for extracting a region of interest characterized by further comprising
