Organ volume measurement method and device using artificial neural network

Abstract

An organ volume measurement method according to an embodiment of the present invention includes: obtaining a plurality of images of the organ and shooting metadata, and preprocessing the plurality of images to obtain a plurality of image patches of a specified size; inputting the plurality of image patches into a neural network model based on a 3D convolutional neural network (CNN) to estimate organ regions corresponding to the plurality of image patches; measuring the volume of the organ using the estimated width of the organ region and the shooting metadata; and measuring the uncertainty value of the neural network model and the uncertainty values ​​of the plurality of images based on the estimation result of the neural network model.

Description

Technical Field

[0001] The present invention relates to a method and device for measuring organ volume using an artificial neural network. More specifically, the present invention relates to a method and device for measuring organ volume based on the detected organ area by taking a 3D image of the organ and detecting the organ area through an artificial neural network.

[0002] Embodiments disclosed in the present invention relate to a method for determining the development state of polycystic kidney disease or polycystic liver disease and a medical electronic device for executing the method. Background Art

[0003] Accurately measuring the volume of the kidneys and liver is a very important step for pathological physiology, treatment mechanism analysis, and evaluation of the effects of therapeutic drugs. In particular, polycystic kidney disease has the characteristic of showing diverse morphologies without morphological consistency in anatomy, so it is very difficult to grasp the morphology of the organs unless you are an expert, and even simple operations are difficult for ordinary workers to handle. It takes a long time and heavy work for doctors to operate all the hundreds of CT images (simple repetitive work). Therefore, through multi-institutional research by global units such as HALT-PKD cohort, CRISP, and SUISSE study, technology development for this problem is being actively carried out, but the scientific reliability of the organ volume itself measured by each institution currently hovers at 86%.

[0004] Polycystic kidney disease is a genetic disorder in which the kidneys become deformed into a honeycomb shape due to multiple cysts filled with fluid. As the disease progresses, the cysts gradually replace normal tissue, the kidneys increase in size, and their function gradually declines until end-stage renal failure. Some cases of polycystic kidney disease may also be accompanied by polycystic liver disease.

[0005] There are approximately 12.5 million patients with polycystic kidney disease worldwide, and it is a disease that affects about one in 500 people. Polycystic kidney disease used to be an incurable disease, but has recently attracted much attention due to the development of new drugs. Summary of the invention

[0006] Technical issues

[0007] The embodiment of the present invention provides a method and apparatus for accurately detecting the region of an organ by using an image obtained from a hospital and accurately measuring the volume of the corresponding organ.

[0008] On the other hand, in the diagnosis of polycystic kidney disease or polycystic liver disease, accurate measurement of kidney or liver volume is very important in morbid physiology or medicine. For example, the morbid physiology hypothesis that renal function is impaired (pressure effect) due to increased kidney and liver volume cannot be accurately evaluated due to the practical reason that "liver volume cannot be accurately measured". For another example, based on the volume of the kidney or liver, doctors can more accurately determine whether a patient has polycystic kidney disease or polycystic liver disease, or the degree of progression of the disease.

[0009] If the volume of kidneys or livers can be accurately measured, the hypothesis related to renal dysfunction can be evaluated more clearly, and its value will be recognized as important research data for functional evaluation of future drugs. In addition, it will greatly help doctors to grasp the appropriate treatment time point so that patients do not develop chronic renal failure.

[0010] There is a method of using a semiautomatic extraction program (2D stereology) to measure the volume of kidneys or livers, but the problem with semiautomatic extraction programs is that the purchase cost is hundreds of millions, and the analysis time is long, 1 to 2 hours per patient. In addition, when using semiautomatic extraction programs, additional labor costs will be incurred.

[0011] In addition, although the region extraction technology such as 2D stereology has been tried in various regions, it is relatively incomplete for kidneys and livers because they are difficult to grasp anatomically. For example, it is reported that the maximum accuracy of kidney region extraction using 2D stereology technology is about 86%.

[0012] Various embodiments disclosed in this application can provide a technology for solving the above problems.

[0013] Technical Solution

[0014] An organ volume measurement method according to an embodiment of the present invention may include: obtaining a plurality of images of the organ and shooting metadata, and preprocessing the plurality of images to obtain a plurality of image patches of a specified size; inputting the plurality of image patches into a neural network model based on a 3D convolutional neural network (CNN) to estimate organ regions corresponding to the plurality of image patches; measuring the volume of the organ using the estimated width of the organ region and the shooting metadata; measuring the uncertainty value of the neural network model and the uncertainty values ​​of a plurality of images based on the estimation result of the neural network model; changing at least one of the plurality of images based on the uncertainty values ​​of the plurality of images; and changing the calibration policy of the neural network model based on the uncertainty value of the neural network model.

[0015] In one embodiment, the multiple images taken of the organ may include a CT image obtained from a Digital Imaging and Communications in Medicine (DICOM) file and a calibration image of the organ, and the shooting metadata may include pixel spacing data about each of the multiple images and depth data of the images.

[0016] In one embodiment, the step of obtaining the multiple image blocks may include: performing data augmentation on a first image included in the multiple images, generating multiple images from the first image, and preprocessing the generated multiple images to obtain multiple image blocks; the data augmentation may include one or more of spatial expansion, color enhancement, noise enhancement and cropping of the image.

[0017] In one embodiment, the multiple images are multiple 3D images of the organ, and the step of obtaining the multiple image blocks may include: sliding the multiple 3D images in a depth direction to obtain the multiple image blocks of a specified size.

[0018] In one embodiment, the neural network model may perform random dropout in the learning step and the derivation step, and the uncertainty value of the neural network model may be measured based on a dispersion value of a probability distribution of result data in the derivation step of the neural network model.

[0019] In one embodiment, the uncertainty values ​​of the plurality of images may be measured based on estimated values ​​of dispersion of the resulting data in the derivation step of the neural network model.

[0020] In one embodiment, the step of changing at least one of the multiple images based on the uncertainty values ​​of the multiple images may include: the step of detecting one or more images of the multiple images whose uncertainty values ​​are above a baseline value; and the step of changing the detected image based on user input of the organ region of the detected image.

[0021] In one embodiment, the method may further include: setting weighted values ​​of a plurality of images according to the changed calibration policy, assigning a larger weighted value to the changed image than to the image before the change, so as to enable the neural network model to learn.

[0022] An organ volume measurement device according to another embodiment of the present invention may include a processor, which can obtain multiple images of an organ and shooting metadata, preprocess the multiple images to obtain multiple image patches of specified sizes, input the multiple image patches into a neural network model based on a 3D convolutional neural network (CNN), estimate the organ regions corresponding to the multiple image patches, measure the volume of the organ using the estimated width of the organ region and the shooting metadata, measure the uncertainty value of the neural network model and the uncertainty values ​​of the multiple images based on the estimation result of the neural network model, change at least one of the multiple images based on the uncertainty values ​​of the multiple images, and change the calibration policy of the neural network model based on the uncertainty value of the neural network model.

[0023] An organ volume measurement method according to an embodiment of the present invention may include: obtaining a plurality of images of the organ and shooting metadata, and preprocessing the plurality of images to obtain a plurality of image patches of a specified size; inputting the plurality of image patches into a neural network model based on a 3D convolutional neural network (CNN) to estimate organ regions corresponding to the plurality of image patches; measuring the volume of the organ using the estimated width of the organ region and the shooting metadata; measuring the uncertainty value of the neural network model and the uncertainty values ​​of a plurality of images based on the estimation result of the neural network model; changing at least one of the plurality of images based on the uncertainty values ​​of the plurality of images; and changing the calibration policy of the neural network model based on the uncertainty value of the neural network model.

[0024] In one embodiment, the multiple images taken of the organ may include a CT image obtained from a Digital Imaging and Communications in Medicine (DICOM) file and a calibration image of the organ, and the shooting metadata may include pixel spacing data about each of the multiple images and depth data of the images.

[0025] In one embodiment, the step of obtaining the multiple image blocks may include: performing data augmentation on a first image included in the multiple images, generating multiple images from the first image, and preprocessing the generated multiple images to obtain multiple image blocks; the data augmentation may include one or more of spatial expansion, color enhancement, noise enhancement and cropping of the image.

[0026] In one embodiment, the multiple images may be multiple 3D images of the organ, and the step of obtaining the multiple image blocks may include: sliding the multiple 3D images in a depth direction to obtain the multiple image blocks of a specified size.

[0027] In one embodiment, the neural network model may perform random dropout in the learning step and the derivation step, and the uncertainty value of the neural network model may be measured based on a dispersion value of a probability distribution of result data in the derivation step of the neural network model.

[0028] In one embodiment, the uncertainty values ​​of the plurality of images may be measured based on estimated values ​​of dispersion of the resulting data in the derivation step of the neural network model.

[0029] In one embodiment, the step of changing at least one of the multiple images based on the uncertainty values ​​of the multiple images may include: the step of detecting one or more images of the multiple images whose uncertainty values ​​are above a baseline value; and the step of changing the detected image based on user input of the organ region of the detected image.

[0030] In one embodiment, the method may further include: setting weighted values ​​of a plurality of images according to the changed calibration policy, assigning a larger weighted value to the changed image than to the image before the change, so as to enable the neural network model to learn.

[0031] An organ volume measurement device according to another embodiment of the present invention may include a processor, which can obtain multiple images of an organ and shooting metadata, preprocess the multiple images to obtain multiple image patches of specified sizes, input the multiple image patches into a neural network model based on a 3D convolutional neural network (CNN), estimate the organ regions corresponding to the multiple image patches, measure the volume of the organ using the estimated width of the organ region and the shooting metadata, measure the uncertainty value of the neural network model and the uncertainty values ​​of the multiple images based on the estimation result of the neural network model, change at least one of the multiple images based on the uncertainty values ​​of the multiple images, and change the calibration policy of the neural network model based on the uncertainty value of the neural network model. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 : is a diagram illustrating an example of a polycystic kidney disease discrimination system including an organ volume measurement device according to an embodiment of the present invention.

[0033] Figure 2 This is a hardware configuration diagram for explaining the internal configuration of an organ volume measurement device for measuring the organ volume of an object according to one embodiment of the present invention.

[0034] Figure 3 This is a hardware block diagram for explaining the structure and operation of the organ volume measurement device according to one embodiment of the present invention.

[0035] Figure 4 It is a sequence diagram of an organ volume measurement method according to an embodiment of the present invention.

[0036] Figure 5 is a diagram illustrating an example of neural network input data according to one embodiment of the present invention.

[0037] Figures 6 to 8 It is a diagram for explaining a method for preprocessing input data according to an embodiment of the present invention.

[0038] Figures 9 and 10 This is a diagram for explaining the structure of an artificial neural network according to one embodiment of the present invention.

[0039] Fig.11 FIG. 4 is a diagram for explaining a method for measuring an organ volume by using a detected organ region according to an embodiment of the present invention.

[0040] Fig.12 is a diagram for explaining a method of updating input data according to an embodiment of the present invention.

[0041] Fig.13 The flowchart shows a method for estimating kidney or liver volume by a medical electronic device according to an embodiment of the present invention.

[0042] Fig.14 The present invention is a flowchart showing a method for estimating kidney or liver volume and updating a deep learning model by a medical electronic device according to an embodiment of the present invention.

[0043] Fig.15 The figure shows the result of extracting and displaying a kidney or liver region in at least one test image by a medical electronic device according to one embodiment of the present invention. DETAILED DESCRIPTION

[0044] The detailed description of the present invention described below refers to the accompanying drawings that illustrate specific embodiments that can be implemented by the present invention as examples. Such embodiments are described in detail so that practitioners can implement the present invention. Although the various embodiments of the present invention are different from each other, it should be understood that they do not need to be mutually exclusive. For example, the specific shapes, structures and characteristics recorded in this specification can be changed from one embodiment to another embodiment without exceeding the spirit and scope of the present invention. In addition, the position and configuration of individual components within each embodiment should also be understood to be able to be changed without exceeding the spirit and scope of the present invention. Therefore, the detailed description described below is not for the purpose of limitation, and the scope of the present invention should include the scope of the claims of the claims and all equivalent scopes. In the drawings, similar figure marks refer to components that are identical or similar in many aspects.

[0045] In order to enable those skilled in the art to easily implement the present invention, multiple embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0046] Refer to the following Figure 1 , a polycystic kidney disease identification system including an organ volume measurement device according to an embodiment of the present invention is described in detail.

[0047] Artificial intelligence (AI) disclosed in the present invention may mean a field that studies artificial intelligence or a methodology that can create it. Machine learning, as a field of artificial intelligence technology, is a technical method in which a computing device learns through data to understand a specific object or condition, or finds and classifies data patterns, and may be an algorithm that enables a computer to analyze data. Machine learning disclosed in the present invention may be understood to include the meaning of an operating method for learning an artificial intelligence model.

[0048] The neural network model of some embodiments of the present invention below can be determined by a neural network architecture search (NAS: Neural Architecture Search) algorithm. The so-called neural network architecture search, as a method of designing an artificial neural network structure through deep learning, can, for example, learn a neural network derived from a recurrent neural network to search for the best artificial neural network structure. In more detail, the neural network model of some embodiments of the present invention can determine the filter size, stride and other metadata of each layer included in the corresponding neural network model based on the NAS algorithm. It should be noted that the aforementioned NAS algorithms all include various forms of algorithms that can be changed and designed by ordinary technicians. As an example, the NAS algorithms include SC-NAS, C-NAS and other algorithms.

[0049] The organ volume measurement device 200 of one embodiment can use multiple images 110 of the photographed organ and multiple images 120 showing the organ region as input data 100. The organ volume measurement device 200 can receive the input data 100 from an external server through the network 10. At this time, the external server can be, for example, a hospital server, and the organ volume measurement device 200 can obtain the input data 100 as a medical image format standard from the hospital server. At this time, the input data 100 including multiple images of the photographed organ included in the input data 100 can be in the format of Digital Imaging and Communications in Medicine (DICOM), and can include CT images, patient information, measurement data and photographing information. However, this is only an example, and the form of the input data 100 of some embodiments of the present invention can certainly be a variety of image file forms such as JPG, PNG, TIF, etc. In one embodiment, the input data 100 can include three-dimensional image data, and at this time, each voxel constituting the image volume can be an arrangement form of pixels. In addition, the multiple images 120 showing the organ region included in the input data 100 can be the correct answer data showing the organ region to be detected by the artificial neural network. That is, the plurality of images 120 showing the organ region may be annotated data (or calibration data) for the organ region. According to some embodiments of the present invention, an artificial neural network for detecting the organ region may perform supervised learning using the aforementioned input data 100 .

[0050] The organ volume measurement device 200 receiving the input data 100 can detect the organ region included in each image using an artificial neural network based on 3D CNN. By using an artificial neural network based on 3D CNN instead of 2D CNN, CT data obtained in the form of a three-dimensional image can be accurately analyzed. However, an ordinary technician can also perform an additional preprocessing process and use a neural network based on 2D CNN to perform the organ volume measurement method of some embodiments of the present invention. In more detail, the organ volume measurement device 200 can obtain multiple images of the organ and shooting metadata, pre-process the multiple images to obtain multiple image patches of specified sizes, input the multiple patch images into a neural network model based on a 3D convolutional neural network (CNN), estimate the organ regions corresponding to the multiple image patches, measure the organ volume using the estimated width of the organ region and the shooting metadata, measure the uncertainty value of the neural network model and the uncertainty values ​​of the multiple images based on the estimation result of the neural network model, change at least one of the multiple images based on the uncertainty value of the input data, and change the calibration policy of the neural network model based on the uncertainty value of the neural network model.

[0051] In addition, in one embodiment, the data on the detected organ region can be used to generate a 2D image or 3D model 300 corresponding to the organ and displayed on the display. At this time, the data on the detected organ region can be revised in response to user input for the 2D image or 3D model 300 displayed on the display, and the revised data can be reused as input data for the artificial neural network.

[0052] Then, the polycystic kidney disease identification device 400 can measure polycystic kidney disease using the volume data of the organ region. However, this is only an example of use when the detected organ region is the kidney or liver. Of course, according to some embodiments of the present invention, the disease or symptom to be identified may be different depending on the detected organ region.

[0053] Refer to the following Figure 2 , the internal structure of the organ volume measurement device 200 is described in detail.

[0054] In one embodiment, an organ volume measurement device 200 for measuring organ volume may include an input / output interface 201, a memory 202, a processor 203, and a communication module 204. The memory 202, as a computer-readable recording medium, may include a permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), and a disk drive. In addition, in the memory 202, program codes and settings for controlling the organ volume measurement device 200, camera images, and posture data of an object may be temporarily or permanently stored.

[0055] The processor 203 performs basic arithmetic, logic and input-output operations, and can be configured to process the commands of the computer program. The commands can be provided to the processor 203 by means of the memory 202 or the communication module 204. For example, the processor 203 can be configured to execute the commands received according to the program code stored in the recording device such as the memory 202.

[0056] The communication module 204 may provide a function for communicating with an external server through the network 10. The external server may be, for example, a hospital server that provides medical images. As an example, a request generated by the processor 203 of the organ volume measurement device 200 according to a program code stored in a recording device such as the memory 202 may be transmitted to the external server through the network 10 according to the control of the communication module 204. Conversely, a control signal or command, content, file, etc. provided according to the control of the processor of the external server may be received by the organ volume measurement device 200 through the network 10 and the communication module 204. For example, a control signal or command of an external server received through the communication module 204 may be transmitted to the processor 203 or the memory 202, and the content or file, etc. may be stored in a storage medium that the organ volume measurement device 200 may also include. In addition, the communication mode of the communication module 204 is not limited, and the network 10 may be a short-range wireless communication network. For example, the network 10 may be a Bluetooth, BLE (Bluetooth Low Energy), or Wifi (Wireless Fidelity) communication network.

[0057] In addition, the input-output interface 201 can receive input from the user and display output data. The input-output interface 201 of one embodiment can display a 2D image or 3D model corresponding to the organ on a display device. Alternatively, input feedback information can be received from the user in the detected organ area.

[0058] Although not shown in the figure, the organ volume measurement device 200 of one embodiment may further include a camera module. The camera module may be a camera module including one or more individual cameras. As an example, the camera module may be a camera module built into the organ volume measurement device 200, or may be a module connected to an independently equipped camera device. In this case, the image of the photographed organ obtained from the camera module may be stored in the memory 202 or a storage medium also included in the organ volume measurement device 200.

[0059] In addition, in another embodiment, the organ volume measurement device 200 may also include a Figure 2 However, it is not necessary to clearly illustrate most of the components of the prior art. For example, the organ volume measurement device 200 may include a battery and a charging device for supplying power to the internal components of the user terminal, and may include at least a part of the above-mentioned input and output devices, or may also include other components such as a transceiver, a GPS (Global Positioning System) module, various sensors, a database, etc.

[0060] Figure 3 This is a hardware block diagram for explaining the structure and operation of the organ volume measurement device according to one embodiment of the present invention.

[0061] In one embodiment, the organ volume measurement device 200 may include a data acquisition unit 210 , a data preprocessing unit 211 , an artificial neural network unit 220 , an uncertainty measurement unit 230 , an artificial neural network updating unit 240 and an organ volume calculation unit 250 .

[0062] In one embodiment, the data acquisition unit 210 can obtain multiple images of the photographed organ and photographing metadata. The multiple images of the photographed organ may include a CT image obtained from a Digital Imaging and Communications in Medicine (DICOM) file and a calibration image of the organ. However, this is just an example, and the input data 100 of some embodiments of the present invention can certainly be in the form of various image files such as JPG, PNG, TIF, etc. The calibrated image, as an image that identifies the organ region, can be the correct answer data for the organ region to be inferred by the artificial neural network. Therefore, the artificial neural network for inferring the organ region in some embodiments of the present invention can use the aforementioned calibration data to perform supervised learning.

[0063] In one embodiment, the data preprocessing unit 211 may perform data augmentation for ensuring sufficient data required for artificial neural network learning. In addition, in one embodiment, the data preprocessing unit 211 may preprocess multiple images to obtain multiple image patches of specified sizes. In one embodiment, the data preprocessing unit 211 for augmenting learning data may perform at least one of spatial augmentation, color augmentation, noise augmentation, and cropping. In addition, the data preprocessing unit 211 for generating multiple image patches may perform image re-sizing required for generating multiple image patches of the same size and image patch extraction that conforms to the morphology of the artificial neural network. In addition, in one embodiment, the data preprocessing unit 211 may slide in the depth direction for a 3D CT image to obtain multiple image patches.

[0064] In one embodiment, the artificial neural network unit 220 may include a neural network learning unit 221 and an organ region derivation unit 222. In one embodiment, the artificial neural network unit 220 may use an artificial neural network based on 3D CNN (Convolitional Neural Network) to perform the learning step and the derivation step. In one embodiment, the neural network learning unit 221 may use image blocks generated based on multiple CT images and multiple calibration images obtained by the data acquisition unit 210 to perform the learning of the aforementioned artificial neural network. In addition, in one embodiment, the organ region derivation unit 222 may use multiple image blocks generated based on multiple CT images to derive the organ region corresponding to each image block.

[0065] On the other hand, the aforementioned artificial neural network based on 3D CNN in some embodiments of the present invention may perform random dropout to prevent overfitting problems. According to one embodiment, the aforementioned artificial neural network may perform random dropout in both the learning step and the derivation step. At this time, the artificial neural network may randomly perform the derivation step based on the specimens of the probability distribution.

[0066] In one embodiment, the uncertainty measurement unit 230 can measure the uncertainty values ​​of the input data and the artificial neural network model. The so-called uncertainty values ​​of the input data and the artificial neural network model are values ​​that indicate whether it is possible to search for specific causes of the input data and the artificial neural network model in which errors occur. Therefore, the uncertainty measurement unit 230 of one embodiment can measure the uncertainty values ​​of the neural network model caused by insufficient data or the structure of all data. At this time, the uncertainty measurement unit 230 can measure the uncertainty values ​​of the neural network based on the dispersion value of the probability distribution of the result data in the derivation step of the artificial neural network model. In addition, even if the data has the same uncertainty value due to randomness, the uncertainty measurement unit 230 of one embodiment can also measure the uncertainty values ​​of the data caused by performing different calibrations. At this time, the uncertainty measurement unit 230 can measure the uncertainty values ​​of multiple images based on the dispersion estimated values ​​of the result data in the derivation step of the artificial neural network model.

[0067] In one embodiment, the artificial neural network updating unit 240 may revise the organ regions respectively estimated by the organ region derivation unit 222 for the plurality of images. In more detail, the artificial neural network updating unit 240 may revise the organ regions detected in the images based on the uncertainty of the artificial neural network model. At this time, the organ volume measurement device 200 may obtain information about the revised organ regions from the user using the input and output interface.

[0068] In addition, the artificial neural network updating unit 240 can change the calibration policy of the artificial neural network model based on the uncertainty value of the input data. In addition, the artificial neural network updating unit 240 can set weight values ​​for each of the multiple images according to the changed calibration policy, and give a larger weight value to the image whose estimated organ region is revised than to other images, and use it as learning data for the artificial neural network.

[0069] In one embodiment, the organ volume calculation unit 250 may estimate the volume of the organ using the width of the organ region and the shooting metadata about the plurality of images, wherein the width of the organ region is estimated using the artificial neural network unit 220 for each of the plurality of image blocks.

[0070] Figure 4 It is a sequence diagram of an organ volume measurement method according to an embodiment of the present invention.

[0071] In step S110 , the organ volume measurement device may obtain a plurality of images of the organ and photographing metadata.

[0072] The multiple images of the organ may include a CT image obtained from a DICOM (Digital Imaging and Communications in Medicine) file and a calibration image of the organ. However, this is just an example, and the input data 100 of some embodiments of the present invention may be in the form of various image files such as JPG, PNG, TIF, etc. In addition, the shooting metadata may include pixel spacing data about each of the multiple images and depth data of the image slices. In one embodiment, the aforementioned multiple images may be multiple 3D images of the organ. The calibration image, as an image that identifies the organ region, may be the correct answer data for the organ region to be inferred by the artificial neural network. Therefore, the artificial neural network for inferring the organ region in some embodiments of the present invention may use the aforementioned calibration data to perform supervised learning. A more detailed description will be described below. Figure 5 In the narrative.

[0073] In step S120, the organ volume measurement device may preprocess the multiple images to obtain multiple image patches of a specified size. In more detail, the organ volume measurement device may perform data augmentation for the multiple images respectively, generate multiple images, and preprocess the generated multiple images to obtain multiple image patches. In one embodiment, the organ volume measurement device may perform data augmentation when the learning data is insufficient. At this time, the organ volume measurement device may perform one or more of space expansion, color enhancement, noise enhancement, and cropping. In one embodiment, space expansion may include at least one of mirroring the multiple images, channel transformation for login error simulation, elastic deformation, rotation, scaling, and resampling. However, it should be noted that this is only an example of space expansion, and the embodiments of the present invention are not limited thereto. In one embodiment, color enhancement may include at least one of brightness addition, brightness multiplication, contrast adjustment, and gamma adjustment. However, it should be noted that this is only an example of color enhancement, and the embodiments of the present invention are not limited thereto. In one embodiment, noise enhancement may utilize at least one of Gaussian noise and Rician noise. However, it should be noted that this is only an example of noise enhancement, and the embodiments of the present invention are not limited thereto. In one embodiment, the cropping may be one of random cropping and center cropping. However, it should be noted that this is only an example of cropping, and the embodiments of the present invention are not limited thereto.

[0074] In addition, the organ volume measurement device can perform image resizing for generating multiple image blocks of the same size and image block extraction that conforms to the morphology of the artificial neural network. In addition, in one embodiment, the organ volume measurement device can slide in the depth direction for the 3D CT image to obtain multiple image blocks. In addition, the organ volume measurement device can generate multiple image blocks by performing overlapping at a specified ratio for a given block size. Moreover, in order to prevent the size of the data from being reduced through convolution operations, the organ volume measurement device can add padding for generating image blocks of a specified size. For example, the organ volume measurement device can add symmetric padding to the width and / or height, and in order to conform to the input data morphology of the artificial neural network based on 3D CNN, zero padding to the depth can be added. At this time, the added padding data can be "0" data. A more detailed description will be given below. Figures 6 to 8 In the narrative.

[0075] In step S130, the organ volume measurement device may input multiple block images into a neural network model based on a 3D convolutional neural network (CNN). In order to prevent the occurrence of overfitting problems, the neural network model may perform random dropout in the learning step and the derivation step. In addition, the neural network model of an embodiment may include an encoding layer and a decoding layer, wherein the encoding layer includes a reduction cell and the decoding layer includes an expansion cell. A more detailed description will be given below. Figures 9 and 10 In the narrative.

[0076] In step S140, the organ volume measurement device can use a neural network based on 3D CNN to estimate the organ region corresponding to each of the multiple image blocks. According to one embodiment, the artificial neural network for estimating the organ region can estimate the organ region for each of the multiple image blocks. In addition, the organ volume measurement device can also perform data post-processing on the result data. For example, the organ volume measurement device can perform at least one of depth spline, depth stitching and unpadding. In one embodiment, the organ volume measurement device can perform depth spline on the result data, and the farther away from the center of each image block, the lower the derivation weight value is assigned. Thus, the result data can be used to obtain a natural stitching result. In addition, in one embodiment, the organ volume measurement device can perform depth stitching on the result data corresponding to each of the multiple image blocks to form a new volume. Moreover, in one embodiment, the organ volume measurement device can also perform un-padding to control the padding added in the data preprocessing step.

[0077] In step S150, the organ volume measuring device can measure the volume of the organ using the estimated width of the organ region and the shooting metadata. More specifically, the organ volume measuring device can calculate the volume of the organ using the pixel interval, the thickness of the image slice, and the number of voxels included in the image block included in the shooting metadata. A more detailed description will be given below. Fig.11 In the narrative.

[0078] In step S160, the organ volume measurement device can measure the uncertainty value of the neural network model and the uncertainty values ​​of each of the multiple image blocks based on the estimated result of the neural network model. In one embodiment, the uncertainty value of the neural network model can be measured based on the dispersion value of the probability distribution of the result data in the derivation step of the neural network model. In addition, the uncertainty value of each of the multiple image blocks can be measured based on the dispersion estimation value of the result data in the derivation step of the neural network model.

[0079] In step S170, the organ volume measurement device may change at least one of the multiple images based on the uncertainty value of the artificial neural network model. The organ volume measurement device may select data that needs to be revised based on the uncertainty value of the model. In addition, the organ volume measurement device may establish a calibration policy based on the uncertainty value of the data. Thus, the neural network of some embodiments of the present invention may perform neural network model update (active learning) using the revised data. That is, the organ volume measurement device detects one or more images whose uncertainty values ​​of the input data are above the reference value, and changes the detected image based on the user input of the organ region of the detected image.

[0080] In step S180, the organ volume measurement device may change the calibration policy of the neural network model based on the uncertainty value of the input data. The organ volume measurement device of one embodiment may perform calibration on each input data according to the changed calibration policy, and assign a greater weight value to the input data with revised labels than to the input data with unrevised labels, so as to enable the neural network model to learn.

[0081] Figure 5 is a diagram illustrating an example of neural network input data according to one embodiment of the present invention.

[0082] (a) is the image data of the photographed organ. The multiple image data shown in (a) can be obtained from a DICOM (Digital Imaging and Communications in Medicine) file. However, this is only an example. The input data 100 of some embodiments of the present invention can be in the form of various image files such as JPG, PNG, TIF, etc. In addition, in one embodiment, the organ volume measurement device can also obtain the photographing metadata about each of the multiple images from the DICOM file. The photographing metadata can also include information related to the pixel size, image depth, voxel size, and number of each of the multiple images.

[0083] (b) is data showing the organ region. (b) as the correct answer data for the organ region to be derived by the artificial neural network may be calibration data showing the organ region. The neural network model of one embodiment may perform supervised learning using the calibration data shown in (b). However, in another embodiment, the neural network model for detecting the organ region may of course perform unsupervised learning using only the image shown in (a).

[0084] Figures 6 to 9It is a diagram for explaining a method for preprocessing input data according to an embodiment of the present invention.

[0085] If reference Figure 6 In one embodiment, the organ volume measurement device may perform data augmentation in order to ensure a sufficient amount of learning data. The organ volume measurement device may perform one or more of spatial expansion, color enhancement, noise enhancement, and cropping. In one embodiment, spatial expansion may include at least one of mirroring each of a plurality of images, channel transformation for login error simulation, elastic deformation, rotation, scaling, and resampling. In one embodiment, color enhancement may include at least one of brightness addition, brightness multiplication, contrast adjustment, and gamma adjustment. In one embodiment, noise enhancement may utilize at least one of Gaussian noise and Rician noise. In one embodiment, cropping may be one of random cropping and center cropping. For example, the image shown in (b) is an example of performing rotation on (a), the image shown in (c) is an example of reducing the brightness of (a), and the image shown in (d) is an example of performing center cropping. As described above, the organ volume measurement device of one embodiment may deform the obtained image in a variety of ways to obtain a sufficient amount of learning data required for the artificial neural network.

[0086] In addition, refer to Figure 7 In one embodiment, the organ volume measurement device may perform a data preprocessing process for obtaining an image block of a specified size. First, the organ volume measurement device may adjust the size of multiple images and extract image blocks at the same scale. Then, the organ volume measurement device may scan in one direction through a 3D sliding window to extract a 3D image of a specified size. In one embodiment, the organ volume measurement device may also extract multiple image blocks by overlapping at a specified ratio of a specified block size. For example, (b) and (c) are examples of 3D image blocks extracted from (a).

[0087] If reference Figure 8In order to adjust the size of the result data of the artificial neural network, the organ volume measurement device of one embodiment can add padding to the input data. According to some embodiments of the present invention, the artificial neural network for estimating the organ region can be a model based on 3D CNN. Therefore, as the operation is performed in the neural network, the input data performs multiple convolution operations, and the size of the result data is inevitably reduced. Therefore, in order to obtain an image block of a specified size, the organ volume measurement device can add padding to the width, height, and depth respectively. For example, Figure (b) is an example of data with depth padding added to (a). Not limited to this, according to some embodiments of the present invention, the image block with padding added can also be an image in which a specific image is copied in a symmetrical form. In one embodiment, when adding padding of a specified size, the additional data can be "0". For example, in order to be used as input data for a neural network based on 3D CNN, padding along the depth direction can be added to a 2D image. At this time, the padding with a specified depth can be composed of "0" data. In another embodiment, the size of the additional padding can also be "0". As mentioned above, the neural network model of some embodiments of the present invention uses 3D images as input data, and thus formal depth information can be appended to the input data. In this case, the depth can be "0".

[0088] Figures 9 and 10 This is a diagram for explaining the structure of an artificial neural network according to one embodiment of the present invention.

[0089] Reference Fig. 9 When receiving the image blocks 501a and 501b generated based on multiple images as input, the neural network 500 can extract the organ regions 502a and 502b corresponding to the respective image blocks. For example, the organ region 502a corresponding to the first image block 501a can be detected based on the first image block 501a, and the organ region 502b corresponding to the second image block 501b can be extracted based on the second image block 501b.

[0090] In addition, in one embodiment, the organ volume measurement device can measure the uncertainty values ​​about the neural network model and the input data. The uncertainty of the model occurs due to insufficient data or the difficulty in representing all the data in the neural network structure. Therefore, in order to prevent the occurrence of overfitting problems, the artificial neural network model of some embodiments of the present invention can perform random dropout in the learning step and the derivation step. In more detail, the organ volume measurement device can perform random predictions based on probability distribution specimens, can sample k sparse neural networks (thinned neural nets), and use the average value of each predicted value to perform the aforementioned random prediction. At this time, the probability distribution can be the probability distribution of the result data obtained in the derivation step of the neural network model, the average value at this time can be obtained as the predicted value, and the dispersion value at this time can be regarded as the uncertainty value.

[0091] The uncertainty of the input data is caused by the randomness that occurs during the data generation process. In more detail, if the uncertainty values ​​of the first data and the second data contained in the random data that is difficult to express by probability are the same, then, although the uncertainty values ​​of the first data and the second data are the same, the first data and the second data are not data obtained according to the specified probability, and thus different labels are applied. When the uncertainty of the input data occurs in this way, the organ volume measurement device of some embodiments of the present invention can measure the uncertainty value of the above-mentioned input data by variance estimation.

[0092] Fig.10 1 is an example diagram of the structure of an artificial neural network model according to an embodiment of the present invention.

[0093] In one embodiment, the neural network model 500 for extracting organ regions is a neural network model based on 3D CNN, which may include an encoding layer 510 and a decoding layer 520, wherein the encoding layer 510 includes a reduction cell, and the decoding layer 520 includes an expansion cell. However, it should be noted that the structure of the neural network model in some embodiments of the present invention is not limited to Fig.10 The content shown may have various structures that can be adopted and deformed by ordinary technicians in the field. In addition, the neural network model of one embodiment of the present invention is not limited thereto, and ordinary technicians may also perform additional preprocessing processes and use a neural network based on 2D CNN to perform the organ volume measurement method of some embodiments of the present invention.

[0094] Fig.11FIG. 4 is a diagram for explaining a method for measuring an organ volume by using a detected organ region according to an embodiment of the present invention.

[0095] According to one embodiment, the volume of the organ of the photographed object can be measured using the result data of the neural network for detecting the organ region. The organ volume measuring device can measure the volume of the organ using the estimated width of the organ region and the photographing metadata. More specifically, the organ volume measuring device can calculate the volume of the organ using the pixel interval, image slice thickness, and the number of voxels contained in the image block included in the photographing metadata. Fig.11 For example, assuming that the interval (or size) of the square pixel is 0.6 mm, the image slice thickness is 1 mm, and the number of voxels is 2,000,000, the volume of the organ is 0.6*0.6*1*2,000,000=0.36 mm2*

[0096] 2,000,000 = 720 ml. As another example, the x-coordinate of a voxel

[0097] Fig.12 is a diagram for explaining a method of updating input data according to an embodiment of the present invention.

[0098] In one embodiment, the data 502 about the multiple organ regions estimated by using the multiple image blocks can be used to generate a 3D model 602 of the corresponding organ region. In one embodiment, the organ volume measurement device can display the data 502 about the organ region or the 3D model 602 of the organ region. Then, the organ volume measurement device can change at least one of the multiple images based on the uncertainty value of the artificial neural network model. That is, the organ volume measurement device can select the data 601, 602, and 603 that need to be revised based on the uncertainty value of the model. That is, the organ volume measurement device can detect one or more images whose uncertainty values ​​of the input data are greater than the reference value, and change the detected image based on the user input of the organ region of the detected image. In addition, the organ volume measurement device can change the calibration policy of the neural network model based on the uncertainty value of the input data. The organ volume measurement device of one embodiment can be based on the changed calibration policy. The organ volume measurement device of one embodiment can perform calibration on each input data based on the changed calibration policy, and assign a larger weight value to the input data with revised labels than to the input data with unrevised labels, so that the neural network model learns. Therefore, the neural network of some embodiments of the present invention can perform neural network model updates (active learning) based on the revised data and new calibration policies.

[0099] In addition, an organ volume measurement device according to an embodiment of the present invention can use the aforementioned organ volume measurement method to verify the effect of a drug. For example, the volume of the patient's kidney and / or liver can be measured to perform a diagnosis of polycystic kidney disease, and the therapeutic effect of taking a drug can also be determined. In more detail, first, an artificial neural network that estimates the above-mentioned organ region can be used to perform kidney volume segmentation of the patient. Then, the patient data before and after drug administration is input, and kidney volume segmentation is performed and then kidney volume is measured. At this time, of course, the pixel size and image slice thickness contained in the medical digital imaging and communication file about the patient's kidney region can be used to measure the corresponding kidney volume. The drug efficacy can be measured based on the kidney volume increase or decrease rate measured according to the above method. Furthermore, an organ volume measurement device according to an embodiment of the present invention can of course provide an interface that can confirm the trend of organ volume changes over time for each patient.

[0100] As an example, the organ volume measurement method of an embodiment of the present invention can be used to measure the volume of the hippocampus. The reduction in the volume of the hippocampus is related to the occurrence of psychotic symptoms of Alzheimer's disease and dementia, so according to one embodiment, the aforementioned mental illness can be diagnosed by measuring the volume of the hippocampus. At this time, the volume of the hippocampus can be measured based on medical images, and the medical images can include CT, MRI images, but the types of medical images are not limited to this, and of course, the volume of the aforementioned hippocampus can also be measured using medical images.

[0101] As another example, the organ volume measurement method of an embodiment of the present invention can be used to measure liver volume. Liver volume is required to calculate the ratio of the remaining liver volume to the total liver volume when a large amount of liver is removed. In liver transplantation, in order to predict the liver function of the donor and the recipient, the liver volume of the donor before the operation needs to be accurately measured. Therefore, according to some embodiments of the present invention, accurate measurement of liver volume can have a positive effect on the success rate of liver transplantation and the prognosis of the patient. Moreover, in one embodiment, when not only the liver volume but also the spleen volume is measured, the success rate of liver surgery and the changes in liver volume and spleen volume can be more accurately measured.

[0102] In another example, the organ volume measurement method of an embodiment of the present invention can be used to measure the volume of a specified cancer tissue. In this case, the response to the anticancer drug can be quantified and objectively measured based on the change in the volume of the specified cancer tissue at a time point before and after the anticancer drug treatment.

[0103] Refer to the following Figures 13 to 15As an example of a device for performing an organ volume measurement method, a method for determining the development state of polycystic kidney or polycystic liver and a medical electronic device for performing the same are described in detail.

[0104] In one embodiment, the medical electronic device may include an input module, a memory, and a processor. According to various embodiments, the composition of the medical electronic device may not be limited to the foregoing. For example, the medical electronic device may omit a part of the foregoing composition, or may also include an unrecorded composition. For example, the medical electronic device may also include at least an output module electrically connected to the processor.

[0105] The input module may obtain at least one test image. The test image may, for example, include a cross-sectional image of at least a portion of a human body, such as a cross-sectional image of a kidney or liver. The test image may be an unspecified image of a kidney or liver region. For example, the test image may be an image of a cross-sectional image of a kidney or liver taken for a patient requiring measurement of the kidney or liver volume.

[0106] According to various embodiments, the test images may be multiple, and the multiple test images may be multiple slice images taken at a specified interval for at least one object. For example, the first test image may be a picture of a first slice taken for at least one object, and the second test image may be a picture of a second slice taken in parallel with a specified interval set in the first slice for at least one object. According to various embodiments, the input module may obtain at least one test image sequentially or simultaneously. For example, the input module may obtain the first test image and the second test image sequentially or simultaneously.

[0107] According to one embodiment, the input module may also obtain multiple learning images and information about the area corresponding to the kidney or the area corresponding to the liver in the multiple learning images. The multiple learning images may be, for example, multiple sample data required to obtain at least one deep learning model. In one embodiment, the multiple learning images may be images that complete the designation of the kidney or liver area. For example, the learning image may be an image that completes the designation of the kidney or liver area or volume measurement in an image of a section including the kidney or liver taken for at least one patient.

[0108] According to one embodiment, the learning image may also be an image that has completed region designation for multiple objects. For example, the learning image may be an image that has completed region designation for both liver-related regions and kidney-related regions. According to various embodiments, the accuracy of a deep learning model obtained using a learning image that has performed region designation for multiple objects is further improved. For example, when a deep learning model is obtained using a learning image that has performed region designation for multiple objects, such as the liver and kidneys, the accuracy of region designation for the liver and kidneys in the test image is further improved.

[0109] According to one embodiment, the region designation of the multiple objects can be achieved by a single signal processing. For example, for a learning image, the region designation of the liver and the region designation of the kidney can be displayed as a single identical color. According to another embodiment, the region designation of the multiple objects can be achieved by different signal processing. For example, for a learning image, the region designation of the liver and the region designation of the kidney can also be displayed in different colors. When the region designation of multiple individuals is achieved with different signals, a deep learning model with higher accuracy can be expected compared to a case where a single signal is used. According to various embodiments, different signal processing of the multiple objects can be achieved using more than two different signals.

[0110] According to various embodiments, the input module can obtain relevant information of multiple learning images or specific areas in multiple learning images in sequence or simultaneously. For example, the input module can obtain relevant information of a specific area in multiple learning images of the first group or multiple learning images of the first group, and then, it can also additionally obtain relevant information of a specific area in multiple learning images of the second group or multiple learning images of the first group. For another example, the input module 110 can also simultaneously obtain relevant information of a specific area in multiple learning images of the first group and multiple learning images of the second group and multiple learning images of the first group and the second group.

[0111] According to various embodiments, the input module may include at least one of a camera module, a scanning module, a communication module, and a user input module. For example, the input module may include a camera module, and the camera module may capture a test image. In one embodiment, the captured test image may be transferred to a processor or a memory. For another example, the input module may include a scanning module, and the scanning module may scan the captured test image and convert it into an electrical signal. In one embodiment, the converted electrical signal may be transferred to a processor or a memory. For another example, the input module may include a communication module, and the communication module may obtain a test image from an external electronic device through wireless communication or wired communication. In one embodiment, the obtained test image may be transferred to a processor or a memory.

[0112] According to one embodiment, the input module can not only obtain the test image, but also obtain the user's input. For example, the input module can include a user input module, and the user input module can obtain various inputs for test image processing from the user. For example, the input module can obtain the interval between the faults corresponding to each test image from the user for multiple test images.

[0113] The memory may include a volatile memory or a non-volatile memory. The volatile memory may be, for example, composed of a RAM (random access memory) (e.g., DRAM (dynamic random access memory), SRAM (static random access memory) or SDRAM (synchronous dynamic random access memory)). The non-volatile memory may be, for example, composed of an OTPROM (one time programmable read-only memory), a PROM (programmable read-only memory), an EPROM (erasable programmable read-only memory), an EEPROM (electrically erasable programmable read-only memory), a mask ROM (mask read-only memory), a flash ROM (flash read-only memory), a transient memory, a hard disk or a solid state drive (SSD).

[0114] According to various embodiments, the memory may, for example, store at least one other software component of the medical electronic device, for example, store commands, information or data related to the program. In various embodiments, the memory may store information obtained from the input module or information processed by the processor. In various embodiments, the information obtained from the input module may, for example, include test images, learning images, information related to specific areas in the learning images, such as information related to areas corresponding to the kidneys or liver, or user input. In various embodiments, the information processed from the processor may, for example, include information related to specific areas in the test images, such as areas corresponding to the kidneys or liver, deep learning models obtained from learning images and information related to specific areas in the learning images, etc.

[0115] The processor may include one or more of a central processing unit (CPU), an application processor (AP), or a communication processor (CP). In various embodiments, the processor may, for example, drive an operating system or an application program, control at least one other component (e.g., a hardware or software component) of a medical electronic device connected to the processor, and may perform various data processing and operations. For example, the processor may perform operations on commands or data received from at least one of the other components (e.g., an input module), and store the results in a memory.

[0116] According to one embodiment, the processor may obtain at least one deep learning model based on multiple learning images stored in the memory and relevant information about specific areas in the multiple learning images, for example, based on relevant information about areas corresponding to the kidney or liver. In one embodiment, the processor may also update the deep learning model. For example, the processor may obtain new learning images and relevant information about areas corresponding to the kidney or liver in the new learning images, and update the deep learning model stored in the memory based on the obtained images and information.

[0117] According to one embodiment, the processor may obtain at least one test image through an input module, and detect an area corresponding to a kidney or an area corresponding to a liver in the at least one test image based on a deep learning model stored in a memory.

[0118] According to one embodiment, the processor may estimate the volume of the kidney or liver contained in at least one test image. For example, the processor may obtain the width of the area detected in at least one test image, and may estimate the volume of the kidney or liver based on the obtained width. For example, the processor may estimate the volume of the kidney or liver based on the interval of at least one test image captured input from the user through the input module, for example, based on the interval between the slice captured by the first test image and the slice captured by the second test image and the obtained width. According to one embodiment, the processor may also determine the development status of polycystic kidney or polycystic liver of the kidney or liver corresponding to the at least one test image based on the estimated kidney or liver volume.

[0119] According to various embodiments, the medical electronic device may also include an output module. The output module may, for example, include at least one display device. The output module may be electrically connected to the processor to output the data transmitted from the processor in a form recognizable to the user. In various embodiments, the user-recognizable form may include at least one of text, voice, image, or video. According to various embodiments, the processor may also use the output module to display the area corresponding to the kidney or liver in the test image, or display the volume of the kidney or liver. According to another embodiment, the processor may also use the output module to display the development status of polycystic kidney or polycystic liver in the kidney or liver.

[0120] Fig.13 The flowchart shows a method for estimating kidney or liver volume by a medical electronic device according to an embodiment of the present invention.

[0121] If reference Fig.13 The method S200 for estimating the volume of kidney or liver by a medical electronic device may include steps S201 to S209. According to various embodiments, the method S200 may omit at least one of steps S201 to S209, or may further include Fig.13According to one embodiment, steps S201 to S209 may be understood as a method executed by means of the aforementioned medical electronic device or processor.

[0122] In step S201, the medical electronic device may obtain at least one deep learning model. In one embodiment, the at least one deep learning model may be obtained based on a plurality of learning images obtained through an input module and relevant information of kidney or liver regions in the plurality of learning images.

[0123] According to one embodiment, at least one deep learning model may be a model for extracting a kidney or liver region from an image including the kidney or liver. For example, at least one deep learning model may be a model for deriving a relationship between an image and a region, for example, deriving a weighted value of a plurality of elements contained in an image. In various embodiments, the plurality of elements and weighted values ​​may be different based on a training method or data of the deep learning model. According to one embodiment, the training method may include v-net training.

[0124] In one embodiment, multiple learning images and relevant information about the kidney or liver area in the multiple learning images can be a collection of previously obtained data. For example, the multiple learning images can be obtained from data previously accumulated by medical institutions, etc. In one embodiment, the relevant information about the corresponding areas of the kidney or liver in the multiple learning images can be obtained by preprocessing the multiple learning images. Preprocessing may, for example, include the original medical institution performing an action of directly displaying the area in the multiple learning images for the multiple learning images. The more types and numbers of the multiple learning images, the higher the accuracy of at least one deep learning model will be.

[0125] In various embodiments, step S201 may be omitted. For example, when there is at least one deep learning model stored in the memory, the medical electronic device may omit step S201 and execute step S203. In this case, the medical electronic device may use at least one deep learning model stored in the memory in the following steps S203 to S209.

[0126] In step S203, the medical electronic device may obtain a test image. The test image may be obtained through an input module, such as through at least one of a camera module, a scanning module, and a communication module. In one embodiment, the test image may be a two-dimensional tomographic image of a patient's body for which a kidney or liver volume measurement is required.

[0127] In step S205, the area corresponding to the kidney or liver in the test image obtained by the medical electronic device in step S203 can be detected. For example, the medical electronic device can apply the deep learning model obtained in step 201 to the obtained test image. In one embodiment, the deep learning model can include weighted value information about multiple elements, and the weighted values ​​can be applied to multiple elements included in the test image. Thus, the medical electronic device can detect a specific area in the test image corresponding to the deep learning model, for example, detect an area corresponding to the kidney or liver.

[0128] In step S207, the medical electronic device may obtain the width of the area detected in step S205. For example, the medical electronic device may obtain the width based on the size of the test image, the magnification of the test image, and / or the ratio of the detected area in the test image.

[0129] In step S209, the medical electronic device may estimate the volume of the kidney or liver included in the test image obtained in step S203. For example, when the test image is continuously photographed at an interval specified for an object, such as a human body including a kidney or a liver, the medical electronic device may estimate the volume of the kidney or the liver based on the specified interval. For example, the test image may include a plurality of images continuously photographed at an interval specified for an object. The medical electronic device may continuously perform step S207 on each of the plurality of images, and may estimate the volume of the kidney or the liver based on the result of step S207 and the specified interval.

[0130] According to various embodiments, the medical electronic device may further include an output module, and the kidney or liver volume estimated in step S209 may be displayed on the output module, so that the user can confirm the development status of polycystic kidney or polycystic liver for the patient corresponding to the test image.

[0131] Through steps S201 to S209 , the medical electronic device can accurately and quickly estimate the volume of the kidney or liver included in the test image.

[0132] Fig.14 The present invention is a flowchart showing a method for estimating kidney or liver volume and updating a deep learning model by a medical electronic device according to an embodiment of the present invention.

[0133] If reference Fig.14 The method S300 for estimating kidney or liver volume and updating deep learning model by medical electronic device may include steps S301 to S307. According to various embodiments, the method S300 may omit at least one of steps S301 to S307, or may further include Fig.14According to one embodiment, steps S301 to S307 may be understood as a method executed by means of the aforementioned medical electronic device or processor. Fig.14 In the description of Fig.13 The repeated parts of the description can be omitted.

[0134] In step S301, the medical electronic device may use the test image to estimate the volume of the kidney or liver included in the test image. Fig.13 Steps S201 to S209 shown in the figure estimate the volume of the kidney or liver included in the test image.

[0135] In step S303, the medical electronic device may determine the development status of polycystic kidney or polycystic liver based on the kidney or liver volume estimated in step S301. For example, the medical electronic device may obtain information related to the reference volume of the kidney or liver of the patient corresponding to the test image through the input module. According to various embodiments, the reference volume may be the normal kidney or liver volume of the patient, or may be the kidney or liver volume measured for the patient several months ago.

[0136] According to one embodiment, the medical electronic device can calculate the level of the kidney or liver volume obtained in step S301 compared to the reference volume, for example, how many times it is equivalent to. The medical electronic device can determine the progression of the patient's polycystic kidney or polycystic liver based on the calculated result.

[0137] In step S305, the medical electronic device may store the test image and the relevant information of the area detected in the test image in the memory. When the area corresponding to the kidney or liver in the test image is detected through step S301, the test image and the detected area may be used as the new learning image and the relevant information of the specific area in the new learning image. The more the number of learning images increases, the higher the accuracy of the deep learning model will be, and thus the medical electronic device may store the new learning image and the relevant information of the specific area in the new learning image in the memory.

[0138] In step S307, the medical electronic device can update (or re-acquire) the deep learning model. The medical electronic device obtains the new learning image and the relevant information of the specific area in the new learning image through step S305, and thus can obtain a new deep learning model or update the original deep learning model together with the multiple learning images and the relevant information of the specific area in the multiple learning images stored in the original memory. In this way, the medical electronic device can develop the deep learning model and more accurately detect the area corresponding to the kidney or liver in the test image.

[0139] Through steps S301 to S307, the medical electronic device can accurately and quickly grasp the development status of polycystic kidney disease or polycystic liver disease, and can improve the accuracy of the deep learning model.

[0140] Fig.15 The figure shows the result of extracting and displaying a kidney or liver region in at least one test image by a medical electronic device according to one embodiment of the present invention.

[0141] If reference Fig.15 , a first test image 410 and a second test image 420 are shown. The first test image 410 and the second test image 420 may be images of a cross-section of a body portion including, for example, a kidney. If the first test image 410 and the second test image 420 are referred to, it may be difficult to clearly determine the boundary between the kidney and other portions by visual inspection.

[0142] In one embodiment, if the deep learning model of the present invention is applied to the first test image 410 and the second test image 420, respectively, a first processed image 411 and a second processed image 421 can be obtained. The first processed image 411 and the second processed image 421 can be images showing the first region 412 and the second region 422 corresponding to the kidney in the first test image 410 and the second test image 420, respectively.

[0143] According to one embodiment, a test image, for example, the first test image 410 or the second test image 420, may also include at least one identification information. For example, the identification information may also include at least one of personal information about the patient, shooting date, shooting time, shooting location, slice interval, or scale information. According to one embodiment, before applying the deep learning model of the present invention, a preprocessing process may also be performed, for example, a process of deleting the identification information in the test image 410 or 420 or a process of extracting the remaining area in the test image 410 or 420 except the identification information may be performed. The preprocessing process may also be performed, for example, with the aid of the aforementioned processor.

[0144] In one embodiment, when the preprocessing process is performed on the test image 410 or 420, the test image 410 or 420 can be de-identified, and the accuracy of extracting the target area can also be improved. That is, the preprocessing process can improve the accuracy of the deep learning model in extracting the area.

[0145] According to one embodiment, the first region 412 and the second region 422 may also be separated and extracted from the first processed image 411 and the second processed image 421. The medical electronic device may obtain the widths of the extracted first region 412 and the second region 422, thereby estimating the kidney volume included in the first test image 410 and the second test image 420.

[0146] The above-described device can be embodied in the form of hardware components, software components, and / or a combination of hardware components and software components. For example, the device and components described in the embodiment can be embodied by one or more general-purpose computers or special-purpose computers such as processors, controllers, ALUs (arithmetic logic units), digital signal processors (digital signal processors), microcomputers, FPGAs (field programmable gate arrays), PLUs (programmable logic units), microprocessors, or any other device that can run instructions and respond to instructions. The processing device can execute an operating system (OS) and one or more software applications executed on the operating system. In addition, the processing device can also access, store, operate, process, and generate data in response to the operation of the software. For ease of understanding, the case where a processing device is used is described, but ordinary technicians in the corresponding technical field can understand that the processing device can include multiple processing elements (processing elements) and / or multiple types of processing elements. For example, the processing device can include multiple processors or a processor and a controller. In addition, it can also be other processing configurations (processing configurations) such as parallel processors (parallel processors).

[0147] Software may include a computer program, code, instruction, or a combination of more than one of them, which may constitute a processing device for operation as required, or independently or collectively command a processing device. Software and / or data may be embodied in some type of machinery, component, physical device, virtual device, computer storage medium or device, or transmitted signal wave, either permanently or temporarily, in order to be parsed by the processing device or to provide instructions or data to the processing device. Software may also be distributed on computer systems connected via a network and stored or run in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

[0148] The method of the embodiment can be embodied in the form of program commands that can be executed by various computer devices and recorded in a computer-readable medium. The computer-readable medium can include program commands, data files, data structures, etc. alone or in combination. The program commands recorded in the medium can be specially designed and constructed for the embodiment, or can also be known and usable by computer software practitioners. In the example of a computer-readable recording medium, magnetic media such as hard disks, floppy disks and tapes, optical recording media such as CD-ROMs (read-only optical disk drives) and DVDs (digital video disks), magneto-optical media such as floptical disks (magneto-optical media), and hardware devices specially constructed for storing and executing program commands such as read-only memories (ROMs), random access memories (RAMs), flash memories, etc. In the example of program commands, there are not only machine language codes made by means of a compiler, but also high-level language codes that can be run by means of a computer using an interpreter, etc. In order to perform the actions of the embodiment, the hardware device can be configured as one or more software modules and run, and vice versa.

[0149] As described above, although the embodiments are described according to the limited embodiments and drawings, as long as a person skilled in the art in the corresponding technical field can make various revisions and modifications from the above description. For example, the described technology is executed in a sequence different from the described method, and / or the described system, structure, device, circuit and other components are combined or combined in a form different from the described method, or replaced or replaced by other components or equivalents, even so, appropriate results can be achieved.

[0150] Therefore, different embodiments, different examples and contents equivalent to the claims also belong to the scope of the following claims.

Claims

1. A method for measuring organ volume, the method for measuring organ volume being performed by means of a computing device, comprising: A step of obtaining a plurality of images of the organ and shooting metadata, and preprocessing the plurality of images to obtain a plurality of image blocks of a specified size; The step of inputting the plurality of image blocks into a neural network model based on a 3D convolutional neural network to infer organ regions corresponding to the plurality of image blocks respectively; The step of measuring the volume of the organ by using the estimated width of the organ region and the photographing metadata; The step of measuring the uncertainty value of the neural network model and the uncertainty values ​​of multiple images based on the estimated results of the neural network model; The step of changing at least one of the plurality of images based on the uncertainty values ​​of the plurality of images; The step of changing at least one of the multiple images comprises: A step of detecting one or more images of the plurality of images whose uncertainty values ​​are above a reference value; the step of altering the detected image based on user input regarding the organ region of the detected image, and The step of changing the calibration policy of the neural network model based on the uncertainty value of the neural network model, and The step of setting weighted values ​​of multiple images according to the changed calibration policy, assigning a larger weighted value to the changed image than the image before the change, and making the neural network model learn.

2. The organ volume measurement method according to claim 1, wherein: taking a plurality of images of the organ including a CT image obtained from a digital imaging and communications in medicine file and a calibration image of the organ, The photographing metadata includes pixel interval data about each of the plurality of images and depth data of the image.

3. The organ volume measurement method according to claim 1, wherein: The step of obtaining the plurality of image blocks comprises: performing data augmentation on a first image included in the plurality of images, generating a plurality of images from the first image, and preprocessing the generated plurality of images to obtain a plurality of image blocks; and The data augmentation includes one or more of space enlargement, color enhancement, noise enhancement or cropping of the image.

4. The organ volume measurement method according to claim 1, wherein: The multiple images are multiple 3D images of the organ. The step of obtaining the plurality of image blocks comprises: The step of sliding the plurality of 3D images in a depth direction to obtain the plurality of image blocks of a specified size.

5. The organ volume measurement method according to claim 1, wherein: The neural network model performs random dropout in the learning step and the derivation step, The uncertainty value of the neural network model is measured based on the dispersion value of the probability distribution of the result data in the derivation step of the neural network model.

6. The organ volume measurement method according to claim 5, wherein: The uncertainty values ​​of the plurality of images are measured based on estimated values ​​of dispersion of the resulting data in the derivation step of the neural network model.

7. An organ volume measurement device, comprising: Memory, which stores instructions; as well as One or more processors configured to execute the instructions to perform the method of claim 1.

8. The organ volume measurement device according to claim 7, wherein: taking a plurality of images of the organ including a CT image obtained from a digital imaging and communications in medicine file and a calibration image of the organ, The photographing metadata includes pixel interval data about each of the plurality of images and depth data of the image.

9. The organ volume measurement device according to claim 7, wherein: The processor is further configured to: performing data augmentation on a first image included in the plurality of images, generating a plurality of images from the first image, and preprocessing the generated plurality of images to obtain a plurality of image blocks; and The data augmentation includes one or more of space enlargement, color enhancement, noise enhancement or cropping of the image.

10. The organ volume measurement device according to claim 7, wherein: The multiple images are multiple 3D images of the organ, and The processor is further configured to: slide the multiple 3D images in a depth direction to obtain the multiple image blocks of a specified size.

11. The organ volume measurement device according to claim 7, wherein: The neural network model performs random dropout in the learning step and the derivation step, The uncertainty value of the neural network model is configured to be measured based on a dispersion value of a probability distribution of result data in a derivation step of the neural network model.

12. The organ volume measurement device according to claim 11, wherein: The uncertainty values ​​of the plurality of images are configured to be measured based on a dispersion estimate of the resulting data in the derivation step of the neural network model.

Images