Electronic device including artificial intelligence model for detecting free air on basis of abdominal CT image, and training method thereof

Abstract

According to the present invention, the electronic device including an artificial intelligence (AI) model for detecting free air on the basis of an abdominal CT image may include: at least one processor; and a memory for storing a computer program executed by the at least one processor, wherein the at least one processor is configured to: acquire a CT image of the abdomen for each patient; perform preprocessing on the acquired CT image to identify a free air area; train an artificial intelligence model by using the preprocessed CT image; and predict the presence / absence and area of perforation in the abdomen on the basis of an input CT image by using the trained artificial intelligence model.

Description

Electronic device including an artificial intelligence model for detecting free air based on abdominal CT images and a learning method thereof

[0001] The present invention relates to an electronic device including an artificial intelligence model and a learning method thereof, and more particularly, to an electronic device including an artificial intelligence model for detecting free air based on abdominal CT images and a learning method thereof.

[0002] Free air (also known as glass air shading) refers to areas of increased gray density or black spots other than the hazy gray areas that appear on CT scans or X-rays, and specifically refers to the presence of air in areas where air should not be present, outside of long-term areas, resulting in black spots.

[0003] Among patients presenting to the emergency room (ED) who are unconscious or complaining of abdominal pain, abdominal CT scans are often performed, but free air is often not detected. However, in patients with peritonitis or intestinal perforation, where free air is detected in less than 5% of the CT images, urgent emergency surgery is necessary. Therefore, even if free air is detected infrequently, all patients should be screened for free air on CT images.

[0004] However, determining the presence or absence of free air in a CT image may vary depending on the skill of the medical staff, and in particular, in emergency rooms, accurate free air detection is physically difficult due to the large number of patients or the need for CT image analysis at night.

[0005] Additionally, in the case of abdominal CT images, it is difficult to distinguish and detect air within the organs and free air outside the organ area due to the presence of the small or large intestine.

[0006] In the existing patent literature, a learning system for medical image processing using CNN was disclosed, but it was not possible to perform free air detection by classifying free air outside the long-term region and other air using the existing patent literature alone.

[0007] Therefore, there was a need to quickly detect free air in abdominal CT images to provide an alarm or guidance to medical staff.

[0008] In order to solve the above-described problem, the present invention provides an electronic device including an artificial intelligence model for detecting free air based on an abdominal CT image, which detects free air located outside an organ region by annotating an abdominal CT image and using a U-NET-based artificial intelligence model, and a learning method thereof.

[0009] The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0010] An electronic device including an artificial intelligence model for detecting free air based on an abdominal CT image according to the present invention for solving the above-described problem comprises at least one processor; and a memory storing a computer program executed by the at least one processor, wherein the at least one processor is configured to acquire a CT image for the abdomen of each patient, perform preprocessing to identify a free air area for the acquired CT image, train an artificial intelligence model using the preprocessed CT image, and predict the presence or absence of a perforation and the area of ​​the perforation in the abdomen for an input CT image using the trained artificial intelligence model.

[0011] A learning method of an electronic device including an artificial intelligence model for detecting free air based on abdominal CT images according to the present invention for solving the above-described problem may include the steps of: acquiring a CT image for the abdomen of each patient; performing preprocessing to identify a free air region for the acquired CT image; training an artificial intelligence model using the preprocessed CT image; and predicting the presence or absence of perforation and the region of perforation in the abdomen for an input CT image using the trained artificial intelligence model.

[0012] In addition, other methods for implementing the present invention, other systems, and computer-readable recording media recording a computer program for executing the above methods may be further provided.

[0013] In addition, a computer program may be further provided that is stored in a medium so that a method for implementing the present invention is performed on a computer.

[0014] According to the above-described problem solving means of the present invention, preprocessing is performed to facilitate free air detection in abdominal CT images, and U-NET-based image segmentation is performed to improve the sensitivity and specificity of free air detection.

[0015] In addition, according to the above-described problem solving means of the present invention, even if the emergency room is crowded with patients or there is a shortage of skilled medical staff, it is possible to provide an auxiliary tool that does not miss a patient with intestinal perforation by providing a professional and specific alarm or guide to the doctor.

[0016] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description below.

[0017] FIG. 1 is a block diagram schematically illustrating the configuration of an electronic device including an artificial intelligence model for detecting free air based on abdominal CT images according to the present invention.

[0018] FIG. 2 is a schematic diagram illustrating the configuration of an artificial intelligence model of an electronic device including an artificial intelligence model for detecting free air based on abdominal CT images according to the present invention.

[0019] FIG. 3 is a schematic diagram illustrating a segmentation process of an electronic device including an artificial intelligence model for detecting free air based on abdominal CT images according to the present invention.

[0020] FIG. 4 is a schematic diagram illustrating an annotation process of an electronic device including an artificial intelligence model for detecting free air based on abdominal CT images according to the present invention.

[0021] FIG. 5 illustrates images before and after filtering preprocessing of an electronic device including an artificial intelligence model for detecting free air based on abdominal CT images according to the present invention.

[0022] FIG. 6 illustrates a process for calculating sensitivity and specificity of an electronic device including an artificial intelligence model for detecting free air based on abdominal CT images according to the present invention.

[0023] FIG. 7 is a flowchart illustrating a learning method of an electronic device including an artificial intelligence model for detecting free air based on abdominal CT images according to the present invention.

[0024] FIG. 8 and FIG. 9 illustrate a process of providing a guide for an electronic device according to one embodiment of the present invention.

[0025] FIG. 10 illustrates a process for determining whether an electronic device is free air according to one embodiment of the present invention.

[0026] FIGS. 11 to 13 illustrate a process of detecting free air by extracting multiple key images from a segmented image by an electronic device according to one embodiment of the present invention.

[0027] FIGS. 14 to 21 illustrate the results and accuracy of the processes performed by an electronic device according to an embodiment of the present invention in FIGS. 11 to 13.

[0028] Throughout the present invention, the same reference numerals denote the same components. The present invention does not describe all elements of the embodiments, and any content that is general in the technical field to which the present invention pertains or that overlaps between the embodiments is omitted. The terms 'part, module, element, block' used in the specification may be implemented in software or hardware, and depending on the embodiments, multiple 'parts, modules, elements, blocks' may be implemented as a single component, or a single 'part, module, element, block' may include multiple components.

[0029] Throughout the specification, when a part is said to be "connected" to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection via a wireless communication network.

[0030] Additionally, when a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0031] Throughout the specification, when we say that an element is "on" another element, this includes not only cases where the element is in contact with the other element, but also cases where another element exists between the two elements.

[0032] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0033] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0034] The identification codes for each step are used for convenience of explanation and do not describe the order of each step. Each step may be performed in a different order than specified unless the context clearly indicates a specific order.

[0035] The operating principle and embodiments of the present invention will be described with reference to the attached drawings below.

[0036] As used herein, the term "device according to the present invention" encompasses a variety of devices capable of performing computational processing and providing results to a user. For example, the device according to the present invention may include a computer, a server device, and a portable terminal, or may be any one of them.

[0037] Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

[0038] The above server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, and a web server.

[0039] The above portable terminal may include, for example, a wireless communication device that ensures portability and mobility, and may include all kinds of handheld-based wireless communication devices such as a PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminal, a smart phone, and a wearable device such as a watch, a ring, a bracelet, an anklet, a necklace, glasses, contact lenses, or a head-mounted device (HMD).

[0040] The artificial intelligence-related functions according to the present invention are operated through a processor and memory. The processor may be composed of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, AP, or DSP (Digital Signal Processor), a graphics-only processor such as a GPU or VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU. One or more processors control the processing of input data according to predefined operating rules or artificial intelligence models stored in memory. Alternatively, if one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0041] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the basic artificial intelligence model is trained using a learning algorithm using a plurality of learning data, thereby creating a predefined operation rules or artificial intelligence model set to perform a desired characteristic (or purpose). This learning may be performed on the device itself on which the artificial intelligence according to the present invention is performed, or may be performed through a separate server and / or system. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0042] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values, and performs neural network operations through operations between the operation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), and examples thereof include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or deep Q-networks.

[0043] According to an exemplary embodiment of the present invention, a processor can implement artificial intelligence. Artificial intelligence refers to a machine learning method based on an artificial neural network that imitates human neurons (biological neurons) to enable machines to learn. Artificial intelligence methodologies can be categorized into supervised learning, in which input data and output data are provided together as training data depending on the learning method, so that the solution (output data) to the problem (input data) is determined; unsupervised learning, in which only input data is provided without output data, so that the solution (output data) to the problem (input data) is not determined; and reinforcement learning, in which a reward (Reward) is provided from an external environment whenever an action (Action) is taken in the current state (State), and learning is performed in a direction to maximize this reward. In addition, artificial intelligence methodologies can be categorized according to the architecture of the learning model. The architectures of widely used deep learning technologies can be categorized into convolutional neural networks (CNNs), recurrent neural networks (RNNs), transformers, and generative adversarial networks (GANs).

[0044] The present device and system may include an artificial intelligence model. The artificial intelligence model may be a single artificial intelligence model or may be implemented as multiple artificial intelligence models. The artificial intelligence model may be composed of a neural network (or artificial neural network) and may include statistical learning algorithms that mimic biological neurons in machine learning and cognitive science. A neural network may refer to a model in general that has problem-solving capabilities by changing the binding strength of synapses through learning, formed by artificial neurons (nodes) that form a network by combining synapses. The neurons of the neural network may include a combination of weights or biases. The neural network may include one or more layers composed of one or more neurons or nodes. For example, the device may include an input layer, a hidden layer, and an output layer. The neural network constituting the device can infer a desired result (output) from an arbitrary input (input) by changing the weights of neurons through learning.

[0045] The processor can create a neural network, train (or learn) a neural network, perform a calculation based on received input data, generate an information signal based on the calculation result, or retrain the neural network. The models of the neural network can include various types of models such as CNN (Convolution Neural Network) such as GoogleNet, AlexNet, VGG Network, R-CNN (Region with Convolution Neural Network), RPN (Region Proposal Network), RNN (Recurrent Neural Network), S-DNN (Stacking-based deep Neural Network), S-SDNN (State-Space Dynamic Neural Network), Deconvolution Network, DBN (Deep Belief Network), RBM (Restrcted Boltzman Machine), Fully Convolutional Network, LSTM (Long Short-Term Memory) Network, Classification Network, etc., but are not limited thereto. The processor can include one or more processors for performing calculations according to the models of the neural network. For example, the neural network can be a deep neural network. It may include a deep neural network.

[0046] Neural networks include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto) Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), Generative Adversarial Network (GAN), Liquid State Machine (LSM), Extreme Learning Machine (ELM), It will be understood by those skilled in the art that any neural network may be included, including but not limited to ESN (Echo State Network), DRN (Deep Residual Network), DNC (Differentiable Neural Computer), NTM (Neural Turning Machine), CN (Capsule Network), KN (Kohonen Network), and AN (Attention Network).

[0047] According to an exemplary embodiment of the present invention, the processor may be configured to perform a CNN (Convolution Neural Network) such as GoogleNet, AlexNet, VGG Network, etc., R-CNN (Region with Convolution Neural Network), RPN (Region Proposal Network), RNN (Recurrent Neural Network), S-DNN (Stacking-based deep Neural Network), S-SDNN (State-Space Dynamic Neural Network), Deconvolution Network, DBN (Deep Belief Network), RBM (Restrcted Boltzman Machine), Fully Convolutional Network, LSTM (Long Short-Term Memory) Network, Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT, SP-BERT, MRC / QA for natural language processing, Text Analysis, Dialog System, GPT-3, GPT-4, Visual Analytics for vision processing, Visual Understanding, Video Synthesis, ResNet Anomaly Detection, Prediction, Time-Series Forecasting, Optimization for data intelligence, Various artificial intelligence structures and algorithms, including Recommendation and Data Creation, can be utilized, but are not limited thereto. Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0048] FIG. 1 is a block diagram schematically illustrating the configuration of an electronic device including an artificial intelligence model for detecting free air based on abdominal CT images according to the present invention. Hereinafter, the electronic device and its learning method according to the present invention will be described with reference to FIGS. 2 to 21.

[0049] The electronic device (100) according to the present invention may include a processor (110), a memory (120), a communication unit (130), and an input / output interface (140). The internal components that the electronic device (100) may include are not limited thereto. The electronic device (1000) of the present invention may perform the functions of the processor (110) through a separate processing server or cloud server instead of the processor (110).

[0050] Referring to FIG. 1, a processor (110) may be implemented to perform operations of the electronic device (100) using a memory (120) that stores data regarding an algorithm for controlling operations of components within the electronic device (100) or a program that reproduces the algorithm, and the data stored in the memory (120). In this case, the processor (110) and the memory (120) may each be implemented as separate chips. Alternatively, the processor (110) and the memory (120) may be implemented as a single chip.

[0051] The memory (120) according to the embodiment can store data supporting various functions of the electronic device (100), a program for the operation of the processor (110), can store input / output data (e.g., images, videos, etc.), and can store a plurality of application programs (or applications) run on the electronic device (100), data for the operation of the electronic device (100), and commands. At least some of these application programs can be downloaded from an external server via wireless communication.

[0052] The memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, an SSD (Solid State Disk type), an SDD (Silicon Disk Drive type), a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk. In addition, the memory may be a database that is separate from the electronic device (100) but is connected by wire or wirelessly.

[0053] The communication unit (130) according to the embodiment may include one or more components that enable communication with an external device, and may include, for example, at least one of a broadcast reception module, a wired communication module, a wireless communication module, a short-range communication module, and a location information module.

[0054] The wired communication module may include various wired communication modules such as a Local Area Network (LAN) module, a Wide Area Network (WAN) module, or a Value Added Network (VAN) module, as well as various cable communication modules such as a Universal Serial Bus (USB), a High Definition Multimedia Interface (HDMI), a Digital Visual Interface (DVI), RS-232 (recommended standard 232), power line communication, or plain old telephone service (POTS).

[0055] The wireless communication module may include a wireless communication module that supports various wireless communication methods such as GSM (global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), UMTS (universal mobile telecommunications system), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, and 6G, in addition to a WiFi module and a Wireless Broadband module.

[0056] The short-range communication module is for short-range communication, and can support short-range communication using at least one of Bluetooth™, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0057] The input / output interface (140) according to the embodiment serves as a passageway for various types of external devices connected to the electronic device (100) of the present invention. This input / output interface (140) may include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module (SIM), an audio I / O (Input / Output) port, a video I / O (Input / Output) port, and an earphone port. The electronic device (100) of the present invention may perform appropriate control related to an external device connected to the input / output interface (140).

[0058] Each component illustrated in Figure 1 represents software and / or hardware components such as a Field Programmable Gate Array (FPGA) and an Application Specific Integrated Circuit (ASIC).

[0059] Accordingly, an electronic device (100) including an artificial intelligence model for detecting free air based on abdominal CT images according to one embodiment of the present invention may include at least one processor (110) and a memory (120) storing a computer program executed by the at least one processor (110).

[0060] The at least one processor (110) may be configured to acquire a CT image for the abdomen of each patient, perform preprocessing to identify a free air area for the acquired CT image, train an artificial intelligence model using the preprocessed CT image, and predict the presence or absence of perforation and the area of ​​perforation in the abdomen for the input CT image using the trained artificial intelligence model.

[0061] Referring to FIG. 2, the artificial intelligence model according to the present invention may include an encoder and a decoder (200). Each layer of the encoder and decoder (200) may be composed of a residual convolution block (220).

[0062] The residual conv block (220) is a block that additionally learns small pieces of information of each block by placing a skip connection in the convolution block (210) composed of a 3x3 convolution layer, a batch normalization layer, and a ReLU layer.

[0063] In other words, the Residual conv block (220) is a CNN (Convolution Neural Network) block equipped with multiple skip connections.

[0064] An abdominal CT image (10) is input to an encoder and decoder (200) composed of a residual convolution block (220), and can be input to a U-NET based segmentation model (300) trained to detect free air in the output CT image (11). The U-NET based segmentation model (300) trained to detect free air in the output CT image (11) can be referred to as FA-NET.

[0065] Referring to FIG. 2, the U-NET-based segmentation model (300) can compare the free air area detected in the output CT image (11) with the ground truth (12) in which the doctor detected the free air, and calculate the degree of overlap between the two images as a Dice score (Sørensen-Dice coefficient score).

[0066] If it matches perfectly, the dice score will be 1, and if it doesn't match at all, the dice score can be 0.

[0067] At this time, the output CT image (11) can be obtained by dividing it with a 3D slicer (230), as shown in FIG. 3, and the CT image (110) can include at least three key images. The drawing number in FIG. 3 shows the image area divided by the 3D slicer (230).

[0068] For example, three to five key images can be extracted from the segmented CT images of each patient. If the artificial intelligence model determines that two or more of the three to five extracted images contain free air, it can be determined that free air exists, and if the artificial intelligence model determines that two or fewer than three of the three to five extracted images contain free air, it can be determined that free air does not exist.

[0069] As an example, the processor (110) of the electronic device (100) may annotate an area determined based on a preset long-term area and the location of the spot as free air.

[0070] For example, referring to FIG. 4, the processor (110) of the electronic device (100) can perform preprocessing by annotating that a spot (32) located within a predetermined long-term area in each of at least three key images is not free air, and annotating that a spot (31) located outside the predetermined long-term area is free air.

[0071] Accordingly, an abdominal CT image (10) can be input into an encoder and decoder (200), and the output CT image (110) can be segmented to extract at least three key images. The U-NET-based segmentation model (300) can detect a free air region in the key image and perform annotation indicating free air in the detected region.

[0072] At this time, as illustrated in FIG. 5, at least one processor (110) of the electronic device (100) according to the present invention can perform windowing on the acquired CT image at a preset threshold value. Through windowing, the CT image can be preprocessed to make it clearer.

[0073] As an example, the preset threshold may be greater than 1200. This can further improve the accuracy of the artificial intelligence model.

[0074] Meanwhile, at least one processor (110) of the electronic device (100) according to the present invention can measure the accuracy of the learned artificial intelligence model based on the Dice score. The Dice score is a value obtained by multiplying the product of the ground truth (12) and the predicted mask by the sum of the output CT image (11) in which the ground truth (12) and the predicted mask are displayed. The higher the Dice score, the higher the accuracy of the artificial intelligence model can be evaluated.

[0075] Also, referring to FIG. 6, abdominal CT images from Test 1 to Test 12 can be acquired for each patient, and at this time, patients can be distinguished into patients with actual intestinal perforation and patients without actual intestinal perforation.

[0076] The ratio of the number of images with free air detected in the output CT images to the number of patients with actual intestinal perforation is called sensitivity, and the ratio of the number of images without free air to the number of patients without actual intestinal perforation is called specificity.

[0077] To assess the accuracy of a trained AI model, sensitivity and specificity can be distinguished. For patients with intestinal perforation, the accuracy of sensitivity and specificity can be assessed by distinguishing between true positives (free air detected) and false negatives (no free air detected), and for patients without intestinal perforation, true negatives (no free air detected) and false positives (free air detected).

[0078] Specifically, as illustrated in FIG. 7, a learning method of an electronic device (100) including an artificial intelligence model for detecting free air based on an abdominal CT image according to the present invention may include a step of obtaining a CT image for the abdomen of each patient (S710), a step of performing preprocessing to check a free air area for the obtained CT image (S720), a step of training an artificial intelligence model using the preprocessed CT image (S730), and a step of predicting the presence or absence of perforation and the area of ​​perforation in the abdomen for the input CT image using the trained artificial intelligence model (S740).

[0079] The step (S740) of predicting the presence or absence and area of ​​intra-abdominal perforation according to the present invention can detect the presence or absence and size of free air in a CT image and output the presence or absence, area, or size of intra-abdominal perforation of a patient.

[0080] As illustrated in FIG. 8, as an embodiment, in the step (S740) of predicting the presence or absence and area of ​​intra-abdominal perforation, if there is intra-abdominal perforation in the patient, a step (S810) of providing a corresponding treatment or emergency surgery guide to a user interface based on the size of the perforation may be further included.

[0081] At this time, the user interface may be the input / output interface (140) described above.

[0082] Alternatively, as illustrated in FIG. 9, as another embodiment, the step (S740) of predicting the presence or absence and area of ​​intra-abdominal perforation may further include, if there is intra-abdominal perforation in the patient, a step (S811) of collecting patient information including a pre-input patient age and the size of the perforation, and a step (S812) of providing a corresponding treatment or emergency surgery guide to a user interface based on the patient information and the size of the perforation.

[0083] Therefore, the artificial intelligence model according to the present invention can not only detect free air using the U-NET-based segmentation model (300), but also assist medical staff in their treatment by providing a corresponding treatment or emergency surgery guide based on the presence and size of free air.

[0084] For example, if no intra-abdominal perforation exists, the medical staff may be guided that no further action is necessary.

[0085] Additionally, if an intra-abdominal perforation exists and the size of the perforation exceeds a threshold, it can guide medical staff to perform emergency surgery, and if it is below the threshold, it can guide that surgical action may be necessary.

[0086] At this time, if the key image is in the upper abdomen, the presence or absence of intra-abdominal perforation can immediately guide medical staff to take appropriate action. Because the upper abdomen lacks the small or large intestine, the likelihood of intra-abdominal air being present is low. Therefore, the possibility of intra-abdominal air being mistaken for free air is low, and thus, even if free air is not detected, the result can be trusted.

[0087] Conversely, if the key image is of the lower abdomen, additional imaging may be suggested even if no perforation is detected in the lower abdomen. This is because it is difficult to distinguish between intra-organ air and extra-organ free air in the lower abdomen, and it is safer to conduct additional verification if the key image corresponds to the lower abdomen.

[0088] Accordingly, the electronic device (100) according to the present invention can separately input the name of an organ for detecting the presence or absence of perforation from the user interface, and when the name of an organ included in a preset lower abdomen is input from the user interface, an additional inspection progress guide can be provided to the user interface.

[0089] Meanwhile, the step (S740) of predicting the presence or absence and area of ​​intra-abdominal perforation according to the present invention may include a step of extracting at least three key images from the segmented CT images of each patient, a step of determining that free air exists if the artificial intelligence model determines that free air exists in two or more of the three extracted images, and a step of determining that free air does not exist if the artificial intelligence model determines that free air exists in less than two of the three extracted images.

[0090] As an example, three to five key images may be extracted from among the segmented CT images, and if it is determined that there is free air in two or more than three of the extracted images, it may be finally determined that there is free air, and if it is determined that there is free air in two or less than three images, it may be finally determined that there is no free air.

[0091] Specifically, referring to FIG. 11, abdominal CT images of five patients and each patient can be acquired from a group of patients with duodenal perforation (1111) and a group of patients without duodenal perforation (1112). In this case, duodenal perforation is merely an exemplary embodiment and all other abdominal perforations can be included.

[0092] By segmenting the free air region from the abdominal CT image of each patient, at least three representative key images can be extracted. Through this, patients with duodenal perforation (1111) can each include key image 1, key image 2, and key image 3, and patients without duodenal perforation (1112) can each include key image 1, key image 2, and key image 3.

[0093] The electronic device (100) according to the present invention can perform free air detection for each key image of each patient.

[0094] Referring to FIG. 12, the electronic device (100) according to the present invention can perform free air detection on each key image of duodenal perforation patients (1111), Pt1, Pt2, Pt3, Pt4, and Pt5. In this case, it can be confirmed that all patients with duodenal perforation have free air, which is a True Positive (TP). In this case, the sensitivity is 100%.

[0095] Referring to FIG. 13, the electronic device (100) according to the present invention can perform free air detection for each key image of Pt1, Pt2, Pt3, Pt4, and Pt5, which are patients (1112) who do not have duodenal perforation. In this case, it can be confirmed that Pt3, Pt4, and Pt5 produce True Negative (TN) results indicating that there is no free air in key image 1, key image 2, and key image 3. However, it can be confirmed that Pt1 produces False Positive results indicating that free air is detected in key image 3, and Pt2 produces False Positive results indicating that free air is detected in all key images 1, key image 2, and key image 3.

[0096] As an example, the electronic device (100) according to the present invention can predict that no free air is detected by determining that the key images are TN not only when all of the key images are TN, but also when two out of three key images are TN.

[0097] This allows Pt1 to have a free air non-detection result that matches the ground truth, while maintaining the specificity at 80% by only obtaining a mismatch result for Pt2.

[0098] Accordingly, the electronic device (100) according to the present invention maintains high sensitivity, but may have low accuracy in terms of specificity. To compensate for this, the final free air detection can be determined based on the results of a majority of key images among multiple key images.

[0099] For example, if TP appears in 2 out of 3 key images, it can be determined that free air has been detected, and if TN appears in 2 out of 3 key images, it can be determined that free air has not been detected.

[0100] Figures 14 to 21 show actual experimental results for the above-described contents.

[0101] Figure 14 shows a true positive image, which is, in order, an abdominal CT image, a free air area detected by FA-NET, which is a U-NET-based segmentation model (300), and a free air area directly diagnosed by a doctor (Ground truth).

[0102] As shown in Fig. 14, the FA-NET Detected FA area and the Ground truth are almost similar, and the Dice score is 0.93, so it can be confirmed that TP is output and the free air detection sensitivity is high.

[0103] Figures 15 and 16 are true positive images with high Dice scores of 0.83 and 0.9, respectively, because the FA-NET Detected FA area and the ground truth are almost similar, while Figure 17 is a false negative image with low Dice scores of 0.57, because the FA-NET Detected FA area and the ground truth are different.

[0104] As a result of an experiment using an electronic device (100) according to the present invention, 439 out of 488 abdominal CT images of patients with duodenal perforation were TP images with high Dice scores as shown in FIGS. 15 and 16, and the rest were FN images with low Dice scores due to errors as shown in FIG. 17.

[0105] Accordingly, it can be seen that the electronic device (100) according to the present invention has a high accuracy with a sensitivity of about 0.9.

[0106] Meanwhile, Fig. 18 shows true negative images for two abdominal CT images, in order from left to right: an abdominal CT image, a free air area detected by FA-NET, a U-NET-based segmentation model (300), and a free air area directly diagnosed by a doctor (Ground truth). This is a patient without free air, and is output with a black background without a separate white area.

[0107] As shown in Fig. 18, it can be confirmed that the FA-NET Detected FA area and the ground truth are the same for the two abdominal CT images, so the Dice score is 1.0 and TN is output for both.

[0108] However, Figs. 19 and 20 are false positive images with a very low Dice score of 0.03 due to differences between the FA-NET Detected FA area and the ground truth. Of the 1,110 abdominal CT images of patients without duodenal perforation, 644 were TN images with high Dice scores, as shown in Fig. 18, and the rest were FP images with low Dice scores due to errors, as shown in Figs. 19 to 10.

[0109] Accordingly, it can be confirmed that the electronic device (100) according to the present invention has a specificity of about 0.58, which is somewhat lower than the sensitivity.

[0110] To compensate for this, the electronic device (100) according to the present invention can divide an abdominal CT image to extract at least three key images and determine the result of outputting a majority of the three key images as the final free air detection result.

[0111] In addition, as shown in Fig. 21, very fine air may be air or noise that exists naturally due to long-term activity rather than through perforation, so if a free air area smaller than a preset area is detected in the area of ​​each image, it can be ignored.

[0112] For example, the electronic device (100) according to the present invention can remove a free air area detected in an area of ​​less than 1% of the total image area and determine the presence or absence of free air only based on a free air area detected in an area exceeding 1% of the total image area.

[0113] This compensates for the accuracy of specificity, which is lower than sensitivity, and allows for high accuracy to be maintained regardless of the presence or absence of free air.

[0114] Therefore, the artificial intelligence model can detect the presence or absence and size of free air in the input CT image and output the presence or area and size of perforation in the patient's abdomen.

[0115] The electronic device (100) according to the present invention, when there is a perforation in the abdomen of the patient, may provide a corresponding treatment or emergency surgery guide to the user interface based on the size of the perforation, or may collect patient information including a pre-input patient age and the size of the perforation, and provide a corresponding treatment or emergency surgery guide to the user interface based on the patient information and the size of the perforation.

[0116] In addition, the electronic device (100) according to the present invention can extract at least three key images from among the segmented CT images of each patient, and if the artificial intelligence model determines that there is free air in two or more of the three extracted images, it can determine that there is free air, and if the artificial intelligence model determines that there is free air in less than two of the three extracted images, it can determine that there is no free air.

[0117] The electronic device (100) according to the present invention further includes a user interface for receiving an organ name to detect the presence or absence of perforation, and when an organ name included in a preset lower abdomen is input in the user interface, an additional inspection progress guide can be provided to the user interface.

[0118] The above-described configuration allows for accurate classification and treatment of critically ill patients with abdominal perforation according to their level of urgency without spending a significant amount of time on patients with abdominal perforation, which account for less than 5% of all emergency room visitors, thereby addressing the shortage of medical staff.

[0119] The disclosed embodiments have been described with reference to the attached drawings as described above. Those skilled in the art will understand that the present disclosure can be implemented in forms other than the disclosed embodiments without altering the technical spirit or essential features of the present invention. The disclosed embodiments are illustrative and should not be construed as limiting.

Claims

1. At least one processor; and A memory storing a computer program executed by at least one processor; At least one processor, Obtain CT images of the abdomen for each patient, Preprocessing is performed to identify the free air area for the acquired CT image, An artificial intelligence model is trained using the above preprocessed CT images, A method configured to predict the presence and area of ​​intra-abdominal perforation for an input CT image using the learned artificial intelligence model, An electronic device comprising an artificial intelligence model for detecting free air based on abdominal CT images.

2. In paragraph 1, At least one processor, It is configured to perform preprocessing by annotating the area determined based on the location of the predetermined organ area and spot in the acquired CT image as free air. An electronic device comprising an artificial intelligence model for detecting free air based on abdominal CT images.

3. In paragraph 2, At least one processor, Windowing is performed at a preset threshold for the acquired CT image, Further configured to segment the acquired CT image using a 3D slicer, An electronic device comprising an artificial intelligence model for detecting free air based on abdominal CT images.

4. In paragraph 3, The above artificial intelligence model, A U-NET based segmentation model trained to detect free air within the preprocessed CT image; and An encoder and a decoder comprising a CNN (Convolution Neural Network) block having multiple skip connections, An electronic device comprising an artificial intelligence model for detecting free air based on abdominal CT images.

5. In paragraph 4, The above artificial intelligence model detects the presence and size of free air in the input CT image and outputs the presence and size of perforation in the patient's abdomen. An electronic device comprising an artificial intelligence model for detecting free air based on abdominal CT images.

6. In paragraph 5, At least one processor, If the patient has an intra-abdominal perforation, the user interface is configured to provide a corresponding treatment or emergency surgical guide based on the size of the perforation. An electronic device comprising an artificial intelligence model for detecting free air based on abdominal CT images.

7. In paragraph 5, At least one processor, If the patient has an intra-abdominal perforation, collect patient information including the patient's age entered in advance and the size of the perforation. configured to provide a corresponding treatment or emergency surgery guide to the user interface based on the above patient information and the size of the perforation, An electronic device comprising an artificial intelligence model for detecting free air based on abdominal CT images.

8. In paragraph 3, At least one processor, Extract at least three key images from the segmented CT images of each patient, If the above artificial intelligence model determines that there is free air in two or more of the three extracted images, it determines that there is free air. If the above artificial intelligence model determines that there is free air in less than two of the three extracted images, it determines that there is no free air. An electronic device comprising an artificial intelligence model for detecting free air based on abdominal CT images.

9. In paragraph 1, At least one processor, Configured to measure the accuracy of an artificial intelligence model learned based on the dice score, An electronic device comprising an artificial intelligence model for detecting free air based on abdominal CT images.

10. In paragraph 1, It further includes a user interface that receives an input of an organ name to detect the presence or absence of a perforation, At least one processor, When the name of an organ included in the preset lower abdomen in the above user interface is entered, it is configured to provide an additional examination progress guide to the user interface. An electronic device comprising an artificial intelligence model for detecting free air based on abdominal CT images.

11. A learning method of an electronic device including an artificial intelligence model for detecting free air based on abdominal CT images, Step of acquiring CT images of the abdomen for each patient; A step of performing preprocessing to identify a free air area for the acquired CT image; A step of training an artificial intelligence model using preprocessed CT images; and A step of predicting the presence and area of ​​intra-abdominal perforation for an input CT image using the learned artificial intelligence model; including; A learning method for an electronic device including an artificial intelligence model.

12. In paragraph 11, The step of performing the above preprocessing is: A step of annotating an area determined based on the location of a predetermined organ area and spot in the acquired CT image as free air, A learning method for an electronic device including an artificial intelligence model.

13. In paragraph 12, The step of performing the above preprocessing is: A step of performing windowing at a preset threshold value for the acquired CT image; and Further comprising a step of dividing the acquired CT image using a 3D slicer, A learning method for an electronic device including an artificial intelligence model.

14. In paragraph 13, The step of training an artificial intelligence model using the above preprocessed CT image is as follows: A step of detecting free air in the preprocessed CT image using the U-NET based segmentation model of the artificial intelligence model; and A step of performing encoding and decoding using a CNN (Convolution Neural Network) block having multiple skip connections of the above artificial intelligence model, A learning method for an electronic device including an artificial intelligence model.

15. In paragraph 14, The step of predicting the presence or absence of intra-abdominal perforation is as follows: Detecting the presence and size of free air in a CT image to output the presence and size of perforation in the patient's abdomen. A learning method for an electronic device including an artificial intelligence model.

Images