Electronic device for determining risk of nodule on basis of plurality of input image sets, operation method thereof, and recording medium for performing same method

Abstract

An electronic device according to various embodiments may: obtain a first input image set obtained by capturing an image of at least a part of a user's body at a first time point; detect at least one first nodule with respect to the first input image set; determine similarity between the at least one first nodule and at least one second nodule, wherein the at least one second nodule is obtained on the basis of a second input image set stored in a memory; and obtain risk information of the at least one first nodule by inputting the first input image set and the second input image set to a risk determination model when the similarity has a value equal to or greater than a pre-configured value, wherein the risk determination model is an artificial intelligence model trained to output risk information of the at least one first nodule on the basis of first size information of the at least one first nodule, first type information of the at least one first nodule, and variation information of the at least one first nodule and the at least one second nodule.

Description

Electronic device for determining the risk of a nodule based on a plurality of input image sets, method of operation thereof, and recording medium for performing the method

[0001] Various embodiments disclosed in this document relate to an electronic device for determining the risk of a nodule based on a plurality of input image sets, a method of operating the same, and a recording medium for performing the method.

[0002] Recently, medical image analysis using computed tomography (CT) images has played an essential role in the early diagnosis and treatment planning of diseases. In particular, techniques for detecting abnormal structures, such as nodules, in CT images are crucial for the early detection of diseases such as lung cancer. However, due to the complexity of CT images and the diverse shapes and sizes of nodules, accurately detecting and classifying nodules to assess their risk level presents a challenging challenge.

[0003] Traditional nodule detection techniques primarily rely on manual or semi-automated analysis. These methods are time-consuming and subject to expert interpretation, which can lead to inconsistent results. Furthermore, small nodules in the early stages or nodules of uncommon shapes can be missed, potentially reducing diagnostic accuracy.

[0004] In addition, traditional risk assessment methods for nodules are mainly based on fragmentary information such as the size or type of the nodules, and these methods have problems in accurately assessing the risk of detected nodules.

[0005] To address these issues, AI-based approaches are being studied. In particular, deep learning technology demonstrates outstanding performance in image recognition and analysis and is expected to have significant potential for automatically detecting and classifying nodules in CT images.

[0006] The above information may be provided as background information to aid in understanding the present disclosure. No claim or determination is made as to whether any of the above is applicable as prior art in connection with the present disclosure.

[0007] Various embodiments disclosed in this document aim to provide an electronic device that detects at least one nodule contained in at least a part of a user's body using artificial intelligence.

[0008] The various embodiments disclosed in this document aim to provide an electronic device that accurately determines the risk level of at least one nodule contained in at least a part of a user's body using artificial intelligence.

[0009] Various embodiments disclosed in this document aim to provide an electronic device that provides a user with information about the risk of at least one nodule contained in at least a part of the user's body using artificial intelligence.

[0010] The various embodiments disclosed in this document aim to provide an electronic device that uses artificial intelligence to determine the risk of at least one nodule contained in at least a part of a user's body, thereby increasing the success rate of early detection and treatment of diseases such as lung cancer.

[0011] According to various embodiments, an electronic device includes a communication circuit for transmitting and receiving data with an external device, at least one processor, and a memory for storing commands, wherein the commands are individually or collectively executed by the at least one processor so that the electronic device obtains a first input image set in which at least a part of a user's body is photographed at a first point in time, detects at least one first nodule with respect to the first input image set, determines a similarity between the at least one first nodule and the at least one second nodule, and the at least one second nodule is obtained based on a second input image set stored in the memory, and when the similarity is equal to or greater than a preset value, inputs the first input image set and the second input image set into a risk assessment model to obtain risk information about the at least one first nodule, and the risk assessment model may be an artificial intelligence model trained to output risk information about the at least one first nodule based on first size information about the at least one first nodule, first type information about the at least one first nodule, and variation information about the at least one first nodule and the at least one second nodule.

[0012] According to various embodiments, a method of operating an electronic device may include: acquiring a first input image set in which at least a part of a user's body is captured at a first point in time; detecting at least one first nodule from the first input image set; determining a similarity between the at least one first nodule and the at least one second nodule; wherein the at least one second nodule is acquired based on a second input image set stored in the electronic device, and when the similarity is equal to or greater than a preset value, inputting the first input image set and the second input image set into a risk assessment model to obtain risk information for the at least one first nodule, wherein the risk assessment model may be an artificial intelligence model trained to output risk information for the at least one first nodule based on first size information for the at least one first nodule, first type information for the at least one first nodule, and variation information for the at least one first nodule and the at least one second nodule.

[0013] According to various embodiments, a non-transitory computer-readable recording medium including a program for executing a control method of an electronic device includes the steps of: obtaining a first input image set in which at least a part of a user's body is photographed at a first point in time; detecting at least one first nodule with respect to the first input image set; determining a similarity between the at least one first nodule and the at least one second nodule; obtaining the at least one second nodule based on the second input image set stored in the electronic device; and, if the similarity is equal to or greater than a preset value, inputting the first input image set and the second input image set into a risk assessment model to obtain risk information for the at least one first nodule, wherein the risk assessment model may be an artificial intelligence model trained to output risk information for the at least one first nodule based on first size information of the at least one first nodule, first type information of the at least one first nodule, and variation information of the at least one first nodule and the at least one second nodule.

[0014] An electronic device according to various embodiments disclosed in this document can detect at least one nodule included in at least a part of a user's body using artificial intelligence.

[0015] An electronic device according to various embodiments disclosed in this document can accurately determine the risk level of at least one nodule included in at least a part of a user's body using artificial intelligence.

[0016] An electronic device according to various embodiments disclosed in this document may provide a user with information about the risk of at least one nodule included in at least a part of the user's body using artificial intelligence.

[0017] An electronic device according to various embodiments disclosed in this document can use artificial intelligence to determine the risk of at least one nodule contained in at least a part of a user's body, thereby increasing the success rate of early detection and treatment of diseases such as lung cancer.

[0018] In addition, various effects may be provided, either directly or indirectly, through this document.

[0019] FIG. 1 is a diagram illustrating a system in various embodiments.

[0020] FIG. 2 is a block diagram of an electronic device according to various embodiments.

[0021] FIG. 3 illustrates a concept for controlling a function related to risk assessment for at least one node of an electronic device, according to various embodiments.

[0022] FIG. 4 is a flowchart illustrating an operation of an electronic device according to various embodiments to obtain risk information based on lung disease information.

[0023] FIG. 5 is a diagram illustrating a set of input images according to various embodiments.

[0024] FIG. 6 is a diagram illustrating at least one artificial intelligence model according to various embodiments.

[0025] FIG. 7 is a flowchart illustrating an operation of an electronic device according to various embodiments to correct primary risk information based on lung disease information.

[0026] FIG. 8 is a diagram illustrating an electronic device obtaining risk information according to various embodiments.

[0027] FIG. 9 is a flowchart illustrating an operation of an electronic device according to various embodiments to display risk information depending on whether or not a patient has a lung disease.

[0028] Figure 10 is a flowchart illustrating an operation of obtaining risk information based on multiple sets of input images.

[0029] FIG. 11 is a diagram illustrating a method for an electronic device to determine a set of nodules according to various embodiments.

[0030] FIG. 12A is a flowchart illustrating an operation of an electronic device to determine similarity according to various embodiments.

[0031] FIG. 12B is a diagram illustrating a method for an electronic device to determine similarity according to various embodiments.

[0032] FIG. 12c is a diagram illustrating a method for an electronic device to determine similarity according to various embodiments.

[0033] FIG. 13 is a flowchart illustrating an operation of an electronic device according to various embodiments to obtain risk information of a nodule by considering whether or not it has a lung disease.

[0034] FIG. 14 is a diagram illustrating an electronic device according to various embodiments obtaining risk information based on a plurality of input image sets.

[0035] In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components.

[0036] Specific structural or functional descriptions for various embodiments are merely illustrative for the purpose of explaining the various embodiments, and the various embodiments may be implemented in various forms and should not be construed as limited to the embodiments described in this specification or application.

[0037] Since various embodiments may have various modifications and take various forms, various embodiments are illustrated in the drawings and described in detail in this specification or application. However, the matters disclosed in the drawings are not intended to specify or limit the various embodiments, and should be understood to include all modifications, equivalents, and alternatives included within the spirit and technical scope of the various embodiments.

[0038] While terms such as "first" and / or "second" may be used to describe various components, these components should not be limited by these terms. These terms are only intended to distinguish one component from another; for example, without departing from the scope of the present disclosure, a first component may be referred to as a "second component," and similarly, a second component may also be referred to as a "first component."

[0039] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. Conversely, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Other expressions that describe the relationship between components, such as "between" and "directly between" or "adjacent to" and "directly adjacent to", should be interpreted similarly.

[0040] The terminology used herein is for the purpose of describing specific embodiments only and is not intended to limit the various embodiments. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this specification, it should be understood that the terms "comprises" or "has" specify the presence of a described feature, number, step, operation, component, part, or combination thereof, but do not exclude in advance the presence or possibility of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0041] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by those of ordinary skill in the art to which this disclosure pertains. Terms defined in commonly used dictionaries should be interpreted to have a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0042] Hereinafter, the present disclosure will be described in detail by describing preferred embodiments of the present disclosure with reference to the attached drawings. The same reference numerals presented in each drawing represent the same components.

[0043]

[0044] FIG. 1 is a diagram illustrating a system in various embodiments.

[0045] Referring to FIG. 1, the system (100) may include a user terminal (110), an electronic device (130), and a database (150).

[0046] The user terminal (110) may be a variety of devices capable of verifying data generated by the electronic device (130) and metadata analyzed therefrom. For example, the user terminal (110) may include a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a medical device, a camera, a wearable device, or a home appliance. The electronic device according to the embodiments of the present document is not limited to the aforementioned devices. In various embodiments, the user terminal (110) may be connected to the electronic device (130) via a network, and multiple user terminals (110) may be connected to the electronic device (130) simultaneously.

[0047] According to various embodiments, the user terminal (110) may include a display. The display may visually provide information to an external party (e.g., a user) of the user terminal (110). According to various embodiments, the display may display various contents (e.g., text, images, videos, icons, and / or symbols). For example, it may display risk information for a node obtained from an electronic device (130). According to various embodiments, the display may include a liquid crystal display (LCD), a light emitting diode (LED) display, or an organic light emitting diode (OLED) display for this purpose.

[0048] The electronic device (130) may include a device that determines the risk level of at least one nodule included in at least a part of the user's body based on a set of input images that have been obtained by capturing at least a part of the user's body. The electronic device (130) may be provided by being included in a computer-readable recording medium by tangibly implementing a program of commands for implementing the same. In other words, the electronic device (130) may be implemented in the form of program commands that can be executed through various computer means and may be recorded in a computer-readable recording medium. In addition, the electronic device (130) may be configured as a computer program that sequentially or non-sequentially performs operations of receiving an image (or a set of images) that has captured at least a part of the user's body and analyzing the same, and the computer program may be stored in a computer-readable recording medium.

[0049] The database (150) may correspond to a storage device that stores various pieces of information generated through an electronic device (130) and / or a user terminal (110).

[0050]

[0051] FIG. 2 is a block diagram of an electronic device according to various embodiments.

[0052] Referring to FIG. 2, the electronic device (130) may be implemented by including a processor (210), a memory (230), a user input / output unit (250), and a communication circuit (270). The components listed above may be operatively or electrically connected to each other. The components of the electronic device (130) illustrated in FIG. 2 may be modified, deleted, or added in part, as an example. For example, the electronic device (130) may further include an output device (e.g., a display (e.g., the display described with reference to FIG. 1)). Hereinafter, when the electronic device (130) includes an output device, various information acquired through the electronic device (130) may be provided through the output device.

[0053] The processor (210) may include at least one processor implemented to provide at least some different functions. The processor (210) may control the overall operation of the electronic device (130) and may be electrically connected to the memory (230), the user input / output unit (250), and the communication circuit (270) to control data flow therebetween. The processor (210) may be implemented as a CPU (Central Processing Unit) of the electronic device (130). According to one embodiment, the processor (210) may include a main processor (e.g., a central processing unit or an application processor) or an auxiliary processor (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently of or together with the main processor. For example, when the electronic device (130) includes a main processor and an auxiliary processor, the auxiliary processor may be configured to use lower power than the main processor or to be specialized for a given function. The auxiliary processor may be implemented separately from the main processor or as a part thereof. The auxiliary processor may control at least a portion of functions or states associated with at least one of the components of the electronic device (130), for example, on behalf of the main processor while the main processor is in an inactive (e.g., sleep) state, or together with the main processor while the main processor is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component. In one embodiment, the auxiliary processor (e.g., neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning.This learning can be performed, for example, in the electronic device (130) itself where the artificial intelligence model is executed, or can be performed through a separate server. The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include a plurality of artificial neural network layers. The artificial neural network can be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model can additionally or alternatively include a software structure. Meanwhile, the operation of the electronic device (130) described below can be understood as the operation of the processor (210).

[0054] The memory (230) may include an auxiliary memory device implemented with a non-volatile memory such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive) and used to store all data required for the electronic device (130), and may include a main memory device implemented with a volatile memory such as a RAM (Random Access Memory). In addition, the memory (230) may include a plurality of instructions that direct the operations of the processor (210) to implement the functions provided by the service. In this case, the processor (210) may include a software server that executes the functions provided by the service based on the plurality of instructions stored in the memory (230).

[0055] The user input / output unit (250) may include an environment for receiving user input and an environment for outputting specific information to the user. For example, the user input / output unit (250) may include an input device including an adapter such as a touchpad, a touch screen, a virtual keyboard, or a pointing device, and an output device including an adapter such as a monitor or a touch screen. In one embodiment, the user input / output unit (250) may correspond to a computing device accessed via a remote connection, in which case the electronic device (130) may function as a server.

[0056] According to various embodiments, the electronic device (130) may include a communication circuit (270). In various embodiments, the communication circuit (270) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the electronic device (130) and an external electronic device (e.g., the user terminal (110) of FIG. 1), and the performance of communication through the established communication channel. The communication circuit (270) may operate independently from the processor (210) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication.

[0057] According to one embodiment, the communication circuit (270) may include a wireless communication module (e.g., a cellular communication module, a short-range wireless communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (e.g., a local area network (LAN) communication module, or a power line communication module). Any of these communication modules may communicate with an external electronic device via a first network (e.g., a short-range communication network such as Bluetooth, WiFi Direct (wireless fidelity direct), or IrDA (infrared data association)) or a second network (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips).

[0058]

[0059] FIG. 3 illustrates a concept for controlling a function related to risk assessment for at least one node of an electronic device, according to various embodiments.

[0060] The electronic device (130) may utilize hardware and / or software modules to support functions related to assessing the risk of at least one nodule. For example, the processor (210) may drive at least one of the input image receiving unit (310), the nodule detection unit (320), the size information obtaining unit (330), the type information obtaining unit (340), the lung disease information obtaining unit (350), the similarity determining unit (360), the variation information obtaining unit (370), and the risk information obtaining unit (380) by executing commands stored in the memory (230). In various embodiments, software modules other than those illustrated in FIG. 3 may be implemented. For example, at least two modules may be integrated into one module, or one module may be split into two or more modules. In addition, work performance may be improved by having hardware and software modules share a single function. For example, the electronic device (130) may include both an encoder implemented as hardware and an encoder implemented as a software module, and some of the data acquired through at least one camera module may be processed by the hardware encoder and some of the data may be processed by the software encoder.

[0061] According to various embodiments, the input image receiving unit (310) may receive an input image set. Here, the input image set may be an image capturing at least a portion of the user's body. For example, the input image set may be an image capturing the user's chest.

[0062] According to various embodiments, the input image receiving unit (310) may obtain sets of input images capturing at least a portion of the user's body at various points in time. For example, the input image receiving unit (310) may obtain a first set of input images capturing at least a portion of the user's body at a first point in time, and a second set of input images of the user captured at least one point in time that is distinct from the first point in time.

[0063] According to various embodiments, the nodule detection unit (320) can detect at least one nodule from a set of input images that capture at least a part of a user's body. For example, the nodule detection unit (320) can obtain a nodule set by detecting at least one nodule from a set of input images through at least one artificial intelligence model (e.g., the first artificial intelligence model (610) described with reference to FIG. 6, the nodule detection model (1113) described with reference to FIG. 11). For example, the nodule detection unit (320) can detect at least one nodule through a segmentation model trained to detect at least one nodule from a set of input images that capture at least a part of a user's body (e.g., the first artificial intelligence model (610) described with reference to FIG. 6, the nodule detection model (1113) described with reference to FIG. 11).

[0064] According to various embodiments, the size information acquisition unit (330) may acquire size information of at least one detected nodule. For example, the size information acquisition unit (330) may input the input image set into at least one artificial intelligence model (e.g., the first artificial intelligence model (610) described with reference to FIG. 6) to acquire size information for each of at least one nodule identified in the input image set. In one embodiment, the size information for the at least one nodule may be determined based on the volume of the at least one nodule.

[0065] According to various embodiments, the size information acquisition unit (330) may utilize at least one artificial intelligence model. For example, the at least one artificial intelligence model may include a model that specifies an evaluation area included in an input image set, a model that determines an abnormal area (e.g., an area determined to include at least one nodule) included in the evaluation area, and a model that determines the volume of the abnormal area to acquire size information.

[0066] According to various embodiments, the type information acquisition unit (340) can determine the type of at least one nodule detected from a set of input images that capture at least a part of the user's body. For example, the type information acquisition unit (340) can acquire type information on at least one nodule in an area determined to be at least one nodule through at least one artificial intelligence model (e.g., the second artificial intelligence model (620) described with reference to FIG. 6, a type detection model). For example, the type information acquisition unit (340) can acquire type information on at least one nodule through a type classification model (type detection model) learned to classify the type of at least one nodule detected by the nodule detection unit (320). The above type information may include various information related to the characteristics of at least one of the nodules, such as classification according to morphological characteristics (solid nodules, partially solid nodules, non-solid nodules), classification according to size (small nodules, medium nodules, giant nodules), classification according to growth and boundary characteristics (nodules with clear boundaries, nodules with unclear boundaries), classification according to pathological characteristics (benign nodules, malignant nodules, metastatic nodules), and other classifications (calcified nodules, air-containing nodules).

[0067] According to various embodiments, the lung disease information acquisition unit (350) may acquire information on a lung disease suffered by a user based on a set of input images that capture at least a portion of the user's body. For example, the lung disease information acquisition unit (350) may acquire lung disease information including whether the user has lung disease (and / or the type of lung disease) based on a set of input images through at least one artificial intelligence model (e.g., the third artificial intelligence model described with reference to FIG. 8, a lung disease detection model). For example, the lung disease information acquisition unit (350) may acquire the lung disease information through a model (lung disease detection model) trained to detect lung diseases such as pneumonia, tuberculosis, sarcoidosis, fungal infection, pulmonary vascular disease, airway disease, and pleural disease based on a set of input images that capture at least a portion of the user's body.

[0068] According to various embodiments, the similarity determination unit (360) may determine the similarity between at least one nodule included in each of the first input image set and the second input image set. For example, the similarity determination unit (360) may determine the similarity between the two input data through at least one artificial intelligence model (e.g., the similarity determination model (1201) of FIG. 12B). According to one embodiment, the similarity determination unit (360) may determine the similarity between at least one first nodule detected in the first input image set and at least one second nodule detected in the second input image set through the nodule detection unit (320). For example, the similarity determination unit (360) may determine the similarity by comparing whether a first activation area determined to include at least one first nodule based on the first input image set and a second activation area determined to include at least one second nodule based on the second input image set are similar. According to various embodiments, the similarity determination unit (360) can determine whether the at least one first nodule and the at least one second nodule are the same nodule. The similarity determination unit (360) can obtain the similarity information based on the determination.

[0069] According to various embodiments, the change information acquisition unit (370) may acquire change information of at least one first nodule detected in the first input image set and at least one second nodule detected in the second input image set, if it is determined that the at least one first nodule and the at least one second nodule are the same nodule. For example, the change information acquisition unit (370) may acquire change information by calculating change in size, type, position, etc. of the at least one first nodule and the at least one second nodule.

[0070] According to various embodiments, the risk information acquisition unit (380) may acquire risk information for at least one nodule detected in an input image set that captures at least a portion of the user's body. For example, the risk information acquisition unit (380) may determine the risk for the at least one nodule based on at least one of size information, type information, and lung disease information for the at least one nodule. For example, the risk information acquisition unit (380) may acquire risk information indicating risk information for the at least one nodule using at least one artificial intelligence model (e.g., the risk determination model (830) described with reference to FIG. 8, the risk determination model (1430) described with reference to FIG. 14). For example, the risk information acquisition unit (380) may acquire primary risk information based on size information and type information for the at least one nodule, and may correct the primary risk information based on the lung disease information to acquire risk information. For example, the risk information acquisition unit (380) may receive size information, type information, and lung disease information for at least one nodule and use an artificial intelligence model trained to determine the risk for at least one nodule to acquire risk information for at least one nodule.

[0071] According to various embodiments, the primary risk information may be determined based on a designated auxiliary indicator (e.g., Lung-RADS (Lung Imaging Reporting and Data System) category).

[0072] According to various embodiments, the risk information acquisition unit (380) may acquire risk information on at least one first nodule detected in the first input image set based on a first input image set in which at least a part of the user's body is captured at a first point in time, and a second input image set in which at least a part of the user's body is captured at at least one point in time that is distinct from the first point in time. For example, the risk information acquisition unit (380) may determine the risk on the at least one first nodule based on first size information and first type information on the at least one first nodule detected based on the first input image set, and variation amount information on the at least one second nodule and the at least one first nodule detected based on the second input image set. For example, the risk information acquisition unit (380) can acquire risk information indicating risk information for the at least one first nodule by using at least one artificial intelligence model (e.g., the risk judgment model (830) described with reference to FIG. 8, the risk judgment model (1430) described with reference to FIG. 14). For example, the risk information acquisition unit (380) can acquire primary risk information based on first size information and first type information for the at least one first nodule, and can acquire risk information for the at least one first nodule by considering change amount information between the at least one first nodule and the at least one second nodule.

[0073] According to one embodiment, the risk information acquisition unit (380) may consider the user's lung disease information when acquiring risk information for the at least one first nodule. For example, the risk information acquisition unit (380) may acquire risk information for the at least one first nodule based on the user's lung disease information, the first size information of the at least one first nodule, the first type information, and the change amount information. For example, the risk information acquisition unit (380) may receive the first size information, the first type information, the change amount information, and the lung disease information for the at least one first nodule, and may acquire risk information for the at least one first nodule using an artificial intelligence model trained to determine the risk of the at least one first nodule.

[0074] According to various embodiments, the electronic device (130) is not limited to the illustrated example and may provide various functions related to risk assessment for at least one nodule. For example, the electronic device (130) may configure various data sets including at least one of size information, type information, risk information, lung disease information, change amount information, and similarity information for at least one nodule through a data set configuration unit (not illustrated).

[0075] Here, the data set may be implemented on a user interface (UI) and displayed to the user via a display. Furthermore, the data set may be displayed to the user via an external device (e.g., the user terminal (110) of FIG. 1, a display device) connected to the electronic device (130).

[0076]

[0077] FIG. 4 is a flowchart (400) showing an operation of an electronic device according to various embodiments to obtain risk information based on lung disease information.

[0078] FIG. 5 is a diagram illustrating a set of input images according to various embodiments.

[0079] FIG. 6 is a diagram illustrating at least one artificial intelligence model according to various embodiments.

[0080] Each of the operations described below may be performed in combination with one another. In addition, among the operations described below, the operations performed by the electronic device (130) may refer to operations performed by the processor (210) of the electronic device (130).

[0081] In addition, the “information” described below may be interpreted to mean “data” or “signal,” and “data” may be understood as a concept that includes both analog data and digital data.

[0082] According to various embodiments, the operations illustrated in FIG. 4 may be performed in various orders, not limited to the order illustrated. Furthermore, according to various embodiments, more operations may be performed than those illustrated in FIG. 4, or at least one operation may be performed less than those illustrated in FIG.

[0083] Referring to FIG. 4, the electronic device (130) may obtain a set of input images capturing at least a portion of the user's body in operation 401.

[0084] Referring to FIG. 5, an input image set (510) is illustrated. The input image set (510) may be an image taken of the user's body, such as an MRI, CT, or X-ray. In one embodiment, the input image set (510) may include a plurality of slices. Meta-information related to the image information, such as slice thickness, resolution, patient information, scan date and time, equipment information, scan direction, use of contrast agent, radiation dose, image format and size, reconstruction parameters, scan range, sequence, and protocol information of the input image set (510), may be stored in the form of DICOM (Digital Imaging and Communications in Medicine) data. That is, the input image set (510) may be stored as a DICOM file, and may be stored in a form that includes metadata together with the image itself.

[0085] According to various embodiments, in operation 403, the electronic device (130) may obtain size information about a set of nodules included in at least a part of the body and at least one nodule included in the set of nodules.

[0086] Referring to FIG. 6, the electronic device (130) can classify at least one nodule included in the input image set (510) into an inactive area (603) or an active area (605) through a first artificial intelligence model (610) (e.g., a nodule detection model). Here, the first artificial intelligence model (610) can specify an evaluation area based on at least one nodule included in the input image set (510). The evaluation area can be a slice that includes at least a portion of at least one nodule. In addition, the evaluation area can include a plurality of slices, and a single continuous nodule can include a plurality of slices.

[0087] According to one embodiment, the first artificial intelligence model (610) may determine at least one nodule portion included in the evaluation area and determine size information (601) of the at least one nodule. For example, the first artificial intelligence model (610) may determine a volume of the at least one nodule and output size information (601) for the at least one nodule based on the volume. In addition, the first artificial intelligence model (610) may classify the evaluation area as an inactive area (603) when the volume of the at least one nodule included in the evaluation area is less than or equal to a cutoff value. Here, the cutoff value is a set value and may be set differently depending on the user.

[0088] In one embodiment, the first artificial intelligence model (610) may be a model trained to identify an evaluation area including at least one nodule and determine the volume of at least one nodule. That is, the first artificial intelligence model (610) may be trained in various ways and is not limited to any one method. Furthermore, the first artificial intelligence model (610) may include at least one sub-AI model, and each of the sub-AI models may be trained in various ways.

[0089] In one embodiment, the first sub-AI model of the first AI model (610) may be trained to automatically identify an evaluation region containing at least one nodule. For example, the first sub-AI model may be trained to identify the presence of pulmonary nodules, tumors, or abnormal tissues in medical images such as computed tomography (CT) or magnetic resonance imaging (MRI). For example, the first sub-AI model may learn the characteristics of abnormal areas determined to be nodules in an image and distinguishing factors from surrounding normal tissues through a deep learning method on an annotated medical image data set.

[0090] Additionally, according to one embodiment, the second sub-AI model of the first AI model (610) may be trained to output size information of at least one detected nodule. In one embodiment, the electronic device (130) may determine size information of at least one nodule based on the number of pixels of at least one nodule on the slice through the first AI model (610). For example, the boundary of at least one nodule may be detected through the second sub-AI model of the first AI model (610), and the volume of at least one nodule may be determined.

[0091] According to one embodiment, the electronic device (130) can determine size information of at least one nodule using DICOM data. For example, the electronic device (130) can determine size information (e.g., volume) of at least one nodule using data such as slice interval and resolution included in the DICOM data.

[0092] According to various embodiments, in operation 405, the electronic device (130) may obtain type information about the at least one node. In various embodiments, the electronic device (130) obtaining type information about the at least one node may be identical to or similar to the operation of the type information obtaining unit (340) described with reference to FIG. 3. Hereinafter, redundant descriptions are omitted.

[0093] Referring to FIG. 6, the electronic device (130) can obtain type information (607) for at least one nodule included in the input image set (510) through the second artificial intelligence model (620) (e.g., type detection model). According to one embodiment, when the volume of at least one nodule in the evaluation area detected through the first artificial intelligence model (610) exceeds a cutoff value, the electronic device (130) determines the evaluation area as an activation area (605) and inputs the activation area (605) into the second artificial intelligence model (620) to obtain type information (607) of the at least one nodule.

[0094] According to one embodiment, the second artificial intelligence model (620) may be a model trained to classify the type of at least one nodule included in the activation area (605) and output type information. For example, the second artificial intelligence model may be a model trained to receive a nodule set including at least one nodule as input and output type information for the at least one nodule.

[0095] In various embodiments, if at least one nodule is classified as an inactive region (603) by the first artificial intelligence model (610), the inactive region (603) may be excluded from analysis by the second artificial intelligence model (620). In addition, the inactive region (603) may be classified as not a nodule in the results for a patient provided through the implementation of the present disclosure, and may thus be excluded from further analysis by an expert. In addition, the active region (605) may be classified as a nodule in the results for a patient provided through the implementation of the present disclosure, and may thus be requested for further analysis by an expert. In other words, classification as an inactive region (6603) or an active region (605) may provide convenience in terms of accuracy and speed in patient analysis to experts.

[0096] According to various embodiments, in operation 407, the electronic device (130) may obtain lung disease information regarding whether the user has a lung-related disease. For example, the electronic device (130) may obtain information about lung diseases, such as whether the user has a lung-related disease, and if so, what type of disease, based on a set of input images that capture at least a part of the user's body. For example, the electronic device (130) may obtain lung disease information including whether the user has a lung disease (and / or the type of lung disease) based on a set of input images through at least one artificial intelligence model (e.g., the third artificial intelligence model described with reference to FIG. 8, a lung disease detection model). For example, the electronic device (130) can obtain the lung disease information through a lung disease detection model, which is an artificial intelligence model trained to detect whether a lung disease such as pneumonia, tuberculosis, sarcoidosis, fungal infection, pulmonary vascular disease, airway disease, pleural disease, etc. exists and the type of lung disease based on an input image set that captures at least a part of the user's body.

[0097] According to various embodiments, in operation 409, the electronic device (130) may obtain risk information for at least one nodule based on the size information, the type information, and the lung disease information. For example, the electronic device (130) may obtain risk information for at least one nodule detected in a set of input images that capture at least a part of the user's body. For example, the electronic device (130) may determine the risk for the at least one nodule based on at least one of the size information, the type information, and the lung disease information for the at least one nodule. For example, the electronic device (130) may obtain risk information indicating risk information for the at least one nodule using at least one artificial intelligence model (e.g., the risk determination model (830) described with reference to FIG. 8, the risk determination model (1430) described with reference to FIG. 14).

[0098]

[0099] FIG. 7 is a flowchart (700) showing an operation of an electronic device according to various embodiments to correct primary risk information based on lung disease information.

[0100] FIG. 8 is a diagram illustrating an electronic device obtaining risk information according to various embodiments.

[0101] Each of the operations described below may be performed in combination with one another. In addition, among the operations described below, the operations performed by the electronic device (130) may refer to operations performed by the processor (210) of the electronic device (130).

[0102] In addition, the “information” described below may be interpreted to mean “data” or “signal,” and “data” may be understood as a concept that includes both analog data and digital data.

[0103] According to various embodiments, the operations illustrated in FIG. 7 may be performed in various orders, not limited to the order illustrated. Furthermore, according to various embodiments, more operations may be performed than those illustrated in FIG. 7, or at least one operation may be performed less than those illustrated in FIG.

[0104] Referring to FIG. 7, the electronic device (130) may obtain primary risk information for at least one nodule based on size information and type information in operation 701. For example, the electronic device (130) may obtain size information and type information for at least one nodule based on a set of input images that capture at least a portion of the user's body through the operation described with reference to FIG. 4.

[0105] For example, referring to FIG. 8, the electronic device (130) can obtain an input image set (810) that captures at least a portion of the user's body (e.g., chest). The electronic device (130) can input the input image set (810) into at least one artificial intelligence model (820) to obtain size information (821), type information (823), and lung disease information (825).

[0106] According to various embodiments, the electronic device (130) may obtain primary risk information for each of at least one target node based on size information (821)) and type information (823)).

[0107] According to various embodiments, in operation 703, the electronic device (130) may obtain risk information by correcting the primary risk information based on the lung disease information. According to one embodiment, the electronic device (130) may obtain risk information (840) using the risk judgment model (830). For example, the electronic device (130) may obtain the primary risk information (831) based on the size information (821) and the type information (823). In addition, for example, the electronic device (130) may obtain risk information (840) by correcting the primary risk information (831) based on whether or not the device has a lung disease (833). For example, the electronic device (130) can determine primary risk information (831) based on auxiliary indicators (e.g., Lung-RADS (Lung Imaging Reporting and Data System) category) specified according to size information (821) and type information (823) through a risk assessment model (830), and can obtain risk information (840) by correcting (e.g., increasing) the numerical value of the primary risk information according to whether or not there is a lung disease (833).

[0108] According to various embodiments, the at least one artificial intelligence model (820) and the risk assessment model (830) may be configured as a single artificial intelligence model. For example, the electronic device (130) may input an input image set (810) into the artificial intelligence model to obtain risk information (840). In this case, the risk information (840) may be obtained through an operation similar to the operation described above.

[0109]

[0110] FIG. 9 is a flowchart (900) showing an operation of an electronic device according to various embodiments to display risk information depending on whether or not a patient has a lung disease.

[0111] Each of the operations described below may be performed in combination with one another. In addition, among the operations described below, the operations performed by the electronic device (130) may refer to operations performed by the processor (210) of the electronic device (130).

[0112] In addition, the “information” described below may be interpreted to mean “data” or “signal,” and “data” may be understood as a concept that includes both analog data and digital data.

[0113] According to various embodiments, the operations illustrated in FIG. 9 may be performed in various orders, not limited to the order illustrated. Furthermore, according to various embodiments, more operations may be performed than those illustrated in FIG. 9, or at least one operation may be performed less than those illustrated in FIG.

[0114] According to various embodiments, the electronic device (130) may output risk information acquired based on a set of input images capturing at least a portion of the user's body. For example, the electronic device (130) may transmit the risk information to an external device (e.g., the user terminal (110) of FIG. 1) so that the risk information is output through the display of the external device.

[0115] Referring to FIG. 9, the electronic device (130) can determine whether the user has a lung disease in operation 901. For example, the electronic device (130) can determine whether the risk information has been corrected based on whether the user has a lung disease when calculating the risk information to be displayed.

[0116] According to various embodiments, in operation 903, if the user has a lung disease (901 - Yes), the electronic device (130) may transmit the risk information to an external device so as to display risk information based on a first visual object. For example, if the risk information for at least one nodule is determined by taking into account that the user has a lung disease, that is, if the risk information is determined by correcting the primary risk information for at least one nodule, the electronic device (130) may display the risk information based on the first visual object. For example, the electronic device (130) may display the risk information based on a first visual object that includes a mark (e.g., a distinguished mark) indicating that the risk information is corrected information depending on whether the user has a lung disease.

[0117] According to various embodiments, the electronic device (130) may transmit the risk information to an external device in operation 905 so as to display the risk information based on a second visual object if the user does not have a lung disease (901-No). For example, the electronic device (130) may display the risk information based on a second visual object if the risk information for at least one nodule is determined by taking into account that the user does not have a lung disease, i.e., if the risk information is determined based on size information and type information for at least one nodule.

[0118] According to one embodiment, the first visual object and the second visual object can be distinguished. Accordingly, the user can distinguish whether the risk for at least one nodule is a risk adjusted due to lung disease. For example, if the risk is determined to be 3 based on the size information and type information of at least one nodule, and if the initial risk information is determined to be 2 based on the size information and type information of at least one nodule and then the risk is adjusted to 3 due to lung disease, the two can be distinguished by using distinct visual objects, even though the risk values ​​are the same.

[0119]

[0120] Figure 10 is a flowchart (1000) showing an operation of obtaining risk information based on multiple input image sets.

[0121] FIG. 11 is a diagram illustrating a method for an electronic device to determine a set of nodules according to various embodiments.

[0122] Each of the operations described below may be performed in combination with one another. In addition, among the operations described below, the operations performed by the electronic device (130) may refer to operations performed by the processor (210) of the electronic device (130).

[0123] In addition, the “information” described below may be interpreted to mean “data” or “signal,” and “data” may be understood as a concept that includes both analog data and digital data.

[0124] According to various embodiments, the operations illustrated in FIG. 10 may be performed in various orders, not limited to the order illustrated. Furthermore, according to various embodiments, more operations may be performed than those illustrated in FIG. 10, or at least one operation may be performed less than those illustrated in FIG.

[0125] Referring to FIG. 10, in operation 1001, the electronic device (130) may acquire a first set of input images taken of at least a portion of the user's body at a first point in time.

[0126] According to various embodiments, in operation 1003, the electronic device (130) can detect at least one first nodule for a first set of input images.

[0127] According to various embodiments, the electronic device (130) can detect at least one second nodule from a second set of input images acquired at at least one time point distinct from the first time point.

[0128] Referring to FIG. 11, the electronic device (130) detects at least one first nodule and at least one second nodule based on the first input image set and the second input image set.

[0129] According to one embodiment, the electronic device (130) can input a first input image set (1111) captured at a first point in time into a nodule detection model (1113) to obtain a first activation area (1115) including at least one first nodule. In addition, according to one embodiment, the electronic device (130) can input a second input image set (1121) captured at at least one point in time distinct from the first point in time into the nodule detection model (1113) to obtain a second activation area (1125) including at least one second nodule.

[0130] According to various embodiments, the nodule detection model (1113) (e.g., the first artificial intelligence model (610) of FIG. 6, the nodule detection model described with reference to FIG. 3) may be an artificial intelligence model trained to detect at least one nodule included in at least a part of the user's body based on an input image set.

[0131] According to various embodiments, the first input image set and the second input image set may include images taken of at least a part of the user's body (e.g., the chest). For example, the first input image set (1111) may be an image of the user's body taken of a first time point, such as an MRI, CT, or X-ray. For example, the second input image set (1121) may be an image of the user's body taken of at least one time point (e.g., a past time point) different from the first time point. According to one embodiment, the first input image set (1111) and the second input image set (1121) may each include a plurality of slides, and may be 3D images. For example, the first input image set (1111) and the second input image set (1121) may each include a plurality of slides including at least one nodule.

[0132] According to one embodiment, the first input image set (1111) and the second input image set (1121) may be images captured in the same format. In this case, the first input image set (1111) may be images captured at a first point in time, and the second input image set (1121) may be images captured at least at a point in time that is distinct from the first point in time, and may be images captured in the past that is temporally earlier than the first point in time.

[0133] Specifically, the electronic device (130) can determine a first activation region (1115) including at least one first nodule included in a first input image set (1111) and a second activation region (1125) including at least one second nodule included in a second input image set (1121) through a nodule detection model (1113). In one embodiment, the nodule detection model (1113) can be composed of at least one artificial intelligence model. For example, the nodule detection model (1113) can specify evaluation regions included in the first input image set (1111) and the second input image set (1121). In addition, the nodule detection model (1113) can determine at least one nodule portion included in the evaluation region and determine a volume of the at least one nodule. In addition, the nodule detection model (1113) can classify the evaluation region as an inactive region when the volume of at least one nodule included in the evaluation region is less than or equal to a cutoff value. Conversely, the nodule detection model (1113) can classify the evaluation area as an activated area if the volume of at least one nodule is greater than the cutoff value. Here, the activated area can be the image itself cropped to the evaluation area and can be determined to a preset size. Meanwhile, at least one nodule can be a set of activated area data including nodules. Each of the at least one first nodule and the at least one second nodule can include at least one piece of activated area data as data determined from images captured at a first time point and at least one time point distinct from the first time point.

[0134] In one embodiment, the electronic device (130) can determine at least one nodule including an active area through the nodule detection model (1113). As described above, the active area can include at least one segmented nodule portion (e.g., at least one first nodule, at least one second nodule). In addition, the active area can include at least one nodule portion included within a preset size and a surrounding area thereof. That is, the electronic device (130) can determine an active area including at least one nodule portion and a surrounding area through the nodule detection model (1113).

[0135] As described above, the activation area can be generated with a preset size. The activation area (520) can include at least one nodule and a surrounding area, and can be generated with a preset size. For example, the activation area (e.g., the first activation area (1115), the second activation area (1125)) can be generated with a size of 10 mm x 10 mm. For another example, the activation area (e.g., the first activation area (1115), the second activation area (1125)) can be generated with a size of 15 mm x 5 mm. The activation area (e.g., the first activation area (1115), the second activation area (1125)) generated in this way can be a target for similarity determination.

[0136] According to various embodiments, in operation 1005, the electronic device (130) may determine a similarity between at least one first nodule and at least one second nodule. For example, the electronic device (130) may acquire at least one second nodule detected from a second input image set acquired at at least one time point distinct from the first time point, and determine a similarity between the at least one first nodule and the at least one second nodule.

[0137] According to various embodiments, the electronic device (130) determines the similarity between at least one first nodule and at least one second nodule as described below with reference to FIGS. 12a, 12b and 12c.

[0138] According to various embodiments, in operation 1007, if the similarity is greater than or equal to a preset value, the electronic device (130) may input the first input image set (1111) and the second input image set (1121) into a risk assessment model to obtain risk information for at least one first nodule. For example, if the electronic device (130) determines that at least one first nodule and at least one second nodule are the same nodule, the electronic device (130) may use the second input image set (1121) together to determine the risk for the at least one first nodule.

[0139] For example, the electronic device (130) may determine the risk for the at least one first nodule based on first size information and first type information for the at least one first nodule detected based on the first input image set (1111), and change amount information for the at least one second nodule and the at least one first nodule detected based on the second input image set (1121). For example, the electronic device (130) may obtain risk information indicating risk information for the at least one first nodule by using at least one artificial intelligence model (e.g., the risk judgment model (1430) described with reference to FIG. 14).

[0140] That is, when determining the risk of at least one first nodule detected from a first input image set (1111) photographed at a first point in time, the electronic device (130) can more accurately determine the risk of at least one first nodule by considering information (e.g., change amount information) based on at least one second nodule detected from a second input image set (1121) photographed at a different point in time (e.g., past point in time).

[0141]

[0142] FIG. 12a is a flowchart illustrating an operation of an electronic device determining similarity according to various embodiments. FIG. 12b is a diagram illustrating a method for an electronic device to determine similarity according to various embodiments. FIG. 12c is a diagram illustrating a method for an electronic device to determine similarity according to various embodiments.

[0143] Each of the operations described below may be performed in combination with one another. In addition, among the operations described below, the operations performed by the electronic device (130) may refer to operations performed by the processor (210) of the electronic device (130).

[0144] In addition, the “information” described below may be interpreted to mean “data” or “signal,” and “data” may be understood as a concept that includes both analog data and digital data.

[0145] According to various embodiments, the operations illustrated in FIG. 12A may be performed in various orders, not limited to the order illustrated. Furthermore, according to various embodiments, more operations may be performed than those illustrated in FIG. 12A, or at least one operation may be performed less than those illustrated in FIG.

[0146] Referring to FIG. 12A, the electronic device (130) may generate a feature map of an activated area in operation 1210. For example, the electronic device (130) may generate a first feature map of a first activated area (1115) including at least one first nodule detected in a first input image set (1111). For example, the electronic device (130) may generate a second feature map of a second activated area (1125) including at least one second nodule detected in a second input image set (1121).

[0147] According to various embodiments, the electronic device (130) may determine the similarity between feature maps in operation 1230. For example, the electronic device (130) may determine the similarity between the first feature map and the second feature map.

[0148] Referring to FIG. 12B, the electronic device (130) can determine the similarity between at least one first nodule included in the first activation area (1115) and at least one second nodule included in the second activation area (1125) through the similarity judgment model (1201). In one embodiment, the electronic device (130) can determine the similarity between at least one first nodule and at least one second nodule by comparing at least one first activation area (1115) and at least one second activation area (1125), respectively.

[0149] For example, the electronic device (130) may classify the activation areas determined to be similar to the activation areas of at least one first nodule among at least one second nodule into a similar nodule set (1203), and may classify the activation areas determined to be dissimilar to all activation areas included in at least one first nodule among at least one second nodule into a dissimilar nodule set (1205). In addition, for example, the electronic device (130) may classify the activation areas determined to be similar to the activation areas included in at least one second nodule among at least one first nodule into a similar nodule set (1203), and may classify the activation areas determined to be dissimilar to the activation areas included in at least one second nodule among at least one first nodule into a dissimilar nodule set (1205).

[0150] In one embodiment, the electronic device (130) can generate a feature map of an activated area and determine the similarity between feature maps through a similarity judgment model (1201). Here, the feature map can be a set of information that emphasizes a specific feature of an image and can be extracted through a convolutional neural network. The similarity judgment model (1201) can determine the similarity between the generated feature maps. For example, the similarity judgment model (1201) can generate feature maps for each of a first activated area (1115) and a second activated area (1125) and compare them to determine the similarity. Here, the method for determining the similarity is not limited to a specific method and can be performed in various ways, such as a statistical method, a distance-based method, or a machine learning method.

[0151] In addition, the similarity judgment model (1201) can determine the similarity between feature maps through geometric matching. The electronic device (130) can determine the similarity between feature maps based on geometric matching through the similarity judgment model (1201). That is, the electronic device (130) can determine the similarity between the first activation area (1115) and the second activation area (1125) through geometric matching. Here, the geometric matching can determine the similarity between feature maps using various transformation models. The similarity judgment model (1201) can determine the similarity between the activation areas by considering the similarity of at least one nodule included in the activation area as well as the similarity of the surrounding area through geometric matching. The similarity judgment model (1201) can determine the similarity between a plurality of feature maps by geometrically transforming the feature map through a transformation model of the feature map.

[0152] In various embodiments, the electronic device (130) may determine at least one nodule included in an activation area as the same nodule if the similarity is greater than or equal to a reference similarity. For example, the electronic device (130) may determine the at least one first nodule and the at least one second nodule as the same nodule if the similarity between a first activation area including at least one first nodule and a second activation area including at least one second nodule is greater than or equal to a preset value. If the at least one first nodule and the at least one second nodule are determined to be the same nodule, the electronic device (130) may consider change amount information confirmed through the at least one second nodule when determining the risk of the at least one first nodule.

[0153]

[0154] FIG. 13 is a flowchart illustrating an operation of an electronic device according to various embodiments to obtain risk information of a nodule by considering whether or not it has a lung disease.

[0155] FIG. 14 is a diagram illustrating an electronic device according to various embodiments obtaining risk information based on a plurality of input image sets.

[0156] Each of the operations described below may be performed in combination with one another. In addition, among the operations described below, the operations performed by the electronic device (130) may refer to operations performed by the processor (210) of the electronic device (130).

[0157] In addition, the “information” described below may be interpreted to mean “data” or “signal,” and “data” may be understood as a concept that includes both analog data and digital data.

[0158] According to various embodiments, the operations illustrated in FIG. 13 may be performed in various orders, not limited to the order illustrated. Furthermore, according to various embodiments, more operations may be performed than those illustrated in FIG. 13, or at least one operation may be performed less than those illustrated in FIG.

[0159] Referring to FIG. 13, in operation 1301, the electronic device (130) can input a first input image set into a lung disease detection model to obtain lung disease information indicating whether the user has a lung-related disease.

[0160] According to various embodiments, the electronic device (130) may obtain risk information of the at least one first nodule based on the first size information, the first type information, the variation information, and the lung disease information in operation 1303. For example, the electronic device (130) may obtain risk information of the at least one first nodule based on the first size information of the at least one first nodule, the first type information of the at least one first nodule, the variation information based on the difference between the at least one first nodule and the at least one second nodule, and the lung disease information.

[0161] For example, referring to FIG. 14, the electronic device (130) may acquire a first input image set (1411) captured at least a portion of the user's body (e.g., chest) at a first point in time. In addition, the electronic device (130) may acquire a second input image set (1413) captured at least one point in time distinct from the first point in time (e.g., a point in time prior to the first point in time).

[0162] According to various embodiments, the electronic device (130) may input a first input image set (1411) into at least one artificial intelligence model (1420) to obtain first size information (1421) indicating a size of at least one first nodule, first type information (1423) indicating a type of the at least one first nodule, and lung disease information (1425). In addition, the electronic device (130) may input a second input image set (1413) into at least one artificial intelligence model to obtain size information (1427) indicating a size of at least one second nodule, and second type information (1429) indicating a type of the at least one second nodule.

[0163] According to one embodiment, the electronic device (130) may input each of the first input image set (1411) and the second input image set (1413) into at least one artificial intelligence model (e.g., the nodule detection model (1113) of FIG. 11, the first artificial intelligence model (610) of FIG. 6) to obtain first size information (1421) and second size information (1427).

[0164] According to one embodiment, the electronic device (130) may input each of the first input image set (1411) and the second input image set (1413) into at least one artificial intelligence model (e.g., the second artificial intelligence model (620) of FIG. 6) to obtain first type information (1423) and second size information (1429).

[0165] According to one embodiment, the electronic device (130) can obtain lung disease information (1425) by inputting a first input image set (1411) into at least one artificial intelligence model (e.g., a lung disease detection model described with reference to FIG. 3).

[0166] According to various embodiments, the electronic device (130) may obtain risk information (1440) of at least one first nodule based on the first size information (1421), the first type information (1423), the change amount information (1435) obtained based on the first size information (1421) and the second size information (1427) and / or the first type information (1423) and the second type information (1429), and the lung disease information (1425) through the risk judgment model (1430).

[0167] In one embodiment, the electronic device (130) may obtain primary risk information (1431) for each of at least one first node being targeted, based on the first size information (1421)) and the first type information (1423)).

[0168] In one embodiment, the electronic device (130) can obtain risk information (1440) by correcting the primary risk information (1431) based on the change information (1435) and the presence or absence of lung disease (1433). According to one embodiment, the electronic device (130) can obtain the risk information (1440) using the risk judgment model (1430). For example, the electronic device (130) can obtain the primary risk information (1431) based on the first size information (1421) and the first type information (1423). In addition, for example, the electronic device (130) can obtain the risk information (1440) by correcting the primary risk information (1431) based on the change information (1435) and the presence or absence of lung disease (1433).

[0169] According to various embodiments, the at least one artificial intelligence model (1420) and the risk assessment model (1430) may be configured as a single artificial intelligence model. For example, the electronic device (130) may input a first input image set (1411) and a second input image set (1413) into the artificial intelligence model to obtain risk information (1440). In this case, the risk information (1440) may be obtained through an operation similar to the operation described above.

[0170] That is, the electronic device (130) can obtain the first input image set (1411) and the second input image set (1413) for the user through user registration information, etc., and input the first input image set (1411) and the second input image set (1413) into at least one artificial intelligence model to obtain risk information (1440) of at least one first nodule included in the first input image set (1411). At this time, if the electronic device (130) determines that at least one first nodule included in the first input image set (1411) and at least one second nodule included in the second input image set (1413) are the same nodule through a similarity judgment model (e.g., the similarity judgment model (1201) of 12b), the electronic device (130) can obtain risk information (1440) by using not only the first size information (1421) and the first type information (1423) for the at least one first nodule, but also the change amount information (1435) between the at least one second nodule and the at least one first nodule.

[0171] According to various embodiments, the electronic device (130) may output risk information (1440) acquired based on a first input image set that captures at least a portion of the user's body. For example, the electronic device (130) may transmit the risk information (1440) to an external device (e.g., the user terminal (110) of FIG. 1) so that the risk information (1440) is output through a display of the external device. In addition, for example, the electronic device (130) may display the risk information (1440) through a display included in (or connected to) the electronic device (130).

[0172] As described above, the electronic device (130) includes a communication circuit for transmitting and receiving data with an external device, at least one processor, and a memory for storing commands, wherein the commands are individually or collectively executed by the at least one processor so that the electronic device obtains a first input image set in which at least a part of the user's body is photographed at a first point in time, detects at least one first nodule with respect to the first input image set, determines a similarity between the at least one first nodule and at least one second nodule, and the at least one second nodule is obtained based on a second input image set stored in the memory, and when the similarity is equal to or greater than a preset value, inputs the first input image set and the second input image set into a risk assessment model to obtain risk information about the at least one first nodule, and the risk assessment model may be configured to obtain the risk information about the at least one first nodule based on first size information about the at least one first nodule, first type information about the at least one first nodule, and variation information about the at least one first nodule and the at least one second nodule. It may be an artificial intelligence model trained to output risk information of at least one first nodule.

[0173] In one embodiment, the instructions are individually or collectively executed by the at least one processor to cause the electronic device to input the first set of input images into a nodule detection model to detect the at least one first nodule of at least a portion of the user's body, and to input the second set of input images into the nodule detection model to detect the at least one second nodule, wherein the nodule detection model may be an artificial intelligence model trained to detect at least one nodule included in at least a portion of the user's body based on the input set of images.

[0174] According to one embodiment, the instructions may be individually or collectively executed by the at least one processor to cause the electronic device to, through the nodule detection model, specify a first evaluation area in the first input image set, obtain first size information of at least one first nodule included in the first evaluation area, and classify the evaluation area as a first activation area if a volume of the at least one first nodule is greater than a cutoff value based on the first size information, and, through the nodule detection model, specify a second evaluation area in the second input image set, obtain second size information of at least one second nodule included in the second evaluation area, and classify the evaluation area as a second activation area if a volume of the at least one second nodule is greater than a cutoff value based on the second size information.

[0175] According to one embodiment, the instructions are individually or collectively executed by the at least one processor to cause the electronic device to generate a first feature map of the first activation area, generate a second feature map of the second activation area, and obtain a similarity between the first feature map and the second feature map through a similarity judgment model, wherein the similarity judgment model may be a model learned to determine a similarity based on activation areas including at least one nodule.

[0176] According to one embodiment, the similarity judgment model can determine the similarity between the first feature map and the second feature map through geometric matching.

[0177] According to one embodiment, the instructions may be individually or collectively executed by the at least one processor to cause the electronic device to input the first activation area into a type detection model to obtain the first type information of the at least one first module, and to input the second activation area into the type detection model to obtain the second type information of the at least one second module, wherein the type detection model may be an artificial intelligence model trained to output a type for at least one nodule.

[0178] According to one embodiment, the commands may be individually or collectively executed by the at least one processor to cause the electronic device to input the first input image set into a lung disease detection model to obtain lung disease information indicating whether the user has a lung-related disease, wherein the lung disease detection model is an artificial intelligence model trained to receive an image including at least a part of the user's body and detect the presence or absence of lung disease, and to obtain risk information of the at least one first nodule based on the first size information, the first type information, the change amount information, and the lung disease information through the risk determination model.

[0179] According to one embodiment, the instructions may be individually or collectively executed by the at least one processor to cause the electronic device to obtain primary risk information of the at least one first nodule based on the first size information, the first type information, and the change amount information through the risk assessment model, and to obtain the risk information by correcting the primary risk information based on the lung disease information.

[0180] In one embodiment, the second input image set may be a set of images acquired at at least one point in time that is distinct from the first point in time.

[0181] In one embodiment, the instructions may be individually or collectively executed by the at least one processor to cause the electronic device to output the risk information through a display of the external device and to transmit the risk information to the external device.

[0182] As described above, the operating method of the electronic device may include an operation of obtaining a first input image set in which at least a part of a user's body is photographed at a first point in time, an operation of detecting at least one first nodule with respect to the first input image set, an operation of determining a similarity between the at least one first nodule and at least one second nodule, an operation of obtaining the at least one second nodule based on a second input image set stored in the electronic device, and, if the similarity is equal to or greater than a preset value, inputting the first input image set and the second input image set into a risk assessment model to obtain risk information for the at least one first nodule, wherein the risk assessment model may be an artificial intelligence model trained to output risk information for the at least one first nodule based on first size information for the at least one first nodule, first type information for the at least one first nodule, and variation information for the at least one first nodule and the at least one second nodule.

[0183] According to one embodiment, the operation of detecting the at least one first nodule includes an operation of inputting the first input image set to a nodule detection model to detect the at least one first nodule in at least a part of the body of the user, and the operation of detecting the at least one second nodule includes an operation of inputting the second input image set to the nodule detection model to detect the at least one second nodule, wherein the nodule detection model may be an artificial intelligence model trained to detect at least one nodule included in at least a part of the body of the user based on the input image set.

[0184] According to one embodiment, the operation of detecting the at least one first nodule may include: an operation of specifying a first evaluation area in the first input image set through the nodule detection model, obtaining first size information of at least one first nodule included in the first evaluation area, and an operation of classifying the evaluation area as a first activation area based on the first size information when a volume of the at least one first nodule is greater than a cutoff value; and the operation of detecting the at least one second nodule may include: an operation of specifying a second evaluation area in the second input image set through the nodule detection model, obtaining second size information of at least one second nodule included in the second evaluation area, and an operation of classifying the evaluation area as a second activation area based on the second size information when a volume of the at least one second nodule is greater than a cutoff value.

[0185] According to one embodiment, the operation of determining the similarity further includes an operation of generating a first feature map of the first activated area, an operation of generating a second feature map of the second activated area, and an operation of obtaining a similarity between the first feature map and the second feature map through a similarity determination model, wherein the similarity determination model may be an artificial intelligence model learned to determine the similarity based on activated areas including at least one nodule.

[0186] According to one embodiment, the method further includes an operation of inputting the first activation area into a type detection model to obtain the first type information of the at least one first nodule, and an operation of inputting the second activation area into the type detection model to obtain the second type information of the at least one second nodule, wherein the type detection model may be an artificial intelligence model trained to output a type for at least one nodule.

[0187] According to one embodiment, the operation of obtaining the risk information may further include an operation of inputting the first input image set into a lung disease detection model to obtain lung disease information indicating whether the user has a lung-related disease; the lung disease detection model is an artificial intelligence model trained to receive an image including at least a part of the user's body and detect the presence or absence of lung disease; and an operation of obtaining risk information of the at least one first nodule based on the first size information, the first type information, the change amount information, and the lung disease information through the risk judgment model.

[0188] According to one embodiment, the operation of obtaining the risk information may further include an operation of obtaining primary risk information of the at least one first nodule based on the first size information, the first type information, and the change amount information through the risk judgment model, and an operation of obtaining the risk information by correcting the primary risk information based on the lung disease information.

[0189] In one embodiment, the second input image set may be a set of images acquired at at least one point in time that is distinct from the first point in time.

[0190] According to one embodiment, the method of operating an electronic device may further include an operation of transmitting the risk information to an external device connected to the electronic device so as to output the risk information through a display of the external device.

[0191] As described above, in a non-transitory computer-readable recording medium including a program for executing a control method of an electronic device, the control method comprises the steps of: obtaining a first input image set in which at least a part of a user's body is photographed at a first point in time; detecting at least one first nodule with respect to the first input image set; determining a similarity between the at least one first nodule and at least one second nodule; wherein the at least one second nodule is obtained based on a second input image set stored in the electronic device, and when the similarity is equal to or greater than a preset value, inputting the first input image set and the second input image set into a risk assessment model to obtain risk information for the at least one first nodule; wherein the risk assessment model may be an artificial intelligence model trained to output risk information for the at least one first nodule based on first size information for the at least one first nodule, first type information for the at least one first nodule, and variation information for the at least one first nodule and the at least one second nodule.

[0192] In this disclosure, each of the phrases "A or B", "at least one of A and B", "at least one of A or B", "A, B, or C", "at least one of A, B, and C", and "at least one of A, B, or C" may include any one of the items listed together in the corresponding phrase, or all possible combinations thereof.

[0193] Terms such as "first," "second," or "first" or "second" may be used simply to distinguish one component from another and do not qualify the components in any other respect (e.g., importance or order).

[0194] The terms "part" and "module" used in various embodiments of the present disclosure may include units implemented in hardware, software, or firmware. For example, they may be used interchangeably with terms such as logic, logic block, component, or circuit. A module may be an integrally formed component or a minimum unit or part of the component that performs one or more functions. The "part" and "module" used in various embodiments of the present disclosure may be stored in an addressable storage medium and implemented by various programs that can be executed by a processor.

[0195] Various embodiments of the present disclosure may be implemented as software (e.g., a program) including one or more commands stored in a memory (230) (e.g., built-in memory or external memory) readable by a device (e.g., an electronic device (130)). The memory (230) may be represented as a storage medium.

[0196] According to one embodiment, the methods according to the various embodiments disclosed in this document may be provided as a computer program product. The computer program product may be traded as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a device-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices.

[0197] According to various embodiments, each component (e.g., a module or a program) of the above-described components may include one or more entities, and some of the entities may be separated and placed in other components. According to various embodiments, one or more components or operations of the above-described components may be omitted, or one or more other components or operations may be added. Additionally or alternatively, a plurality of components (e.g., a module or a program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each component of the plurality of components in a manner identical to or similar to that performed by the corresponding component among the plurality of components prior to the integration.

[0198] According to various embodiments, the operations performed by a module, program, or other component may be performed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be performed in a different order, omitted, or one or more other operations may be added.

[0199] The methods according to the embodiments described in the claims or specification of the present disclosure may be implemented in the form of hardware, software, or a combination of hardware and software.

[0200] When implemented in software, a computer-readable storage medium storing one or more programs (software modules) may be provided. The one or more programs stored in the computer-readable storage medium are configured for execution by one or more processors within an electronic device. The one or more programs include instructions that cause the electronic device to execute methods according to the embodiments described in the claims or specification of the present disclosure.

[0201] In the present disclosure, a function or operation performed by an electronic device may be performed by one or more processors executing one or more instructions stored in a memory. The function or operation of the electronic device mentioned in the present disclosure may be performed by one processor executing one or more instructions, or may be performed by a combination of multiple processors executing one or more instructions. The processor mentioned in the present disclosure may be understood to include a circuit for performing calculations or controlling other components of the electronic device. For example, the one or more processors may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), a micro controller unit (MCU), a sensor hub, a supplementary processor, a communication processor, an application processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a neural processing unit (NPU), a system on a chip (SoC), or an integrated circuit implemented to execute one or more instructions, and may have a plurality of cores.

[0202] In the present disclosure, a program (software module, software) may be stored in a non-volatile memory including a random access memory, a flash memory, a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic disc storage device, a compact disc ROM (CD-ROM), a digital versatile disc (DVD) or other forms of optical storage devices, a magnetic cassette. Or, it may be stored in a memory formed by a combination of some or all of these. The memory may be formed by a single storage medium or may be formed by a combination of multiple storage media. The one or more commands may be stored in a single storage medium or may be distributed and stored in multiple storage media.

Claims

1. In electronic devices, A communication circuit that transmits and receives data with an external device; at least one processor; and Contains memory for storing commands, The above instructions are individually or collectively executed by the at least one processor so that the electronic device: Acquire a first set of input images of at least a part of the user's body at a first point in time, Detecting at least one first nodule for the first input image set, Determining the similarity between at least one first nodule and at least one second nodule, wherein the at least one second nodule is obtained based on a second input image set stored in the memory; and If the similarity is greater than or equal to a preset value, the first input image set and the second input image set are input into a risk assessment model to obtain risk information for at least one first nodule. An electronic device, wherein the risk assessment model is an artificial intelligence model trained to output risk information of the at least one first nodule based on first size information of the at least one first nodule, first type information for the at least one first nodule, and change amount information of the at least one first nodule and the at least one second nodule.

2. In claim 1, The above instructions are individually or collectively executed by the at least one processor so that the electronic device: Inputting the first input image set to a nodule detection model to detect at least one first nodule of at least a part of the user's body, Inputting the second input image set into the nodule detection model to detect at least one second nodule, and An electronic device wherein the above nodule detection model is an artificial intelligence model trained to detect at least one nodule included in at least a part of the user's body based on an input image set.

3. In claim 2, The above instructions are individually or collectively executed by the at least one processor so that the electronic device: Through the above nodule detection model, a first evaluation area is specified in the first input image set, and the first size information of at least one first nodule included in the first evaluation area is obtained, Based on the first size information, if the volume of at least one first nodule is greater than the cutoff value, the evaluation area is classified as a first activation area, Through the above nodule detection model, a second evaluation area is specified in the second input image set, and second size information of at least one second nodule included in the second evaluation area is obtained, and An electronic device that classifies the evaluation area as a second activation area based on the second size information when the volume of at least one second nodule is greater than a cutoff value.

4. In claim 3, The above instructions are individually or collectively executed by the at least one processor so that the electronic device, through a similarity judgment model: Generate a first feature map of the first activation area, Generating a second feature map of the second activation area, and To obtain the similarity between the first feature map and the second feature map, An electronic device, wherein the above similarity judgment model is a model learned to judge similarity based on activated regions containing at least one nodule.

5. In claim 4, An electronic device in which the above similarity judgment model determines the similarity between the first feature map and the second feature map through geometric matching.

6. In claim 3, The above instructions are individually or collectively executed by the at least one processor so that the electronic device: Inputting the first activation area into a type detection model to obtain the first type information of the at least one first nodule, and By inputting the second activation area into the type detection model, second type information of at least one second nodule is obtained, An electronic device, wherein the above type detection model is an artificial intelligence model trained to output a type for at least one nodule.

7. In claim 1, The above instructions are individually or collectively executed by the at least one processor so that the electronic device: The first input image set is input into a lung disease detection model to obtain lung disease information indicating whether the user has a lung-related disease, and the lung disease detection model is an artificial intelligence model trained to detect the presence or absence of lung disease by receiving an image including at least a part of the user's body, and An electronic device that obtains risk information of at least one first nodule based on the first size information, the first type information, the change amount information, and the lung disease information through the risk assessment model.

8. In claim 7, The above instructions are individually or collectively executed by the at least one processor so that the electronic device, through the risk assessment model: Obtaining primary risk information of at least one first nodule based on the first size information, the first type information, and the change amount information, and An electronic device that obtains the risk information by correcting the first risk information based on the lung disease information.

9. In claim 1, An electronic device, wherein the second input image set is a set of images acquired at at least one point in time that is distinct from the first point in time.

10. In claim 1, The above instructions are individually or collectively executed by the at least one processor so that the electronic device: An electronic device that transmits the risk information to the external device so as to output the risk information through the display of the external device.

11. In the method of operating an electronic device, An act of acquiring a first set of input images of at least a portion of the user's body at a first point in time; An operation of detecting at least one first nodule for the first input image set; An operation of determining a similarity between at least one first nodule and at least one second nodule; wherein the at least one second nodule is obtained based on a second input image set stored in the electronic device; and If the similarity is greater than or equal to a preset value, an operation of inputting the first input image set and the second input image set into a risk judgment model to obtain risk information for at least one first nodule is included. A method for operating an electronic device, wherein the risk assessment model is an artificial intelligence model trained to output risk information of the at least one first module based on first size information of the at least one first module, first type information for the at least one first module, and change amount information of the at least one first module and the at least one second module.

12. In claim 11, The operation of detecting at least one first nodule includes an operation of inputting the first input image set into a nodule detection model to detect the at least one first nodule of at least a part of the user's body, and The operation of detecting at least one second nodule includes an operation of inputting the second input image set into the nodule detection model to detect the at least one second nodule, A method of operating an electronic device, wherein the nodule detection model is an artificial intelligence model trained to detect at least one nodule included in at least a part of the user's body based on an input image set.

13. In claim 12, The operation of detecting at least one first nodule is: An operation of specifying a first evaluation area in the first input image set through the above nodule detection model and obtaining the first size information of at least one first nodule included in the first evaluation area; and Based on the first size information, an operation is included to classify the evaluation area as a first activation area when the volume of at least one first nodule is greater than a cutoff value. The operation of detecting at least one second nodule is: An operation of specifying a second evaluation area in the second input image set through the above nodule detection model and obtaining second size information of at least one second nodule included in the second evaluation area; and An operating method of an electronic device, comprising an operation of classifying the evaluation area as a second activation area based on the second size information, when the volume of the at least one second nodule is greater than a cutoff value.

14. In claim 13, The above similarity judgment operation is performed through a similarity judgment model: An operation of generating a first feature map of the first activated region; An operation of generating a second feature map of the second activated area; and Further comprising an operation of obtaining similarity between the first feature map and the second feature map, A method of operating an electronic device, wherein the above similarity judgment model is an artificial intelligence model learned to judge similarity based on activated areas containing at least one nodule.

15. A non-transitory computer-readable recording medium including a program for executing a control method of an electronic device, wherein the control method is: A step of acquiring a first set of input images of at least a part of the user's body at a first point in time; A step of detecting at least one first nodule for the first input image set; A step of determining the similarity between at least one first nodule and at least one second nodule; wherein the at least one second nodule is obtained based on a second input image set stored in the electronic device; and If the similarity is greater than or equal to a preset value, a step of inputting the first input image set and the second input image set into a risk assessment model to obtain risk information for at least one first nodule is included. A computer-readable recording medium, wherein the risk assessment model is an artificial intelligence model trained to output risk information of the at least one first nodule based on first size information of the at least one first nodule, first type information for the at least one first nodule, and change amount information of the at least one first nodule and the at least one second nodule.

Images