Artificial intelligence-based associated image providing device, method and computer program including same

Abstract

The present invention relates to a device for providing an artificial intelligence-based associated image, wherein the device is an electronic device comprising: a memory including at least one instruction; and at least one processor electrically connected to the memory and configured to perform the at least one instruction. The at least one processor may perform the steps of: receiving a first image set and a second image set including information about an abnormal part of a user; providing a portion of image data selected from the second image set; and providing image data associated with the portion of image data of the first image set to be simultaneously displayed, wherein image data, which is not associated with the first image set, of the second image set may be labeled.

Description

Artificial intelligence-based related image providing device, method and computer program including the same

[0001] The present invention relates to a technology for providing related images based on artificial intelligence.

[0002]

[0003] Lung cancer is the most common cancer worldwide, making early detection and continuous monitoring crucial. Computed tomography (CT) is a key imaging technique for diagnosing and monitoring lung cancer. CT images allow healthcare professionals to obtain detailed images of lung tissue and identify nodules. While existing technologies can capture nodules in CT images, they remain limited in analyzing changes over time and assessing relationships between nodules.

[0004] Specifically, current technologies can identify nodules at a single point in time, but are limited in effectively comparing nodule changes across images from multiple points in time, and especially in identifying whether a new nodule is derived from an existing nodule or is newly generated.

[0005] Furthermore, existing technologies struggle to simultaneously visualize and analyze related nodules, hindering medical professionals from comprehensively understanding the relationships between nodules and their progression. These limitations can reduce the accuracy of lung cancer diagnosis and management.

[0006]

[0007] One embodiment of the present invention relates to an electronic device capable of tracking changes in lesions contained in images having time intervals.

[0008] One embodiment of the present invention relates to an electronic device capable of analyzing images with time intervals to identify each lesion and calculate a similarity for the lesion location.

[0009] One embodiment of the present invention relates to an electronic device capable of analyzing images with time intervals to determine whether a newly formed lesion is derived from an existing lesion or whether it has newly developed.

[0010] One embodiment of the present invention relates to an electronic device capable of simultaneously providing related images.

[0011] One embodiment of the present invention relates to an electronic device capable of providing labeled images, whether related or not.

[0012]

[0013] An artificial intelligence-based associated image providing device according to one embodiment of the present invention is an electronic device, comprising: a memory including at least one instruction; and at least one processor electrically connected to the memory and configured to perform the at least one instruction; wherein the at least one processor includes an operation of receiving a first image set and a second image set including information about an abnormal part of a user; an operation of providing some image data selected from the second image set; and an operation of providing image data associated with the some image data from the first image set to be displayed simultaneously, wherein image data from the second image set that is not associated with the first image set may be labeled.

[0014] The first image set may be a set of data obtained by analyzing an image of the user taken at a first point in time through the first artificial intelligence model, and the second image set may be a set of data obtained by analyzing an image of the user taken at a second point in time through the first artificial intelligence model, the second point in time being a point in time after the first point in time.

[0015] The first image set may include information on one or more of the size, condition, location, or category of the abnormal area.

[0016] The at least one processor may indicate the location of the abnormal area on the image and provide the partial image data and the associated image data.

[0017] The at least one processor may provide the partial image data and the associated image data by displaying them in at least one direction among a sagittal plane, a coronal plane, and a horizontal plane.

[0018] The at least one processor may compare, through a second artificial intelligence model, an abnormal part of the user included in the partial image data with an abnormal part of the user included in the second image set to determine image data associated with the partial image data.

[0019] The at least one processor can determine, through the second artificial intelligence model, new nodule data that is not related to the first image set among the second image set, and label the new nodule data.

[0020] The at least one processor may determine, through the second artificial intelligence model, extinction nodule data that is not related to the second image set among the first image set, and label the extinction nodule data.

[0021] A method for providing an artificial intelligence-based related image according to one embodiment of the present invention comprises the steps of: receiving a first image set and a second image set including information on an abnormal part of a user by a memory and at least one processor electrically connected to the memory; providing some image data selected from the second image set; and providing image data associated with the some image data from the first image set to be displayed simultaneously, wherein image data from the second image set that is not associated with the first image set may be labeled.

[0022] The present invention can be implemented as a computer program recorded on a computer-readable recording medium to perform an artificial intelligence-based related image providing method according to one embodiment of the present invention.

[0023]

[0024] The disclosed technology may have the following effects. However, this does not mean that a particular embodiment must include all or only the following effects, and thus the scope of the disclosed technology should not be construed as being limited thereby.

[0025] An electronic device according to one embodiment of the present invention is capable of tracking changes in a lesion included in images having time intervals.

[0026] An electronic device according to one embodiment of the present invention can analyze images with time intervals to identify each lesion and calculate a similarity for the lesion location.

[0027] An electronic device according to one embodiment of the present invention can analyze images with time intervals to determine whether a newly generated lesion is derived from an existing lesion or newly generated.

[0028] An electronic device according to one embodiment of the present invention is capable of simultaneously providing related images.

[0029] An electronic device according to one embodiment of the present invention can provide images that are related or not while being labeled.

[0030]

[0031] FIG. 1 is a drawing illustrating a system of the present invention according to one embodiment of the present invention.

[0032] FIG. 2 is a drawing illustrating the physical configuration of the present invention according to one embodiment of the present invention.

[0033] FIG. 3 is a drawing illustrating a functional configuration of the present invention according to one embodiment of the present invention.

[0034] FIG. 4 is a diagram illustrating a functional configuration according to one embodiment of a novel nodule detection device based on a geometric matching map.

[0035] FIG. 5 is a drawing illustrating a captured image according to one embodiment of the present invention.

[0036] FIG. 6 is a drawing illustrating a method for determining a nodule set according to one embodiment of the present invention.

[0037] FIG. 7 is a diagram illustrating a method for determining similarity between feature maps according to one embodiment of the present invention.

[0038] FIG. 8 is a drawing illustrating a method for classifying a new nodule according to one embodiment of the present invention.

[0039] FIG. 9 is a diagram illustrating a sequence according to one embodiment of a novel nodule detection method based on a geometric matching map.

[0040] FIG. 10 is a diagram illustrating a functional configuration according to one embodiment of an artificial intelligence-based related image providing device.

[0041] FIG. 11 is a drawing illustrating a method for displaying an image set according to one embodiment of the present invention.

[0042] FIG. 12 is a diagram illustrating a method for determining image data associated with some image data according to one embodiment of the present invention.

[0043] Figure 13 is a diagram illustrating a sequence according to one embodiment of a method for providing related images based on artificial intelligence.

[0044]

[0045] The description of the present invention is merely an example for structural and functional explanation, and therefore, the scope of the present invention should not be construed as being limited by the embodiments described in the text. That is, since the embodiments can be modified in various ways and can take various forms, the scope of the present invention should be understood to include equivalents that can realize the technical idea. In addition, the purposes or effects presented in the present invention do not mean that a specific embodiment must include all of them or only such effects, and therefore, the scope of the present invention should not be construed as being limited thereby.

[0046] Meanwhile, the meaning of the terms described in this application should be understood as follows.

[0047] Terms such as "first" and "second" are intended to distinguish one component from another, and the scope of the rights should not be limited by these terms. For example, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

[0048] When a component is said to be "connected" to another component, it should be understood that while it may be directly connected to that other component, there may also be other components intervening. Conversely, when a component is said to be "directly connected" to another component, it should be understood that there are no other intervening components. Similarly, other expressions describing relationships between components, such as "between" and "directly between," or "adjacent to" and "directly adjacent to," should be interpreted similarly.

[0049] Singular expressions should be understood to include plural expressions unless the context clearly indicates otherwise, and terms such as "comprises" or "have" should be understood to specify the presence of a feature, number, step, operation, component, part or combination thereof, but not to exclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts or combinations thereof.

[0050] For each step, the identifiers (e.g., a, b, c, etc.) are used for convenience of explanation and do not describe the order of the steps. The steps may occur in a different order than stated unless the context clearly dictates a specific order. That is, the steps may occur in the same order as stated, may be performed substantially simultaneously, or may be performed in the opposite order.

[0051] The present invention can be implemented as computer-readable code on a computer-readable recording medium, and the computer-readable recording medium includes all types of recording devices that store data that can be read by a computer system. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc., and also includes those implemented in the form of a carrier wave (e.g., transmission via the Internet). Furthermore, the computer-readable recording medium can be distributed across network-connected computer systems, so that the computer-readable code can be stored and executed in a distributed manner.

[0052] Unless otherwise defined, all terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted to be consistent with their meaning within the context of the relevant technology, and should not be interpreted as having an idealized or overly formal meaning unless explicitly defined herein.

[0053]

[0054] A novel nodule detection device based on a geometric matching map according to one embodiment of the present invention is an electronic device, comprising: a memory including at least one instruction; and at least one processor electrically connected to the memory and configured to perform the at least one instruction; wherein the at least one processor is capable of performing the following operations: receiving a first image captured at a first point in time and a second image captured at a second point in time; determining a first nodule set included in the first image and a second nodule set included in the second image through a first artificial intelligence model; determining a similarity between the first nodule set and the second nodule set through a second artificial intelligence model; and determining an activated area among the second nodule set, which does not have a similarity higher than a preset reference similarity, as a new nodule.

[0055] The above second time point may be later in time than the above first time point.

[0056] The at least one processor can determine, through the first artificial intelligence model, a set of nodules including an activation area, the activation area including an abnormal area and a surrounding area.

[0057] The above first artificial intelligence model can be trained with a dataset including abnormal areas to determine the activation area.

[0058] The above-mentioned activation area can be created with a preset size.

[0059] The second artificial intelligence model can generate a feature map of the activated area and determine the similarity between the feature maps.

[0060] The second artificial intelligence model can determine the similarity between the feature maps through geometric matching.

[0061] The at least one processor may perform an operation of determining, through a third artificial intelligence model, whether the new nodule has been spread from a nodule included in the first nodule set.

[0062] The third artificial intelligence model can analyze the texture of the new nodule to determine whether the new nodule is diffused from a nodule included in the first nodule set.

[0063] A novel nodule detection method based on a geometric matching map according to one embodiment of the present invention may include the steps of: receiving an input image by a memory and at least one processor electrically connected to the memory; determining a volume of an abnormal region included in the evaluation region by specifying an evaluation region of the input image through a first artificial intelligence model; classifying the evaluation region as an inactive region when the volume of the abnormal region is less than or equal to a cutoff value; and determining whether the evaluation region is an active region through a second artificial intelligence model corresponding to the volume of the abnormal region when the volume of the abnormal region is greater than the cutoff value.

[0064] A novel nodule detection method based on a geometric matching map according to one embodiment of the present invention can be implemented as a computer program recorded on a computer-readable recording medium.

[0065]

[0066] FIG. 1 is a diagram illustrating a system of the present invention according to one embodiment of the present invention. Referring to FIG. 1, the system (100) may include a user terminal (110), an electronic device (130), and a database (150).

[0067] The user terminal (110) may be implemented as a smartphone or wearable device capable of checking data generated by the electronic device (130) and metadata analyzed therefrom, but is not necessarily limited thereto and may also be implemented as various devices such as a tablet PC. The user terminal (110) may be connected to the electronic device (130) via a network, and multiple user terminals (110) may be connected to the electronic device (130) simultaneously.

[0068] The electronic device (130) may include a first electronic device (131) and a second electronic device (132). The first electronic device (131) may be a novel nodule detection device based on a geometric matching map. The second electronic device (132) may be an artificial intelligence-based associated image providing device. The electronic device (130) may be provided by being included in a computer-readable recording medium by tangibly implementing a program of commands for implementing the same. In other words, the electronic device (130) may be implemented in the form of program commands that may be executed through various computer means and may be recorded in a computer-readable recording medium. In addition, the electronic device (130) may be configured as a computer program that sequentially or non-sequentially performs operations of receiving a first image and a second image and analyzing the same, and the computer program may be stored in a computer-readable recording medium.

[0069] The database (150) may correspond to a storage device that stores various pieces of information generated through an electronic device (130).

[0070]

[0071] FIG. 2 is a diagram illustrating a physical configuration of an electronic device (130) according to one embodiment. Referring to FIG. 2, the electronic device (130) may be implemented to include a processor (210), a memory (230), a user input / output unit (250), and a network input / output unit (270).

[0072] The processor (210) may include at least one processor implemented to provide at least some different functions. The processor (210) may control the overall operation of the electronic device (130) and may be electrically connected to the memory (230), the user input / output unit (250), and the network input / output unit (270) to control data flow therebetween. The processor (210) may be implemented as a CPU (Central Processing Unit) of the electronic device (130). According to one embodiment, the processor (210) may include a main processor (e.g., a central processing unit or an application processor) or an auxiliary processor (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor) that can operate independently of or together with the main processor. For example, when the electronic device (130) includes a main processor and an auxiliary processor, the auxiliary processor may be configured to use lower power than the main processor or to be specialized for a given function. The auxiliary processor may be implemented separately from the main processor or as a part thereof. The auxiliary processor may control at least a portion of functions or states associated with at least one of the components of the electronic device (130), for example, on behalf of the main processor while the main processor is in an inactive (e.g., sleep) state, or together with the main processor while the main processor is in an active (e.g., application execution) state. In one embodiment, the auxiliary processor (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component. In one embodiment, the auxiliary processor (e.g., neural network processing unit) may include a hardware structure specialized for processing artificial intelligence models. The artificial intelligence models may be generated through machine learning.This learning can be performed, for example, in the electronic device (130) itself where the artificial intelligence model is executed, or can be performed through a separate server. The learning algorithm can include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model can include a plurality of artificial neural network layers. The artificial neural network can be one of a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model can additionally or alternatively include a software structure. Meanwhile, the operation of the electronic device (130) described below can be understood as the operation of the processor (210).

[0073] The memory (230) may include an auxiliary memory device implemented with a non-volatile memory such as an SSD (Solid State Drive) or an HDD (Hard Disk Drive) and used to store all data required for the electronic device (130), and may include a main memory device implemented with a volatile memory such as a RAM (Random Access Memory). In addition, the memory (230) may include a plurality of instructions that direct the operations of the processor (210) to implement the functions provided by the service. In this case, the processor (210) may include a software server that executes the functions provided by the service based on the plurality of instructions stored in the memory (230).

[0074] The user input / output unit (250) may include an environment for receiving user input and an environment for outputting specific information to the user. For example, the user input / output unit (250) may include an input device including an adapter such as a touchpad, a touch screen, a virtual keyboard, or a pointing device, and an output device including an adapter such as a monitor or a touch screen. In one embodiment, the user input / output unit (250) may correspond to a computing device accessed via a remote connection, in which case the electronic device (130) may function as a server.

[0075] The network input / output unit (270) includes an environment for connecting to an external device or system via a network, and may include an adapter for communication such as a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), and a value added network (VAN).

[0076]

[0077] FIG. 3 is a diagram illustrating the functional configuration of an electronic device (130) according to one embodiment of the present invention. As described above, the electronic device (130) may include a first electronic device (131) and a second electronic device (132). The first electronic device (131) may be a novel nodule detection device based on a geometric matching map. The second electronic device (132) may be an artificial intelligence-based associated image providing device.

[0078]

[0079] FIG. 4 is a diagram illustrating the functional configuration of a novel nodule detection device based on a geometric matching map according to one embodiment of the present invention. Referring to FIG. 4, the novel nodule detection device based on a geometric matching map may include an image receiving unit (410), a nodule determination unit (430), a similarity determination unit (450), a new nodule determination unit (470), and a nodule diffusion determination unit (490).

[0080] The image receiving unit (410) can receive a first image (511) captured at a first point in time and a second image (512) captured at a second point in time. The first image (511) and the second image may be input images (510) composed of a plurality of slides. The first image (511) may be captured at a first point in time and analyzed. Referring to FIGS. 5 and 6, the first image (511) and the second image (512) may be images of the user's body captured by MRI, CT, X-ray, etc. In addition, the first image (511) and the second image (512) may include a plurality of slides. Specifically, the first image (511) and the second image (512) may include a plurality of slides and may be 3D images. For example, the first image (511) and the second image (512) may include a plurality of slides including a specific nodule. The first image (511) and the second image (512) may be images captured in the same format. The first point in time may be the point in time when the first image (511) is captured, and may be the day on which the first capture of the user is performed. The image receiving unit (410) may receive the first image (511) captured at the first point in time, and may receive the second image (512) captured at a second point in time that is later than the first point in time.

[0081] The nodule determination unit (430) can determine the first nodule set (511a) included in the first image (511) and the second nodule set (512a) included in the second image (512) through the first artificial intelligence model (610). First, the artificial intelligence models described herein can include a first artificial intelligence model (610), a second artificial intelligence model (620), and a third artificial intelligence model (630). Here, each artificial intelligence model can include multiple models, and for convenience, can be understood as an artificial intelligence model that performs a specific function. The first artificial intelligence model (610) can specify an evaluation area included in the first image (511) and the second image (512). In addition, the first artificial intelligence model (610) can determine an abnormal area (521) included in the evaluation area, and determine the volume of the abnormal area (521). In addition, the first artificial intelligence model (610) can classify the evaluation area as an inactive area if the volume of the abnormal area (521) included in the evaluation area is less than or equal to the cutoff value. Conversely, the first artificial intelligence model (610) can classify the evaluation area as an activated area (520) if the volume of the abnormal area (521) is greater than or equal to the cutoff value. Here, the activated area (520) can be the image itself cropped to the evaluation area and can be determined to a preset size, which will be described in detail below. Here, the nodule set can be a set of activated area data including nodules, i.e., the abnormal area (521). The first nodule set (511a) and the second nodule set (512a) are data determined from images captured at the first time and the second time, respectively, and can each include at least one piece of activated area data.

[0082] In one embodiment, the nodule determination unit (430) can determine a nodule set including an activated region (520) - the activated region (520) including an abnormal region (521) and a surrounding region - through the first artificial intelligence model (610). As described above, the activated region (520) can include a segmented abnormal region (521). In addition, the activated region (520) can include an abnormal region (521) included within a preset size and a surrounding region thereof. Through this, a similarity between the activated regions (520) can be calculated, which will be described in detail below. That is, the nodule determination unit (430) can determine an activated region (520) including an abnormal region and a surrounding region through the first artificial intelligence model (610), and determine a nodule set including these activated regions (520).

[0083] Here, the first artificial intelligence model (610) can be trained with a dataset including an abnormal region (521) to determine an activation region (520). The first artificial intelligence model (610) can be trained in various ways and is not limited to any one method. In addition, the first artificial intelligence model (610) includes a first sub-AI model and a second sub-AI model, and each of the sub-AI models can be trained in various ways. Specifically, the first sub-AI model can be trained to automatically specify an evaluation region including an abnormal region. For example, the first sub-AI model can be trained to identify the presence of lung nodules, tumors, or abnormal tissues in medical images such as computed tomography (CT) or magnetic resonance imaging (MRI). As a specific example, the first sub-AI model can learn the characteristics of an abnormal region (521) in an image and the distinguishing factors from the surrounding normal tissues through a deep learning method on an annotated medical image dataset. Additionally, the second sub-AI model can be trained to calculate the volume of the abnormal area (521). Here, for the purpose of explanation, the first AI model (610) is divided into the first sub-AI model and the second sub-AI model, but in the implementation of the present invention, this can be designed as a single integrated model.

[0084] As described above, the activation area (520) can be created with a preset size. The activation area (520) can include the abnormal area (521) and the surrounding area, and can be created with a preset size. For example, the activation area (520) can be created with a size of 10 mm x 10 mm. As another example, the activation area (520) can be created with a size of 15 mm x 5 mm. The activation area (520) created in this way can be the target of similarity determination.

[0085] Referring to FIG. 7, the similarity determination unit (450) can determine the similarity between the first nodule set (511a) and the second nodule set (512a) through the second artificial intelligence model (620). The similarity determination unit (450) can determine the similarity between the first nodule set (511a) and the second nodule set (512a) by comparing at least one activated region included in the first nodule set (511a) with at least one activated region included in the second nodule set (512a). For example, the similarity determination unit (450) may compare the first activation area and the second activation area included in the first nodule set (511a) with the third activation area and the fourth activation area included in the second nodule set (512a) to determine whether the first activation area and the third activation area are similar, determine whether the first activation area and the fourth activation area are similar, determine whether the second activation area and the third activation area are similar, and determine whether the second activation area and the fourth activation area are similar. Here, the second artificial intelligence model (620) may be trained in various ways and is not limited to any one way. Specifically, the second artificial intelligence model may be trained to determine the similarity between the activation areas.

[0086] Through this, the similarity determination unit (450) can classify the activation areas determined to be similar to any one of the activation areas of the first nodule set (511a) among the second nodule set (512a) into a similar nodule set (710), and classify the activation areas determined to be similar to all activation areas included in the first nodule set (511a) among the second nodule set (512a) into a dissimilar nodule set (720). In addition, as another example, the similarity determination unit (450) can classify the activation areas determined to be similar to the activation areas included in the second nodule set (512a) among the first nodule set (511a) into a similar nodule set (710), and classify the activation areas determined to be dissimilar to the activation areas included in the second nodule set (512a) among the first nodule set (511a) into a dissimilar nodule set (720).

[0087] Here, the second artificial intelligence model (620) can generate a feature map of the activated area and determine the similarity between the feature maps. Here, the feature map can be a set of information that emphasizes a specific feature of the image and can be extracted through a convolutional neural network. The second artificial intelligence model (620) can determine the similarity between the generated feature maps. For example, the second artificial intelligence model (620) can generate feature maps of the first activated area and the second activated area included in the first nodule set (511a) and the third activated area and the fourth activated area included in the second nodule set (512a), respectively, and compare them to determine the similarity. More specifically, the second artificial intelligence model (620) can determine the similarity by comparing the feature map of the first activated area included in the first nodule set (511a) with the feature map of the third activated area included in the second nodule set (512a). Here, the method of determining similarity is not limited to a specific method and can be performed in various ways, such as statistical methods, distance-based methods, or machine learning methods.

[0088] In addition, the second artificial intelligence model (620) can determine the similarity between feature maps through geometric matching. The similarity determination unit (450) can determine the similarity between feature maps based on geometric matching through the second artificial intelligence model. That is, the similarity determination unit (450) can determine the similarity between at least one activated region included in the first nodule set (511a) and at least one activated region included in the second nodule set (512a) through geometric matching. Here, the geometric matching can determine the similarity between feature maps using various deformation models. The second artificial intelligence model (620) can determine the similarity between activated regions (520) by considering not only the similarity of the abnormal region (521) included in the activated region (520) but also the similarity of the surrounding regions through geometric matching. The second artificial intelligence model (620) can determine the similarity between multiple feature maps by geometrically transforming the feature map through a transformation model of the feature map.

[0089] Referring to FIG. 8, the new nodule determination unit (470) may determine an activated area having a similarity higher than or equal to a preset standard similarity among the second nodule set (512a) as a new nodule (720a). The new nodule determination unit (470) may determine an activated area having a similarity lower than or equal to a preset standard similarity among the second nodule set (512a) as a new nodule based on the similarity determined through the second artificial intelligence model (620). For example, the new nodule determination unit (470) may compare a third activated area among the second nodule set with a first activated area and a second activated area among the first nodule set, and if none of the calculated similarities exceed a preset value, the third activated area may be determined as a new nodule. For another example, the new nodule determination unit (470) may compare the fourth activation area among the second nodule set with the first activation area and the second activation area among the first nodule set, and if the similarity between the fourth activation area and the first activation area among the calculated similarities exceeds a preset standard and the similarity between the fourth activation area and the second activation area is below the preset standard, the fourth activation area may be determined not to be a new nodule. Here, the preset standard may be determined by the controller.

[0090] Through the third artificial intelligence model (630) of the nodule diffusion determination unit (490), it can be determined whether a new nodule has spread from a nodule included in the first nodule set. In addition, through the third artificial intelligence model (630) of the nodule diffusion determination unit (490), it can be determined whether a new nodule is a generated nodule that has occurred regardless of metastasis. Here, the third artificial intelligence model (630) can be trained in various ways and is not limited to any one way. Specifically, the third artificial intelligence model can be trained to compare the properties of the new nodule with the properties of an existing nodule. For example, the third artificial intelligence model (630) can determine whether a new nodule has spread from an existing nodule or has been newly generated by considering how close the new nodule is to a nodule included in the first nodule set, whether it exhibits the shape of a metastasized cancer, the density and texture of the nodule, the passage of time, etc.

[0091] Here, the third artificial intelligence model (630) can analyze the texture of the new nodule to determine whether the new nodule is diffused from a nodule included in the first nodule set. For example, if the new nodule exhibits an irregular or rough texture, the third artificial intelligence model (630) can determine that the new nodule is a generated nodule. For another example, if the new nodule has a similar texture and density to existing nodules within a certain distance, the third artificial intelligence model (630) can determine that the new nodule is a diffused nodule.

[0092] Fig. 9 is a diagram illustrating a sequence according to one embodiment of a novel nodule detection method based on a geometric matching map. The novel nodule detection method based on a geometric matching map can receive an input image (S910).

[0093] A novel nodule detection method based on a geometric matching map can determine the volume of an abnormal area included in the evaluation area by specifying the evaluation area of ​​the input image through a first artificial intelligence model (S920).

[0094] A novel nodule detection method based on a geometric matching map can classify the evaluation area as an inactive area if the volume of the abnormal area is less than or equal to a cutoff value (S930).

[0095] A novel nodule detection method based on a geometric matching map can determine whether the evaluation area is an activated area through a second artificial intelligence model corresponding to the volume of the abnormal area when the volume of the abnormal area is greater than a cutoff value (S940).

[0096]

[0097] Referring to FIG. 10, an artificial intelligence-based associated image providing device may include an image set receiving unit (1010), an image data providing unit (1030), and an image data labeling unit (1050).

[0098] The image set receiving unit (1010) may receive a first image set and a second image set that include information about abnormal areas of a user. Each image set may be a set of image slides that include an activated area for a specific user. In other words, an image set may be a set of at least one slide that includes an activated area among images (slides) for a specific user. In addition, an image set may be a concept that includes not only a simple image, but also information such as the size of an image and a nodule included in the image, which will be described in detail below.

[0099] In one embodiment, the first image set (1120) may be a set of data analyzed from an image of the user captured at a first point in time using the first artificial intelligence model (610). Here, the first artificial intelligence model (610) may determine an activated area (520) containing an abnormal area (521) within the user's video or image, as described above. In addition, the first artificial intelligence model (610) may determine the size, condition, location, etc. of the abnormal area (521), i.e., the nodule. In addition, the first point in time may be a point in time prior to the second point in time, as described above, and may be the time when the user's image is first acquired.

[0100] In one embodiment, the first image set may include information on any one or more of the size, condition, location, or category of the abnormal area (521). The image set receiving unit (1010) may receive the first image set including information on the abnormal area (521) analyzed through the first artificial intelligence model (610). The first image set (1120) may include all of information on at least one abnormal area (521), including an image of the abnormal area (521), and information on the size, condition, location, and category of the abnormal area (521). For example, the first image set (1120) may include an image of a specific abnormal area and information indicating that the size of the abnormal area is 5.0 mm, the attribute of the abnormal area is calcification, the abnormal area is located in the LUL (left upper eyelid), and the category of the abnormal area corresponds to 1. The second image set may also include all of the information similar to the first image set. Accordingly, the processor can analyze the abnormal area (521) through the first artificial intelligence model (610) and transmit the analyzed information to the image set receiving unit (1010).

[0101] Here, the second image set (1130) may be a set of data analyzed by using the first artificial intelligence model (610) for images of the user captured at a second point in time, the second point in time being a point after the first point in time. The second image set (1130), like the first image set (1120), may include information about the same user, but may be data analyzed at a different time for data captured at a different time. However, the second image set (1130) may include sets of data analyzed by using the same artificial intelligence model used for the analysis in the first image set (1120). Additionally, as an exception, the second image set (1130) may include sets of data analyzed by using a different model from the artificial intelligence model used for the analysis in the first image set (1120). The second image set (1130), like the first image set (1120), may also include information about at least one abnormal area of ​​the user.

[0102] Referring to FIG. 11, the image data providing unit (1030) may provide some image data selected from the second image set (1130). In addition, the image data providing unit (1030) may provide at least one of the user's name, age, or disease stage related to the first image set (1120) and the second image set (1130) as user data (1110). When any one of the at least one abnormal area included in the second image set (1130) is selected, the image data providing unit (1030) may provide the selected image data, i.e., a slide image. By providing some image data selected from the second image set (1130), the image data providing unit (1030) may cause the user input / output unit (250) to display the corresponding some images. Here, the selection may be performed by a controller. For example, the image data providing unit (1030) can display some image data selected from the second image set (1130) as second image data (1131), and simultaneously display first image data (1121) that is data associated with the selected some image data, which will be described in detail below.

[0103] In one embodiment, the image data providing unit (1030) may provide image data and associated image data by indicating the location of an abnormal area on an image. Specifically, the image data providing unit (1030) may provide image data by indicating the location of the abnormal area determined through the first artificial intelligence model (610). For example, when displaying the second image data (1131) from the second image set (1130), the image data providing unit (1030) may indicate and provide the second abnormal area location (1131a). As another example, when displaying the second image data (1131) associated with the second image data (1131), the image data providing unit (1030) may indicate and provide the first abnormal area location (1121a). Here, the associated image data may be determined through the second artificial intelligence model (620), which will be described in detail below.

[0104] In one embodiment, the image data providing unit (1030) may provide some image data and associated image data by displaying them in at least one direction among the sagittal plane, the coronal plane, and the axial plane. The image data providing unit (1030) may display some image data and associated image data in at least one direction among the sagittal plane, the coronal plane, and the axial plane, and may display them in two or more ways simultaneously. In addition, the image data providing unit (1030) may provide some image data and associated image data in an adjustable oblique plane. For example, the image data providing unit (1030) may display the second image data centered on the sagittal plane, but may display the coronal plane and the horizontal plane at different locations from the sagittal plane. Through this implementation of the present invention, it may be possible for experts to more easily compare disease data of users with a time difference.

[0105] In one embodiment, the image data providing unit (1030) may provide image data associated with some image data of the first image set (1120) to be displayed simultaneously. As described above, the image data providing unit (1030) may provide first image data (1121) associated with second image data (1131) included in the second image set (1130) to be displayed simultaneously.

[0106] Referring to FIG. 12, image data associated with some image data may be image data determined to be associated with some image data by comparing an abnormal part of the user included in some image data with an abnormal part of the user included in a second image set (1130) through a second artificial intelligence model (620). As described above, the second artificial intelligence model (620) may determine the similarity between the abnormal part included in a first image captured at a first point in time and a second image captured at a second point in time. Specifically, the image data providing unit (1030) may determine first image data (1121) that shows a similarity higher than a preset similarity with the second image data (1131) through the second artificial intelligence model (620), and provide the first image data (1121) as data associated with the second image data (1131).

[0107] The image data labeling unit (1050) can label (1132) image data that is not related to the first image set (1120) among the second image set (1130). For example, the image data labeling unit (1050) can label abnormal areas that are related to abnormal areas included in the first image set (1120) among the second image set (1130) without labeling them, and the image data labeling unit (1050) can label abnormal areas that are not related to abnormal areas included in the first image set (1120) among the second image set (1130). Through this, the image data that is not related to the first image set (1120) can be labeled (1132) and provided to the user. For example, the image data labeling unit (1050) can label the second image data when the second image data is not related to at least one abnormal area included in the first image set (1120).

[0108] Here, the image data unrelated to the first image set (1120) may be a new nodule unrelated to the first image set (1120) among the second image set (1130) determined by the second artificial intelligence model (620). As described above, the second artificial intelligence model (620) may detect similarity by comparing at least one abnormal region included in the second image set with at least one abnormal region included in the first image set. Through this, the image data labeling unit (1050) may label (1122) such new nodule as new nodule data.

[0109] In one embodiment, the image data labeling unit (1050) may determine, through the second artificial intelligence model (620), images among the first image set (1120) that are determined to be unrelated to the second image set (1130) as extinction nodule data, and label (1122) the corresponding extinction nodule data. For example, the image data labeling unit (1050) may compare at least one abnormal area included in the second image set (1130) with first image data included in the first image set, and when the first image data is determined to be dissimilar, label the corresponding first image data as extinction nodule data. Through this, an expert may be able to intuitively recognize that a nodule existed at the first time point but does not exist at the second time point.

[0110]

[0111] Referring to FIG. 13, the artificial intelligence-based related image providing method can receive a first image set and a second image set including information about a user's node (S1310).

[0112] The artificial intelligence-based related image providing method can provide some image data selected from the second image set (S1320).

[0113] The artificial intelligence-based associated image providing method can provide image data related to some image data of the first image set to be displayed simultaneously (S1330).

[0114]

[0115] Although the present invention has been described above with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

Claims

1. As an electronic device, a memory containing at least one instruction; and At least one processor electrically connected to said memory and configured to perform said at least one instruction; At least one processor, An operation of receiving a first image set and a second image set containing information about an abnormal area of ​​a user; An operation of providing some image data selected from the second image set; and An operation for simultaneously providing image data associated with some of the image data among the first image set, Image data among the second image set that is not related to the first image set is labeled, Electronic devices.

2. In paragraph 1, The above first image set is, A set of data analyzed from the user's image taken at the first point in time through the first artificial intelligence model, The second set of images above is, Characterized in that it is a set of data analyzed by the first artificial intelligence model, wherein the second time point is a time point after the first time point, of the user's image captured at a second time point. Electronic devices.

3. In paragraph 1, The above first image set is, characterized by including information on one or more of the size, condition, location or category of the abnormal part; Electronic devices.

4. In paragraph 3, At least one processor, Characterized in that it provides the partial image data and the associated image data by indicating the location of the abnormal part on the image. Electronic devices.

5. In paragraph 4, At least one processor, Characterized in that it provides the partial image data and the associated image data by displaying in at least one direction among the sagittal plane, the coronal plane, and the horizontal plane. Electronic devices.

6. In paragraph 1, At least one processor, A second artificial intelligence model is used to compare the abnormal part of the user included in the partial image data with the abnormal part of the user included in the second image set, thereby determining image data associated with the partial image data. Electronic devices.

7. In paragraph 6, At least one processor, Through the second artificial intelligence model, new nodule data that is not related to the first image set among the second image set is determined, characterized by labeling the above new nodule data, Electronic devices.

8. In paragraph 6, At least one processor, Through the second artificial intelligence model, the extinction nodule data that is not related to the second image set among the first image set is determined, characterized in that the above extinction nodule data is labeled, Electronic devices.

9. By means of a memory and at least one processor electrically connected to the memory, Receiving a first image set and a second image set containing information about an abnormal part of the user; and A step of providing some image data selected from the second image set; and A step of providing image data associated with some of the image data among the first image set to be displayed simultaneously, Image data among the second image set that is not related to the first image set is labeled, method.

10. A computer program recorded on a computer-readable recording medium that is combined with a computer as hardware and enables the method of claim 9 to be performed.

Images