Method and apparatus for providing lesion information

Abstract

This method by which a computing device comprising at least one processor provides lesion information may comprise the steps of: acquiring a medical image; generating at least one prior image from the medical image; and outputting the medical image and the at least one prior image on the basis of a time point corresponding to the at least one prior image. The method for providing lesion information may further comprise the steps of: detecting at least one region of interest from the medical image and the prior image; quantifying lesions in the at least one region of interest; and detecting changes in lesions for each region of interest.

Description

Method and device for providing lesion information

[0001] Embodiments relate to a method and apparatus for providing information about a lesion included in a medical image.

[0002] There are methods for analyzing medical images using various image processing technologies to detect lesions and provide information about findings or diseases. These methods can help medical professionals reduce the time required to analyze medical images and enable more accurate diagnoses.

[0003] Meanwhile, because there are often no images of the same patient in the same atmosphere taken over a long period of time, there are limitations in identifying the condition before the lesion occurred or in identifying the progression of the lesion up to the time the medical image was acquired.

[0004] The embodiments aim to provide a method and device for providing a progression course of a lesion.

[0005] The embodiments aim to provide a method and device for providing quantitative information on lesions by region in medical images.

[0006] However, the scope of the embodiments is not limited to the technical tasks described above, and the scope of the embodiments may be expanded to other technical tasks that can be inferred by a person skilled in the art based on the entire described content.

[0007] A method for providing lesion information may be performed by a computing device including at least one processor, and may include the steps of: acquiring a medical image; generating at least one previous image from the medical image; and outputting the medical image and the at least one previous image based on a time point corresponding to the at least one previous image. The method for providing lesion information may further include the steps of: detecting at least one region of interest from the medical image and the previous image; quantifying lesions within at least one region of interest; and detecting changes in lesions for each region of interest.

[0008] Embodiments may provide methods and devices for providing a progression course of a lesion.

[0009] Embodiments may provide a method and device for providing quantitative information on lesions by region in medical images.

[0010] Figures 1a and 1b illustrate a method for providing lesion information according to one embodiment of the present invention.

[0011] Figure 2 shows fundus images at each time point according to one embodiment of the present invention.

[0012] Figure 3 shows a previous image and a lesion image according to one embodiment of the present invention.

[0013] Figure 4 illustrates a diffusion model according to one embodiment of the present invention.

[0014] FIG. 5 illustrates a method for detecting a region of interest and quantifying a lesion according to one embodiment of the present invention.

[0015] Figure 6 illustrates a lesion according to one embodiment of the present invention.

[0016] Fig. 7 illustrates a lesion information providing device according to one embodiment of the present invention.

[0017] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The detailed description below with reference to the accompanying drawings is intended to illustrate preferred embodiments rather than merely illustrate embodiments that can be implemented according to the present invention. The following detailed description includes details to provide a thorough understanding of the embodiments. However, it will be apparent to those skilled in the art that the embodiments can be practiced without these details.

[0018] While most terms used in this specification are commonly used in the field, some terms were arbitrarily selected by the applicant, and their meanings are described in detail as needed. Therefore, the embodiments should be understood based on the intended meaning of the terms, not simply their names or meanings.

[0019] Figures 1a and 1b illustrate a method for providing lesion information according to one embodiment of the present invention.

[0020] Referring to FIG. 1a, a method for providing lesion information according to one embodiment of the present invention may include a step of acquiring a medical image (S100), a step of generating a previous image (S101), and a step of outputting the generated previous image (S105). The step of generating a previous image (S101) is specifically described in FIG. 2.

[0021] Specifically, a method for providing lesion information performed by a computing device (700) including at least one processor (703) may include the steps of: acquiring a medical image; generating at least one previous image from the medical image; and outputting the medical image and the at least one previous image based on a time point corresponding to the at least one previous image.

[0022] Referring to FIG. 1b, a method for providing lesion information according to an embodiment of the present invention may include a step of acquiring a medical image (S100), a step of generating a previous image (S101), a step of detecting a region of interest (S102), a step of quantifying a lesion (S103), a step of detecting changes by region (S104), and / or a step of outputting an image including the detected changes by region (S106).

[0023] Specifically, the method for providing a lesion may further include the steps of: detecting at least one region of interest in a medical image and a previous image; quantifying a lesion within at least one region of interest; detecting a change in the lesion for each region of interest; and / or outputting an image including a change in the quantified lesion for the region of interest.

[0024] The step of acquiring a medical image (S100) may acquire a medical image containing a progressive lesion. The medical image may refer to a fundus image, but is not necessarily limited to a fundus image. The acquired fundus image may include a deepened lesion and / or treatment marks. The acquired fundus image may be, for example, the acquired image of FIG. 2, FIG. 3, or FIG. 5.

[0025] The step (S101) of generating a previous image of a medical image from a medical image may generate a medical image corresponding to a time point earlier than the time point of the acquired medical image. The previous image refers to a previous fundus image at a time point (t<1) earlier than the acquisition time point (t=1). The previous fundus image may be generated by the inverse transformation (data from noise) of the diffusion model. Meanwhile, the acquisition time point refers to the time point when the medical image is actually captured, not the time point when the medical image is input to a device or model that performs the method according to the present invention. In addition, the previous image and the previous fundus image refer to images generated by the device or model. For a detailed description of the step (S101) of generating the previous image, refer to the descriptions of FIGS. 2 and 3.

[0026] The step (S102) of detecting at least one region of interest (ROI) in a medical image and a previous image may detect a region of interest (ROI) from the acquired medical image and / or the previous images. The region of interest may be, for example, multiple regions of interest (500) of the fundus image of FIG. 5. The multiple regions of interest may be classified as S1, S2, S3, S4, S5, S6, S7, S8, and S9 according to the region of the fundus. The multiple regions of interest for the acquired image including a lesion may include the lesion for each region of interest of the fundus. The multiple regions of interest for the previous image of the acquired image may include a deepened lesion region or a normal region without a lesion for each region of interest. If the previous image is a normal image without a lesion, the region of interest may not be derived. In order to provide lesion quantification information for each region of the fundus, the region of interest of the fundus images may be detected according to the degree of lesion progression. The operation of the step (S102) of detecting the region of interest is described again in FIG. 5.

[0027] The step of quantifying lesions within at least one region of interest (S103) can quantify the region containing the lesion within the region of interest. For example, the pixel area of ​​the region of the lesion can be quantified. By quantifying the lesion within the region of interest, information on the change in the progressive lesion over time can be derived. The area of ​​the lesion within the region of interest can be expressed as a percentage, etc. The area of ​​the lesion compared to the entire fundus image or the retinal area within the region can be expressed as a percentage, etc. Quantification information of the lesion, such as the area, size, and length related to the lesion, can be generated. For a specific description of a method for quantifying the region of interest of a medical image by region, refer to FIG. 4.

[0028] In the step (S104) of detecting changes in lesions by region of interest, changes in lesions by region can be detected from the quantified lesions. Information on progression from a normal region to a lesion region over time can be detected.

[0029] Meanwhile, a method for providing lesion information according to an embodiment of the present invention may further include a step of determining whether a lesion is included in the acquired medical image. If it is determined that the acquired medical image includes a lesion, the step of generating a previous image (S101) may generate a previous image including a lesion with a less advanced stage than the lesion. If it is determined that the acquired medical image does not include a lesion, the step of generating a previous image (S101) may generate an image identical to the acquired medical image as the previous image, or may not perform a separate operation. In addition, if it is determined that the acquired medical image does not include a lesion, the step of outputting the generated previous image may output an alarm indicating that a condition for generating the previous image is not satisfied. In this case, the alarm may be a visual alarm including text or an image, or an auditory alarm including a sound.

[0030] Although the acquired medical image does not contain a lesion, it may contain a mark from a treatment of the lesion. The mark from the treatment of the lesion may be, for example, a laser mark. If it is determined that the acquired medical image contains a mark from the treatment of the lesion, the step of generating a previous image (S101) may generate previous images in which the mark from the treatment of the lesion gradually disappears and a normal image in which the mark from the treatment of the lesion has completely disappeared. If it is determined that the acquired medical image contains a mark from the treatment of the lesion, the step of outputting the generated previous image may output an alarm indicating that the acquired medical image does not contain a lesion, that the lesion has been treated, but that the mark from the treatment of the lesion remains. The alarm may be a visual alarm including text or an image, or an auditory alarm including a sound. The user may be asked to select whether he or she wants to view a normal image without a mark from the treatment of the lesion, indicating whether the treatment was successful. The user may check how much of the mark from the treatment of the lesion remains even after the treatment of the lesion, and may compare it with a normal image without a mark from the treatment of the lesion to check the current status.

[0031] The step of determining whether the acquired medical image contains a lesion can determine whether the lesion is included using a machine learning model or an artificial neural network model trained to detect lesions in medical images, and the machine learning model or the artificial neural network model can be trained using medical images labeled with the location and type of lesion as learning data.

[0032] The step of determining whether the acquired medical image contains a treatment scar can be performed using a machine learning model or artificial neural network model trained to detect treatment scars in medical images. The machine learning model or artificial neural network model can be trained using medical images labeled with treatment scars as training data.

[0033] The output step (S105) can acquire a medical image (S100), generate a previous image from the medical image (S101), and then output the previous image. The output previous image is similar to the previous image or normal image illustrated in FIG. 2 as an example.

[0034] The output step (S106) outputs a previous image from the acquired medical image, similar to the output step (S105), but additionally detects a region of interest (S102), quantifies a lesion within the region of interest (S103), detects a change in the quantified lesion in the region of interest (S104), and outputs a medical image (S106) including a change in the detected lesion.

[0035] Figure 2 shows fundus images at each time point according to one embodiment of the present invention.

[0036] Figure 2 illustrates a fundus image acquired by a medical image acquisition step (S100) and a previous image generated by a previous image generation step (S101).

[0037] According to the embodiments, the method can acquire a fundus image, as illustrated in FIG. 2, and generate fundus images from time points prior to the time point at which the acquired fundus image was captured. If the acquired image includes an aggravated lesion, the fundus images from previous time points can represent the progression of the lesion. For example, over time, a normal fundus image, an image from the initial point at which a lesion developed, and an image from a point at which the lesion worsened can all be displayed.

[0038] By generating fundus images from previous time points, it is possible to provide clinically important region-specific quantification information to medical professionals, and to provide users, such as patients and / or family members, with easily understandable information on the progression of lesions. In this regard, the previous image generation step (S101), the region of interest detection step (S102), and / or the lesion quantification step (S103) are described in detail in FIGS. 4, 5, and 6.

[0039] In addition, the post-treatment image may show that the lesion has been reduced or disappeared, and may include laser marks, etc., that have been left after the treatment. The fundus image acquired by the medical image acquisition step (S100) may be a fundus image with an aggravated lesion, or may be a fundus image after the lesion has been treated. The medical image acquisition step (S100) acquires a fundus image after the actual lesion has been treated, and from the fundus image including the treatment marks, e.g., laser marks, etc., a fundus image of a previous point in time, i.e., fundus images of a previous point in time in which the treatment marks have gradually disappeared, and a normal fundus image in which the treatment marks have completely disappeared, may be generated by the previous image generation step (S101). The detailed operation of the previous image generation step (S101) is described in detail in FIG. 3.

[0040] In addition, since the previous image includes a normal image from before the lesion occurred and images at each point in time according to the degree of lesion progression, the previous image corresponding to a specific point in time selected by the user can be provided or changes in the lesion can be confirmed in response to an input of selecting a specific time interval.

[0041] According to one embodiment of the present invention, when time information for selecting a specific time point is input by a user, a previous image corresponding to the time point can be generated. For example, if the time point at which an input medical image was captured is called time point 1, an image of time point 2, which is a time point prior to time point 1, can be generated, an image of time point 3, which is a time point prior to time point 2, can be generated, and an image of time point 4, which is a time point prior to time point 3, can be generated. In this case, the image of time point 4 that is finally generated can be a normal image without a lesion. The degree of lesion can be expressed from none to low, or from low to severe from the normal image to the input medical image. Meanwhile, although this example describes that images up to time point 4 can be generated, the embodiment of the present invention is not necessarily limited thereto, and it should be understood that images corresponding to more or fewer time points can be generated depending on a user's selection or preset settings.

[0042] Referring to FIG. 2, the previous images include a second image at a second time point, a third image at a third time point, and a fourth image at a fourth time point, and it can be understood that the second time point, the third time point, and the fourth time point are time points in the past compared to the time point at which the input medical image was taken. In addition, the quantification value of the lesion of the second image may be lower than the quantification value of the lesion of the input medical image, the quantification value of the lesion of the second image may be lower than the quantification value of the lesion of the first image, and the quantification value of the lesion of the third image may be lower than the quantification value of the lesion of the second image. In addition, the quantification value of the lesion of the fourth image may be lower than the quantification value of the lesion of the third image. The quantification value may be a value indicating the malignancy of the lesion, the number of lesions, the size of the lesion area, etc. For example, when the images are arranged from the past time point to the present time point, they may be a fourth image, a third image, a second image, and an input medical image.

[0043] Figure 3 shows a previous image and a lesion image according to one embodiment of the present invention.

[0044] FIG. 3 illustrates an input medical image at the time of capture in FIG. 2 and one or more previous images from a time point in the past than the time of capture. If a time point is defined by a t value, the time point (t) of the input medical image including an aggravated lesion at the time of capture may be defined as 1, and a t value less than 1, for example, 0 to 0.999, may be assigned to one or more previous images from the past time point. For the convenience of explanation, the case where the acquisition time point (t) is 1 is assumed, but it is not necessarily limited to t = 1, and it is obvious to those skilled in the art that t may be set to a specific other value. In addition, it is sufficient if the time point of the previous image is less than the acquisition time point, and a value less than 0 may also be acceptable.

[0045] From the lesion image of t=1 acquired by the medical image acquisition step (S100), a previous image in which the lesion of the past time point gradually disappears or a previous image (normal image) in which all lesions have disappeared can be generated by the previous image generation step (S101). In addition, when the captured input medical image includes a mark of a lesion treated, for example, a laser mark, a previous image in which the treatment mark disappears or a previous image (normal image) in which all treatment marks have disappeared can be generated from the lesion image (t=1) in which the treatment mark remains by the previous image generation step (S101). Hereinafter, the detailed operation of the previous image generation step (S101) will be described with reference to FIG. 4.

[0046] Figure 4 shows a diffusion model according to embodiments.

[0047] FIG. 4 illustrates a process of generating one or more previous images based on a diffusion model in the previous image generation step (S101) of FIG. 1.

[0048] As described in FIG. 3, the image acquired at time t=1 is an actual captured image containing a lesion, and the diffusion model according to an embodiment of the present invention of FIG. 4 can use this as input to generate at least one previous image corresponding to a time earlier than t=1.

[0049] A type of diffusion model can include the Stable Diffusion model. Stable diffusion utilizes perceptual compression and semantic compression. Perceptual compression is a Pixel Space Model utilizing an autoencoder, while semantic compression is a Latent Space Model, which can generate images using various conditions (text, semantic masks, images, etc.).

[0050] A diffusion model can include forward and reverse transformations, where forward transformation refers to the transformation from data to noise, and reverse transformation refers to the transformation from noise to data.

[0051] DDPM (Denoising Diffusion Probabilistic Models, Ho, Jonathan, et al. “Denoising Diffusion Probabilistic Models.” NIPS 2020) is a diffusion generative model that consists of a forward process that gradually adds Gaussian noise until the original image becomes a complete Gaussian noise (Isotropic Gaussian Noise) image, and a backward process that is a denoising process that gradually removes such noise. For example, when noise is added to the original image (x0) at each step, it takes several steps to create a complete Gaussian noise (x T) is made. Here, the diffusion model is the current point image (x t ) is given, x t The previous time point image (x) with one step less noise added from t-1 ) is learned to generate p(x). That is, the diffusion model is t-1 |x t ) distribution to learn the correct answer distribution q(x t-1 |x t , x0) to be as similar as possible. Since the correct answer distribution q satisfies the equation below according to the Gaussian distribution, if it is trained to predict only the noise at a specific point in time, it is a complete noise image (x t ) can be used to generate the original image (x0).

[0052]

[0053]

[0054]

[0055] The learning method of the diffusion model using this is a forward transformation that converts the acquired image containing the deepened lesion into a Gaussian noise image, and the time condition of t = 1 is converted to the latent space by a text encoder. The text encoder can use, for example, CLIP (Learning transferable visual models from natural language supervision, Radford et al. "Learning transferable visual models from natural language supervision." PMLR 2021). The Stable Diffusion model learns to restore the converted image and condition to the input image through the denoising process. In the same way, the data distribution is learned through a large amount of data with a normal image without a lesion and the condition of t = 0.

[0056] Inference is the process by which a machine learning model makes predictions on new data after completing the training phase. In other words, it refers to the process of predicting outcomes for unknown data not used in the model's training. This involves generating results for new input data based on the patterns learned during the training phase.

[0057] As an inference process, if the acquired image and the desired time point (t<1) are input to the Stable Diffusion model, the time point is converted to Latent Space by CLIP, and the image is converted to Latent Space through an autoencoder, and these can be combined through a denoising process to generate a fundus image of the desired time point.

[0058] To learn a diffusion model, a diffusion model can be used to input a desired time point (a time point without a lesion or between acquired images) and an acquired image, and predict and generate a fundus image at that time point. The learning model according to the embodiments randomly assigns a condition t having a value less than or equal to 1 for a normal fundus image, and assigns a condition t=1 for the acquired image, thereby learning the distribution of these.

[0059] The method for providing a lesion may further include a step of assigning a point in time having a specific value to a lesion image of a medical image, and randomly assigning a point in time having a value less than the specific value to a normal image of the lesion image and at least one previous image having a lesion value lower than the value of the lesion (e.g., lesion-related area, size, length, etc.) of the lesion image, thereby learning the distribution of the lesion image, the normal image, and the at least one previous image based on a diffusion model.

[0060] The method for providing a lesion may include training a learning model, and the step of generating at least one previous image from a medical image may include predicting a previous image corresponding to a time point preceding the time point at which the medical image was captured, based on a diffusion model. Based on information about the previous time point, multiple previous images between the time point at which the image was captured and the previous time point may be generated.

[0061] FIG. 5 illustrates a method for detecting a region of interest and quantifying a lesion according to one embodiment of the present invention.

[0062] Referring to FIG. 5, a method according to an embodiment of the present invention can detect quantitative information by region and / or finding from a fundus image. The region of interest detection step (S102) and / or lesion quantification step (S103) of FIG. 1 can derive a region of interest of a medical image including an acquired image and a previous image, as shown in FIG. 5, and quantify lesions by region of interest. For example, the region of interest can include nine regions of interest, and quantitative information by finding for each region of interest can be generated.

[0063] The region of interest detection step (S102) can detect nine regions of interest (500, 501: S1 to S9) from a medical image. When the medical image is a fundus image, for example, as in FIG. 2 or FIG. 3, regions of interest such as regions S1 to S9 (500, 501) can be derived from the image acquired by the medical image acquisition step (S100) and / or the previous image generated in the previous image generation step (S101). The region of interest can be generated as three circles and a 45-degree diagonal and a 135-degree diagonal with respect to the center (fovea) of the fundus image. The region of interest (500) is illustrated again as nine regions of interest (502). The diameter of the optic disc can be defined as 1 ODD (optic disc diameter) or 1 DD (disc diameter). The three circles constituting the regions of interest (502) have radii of 0.5 ODD, 1.5 ODD, and 2.5 ODD. Among the nine regions of interest (500, 501, and 502) denoted as S1 to S9, the smallest circle having a radius of 0.5 ODD is defined as S1. The regions within the next circle having a radius larger than S1 are defined as S2, S3, S4, and S5 in a clockwise direction. The regions within the largest circle are defined as S7, S8, and S9 in a clockwise direction. The order of designating S1 to S9 may vary depending on the embodiment.

[0064] The lesion quantification step (S103) can quantify the lesions included in the regions of interest and display them as quantified information (501). The quantified information (501) can include the number of pixels (px) corresponding to the lesions included in each region of interest and / or a relative percentage (%) represented by the number of pixels corresponding to the lesions in each region of interest compared to the area of ​​the fundus region excluding the background of the fundus image. The lesion quantification step (S103) can quantify the number of pixels corresponding to the lesions and the ratio of the lesion area to the fundus region when the region of interest includes a lesion.

[0065] For example, if the region of interest detection step (S102) detects regions of interest (500, 501, 502) such as S1 to S9, and among these, hard exudates are detected only in S3, S4, S7, and S8, the lesion quantification step (S103) can derive quantitative values ​​(2135px, 980px, 1659px, 206px) and quantitative ratios (0.17%, 0.06%, 0.12%, 0.02%) from the regions of interest where lesions are detected.

[0066] According to one embodiment of the present invention, lesions may include not only hard exudates, but also microaneurysms, hemorrhages, soft exudates, and yellow substances accumulated in the retina (Drusen).

[0067] For example, the region of interest (503) may include yellow material (Drusen) accumulated on the retina, the region of interest (504) may include hemorrhage, and the region of interest (505) may include exudate.

[0068] The step of detecting at least one region of interest in a medical image and a previous image of a method for providing lesion information includes detecting at least one region of interest based on a diameter of an optic nerve head from a center of a fundus region of the medical image and the previous image, and the at least one region of interest may include a lesion related to at least one of microaneurysm, hemorrhage, soft exudates, hard exudates, or drusen accumulated on the retina.

[0069] The step of quantifying the lesion within at least one region of interest may include calculating the number of pixels of the lesion within the at least one region of interest and calculating a ratio of the area occupied by the number of pixels of the lesion to the fundus area.

[0070] Figure 6 illustrates a lesion according to one embodiment of the present invention.

[0071] Referring to Fig. 6, examples of lesions according to the above-described embodiments can be seen. Fig. 6(a) illustrates a fundus image with a lesion, Fig. 6(b) illustrates microaneurysms, Fig. 6(c) illustrates hemorrhages, Fig. 6(d) illustrates hard exudates, Fig. 6(e) illustrates soft exudates, and Fig. 6(f) illustrates the optic disc (not a lesion).

[0072] A method according to embodiments may generate an image of a time point preceding the time point at which an input medical image was acquired, and may generate a fundus image representing the progression of a lesion from the previous time point to the current time point. One or more images of the previous time points may be generated, and if multiple images of the previous time points are generated, the image corresponding to the earliest time point may be an image of a normal state before the development of a lesion. In addition, if selection information for selecting a specific time point is input from the user, quantitative information of the fundus image derived in response to the selection information may be generated. A method according to embodiments may provide medical staff and / or users with a quantitative comparative analysis between the input image and the generated image, and provide information on how the lesion has worsened.

[0073] The method according to the embodiments relates to a method and device for generating an image of a previous time point based on an input medical image and providing information on changes in a lesion therefrom. Specifically, according to the method according to the present invention, a computing device can acquire a medical image and generate an image of a previous time point of the acquired medical image. In addition, by detecting a region of interest from the acquired medical image and the generated image and obtaining quantitative information through a lesion segmentation model, change information by region / lesion can be provided. The method according to the embodiments can provide information on the progression of a lesion included in a medical image efficiently and in a way that increases user convenience by analyzing an image of a lesion and generating a previous image.

[0074] Fig. 7 illustrates a lesion information providing device according to one embodiment of the present invention.

[0075] The lesion information providing device (700) according to the embodiments performs the lesion information providing method of FIG. 1.

[0076] The lesion information providing device (700) may include an interface unit (701), a memory (702), and / or a processor (703).

[0077] The interface unit (701) acquires medical images.

[0078] The memory (702) is connected to the interface (701) and stores medical images and information about medical images, and information for quantification.

[0079] The processor (703) may be connected to the interface unit (701) and / or the memory (702). The memory (702) may store instructions for quantification. The instructions stored in the memory may be configured to cause the processor to acquire a medical image; generate at least one previous image from the medical image; and output the medical image and the at least one previous image based on a time point corresponding to the at least one previous image.

[0080] The processor (703) may be further configured to detect at least one region of interest in the medical image and the previous image; quantify a lesion within at least one region of interest; and detect changes to the lesion for each region of interest.

[0081] The processor (703) may be further configured to assign a point in time having a specific value to a lesion image of a medical image, and randomly assign a point in time having a value less than the specific value to a normal image of the lesion image and at least one previous image having a lesion value lower than the lesion value of the lesion image, thereby learning the distribution of the lesion image, the normal image, and the at least one previous image based on a diffusion model.

[0082] Generating at least one previous image from a medical image includes predicting a previous image corresponding to a time point preceding a time point at which the medical image was captured, based on a diffusion model, and a plurality of previous images between the time point at which the medical image was captured and the previous time point can be generated based on information about the previous time point.

[0083] The plurality of previous images include a first image at a first time point, a second image at a second time point, and a third image at a third time point, wherein the first time point, the second time point, and the third time point are time points in the past relative to a current time point of the medical image, and the quantification value of the lesion of the first image is lower than the quantification value of the lesion of the medical image, the quantification value of the lesion of the second image is lower than the quantification value of the lesion of the first image, and the quantification value of the lesion of the third image is lower than the quantification value of the lesion of the second image.

[0084] The medical image includes lesions of the fundus by region of the fundus, and the lesions may include at least one of microaneurysms, hemorrhages, soft exudates, hard exudates, or yellowish material accumulated on the retina (Drusen).

[0085] The medical images include treatment marks of lesions in the fundus, and the treatment marks may include laser marks.

[0086] Detecting at least one region of interest in the medical image and the previous image includes: detecting at least one region of interest based on a diameter of the optic disc from the center of the fundus region of the medical image and the previous image, wherein the at least one region of interest includes a lesion related to at least one of microaneurysm, hemorrhage, soft exudates, hard exudates, or drusen.

[0087] Quantifying the lesion within at least one region of interest comprises: calculating the number of pixels of the lesion within at least one region of interest, and calculating the ratio of the area occupied by the number of pixels of the lesion to the area of ​​the fundus.

[0088] The embodiments have the effect of predicting images from a previous point in time based on a user's selection and providing quantitative change information by region. The devices and methods according to the embodiments have the effect of generating images from a previous point in time of a medical image, detecting regions of interest from the acquired and generated images, obtaining quantitative information through a lesion segmentation model, and providing change information by region.

[0089] The embodiments have been described in terms of methods and / or devices, and the descriptions of methods and devices may be applied complementarily.

[0090] For the convenience of explanation, each drawing has been described separately, but it is also possible to design a new embodiment by combining the embodiments described in each drawing. In addition, designing a computer-readable recording medium having a program recorded thereon for executing the previously described embodiments, as needed by a person skilled in the art, also falls within the scope of the embodiments. The devices and methods according to the embodiments are not limited to the configurations and methods of the embodiments described above, but the embodiments may be configured by selectively combining all or part of the embodiments so that various modifications can be made. Although preferred embodiments of the embodiments have been illustrated and described, the embodiments are not limited to the specific embodiments described above, and various modifications can be made by a person skilled in the art to which the present invention pertains without departing from the gist of the embodiments claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the embodiments.

[0091] The various components of the devices of the embodiments may be implemented by hardware, software, firmware, or a combination thereof. The various components of the embodiments may be implemented by a single chip, for example, a single hardware circuit. According to embodiments, the components according to the embodiments may be implemented by separate chips. According to embodiments, at least one of the components of the devices of the embodiments may be configured with one or more processors capable of executing one or more programs, and the one or more programs may perform, or include instructions for performing, one or more of the operations / methods according to the embodiments. The executable instructions for performing the methods / operations of the devices of the embodiments may be stored in non-transitory CRMs or other computer program products configured to be executed by one or more processors, or may be stored in temporary CRMs or other computer program products configured to be executed by one or more processors. In addition, the memory according to the embodiments may be used as a concept including not only volatile memory (e.g., RAM, etc.), but also non-volatile memory, flash memory, PROM, etc. Additionally, it may include implementations in the form of carrier waves, such as transmissions via the Internet. Furthermore, processor-readable recording media may be distributed across network-connected computer systems, allowing processor-readable code to be stored and executed in a distributed manner.

[0092] In this document, “ / ” and “,” are interpreted as “and / or”. For example, “A / B” is interpreted as “A and / or B”, and “A, B” is interpreted as “A and / or B”. Additionally, “A / B / C” means “at least one of A, B, and / or C”. Also, “A, B, C” means “at least one of A, B, and / or C”. Additionally, “or” in this document is interpreted as “and / or”. For example, “A or B” can mean 1) “A” only, 2) “B” only, or 3) “A and B”. In other words, “or” in this document can mean “additionally or alternatively”.

[0093] Terms such as first, second, etc. may be used to describe various components of the embodiments. However, the various components according to the embodiments should not be limited in their interpretation by the above terms. These terms are merely used to distinguish one component from another. For example, a first user input signal may be referred to as a second user input signal. Similarly, a second user input signal may be referred to as a first user input signal. The use of these terms should be interpreted as not departing from the scope of the various embodiments. Although a first user input signal and a second user input signal are both user input signals, they do not mean the same user input signals unless the context clearly indicates otherwise.

[0094] The terminology used to describe the embodiments is for the purpose of describing particular embodiments and is not intended to be limiting of the embodiments. As used in the description of the embodiments and in the claims, the singular is intended to include the plural unless the context clearly dictates otherwise. The expressions “and / or” are used to mean all possible combinations of terms. The expression “includes” describes the presence of features, numbers, steps, elements, and / or components, but does not mean that additional features, numbers, steps, elements, and / or components are not included. Conditional expressions such as “if” or “when” used to describe the embodiments are not intended to be limited to only optional cases. When a specific condition is satisfied, a related action is performed in response to a specific condition, or a related definition is intended to be interpreted.

[0095] Additionally, the operations according to the embodiments described in this document may be performed by a transceiver device including a memory and / or a processor according to the embodiments. The memory may store programs for processing / controlling the operations according to the embodiments, and the processor may control various operations described in this document. The processor may be referred to as a controller, etc. The operations according to the embodiments may be performed by firmware, software, and / or a combination thereof, and the firmware, software, and / or a combination thereof may be stored in the processor or in the memory.

[0096] Meanwhile, the operations according to the embodiments described above may be performed by a transmitting device and / or a receiving device according to the embodiments. The transmitting / receiving device may include a transmitting / receiving unit for transmitting and receiving media data, a memory for storing instructions (program code, algorithm, flowchart, and / or data) for a process according to the embodiments, and a processor for controlling the operations of the transmitting / receiving device.

[0097] The processor may be referred to as a controller, etc., and may correspond to, for example, hardware, software, and / or a combination thereof. The operations according to the above-described embodiments may be performed by the processor.

[0098] As described above, the relevant contents have been described in the best form for carrying out the embodiments.

[0099] As described above, the embodiments can be applied in whole or in part to a device and system for generating quantitative information by region based on fundus images.

[0100] A person skilled in the art may make various changes or modifications to the embodiments within the scope of the embodiments.

[0101] Embodiments may include modifications / changes, which do not depart from the scope of the claims and their equivalents.

Claims

1. A method for providing lesion information performed by a computing device including at least one processor, Step of acquiring medical images; generating at least one previous image from said medical image; and A step of outputting the medical image and the at least one previous image based on a time corresponding to the at least one previous image; comprising; How to provide lesion information.

2. In paragraph 1 A step of detecting at least one region of interest in the medical image and the previous image; a step of quantifying a lesion within at least one region of interest; and A step of detecting changes in lesions for each region of interest; further comprising; How to provide lesion information.

3. In paragraph 1 Assign a point in time having a specific value to the lesion image of the above medical image, By randomly assigning a time point having a value lower than the specific value to at least one previous image having a lesion value lower than the lesion value of the normal image of the above lesion image and the lesion image, Further comprising a step of learning the distribution of the lesion image, the normal image and the at least one previous image based on a diffusion model. How to provide lesion information.

4. In paragraph 3, The step of generating at least one previous image from the medical image includes predicting a previous image corresponding to a time point preceding the time point at which the medical image was captured, based on the diffusion model, Based on the information about the previous point in time, multiple previous images between the shooting point in time and the previous point in time are generated. How to provide lesion information.

5. In paragraph 4, The above multiple previous images include a first image at a first time point and a second image at a second time point, The above first time point and the above second time point are time points in the past relative to the present time point of the medical image, The quantification value of the lesion of the first image is lower than the quantification value of the lesion of the medical image, The quantification value of the lesion in the second image is lower than the quantification value of the lesion in the first image. How to provide lesion information.

6. In paragraph 1, The above medical image includes lesions of the fundus by region of the fundus, The lesion includes at least one of microaneurysm, hemorrhage, soft exudates, hard exudates, or drusen. How to provide lesion information.

7. In paragraph 1, The above medical images include treatment marks on lesions in the fundus, The above treatment marks include laser marks. How to provide lesion information.

8. In paragraph 2, The step of detecting at least one region of interest in the above medical image and the above previous image is: detecting at least one region of interest based on a diameter of the optic disc from the center of the fundus region of the medical image and the previous image; The at least one area of ​​interest comprises a lesion involving at least one of microaneurysm, hemorrhage, soft exudates, hard exudates, or drusen. How to provide lesion information.

9. In paragraph 8, The step of quantifying lesions within at least one region of interest comprises: Comprising calculating the number of pixels corresponding to the lesion included in the at least one region of interest, and calculating the ratio of the area occupied by the number of pixels corresponding to the lesion to the fundus region. How to provide lesion information.

10. Interface unit for acquiring medical images; memory connected to the above interface unit; and A processor connected to said memory; wherein instructions stored in said memory cause the processor to: Acquire medical images; generating at least one previous image from said medical image; and configured to output the medical image and the at least one previous image based on a time point corresponding to the at least one previous image; Lesion information provider device.

11. A computer program stored in a computer-readable storage medium, wherein the computer program, when executed on one or more processors, performs the following operations to provide lesion information, wherein the operations are: Step of acquiring medical images; generating at least one previous image from said medical image; and A step of outputting the medical image and the at least one previous image based on a time corresponding to the at least one previous image; comprising; A computer program stored on a computer-readable storage medium.

Images