Cervical image generating system and method using generative ai model

Abstract

The present invention relates to a cervical image generating system and method using a generative AI model, and, more specifically, to a cervical image generating system and method using a generative AI model in order to generate a virtual cervical positive image by using a diffusion probability model and a transfer learning technique. According to one embodiment of the present invention, the cervical image generating system using a generative AI model comprises: a data collection unit for acquiring and collecting cervical image data; a data learning unit for learning, by means of the generative AI model, the cervical image data collected by the data collection unit; and an inference unit, which infers input data by means of a cervical image generation model derived from the data learning unit, so as to generate and output a virtual cervical image.

Description

System and method for generating cervical images using a generative AI model

[0001] The present invention relates to a system and method for generating a cervical image using a generative AI model, and more particularly, to a system and method for generating a cervical image using a generative AI model to generate a virtual benign cervical image using a diffusion probability model and a transfer learning technique.

[0002] Unless otherwise indicated herein, the materials described in this section are not prior art to the claims of this application, and their inclusion in this section is not intended to be admitted as prior art.

[0003] The female uterus is generally divided into a body and a neck. The body (uterus) accounts for approximately 75% of the uterus, while the neck (cervix) connects to the vagina. Cancer that occurs in the cervix, the neck of the uterus, is called cervical cancer.

[0004] According to statistics from the Health Insurance Review & Assessment Service, over 50,000 patients have visited hospitals each year for cervical cancer over the past five years. Of these, patients in their 40s account for approximately 25%, followed by those in their 50s and 30s, each accounting for over 20%. As such, cervical cancer incidence statistics show it to be one of the most dangerous cancers for women worldwide.

[0005] However, according to the 2020 cancer registration statistics from the National Cancer Center, the incidence of cervical cancer has been decreasing from 16.7 to 14.2 between 2009 and 2013 between 2014 and 2018.

[0006] This trend has resulted in a continuous decrease in the number of positive image data required for training AI-based cervical cancer grading models, leading to data imbalance between negative and positive image classes. Consequently, securing positive cervical cancer data is essential to continuously improve the performance of AI-based cervical cancer grading models.

[0007] In this context, there is a need to continuously augment the training data by generating virtual cervical images to enhance the performance of AI-based cervical cancer classification models and improve the accuracy of cervical cancer diagnosis. In particular, virtual image generation techniques are effective for images of positive cervical cancer, as data acquisition is essential due to the declining incidence rate in Korea.

[0008] In relation to such artificial intelligence-based cervical cancer diagnosis, the applicant of the present invention, in the invention of the “Artificial Intelligence-Based Cervical Cancer Screening Service System (Patent Registration No. 10-2462975, announced on November 8, 2022)” previously filed and registered, proposed a method to solve the above-mentioned problem by improving the accuracy, objectivity, and speed of cervical cancer screening using an artificial intelligence learning model.

[0009] However, in order to more effectively utilize the AI-based cervical cancer screening service system of the above-mentioned prior application patent invention and to improve the performance of the AI-based cervical cancer grade classification model constructed for the screening service, it is necessary to supplement the decreasing number of positive image data to resolve the data imbalance problem between negative and positive image classes and to generate virtual cervical images to increase the learning data.

[0010] In this regard, Patent Publication No. 10-2041402 (announced on November 7, 2019) discloses a prior art regarding a “cervical learning data generation system and a cervical learning data classification method”, but the prior art does not relate to a cervical benign image generation method for generating a virtual cervical image using an artificial intelligence diffusion probability model and transfer learning technique intended by the present invention, but only relates to a classification method for classifying cervical image data for building an AI learning system related to the cervix, and thus cannot resolve the aforementioned problem.

[0011] [Prior Art Literature]

[0012] {Patent Document}

[0013] (Patent Document 1) Patent Registration No. 10-2462975 (Published on November 8, 2022)

[0014] (Patent Document 2) Patent Registration No. 10-2041402 (Published on November 7, 2019)

[0015] The present invention has been devised to improve upon the above-mentioned conventional problems, and the purpose of the present invention is to provide a system and method for generating cervical images using a generative AI model by resolving the problem of data imbalance between negative and positive image classes required for constructing an AI-based cervical cancer grading classification model, applying an AI diffusion probability model to these image data and using a transfer learning technique to generate virtual cervical benign images, and thereby continuously enhancing the performance of the AI-based cervical cancer grading classification model.

[0016] In addition, it is obvious that the technical tasks are not limited to the technical tasks described above, and other technical tasks may be derived from the following description.

[0017] A system for generating a cervical image using a generative AI model according to one embodiment of the present invention is characterized by including: a data collection unit that acquires and collects cervical image data; a data learning unit that learns the cervical image data collected by the data collection unit using a generative AI model; and an inference unit that generates and outputs a virtual cervical image by inferring input data using the cervical image generation model derived from the data learning unit.

[0018] According to a preferred embodiment of the present invention, the generative AI model is characterized in that it is a diffusion probability model.

[0019] According to a preferred embodiment of the present invention, the data collection unit is characterized by including a camera photographing unit for photographing and acquiring a cervical image; a cervical image reading unit for a medical professional to read the photographed cervical image; and a database server storage unit for storing the read image data by a communication means.

[0020] According to a preferred embodiment of the present invention, the data learning unit is characterized by including a training data set classification unit that is transmitted and classified from the database server storage unit; a training data set preprocessing unit that preprocesses the training data set; and a diffusion probability model learning unit that learns the preprocessed data by a diffusion probability model.

[0021] According to a preferred embodiment of the present invention, the inference unit is characterized by including an inference input unit that inputs an input image or a prompt; an inference preprocessing unit that preprocesses the input data; a cervical image generation model inference unit that applies the preprocessed data to a cervical image generation model learned by a diffusion probability model; and an output unit that outputs a virtual cervical image generated by the generation model inference unit.

[0022] According to a preferred embodiment of the present invention, the present invention is characterized by including a database for learning an artificial intelligence-based cervical cancer grade classification model that stores a virtual cervical benign image generated by the inference unit after being read by medical staff.

[0023] A method for generating a cervical image using a generative AI model according to a preferred embodiment of the present invention is characterized by including: a data collection step for acquiring and collecting cervical image data; a data learning step for learning the cervical image data collected in the data collection step by a generative AI model; and an inference step for generating and outputting a virtual cervical image by inferring input data using the cervical image generation model derived in the data learning step.

[0024] According to a preferred embodiment of the present invention, the generative AI model is characterized in that it is a diffusion probability model.

[0025] According to a preferred embodiment of the present invention, the data collection step is characterized by including a step of capturing a cervical image by a camera photographing unit; a cervical image reading step of having a medical staff read the captured cervical image; and a database server storage step of storing the read image data by a communication means.

[0026] According to a preferred embodiment of the present invention, the data learning step is characterized by including a training data set classification step in which a training data set is transmitted from the database server storage unit and classified; a training data set preprocessing step in which the training data set is preprocessed; and a diffusion probability model learning step in which the preprocessed data is learned by a diffusion probability model.

[0027] According to a preferred embodiment of the present invention, the inference step is characterized by including an inference input step of inputting an input image or a prompt into an input unit; an inference preprocessing step of preprocessing the input data; a cervical image generation model inference step of applying the preprocessed data to a cervical image generation model learned by a diffusion probability model; and an output step of outputting a virtual cervical image generated in the generation model inference step through an output unit.

[0028] According to a preferred embodiment of the present invention, the present invention is characterized by including a reading step in which a medical professional reads whether the virtual cervical benign image generated in the inference step is an actual benign image; and a storage step in which, if the virtual cervical benign image is identical to the actual benign image after the reading step, the image data is stored in a database for learning an artificial intelligence-based cervical cancer grade classification model.

[0029] According to a preferred embodiment of the present invention, the present invention is characterized by including a retraining step of retraining an artificial intelligence-based cervical cancer grade classification model based on negative and positive image data stored in a database for training the artificial intelligence-based cervical cancer grade classification model.

[0030] According to the present invention, there is a beneficial effect of resolving the problem of data imbalance between negative and positive image classes required to build an artificial intelligence-based cervical cancer grading classification model.

[0031] According to the present invention, there is an advantageous effect of enhancing the performance of a continuous artificial intelligence-based cervical cancer grading model by generating and retraining virtual cervical benign images using a transfer learning technique and applying a diffusion probability model to a generative AI model for negative and positive image data.

[0032] The effects of the present invention are not limited to the effects described above, and should be understood to include all effects that can be inferred from the detailed description of the present invention or the composition of the invention described in the claims.

[0033] Figure 1 is a block diagram of the entire cervical image generation system using the generative AI model of the present invention.

[0034] Figure 2 is a detailed block diagram of the data learning unit of Figure 1.

[0035] Figure 3 is a detailed block diagram of the inference unit of Figure 1.

[0036] Figure 4 is a process configuration flowchart of a cervical image generation system using a generative AI model according to embodiments 1 and 2 of the present invention.

[0037] FIG. 5a is an example photograph of a virtual benign image generated by a system and method for generating a cervical image using a generative AI model according to Example 1 of the present invention.

[0038] FIG. 5b is an example photograph of a virtual benign image generated by a system and method for generating a cervical image using a generative AI model according to Example 2 of the present invention.

[0039]

[0040] Hereinafter, with reference to the attached drawings, a system and method for generating cervical images using a generative AI model according to a preferred embodiment will be described in detail.

[0041] For reference, in the drawings below, each component is omitted or schematically illustrated for convenience and clarity, and the size of each component does not reflect the actual size. The same reference numerals refer to the same components throughout the specification, and drawing numerals for the same components are omitted in individual drawings. In addition, detailed descriptions of well-known functions and components that may unnecessarily obscure the gist of the present invention are omitted.

[0042] Hereinafter, a preferred embodiment of the present invention will be described with reference to FIGS. 1 to 5. As shown in FIG. 1, a cervical image generation system (1000) using a generative AI model of the present invention is characterized by including: a data collection unit (100) for acquiring and collecting cervical image data; a data learning unit (200) for learning the cervical image data collected by the data collection unit (100) by a generative AI model; and an inference unit (300) for generating and outputting a virtual cervical image by inferring input data by the cervical image generation model derived by the data learning unit (200).

[0043] The present invention generates a virtual cervical image using a generative AI model. The generative AI model uses a diffusion probability model, and the present invention also creates a cervical image generation model using a transfer learning technique. Furthermore, although the diffusion probability model is used in the embodiment of the present invention, it is also possible to use, and is not excluded, the types of generative AI models described below, such as generative adversarial networks (GANs), variational autoencoders (VAEs), and autoregressive models.

[0044] The overall production system through the features of the present invention described above is summarized as follows.

[0045] That is, the cervical image captured by the camera (110) is stored in the database server (130) by indicating whether it is negative / positive and the lesion through the interpretation (120) of a medical professional. The training data set is classified (210) in the stored database server (130) and preprocessed in the preprocessing unit (220). Based on the preprocessed data set, the generative AI model learning unit (230) learns by a diffusion probability model. Thereafter, the process of inferring this is performed in the inference unit (300), which is performed by inputting data from the inference input unit (310) of the input image or prompt as inference data in the preprocessing unit (320), and inputting the preprocessed inference data into the learned cervical image generation model inference unit (330) to generate a virtual image through the virtual cervical image output unit (300).

[0046] More specifically, referring to Fig. 1, first, the cervical image data collection unit (100) acquires a cervical image from a patient (110), and then, through various routes such as communication means, a medical professional reads whether the acquired cervical image is negative or positive (120), and if positive, a lesion is marked. Then, the positive image with the lesion marked as a result of the reading is stored in the database server (130).

[0047] As described above, the data collection unit (100) according to a preferred embodiment of the present invention may include a camera photographing unit (110) for photographing and acquiring a cervical image, a cervical image reading unit (120) for medical staff to read the photographed cervical image, and a database server storage unit (130) for storing the image reading unit (120) by a communication means.

[0048] In addition, according to a preferred embodiment of the present invention, as shown in Fig. 1, the data learning unit (200) undergoes a learning process based on the cervical image data collected in the data collection unit (100) described above. That is, the data learning unit (200) may include a training data set classification unit (210) that is transmitted from the database server storage unit (130) and classified, a training data set preprocessing unit (220) that preprocesses the training data set, and a diffusion probability model learning unit (230) that learns the preprocessed data by a generative AI model (diffusion probability model).

[0049] In a preferred embodiment of the present invention described above, a cervical positive image generation technique is described through FIG. 2.

[0050] That is, as shown in Fig. 2, the present invention develops three representative cervical image generation techniques (131). (1) An unconditional image generation technique (132) is used to generate a desired number of virtual cervical images, (2) an image conversion technique (133) is used to generate virtual cervical images through domain-to-domain conversion, such as changing a negative cervix into a positive cervix, and (3) a conditional image generation technique (134) is used to generate conditional cervical images according to various diseases and situations.

[0051] In other words, as described above, the cervical image is captured using a cervical magnification device (camera), then interpreted by a specialist, and stored in a database along with characteristics such as whether it is negative / positive and whether there is a lesion. The images stored in the database are preprocessed into training data suitable for the purpose using the three techniques introduced above. After that, the preprocessed cervical image dataset is input as training data to a diffusion probability model, which is a generative AI model, and trained. The input image or prompt is preprocessed to fit the trained model and inputted as the input of the trained model, and a virtual cervical image is generated through the inference process of the inference unit.

[0052] In addition, according to a preferred embodiment of the present invention, as shown in Fig. 1, a virtual benign image can be generated by the cervical image generation model (240) learned in the data learning unit (200) described above, and transmitted to the inference unit (300) to generate a virtual benign cervical image. That is, the inference unit (300) may include an inference input unit (310), an inference preprocessing unit (320) that preprocesses the input data, and an output unit (340) that applies the preprocessed data to the cervical image generation model inference unit (330) learned by the diffusion probability model, and outputs the virtual cervical image generated here.

[0053] In addition, according to a preferred embodiment of the present invention, as shown in FIG. 4, the virtual cervical benign image generated by the inference unit (300) may be stored in an artificial intelligence-based cervical cancer grade classification model learning database (500) after being interpreted by medical staff. That is, if the virtual cervical benign image generated by the output is evaluated by a medical professional as an image of the same level as an actual benign image, it may be stored in the artificial intelligence-based cervical cancer grade classification model learning database (500) as image data for improving the performance of the artificial intelligence-based cervical cancer grade classification model constructed for more effective use in an artificial intelligence-based cervical cancer screening service system to be actually used.

[0054] The following describes a method for generating a cervical image using a generative AI model according to one embodiment of the present invention.

[0055] The method for generating a cervical image using a generative AI model of the present invention is characterized by including a data collection step (S100) of acquiring and collecting cervical image data, a data learning step (S200) of learning the cervical image data collected in the data collection step (S100) by a generative AI model, and an inference step (S300) of generating and outputting a virtual cervical image by inferring input data by the cervical image generation model derived from the data learning unit (200).

[0056] The above data collection step (S100) may include a step (S110) of capturing a cervical image by a camera camera (110), a step (S120) of reading the cervical image by a medical professional, and a step (S130) of storing the read image data in a database server by a communication means.

[0057] In addition, the data learning step (S200) is characterized by including a training data set classification step (S210) in which a training data set is transmitted from the database server storage unit (130) and classified, a training data set preprocessing step (S220) in which the training data set is preprocessed, and a diffusion probability model learning step (S230) in which the preprocessed data is learned by a diffusion probability model.

[0058] In addition, the above-described inference step (S300) is characterized by including an inference input step (S310) for inputting an input image or prompt to be generated into an input unit (310), an inference preprocessing step (S320) for preprocessing the input data, a cervical image generation model inference step (S330) for applying the preprocessed data to a cervical image generation model learned by a diffusion probability model, and an output step (S340) for outputting a virtual cervical image generated in the generation model inference step (S330) through an output unit.

[0059] According to a preferred embodiment of the present invention, the present invention may further include a reading step (S400) in which a medical professional reads whether the virtual cervical benign image generated by the inference unit (300) is an actual benign image, and a storage step (S500) in which, after the reading step (S400), if the virtual cervical benign image is identical to the actual benign image, the image data is stored in a database (500) for learning an artificial intelligence-based cervical cancer grade classification model.

[0060] According to a preferred embodiment of the present invention, a retraining step (S600) for retraining the artificial intelligence-based cervical cancer grade classification model based on negative and positive image data stored in the database (500) for training the artificial intelligence-based cervical cancer grade classification model may be further included.

[0061] Embodiment 1 of the present invention described above selects an image conversion method (133) that generates a positive image based on a negative cervical image as shown in FIGS. 2 to 4, and Embodiment 2 selects a conditional image generation method (134) that generates a positive cervical image based on the positive cervical image.

[0062] When explaining Embodiments 1 and 2 through FIG. 4, the upper process of Embodiment 1 in FIG. 4 selects an image conversion method (133) to classify a negative cervical image in the classification unit (210) and executes an image generation process, and the lower process of Embodiment 2 selects a conditional image generation method (134) to classify a positive cervical image in the classification unit (210) and executes an image generation process.

[0063] More specifically, as shown in FIG. 4, the image conversion generation technique (133) of Example 1 requires three pairs of inputs to be made into the training data (220). That is, a negative cervix image is designated as the image before conversion, the prompt is designated as “positive cervix,” and the image after conversion is designated as a positive cervix image. This is preprocessed as the training data (220) of the image conversion technique (133), and then diffusion probability model learning (230) is performed. After diffusion probability model learning (230) is completed, the image before conversion is designated as a negative image as inference data (310, 320), and the prompt is designated as “positive cervix.” By preprocessing the above inference data and using it as input to the diffusion probability model to perform inference (330), a virtual positive cervix image is generated and output (340).

[0064] In addition, the conditional image generation technique (134) of Example 2 of FIG. 4 collects and classifies positive images from the database server (130) and uses them as images of training data (220), and the prompt specifies a condition as "positive cervix." The training data is preprocessed to learn the diffusion probability model (230). When the diffusion probability model learning (230) is completed, the prompt is specified as "positive cervix" as inference data (310, 320) and the number of images to be generated is specified. The above inference data is preprocessed and used as input to the diffusion probability model to perform inference (330), and a virtual positive cervix image is generated and output (340).

[0065] In addition, after the processes of the above-described embodiments 1 and 2 are completed, a medical staff reading (verification) step (400, S400) is performed to determine whether the image has the characteristics of an actual benign image. The reading (verification) step (400, S400) includes a specialist reading process, and if the image has the characteristics of a benign image, it is stored in a database for learning an artificial intelligence-based cervical cancer grade classification model (500, S500). The performance of the artificial intelligence-based cervical cancer grade classification model can be enhanced through a retraining process (600, S600) using the stored benign image.

[0066] Meanwhile, in the present invention, the 'unconditional image generation technique' is not considered a separate embodiment, but is added only as explanatory material to help understanding of the present invention.

[0067] In addition, FIG. 5a shows an example photograph of a virtual positive image generation according to Embodiment 1 of the present invention, and FIG. 5b shows an example photograph of a virtual positive image generation according to Embodiment 2 of the present invention.

[0068] Below, in order to help understand the preferred embodiment of the present invention described above, the types of generative AI models related to the present invention, namely the diffusion probability model and transfer learning, will be described in detail first, and other types of generative AI models will be briefly described later.

[0069] Description of Generative AI Models

[0070] Machine learning models can be broadly divided into classification and generative models, depending on their function. Classification models typically transform high-dimensional input data into low-dimensional labeled data, while generative models, on the other hand, generate high-dimensional data from relatively low-dimensional data.

[0071] Here, the types of generative AI models that generate the above images are explained, including generative adversarial networks (GANs), variational autoencoders (VAEs), autoregressive models, and diffusion models. These models are briefly described below.

[0072] First, the generative AI model used in the present invention is trained using a transfer learning method using a diffusion probability model.

[0073] Description of the Diffusion Probabilistic Model

[0074] The idea of ​​the diffusion probability model originated from Langevin dynamics, which describes the phenomenon in which molecules gathered in space are evenly distributed throughout space through diffusion. This is explained through the following examples.

[0075]

[0076] The movement of each molecule follows a Gaussian distribution, and can be expressed as a normal distribution with a mean and standard deviation. If the movement of each molecule can be calculated at each time point, the diffusion process can be reversed.

[0077]

[0078] By applying this directly to the pixel values ​​of the image, we add a noise value that follows a normal distribution to the pixel values ​​of the image, just as we viewed the movement of molecules as noise that follows a normal distribution.

[0079]

[0080] X0 is the original image. The image created by adding noise to all pixel values ​​of this image is X1. In the same way as the Diffusion Process of molecules, noise is added to all pixel values ​​from the image every time t. The image created through T times is a complete noise image, and in terms of molecules, it is a state where x0 is clumped together in the Diffusion Process, x T This means that the particles are completely evenly distributed. Furthermore, as mentioned earlier, if we can calculate the noise added at each time point, this process can be reversed. In the case of molecules, it is possible to revert from a state of even distribution across all space to a state of clustering in a single space.

[0081]

[0082] For the above image, if we can compute the noise added to the image pixel values ​​at each time t, then the noisy image x T It is possible to revert from the input image x0 to the input image x0. This can be explained by the following formula.

[0083]

[0084] The input image is x0 on the far right, and the noise image is x TThis process is divided into forward and reverse transformations. Forward transformation transforms the input image into a noise image, while reverse transformation generates input data from the noise image. Therefore, by utilizing the method of generating input data from a noise image, the diffusion probability model parameterizes the reverse transformation step and trains it as a deep learning model, generating input images from the noise image through the trained reverse transformation. Thus, the diffusion probability model can be considered a parameterized Markov chain trained to generate images after a finite time. It has the Markov property that the current state (t) depends on the previous state (t-1). The purpose of the diffusion probability model is to make the noise image that has undergone the forward and reverse transformation steps similar to the probability distribution of the input image. To this end, training progresses by updating the mean and standard deviation, which are the noise generation probability distribution parameters, during the reverse transformation step. Furthermore, since actual noise generation always samples from a Gaussian probability distribution with different values, the generated image continuously changes. Furthermore, the resulting image generated from the reverse transformation varies depending on the training data.

[0085] <Description of Learning in a Diffusion Probability Model>

[0086] In the present invention, a generative AI model uses a neural network, similar to the neurons in the human brain, to learn patterns and features from existing data. This model can then generate new data that matches the learned patterns and features. For example, a generative AI model trained on a set of cervical images can generate virtual cervical images similar to the trained cervical images.

[0087] As mentioned in Examples 1 and 2 above, the generative AI model of the present invention has three techniques: an unconditional image generation technique (132), a conditional image generation technique (133), and an image transformation technique (134). That is, although not a separate embodiment in the present invention, the unconditional image generation technique (132) generates a virtual cervix image using only a set of cervical images. In addition, the image transformation technique (134), which is Embodiment 1 of the present invention, generates a cervix image through methods such as style transformation, image restoration, domain-to-domain transformation, and location change. In addition, the conditional image generation technique (133), which is Embodiment 2 of the present invention, generates a cervix image based on the characteristics of a specific cervix.

[0088] The differences between the above unconditional image generation technique (132) and the conditional image generation technique (133) are as follows. For example, the unconditional image generation technique (132) can generate a large number of cervical images because it can generate images without data, while the conditional image generation technique (133) can generate cervical images tailored to specific situations. Examples include benign cervical images, negative cervical images, and cervical images in which white lesions are clearly visible.

[0089] <Description of Transfer Learning>

[0090] Furthermore, the present invention trains a diffusion probability model using transfer learning techniques. Transfer learning refers to the application of a trained model to a different task to achieve a specific goal. It utilizes the knowledge of a model already trained on a different dataset to a new task or dataset. It has the advantage of achieving high performance even with a small dataset and creating a model specialized for the desired test or domain. Transfer learning is divided into two stages: pretraining and fine-tuning. In the pretraining stage, the model is trained on a large dataset to learn general features. For example, when training an image recognition model, a large dataset like ImageNet is used to learn general features (e.g., edges, textures, etc.) of various images. In the fine-tuning stage, the pretrained model is tailored to a specific task. This process uses a small dataset to adapt the model to a specific domain. For example, a pretrained model is adapted to generate cervical images. This process uses the cervical dataset and retrains the last few layers of the model, achieving high performance with limited data and resources.

[0091]

[0092] As shown in Fig. 2, the first unconditional image generation technique (132) collects cervical images from a database server (130) and preprocesses them into training data (210). The preprocessed cervical image set is fed into the basic algorithm of the diffusion probability model (230) and trained.

[0093] Additionally, as shown in Fig. 2, the second image transformation method (133) preprocesses the training data into three pairs: a pre-transformation image, a prompt containing conditions, and a post-transformation image. The preprocessed dataset is fed into the Image to Image algorithm and trained.

[0094] Additionally, as illustrated in FIG. 2, the third conditional image generation technique (134) sets a condition containing a specific situation or case as a prompt and pairs two images that meet the condition, preprocessing them as training data. The preprocessed dataset is then fed into the Text to Image algorithm for training.

[0095]

[0096] As shown in Fig. 3, the first unconditional image generation technique (302) can generate a virtual cervical image by specifying the number of images to be generated from a diffusion probability model learned based on a set of cervical images.

[0097] Additionally, the second image transformation method (303) is a technique for transforming an input image into another image with specific properties or styles. A virtual image is generated based on the input image (311) and a prompt (310). For example, a negative cervix image and a prompt "positive cervix" are transformed into a virtual positive cervix image. Examples of transformation techniques include style transformation, image restoration, cross-domain transformation, and position transformation. Style transformation transforms a given image into a specific style. Specific styles include race, doctor, etc. If the characteristics of the cervix vary by race or doctor, the cervix image can be generated using a style transformation technique. Image restoration restores damaged or incomplete images. If a cervix image captured using a camera contains noise or is damaged, it can be transformed into a complete image. Cross-domain transformation transforms an image from one domain into another. For example, a negative cervix image can be transformed into a positive cervix image, or a positive cervix image can be transformed into a negative cervix image. Finally, positional transformation involves altering the position or appearance of a specific object. For example, transforming an image of the cervix's deformation zone into an image that shows the deformation zone. Or, transforming an image of the cervix's outer opening into an image that shows the outer opening.

[0098] In addition, the third conditional image generation technique (304) can generate a virtual negative cervix image by learning a negative cervix image. Or, it can generate a virtual positive cervix image by learning a positive cervix image. This generation technique is called conditional image generation. The condition is called a prompt (310), and the prompt is given as text. For example, in the conditional image generation model, if "negative cervix" is entered in the prompt and N images to be generated are specified (312), N negative cervix images are generated. In addition, various images can be generated based on all visual characteristics of the cervix. (Ex. benign cervix image, cervix image including a specific lesion, cervix image where the external os is not visible, etc. All situations can be learned and generated.)

[0099] As described above, a virtual cervical image is generated based on three major methods. As input methods for each technique, the unconditional image generation technique (302) uses the number of images to be generated (312), the image conversion technique (303) uses the image before conversion (311) and a prompt (310), and the conditional image generation technique (304) can be input using a prompt (310) and the number of images to be generated (312).

[0100] Meanwhile, among the types of generative AI models, we explain generative adversarial networks (GANs), variational autoencoders (VAEs), and autoregressive models.

[0101] Generative Adversarial Networks (GANs) Explained

[0102] A generative adversarial network (GAN) is a model comprised of two neural networks: a generator and a discriminator. In this model, the generator and discriminator compete against each other to generate data. The generator generates new data, and the discriminator determines whether this data is real or fake.

[0103]

[0104] The generator learns to generate data similar to real data by receiving the generated z, and the discriminator learns to distinguish between real data and fake data generated by the generator. The generator learns to identify the distribution of the input data and reproduces this distribution so that it is identical to the distribution of the original data. The discriminator distinguishes between real and fake data and estimates the probability of each. The ultimate goal of a generative adversarial network is to generate data that closely resembles the distribution of real data, and the generator tries to generate fake data that is similar to real data so that the discriminator cannot falsely identify it. Through this process, the performance of the generator and discriminator gradually improves, and the ultimate goal is to make the discriminator indistinguishable from real data.

[0105] Description of Variational Autoencoders (VAEs)

[0106] The variational autoencoder is a generative AI model that learns compressed representations of training data as probability distributions, and is used to generate new sample data by creating variations of the learned representations. First, the basic concept of an autoencoder is as follows. The process of inferring original data into latent variables suitable for the purpose is called encoding, and the process of restoring the encoded data back to the original data form is called decoding. A neural network composed of an encoder and a decoder in this way is called an autoencoder. While most autoencoders learn a discrete latent space model, a variational autoencoder learns a continuous latent variable model. Instead of a single encoding vector for the latent space, a variational autoencoder models two different vectors: a mean vector and a standard deviation vector.

[0107]

[0108] The input puppy image is connected to an encoder network. The encoder compresses the input image and transforms it into a low-dimensional latent space (latent vector). It's important to note that the encoder doesn't simply transform it into a single vector; it estimates two parameters: the mean and variance. These parameters define the probability distribution in the latent space. The latent vector is sampled based on the mean and variance obtained by the encoder. This is a random vector that follows a normal distribution, and a random sampling process is added during this process. This randomness allows the variational autoencoder to function as a generative AI model, not just a simple compressive model. The sampled latent vector is then passed to the decoder network, which reconstructs an image similar to the original. The decoder then receives the latent vector and transforms it back into high-dimensional image data. The final output from the decoder is an image very similar to the original input image. However, because the sampling of the latent vector involves randomness, it is not exactly the same as the original.

[0109] Description of Autoregressive Models

[0110] The term "autoregressive" above refers to a model that regresses against itself. This means predicting the next pixel value based on the previous pixel value. An autoregressive model generates an image through two processes: setting a pixel sequence and making a prediction based on the previous pixel.

[0111]

[0112] The image above is represented as a two-dimensional array. The autoregressive model unrolls the two-dimensional image into a one-dimensional sequence and sequentially processes each pixel. After generating the first pixel, it predicts and generates the second pixel based on that pixel. This process uses information from previous pixels to calculate the value of the next pixel. To achieve this, the model estimates the probability distribution of the next pixel using information from previously generated pixels and samples and generates new pixel values ​​based on that distribution. This process is repeated until all pixels are generated, resulting in the final image. Furthermore, images are generated pixel by pixel using recurrent neural networks or convolutional neural networks.

[0113] As described above, according to the present invention, in order to effectively perform an AI-based cervical cancer screening service, the problem of data imbalance between negative and positive image classes required for constructing an AI-based cervical cancer grade classification model can be resolved, and further, according to the present invention, by applying a diffusion probability model and a transfer learning technique to a generative AI model for negative and positive image data, a virtual cervical benign image can be generated and retrained, thereby enhancing the performance of a continuous AI-based cervical cancer grade classification model.

[0114] Although the preferred embodiments of the present invention have been described with reference to the attached drawings, the embodiments described in this specification and the configurations illustrated in the drawings are only the most preferred embodiments of the present invention and do not represent all of the technical ideas of the present invention, and therefore, it should be understood that there may be various equivalents and modified examples that can replace them at the time of filing this application. Therefore, the embodiments described above should be understood as illustrative and not restrictive in all respects, and the scope of the present invention is indicated by the claims described below rather than the detailed description, and all changes or modified forms derived from the meaning and scope of the claims and equivalent concepts should be interpreted as being included in the scope of the present invention.

[0115]

[0116] {Explanation of symbols}

[0117] 100: Data Collection Department

[0118] 110: Camera shooting section (image acquisition section)

[0119] 120: Cervical image interpretation unit

[0120] 130: Database server storage

[0121] 131: Generation Technique Selection Section

[0122] 132: Unconditional Image Generation Technique Selection Section

[0123] 133: Image Transformation Method Selection Section

[0124] 134: Conditional Image Generation Technique Selection Section

[0125] 143: Condition setting section

[0126] 144: Condition setting section

[0127] 200: Data Learning Department

[0128] 210: Training data set classification section

[0129] 220: Training Data Set Preprocessing

[0130] 230: Generative AI Model Learning Unit (Diffusion Probability Model)

[0131] 240: Cervical image generation model

[0132] 300: Inference Department

[0133] 301: Inference Generation Technique Selection Section

[0134] 302: Selection of unconditional image inference generation technique

[0135] 303: Image Transformation Method Inference Selection Section

[0136] 304: Conditional Image Inference Generation Technique Selection Section

[0137] 310: Inference input (input image or prompt)

[0138] 311: Image input section

[0139] 312: Setting the number of generated images

[0140] 320: Inference Preprocessing Unit

[0141] 330: Cervical Image Generation Model Inference Unit

[0142] 340; Virtual cervical benign image generation output section

[0143] 400: Medical staff interpretation of virtual generated images (positive)

[0144] 500: Database for AI-based cervical cancer grading model classification training

[0145] 600: AI-based cervical cancer grading model retraining department

[0146] 1000: Cervical Image Generation System Using Generative AI Models

Claims

1. Data collection unit for acquiring and collecting cervical image data; A data learning unit that learns the cervical image data collected from the above data collection unit using a generative AI model; A cervical image generation system using a generative AI model, characterized in that it includes an inference unit that generates and outputs a virtual cervical image by inferring input data using a cervical image generation model derived from the data learning unit.

2. In paragraph 1, A cervical image generation system using a generative AI model, characterized in that the generative AI model is a diffusion probability model.

3. In paragraph 2, The above data collection unit, A camera unit that captures images of the cervix; A cervical image interpretation unit in which the medical staff interprets the cervical image taken above; and A cervical image generation system using a generative AI model, characterized in that it includes a database server storage unit that stores the read image data by a communication means.

4. In paragraph 2, The above data learning unit, A training data set classification unit that is transmitted and classified from the above database server storage unit; and A training data set preprocessing unit for preprocessing the above training data set; and A system for generating cervical images using a generative AI model, characterized in that it includes a diffusion probability model learning unit that learns the above preprocessed data by a diffusion probability model.

5. In paragraph 2, The above reasoning part is, An inference input section that inputs an input image or prompt; and An inference preprocessing unit that preprocesses the above input data; and A cervical image generation model inference unit that applies the above preprocessed data to a cervical image generation model learned by a diffusion probability model; and A cervical image generation system using a generative AI model, characterized in that it includes an output unit that outputs a virtual cervical image generated by the generative model inference unit.

6. In any one of paragraphs 1 to 5, A cervical image generation system using a generative AI model, characterized in that it includes a database for learning an artificial intelligence-based cervical cancer grade classification model that stores virtual cervical benign images generated from the above inference unit after being read by medical staff.

7. Data collection step for acquiring and collecting cervical image data; A data learning step in which the cervical image data collected in the above data collection step is learned by a generative AI model; A method for generating a cervical image using a generative AI model, characterized by including an inference step for generating a virtual cervical image by inferring input data using a cervical image generation model derived in the above data learning step and outputting the generated virtual cervical image.

8. In paragraph 7, A method for generating a cervical image using a generative AI model, characterized in that the generative AI model is a diffusion probability model.

9. In paragraph 8, The above data collection steps are: Step of acquiring a cervical image by a camera; and A cervical image interpretation step in which the medical staff interprets the above-mentioned cervical image; and A method for generating a cervical image using a generative AI model, characterized in that it includes a database server storage step for storing the read image data by a communication means.

10. In paragraph 8, The data learning stage is A training data set classification step in which the training data set is transmitted from the database server storage unit and classified; and A training data set preprocessing step for preprocessing the above training data set; and A method for generating a cervical image using a generative AI model, characterized in that it includes a diffusion probability model learning step for learning the above preprocessed data by a diffusion probability model.

11. In paragraph 8, The above inference step is, An inference input step that inputs an input image or prompt into the input field; and An inference preprocessing step for preprocessing the above input data; and A cervical image generation model inference step that applies the above preprocessed data to a cervical image generation model learned by a diffusion probability model; and A method for generating a cervical image using a generative AI model, characterized in that it includes an output step for outputting a virtual cervical image generated in the above generative model inference step by an output unit.

12. In any one of paragraphs 7 to 11, A reading step in which medical staff reads whether the virtual cervical benign image generated in the above inference step is an actual benign image; and A method for generating a cervical image using a generative AI model, characterized in that it includes a storage step of storing the image data in a database for learning an artificial intelligence-based cervical cancer grade classification model if the image data is identical to an actual benign image after the reading step.

13. In paragraph 12, A method for generating a cervical image using a generative AI model, characterized in that it includes a retraining step of retraining an artificial intelligence-based cervical cancer grade classification model based on negative and positive image data stored in a database for learning the artificial intelligence-based cervical cancer grade classification model.

Images