Method for training artificial intelligence model that distinguishes between epithelium and stroma, and electronic device for performing same

Abstract

The present disclosure relates to a method for training an artificial intelligence model that distinguishes between epithelium and stroma, and an electronic device for performing same. The method according to one embodiment of the present disclosure may comprise the steps of: acquiring a tissue image including RGB channels; acquiring annotation data for segmenting the tissue image into an epithelial region and a stromal region; setting a value of a first channel from among the RGB channels to a maximum value with respect to the stromal region of the tissue image and setting a value of a second channel from among the RGB channels to a maximum value with respect to the epithelial region of the tissue image, thereby acquiring a composite tissue image; and training the artificial intelligence model by using the tissue image as input data and the composite tissue image as ground truth data.

Description

Method for training an artificial intelligence model that distinguishes between epithelium and stroma, and an electronic device for performing the same

[0001] The present disclosure relates to a method for training an artificial intelligence model that distinguishes between epithelium and stroma and an electronic device for performing the same, and more specifically, to a method and device for distinguishing between epithelium and stroma within a tissue image using an artificial intelligence model with an encoder-decoder structure.

[0002] The differentiation of epithelium and stroma in tissue images is a critical process in pathological diagnosis, and AI-based technologies to automate this process are being actively researched in the field of medical image analysis. In particular, image segmentation and pattern recognition technologies are showing promising results in distinguishing between epithelium and stroma, contributing to improved accuracy and efficiency in pathological diagnosis. Current technologies are becoming increasingly sophisticated in analyzing high-resolution tissue images to differentiate between the two, thereby increasing the potential for automating pathological diagnosis.

[0003] However, tissue images generally have very high resolution, so they are often divided into small patches for analysis. In this process, the annotation data required for training AI models is typically provided labeled at the level of very large Whole Slide Images (WSI). Consequently, there may be instances where the epithelium and stroma are not clearly distinguished at the small patch level. This prevents the model from accurately recognizing the boundary between the epithelium and stroma during training, which can ultimately affect diagnostic accuracy.

[0004] In particular, data labeled at the WSI level may not adequately reflect the detailed structure of high-resolution images. This leads to the problem of unclear learning of the boundaries between the epithelium and stroma at the small patch level, which can hinder AI models from providing reliable results in real-world diagnostic settings. Therefore, at the current level of technology, this issue remains a significant challenge in improving the accuracy and reliability of tissue image analysis.

[0005] The object of the present disclosure is to provide a method for training an artificial intelligence model that distinguishes epithelium and stroma, which uses an artificial intelligence model to distinguish specific regions within an image and trains the model, and an electronic device for performing the same.

[0006] In one embodiment of the present disclosure, a method for training an artificial intelligence model that distinguishes between epithelium and stroma may be provided. The method may include the steps of: acquiring a tissue image including RGB channels; acquiring annotation data that distinguishes the tissue image into an epithelial region and a stroma region; acquiring a synthetic tissue image by setting the value of a first channel among the RGB channels to a maximum value for the stroma region of the tissue image and setting the value of a second channel among the RGB channels to a maximum value for the epithelial region of the tissue image; and training an artificial intelligence model using the tissue image as input data and the synthetic tissue image as ground truth data.

[0007] In one embodiment of the present disclosure, the artificial intelligence model includes an encoder and a decoder, the encoder outputs an embedding vector with the tissue image as input, and the decoder outputs the synthetic tissue image with the embedding vector as input.

[0008] In one embodiment of the present disclosure, the method comprises the steps of: acquiring a whole slide image of the entire tissue; acquiring whole annotation data corresponding to the whole slide image; dividing the whole slide image into a plurality of patch images; and acquiring metadata for each of the plurality of patch images based on the whole annotation data, wherein one of the plurality of patch images corresponds to a tissue image and the metadata may correspond to the annotation data.

[0009] In one embodiment of the present disclosure, the synthetic tissue image may be an image in which the appearance of the tissue image is maintained and the epithelial region and the stromal region are distinguished by color.

[0010] In one embodiment of the present disclosure, the values ​​of the RGB channels of the tissue image may not be changed for the region excluding the epithelial region and the stroma region.

[0011] In one embodiment of the present disclosure, the step of acquiring the synthetic tissue image may include the step of acquiring the synthetic tissue image by setting the value of the third channel among the RGB channels to a maximum value for at least one of the epithelial region and the stroma region.

[0012] In one embodiment of the present disclosure, the first channel may be an R channel, the second channel may be a G channel, and the third channel may be a B channel. The step of acquiring the synthetic tissue image may include the step of setting the value of the R channel to a maximum value for the stromal region, and the step of setting the values ​​of the G channel and the B channel to a maximum value for the epithelial region.

[0013] In one embodiment of the present disclosure, the values ​​of the G channel and the B channel may be maintained for the substrate region, and the value of the R channel may be maintained for the epithelial region.

[0014] In one embodiment of the present disclosure, the step of acquiring the synthetic tissue image may include: adding the value of the first channel to the epithelial region of the tissue image by a maximum value; adding the value of the second channel to the stromal region of the tissue image by a maximum value; determining whether at least one of the value of the first channel and the value of the second channel exceeds the maximum value; and setting the value exceeding the maximum value as the maximum value based on the determination that at least one of the value of the first channel and the value of the second channel exceeds the maximum value.

[0015] In one embodiment of the present disclosure, an electronic device may be provided. The electronic device may include at least one processor including a processing circuit and at least one memory storing at least one instruction. By the at least one processor executing the at least one instruction, the electronic device may acquire a tissue image including RGB channels, acquire annotation data that divides the tissue image into an epithelial region and a stromal region, acquire a synthetic tissue image by setting the value of a first channel among the RGB channels to a maximum value for the stromal region of the tissue image and setting the value of a second channel among the RGB channels to a maximum value for the epithelial region of the tissue image, and train an artificial intelligence model using the tissue image as input data and the synthetic tissue image as ground truth data.

[0016] According to one embodiment of the present disclosure, by obtaining annotation data by distinguishing the epithelial and stromal regions of a tissue image and generating a synthetic tissue image by adjusting RGB channel values, an artificial intelligence model can be effectively trained to clearly distinguish between the epithelium and the stromal region even at the small patch level. According to one embodiment of the present disclosure, by training an artificial intelligence model using a synthetic tissue image as ground truth data, an artificial intelligence model capable of accurately distinguishing between the epithelium and the stromal region even in high-resolution tissue images can be implemented.

[0017] FIG. 1 is a conceptual diagram showing a method for training an artificial intelligence model according to one embodiment of the present disclosure.

[0018] FIGS. 2a and 2b are conceptual diagrams illustrating an image synthesis method according to one embodiment of the present disclosure.

[0019] FIGS. 2a and 2b are conceptual diagrams illustrating an image synthesis method according to one embodiment of the present disclosure.

[0020] FIG. 3 is a conceptual diagram illustrating, exemplarily, a method for processing annotation data according to one embodiment of the present disclosure.

[0021] FIG. 4 is a conceptual diagram exemplarily showing a method of dividing an entire slide image into patch images according to one embodiment of the present disclosure.

[0022] FIG. 5 is a conceptual diagram showing the inference operation of an artificial intelligence model according to one embodiment of the present disclosure.

[0023] FIG. 6 is a conceptual diagram showing the structure of an artificial intelligence model according to one embodiment of the present disclosure.

[0024] FIG. 7 is a block diagram showing an electronic device according to one embodiment of the present disclosure.

[0025] FIG. 8 is a flowchart showing a method for training an artificial intelligence model that distinguishes between epithelium and stroma according to one embodiment of the present disclosure.

[0026] FIG. 8 is a flowchart showing a method for training an artificial intelligence model that distinguishes between epithelium and stroma according to one embodiment of the present disclosure.

[0027] Figure 10 is a flowchart showing the detailed steps of step S830 of Figure 8.

[0028]

[0029] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. The embodiments will be described clearly and in detail so that a person skilled in the art can easily practice the present disclosure. However, the scope of the rights is not limited or restricted by these embodiments. Identical or similar reference numerals are used for similar components in each drawing, and redundant descriptions of identical or similar components are omitted.

[0030] The terms used in the following description have been selected as common and universal in the relevant technical field, but other terms may exist depending on technological development and / or changes, conventions, preferences of the skilled technician, etc. Therefore, the terms used in the following description should not be understood as limiting the technical concept, but as illustrative terms to explain the embodiments.

[0031] In addition, there are terms arbitrarily selected by the applicant in specific cases, and their detailed meanings will be described in the relevant explanatory section. Therefore, the terms used in the description below must be understood not merely as their names, but based on their meanings and the content throughout the specification.

[0032] Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art as described in this specification. Additionally, terms including ordinal numbers, such as "first" or "second," used in this specification may be used to describe various components, but said components should not be limited by said terms. Such terms are used solely for the purpose of distinguishing one component from another.

[0033] When a part of a specification is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, terms such as "part" or "module" as used in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware or software, or as a combination of hardware and software.

[0034] Embodiments of the present disclosure are described below with reference to the attached drawings so that those skilled in the art can easily implement them. However, the present disclosure may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present disclosure in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification have been given similar reference numerals. Also, the reference numerals used in each drawing are for the purpose of explaining each drawing, and different reference numerals used in different drawings are not intended to indicate different elements. The present disclosure will be described in detail below with reference to the attached drawings.

[0035] In the present disclosure, functions related to 'Artificial Intelligence' are operated through a processor and memory. The processor may be composed of one or more processors. In this case, the one or more processors may be general-purpose processors such as CPUs, APs, and DSPs (Digital Signal Processors), graphics-dedicated processors such as GPUs and VPUs (Vision Processing Units), or AI-dedicated processors such as NPUs. The one or more processors control the processing of input data according to predefined operation rules or AI models stored in memory. Alternatively, if the one or more processors are AI-dedicated processors, the AI-dedicated processors may be designed with a hardware structure specialized for processing a specific AI model.

[0036] The predefined rules of operation or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that a basic artificial intelligence model is trained using a number of training data by a learning algorithm, thereby creating predefined rules of operation or artificial intelligence models configured to perform desired characteristics (or objectives). Such learning may be performed on the device itself where the artificial intelligence according to the present disclosure is executed, or it may be performed through a separate server and / or system. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but are not limited to the examples described above.

[0037] In the present disclosure, an 'artificial intelligence model' may be composed of a plurality of neural network layers. Each of the plurality of neural network layers has a plurality of weight values ​​and performs neural network operations through operations between the results of operations of a previous layer and the plurality of weights. The plurality of weights possessed by the plurality of neural network layers may be optimized by the learning results of the deep neural network model. For example, the plurality of weights may be updated so that the loss value or cost value obtained from the deep neural network model during the learning process is reduced or minimized. For example, the deep neural network model may include, but is not limited to, a CNN (Convolutional Neural Network), DNN (Deep Neural Network), RNN (Recurrent Neural Network), RBM (Restricted Boltzmann Machine), DBN (Deep Belief Network), BRDNN (Bidirectional Recurrent Deep Neural Network), or Deep Q-Networks.

[0038] In the present disclosure, the term 'artificial intelligence model' may refer to an encoder-decoder based neural network model that generates a structured output similar to the input.

[0039] In the present disclosure, "tissue" may refer to a collection of cells that perform a specific function within an organism. In the present disclosure, "epithelial" may refer to a layer of cells that biologically cover the surface of skin or organs. In the present disclosure, "matrix" may refer to a structural support or environment that biologically surrounds cells.

[0040] In the present disclosure, 'RGB channels' may refer to red (R), green (G), and blue (B) channels used to represent colors in digital images. In one embodiment of the present disclosure, RGB channels may be used to distinguish between epithelial and stromal regions of a tissue image and to generate a synthetic tissue image.

[0041] In the present disclosure, 'annotation data' may refer to labeling data that provides additional information about the data. In one embodiment of the present disclosure, the annotation data includes information that separates a tissue image into an epithelial region and a stromal region, and may consist of image coordinate values ​​and labeling values.

[0042] In the present disclosure, 'ground truth data' may refer to ground truth data referenced during model training. In one embodiment of the present disclosure, ground truth data is defined as synthetic tissue images and may be used as a standard for training an artificial intelligence model.

[0043] In the present disclosure, 'embedding vector' may refer to a vector representation that transforms high-dimensional data into a low-dimensional space. In one embodiment of the present disclosure, the embedding vector is generated by an encoder and used by a decoder to generate a synthetic tissue image.

[0044] In the present disclosure, 'Whole Slide Image (WSI)' may refer to a digital image file capturing the entire tissue. In one embodiment of the present disclosure, the whole slide image may be divided into a plurality of patch images.

[0045] FIG. 1 is a conceptual diagram showing a method for training an artificial intelligence model according to one embodiment of the present disclosure.

[0046] Referring to FIG. 1, the tissue image (10) is an image of a tissue that may show the structural and morphological characteristics of the tissue. The tissue image (10) may be one of the patch images divided from the entire slide image. The tissue image (10) may be input into an image synthesis module (100) to be used to generate a synthetic tissue image (30), and may be input into an artificial intelligence learning module (200) to be used to learn an artificial intelligence model (210). For example, the tissue image (10) may be pathological tissue, biological tissue, cell tissue, vascular tissue, nerve tissue, muscle tissue, or connective tissue. However, the present disclosure is not limited thereto.

[0047] The tissue image (10) includes RGB channels and can be distinguished into an epithelial region (ER) and a stromal region (SR) through annotation data (20). By adjusting the values ​​of the RGB channels, it can be used to generate a synthetic tissue image (30) that can clearly distinguish between the epithelial region (ER) and the stromal region (SR). According to one embodiment of the present disclosure, this can contribute to generating ground truth data required for training an artificial intelligence model (210).

[0048] The tissue image (10) may be an image segmented from a high-resolution image, and as it is segmented from a high-resolution image, an inaccuracy of the annotation data (20) may occur. According to one embodiment of the present disclosure, through the synthesis of the tissue image (10) and the annotation data (20), information can be provided that can clearly distinguish incorrectly labeled areas, such as other areas (OR).

[0049] Annotation data (20) may include information distinguishing between epithelial regions (ER) and stromal regions (SR) within a tissue image (10). For example, annotation data (20) may be data consisting of coordinate values ​​and labeling values ​​(whether epithelial or stromal) in an image frame of the tissue image (10). Annotation data (20) may be used to generate a synthetic tissue image (30) or to train an artificial intelligence model (210) by providing region information related to the tissue image (10). For example, annotation data (20) may be an epithelial region (ER), stromal region (SR), other region (OR), background region, boundary region, overlapping region, or undefined region. However, the present disclosure is not limited thereto.

[0050] According to one embodiment of the present disclosure, the values ​​of the RGB channels of the tissue image (10) can be set differently for each region to clearly distinguish between the epithelial region (ER) and the stromal region (SR). For example, the values ​​of the first channel among the RGB channels can be set to the maximum value for the stromal region (SR), and the values ​​of the second channel among the RGB channels can be set to the maximum value for the epithelial region (ER), thereby clearly distinguishing between the epithelial region (ER) and the stromal region (SR).

[0051] The annotation data (20) can be used to generate a synthetic tissue image (30) based on information distinguishing between the epithelial region (ER) and the stromal region (SR). Through this, the synthetic tissue image (30) can be used as ground truth data in the training of the artificial intelligence model (210).

[0052] The annotation data (20) may include other regions (OR). Other regions (OR) may refer to regions that are not actually epithelial regions (ER) or stromal regions (SR) within regions labeled as epithelial regions (ER) or stromal regions (SR). When training an artificial intelligence model (210) using inaccurate annotation data as ground truth data, a problem may occur where other regions (OR) are trained as epithelial regions (ER) or stromal regions (SR); however, when training an artificial intelligence model (210) using a synthetic tissue image (30) according to one embodiment of the present disclosure as ground truth data, accurate results can be derived from inaccurate annotation data.

[0053] The image synthesis module (100) can generate a synthetic tissue image (30) based on a tissue image (10) and annotation data (20). The image synthesis module (100) can generate a synthetic tissue image (30) in which the stromal region (SR) and the epithelial region (ER) are distinguished by manipulating the RGB channels of the tissue image (10). The image synthesis module (100) can generate a synthetic tissue image in which the stromal region (SR) and the epithelial region (ER) are clearly distinguished by color by setting the value of the first channel for the stromal region (SR) to a maximum value and the value of the second channel for the epithelial region (ER) to a maximum value. Additionally, the image synthesis module (100) can generate a synthetic tissue image (30) in which a mislabeled region, such as an other region (OR), is clearly expressed as not belonging to either the stromal region (SR) or the epithelial region (ER).

[0054] The synthetic tissue image (30) is an image generated based on the tissue image (10) and annotation data (20), and can provide an image in which the epithelial region (ER) and the stromal region (SR) are clearly distinguished. It includes RGB channels and can be used as ground truth data for training an artificial intelligence model (210) in an artificial intelligence learning module (200). The synthetic tissue image (30) can be generated by setting specific values ​​of the RGB channels to clearly distinguish the epithelial region (ER) and the stromal region (SR).

[0055] The artificial intelligence learning module (200) can train the artificial intelligence model (210) using a synthetic tissue image (30). In one embodiment of the present disclosure, the artificial intelligence model (210) can distinguish between the epithelial region (ER) and the stromal region (SR) based on an encoder-decoder structure. It can be trained in a direction that reduces the loss value derived by comparing an image generated from the tissue image (10) with the synthetic tissue image (30). The artificial intelligence model (210) can derive a result that distinguishes between the epithelial region (ER) and the stromal region (SR) from a tissue image (10) containing RGB channels.

[0056] The artificial intelligence model (210) can be trained to distinguish between the epithelial region (ER) and the stromal region (SR) for various types of tissue images (10). The accuracy of distinguishing between the epithelial region (ER) and the stromal region (SR) can be increased depending on the resolution, color, or characteristics of the tissue image (10). The performance of distinguishing between the epithelial region (ER) and the stromal region (SR) can be further improved by utilizing additional training data.

[0057] FIGS. 2a and 2b are conceptual diagrams illustrating an image synthesis method according to one embodiment of the present disclosure. Regarding the image synthesis module (100), tissue image (10), annotation data (20), and synthetic tissue image (30), details that overlap with those described in FIG. 1 will be omitted.

[0058] Referring to FIG. 2a, the image synthesis module (100) can generate a synthetic tissue image (30) by separating the RGB channels of the tissue image (10) and adjusting the values ​​of the RGB channels for each of the epithelial region and the stromal region of the tissue image (10) based on annotation data (20). The image synthesis module (100) can provide ground truth data for training an artificial intelligence model that distinguishes between the epithelial region and the stromal region.

[0059] In one embodiment of the present disclosure, the image synthesis module (100) can add a maximum value to the value of a first channel corresponding to the stromal region of the annotation data (20) and add a maximum value to the value of a second channel corresponding to the epithelial region for the RGB value of a specific pixel of the tissue image (10). If the result of the addition exceeds the maximum value, the excess value can be subtracted to adjust the value of the corresponding channel so that it does not exceed the maximum value. A synthesized tissue image (30) generated through this process can be output.

[0060] The tissue image (10) may provide an original image of tissue including epithelium and stroma. The tissue image (10) includes RGB channels and can be used to generate a synthetic tissue image (30) in which the epithelium and stroma are clearly distinguished by setting the values ​​of the RGB channels differently based on annotation data (20) that distinguishes the epithelial region and the stroma region. For regions other than the epithelial region and the stroma region, the values ​​of the RGB channels of the tissue image (10) may be maintained so as not to change.

[0061] The image synthesis module (100) can adjust the respective channel values ​​for the epithelial region and the stromal region based on the RGB values ​​of a specific pixel. The image synthesis module (100) can generate a synthetic tissue image (30) by setting the value of the second channel (e.g., G channel) for the epithelial region to a maximum value and the value of the first channel (e.g., R channel) for the stromal region to a maximum value. In one embodiment of the present disclosure, the set value may be a value defined by the user or manufacturer rather than a maximum value.

[0062] The annotation data (20) includes information that distinguishes the tissue image (10) into an epithelial region and a stromal region, and can provide data indicating whether each pixel corresponds to an epithelial region or a stromal region. It is designed to clearly distinguish between the epithelial region and the stromal region, and can be used to set the value of the second channel among the RGB channels to a maximum value for the epithelial region and the value of the first channel to a maximum value for the stromal region.

[0063] A synthetic tissue image (30) can generate an image in which the epithelial region and the stroma region are distinguished by color while maintaining the appearance of the tissue (structural characteristics and / or morphological characteristics of the tissue) in the tissue image (10). The synthetic tissue image (30) can be used as ground truth data for training an artificial intelligence model that distinguishes between the epithelium and the stroma. The synthetic tissue image (30) can be generated by setting the value of the first channel for the stroma region among the RGB channels to a maximum value and the value of the second channel for the epithelial region to a maximum value. The synthetic tissue image (30) is generated so that the values ​​of the RGB channels of the tissue image (10) are not changed for the region excluding the epithelial region and the stroma region, and can be adjusted so that the RGB channel values ​​do not exceed the maximum value. According to one embodiment of the present disclosure, this can contribute to improving the performance of an artificial intelligence model that distinguishes between the epithelium and the stroma. According to one embodiment of the present disclosure, the annotation data (20) does not maintain the appearance of the tissue and distinguishes the epithelial region and the stroma region, but the synthetic tissue image (30) maintains the appearance of the tissue and can be designed to clearly distinguish the epithelium and stroma even when based on incorrect annotation data.

[0064] The image synthesis module (100) can generate a synthetic tissue image (30) by adjusting (or manipulating) the values ​​of RGB channels to distinguish between the epithelial region and the stromal region within the tissue image (10). The image synthesis module (100) can clearly distinguish between the epithelial region and the stromal region by setting the value of a specific channel to a maximum value based on the annotation data (20) or by adding the maximum value to the value of a specific channel. If the result of the addition exceeds the maximum value, the image synthesis module (100) can subtract the excess value to adjust the value of the corresponding channel so that it does not exceed the maximum value.

[0065] For example, as illustrated in FIG. 2a, the first channel may be an R channel, the second channel may be a G channel, and the third channel may be a B channel, but the present disclosure is not limited thereto. The image synthesis module (100) may set the value of the first channel of the tissue image (10) corresponding to the matrix region of the annotation data (20) to a maximum value. The image synthesis module (100) may set the value of the second channel of the tissue image (10) corresponding to the epithelial region of the annotation data (20) to a maximum value. For example, the RGB values ​​of a specific pixel of the tissue image (10) may be (a, b, c). The value of the first channel may be a, the value of the second channel may be b, and the value of the third channel may be c. The image synthesis module (100) may add the maximum value (Max_a) of the first channel to the value (a) of the first channel of the tissue image (10) corresponding to the matrix region of the annotation data (20). The image synthesis module (100) can add the maximum value (Max_b) of the second channel to the value (b) of the second channel of the tissue image (10) corresponding to the epithelial region of the annotation data (20). If the value of the corresponding channel exceeds the maximum value as a result of the addition, the image synthesis module (100) can adjust the value of the corresponding channel so that it does not exceed the maximum value by subtracting the excess amount.

[0066] In one embodiment of the present disclosure, the maximum value may be a value between 0 and 255 for each of the RGB channels, and the maximum value may be 255.

[0067] In one embodiment of the present disclosure, the structural and / or morphological characteristics of the tissue can be distinguished more clearly as the value of the R channel in the substrate region increases. Therefore, by processing the value of the R channel in the substrate region to a maximum value, the performance of the trained artificial intelligence model can be improved.

[0068] Referring to FIG. 2b, the image synthesis module (100) can separate the RGB channels of the tissue image (10) into a first channel, a second channel, and a third channel. A synthetic tissue image (30) can be generated by setting the value of the first channel corresponding to the stromal region of the annotation data (20) to a maximum value, and setting the values ​​of the second channel and the third channel corresponding to the epithelial region to a maximum value. The synthetic tissue image (30) may be in a state where the value of the first channel in the stromal region is a maximum value, and the values ​​of the second channel and the third channel in the epithelial region are set to a maximum value. However, the present disclosure is not limited thereto, and the values ​​of two channels in the stromal region may be set to a maximum value, and the value of one channel in the epithelial region may be set to a maximum value.

[0069] For example, the RGB values ​​of a specific pixel of the tissue image (10) may be (a, b, c). The value of the first channel may be a, the value of the second channel may be b, and the value of the third channel may be c. The image synthesis module (100) may add the maximum value of the first channel (Max_a) to the value of the first channel (a) of the tissue image (10) corresponding to the stromal region of the annotation data (20). The image synthesis module (100) may add the maximum value of the second channel (Max_b) to the value of the second channel (b) of the tissue image (10) corresponding to the epithelial region of the annotation data (20). The image synthesis module (100) may add the maximum value of the second channel (Max_c) to the value of the second channel (c) of the tissue image (10) corresponding to the epithelial region of the annotation data (20). The image synthesis module (100) can adjust the value of a corresponding channel so that it does not exceed the maximum value by subtracting the excess amount when the value of the corresponding channel exceeds the maximum value as a result of the addition. The image synthesis module (100) can output a synthetic tissue image (30) as the result of the addition.

[0070] FIG. 3 is a conceptual diagram illustrating an exemplary method for processing annotation data according to one embodiment of the present disclosure. Regarding the stroma region (SR), epithelial region (ER), and other region (OR), details that overlap with those described in FIG. 1 will be omitted.

[0071] Referring to FIG. 3, unlike the annotation data (20) illustrated in FIG. 1, the annotation data (21) can provide data for distinguishing the epithelial region (ER), stromal region (SR), and other region (OR) within the tissue image. That is, the annotation data (21) may include other regions (OR) other than the epithelial region (ER) and stromal region (SR). The other region (OR) may refer to the entire region or a partial region other than the epithelial region (ER) and stromal region (SR).

[0072] Annotation data (21) can be separated into stromal region annotation data (22) and epithelial region annotation data (23), and a synthetic tissue image can be generated using the separated data. Data corresponding to other regions (OR) may not be utilized during the synthesis process. According to one embodiment of the present disclosure, when used as training data for an artificial intelligence model, data corresponding to other regions (OR) is not utilized, thereby simplifying the training of the artificial intelligence model.

[0073] However, the present disclosure is not limited thereto, and other regions (OR) are data distinguished from epithelial regions (ER) and stromal regions (SR) within tissue images and can be used for training artificial intelligence models.

[0074] The stromal region annotation data (22) or the epithelial region annotation data (23) may include information indicating the characteristics of the region, for example, pixel-level location information, color information, texture information, boundary information, shape information, density information, or pattern information. However, the present disclosure is not limited thereto. According to one embodiment of the present disclosure, the stromal region annotation data (22) or the epithelial region annotation data (23) may include additional data processing to more clearly distinguish the boundaries of the region within the tissue image. For example, the stromal region annotation data (22) or the epithelial region annotation data (23) may include information that highlights the boundary line between the stromal region (SR) and the epithelial region (ER), or highlights a specific pattern within the region. Such additional data processing may support an artificial intelligence model in learning the stromal region (SR) more accurately.

[0075] FIG. 4 is a conceptual diagram exemplarily showing a method of dividing an entire slide image into patch images according to one embodiment of the present disclosure. Regarding the tissue image (10), details that overlap with those described in FIG. 1, FIG. 2a, and FIG. 2b will be omitted.

[0076] Referring to FIG. 4, the whole slide image (40) may provide an image including the overall structure of the tissue. For example, the whole slide image (40) may be a digital pathology image, a microscopic image, a tissue slide scan image, a high-resolution tissue image, a tissue sample image, a tissue cross-sectional image, or a biological tissue image. However, the present disclosure is not limited thereto.

[0077] The entire slide image (40) may be divided into a plurality of patch images, such as the tissue image (10). However, the present disclosure is not limited thereto, and the entire slide image (40) may first be divided into a plurality of Region of Interest (ROI) images in which tissue cells exist, and then each of the plurality of ROI images may be divided into a plurality of patch images.

[0078] The entire slide image (40) can be input into an image preprocessing module (300) and divided into multiple patch images. Additionally, the entire slide image (40) can provide basic data for generating annotation data for each of the divided patch images.

[0079] The image preprocessing module (300) can divide the entire slide image (40) into a plurality of patch images and generate metadata for each of the divided patch images. For example, the image preprocessing module (300) may be an image splitting device, a data processing device, an image analysis device, a data generation device, an image conversion device, a data classification device, or a data mapping device. However, the present disclosure is not limited thereto.

[0080] The image preprocessing module (300) can generate annotation data for each of the patch images generated based on the entire annotation data as metadata. The metadata may be annotation data in which the epithelial region and the stroma region of each of the patch images are distinguished. Additionally, the image preprocessing module (300) can generate data that can clearly express the distinction between the epithelium and the stroma.

[0081] In one embodiment of the present disclosure, the image preprocessing module (300) can distinguish between the epithelial region and the stromal region of each patch image based on annotation data generated using an artificial intelligence model with an encoder-decoder structure. The image preprocessing module (300) can generate data in which the epithelium and the stromal region are clearly distinguished by setting the values ​​of the RGB channels differently, and such data can be used for training the artificial intelligence model.

[0082] According to one embodiment of the present disclosure, the entire slide image (40) is a high-resolution image, and the entire annotation data corresponding to the entire slide image (40) may contain inaccurate labeling information at the level of small patch units. That is, the higher the resolution and the larger the size, the higher the inaccuracy of the labeling information may be. According to one embodiment of the present disclosure, ground truth data that presents an accurate answer from inaccurate labeling information can be generated through the image synthesis method described in FIGS. 1 to 2b.

[0083] FIG. 5 is a conceptual diagram showing the inference operation of an artificial intelligence model according to one embodiment of the present disclosure. Regarding the artificial intelligence model (210), details that overlap with those described in FIG. 1 will be omitted.

[0084] Referring to FIG. 5, the input image (50) may refer to image data that can be used as input to an artificial intelligence model (210). The input image (50) may be in the form of a tissue image. For example, the input image (50) may be a tissue image, a medical image, a microscope image, a digital pathology image, a biological sample image, a cell image, or a tissue slide image. However, the present disclosure is not limited thereto.

[0085] The artificial intelligence model (210) can receive an input image (50) as input and generate (or infer) an output image (60) in which the epithelial region and the stromal region are distinguished while maintaining the tissue appearance of the input image (50). The artificial intelligence model (210) may include an encoder (211) and a decoder (212). The encoder (211) takes the input image (50) as input and outputs an embedding vector, and the decoder (212) takes the embedding vector as input and outputs a synthetic tissue image.

[0086] The encoder (211) can receive an input image (50) as input and output an embedding vector. The encoder (211) is a component of the artificial intelligence model (210) and can perform the role of extracting features of the input image (50) and converting them into an embedding vector. For example, the encoder (211) may be a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Transformer, an Autoencoder, a Graph Neural Network (GNN), a Deep Neural Network (DNN), or a Restricted Boltzmann Machine (RBM). However, the present disclosure is not limited thereto.

[0087] In one embodiment of the present disclosure, the encoder (211) may be designed to be part of a pre-trained artificial intelligence model (210) to accommodate various resolutions and sizes of the input image (50). By applying a multi-scale feature extraction technique, the detailed features and overall structure of the input image (50) can be analyzed simultaneously. Through this, the encoder (211) can effectively provide information for distinguishing between the epithelium and the stroma while maintaining the tissue appearance of the input image (50).

[0088] In one embodiment of the present disclosure, the encoder (211) may include a preprocessing step of the input image (50). By performing noise removal, color correction, or resolution adjustment, the quality of the input image (50) can be improved and the accuracy in the learning and inference process can be increased.

[0089] The decoder (212) is a component of the artificial intelligence model (210) and can receive an embedding vector as input and output a synthetic tissue image. The decoder (212) can perform the role of generating an output image based on the input embedding vector. For example, the decoder (212) may be a Convolutional Neural Network (CNN), a Recurrent Neural Network (RNN), a Transformer, an Autoencoder, a Generative Adversarial Network (GAN), a Variational Autoencoder (VAE), or a U-Net structure. However, the present disclosure is not limited thereto.

[0090] The decoder (212) can generate a synthetic tissue image including RGB channels. The decoder (212) can output a synthetic tissue image in which the epithelial region and the stromal region are distinguished by color while maintaining the appearance of the tissue image based on the input embedding vector.

[0091] In one embodiment of the present disclosure, the decoder (212) is part of a pre-trained artificial intelligence model (210) and can generate an output image (60) in which the epithelium and stroma are distinguished based on an embedding vector extracted from an input image (50). The output image (60) can represent an image in which the epithelial region and the stroma region are distinguished while maintaining the tissue appearance of the input image (50).

[0092] FIG. 6 is a conceptual diagram showing the structure of an artificial intelligence model according to one embodiment of the present disclosure. Regarding the artificial intelligence model (210), details that overlap with those described in FIG. 1 and FIG. 5 will be omitted.

[0093] The artificial intelligence model (210) may be composed of at least one input layer (IL), at least one downsampling layer (DL), at least one bottleneck layer (BL), at least one upsampling layer (UL), and at least one output layer (OL). The input layer (IL) may receive an input image, such as a tissue image (or a patch image of a tissue sample), and generate a first feature vector. The downsampling layer (DL) may reduce the first feature vector to generate a second feature vector. The bottleneck layer (BL) may extract a third feature vector based on the second feature vector. The upsampling layer (UL) may expand the third feature vector to generate a fourth feature vector. The output layer (OL) may receive the first feature vector and the fourth feature vector and finally output an output image in which the epithelial region and the stromal region are separated in the input image.

[0094] The input layer (IL) can receive an input image and output a first feature vector. The input layer (IL) can extract features of the input image through a convolution operation. The input layer (IL) is an initial layer of the artificial intelligence model (210) and can transmit the first feature vector to the downsampling layer (DL).

[0095] In one embodiment of the present disclosure, the input layer (IL) may have a structure capable of performing various types of convolution operations. For example, the input layer (IL) may perform 2D convolution, 3D convolution, deep convolution, or multi-channel convolution. However, the present disclosure is not limited thereto. The input layer (IL) may have a flexible structure capable of processing various types of patch inputs.

[0096] The downsampling layer (DL) can reduce the input first feature vector to output a second feature vector. For example, the downsampling layer (DL) may use methods such as max pooling, average pooling, and stride convolution. However, the present disclosure is not limited thereto.

[0097] The bottleneck layer (BL) is the deepest layer in the artificial intelligence model (210) and can extract key features of the input data. The bottleneck layer (BL) can operate between the downsampling layer (DL) and the upsampling layer (UL). The bottleneck layer (BL) can receive a second feature vector as input and output a third feature vector.

[0098] The upsampling layer (UL) can take the second feature vector and the third feature vector as input and output a fourth feature vector. The upsampling layer (UL) can concatenate the second feature vector to the third feature vector. The upsampling layer (UL) can expand the third feature vector to output a fourth feature vector. The upsampling layer (UL) can perform up-convolution on the third feature vector. The fourth feature vector can be passed to the output layer (OL).

[0099] The output layer (OL) can ultimately generate an output image. The output layer (OL) takes a first feature vector and a fourth feature vector as input and can infer an output image in which the structural and / or morphological characteristics of the input image are preserved and the stroma and epithelium are distinguished by color. The upsampling layer (UL) can concatenate the first feature vector to the fourth feature vector. For example, the output layer (OL) can generate a final output image by processing the input feature vectors through a convolution operation.

[0100] In one embodiment of the present disclosure, the output layer (OL) may output an image having RGB channels. According to one embodiment of the present disclosure, the output layer (OL) may output an image in which the stromal portion and the epithelial portion are visually distinguished within normal tissue. For example, in a portion presumed to be the stromal portion within a patch, the value of at least one first channel (e.g., R channel) among the RGB channels may be set to a first value (e.g., 255). For example, in a portion presumed to be the epithelial portion within a patch, the value of at least one channel (e.g., second channel (G channel) and third channel (B channel)) excluding at least one first channel among the RGB channels may be set to a second value (e.g., 255). However, the present disclosure is not limited thereto, and at least one value within the RGB channel may be arbitrarily adjusted so that the output layer (OL) outputs an image in which the stromal portion and the epithelial portion are visually distinguished.

[0101] In one embodiment of the present disclosure, the output layer (OL) may have a flexible structure that can be applied to various artificial intelligence models. The output layer (OL) may be replaced with the output layer of another artificial intelligence model, thereby enabling analysis of various tissue samples. This structural flexibility allows the output layer (OL) to be applied to various application fields.

[0102] FIG. 7 is a block diagram showing an electronic device according to one embodiment of the present disclosure. Regarding the image synthesis module (100), artificial intelligence learning module (200), artificial intelligence model (210), and image preprocessing module (300), details that overlap with those described in FIG. 1, FIG. 2a, FIG. 2b, FIG. 4, FIG. 5, and FIG. 6 will be omitted.

[0103] Referring to FIG. 7, the electronic device (1000) can train and execute an artificial intelligence model that distinguishes between epithelium and stroma within a tissue image. The electronic device (1000) may include a processor (1100), storage (1200), a communication interface (1300), and memory (1400). For example, the electronic device (1000) may be a computer, a server, a smartphone, a tablet, an embedded system, a network device, a data processing device, or other electronic device. However, the present disclosure is not limited thereto.

[0104] The electronic device (1000) can acquire a tissue image including RGB channels and annotation data that separates the tissue into an epithelial region and a stromal region. The electronic device (1000) can generate a synthetic tissue image by setting the value of the first channel among the RGB channels to the maximum value for the stromal region of the tissue image and the value of the second channel among the RGB channels to the maximum value for the epithelial region. The electronic device (1000) can train an artificial intelligence model using the tissue image as input data and the synthetic tissue image as ground truth data.

[0105] In one embodiment of the present disclosure, the electronic device (1000) can generate ground truth data to distinguish between the epithelium and stroma within a tissue image and train an artificial intelligence model based thereon. By executing the trained artificial intelligence model, the epithelium and stroma within the tissue image can be distinguished, thereby clearly representing the appearance of the tissue image and providing data in which the epithelium and stroma are clearly distinguished.

[0106] The processor (1100) can perform overall control and data processing of the electronic device (1000). The processor (1100) can implement the functions of the electronic device (1000) by executing various modules stored in memory (1400). For example, the processor (1100) may be a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing unit (DSP), an application processor, a microcontroller, a neural network processing unit (NPU), or a field programmable gate array (FPGA). However, the present disclosure is not limited thereto.

[0107] The processor (1100) can train and execute an artificial intelligence model (210) for distinguishing between epithelium and stroma within a tissue image. The processor (1100) can acquire a tissue image containing RGB channels and annotation data that distinguishes the tissue into epithelial regions and stroma regions. The processor (1100) can generate a synthetic tissue image by setting the value of the first channel among the RGB channels to the maximum value for the stroma region of the tissue image and the value of the second channel to the maximum value for the epithelial region. The generated synthetic tissue image can be used as ground truth data to train the artificial intelligence model (210).

[0108] In one embodiment of the present disclosure, the processor (1100) may be designed to process various types of tissue images. By analyzing the resolution, color, or format of the input tissue image and performing a suitable preprocessing process, the learning and execution performance of the artificial intelligence model (210) can be optimized. Additionally, the processor (1100) may analyze the distribution of epithelium and stroma within the tissue image based on the results of the learned artificial intelligence model (210), and visually represent this to provide it to the user.

[0109] Storage (1200) can store data and programs of an electronic device (1000). Storage (1200) can be connected to memory (1400) to support the storage and retrieval of data. For example, storage (1200) may be a hard disk drive (HDD), a solid-state drive (SSD), flash memory, an optical disk drive, network storage, cloud storage, or magnetic tape. However, the present disclosure is not limited thereto.

[0110] Storage (1200) can store data for distinguishing between epithelium and stroma within a tissue image. It can store a tissue image containing RGB channels, annotation data separated into epithelial and stroma regions, and a synthetic tissue image with different RGB channel values ​​set for each of the epithelial and stroma regions. This data can provide data necessary for training an artificial intelligence model (210).

[0111] In one embodiment of the present disclosure, the storage (1200) can store a preprocessed tissue image and provide data in a form suitable for the learning and execution of an artificial intelligence learning module (200) and an artificial intelligence model (210). Additionally, the storage (1200) can store data received through a communication interface (1300) and provide it to other components of the electronic device (1000).

[0112] The communication interface (1300) can perform data transmission and reception with an external device from the electronic device (1000). Data can be transmitted and received via wired or wireless communication methods, and may include, for example, Bluetooth, Wi-Fi, Ethernet, cellular network, NFC, satellite communication, optical communication, etc. However, the present disclosure is not limited thereto.

[0113] The communication interface (1300) can receive data including tissue images from an external device or transmit it to an external device. The communication interface (1300) is connected to a processor (1100) to process signals generated during the data transmission and reception process, and is connected to a memory (1400) to store received data or transmit stored data.

[0114] In one embodiment of the present disclosure, the communication interface (1300) can transmit result data of the learned artificial intelligence model (210) to an external device and can perform a signal processing function to maintain the integrity of the data during the transmission process.

[0115] The memory (1400) can store and execute programs and data of the electronic device (1000). The memory (1400) may include an image synthesis module (100), an artificial intelligence learning module (200), and an image preprocessing module (300). For example, the memory (1400) may be RAM, ROM, flash memory, HDD, SSD, optical disk drive, or memory card. However, the present disclosure is not limited thereto.

[0116] The memory (1400) can store and execute an image synthesis module (100) that generates a synthetic tissue image, which is ground truth data for distinguishing the epithelium and stroma within the tissue image. The generated data can be provided to an artificial intelligence learning module (200) and used to train an artificial intelligence model (210). Additionally, the memory (1400) can store and execute an image preprocessing module (300) that preprocesses the input tissue image and converts it into a form suitable for the artificial intelligence learning module (200) and the artificial intelligence model (210).

[0117] The image synthesis module (100) can generate ground truth data to distinguish between the epithelium and the stroma within a tissue image. By setting different RGB channel values ​​for each of the epithelial region and the stroma region, a synthetic tissue image in which the epithelium and the stroma are clearly distinguished can be generated. The generated synthetic tissue image can be provided to an artificial intelligence learning module (200).

[0118] The artificial intelligence learning module (200) can train the artificial intelligence model (210). It can receive ground truth data generated from the image synthesis module (100) to generate training data, and perform training of the artificial intelligence model (210) based on this. The trained result can be saved or utilized in other modules.

[0119] The artificial intelligence model (210) can distinguish between epithelium and stroma within a tissue image. It can operate using a tissue image containing RGB channels as input data and based on annotation data separated into epithelial and stroma regions. The learned artificial intelligence model (210) can distinguish between the epithelium and stroma of the tissue image more accurately.

[0120] The image preprocessing module (300) can preprocess an input tissue image and convert it into a form suitable for the artificial intelligence learning module (200) and the artificial intelligence model (210). It can generate a synthetic tissue image by analyzing the RGB channel values ​​of the tissue image and processing annotation data to distinguish between epithelial and stromal regions. The generated synthetic tissue image can be provided as training data for the artificial intelligence learning module (200) and the artificial intelligence model (210). In one embodiment of the present disclosure, the image preprocessing module (300) can divide the entire slide image into a plurality of patch images. The image preprocessing module (300) can provide each of the plurality of patch images to the artificial intelligence model (210) or the image synthesis module (100).

[0121] FIG. 8 is a flowchart illustrating a method for training an artificial intelligence model that distinguishes between epithelium and stroma according to one embodiment of the present disclosure. The method for training an artificial intelligence model that distinguishes between epithelium and stroma according to one embodiment of the present disclosure includes steps S810 to S840, at least some of the steps may be omitted, and at least one other step may be added between the steps. Below, it is described that an electronic device (1000) performs the steps, but a processor (1100) of the electronic device (1000) may perform the steps.

[0122] Referring to FIG. 7 in conjunction with FIG. 8, at step S810, the electronic device (1000) can acquire a tissue image including RGB channels. The electronic device (1000) may use an optical device or a digital imaging device to acquire the tissue image. For example, the tissue image may be acquired through a microscope, a digital camera, or a scanner. However, the present disclosure is not limited thereto. The electronic device (1000) can acquire the tissue image at high resolution to clearly distinguish between the epithelial and stromal regions.

[0123] In one embodiment of the present disclosure, step S810 may include applying various optical filters or illumination conditions to acquire a tissue image. For example, a specific structure of the tissue may be emphasized using light of a specific wavelength, or an image containing depth information may be acquired through multi-focus imaging.

[0124] In step S820, the electronic device (1000) can acquire annotation data that separates a tissue image into epithelial regions and stromal regions. The electronic device (1000) may use an image processing algorithm or an artificial intelligence model to generate annotation data. For example, the electronic device (1000) may utilize a boundary detection algorithm, a color segmentation algorithm, or a deep learning-based segmentation network. However, the present disclosure is not limited thereto. The electronic device (1000) can generate annotation data. The electronic device (1000) may receive user input from an expert and modify the annotation data based on said user input.

[0125] In one embodiment of the present disclosure, step S820 may include applying a multi-scale analysis or a texture-based segmentation technique to generate annotation data. For example, the electronic device (1000) may analyze image features in different ways depending on the resolution or size of the tissue image to more precisely distinguish between epithelial and stromal regions.

[0126] In step S830, the electronic device (1000) can obtain a synthetic tissue image by setting the value of the first channel among the RGB channels to a maximum value for the stromal region of the tissue image and the value of the second channel among the RGB channels to a maximum value for the epithelial region of the tissue image. The electronic device (1000) can perform pixel-level color manipulation to generate the synthetic tissue image. For example, the R channel value of the stromal region and the G channel value of the epithelial region can be maximized to distinguish the two regions by color. However, the present disclosure is not limited thereto. The electronic device (1000) can clearly distinguish the epithelial and stromal regions while maintaining the appearance of the synthetic tissue image.

[0127] In one embodiment of the present disclosure, step S830 may include setting the value of a third channel among the RGB channels to a maximum value for at least one of the epithelial region and the stroma region. For example, the electronic device (1000) may generate a synthetic tissue image by maximizing the value of the R channel for the stroma region and maximizing the values ​​of the G channel and B channel for the epithelial region. Additionally, by maintaining the values ​​of the G channel and B channel for the stroma region and maintaining the value of the R channel for the epithelial region, the appearance of the original tissue image may be preserved.

[0128] In step S840, the electronic device (1000) can train an artificial intelligence model using tissue images as input data and synthetic tissue images as ground truth data. The electronic device (1000) can use a large number of tissue images and synthetic tissue images for training the artificial intelligence model. For example, the electronic device (1000) can train a model including an encoder and a decoder. The encoder can take a tissue image as input and output an embedding vector, and the decoder can take the embedding vector as input and output a synthetic tissue image. However, the present disclosure is not limited thereto. The electronic device (1000) can optimize training performance by applying various artificial intelligence model structures.

[0129] In one embodiment of the present disclosure, step S840 may include the step of applying a data augmentation technique for training an artificial intelligence model. For example, the electronic device (1000) may diversify the training data by rotating, scaling, or color-changing the tissue image. This may improve the generalization performance of the artificial intelligence model.

[0130] FIG. 9 is a flowchart illustrating a method for training an artificial intelligence model that distinguishes between epithelium and stroma according to one embodiment of the present disclosure. The method for training an artificial intelligence model that distinguishes between epithelium and stroma according to one embodiment of the present disclosure includes steps S910 to S940, at least some of the steps may be omitted, and at least one other step may be added between the steps. Below, it is described that an electronic device (1000) performs the steps, but a processor (1100) of the electronic device (1000) may perform the steps.

[0131] With reference to FIG. 1 and FIG. 9, in step S910, the electronic device (1000) may acquire a whole slide image of the entire tissue. The electronic device (1000) may include an optical device and a digital image sensor to generate a high-resolution image. For example, the whole slide image may be used to digitize a tissue sample for pathological analysis. However, the present disclosure is not limited thereto.

[0132] In one embodiment of the present disclosure, step S910 may include fixing a tissue sample to a slide and scanning it with an optical device to obtain a full slide image. For example, the optical device may be a microscope or a scanner. However, the present disclosure is not limited thereto.

[0133] In step S920, the electronic device (1000) can acquire whole annotation data corresponding to the entire slide image. The electronic device (1000) can use an image analysis algorithm to identify specific regions of the tissue and generate annotation data for those regions. For example, the annotation data may include the pathological condition of the tissue, cell density, or the expression level of specific markers. However, the present disclosure is not limited thereto.

[0134] In one embodiment of the present disclosure, step S920 may include the step of analyzing tissue images using a machine learning algorithm to obtain the entire annotation data. For example, the machine learning algorithm may be a deep learning model, a random forest, a support vector machine (SVM), or a k-nearest neighbor (k-NN). However, the present disclosure is not limited thereto.

[0135] In step S930, the electronic device (1000) may divide the entire tissue image into a plurality of patch images. The electronic device (1000) may divide the entire slide image into a grid of a fixed size or divide it based on a Region of Interest (ROI). For example, the patch images may be 256x256 pixels, 512x512 pixels, or 1024x1024 pixels in size. However, the present disclosure is not limited thereto.

[0136] In one embodiment of the present disclosure, step S930 may include the step of dividing each of the plurality of patch images so as to overlap. For example, the overlap ratio may be 10%, 20%, 30%, 40%, or 50%. However, the present disclosure is not limited thereto.

[0137] In step S940, the electronic device (1000) may acquire metadata for each of the plurality of patch images based on the entire annotation data. The electronic device (1000) may analyze the characteristics of the tissue included in each patch image and generate metadata for said characteristics. For example, the metadata may include location information of the patch image, labeling values ​​of the patch image (e.g., whether it is stroma or epithelium), pathological state of the tissue, cell density, expression levels of specific markers, or morphological features of the tissue. However, the present disclosure is not limited thereto. After step S940, the procedure may proceed to step S810 illustrated in FIG. 8.

[0138] FIG. 10 is a flowchart showing the detailed steps of step S830 of FIG. 8. Step S830 according to one embodiment of the present disclosure includes steps S1010 to S1040, at least some of the steps may be omitted, and at least one other step may be added between the steps. Below, it is described that an electronic device (1000) performs the steps, but a processor (1100) of the electronic device (1000) may perform the steps.

[0139] Referring to FIG. 10, in step S1010, the electronic device (1000) may add the value of the first channel to the epithelial region of the tissue image by the maximum value. The electronic device (1000) may analyze pixel data of the tissue image and annotation data to identify the epithelial region. The electronic device (1000) may extract the first channel value of the pixel corresponding to the epithelial region and perform an operation of adding the maximum value to the value. However, the present disclosure is not limited thereto.

[0140] In one embodiment of the present disclosure, step S1010 may include identifying an epithelial region of a tissue image based on annotation data and adjusting a first channel value based on pixel data of the epithelial region. For example, additional image processing algorithms may be applied to further clarify the boundaries of the epithelial region. Such algorithms may include various methods such as filtering, edge detection, or histogram analysis.

[0141] In step S1020, the electronic device (1000) may add the value of the second channel to the matrix region of the tissue image by the maximum value. The electronic device (1000) may analyze pixel data of the tissue image and annotation data to identify the matrix region. The electronic device (1000) may extract the second channel value of the pixel corresponding to the matrix region in the tissue image and perform an operation of adding the maximum value to the value. However, the present disclosure is not limited thereto.

[0142] In one embodiment of the present disclosure, step S1020 may include identifying a matrix region of a tissue image and adjusting a second channel value based on pixel data of the matrix region. For example, additional image processing algorithms may be applied to further clarify the characteristics of the matrix region. Such algorithms may include various methods such as color segmentation, texture analysis, or clustering.

[0143] In step S1030, the electronic device (1000) may determine whether at least one of the values ​​of the first channel and the second channel exceeds the maximum value. The electronic device (1000) may analyze the value of each pixel to compare the values ​​of the first channel and the second channel. For example, the electronic device (1000) may determine whether the first channel value and the second channel value of each pixel exceed the maximum value by comparing them. This comparison may be performed on a pixel-by-pixel basis, and whether the value exceeds the maximum value may be stored as a flag or status value. However, the present disclosure is not limited thereto.

[0144] In one embodiment of the present disclosure, step S1030 may include a step of comparing the values ​​of the first channel and the second channel and determining whether they exceed. For example, step S1030 may include an additional data processing step for setting a specific threshold value to determine whether they exceed, or for visually displaying the value that exceeds the threshold.

[0145] In step S1040, the electronic device (1000) may set a value exceeding the maximum value as the maximum value based on determining that at least one of the values ​​of the first channel and the second channel exceeds the maximum value. The electronic device (1000) may modify pixel data to adjust the excess value. For example, the excess value may be clipped to the maximum value, or a ratio adjustment may be performed on the excess value. Through such adjustment, distortion caused by data overflow can be prevented.

[0146] In one embodiment of the present disclosure, step S1040 may include the step of adjusting an excess value and storing or outputting the adjusted value. For example, the adjusted value may be stored as new image data or output to a display device so as to be visually checked.

[0147] A method according to one embodiment of the present disclosure may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0148] In the embodiments described above, components according to the technical concept of the present disclosure have been described using terms such as first, second, third, etc. However, terms such as first, second, third, etc. are used to distinguish components from one another and do not limit the technical concept of the present disclosure. Terms such as first, second, third, etc. do not imply an order or any numerical meaning.

[0149] The embodiments described above are specific embodiments for carrying out the present disclosure. It should be understood that the present disclosure includes not only the embodiments described above, but also embodiments that can be simply modified or easily modified using the embodiments described above. Accordingly, the scope of the present disclosure should not be limited to the embodiments described above, but should be defined by the claims set forth below, as well as equivalents to the claims of the present invention.

Claims

1. Regarding a method for training an artificial intelligence model that distinguishes between epithelium and stroma, A step of acquiring a tissue image including RGB channels; A step of acquiring annotation data that classifies the above tissue image into epithelial and stromal regions; A step of obtaining a synthetic tissue image by setting the value of the first channel among the RGB channels to a maximum value for the stroma region of the tissue image and setting the value of the second channel among the RGB channels to a maximum value for the epithelial region of the tissue image; A method comprising the step of training an artificial intelligence model using the tissue image as input data and the synthetic tissue image as ground truth data.

2. In Paragraph 1, The above artificial intelligence model includes an encoder and a decoder, and The above encoder takes the tissue image as input and outputs an embedding vector, and A method in which the above decoder takes the above embedding vector as input and outputs the above synthetic tissue image.

3. In Paragraph 1, The above method is, A step of acquiring a full slide image of the entire tissue; A step of obtaining all annotation data corresponding to the above-mentioned entire slide image; A step of dividing the entire slide image into a plurality of patch images; and Based on the entire annotation data above, the method includes the step of obtaining metadata for each of the plurality of patch images, and A method in which one of the plurality of patch images corresponds to a tissue image, and the metadata corresponds to the annotation data.

4. In Paragraph 1, A method in which the synthetic tissue image is an image in which the appearance of the tissue image is maintained and the epithelial region and the stroma region are distinguished by color.

5. In Paragraph 1, A method in which the values ​​of the RGB channels of the tissue image are not changed for the region excluding the epithelial region and the stroma region.

6. In Paragraph 1, The step of acquiring the above synthetic tissue image is: A method comprising the step of obtaining the synthetic tissue image by setting the value of the third channel among the RGB channels to a maximum value for at least one of the epithelial region and the stroma region.

7. In Paragraph 6, The above first channel is an R channel, and The above second channel is a G channel, and The above third channel is channel B, and The step of acquiring the above synthetic tissue image is: A step of setting the value of the R channel to a maximum value for the substrate region; and A method comprising the step of setting the values ​​of the G channel and the B channel to a maximum value for the epithelial region.

8. In Paragraph 7, For the above substrate region, the values ​​of the G channel and the B channel are maintained, and A method in which the value of the R channel is maintained for the above epithelial region.

9. In Paragraph 1, The step of acquiring the above synthetic tissue image is: A step of adding the value of the first channel to the epithelial region of the tissue image by the maximum value; A step of adding the value of the second channel to the substrate region of the tissue image by the maximum value; A step of determining whether at least one of the value of the first channel and the value of the second channel exceeds the maximum value; and A method comprising the step of setting a value exceeding the maximum value as the maximum value based on determining that at least one of the value of the first channel and the value of the second channel exceeds the maximum value.

10. In an electronic device, At least one processor including a processing circuit; and It includes at least one memory that stores at least one instruction, By the above at least one processor executing the above at least one instruction, the electronic device: Acquire a tissue image including RGB channels, and Acquire annotation data that classifies the above tissue image into epithelial and stromal regions, and A synthetic tissue image is obtained by setting the value of the first channel among the RGB channels to the maximum value for the stroma region of the tissue image and setting the value of the second channel among the RGB channels to the maximum value for the epithelial region of the tissue image. An electronic device that uses the above tissue image as input data and the above synthetic tissue image as ground truth data to train an artificial intelligence model.

Images