Artificial intelligence-based apparatus and method for examining and diagnosing thyroid frozen section

Abstract

The present disclosure relates to an artificial intelligence-based apparatus and method for examining and diagnosing a thyroid frozen section, and the apparatus includes: a memory; and a processor including an artificial intelligence engine for examining and diagnosing the thyroid frozen section, wherein the processor may obtain a first frozen section image of the thyroid, obtain a second frozen section image corresponding to the first frozen section image of the thyroid, and generate and provide a diagnosis result for a frozen section examination of the thyroid on the basis of the first frozen section image and the second frozen section image by using the artificial intelligence engine.

Description

Artificial intelligence-based thyroid frozen section examination and diagnosis device and method

[0001] The present disclosure relates to an artificial intelligence-based thyroid frozen section examination and diagnosis device and method.

[0002] Frozen section examination is a method of pathological examination performed during surgery to determine the direction of surgery or the extent of surgery.

[0003] Typically, pathological specimens are made into pathological slides for pathological diagnosis through the process of fixation, dehydration, embedding, sectioning, and staining. It usually takes one day for the fixation, dehydration, and embedding steps to be completed.

[0004] On the other hand, frozen section examination has the advantage of being able to be performed during surgery, as it takes about 30 minutes to produce a pathology slide after freezing, cutting, and staining without going through the fixation, dehydration, and embedding stages.

[0005] However, pathology slides produced in this way have many problems, such as the cells being clumped together and the staining pattern being irregular, which is disadvantageous in histomorphology compared to slides that have undergone conventional fixation, dehydration, and embedding, and thus there is a high risk of misdiagnosis.

[0006] Such misdiagnosis can have a decisive impact on, for example, the decision on the direction or scope of surgery, leading to irreversible results, and therefore requires countermeasures.

[0007] The present disclosure aims to provide a device and method for providing thyroid frozen section examination and diagnostic information based on artificial intelligence in conjunction with digital pathology.

[0008] An artificial intelligence-based frozen section examination diagnostic device according to at least one of various embodiments of the present disclosure comprises: a memory; and a processor including an artificial intelligence engine for the thyroid frozen section examination and diagnosis, wherein the processor can obtain a first frozen section image of the thyroid, obtain a second frozen section image corresponding to the first frozen section image of the thyroid, and generate and provide a diagnostic result for the frozen section examination of the thyroid based on the first frozen section image and the second frozen section image using the artificial intelligence engine.

[0009] The processor according to at least one of the various embodiments of the present disclosure may generate a second frozen section image corresponding to the first frozen section image of the thyroid gland using the artificial intelligence engine.

[0010] The processor according to at least one of the various embodiments of the present disclosure may generate the second frozen section image by converting the first frozen section image into a HE (Histogram Equalization) image.

[0011] The artificial intelligence engine according to at least one of the various embodiments of the present disclosure may include a plurality of models required for preprocessing for learning.

[0012] The first model according to at least one of the various embodiments of the present disclosure may be a preprocessing model for learning diagnostic content from a frozen section image.

[0013] The first model according to at least one of the various embodiments of the present disclosure may be a model that learns by labeling frozen section images and diagnostic content.

[0014] The second model according to at least one of the various embodiments of the present disclosure may be a preprocessing model for learning related to conversion from a frozen section image to the HE image.

[0015] The second model according to at least one of the various embodiments of the present disclosure may be a model that learns by labeling frozen section images and HE images.

[0016] The third model according to at least one of the various embodiments of the present disclosure may be a preprocessing model for learning diagnostic content from HE converted images.

[0017] The third model according to at least one of the various embodiments of the present disclosure may be a model that learns by labeling HE converted images and diagnostic content.

[0018] The processor according to at least one of the various embodiments of the present disclosure may determine at least two of the plurality of models as an ensemble.

[0019] The processor according to at least one of the various embodiments of the present disclosure can control changing the ensemble weights of each model.

[0020] The processor according to at least one of the various embodiments of the present disclosure can control the weighting to be changed according to the size of the tissue used in the frozen section image.

[0021] The processor according to at least one of the various embodiments of the present disclosure may set a high weight of a frozen section image when the size of the tissue is greater than or equal to a threshold, and may set a low weight of a HE converted image when the size of the tissue is less than or equal to the threshold.

[0022] An artificial intelligence-based thyroid frozen section examination diagnosis method, performed by a processor of an electronic device according to at least one of various embodiments of the present disclosure, may include the steps of: acquiring a first frozen section image of the thyroid gland; acquiring a second frozen section image corresponding to the first frozen section image of the thyroid gland; and generating and providing a diagnosis result for the frozen section examination of the thyroid gland based on the first frozen section image and the second frozen section image.

[0023] According to at least one of the various embodiments of the present disclosure, there is an advantage in that thyroid frozen section examination and diagnosis can be performed quickly and accurately through a device and method that provides thyroid frozen section examination and diagnosis information based on artificial intelligence in conjunction with digital pathology.

[0024] FIG. 1 is a schematic diagram of an artificial intelligence-based thyroid frozen section examination diagnostic system according to one embodiment of the present disclosure.

[0025] Figure 2 is a block diagram of the image acquisition device of Figure 1.

[0026] Figures 3 to 6 are block diagrams of the data processing unit or control unit of Figure 2.

[0027] FIG. 7 is a flowchart illustrating an artificial intelligence-based thyroid frozen section examination diagnosis method according to one embodiment of the present disclosure.

[0028] FIG. 8 is a drawing illustrating an example of a frozen section, a frozen section image, and an HE converted image of FIG. 7.

[0029] FIGS. 9 and 10 are flowcharts illustrating an artificial intelligence-based thyroid frozen section examination diagnostic method according to another embodiment of the present disclosure.

[0030] Throughout this disclosure, the same reference numerals denote the same components. This disclosure does not describe all elements of the embodiments, and any content that is common in the technical field to which this disclosure pertains or that overlaps between embodiments is omitted. The terms "part, module, element, block" used in the specification may be implemented in software or hardware, and depending on the embodiments, multiple "parts, modules, elements, blocks" may be implemented as a single component, or a single "part, module, element, block" may include multiple components.

[0031] Throughout the specification, when a part is said to be "connected" to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection via a wireless communication network.

[0032] Additionally, when a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0033] Throughout the specification, when we say that an element is "on" another element, this includes not only cases where the element is in contact with the other element, but also cases where another element exists between the two elements.

[0034] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0035] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0036] The identification codes for each step are used for convenience of explanation and do not describe the order of each step. Each step may be performed in a different order than specified unless the context clearly indicates a specific order.

[0037] The operating principle and embodiments of the present disclosure are described below with reference to the attached drawings.

[0038] As used herein, the term "device according to the present disclosure" encompasses a variety of devices capable of performing computational processing and providing results to a user. For example, the device according to the present disclosure may include a computer, a server device, and a portable terminal, or may be any one of them.

[0039] Here, the computer may include, for example, a notebook, desktop, laptop, tablet PC, slate PC, etc. equipped with a web browser.

[0040] The above server device is a server that processes information by communicating with an external device, and may include an application server, a computing server, a database server, a file server, a game server, a mail server, a proxy server, a web server, etc.

[0041] The above portable terminal may include, for example, a wireless communication device that ensures portability and mobility, and may include all kinds of handheld-based wireless communication devices such as a PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), WiBro (Wireless Broadband Internet) terminal, smart phone, etc., and wearable devices such as a watch, ring, bracelet, anklet, necklace, glasses, contact lens, or head-mounted device (HMD).

[0042] The artificial intelligence-related functions according to the present disclosure are operated via a processor and memory. The processor may be comprised of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a DSP (Digital Signal Processor), a graphics-only processor such as a GPU or a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU. One or more processors control the processing of input data according to predefined operating rules or artificial intelligence models stored in memory. Alternatively, if one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0043] The predefined operation rules or artificial intelligence models are characterized by being created through learning. Here, being created through learning means that the basic artificial intelligence model is learned by a learning algorithm using a plurality of learning data, thereby creating a predefined operation rules or artificial intelligence model set to perform a desired characteristic (or purpose). This learning may be performed on the device itself on which the artificial intelligence according to the present disclosure is performed, or may be performed through a separate server and / or system. Examples of the learning algorithm include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0044] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values, and performs neural network operations through operations between the operation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), and examples thereof include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or deep Q-networks.

[0045] According to an exemplary embodiment of the present disclosure, a processor can implement artificial intelligence. Artificial intelligence refers to a machine learning method based on an artificial neural network that imitates human neurons (biological neurons) to enable machines to learn. Artificial intelligence methodologies can be categorized into supervised learning, in which input data and output data are provided together as training data depending on the learning method, so that the solution (output data) to the problem (input data) is determined; unsupervised learning, in which only input data is provided without output data, so that the solution (output data) to the problem (input data) is not determined; and reinforcement learning, in which a reward (Reward) is provided from an external environment whenever an action (Action) is taken in the current state (State), and learning is performed in a direction to maximize this reward. In addition, artificial intelligence methodologies can be categorized according to the architecture of the learning model. The architectures of widely used deep learning technologies can be categorized into convolutional neural networks (CNNs), recurrent neural networks (RNNs), transformers, and generative adversarial networks (GANs).

[0046] The present device and system may include an artificial intelligence model. The artificial intelligence model may be a single artificial intelligence model or may be implemented as multiple artificial intelligence models. The artificial intelligence model may be composed of a neural network (or artificial neural network) and may include statistical learning algorithms that mimic biological neurons in machine learning and cognitive science. A neural network may refer to a model in general that has problem-solving capabilities by changing the binding strength of synapses through learning, formed by artificial neurons (nodes) that form a network by combining synapses. The neurons of the neural network may include a combination of weights or biases. The neural network may include one or more layers composed of one or more neurons or nodes. For example, the device may include an input layer, a hidden layer, and an output layer. The neural network constituting the device can infer a desired result (output) from an arbitrary input (input) by changing the weights of neurons through learning.

[0047] The processor can create a neural network, train (or learn) a neural network, perform a calculation based on received input data, generate an information signal based on the calculation result, or retrain the neural network. The models of the neural network can include various types of models such as CNN (Convolution Neural Network) such as GoogleNet, AlexNet, VGG Network, R-CNN (Region with Convolution Neural Network), RPN (Region Proposal Network), RNN (Recurrent Neural Network), S-DNN (Stacking-based deep Neural Network), S-SDNN (State-Space Dynamic Neural Network), Deconvolution Network, DBN (Deep Belief Network), RBM (Restrcted Boltzman Machine), Fully Convolutional Network, LSTM (Long Short-Term Memory) Network, Classification Network, etc., but are not limited thereto. The processor can include one or more processors for performing calculations according to the models of the neural network. For example, the neural network can be a deep neural network. It may include a deep neural network.

[0048] Neural networks include CNN (Convolutional Neural Network), RNN (Recurrent Neural Network), perceptron, multilayer perceptron, FF (Feed Forward), RBF (Radial Basis Network), DFF (Deep Feed Forward), LSTM (Long Short Term Memory), GRU (Gated Recurrent Unit), AE (Auto Encoder), VAE (Variational Auto) Encoder), DAE (Denoising Auto Encoder), SAE (Sparse Auto Encoder), MC (Markov Chain), HN (Hopfield Network), BM (Boltzmann Machine), RBM (Restricted Boltzmann Machine), DBN (Depp Belief Network), DCN (Deep Convolutional Network), DN (Deconvolutional Network), DCIGN (Deep Convolutional Inverse Graphics Network), Generative Adversarial Network (GAN), Liquid State Machine (LSM), Extreme Learning Machine (ELM), It will be understood by those skilled in the art that any neural network may be included, including but not limited to ESN (Echo State Network), DRN (Deep Residual Network), DNC (Differentiable Neural Computer), NTM (Neural Turning Machine), CN (Capsule Network), KN (Kohonen Network), and AN (Attention Network).

[0049] According to an exemplary embodiment of the present disclosure, the processor may be configured to perform a process for generating a CNN (Convolution Neural Network) such as GoogleNet, AlexNet, VGG Network, Region with Convolution Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restrcted Boltzman Machine (RBM), Fully Convolutional Network, Long Short-Term Memory (LSTM) Network, Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT, SP-BERT, MRC / QA for natural language processing, Text Analysis, Dialog System, GPT-3, GPT-4, Visual Analytics for vision processing, Visual Understanding, Video Synthesis, ResNet for data intelligence, Anomaly Detection, Prediction, Time-Series Forecasting, Various artificial intelligence structures and algorithms, including optimization, recommendation, and data creation, can be utilized, but are not limited thereto. Hereinafter, embodiments of the present disclosure will be described in detail with reference to the attached drawings.

[0050] As mentioned above, frozen section examination represents a method of pathological examination performed, for example, to determine the direction of surgery or the extent of surgery during surgery.

[0051] In this regard, the present specification discloses a digital pathology-linked artificial intelligence-based thyroid frozen section examination and diagnosis system, device and method according to at least one of the various embodiments of the present disclosure.

[0052] In the present disclosure, for example, a thyroid frozen section examination reading algorithm, a virtual HE image conversion algorithm for a conventional fixation process (fixation, dehydration, embedding, thin section staining) of a thyroid frozen section image, a thyroid frozen section virtual HE conversion image reading algorithm, a thyroid frozen section examination reading and HE conversion image integration diagnosis algorithm, etc. are disclosed for frozen section examination and diagnosis based on artificial intelligence.

[0053] Accordingly, according to the present disclosure, a user, for example, a medical institution, can be provided with HE converted images of thyroid frozen section images, thyroid frozen section image diagnosis, thyroid frozen section HE converted image diagnosis, final comprehensive diagnosis result information, etc.

[0054] The present disclosure can compare first and second frozen section images. Furthermore, the present disclosure can generate and provide AI-based diagnostic information for each image. Subsequently, a final diagnosis can be generated and provided based on the AI-based analysis results of the two images. Thus, the present disclosure offers the advantage of increasing the accuracy of diagnostic results and facilitating rapid and easy frozen section examination and diagnosis.

[0055] In the above, the first frozen section image is an original (or raw) frozen section image.

[0056] In the above, the second frozen section image may represent an image generated by converting the first frozen section image based on artificial intelligence. The second frozen section image may include, for example, a Histogram Equalization (HE) image. However, the present disclosure is not limited thereto.

[0057] With reference to the attached drawings, various embodiments of the present disclosure are described as follows.

[0058] FIG. 1 is a schematic diagram of an artificial intelligence-based thyroid frozen section examination diagnostic system (1) according to one embodiment of the present disclosure.

[0059] Figure 2 is a block diagram of the image acquisition device (10) of Figure 1.

[0060] Figures 3 to 6 are block diagrams of the data processing unit (220) or control unit (230) of Figure 2.

[0061] An artificial intelligence-based thyroid frozen section examination diagnostic system (1) according to at least one of the various embodiments of the present disclosure can provide a diagnostic result of a frozen section examination based on an image of a frozen section of a thyroid gland (hereinafter referred to as a "first frozen section image") and a second frozen section image converted to correspond to the first frozen section image. In this case, the second frozen section image may include an image converted using the HE method. However, the present disclosure is not limited thereto.

[0062] Referring to FIG. 1, an artificial intelligence-based thyroid frozen section examination diagnostic system (1) according to at least one of the various embodiments of the present disclosure may be configured to include an image acquisition device (10) and a terminal (30).

[0063] At this time, the artificial intelligence-based thyroid frozen section examination diagnostic system (1) according to at least one of the various embodiments of the present disclosure may further include a server (20). However, the server (20) may not be an essential component.

[0064] The server (20) can duplicate or replace all or part of the functions or components of the image acquisition device (10) described below.

[0065] These servers (20) may be provided in the form of a cloud, and may be implemented to include various data processing modules for signal processing while being located remotely.

[0066] The image acquisition device (10) can obtain a frozen section image, i.e., a first frozen section image, as in (b) of FIG. 8, after a tissue, for example, as in (a) of FIG. 8, collected from a user (i.e., a target patient) for frozen section examination, is frozen and created as a frozen section slide (or pathology slide), through digital pathology.

[0067] The image acquisition device (10) can acquire a second frozen section image from the first frozen section image acquired after the digital pathology process for the target patient.

[0068] The image acquisition device (10) can generate frozen section examination results or diagnostic results for the biological tissue of the target patient based on the first frozen section image and the second frozen section image and provide the results to the terminal (30) via the terminal (30) or server (20).

[0069] All or part of the functions of the image acquisition device (10) described above may be handled by the server (20).

[0070] In this case, when a first frozen section image is uploaded from the image acquisition device (10), the server (20) can generate a second frozen section image based on the uploaded first frozen section image.

[0071] The server (20) can generate frozen section examination results or diagnostic results for the biological tissue of the target patient based on the first frozen section image and the second frozen section image and provide the results to the terminal (30) via the terminal (30) or / and the image acquisition device (10).

[0072] The terminal (30) can request a frozen section examination of the thyroid tissue of the target patient from the image acquisition device (10) or / and the server (20), obtain the results of the frozen section examination as described above, and provide related information through an output device (not shown) such as a display or speaker.

[0073] Meanwhile, depending on the embodiment, the terminal (30) may not be an essential component. For example, the terminal (30) may be replaced by the image acquisition device (10). In this case, it is preferable that the image acquisition device (10) be equipped with the aforementioned output device to provide relevant information.

[0074] In the above, the output device may include a display, a speaker (at this time, the speaker does not necessarily need to be built into the terminal (30), and a speaker that can output data in conjunction with the terminal (30) is sufficient).

[0075] The connection between each component illustrated in Fig. 1 may refer to a wired / wireless communication method. For convenience of explanation, the present disclosure uses a wireless communication method such as Bluetooth, BLE (Bluetooth™ Low Energy), Zigbee, or Wi-Fi as an example.

[0076] In FIG. 2, components of an image acquisition device (10) according to at least one of the various embodiments of the present disclosure are illustrated.

[0077] At this time, the image acquisition device (10) can be viewed as, for example, an artificial intelligence-based thyroid frozen section examination diagnostic device.

[0078] An artificial intelligence-based thyroid frozen section examination diagnostic device (10) may be configured to include a memory (240) and a processor.

[0079] The processor may include an artificial intelligence engine (AI engine) for the above thyroid frozen section examination and diagnosis.

[0080] The processor may include a communication module (210), a data processing unit (220), a control unit (230), etc., to obtain a first frozen section image of the thyroid gland, obtain a second frozen section image corresponding to the first frozen section image of the thyroid gland, and generate and provide a diagnostic result for a frozen section examination of the thyroid gland based on the first frozen section image and the second frozen section image using the artificial intelligence engine.

[0081] The communication module (110) can provide a wireless communication interface environment to support a communication environment so that the artificial intelligence-based thyroid frozen section examination diagnostic device (10) can exchange data with the server (20) and / or terminal (30). In this case, the communication module (110) can be an LTE, LTE-A, 5G, or Wi-Fi communication module.

[0082] The data processing unit (220) can process data acquired through the communication module (210) under the control of the control unit (240).

[0083] The control unit (230) can control the operation of each component of the artificial intelligence-based thyroid frozen section examination diagnostic device (10). The specific operation of the control unit (230) will be described later.

[0084] The memory (240) can store various information in advance or update stored information.

[0085] The memory (240) can temporarily store data acquired from an artificial intelligence-based thyroid frozen section examination diagnostic device (10) or data acquired from a terminal (30) or server (20).

[0086] Memory (240) may store information regarding software, such as applications and programs, or related processing engines, as needed for frozen section examination and diagnosis according to the present disclosure. The processing engine may include, for example, an artificial intelligence learning engine. Such an engine may be implemented within the data processing unit (220) and / or the control unit (230).

[0087] In FIG. 2, the memory (240) may include at least one type of storage medium among a flash memory type, a hard disk type, an SSD (Solid State Disk type), an SDD (Silicon Disk Drive type), a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk.

[0088] In Fig. 2, only one memory (240) is illustrated for convenience, but it is not limited thereto and there may be multiple memory (240).

[0089] In FIG. 2, the memory (240) is illustrated and described as being built into the artificial intelligence-based thyroid frozen section examination diagnostic device (10) for convenience, but is not limited thereto. For example, in FIG. 2, the memory (240) may also be implemented in the form of a DB (Database) server located externally or remotely.

[0090] In FIG. 2, the memory and processor may be implemented as separate chips. Alternatively, the memory and processor may be implemented as a single chip.

[0091] At least one component may be added or deleted to correspond to the performance of the components illustrated in Figure 2. Furthermore, it will be readily apparent to those skilled in the art that the relative positions of the components may be altered to correspond to the performance or structure of the system.

[0092] Meanwhile, each component illustrated in FIG. 2 refers to software and / or hardware components such as Field Programmable Gate Array (FPGA) and Application Specific Integrated Circuit (ASIC).

[0093] The following describes the operation of a processor according to another embodiment of the present disclosure.

[0094] First, referring to FIG. 3, the data processing unit (220) or control unit (230) of the processor may be configured to include, for example, a data preprocessing unit (310), an artificial intelligence learning engine (320), an image comparison unit (330), a diagnostic information generation unit (340), etc.

[0095] Although some of the components illustrated in FIG. 3 may be included in the data processing unit (220) and the rest may be included in the control unit (230).

[0096] The data preprocessing unit (310) can preprocess input data to generate or extract data that can be used in the artificial intelligence learning engine (320).

[0097] The artificial intelligence learning engine (320) can learn a model using data (e.g., a learning dataset) preprocessed in the data preprocessing unit (310).

[0098] The artificial intelligence learning engine (320) may also update the learned model based on data fed back from the server (20) and / or terminal (30). The model updated by the artificial intelligence learning engine (320) may be transmitted to the memory (240) and updated.

[0099] The image comparison unit (330) can compare and analyze input images.

[0100] The image comparison unit (330) can compare, for example, the first frozen section image and the second frozen section image that are input.

[0101] This image comparison unit (330) may be included as a component of, for example, an artificial intelligence learning engine (320) or a diagnostic information generation unit (340) described later.

[0102] The diagnostic information generation unit (340) can generate diagnostic information for a requested frozen section examination of a target patient's thyroid tissue based on the comparison results of the image comparison unit (330) using a model learned by the artificial intelligence learning engine (320). The diagnostic information thus generated can be output to the terminal (30).

[0103] Next, referring to FIG. 4, the data processing unit (220) or / and the control unit (230) may be configured to include a data preprocessing unit (405) and k diagnostic models (where k is a natural number).

[0104] A diagnostic model may represent a learning model for generating diagnostic information.

[0105] The data preprocessing unit (405) can preprocess and classify data to be learned from each diagnostic model. The classified data can then be input into the corresponding diagnostic model and used for learning.

[0106] For example, assuming k is 3, the first diagnostic model may be a frozen section-based diagnostic learning model by labeling the frozen section image and the diagnostic content.

[0107] The second diagnostic model may be a diagnostic learning model that learns by labeling thyroid frozen section images and HE converted images in order to obtain thyroid frozen section HE converted images from thyroid frozen section images.

[0108] And the third diagnostic model may be a frozen section-based diagnostic learning model that labels the thyroid frozen section HE converted image and the diagnostic content.

[0109] The processor can determine at least two diagnostic models as an ensemble depending on k. For example, the processor can determine the first diagnostic model and the third diagnostic model as an ensemble.

[0110] Even if the processor determines at least two diagnostic models as an ensemble, it can individually learn and utilize each diagnostic model determined as an ensemble.

[0111] Next, referring to FIG. 5, the data processing unit (220) or / and the control unit (230) may be configured to include a frozen section classification model (510) and m diagnostic models (where m is a natural number).

[0112] In the frozen section classification model (510), the diagnostic content used for learning may differ depending on the organ.

[0113] The frozen section classification model (510) can be classified to learn diagnostic information such as cancer or benign in relation to the thyroid gland, but is not limited thereto.

[0114] Therefore, the two (e.g., for thyroid, m is 2) diagnostic models may include a cancer diagnostic model (520) and a benign diagnostic model (530).

[0115] Each diagnostic model can receive and learn preprocessed and classified data from the frozen section classification model (510).

[0116] Referring to FIG. 6, the data processing unit (220) or / and the control unit (230) may be configured to include a frozen section classification model (610) and r diagnostic models (where r is a natural number).

[0117] In the frozen section classification model (610), the diagnostic content used for learning may differ depending on the organ.

[0118] The frozen section classification model (610) can be classified so that each diagnostic model can learn according to the size of the thyroid biopsy tissue.

[0119] For convenience of explanation, assuming that r is 3, the diagnostic model may include a first size diagnostic model (620), a second size diagnostic model (630), and a third size diagnostic model (640).

[0120] Each diagnostic model can receive and learn preprocessed and classified data from the frozen section classification model (610).

[0121] The preprocessing and classification of the frozen section data in FIGS. 4 to 6 described above may be for the purpose of controlling the weights of the determined or to-be-determined ensemble, but is not limited thereto.

[0122] In connection with the present disclosure, the ensemble weights of each diagnostic model may be changed depending on the type of body organ. For example, if the body organ is the thyroid, a higher weight may be assigned to the third model in Figure 4 compared to the other models.

[0123] In connection with the present disclosure, weights may be controlled and changed depending on the size of the thyroid biopsy tissue. Although FIG. 6 exemplifies three sizes, when dividing into two sizes, if the size of the thyroid biopsy tissue for frozen section examination is greater than a threshold, a higher weight may be set to the thyroid nodule, i.e., the first frozen section image, and otherwise, i.e., if the size of the thyroid biopsy tissue is less than the threshold, a lower weight may be set to the HE converted image, i.e., the second frozen section image.

[0124] As described above, the artificial intelligence engine according to at least one of the various embodiments of the present disclosure may include a plurality of models required for preprocessing for learning.

[0125] At this time, the first model may be a preprocessing model for learning diagnostic information from frozen section images. The first model may be a model that learns by labeling frozen section images and diagnostic information.

[0126] The second model may be a preprocessing model for learning related to converting frozen section images into HE images. The second model may be a model that learns by labeling frozen section images and HE images.

[0127] Additionally, the third model may be a preprocessing model for learning diagnostic content from HE converted images. The third model may be a model that learns by labeling HE converted images and diagnostic content.

[0128] The processor according to at least one of the various embodiments of the present disclosure may determine at least two of the plurality of models as an ensemble.

[0129] At this time, the processor can control the change of the ensemble weights of each model.

[0130] FIGS. 7, 9 and 10 are flowcharts illustrating an artificial intelligence-based thyroid frozen section examination diagnostic method according to the present disclosure.

[0131] FIG. 8 is a drawing illustrating an example of a frozen section, a frozen section image, and an HE converted image of FIG. 7.

[0132] The method of operation of an artificial intelligence-based thyroid frozen section examination diagnostic device (10) linked to digital pathology according to at least one of the various embodiments of the present disclosure may be as follows.

[0133] In operation S110, thyroid tissue of the target patient can be collected for frozen section examination as shown in (a) of FIG. 8.

[0134] In S120 motion, thyroid biopsy tissue collected for frozen section examination in S110 motion may be frozen.

[0135] In operation S130, the frozen thyroid biopsy tissue of the subject patient in operation S120 can be transferred to a pathology slide and digital pathology operation can be performed.

[0136] In operation S140, the processor can obtain a first frozen section image as shown in (b) of FIG. 8 based on the digital pathology results performed in operation S130.

[0137] In operation S150, the processor can HE-convert the first frozen section image acquired in operation S140 based on artificial intelligence to acquire a second frozen section image as shown in (c) of FIG. 8.

[0138] In operation S160, the processor can generate (or obtain) each image-based primary diagnostic information.

[0139] In operation S160, the processor can obtain primary diagnostic information generated based on the first frozen section image and primary diagnostic information generated based on the second frozen section image, respectively.

[0140] In operation S170, the processor can merge the primary diagnostic information acquired based on each frozen section image in operation S160 to generate secondary diagnostic information and provide it to a terminal (30), etc.

[0141] To summarize FIG. 7, a digital pathology-linked artificial intelligence-based thyroid frozen section examination diagnostic method according to at least one of various embodiments of the present disclosure acquires a first frozen section image of the thyroid gland, acquires a second frozen section image corresponding to the first frozen section image of the thyroid gland, and generates and provides a diagnostic result for the frozen section examination of the thyroid gland based on the first frozen section image and the second frozen section image.

[0142] Next, referring to FIG. 9, in operation S210, the processor can identify a body organ to be examined by frozen section.

[0143] In operation S220, the processor can determine ensemble weights of each diagnostic model based on the identified frozen section examination target body organ.

[0144] In operation S230, the processor can determine whether a change in the weights of the ensemble is required.

[0145] If the processor determines that the ensemble weights do not need to be changed based on the S230 operation judgment result, the processor can ignore or discard the previous decision.

[0146] In operation S240, if the processor determines that a change in the ensemble weight is necessary as a result of the judgment in operation S230, it can control the change in the ensemble weight of each model.

[0147] Finally, referring to FIG. 10, in operation S310, the processor can identify the size of the tissue to be inspected for the slit.

[0148] In operation S320, the processor can determine whether the size of the identified tissue fragment to be inspected is greater than a threshold.

[0149] If the processor determines that the size of the tissue to be examined for the identified homogeneous section is greater than a threshold value as a result of the S320 operation judgment, a higher weight can be assigned to the homogeneous section (S330). At this time, the homogeneous section can represent a first homogeneous section image.

[0150] If the processor determines that the size of the tissue to be examined for the identified homogeneous section is less than the threshold as a result of the S320 operation judgment, it can assign a lower weight to the HE converted image (i.e., the second homogeneous section image) rather than the homogeneous section (S340).

[0151] When an event occurs, such as changing the weight of the ensemble among the aforementioned contents or changing the weight according to the size of the thyroid tissue, the processor can notify the server (20) or / and the terminal (30) of the fact.

[0152] After the processor provides the diagnosis results of a thyroid biopsy of a target patient to the server (20) or / and the terminal (30), if positive feedback or negative feedback is received from the terminal (30), the processor can update the aforementioned diagnostic learning model with reference to the received feedback.

[0153] For example, if the negative feedback from the medical institution via the aforementioned terminal (30) exceeds a predetermined number of times or is continuous, the processor may reset all or at least some of the pre-learned diagnostic models and re-perform the learning process. In this case, the preprocessing model may also be reset after initialization.

[0154] The processor can derive and pre-store correlations between result values ​​based on ensemble combinations or ensemble weights in memory (240). These stored correlations can be reflected in future updates. For example, when one model is updated, other correlated models can also be updated. In this case, models updated based on correlations may be updated at a lower rate than the target model.

[0155] The processor can store the target patient's gender, age group, situation and prediction information before frozen section, situation and diagnosed information after frozen section, etc. in the memory (240).

[0156] The processor can arbitrarily adjust the weights of the model or / and ensemble for a new target patient based on information stored in the memory (240).

[0157] The present disclosure can also provide information related to digital pathology-linked artificial intelligence-based thyroid frozen section examination and diagnosis in a combined form of two or more of the above-described embodiments.

[0158] Meanwhile, the disclosed embodiments may be implemented in the form of a recording medium storing computer-executable instructions. The instructions may be stored in the form of program code, and when executed by a processor, may generate program modules to perform the operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium.

[0159] Computer-readable storage media include all types of storage media that store instructions that can be deciphered by a computer. Examples include read-only memory (ROM), random access memory (RAM), magnetic tape, magnetic disks, flash memory, and optical data storage devices.

[0160] The disclosed embodiments have been described with reference to the attached drawings as described above. Those skilled in the art will understand that the present disclosure can be implemented in forms other than the disclosed embodiments without altering the technical spirit or essential features of the present disclosure. The disclosed embodiments are illustrative and should not be construed as limiting.

Claims

1. In an AI-based thyroid frozen section examination diagnostic device, memory; and A processor including an artificial intelligence engine for the above thyroid frozen section examination and diagnosis, The above processor, Obtaining a first frozen section image of the thyroid gland, obtaining a second frozen section image corresponding to the first frozen section image of the thyroid gland, and generating and providing a diagnostic result for a frozen section examination of the thyroid gland based on the first frozen section image and the second frozen section image using the artificial intelligence engine. An artificial intelligence-based thyroid frozen section examination diagnostic device.

2. In claim 1, The above processor, Using the artificial intelligence engine, a second frozen section image corresponding to the first frozen section image of the thyroid gland is generated. An artificial intelligence-based thyroid frozen section examination diagnostic device.

3. In claim 2, The above processor, Converting the first frozen section image into a HE (Histogram Equalization) image to generate the second frozen section image. An artificial intelligence-based thyroid frozen section examination diagnostic device.

4. In claim 3, The above artificial intelligence engine, Contains multiple models required for preprocessing for learning. An artificial intelligence-based thyroid frozen section examination diagnostic device.

5. In claim 4, The above first model is a preprocessing model for learning diagnostic content from frozen section images. An artificial intelligence-based thyroid frozen section examination diagnostic device.

6. In claim 5, The above first model is a model that learns by labeling frozen section images and diagnostic content. An artificial intelligence-based thyroid frozen section examination diagnostic device.

7. In claim 6, The above second model is a preprocessing model for learning related to conversion from a frozen section image to the HE image. An artificial intelligence-based thyroid frozen section examination diagnostic device.

8. In claim 7, The above second model is a model that learns by labeling frozen section images and HE images. An artificial intelligence-based thyroid frozen section examination diagnostic device.

9. In claim 8, The above third model is a preprocessing model for learning diagnostic content from HE converted images. An artificial intelligence-based thyroid frozen section examination diagnostic device.

10. In claim 9, The above third model is a model that learns by labeling HE converted images and diagnostic content. An artificial intelligence-based thyroid frozen section examination diagnostic device.

11. In claim 10, The above processor, At least two of the above multiple models are determined as an ensemble, An artificial intelligence-based thyroid frozen section examination diagnostic device.

12. In claim 11, The above processor, Controlling the ensemble weights of each of the above models, An artificial intelligence-based thyroid frozen section examination diagnostic device.

13. In claim 12, The above processor, Controlling the weighting according to the size of the tissue used in the above frozen section image. An artificial intelligence-based thyroid frozen section examination diagnostic device.

14. In claim 13, The above processor, If the size of the above organization is greater than the threshold, the weight of the frozen section image is set high, If the size of the above organization is less than a threshold, the weight of the HE converted image is set low. An artificial intelligence-based thyroid frozen section examination diagnostic device.

15. In a method for diagnosing thyroid frozen section examination based on artificial intelligence, performed by a processor of an electronic device, A step of obtaining a first frozen section image of the thyroid gland; A step of obtaining a second frozen section image corresponding to the first frozen section image of the thyroid gland; and Comprising a step of generating and providing a diagnostic result for a frozen section examination of the thyroid gland based on the first frozen section image and the second frozen section image. An artificial intelligence-based thyroid frozen section examination diagnostic method.

Images