Estimation device, estimation method, learning model, learning model generation method, and computer program

Abstract

To provide an estimation device, a learning model, a method for generating a learning model, and a computer program.SOLUTION: An estimation device includes: an acquisition part for acquiring an oral cavity image that can be obtained by imaging an oral cavity of a subject; an estimation part for estimating whether there is any lesion in an oral mucous membrane of the subject from the oral cavity image acquired by the acquisition part using a learning model that is configured to output information relating to a lesion in an oral mucous membrane with respect to input of the oral cavity image; and an output part for outputting an estimated result by the estimation part.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an estimation device, a learning model, a method for generating a learning model, and a computer program. [Background technology]

[0002] There are various types of oral mucosal diseases, such as oral malignant tumors and stomatitis, but diagnosis is often difficult. In particular, early oral malignant tumors often present clinical symptoms similar to stomatitis, and are often overlooked by medical professionals who are not specialists. Many facilities are currently conducting research into causative genes and prognostic factors for oral malignant tumors, and the inventors have also reported on pathological prognostic predictors of oral malignant tumors (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Al-Shareef H, Hiraoka S, et al. ONCOLOGY REPORTS:2016 Summary of the Invention [Problem to be solved by the invention]

[0004] However, research in the oral cavity has lagged behind other fields, and there is no prospect of establishing a simple diagnostic support system that can be put into practical use.

[0005] The present invention aims to provide an estimation device, a learning model, a method for generating a learning model, and a computer program capable of estimating lesions in the oral mucosa. [Means for solving the problem]

[0006] An estimation device according to one embodiment of the present invention includes an acquisition unit that acquires oral images obtained by imaging the inside of the oral cavity of a subject, an estimation unit that estimates the presence or absence of a lesion in the oral mucosa of the subject from the oral image acquired by the acquisition unit using a learning model configured to output information regarding lesions in the oral mucosa in response to an input of the oral image, and an output unit that outputs the estimation result of the estimation unit.

[0007] A learning model according to one embodiment of the present invention comprises an input layer to which oral cavity images obtained by imaging the inside of a subject's oral cavity are input, an output layer to output information relating to lesions in the oral mucosa, and an intermediate layer that uses oral cavity images and annotations for the oral cavity images as training data to learn the relationship between the oral cavity images input to the input layer and the information output by the output layer, and when an oral cavity image is input to the input layer, causes a computer to perform calculations in the intermediate layer and output information relating to lesions in the oral mucosa from the output layer.

[0008] A method for generating a learning model according to one embodiment of the present invention uses a computer to acquire training data including oral cavity images obtained by imaging the inside of a subject's oral cavity and annotations for the oral cavity images, and generates a learning model that outputs information regarding lesions in the oral mucosa in response to input of oral cavity images based on the acquired training data.

[0009] A computer program according to one embodiment of the present invention is a computer program for causing a computer to execute a process of acquiring an oral cavity image obtained by imaging the inside of a subject's oral cavity, estimating the presence or absence of a lesion in the oral mucosa from the acquired oral cavity image using a learning model configured to output information regarding a lesion in the oral mucosa in response to an input of the oral cavity image, and outputting the estimation result. Effect of the Invention

[0010] According to the present application, lesions in the oral mucosa can be estimated. [Brief description of the drawings]

[0011] [Figure 1]FIG. 1 is a block diagram illustrating a configuration of an estimation device according to a first embodiment. [Diagram 2] FIG. 2 is a schematic diagram showing an example of an oral cavity image. [Diagram 3] FIG. 2 is a schematic diagram showing an example of the configuration of a learning model. [Figure 4] 4 is a flowchart illustrating a procedure of a process executed by the estimation device according to the first embodiment. [Diagram 5] FIG. 4 is a schematic diagram showing an example of an output from the estimation device. [Figure 6] 10 is a flowchart illustrating a procedure of a process executed by an estimation device 1 according to a second embodiment. [Figure 7] FIG. 11 is a block diagram illustrating a configuration of an estimation device according to a third embodiment. [Figure 8] FIG. 13 is a schematic diagram showing an extraction example. [Figure 9] 11 is a flowchart illustrating a procedure of a process executed by an estimation device according to a third embodiment. [Figure 10] FIG. 2 is a block diagram illustrating a configuration of a server device. [Figure 11] FIG. 1 is a conceptual diagram illustrating an example of an oral cavity image database. [Figure 12] 11 is a flowchart illustrating a procedure for generating a learning model. [Figure 13] FIG. 13 is a block diagram illustrating a configuration of an estimation device according to a fifth embodiment. [Figure 14] FIG. 13 is a schematic diagram showing an example of an enlarged image. [Figure 15] FIG. 2 is a schematic diagram showing an example of the configuration of a learning model. [Figure 16] FIG. 13 is a conceptual diagram showing an example of an oral cavity image database according to the fifth embodiment. [Figure 17] 11 is a flowchart illustrating a procedure for generating a learning model. [Figure 18] 13 is a flowchart showing an estimation procedure using a learning model. [Figure 19] FIG. 13 is a schematic diagram showing an example of presentation of an observation site. [Figure 20]FIG. 13 is a schematic diagram showing an example of an interface screen for accepting designation of a target portion. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the present invention will be described in detail with reference to the drawings showing embodiments thereof. (Embodiment 1) 1 is a block diagram for explaining the configuration of an estimation device 1 according to the first embodiment. The estimation device 1 is a computer device installed in a facility such as a hospital, and estimates the presence or absence of a lesion in the oral mucosa of a subject from an oral cavity image obtained by capturing an image of the oral cavity of the subject. The estimation device 1 provides diagnostic support by presenting the estimation result to a diagnosing person such as a doctor.

[0013] The estimation device 1 includes an input unit 11, a control unit 12, a storage unit 13, an output unit 14, a communication unit 15, and an operation unit 16.

[0014] The input unit 11 includes an input interface to which image data of an oral cavity image is input. The input interface is, for example, an interface for connecting an imaging device for imaging the oral cavity of a subject. The imaging device is a digital camera or a digital video camera, and outputs image data in which each pixel is expressed by RGB gradation values, for example. The input unit 11 acquires image data of the oral cavity image from the imaging device connected to the input interface. The input interface may be an interface for accessing a recording medium on which the captured image data is recorded. The input unit 11 acquires image data related to the oral cavity image by reading out the image data recorded on the recording medium. The image data acquired by the input unit 11 is output to the control unit 12 and stored in the storage unit 13 via the control unit 12.

[0015] The control unit 12 includes, for example, a central processing unit (CPU), a read only memory (ROM), and a random access memory (RAM). The ROM included in the control unit 12 stores a control program for controlling the operation of each hardware part included in the estimation device 1. The CPU in the control unit 12 executes the control program stored in the ROM and various computer programs stored in a storage unit 13 described later, and controls the operation of each hardware part to realize a function of estimating the presence or absence of a lesion in the oral mucosa from an oral cavity image. The RAM included in the control unit 12 temporarily stores data used during the execution of calculations.

[0016] The control unit 12 is configured to include a CPU, a ROM, and a RAM, but may be one or more arithmetic circuits including a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), a quantum processor, a volatile or non-volatile memory, etc. The control unit 12 may also include functions such as a clock that outputs date and time information, a timer that measures the elapsed time from when a measurement start instruction is given until when a measurement end instruction is given, and a counter that counts numbers.

[0017] The storage unit 13 includes a storage device such as a hard disk drive (HDD), a solid state drive (SSD), an electronically erasable programmable read only memory (EEPROM), etc. The storage unit 13 stores a computer program executed by the control unit 12, a learning model 130 used in a process of estimating the presence or absence of a lesion in the oral mucosa, and the like.

[0018] The computer program stored in the storage unit 13 includes an estimation processing program P1 for causing the estimation device 1 to execute a process of estimating the presence or absence of a lesion in the oral mucosa from the acquired oral cavity image using the learning model 130.

[0019] The computer program stored in the storage unit 13 may be provided by a non-transitory recording medium M1 on which the computer program is readably recorded. The recording medium M1 is, for example, a portable memory such as a CD-ROM, a USB memory, a Secure Digital (SD) card, a micro SD card, or a Compact Flash (registered trademark). The control unit 12 reads various programs from the recording medium M1 via, for example, the input unit 11, and installs the various programs thus read into the storage unit 13.

[0020] The learning model 130 is configured to output information on lesions in the oral mucosa in response to an input of an oral cavity image. The learning model 130 is described by its definition information. The definition information of the learning model 130 includes structural information of the learning model 130, various parameters such as weights and biases between nodes used in the learning model 130, and the like. In this embodiment, the learning model 130, which has been trained in advance by a predetermined learning algorithm using oral cavity images and annotations for the oral cavity images as training data, is stored in the storage unit 13.

[0021] The control unit 12 executes the estimation processing program P1 stored in the storage unit 13, and obtains information about a lesion from the learning model 130 by providing image data of an oral cavity image to the learning model 130. The control unit 12 estimates the presence or absence of a lesion in the oral mucosa based on the information about the lesion obtained from the learning model 130.

[0022] The output unit 14 includes an output interface for connecting an output device. An example of the output device is a display device 140 including a liquid crystal panel or an organic EL (Electro-Luminescence) panel. When outputting the estimation result, the control unit 12 generates display data to be displayed on the display device 140, and outputs the generated display data to the display device 140 via the output unit 14, thereby causing the display device 140 to display the estimation result.

[0023] The communication unit 15 includes a communication interface for transmitting and receiving various data. The communication interface included in the communication unit 15 is, for example, a communication interface conforming to a communication standard of a LAN (Local Area Network) used in WiFi (registered trademark) or Ethernet (registered trademark). When data to be transmitted is input from the control unit 12, the communication unit 15 transmits the data to be transmitted to a specified destination. When data transmitted from an external device is received, the communication unit 15 outputs the received data to the control unit 12.

[0024] The operation unit 16 is equipped with an input interface such as various operation buttons, switches, a touch panel, etc., and receives various operation information and setting information. The control unit 12 performs appropriate control based on the operation information input from the operation unit 16, and stores the setting information in the storage unit 13 as necessary.

[0025] Next, the oral cavity image input to the estimation device 1 will be described. Fig. 2 is a schematic diagram showing an example of an oral cavity image. The oral cavity image in this embodiment is an image obtained by capturing an image of the inside of the oral cavity of a subject using an imaging device. In the example of Fig. 2, an oral cavity image captured so as to include the left side of the tongue of the subject is shown.

[0026] In this embodiment, it is sufficient that at least a portion of the oral mucosa is included in the oral cavity image. The oral mucosa includes at least a portion of the subject's tongue, upper lip, hard palate, soft palate, uvula, tonsils, buccal mucosa, floor of the mouth, gums, and lower lip. The oral cavity image may also include objects other than the oral mucosa, such as the subject's teeth, the imager's fingers, and other structures.

[0027] Next, the learning model 130 used in the estimation device 1 will be described. 3 is a schematic diagram showing a configuration example of the learning model 130. The learning model 130 is, for example, a learning model based on CNN (Convolutional Neural Networks), and includes an input layer 131, an intermediate layer 132, and an output layer 133. The learning model 130 is trained in advance to output information related to a lesion in the oral mucosa in response to an input of an oral cavity image.

[0028] Image data of an oral cavity image is input to the input layer 131. The image data of an oral cavity image input to the input layer 131 is sent to the intermediate layer 132.

[0029] The intermediate layer 132 is composed of, for example, a convolution layer 132a, a pooling layer 132b, and a fully connected layer 132c. The convolution layers 132a and the pooling layers 132b may be provided alternately in multiple layers. The convolution layer 132a and the pooling layer 132b extract features of the oral cavity image input through the input layer 131 by calculation using the nodes of each layer. The fully connected layer 132c combines data from which feature parts are extracted by the convolution layer 132a and the pooling layer 132b into one node, and outputs feature variables transformed by an activation function. The feature variables are output to the output layer 133 through the fully connected layer 132c.

[0030] The output layer 133 includes one or more nodes. Based on the feature variables input from the fully connected layer 132c of the intermediate layer 132, the output layer 133 converts the feature variables into probabilities using a softmax function, and outputs the probability that the oral image belongs to each category from each node. That is, in this embodiment, the probability that the oral image belongs to each category is output as information about the lesion. The categories for classifying the oral images can be arbitrarily set to include at least one lesion belonging to oral malignant tumors, precancerous lesions, benign tumors, traumatic ulcers, inflammatory diseases, viral diseases, fungal infections, autoimmune diseases, stomatitis, angular cheilitis, decubitus ulcers, organic changes in the tongue surface mucosa, or graft-versus-host disease. For example, the classification categories may include at least one of oral cancer and oral sarcoma belonging to oral malignant tumors, leukoplakia, erythema, and lichen planus belonging to precancerous lesions, gingivitis, periodontitis, jaw inflammation, osteomyelitis of the jaw, and drug-induced osteonecrosis of the jaw belonging to inflammatory diseases, herpes, shingles, herpangina, and hand, foot, and mouth disease belonging to viral diseases, oral candidiasis belonging to fungal infections, pemphigus, pemphigoid, and Behcet's disease belonging to autoimmune diseases, and geographic tongue, grooved tongue, black hairy tongue, and median rhomboid glossitis belonging to organic changes in the tongue surface mucosa. Furthermore, the classification categories may include pigmentation not belonging to lesions, and may include normal states not belonging to lesions and pigmentation.

[0031] 3 shows a learning model 130 in which n categories are set as categories for classifying oral cavity images. This learning model 130 is configured to output a probability X1 of oral malignant tumor, a probability X2 of leukoplakia, a probability X3 of lichen planus, ..., a probability Xn of normality from each node of the output layer 133. The number of categories (=n) set may be one or more.

[0032] The control unit 12 of the estimation device 1 acquires the probability for each of the lesions set as a category to be classified from the output layer 133 of the learning model 130, and estimates the presence or absence of a lesion in the oral mucosa based on the acquired probability. For example, when only the probability X1 of an oral malignant tumor exceeds a threshold (e.g., 80%), the control unit 12 can estimate that a malignant tumor has developed in the oral cavity of the subject. The same applies to the case where any one of the probabilities X2, X3, ..., Xn-1 exceeds the threshold. On the other hand, when none of the probabilities X1, X2, ..., Xn-1 exceeds the threshold, or when the probability Xn indicating normality exceeds the threshold, the control unit 12 can estimate that no lesion is present in the oral cavity of the subject.

[0033] In addition, although the learning model 130 based on CNN is shown in the example of Fig. 3, the machine learning model for constructing the learning model 130 can be set arbitrarily. For example, instead of CNN, a learning model based on R-CNN (Region-based CNN), YOLO (You Only Look Once), SSD (Single Shot Detector), etc. may be set. Also, an algorithm such as U-Net may be used to output information on an area estimated to be a lesion.

[0034] 4 is a flowchart illustrating the procedure of processing executed by the estimation device 1 according to embodiment 1. The control unit 12 of the estimation device 1 executes an estimation processing program P1 stored in the storage unit 13 to perform the following estimation processing.

[0035] The control unit 12 acquires an oral cavity image through the input unit 11 (step S101), and performs a calculation using the learning model 130 by providing the acquired oral cavity image to the input layer 131 of the learning model 130 (step S102). The image data of the oral cavity image provided to the input layer 131 of the learning model 130 is sent to the intermediate layer 132. In the intermediate layer 132, a calculation is performed using an activation function including weights and biases between nodes. The convolution layer 132a and the pooling layer 132b of the intermediate layer 132 extract image features. The data of the feature parts extracted by the convolution layer 132a and the pooling layer 132b are connected to each node constituting the fully connected layer 132c, and are converted into feature variables by the activation function. The converted feature variables are output to the output layer 133 through the fully connected layer 132c. The output layer 133 converts the feature variables input from the fully connected layer 132c of the intermediate layer 132 into probabilities using a softmax function, and outputs the probability of belonging to each category from each node.

[0036] The control unit 12 acquires the calculation result from the learning model 130, and estimates the presence or absence of a lesion in the oral mucosa based on the acquired calculation result (step S103). As described above, each node constituting the output layer 133 of the learning model 130 outputs a probability for each of the lesions set as a category to be classified. The control unit 12 can estimate the presence or absence of a lesion based on the probability output from each node of the output layer 133.

[0037] The control unit 12 outputs the estimation result through the output unit 14 (step S104). Specifically, the control unit 12 generates display data for displaying the estimation result on the display device 140, and outputs the generated display data to the display device 140, thereby causing the display device 140 to display the estimation result. The display mode of the estimation result can be set arbitrarily. For example, the control unit 12 may generate display data including characters or figures representing the presence or absence of a specific lesion (e.g., oral malignant tumor), output the display device 140, and display the presence or absence of the specific lesion by the characters or figures on the display device 140. The control unit 12 may also generate display data including a probability value corresponding to each lesion, output the display device 140, and display the probability value corresponding to each lesion as numerical information on the display device 140.

[0038] Fig. 5 is a schematic diagram showing an example of output from the estimation device 1. The example in Fig. 5 shows a state in which a subject ID for identifying the subject, the subject's name, the oral cavity image used in the estimation process, the probability of falling into each category, and text information showing the estimation result are displayed on the display device 140. In addition, the learning model 130 may be used to identify the position of the lesion in the oral cavity image, and information on the identified position of the lesion may also be displayed.

[0039] As described above, in this embodiment, the presence or absence of a lesion in the oral mucosa is estimated using the machine learning learning model 130 including deep learning, and the estimation result is output. Therefore, by using the estimation result for diagnostic support, the possibility of overlooking a lesion can be reduced.

[0040] In this embodiment, the estimation device 1 has been described as a computer device installed in a facility such as a hospital, but the estimation device 1 may be a server device accessible by communication from a computer device installed in a facility such as a hospital. In this case, the estimation device 1 acquires oral cavity images of the subject's oral cavity from the computer device of the hospital or the like via communication, and estimates the presence or absence of a lesion in the oral mucosa based on the acquired oral cavity images. The estimation device 1 transmits the estimation result to the computer device of the hospital or the like via communication. The computer device provides diagnostic support to doctors and the like by displaying the estimation result received from the estimation device 1 on a display device.

[0041] (Embodiment 2) In the second embodiment, a configuration for performing preprocessing on an oral cavity image before executing a calculation using the learning model 130 will be described.

[0042] 6 is a flowchart illustrating the procedure of processing executed by the estimation device 1 according to embodiment 2. The control unit 12 of the estimation device 1 executes an estimation processing program P1 stored in the storage unit 13 to perform the following estimation processing.

[0043] The control unit 12 acquires an oral cavity image through the input unit 11 (step S201) and performs preprocessing on the acquired oral cavity image (step S202). The control unit 12 can perform, for example, gamma correction as preprocessing on the oral cavity image. That is, assuming that the luminance value of each pixel before correction is x, the luminance value of each pixel after correction is y, and the gamma value is γ, the control unit 12 calculates the luminance value of each pixel after correction as y=x 1 / γWhen imaging the oral cavity using an imaging device, a strong light such as a strobe light may be used with a short emission time. In this case, the entire oral cavity image may become unnecessarily bright (whitish), and the strobe light or the like may be reflected by the mucous membrane in the oral cavity, causing a phenomenon in which the gradation of the bright part is lost (whiteout), which may make it difficult to accurately identify the lesion. On the other hand, if the gamma value is set to a small value, for example, less than 0.2, the entire image becomes too dark and it becomes difficult to find the lesion. For this reason, it is preferable to set the gamma value to a value of 0.2 or more and less than 1.0. The control unit 12 may statistically analyze the luminance value of each pixel for the oral cavity image before correction and set the gamma value based on, for example, the median value of the luminance value. The control unit 12 may also acquire model information of the imaging device, strobe, etc. used to image the oral cavity, and set the gamma value according to the acquired model information. In the latter case, a table that defines the correspondence between the model information of the imaging device and the gamma value is prepared in the storage unit 13, and the gamma value may be determined by referring to this table.

[0044] Level correction may be performed as pre-processing of the oral cavity image. For example, if the oral cavity image to be corrected has a histogram peak in the range of 100 to 240 among the input levels of 0 to 255, correction may be performed to change the shadow point to the inside of the histogram peak (for example, 110). This pre-processing makes it possible to correct the gradation value of an image that is generally whitish due to the influence of strobe light, etc., and by using the corrected oral cavity image, it becomes possible to easily distinguish diseased areas. It is not necessary to perform correction on highlight points.

[0045] The control unit 12 executes a calculation by the learning model 130 by providing the preprocessed oral cavity image to the input layer 131 of the learning model 130 (step S203). That is, the control unit 12 executes a calculation using a weight and a bias in each node of the input layer 131, the intermediate layer 132, and the output layer 133 constituting the learning model 130, similarly to the first embodiment.

[0046] The control unit 12 acquires the calculation result from the learning model 130, and estimates the presence or absence of a lesion in the oral mucosa based on the acquired calculation result (step S204). Each node constituting the output layer 133 of the learning model 130 outputs a probability for each of the lesions set as a category to be classified. The control unit 12 estimates the presence or absence of a lesion based on the probability output from each node of the output layer 133.

[0047] The control unit 12 outputs the estimation result through the output unit 14 (step S205). Specifically, the control unit 12 generates display data for displaying the estimation result on the display device 140, and outputs the generated display data to the display device 140, thereby causing the display device 140 to display the estimation result. The display manner of the estimation result is the same as in the first embodiment.

[0048] In the present embodiment, the gamma correction is performed as pre-processing of the oral cavity image, but the gamma correction is not limited to this, and brightness correction may be performed according to a preset tone curve. Also, the pre-processing may be performed, such as contrast correction, saturation correction, or the like, instead of brightness correction.

[0049] (Embodiment 3) In the third embodiment, a configuration will be described in which the estimation device 1 extracts a region corresponding to the oral mucosa from an oral cavity image, and estimates the presence or absence of a lesion in the oral mucosa from the oral cavity image of the extracted region.

[0050] 7 is a block diagram illustrating the configuration of an estimation device 1 according to embodiment 3. The estimation device 1 includes an input unit 11, a control unit 12, a storage unit 13, an output unit 14, a communication unit 15, and an operation unit 16. These components are similar to those in embodiment 1, and therefore detailed descriptions thereof will be omitted.

[0051] In addition to the learning model 130 and the estimation processing program P1, the storage unit 13 also stores a region extraction program P2. The region extraction program P2 is a computer program for causing the estimation device 1 to execute a process of extracting a region corresponding to the oral mucosa of the subject from an oral cavity image. A known region extraction algorithm is used for the region extraction program P2.

[0052] One example of a region extraction algorithm is the GrabCut algorithm, which learns the distribution of pixel values ​​in the foreground and background regions using a Gaussian Mixture Model (GMM), and for pixels set as unknown regions, calculates the likelihood of them being foreground or background based on the relationship between the foreground and background regions based on the statistical values ​​of the pixel values, thereby separating and extracting the foreground and background regions.

[0053] The estimation device 1 according to the present embodiment extracts a region corresponding to the oral mucosa from the entire image region of the oral cavity image as a foreground region, and extracts the entire image region excluding the region corresponding to the oral mucosa as a background region. The background region may include the subject's teeth, the photographer's fingers, various tools, etc., but the foreground region can be separated from the background region and extracted by using a region extraction algorithm such as the GrabCut algorithm.

[0054] In this embodiment, the region corresponding to the oral mucosa (foreground region) is separated from other regions (background region), but it is also possible to extract the region corresponding to a specific oral mucosa (e.g., tongue) as the foreground region and extract the other regions including the oral mucosa as the background region.

[0055] FIG. 8 is a schematic diagram showing an extraction example. The example in FIG. 8 shows the result of separating the entire image area of ​​the oral cavity image into an area (foreground area) corresponding to the oral mucosa of the subject and the other area (background area). In FIG. 8, the background area is shown as a hatched area. It can be seen that the background area includes an area corresponding to the teeth of the subject and an area outside the oral cavity. The estimation device 1 estimates the presence or absence of a lesion in the oral mucosa by transferring the oral cavity image of the area (foreground area) corresponding to the oral mucosa of the subject to the learning model 130.

[0056] 9 is a flowchart illustrating the procedure of processing executed by the estimation device 1 according to embodiment 3. The control unit 12 of the estimation device 1 executes an estimation processing program P1 and a region extraction program P2 stored in the storage unit 13, thereby performing the following estimation processing.

[0057] The control unit 12 acquires an oral cavity image through the input unit 11 (step S301), and extracts an area corresponding to the oral mucosa from the entire image area of ​​the acquired oral cavity image (step S302). By the process of step S302, the areas corresponding to the subject's teeth, the imager's fingers, and various instruments are removed from the oral cavity image.

[0058] Next, the control unit 12 executes a calculation using the learning model 130 by providing the oral cavity image in which the region corresponding to the oral cavity region has been extracted (an image in which the portion corresponding to the subject's teeth, etc. has been removed) to the input layer 131 of the learning model 130 (step S303). The oral cavity image data provided to the input layer 131 of the learning model 130 is sent to the intermediate layer 132. In the intermediate layer 132, a calculation is executed using an activation function including weights and biases between nodes. Image features are extracted in the convolution layer 132a and pooling layer 132b of the intermediate layer 132. The data of the feature parts extracted by the convolution layer 132a and pooling layer 132b are connected to each node constituting the fully connected layer 132c and converted into feature variables by the activation function. The converted feature variables are output to the output layer 133 through the fully connected layer 132c. The output layer 133 converts the feature variables input from the fully connected layer 132c of the intermediate layer 132 into probabilities using a softmax function, and outputs the probability of belonging to each category from each node.

[0059] The control unit 12 acquires the calculation result from the learning model 130, and estimates the presence or absence of a lesion in the oral mucosa based on the acquired calculation result (step S304). As described above, each node constituting the output layer 133 of the learning model 130 outputs a probability for each of the lesions set as a category to be classified. The control unit 12 can estimate the presence or absence of a lesion based on the probability output from each node of the output layer 133.

[0060] The control unit 12 outputs the estimation result through the output unit 14 (step S305). Specifically, the control unit 12 generates display data for displaying the estimation result on the display device 140, and outputs the generated display data to the display device 140, thereby displaying the estimation result on the display device 140. The display mode of the estimation result can be set arbitrarily. For example, the control unit 12 may generate display data including characters or figures representing the presence or absence of a specific lesion (e.g., oral malignant tumor), output the display device 140, and display the presence or absence of the specific lesion by the characters or figures on the display device 140. The control unit 12 may also generate display data including a probability value corresponding to each lesion, output the display device 140, and display the probability value corresponding to each lesion as numerical information on the display device 140.

[0061] As described above, in the third embodiment, since the estimation process can be performed after removing parts unnecessary for estimating a lesion, it is possible to improve the estimation accuracy.

[0062] In the third embodiment, image processing such as gamma correction may also be performed as preprocessing before inputting the oral cavity image to the learning model 130.

[0063] (Embodiment 4) In the fourth embodiment, a method for generating the learning model 130 will be described.

[0064] The learning model 130 used in the estimation device 1 is generated, for example, in a server device 2 that is communicatively connected to the estimation device 1.

[0065] 10 is a block diagram illustrating the configuration of the server device 2. The server device 2 includes a control unit 21, a storage unit 22, an input unit 23, a communication unit 24, an operation unit 25, and a display unit .

[0066] The control unit 21 includes, for example, a CPU, a ROM, a RAM, etc. The ROM included in the control unit 21 stores a control program and the like for controlling the operation of each hardware unit included in the server device 2. The CPU in the control unit 21 executes the control program stored in the ROM and various programs stored in the storage unit 22, and controls the operation of each hardware unit.

[0067] The control unit 21 is not limited to the above-mentioned configuration. The control unit 21 is not limited to a configuration including a CPU, a ROM, and a RAM. The control unit 21 may be, for example, one or more control circuits or arithmetic circuits including a GPU, an FPGA, a DSP, a volatile or non-volatile memory, etc. The control unit 21 may also have functions such as a clock that outputs date and time information, a timer that measures the elapsed time from when a measurement start instruction is given to when a measurement end instruction is given, and a counter that counts numbers.

[0068] The storage unit 22 includes a storage device such as a hard disk drive. The storage unit 22 stores various computer programs executed by the control unit 21, various data used by the computer programs, data acquired from the outside, and the like. An example of a computer program stored in the storage unit 22 is a model generation program P3 for generating a learning model. The storage unit 22 also includes an oral cavity image database (oral cavity image DB) 220 that stores oral cavity images and annotations of the oral cavity images in association with each other.

[0069] The input unit 23 includes an input interface for acquiring data and programs from a recording medium on which various data or programs are recorded. The various data and programs input through the input unit 23 are stored in the storage unit 22.

[0070] The communication unit 24 includes a communication interface that connects to the communication network N. The communication network N is the Internet, a LAN for a specific purpose, or a Wide Area Network (WAN), etc. The communication unit 24 transmits data to be transmitted to the estimation device 1 to the estimation device 1 via the communication network N. The communication unit 24 also receives data transmitted from the estimation device 1 to the server device 2 via the communication network N.

[0071] The operation unit 25 includes an input interface such as a keyboard and a mouse, and receives various types of operation information and setting information. The control unit 21 performs appropriate control based on the operation information input from the operation unit 25, and stores the setting information in the storage unit 22 as necessary.

[0072] The display unit 26 includes a display device such as a liquid crystal display panel, an organic EL display panel, or the like, and displays information to be notified to the administrator of the server device 2, etc., based on a control signal output from the control unit 21.

[0073] In this embodiment, the server device 2 is configured to include an operation unit 25 and a display unit 26, but the operation unit 25 and the display unit 26 are not essential, and the server device 2 may be configured to accept operations via an externally connected computer and output information to be notified to the external computer.

[0074] FIG. 11 is a conceptual diagram showing an example of the oral cavity image database 220. In the oral cavity image database 220, oral cavity images and annotations for the oral cavity images are stored in association with each other. The oral cavity images include, for example, images of oral cavities with malignant tumors, images of oral cavities with morphologies specific to oral mucosal diseases (for example, ulcers, erosions, swellings, etc.). The annotations include the doctor's diagnosis. The diagnosis includes a pathological diagnosis result or a definitive diagnosis result, and in this embodiment, is used as label data indicating that the associated and stored oral cavity image is normal, or label data indicating which pathology it is. The annotations may also include information such as the subject ID and the subject's name.

[0075] The procedure for generating a learning model in the server device 2 will be described below. 12 is a flowchart for explaining the procedure for generating a learning model. The control unit 21 of the server device 2 accesses the oral cavity image database 220 in the storage unit 22 and acquires training data used to generate the learning model (step S401). The training data includes, for example, oral cavity images and annotations for the oral cavity images. In the initial stage of generating the learning model, the training data prepared by the administrator of the server device 2 or the like is set. In addition, as the learning progresses, the estimation result by the learning model 130 and the oral cavity images used in the estimation process may be acquired from the estimation device 1, and the acquired data may be set as the training data.

[0076] Next, the control unit 21 inputs the image data included as training data into the learning model for learning (step S402), and obtains the calculation result from the learning model (step S403). Before the learning starts, the definition information describing the learning model is set to an initial setting value. The calculation by this learning model is the same as the calculation by the learning model 130 in the estimation process.

[0077] Next, the control unit 21 evaluates the calculation result obtained in step S403 (step S404) and determines whether learning is complete or not (step S405). Specifically, the control unit 21 can evaluate the calculation result using an error function (also called an objective function, loss function, or cost function) based on the calculation result obtained in step S403 and the training data. The control unit 21 determines that learning is complete when the error function becomes equal to or smaller than a threshold (or equal to or larger than a threshold) in the process of optimizing (minimizing or maximizing) the error function by a gradient descent method such as the steepest descent method. In order to avoid the problem of overlearning, a method such as cross-validation or early termination may be adopted to terminate learning at an appropriate timing.

[0078] When it is determined that the learning is not completed (S405: NO), the control unit 21 updates the weights and biases between the nodes of the learning model (step S406) and returns the process to step S401. The control unit 21 can update the weights and biases between the nodes by using the backpropagation method that sequentially updates the weights and biases between the nodes from the output layer to the input layer of the learning model.

[0079] When it is determined that the learning is completed (S405: YES), the control unit 21 stores the learning model in the storage unit 22 as a learned learning model (step S407), and ends the process according to this flowchart.

[0080] As described above, in this embodiment, the learning model 130 used in the estimation device 1 can be generated in the server device 2. In response to a request from the estimation device 1, the server device 2 transmits the generated learning model to the estimation device 1. The estimation device 1 receives the learning model from the server device 2, stores it in the storage unit 13, and then executes the estimation processing program P1, thereby performing lesion estimation processing.

[0081] Furthermore, the server device 2 may be configured to collect new oral cavity images and annotations for the oral cavity images at an appropriate timing after the completion of learning, and re-learn the learning model using these data. The oral cavity image may be an oral cavity image (see FIG. 2) obtained by capturing an image including at least a part of the oral mucosa, or an oral cavity image (see FIG. 8) obtained by extracting an area corresponding to the oral mucosa. In addition, when displaying the estimation result, the estimation device 1 may accept a selection (diagnosis result) of whether the estimation result is correct or not, and transmit the accepted diagnosis result to the server device 2 as an annotation. The procedure of re-learning is exactly the same as the procedure of generating the learning model, and re-learning is performed by inputting the oral cavity image included in the training data to the learning model and evaluating the error between the calculation result obtained as the output of the learning model and the annotation included in the training data.

[0082] (Embodiment 5) In the fifth embodiment, a configuration for estimating the presence or absence of a lesion in the oral mucosa using a magnified image of the oral cavity will be described.

[0083] 13 is a block diagram for explaining the configuration of an estimation device 1 according to embodiment 5. The estimation device 1 includes an input unit 11, a control unit 12, a storage unit 13, an output unit 14, a communication unit 15, and an operation unit 16. These components are similar to those in embodiment 1, and therefore detailed explanations thereof will be omitted. It is decided.

[0084] In the fifth embodiment, a learning model is prepared for each part of the oral cavity, such as the tongue, upper lip, hard palate, soft palate, uvula, palatine tonsils, buccal mucosa, floor of the mouth, gums, and lower lip. The learning model 130-k (k=1, 2, ..., m; m is an integer of 1 or more) for each part is stored in the storage unit 13. The learning model 130-k is configured to output information about a lesion in the oral mucosa in response to an input of an enlarged image, which will be described later. Note that, although the present embodiment is configured to prepare the learning model 130-k for each part of the oral cavity, a common learning model may be used for several parts. For example, a common learning model may be prepared for the hard palate and the soft palate, and a common learning model may be prepared for the upper lip and the lower lip. Also, a different learning model may be used for each divided region obtained by dividing a specific part. For example, different learning models may be prepared for the upper surface region and the lower surface region of the tongue.

[0085] FIG. 14 is a schematic diagram showing an example of a magnified image. The magnified image in this embodiment is an oral cavity image obtained by magnifying the oral cavity of a subject and capturing an image of the oral cavity of the subject. Such a magnified image is obtained by capturing an image of the oral cavity of the subject using a contact-type cell observation device such as a contact endoscope system. In this embodiment, in order to observe the shape and distribution of epithelial cell nuclei, the observation site is stained with a dye such as methylene blue, and the observation site is imaged at a magnification of about 500 times with the imaging unit of the contact-type cell observation device pressed against the observation site.

[0086] The example of FIG. 14 shows an enlarged image of a part of the tongue of a subject, which is enlarged 500 times. Due to space limitations, the enlarged image is shown in grayscale, but the actual enlarged image obtained from the contact-type cell observation device is a color image. In the enlarged image, epithelial cell nuclei stained with a staining agent are observed. The example of FIG. 14 shows a state in which a large number of epithelial cell nuclei are observed as small circular or oval regions that are elongated and crushed from a circular shape. In this embodiment, the learning model 130-k is generated by learning the characteristics of normal epithelial cell nuclei, such as shape, stainability, distribution and arrangement, and nucleus / cytoplasm ratio, and the characteristics of epithelial cell nuclei in the lesion area, such as shape, stainability, distribution and arrangement, and nucleus / cytoplasm ratio, for each of the above-mentioned sites. The method of generating the learning model 130-k will be described in detail later.

[0087] 15 is a schematic diagram showing an example of the configuration of the learning model 130-k. The configuration of the learning model 130-k is the same as the configuration of the learning model 130 described in the first embodiment. That is, the learning model 130-k is a learning model using CNN, and includes an input layer 131, an intermediate layer 132, and an output layer 133. The learning model 130 is configured to perform a calculation in the intermediate layer 132 when the enlarged image described above is input to the input layer 131, and to output information related to a lesion in the oral mucosa from the output layer 133.

[0088] The example of FIG. 15 shows a learning model 130-k in which n categories are set as categories for classifying enlarged images. This learning model 130-k is configured to output, from each node of the output layer 133, a probability X1 of oral malignant tumor, a probability X2 of leukoplakia, a probability X3 of lichen planus, ..., a probability Xn of normality. The number of categories (=n) to be set may be one or more. The category for classifying the enlarged image may be set for each target site. For example, when the target site is the tongue, the category may be set as a category for classifying lesions belonging to organic changes of the tongue surface mucosa, such as geographic tongue, grooved tongue, black hairy tongue, and median rhomboid glossitis, in addition to oral malignant tumor, leukoplakia, and lichen planus. When the target site is the gums, the category may be set as a category for classifying gingivitis, periodontitis, and the like, in addition to oral malignant tumor, leukoplakia, and lichen planus. The same applies to other target sites, and a category for classifying may be set for each target site.

[0089] In addition, although the learning model 130 based on CNN is shown in the example of FIG. 15, the machine learning model for constructing the learning model 130-k can be set arbitrarily. For example, instead of CNN, a learning model based on R-CNN, YOLO, SSD, or the like may be set. Also, information on an area estimated to be a lesion may be output using an algorithm such as U-Net.

[0090] The learning model 130-k is generated, for example, in a server device 2 accessible from the estimation device 1. Prior to generating the learning model 130-k, the server device 2 collects data including an enlarged image, a subject ID, a subject name, and a doctor's diagnosis for each target site, and associates the information with each other and registers it in the oral cavity image database 220. Fig. 16 is a conceptual diagram showing an example of the oral cavity image database 220 in the fifth embodiment.

[0091] The server device 2 generates the learning model 130-k by using the data registered in the oral cavity image database 220 as training data. FIG. 17 is a flowchart for explaining the procedure for generating the learning model 130-k. The control unit 21 of the server device 2 accepts the designation of a target site through, for example, the operation unit 25 (step S501), and selects data such as an enlarged image associated with the designated target site and annotations (diagnosis results) for the enlarged image stored in the oral cavity image database 220 as training data (step S502). In the initial stage of generating the learning model 130-k, the training data is set to data prepared by the administrator of the server device 2, etc. As the learning progresses, the estimation result by the learning model 130-k and the enlarged image used in the estimation process may be acquired from the estimation device 1, and the acquired data may be set as training data.

[0092] The control unit 21 inputs the image data of the enlarged image selected as training data to the learning model 130-k for learning (step S503), and obtains the calculation result from the learning model 130-k (step S504). Before the learning starts, the definition information describing the learning model 130-k is set to an initial setting value. The calculation procedure using this learning model 130-k is the same as that of the fourth embodiment.

[0093] Next, the control unit 21 evaluates the calculation result obtained in step S504 (step S505) and determines whether learning is complete or not (step S506). Specifically, the control unit 21 can evaluate the calculation result using an error function (also called an objective function, loss function, or cost function) based on the calculation result obtained in step S504 and the training data. The control unit 21 determines that learning is complete when the error function becomes equal to or smaller than a threshold (or equal to or larger than a threshold) in the process of optimizing (minimizing or maximizing) the error function by a gradient descent method such as the steepest descent method. In order to avoid the problem of overlearning, a method such as cross-validation or early termination may be adopted to terminate learning at an appropriate timing.

[0094] When it is determined that the learning is not completed (S506: NO), the control unit 21 updates the weights and biases between the nodes of the learning model (step S507) and returns the process to step S502. The control unit 21 can update the weights and biases between the nodes by using the backpropagation method that sequentially updates the weights and biases between the nodes from the output layer to the input layer of the learning model.

[0095] When it is determined that the learning is completed (S506: YES), the control unit 21 stores the learned learning model 130-k in the storage unit 22 (step S508), and ends the process according to this flowchart.

[0096] The server device 2 transmits the generated learning model 130-k together with the information on the target body part to the estimation device 1. The estimation device 1 stores the learning model 130-k transmitted from the server device 2 in the storage unit 13 in association with the information on the target body part.

[0097] 18 is a flowchart showing an estimation procedure using the learning model 130-k. The control unit 12 of the estimation device 1 executes an estimation processing program P1 stored in the storage unit 13 to perform the following estimation processing.

[0098] The control unit 12 acquires an enlarged image obtained by imaging the oral cavity using, for example, a contact-type cell observation device (step S511). The control unit 12 may acquire the enlarged image through the input unit 11, or may acquire the enlarged image through the communication unit 15. In addition, if an estimation result using an oral cavity image at normal magnification (for example, an estimation result as shown in FIG. 5) is obtained in advance, when performing observation using a contact-type cell observation device, the control unit 12 may display the subject ID, the subject name, the estimated lesion name, and the position of the lesion on the display device 140 to present the observer with information on the observation site. For example, a primary medical institution may perform a preliminary estimation by providing an oral cavity image at normal magnification to the learning model 130, and when performing observation using a contact-type cell observation device at a secondary medical institution, the estimation result obtained at the primary medical institution may be presented to the secondary medical institution. In this case, the estimation device 1 in which the learning model 130 is installed may be provided at the primary medical institution, and the estimation device 1 in which the learning model 130-k is installed may be provided at the secondary medical institution. Also, the learning model 130 and the learning model 130-k may be installed in a server device accessible from both the primary medical institution and the secondary medical institution. Fig. 19 is a schematic diagram showing an example of the presentation of the observation site. The observer adjusts the observation position based on the information of the observation site presented on the display device 140, and takes an image of the oral cavity using the contact-type cell observation device.

[0099] Next, the control unit 12 accepts the designation of the target part (step S512). When accepting the designation of the target part, the control unit 12 may display an interface screen on the display device 140. FIG. 20 is a schematic diagram showing an example of an interface screen 1400 for accepting the designation of the target part. For example, a pull-down menu 1401 for accepting the selection of the target part and a start button 1402 for accepting an instruction to start the estimation process are arranged on the interface screen 1400. The pull-down menu 1401 is configured to display the names of target parts (tongue, upper lip, hard palate, soft palate, uvula, palatine tonsils, buccal mucosa, floor of the mouth, gums, lower lip, etc.) that can be designated by an operation using the operation unit 16, and to accept the designation of the part to be estimated from the displayed name of the target part. The start button 1402 is configured to accept an instruction to start the estimation process by an operation using the operation unit 16. In the example of FIG. 20, an image of the oral cavity captured at normal magnification is shown, but an enlarged image captured using a contact-type cell observation device may be displayed on the interface screen 1400.

[0100] When the target part is designated in step S512 and an instruction to start the estimation process is received, the control unit 12 selects the learning model 130-k according to the designated target part (step S513), and executes a calculation by the learning model 130-k by providing the enlarged image acquired in step S511 to the selected learning model 130-k (step S514). The calculation procedure using the learning model 130-k is the same as that in the first embodiment.

[0101] The control unit 12 acquires the calculation result from the learning model 130-k, and estimates the presence or absence of a lesion in the oral mucosa based on the acquired calculation result (step S515). As described above, each node constituting the output layer 133 of the learning model 130-k outputs a probability for each of the lesions set as a category to be classified. The control unit 12 can estimate the presence or absence of a lesion based on the probability output from each node of the output layer 133.

[0102] The control unit 12 outputs the estimation result through the output unit 14 (step S516). Specifically, the control unit 12 generates display data for displaying the estimation result on the display device 140, and outputs the generated display data to the display device 140, thereby displaying the estimation result on the display device 140. The display mode of the estimation result can be set arbitrarily. For example, the control unit 12 may generate display data including characters or figures representing the presence or absence of a specific lesion (e.g., oral malignant tumor), output the display device 140, and display the presence or absence of the specific lesion by the characters or figures on the display device 140. The control unit 12 may also generate display data including a probability value corresponding to each lesion, output the display device 140, and display the probability value corresponding to each lesion as numerical information on the display device 140. An example of the output of the estimation device 1 is the same as that shown in FIG. 5.

[0103] As described above, in the fifth embodiment, by using the learning model 130-k, the presence or absence of a lesion in the oral mucosa can be estimated based on features such as the shape, staining, distribution and arrangement of epithelial cell nuclei observed in a magnified image, and the nucleus / cytoplasm ratio. In addition, by estimating the presence or absence of a lesion using such a learning model 130-k, it is possible to perform a minimally invasive and accurate diagnosis, and it is suggested that a tissue diagnosis comparable to a biopsy diagnosis can be performed. In addition, the diagnosis results by the learning model 130 may be stored in the memory unit 13 and used to clarify the risk of lesion occurrence in the oral mucosa, prognostic predictors, and the like.

[0104] The embodiments disclosed herein are illustrative in all respects and should not be considered as limiting. The scope of the present invention is defined by the claims, not by the above meaning, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0105] 1 Estimation device 2. Server equipment 11 Input section 12 Control section 13 Storage section 14 Output section 15 Communications Department 16 Control section 21 Control section 22 Memory section 23 Input section 24 Communications Department 25 Control section 26 Display section 130 Learning Model 130-1, 130-2, …, 130-m learning model 220 Oral Image Database P1 Estimation Processing Program P2 Region Extraction Program P3 Model Generator

Claims

1. an acquisition unit that acquires an enlarged image by imaging a specific site in the oral cavity of a subject at a magnification sufficient to observe the shape of epithelial cell nuclei; an estimation unit that estimates the presence or absence of a lesion in the specific site from the enlarged image acquired by the acquisition unit, using a learning model configured to output information about a lesion in the enlarged image when an enlarged image of the oral cavity is input; an output unit that outputs an estimation result of the estimation unit; An estimation device comprising:

2. The learning model is generated by learning the shape characteristics of normal epithelial cell nuclei and the shape characteristics of epithelial cell nuclei in a lesion area. The estimation device according to claim 1 .

3. The learning model is further generated by learning the characteristics of staining, distribution, arrangement, and nucleus / cytoplasm ratio of normal epithelial cell nuclei and the characteristics of staining, distribution, arrangement, and nucleus / cytoplasm ratio of epithelial cell nuclei in a lesion area. The estimation device according to claim 2 .

4. The enlarged image is an image obtained by capturing an image using a contact-type cell observation device. The estimation device according to any one of claims 1 to 3.

5. When an image of a subject's oral cavity (excluding an enlarged image captured by enlarging the observation area to such an extent that the shape of epithelial cell nuclei can be observed) is input, an inference result including the name and location of the lesion inferred in the oral mucosa of the subject is obtained from a primary medical institution using a learning model configured to output information regarding the lesion in the oral mucosa of the subject, The acquisition unit acquires an enlarged image obtained by imaging a specific site identified based on information on the name and location of a lesion included in the estimation result at a magnification sufficient to observe the shape of an epithelial cell nucleus. The estimation device according to any one of claims 1 to 4.

6. It is possible to use a plurality of learning models trained individually for each site in the oral cavity, a reception unit that receives a designation of a part captured in the enlarged image acquired by the acquisition unit; a selection unit that selects one learning model from the plurality of learning models according to the part specified through the reception unit; Equipped with The estimation unit estimates the presence or absence of a lesion in the specified region by using the one learning model selected by the selection unit. The estimation device according to any one of claims 1 to 5.

7. The estimation unit estimates the presence or absence of at least one lesion belonging to oral malignant tumor, precancerous lesion, benign tumor, traumatic ulcer, inflammatory disease, viral disease, fungal infection, autoimmune disease, stomatitis, angular cheilitis, decubitus ulcer, organic change of the tongue surface mucosa, or graft-versus-host disease. The estimation device according to any one of claims 1 to 6.

8. A magnified image is obtained by imaging a specific site in the oral cavity of a subject at a magnification sufficient to observe the shape of epithelial cell nuclei, When a magnified image of the oral cavity is input, a learning model configured to output information about a lesion in the magnified image is used to estimate the presence or absence of a lesion in the specific site from the acquired magnified image; Output the estimation result The estimation method is carried out by a computer.

9. an input layer to which an enlarged image obtained by imaging a specific site in the oral cavity of a subject at a magnification large enough to observe the shape of epithelial cell nuclei is input; An output layer that outputs information regarding a lesion in the specific site; and An intermediate layer that uses an enlarged image captured at a magnification sufficient to observe the shape of the epithelial cell nucleus and annotations for the enlarged image as training data to learn the relationship between the enlarged image input to the input layer and the information output by the output layer. Equipped with When an enlarged image is input to the input layer, the computer is operated so as to perform a calculation in the intermediate layer and output information regarding a lesion in the specific region from the output layer. Learning model.

10. By computer, Obtaining training data including an enlarged image obtained by enlarging and imaging a specific site in the oral cavity of a subject to a degree that allows observation of the shape of an epithelial cell nucleus and annotations for the enlarged image; A learning model is generated that outputs information about a lesion in the specific region in response to an input of an enlarged image based on the acquired training data. How to generate a learning model.

11. On the computer, Obtaining an enlarged image by imaging a specific site in the oral cavity of the subject at a magnification sufficient to observe the shape of epithelial cell nuclei; When an enlarged image of the oral cavity is input, a learning model configured to output information about a lesion in the enlarged image is used to estimate the presence or absence of a lesion in the specific area from the acquired enlarged image; Output the estimation result A computer program for executing a process.

Images