Program, storage medium, system, trained model, and judgment method

Abstract

To provide a program, a storage medium, a system, a learned model, and a determination method with which the presence or absence of the onset of a subject's appendicitis can be determined.SOLUTION: In a program for causing a computer to function as model generation means 22, receiving means 24, and operation means 26, the model generation means 22 uses training data 32 including a tongue surface image and information on the presence or absence of the onset of appendicitis to generate, through deep learning, a learned model 34 having the tongue surface image as input and the information on the presence or absence of the onset of appendicitis as output. The receiving means 24 receives an image of a tongue surface of a subject for whom the presence or absence of the onset of appendicitis should be diagnosed. The operation means 26 performs operation related to the presence or absence of the onset of appendicitis when receiving input of the tongue surface image received by the receiving means 24 based on the learned model 34 generated by the model generation means 22.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a program, a storage medium, a system, a trained model, and a determination method. [Background technology]

[0002] Tongue diagnosis, which uses the condition of the tongue as one of the indicators of the state of the body, has long been known in traditional Chinese medicine. A tongue diagnosis device that uses this theory of tongue diagnosis is known, for example, as disclosed in Patent Document 1. The tongue diagnosis device disclosed in Patent Document 1 allows doctors who do not have the skills to perform tongue diagnosis according to traditional Chinese medicine, doctors who are in remote locations and therefore cannot directly examine the patient's tongue, and even non-physician pharmacists to obtain tongue diagnosis results based on traditional Chinese medicine. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-028058 Summary of the Invention [Problem to be solved by the invention]

[0004] However, there has not been a tongue diagnosis device that can determine whether or not a subject has developed acute appendicitis.

[0005] The present invention has been made in consideration of these points, and aims to provide a program, storage medium, system, trained model, and determination method that can determine whether or not a subject has developed acute appendicitis. [Means for solving the problem]

[0006] The present invention provides a program that causes a computer to function as a model generation unit, a reception unit, and a calculation unit, the program comprising: the model generation means uses training data including a tongue surface image and information regarding the presence or absence of the onset of acute appendicitis to generate a trained model by deep learning, which inputs the tongue surface image and outputs information regarding the presence or absence of the onset of acute appendicitis; the receiving means receives a tongue surface image of a subject to be diagnosed for the presence or absence of acute appendicitis; The calculation means is a program that performs calculations regarding the presence or absence of the onset of acute appendicitis when a tongue surface image accepted by the acceptance means is input, based on the trained model generated by the model generation means.

[0007] The present invention provides a storage medium storing a program that causes a computer to function as a model generation unit, a reception unit, and a calculation unit, the model generation means uses training data including a tongue surface image and information regarding the presence or absence of the onset of acute appendicitis to generate a trained model by deep learning, which inputs the tongue surface image and outputs information regarding the presence or absence of the onset of acute appendicitis; the receiving means receives a tongue surface image of a subject to be diagnosed for the presence or absence of acute appendicitis; The calculation means is a storage medium that performs calculations regarding the presence or absence of the onset of acute appendicitis when a tongue surface image accepted by the acceptance means is input, based on the trained model generated by the model generation means.

[0008] The present invention provides a model generation means for generating a trained model by deep learning using teacher data including a tongue surface image and information on the presence or absence of the onset of acute appendicitis, and for inputting a tongue surface image and outputting information on the presence or absence of the onset of acute appendicitis; a receiving means for receiving a tongue surface image of a subject to be diagnosed for the presence or absence of acute appendicitis; a calculation means for calculating whether or not acute appendicitis has occurred when the tongue surface image received by the receiving means is input based on the trained model generated by the model generation means; and It is a system equipped with the above.

[0009] The present invention is a trained model that is generated by deep learning using training data including a tongue surface image and information regarding the presence or absence of acute appendicitis, and that inputs a tongue surface image and outputs information regarding the presence or absence of acute appendicitis.

[0010] The present invention provides a determination method performed by a computer, comprising: a step of generating a trained model by deep learning using training data including a tongue surface image and information regarding the presence or absence of the onset of acute appendicitis, the trained model inputting the tongue surface image and outputting information regarding the presence or absence of the onset of acute appendicitis; receiving a tongue surface image of a subject to be diagnosed for the presence or absence of acute appendicitis; A step of performing a calculation regarding the presence or absence of acute appendicitis when a received tongue surface image is input based on the generated trained model; The determination method includes the following steps. [Effects of the Invention]

[0011] According to the program, storage medium, system, trained model, and determination method of the present invention, it is possible to determine whether or not a subject has developed acute appendicitis. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a schematic diagram illustrating a configuration of a diagnostic system according to an embodiment of the present invention; [Figure 2] FIG. 2 is an explanatory diagram showing the generation of a trained model in the diagnostic system shown in FIG. 1 and calculations when a tongue surface image is input. [Figure 3] 2 is a flowchart showing a method for generating a trained model and a method for determining whether or not acute appendicitis has occurred by the diagnostic system shown in FIG. 1. [Figure 4] This is a tongue surface image calculated to have a high probability of acute appendicitis. [Figure 5] 5 is a heat map generated for the tongue surface image shown in FIG. 4. [Figure 6]This is a superimposed image generated by superimposing the tongue surface image shown in FIG. 4 and the heat map shown in FIG. 5. [Figure 7] This is a tongue surface image calculated to have an extremely low probability of developing acute appendicitis. [Figure 8] 8 is a heat map generated for the tongue surface image shown in FIG. 7. [Figure 9] This is a superimposed image generated by superimposing the tongue surface image shown in FIG. 7 and the heat map shown in FIG. 5. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Figures 1 to 9 are diagrams showing a diagnostic system according to this embodiment and an image of a tongue surface to be diagnosed.

[0014] First, the configuration of the diagnostic system according to this embodiment will be described with reference to Figures 1 and 2. As shown in Figure 1, etc., the diagnostic system 10 is configured from a computer and the like, and includes a control unit 20, a storage unit 30, a display unit 50, an operation unit 52, and a communication unit 54.

[0015] The control unit 20 is composed of a CPU (Central Processing Unit) and the like, and controls the operation of the diagnostic system 10. Specifically, the control unit 20 functions as a model generating means 22, a receiving means 24, a calculating means 26, and an output means 28 by executing a program 36 stored in a storage unit 30, which will be described later.

[0016] The model generation means 22 uses training data 32 including a tongue surface image and information regarding the presence or absence of the onset of acute appendicitis to generate a trained model 34 by deep learning, which inputs the tongue surface image and outputs information regarding the presence or absence of the onset of acute appendicitis. Details of the method for generating the trained model 34 by the model generation means 22 will be described later.

[0017] The receiving means 24 receives a tongue surface image of a subject who should be diagnosed for the presence or absence of acute appendicitis (i.e., a subject suspected of having acute appendicitis). Specifically, when a tongue surface image is transmitted to the diagnostic system 10 from an external device (e.g., an imaging device such as a digital camera or a mobile communication terminal such as a smartphone) via the communication unit 54 described below, the receiving means 24 receives the tongue surface image.

[0018] The calculation means 26 performs calculations based on the trained model 34 generated by the model generation means 22 regarding the presence or absence of the onset of acute appendicitis when the tongue surface image received by the reception means 24 is input.

[0019] The output means 28 outputs the calculation results obtained by the calculation means 26. The information output by the output means 28 is displayed on a display unit 50 or transmitted by a communication unit 54 to a device other than the diagnostic system 10.

[0020] The storage unit 30 is configured with, for example, a hard disk drive (HDD), a random access memory (RAM), a read-only memory (ROM), and a solid state drive (SSD). Furthermore, the storage unit 30 is not limited to being built into the diagnostic system 10, but may be a storage medium (for example, a USB memory) that can be detachably attached to the diagnostic system 10. In this embodiment, the storage unit 30 stores a trained model 34, a program 36, and the like.

[0021] The display unit 50 is, for example, a monitor or the like, and is configured to display various screens by receiving a display command signal from the control unit 20. The operation unit 52 is, for example, a keyboard or the like, and is configured to be able to give various commands to the control unit 20. In this embodiment, a display operation unit such as a touch panel in which the display unit 50 and the operation unit 52 are integrated may be used. The communication unit 54 includes a communication interface for transmitting and receiving signals to and from an external device wirelessly or via a wired connection.

[0022] Next, a diagnostic method using such diagnostic system 10 (specifically, a method for determining whether or not acute appendicitis has occurred) will be described with reference to Figures 2 and 3. Figure 2 is an explanatory diagram showing the generation of a trained model in diagnostic system 10 shown in Figure 1 and the calculations performed when a tongue surface image is input, and Figure 3 is a flowchart showing the determination method performed by diagnostic system 10 shown in Figure 1. The operations described below are performed by each of means 22, 24, 26, and 28 as a result of control unit 20 executing program 36 stored in storage unit 30.

[0023] First, in the diagnostic system 10, the model generation means 22 generates a trained model 34 in advance (STEPs 1 and 2). The method for generating the trained model 34 by the model generation means 22 is described in detail below. The model generation means 22 is configured with a neural network (NN), such as a convolutional neural network (CNN), a vision transformer (ViT), or a combination thereof. Such a neural network has an input layer to which training data 32 including a tongue surface image and information on the presence or absence of acute appendicitis is input, a middle layer to which parameters are trained using the training data 32, an output layer to output a prediction of the presence or absence of acute appendicitis, and a function to output a trained model 34 configured to include the trained parameters. When such a neural network receives the training data 32, it performs calculations in the middle layer, and outputs a prediction of the presence or absence of acute appendicitis from the output layer. Such a neural network can be newly configured or a known one can be used.

[0024] Here, the model generation means 22 may generate the trained model 34 by deep learning by adding the training data 32 to an existing model. The existing model is a model and trained parameters that have already been subjected to deep learning using tongue surface images or a large number of images other than tongue surface images. The trained model 34 can be efficiently created by transfer learning of such an existing model. In other words, even if the amount of training data 32 is small, a highly accurate trained model 34 can be generated. As the existing model, for example, MobileNet, a form of CNN, or ViT can be used. Note that this embodiment is not limited to such an embodiment. In another embodiment, the model generation means 22 may construct a new model without using an existing model, learn parameters using only the training data 32, and generate a trained model 34 including the trained parameters by deep learning.

[0025] Furthermore, in this embodiment, the teacher data 32 includes a training set, a validation set, and a test set (either the validation set or the test set may be absent), and the model generation means 22 first generates a trained model 34 using the training set (STEP 1), and then verifies the generated trained model 34 using the validation set and the test set (STEP 2). When the model generation means 22 creates the trained model 34 using the training set as the teacher data 32, it is not possible to evaluate overfitting, overlearning, or generalization performance for unknown data using only the teacher data 32 itself. For this reason, the generalization performance is verified using the validation set and the test set. A known method or a new method can be used as a verification method using the validation set and the test set.

[0026] Next, details of the calculation method by the calculation means 26 will be described. When the tongue surface image of a subject to be diagnosed for the presence or absence of acute appendicitis is received by the reception means 24 (STEP 3), the calculation means 26 performs a calculation regarding the presence or absence of acute appendicitis when the tongue surface image received by the reception means 24 is input, based on the trained model 34 generated by the model generation means 22 (STEP 4). Specifically, when performing the calculation regarding the presence or absence of acute appendicitis, the calculation means 26 calculates at least one of the probability of the presence or absence of acute appendicitis.

[0027] More specifically, the calculation means 26 first performs image processing on the tongue surface image (see FIGS. 4 and 7) received by the receiving means 24 based on the trained model 34 generated by the model generation means 22. This involves calculation of trained parameters and the tongue surface image, an example of which is extraction of image features as shown in a heat map (see FIGS. 5 and 8). If the extracted features are a heat map, a superimposed image (see FIGS. 6 and 9) is generated as the calculation result by overlaying it on the tongue surface image, and a calculation regarding the presence or absence of acute appendicitis is performed based on this generated superimposed image. The extracted features are not limited to heat maps (see FIGS. 5 and 8), and other features may also be extracted by the model generation means 22. In another aspect, for example, the calculation means 26 calculates trained parameters on the tongue surface image received by the receiving means 24 to extract features such as a wide-ranging dependency between the tongue surface image received by the receiving means 24 and the prediction of the presence or absence of appendicitis. If other features are extracted, the calculation results will be different from those of the superimposed images (see Figures 6 and 9), but in any case, appropriate calculations are performed between the tongue surface image and the extracted features to predict the presence or absence of acute appendicitis. Figures 4 to 6 show tongue surface images, heat maps, and superimposed images calculated to indicate a high probability of acute appendicitis, while Figures 7 to 9 show tongue surface images, heat maps, and superimposed images calculated to indicate an extremely low probability of acute appendicitis.

[0028] Thereafter, the calculation result by the calculation means 26 is output by the output means 28 (STEP 5). Specifically, at least one or both of the probability of acute appendicitis occurring and the probability of acute appendicitis not occurring are displayed on the display unit 50 or transmitted to an external device by the communication unit 54. This allows the user of the diagnostic system 10 to refer to the calculation result by the calculation means 26 (i.e., the probability of acute appendicitis occurring) when diagnosing whether or not the subject whose tongue surface image has been captured has acute appendicitis.

[0029] Furthermore, when an input is made via the operation unit 52 or the like that the probability of the onset of acute appendicitis calculated by the calculation means 26 is valid based on the actual onset of acute appendicitis in the subject, the tongue surface image received by the reception means 24 and the probability of the onset of acute appendicitis calculated by the calculation means 26 may be input to the model generation means 22 as training data 32, thereby updating the trained model 34 generated by the model generation means 22 each time. This increases the amount of training data 32 input to the model generation means 22 each time a diagnosis of a tongue surface image is made by the diagnostic system 10, thereby making it possible to make the trained model 34 more accurate.

[0030] According to the diagnostic system 10, program 36, storage medium (specifically, storage unit 30), and determination method of the present embodiment configured as described above, the model generation means 22 uses training data 32 including a tongue surface image and information regarding the presence or absence of acute appendicitis to generate a trained model 34 by deep learning. The trained model 34 receives a tongue surface image of a subject whose acute appendicitis is to be diagnosed. The calculation means 26 then performs a calculation regarding the presence or absence of acute appendicitis when the tongue surface image received by the calculation means 24 is input, based on the trained model 34 generated by the model generation means 22. In this way, when a tongue surface image is input, the trained model 34 generated by deep learning is used to perform a calculation regarding the presence or absence of acute appendicitis, thereby improving the accuracy of determining the presence or absence of acute appendicitis.

[0031] The system, program, storage medium, and determination method according to the present invention are not limited to the above-described aspects, and various modifications can be made.

[0032] For example, the program that causes a computer (specifically, the diagnostic system 10) to function as the model generating means 22, the receiving means 24, and the calculating means 26 is not limited to the program stored in the storage unit 30. Such a program may be stored in a recording medium such as a USB memory attached to the computer, or may be transmitted to the computer from an external device via the communication unit 54.

[0033] Furthermore, the model generation means 22 is not limited to generating the trained model 34 from the training data 32 using a neural network. In other examples of the diagnostic system according to the present invention, the model generation means 22 may generate the trained model 34 using other types of machine learning, such as XGBoost, LightGB, M-decision tree learning, association rule learning, and clustering.

[0034] Furthermore, the diagnostic system according to the present invention is not limited to one in which the computing means uses a trained model to perform a calculation regarding the presence or absence of acute appendicitis based on a tongue surface image. Another aspect of the diagnostic system may be one in which the computing means uses a trained model configured to include trained parameters that input a tongue surface image and output information regarding the presence or absence of acute appendicitis (prediction of the presence or absence of onset) and perform a calculation regarding the presence or absence of a disease other than acute appendicitis (e.g., digestive system diseases represented by stomach and liver diseases, or renal and urinary system diseases represented by kidney diseases) based on the tongue surface image of the subject, using a trained model that is generated by deep learning using training data including information regarding the presence or absence of the tongue surface image and diseases other than acute appendicitis (e.g., digestive system diseases represented by stomach and liver diseases, or renal and urinary system diseases represented by kidney diseases).

[0035] Furthermore, in the above description, the trained model 34 and the program 36 are stored in the storage unit 30, but the present embodiment is not limited to this. The trained model 34 and the program 36 may be stored in a storage unit of a device (e.g., a server) separate from the diagnostic system 10. Furthermore, an image taken with a smartphone or the like may be uploaded to a web server or the like, and the program or trained model on the server may be used to display only the results on the server, or only the results may be downloaded to a smartphone or the like and displayed. [Example]

[0036] [Example] Next, an example will be described in which the trained model 34 is generated and calculations are performed by the calculation means 26 using the diagnostic system 10 configured as shown in FIGS.

[0037] First, a trained model 34 was generated by the model generation means 22 using multiple sets of training data 32. Specifically, multiple random cross-validations based on stratified sampling were performed to prepare 90 sets of training data 32, including 66 training sets of men and women aged 20 years or older, 12 validation sets, and 12 test sets. Each set of training data 32 contained tongue surface images and information on the presence or absence of acute appendicitis. The model generation means 22 then generated a trained model 34 by deep learning by adding 66 training sets (41 sets with acute appendicitis and 25 sets without acute appendicitis) to the existing model. Specifically, as a deep learning method, a MobileNet (http: / / www.image-net.org / ) model trained on other image data and its trained parameters were given as initial values, and the parameters were fixed in the front layer of the model, while the rear layer was retrained with the 66 training data, a process known as transfer learning. Then, a newly generated trained model34, which used the retrained parameters as trained parameters, was validated using 12 validation sets (7 sets with acute appendicitis, 5 sets without) and 12 test sets (7 sets with acute appendicitis, 5 sets without). Specifically, a random training set, validation set, and test set split method was performed multiple times as a validation method using the validation set and test set. The parameters after final training were then used as trained parameters.

[0038] Using the trained model 34 thus generated, the calculation means 26 performed a calculation regarding the presence or absence of acute appendicitis based on tongue surface images input into the diagnostic system 10, and the calculation results (specifically, the proportion of people with acute appendicitis) were displayed on the display unit 50. When calculations were performed on a total of 12 tongue surface images of men and women aged 20 or older, 7 of which showed the presence of acute appendicitis and 5 of which showed no acute appendicitis, the presence or absence of acute appendicitis could be correctly diagnosed for 11 of the tongue surface images. In other words, the accuracy rate was approximately 92%.

[0039] Comparative Example Experts visually inspected the subjects' tongue surface images to determine whether or not they had acute appendicitis. Specifically, the images were divided into nine sections, three vertical and three horizontal, and two examiners scored each section, assigning a score of 0 for no visible tongue coating, 1 for thin tongue coating with visible papillae, and 2 for thick tongue coating with no visible papillae. The average score was calculated. A score of 10 or higher indicated a high likelihood of acute appendicitis, while a score of less than 4 indicated a low likelihood of acute appendicitis. Based on these results, trained experts visually inspected a total of 12 tongue surface images of men and women aged 20 or older, and the accuracy rate was approximately 58%.

[0040] In this way, when a trained model 34 is generated by deep learning using training data 32 containing tongue surface images and information regarding the presence or absence of acute appendicitis, and this trained model 34 is used to perform calculations regarding the presence or absence of acute appendicitis based on tongue surface images, it has been found that a higher accuracy rate can be obtained than when an expert visually diagnoses the presence or absence of acute appendicitis by visually inspecting the subject's tongue surface image. [Explanation of symbols]

[0041] 10 Diagnostic Systems 20 Control Unit 22 Model Generation Method 24 Reception methods 26 Calculation means 28 Output Method 30 Storage section 32 Training data 34 trained models 36 Programs 50 Display 52 Operation section 54 Communications Department

Claims

1. A program that causes a computer to function as a model generation means, a reception means, and a calculation means, the model generation means uses training data including a tongue surface image and information regarding the presence or absence of the onset of acute appendicitis to generate a trained model by deep learning, which inputs the tongue surface image and outputs information regarding the presence or absence of the onset of acute appendicitis; the receiving means receives a tongue surface image of a subject to be diagnosed for the presence or absence of acute appendicitis; The calculation means is a program that performs calculations to generate information regarding the presence or absence of the onset of acute appendicitis when a tongue surface image accepted by the acceptance means is input, based on the trained model generated by the model generation means.

2. 2. The program according to claim 1, wherein the calculation means calculates at least one of a probability of acute appendicitis occurring and a probability of acute appendicitis not occurring when calculating whether acute appendicitis has occurred.

3. 3. The program according to claim 1, wherein the calculation means generates a heat map for the tongue surface image accepted by the acceptance means based on the trained model generated by the model generation means, and performs calculations regarding the presence or absence of acute appendicitis based on a superimposed image generated by overlaying the heat map on the tongue surface image accepted by the acceptance means.

4. The program according to claim 1 , wherein the model generation means generates the trained model by deep learning by adding the training data to an existing model.

5. The teacher data includes a training set and at least one of a validation set and a test set; 5. The program according to claim 1, wherein the model generation means verifies the trained model generated using the training set using at least one of the validation set and the test set.

6. The program according to claim 1 , wherein the training data includes a tongue surface image received by the receiving means and a calculation result by the calculating means.

7. A storage medium storing a program that causes a computer to function as a model generation means, a reception means, and a calculation means, the model generation means uses training data including a tongue surface image and information regarding the presence or absence of the onset of acute appendicitis to generate a trained model by deep learning, which inputs the tongue surface image and outputs information regarding the presence or absence of the onset of acute appendicitis; the receiving means receives a tongue surface image of a subject to be diagnosed for the presence or absence of acute appendicitis; The calculation means performs calculations to generate information regarding the presence or absence of the onset of acute appendicitis when a tongue surface image accepted by the acceptance means is input based on the trained model generated by the model generation means.

8. a model generation means for generating a trained model by deep learning using training data including a tongue surface image and information on the presence or absence of the onset of acute appendicitis, and inputting the tongue surface image and outputting information on the presence or absence of the onset of acute appendicitis; a receiving means for receiving a tongue surface image of a subject to be diagnosed for the presence or absence of acute appendicitis; a computing means for performing a calculation to generate information regarding the presence or absence of the onset of acute appendicitis when the tongue surface image received by the receiving means is input based on the trained model generated by the model generating means; A system equipped with

9. A determination method performed by a computer, comprising: a step of generating a trained model by deep learning using training data including a tongue surface image and information regarding the presence or absence of the onset of acute appendicitis, the trained model inputting the tongue surface image and outputting information regarding the presence or absence of the onset of acute appendicitis; receiving a tongue surface image of a subject to be diagnosed for the presence or absence of acute appendicitis; performing a calculation to generate information regarding the presence or absence of the onset of acute appendicitis when the accepted tongue surface image is input based on the generated trained model; The determination method includes:

Images