Eardrum classification apparatus, eardrum classification system, and eardrum classification program

Abstract

To classify eardrum pathologies with a higher correct answer rate in comparison to conventional techniques.SOLUTION: An eardrum classification apparatus 30 includes: a communication unit that receives input of image data including an eardrum; a first classification unit 31b that extracts features from the input image data by using an algorithm and classifies the eardrum as having acute otitis media or something other than acute otitis media; and a second classification unit 31c that extracts features, by using an algorithm, from the image data classified as having something other than acute otitis media, and classifies the eardrum as being normal or having exudative otitis media. On the basis of the classification results given by the first classification unit 31b and the second classification unit 31c, the communication unit provides an output indicating acute otitis media, normal eardrum, or exudative otitis media.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an apparatus for classifying the presence or absence of a pathological condition of the eardrum from an image of the eardrum.

Background Art

[0002] Conventionally, for example, Japanese Patent Application Laid-Open No. 2019-534723 (Patent Document 1) is known as a method for classifying the pathological condition of the eardrum from an image of the eardrum. The conventional method determines whether the eardrum is normal or abnormal based on an image of the state of the eardrum.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the inventor has found that there are still points to be improved in the conventional method as described above. That is, it is difficult to make an accurate diagnosis, such as misjudging a normal eardrum as abnormal, or misjudging an abnormal eardrum as normal, or misjudging whether otitis media is acute or exudative, and there is room for improvement in the correct answer rate.

[0005] In addition, there are many needs to receive a diagnosis of the eardrum at midnight when hospitals and clinics are not accepting consultations, on Sundays and holidays. In particular, parents who are busy balancing work and child-rearing often fail to notice abnormalities deep in the earholes of infants and children, worsening the inflammation around the eardrum, and often wish to receive consultations at midnight, on Sundays and holidays.

[0006] In addition, in remote areas, medical services are not sufficient, and it is difficult to receive consultations at hospitals and clinics due to the absence of pediatricians and otolaryngologists.

[0007] On the one hand, early detection of otitis media is essential for appropriate treatment. Delayed diagnosis leads to delayed appropriate treatment.

[0008] In view of the above situation, an object of the present invention is to provide a technology that can simply and correctly determine the condition of the area including the eardrum at an appropriate timing even when medical services such as hospitals and clinics are limited.

Means for Solving the Problems

[0009] For this purpose, the eardrum classification device according to the present invention includes a communication unit to which image data including the eardrum is input, a first classification unit that extracts characteristics from the image data by an algorithm and classifies whether it is acute otitis media or other than acute otitis media regarding the eardrum, and a second classification unit that extracts characteristics from the image data classified as other than acute otitis media by an algorithm and classifies whether it is a normal eardrum or otitis media with effusion regarding the eardrum. Based on the classification results of the first and second classification units, acute otitis media, normal eardrum, or otitis media with effusion is output from the communication unit.

[0010] According to the present invention, since two-stage two-classification is performed, a classification result with a higher correct answer rate than before can be obtained. Therefore, even if it is not possible to visit a medical institution, support equivalent to a medical examination can be received, and the symptoms can be known at an appropriate timing. The eardrum classification device of the present invention is not particularly limited in terms of hardware configuration and software configuration. The eardrum classification device can be, for example, a server on a network or a cloud.

[0011] Highlights or foreign objects reflected in the image have an adverse effect on classification. When photographing the eardrum, it is difficult for ordinary people who are not used to taking pictures to adjust the amount of light emission required for shooting or to photograph the eardrum so that highlights or foreign objects are not reflected. Therefore, as one aspect of the present invention, a trimming unit is provided that trims the eardrum from the image data input to the communication unit and provides it to the first classification unit. According to such an aspect, it is possible to remove the reflection of highlights and foreign objects from the image to be classified, so the correct answer rate of classification is improved. In addition, machine learning can be effectively performed. As another aspect of the present invention, the eardrum may be trimmed before being input to the communication unit, such as when photographing the eardrum or when transmitting image data to the communication unit, and the trimmed image data may be input to the communication unit of the eardrum classification device.

[0012] As a preferred aspect of the present invention, a trimming unit is provided that trims a predetermined portion of the eardrum from the image data input to the communication unit and provides it to the first classification unit. According to such an aspect, since at least one of the first classification unit and the second classification unit focuses on a predetermined portion of the eardrum rather than the entire eardrum, the correct answer rate of classification can be improved or machine learning can be effectively performed. For example, by trimming the lower half of the eardrum, cases with prominent lesions in the lower half, such as exudative otitis media, can be correctly classified, and furthermore, machine learning can be effectively performed. Alternatively, for example, the central part of the eardrum is trimmed.

[0013] The eardrum classification device of the present invention is equipped with artificial intelligence (AI). The AI model of the AI algorithm is not particularly limited. As a preferred aspect of the present invention, at least one of the first classification unit and the second classification unit takes in labeled teacher data and performs supervised machine learning. According to such an aspect, the correct answer rate is increasingly improved as the eardrum classification device accumulates machine learning.

[0014] The tympanic membrane classification system of the present invention includes the above-described tympanic membrane classification device, an ear scope for photographing the tympanic membrane, and a terminal that receives the image data of the tympanic membrane from the ear scope and outputs it to the communication unit of the tympanic membrane classification device. According to such an aspect, in an ordinary household lacking medical skills, by operating the ear scope and the terminal at home, the symptoms of the tympanic membrane can be known. The terminal can, for example, simply display the classification result or optionally display information related to the classification result. The ear scope only needs to be able to photograph the innermost part of the ear canal, and is not limited to medical devices, as long as it can handle a considerable resolution (DPI) for the photographing unit and the image. Preferably, the ear scope preferably has a photographing unit capable of photographing an image with 4 million or more pixels. More preferably, the ear scope has a photographing unit capable of photographing an image with 8 million or more pixels. The device and hardware configuration of the terminal are not particularly limited.

[0015] As an aspect of the present invention, the terminal has a screen for displaying the image photographed by the ear scope and a guiding means for guiding the tympanic membrane to be displayed at the center of the screen on the screen, and outputs the image data with the tympanic membrane photographed at the center to the tympanic membrane classification device. According to such an aspect, since the terminal has the guiding means, even a user lacking medical skills can easily photograph the tympanic membrane image D´. The guiding means is not particularly limited, but is, for example, a circular frame displayed at the center of the terminal screen.

[0016] As an aspect of the present invention, the ear scope is a digital camera provided in an ear canal cleaning structure. According to such an aspect, with an ear canal cleaning instrument equipped with a digital camera, the symptoms of the tympanic membrane can be known.

[0017] As a preferred aspect of the present invention, the tympanic membrane classification system includes a terminal for medical institutions that can communicate with the tympanic membrane classification device and the terminal. According to such an aspect, support substantially equivalent to a medical examination can be received from a medical institution.

[0018] Glare and foreign objects reflected in the image have an adverse effect on classification. When photographing the eardrum, it is difficult for ordinary people who are not used to taking pictures to adjust the amount of light emission required for photographing or to photograph the eardrum so that glare and foreign objects are not reflected. Therefore, as a preferred aspect of the present invention, the ear scope or the terminal has a trimming unit that trims the eardrum from an image including the eardrum to create image data. According to such an aspect, it is possible to remove the reflection of glare and foreign objects from the image to be classified, so that the correct classification rate is improved. In addition, machine learning can be effectively performed. From the perspective of the eardrum classification device, trimming is executed on the ear scope side. Alternatively, in another aspect, trimming may be executed on the eardrum classification device side.

[0019] As a preferred aspect of the present invention, the guiding means displays on the screen whether the image displayed on the screen is suitable for classification. According to such an aspect, even a user who is not used to taking pictures can take an eardrum image.

[0020] As a preferred aspect of the present invention, the terminal displays on the screen the probabilities regarding the classification results of the first and second classification units. According to such an aspect, it is possible to alert the user. As another aspect, a threshold may be set for the probability, and if it is equal to or higher than the threshold, a notification prompting the user to go to a hospital or a clinic may be displayed on the screen.

[0021] Each operation (process) in the classification method of the present invention can also be realized in the form of a computer-executable program stored in various machine-readable storage media. The eardrum classification program of the present invention is a program that causes a computer to classify image data including the eardrum. From the image data, characteristics are extracted by an algorithm, and a first classification step of classifying whether the eardrum is related to acute otitis media or other than acute otitis media, and from the image data classified as other than acute otitis media, characteristics are extracted by an algorithm, and a second classification step of classifying whether the eardrum is a normal eardrum or otitis media with effusion, and based on the classification results of the first and second classification steps, a step of outputting acute otitis media, normal eardrum, or otitis media with effusion is executed by the computer.

[0022] The present invention includes program code for causing a computer to perform classification, a program product readable by a computer, and a storage medium storing the program code. The storage medium may include, for example, but is not limited to, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. The storage medium storing the program code is provided directly or indirectly to a system or device, and a computer, ROM, RAM, or central processing unit (CPU) of the system or device reads and executes the above-described program code. Embodiments of the present invention are not limited to programs and can be implemented by other software and firmware. The present invention may be an apparatus or a method.

Effects of the Invention

[0023] Thus, according to the present invention, without the need to visit a medical institution, the condition of the eardrum can be known with a high correct answer rate, so that the condition of the area including the eardrum can be grasped at an appropriate timing regardless of time and place, and a missed diagnosis can be prevented. In addition, regional differences in medical services are eliminated. Furthermore, by being used in the medical field, the efficiency of diagnosis is improved, contributing to labor saving at the site.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0025] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a schematic overall view showing an eardrum classification system according to an embodiment of the present invention. The eardrum classification system includes an ear scope 10, a terminal 20, and an eardrum classification device 30. The ear scope 10 has a rod-shaped main body 11, an ear canal cleaning structure 12 provided at the tip of the main body, and a light emitting unit 14 and a photographing unit 15 attached to the ear canal cleaning structure 12. The photographing unit is, for example, a digital camera.

[0026] The terminal 20 has a screen 21 and can communicate with the ear scope 10. An image B taken by the photographing unit 15 is projected onto the screen 21. The image B includes a circular frame C as an example.

[0027] The user of the ear scope 10 holds the main body 11 by hand, inserts the ear canal cleaning structure 12 into the external auditory canal H of the subject, for example, a child, and removes earwax on the inner peripheral surface of the external auditory canal H. Alternatively, the user causes the light emitting unit 14 to emit light, photographs the inside of the external auditory canal H with the photographing unit 15, and projects the inside of the external auditory canal H onto the screen 21.

[0028] The user may move the ear scope 10 while checking the screen 21 so that the image of the eardrum D (hereinafter referred to as the eardrum image D´) comes to the center of the frame C. The frame C serves as a mark to guide the user to take a picture of the eardrum image D´. The image B projected by the imaging unit 15 onto the screen 21 changes moment by moment. While moving the ear scope 10 while checking the eardrum image D´ included in the image B, when the eardrum image D´ is projected large at the center of the frame C, the user may select and capture an eardrum image D´ with less (highlight F) whiteening of the image due to light reflection, glare, and color bleeding while avoiding image blurring and camera shake. Note that the user is not particularly limited, does not need to be proficient in photography techniques, does not need to be a medical professional, and is an ordinary person.

[0029] The terminal 20 can communicate with an external eardrum classification device 30 (specifically, the communication unit 33) via wireless communication or wired communication, for example, network means NW such as the Internet. The terminal 20 can be substituted with a widely popular mobile phone terminal, for example, a smartphone, a tablet terminal, or a terminal equipped with other communication functions such as Wi-fi (registered trademark) and bluetooth (registered trademark). By submitting the captured image B (strictly speaking, the image data of the image B) to the eardrum classification device 30, the user can have the eardrum classification device 30 classify whether the eardrum D is a normal eardrum or affected by acute or exudative otitis media. Such classification can be rephrased as judgment or diagnosis.

[0030] The tympanic membrane classification device 30 includes a classification unit 31 having an artificial intelligence (AI) that drives an original algorithm, a memory unit 32 that stores a large number of image data (synonymous with images), and a communication unit 33 that communicates with the outside. The tympanic membrane classification device 30 of the present embodiment is a server connected to the Internet. The memory unit 32 stores a large number of image data of normal tympanic membranes labeled as normal tympanic membranes. Also, a large number of image data of acute otitis media labeled as acute otitis media are stored. Also, a large number of image data of secretory otitis media labeled as secretory otitis media are stored. These data are used as labeled teacher data for the machine learning of the classification unit 31. The classification unit 31 performs machine learning using a plurality of teacher data as teaching materials to improve the correct classification rate. Note that the correct classification rate is understood to be synonymous with the result of scoring and accumulating the correctness of the classification results of the classification unit 31 by an external institution different from the tympanic membrane classification device 30, for example, a human doctor.

[0031] When the classification unit 31 is input with an image (synonymous with image data, the same hereinafter) including the tympanic membrane image D´, it extracts characteristics for classification from the tympanic membrane image D´, processes the characteristics by an algorithm, classifies whether the tympanic membrane image D´ is a normal tympanic membrane, acute otitis media, or secretory otitis media, and outputs the classification result. The characteristics refer to the color distribution of the tympanic membrane image D´, the size of the shape of the distribution, etc., and are defined inside the classification unit 31 according to the algorithm.

[0032] The communication unit 33 inputs image data from the outside via wireless communication or wired communication such as network means NW and outputs the image data to the classification unit 31. Also, the communication unit 33 can communicate with the computer terminal of the medical institution 40 via the same communication means. Alternatively, the tympanic membrane classification device 30 may be a computer installed in a medical institution. In any case, labeled teacher data is gradually accumulated in the memory unit 32. As a result, the learning amount of the machine learning of the classification unit 31 increases, and the correct classification rate improves. Also, the tympanic membrane image D´ newly input from the terminal 20 to the tympanic membrane classification device 30 is later labeled by a doctor of the medical institution 40. As a result, teacher data is gradually accumulated in the memory unit 32.

[0033] The classification unit 31 has a trimming unit, a first classification unit, and a second classification unit. In the present embodiment, as shown in FIG. 3, the trimming unit in the classification unit 31 trims the eardrum image D' from the image B in (a) by an algorithm. As a result, the brilliance F away from the eardrum image D' reflected in the image B, the color of the inner peripheral surface of the external auditory canal H, and foreign matters G such as body hair and earwax do not affect the classification. (b) represents the image data related to the eardrum image D' after trimming.

[0034] FIG. 4 is a schematic diagram showing the algorithms sequentially executed by the first classification unit and the second classification unit in the classification unit 31. Such algorithms are executed by ResNet (Residual Network), VGG16 (Visual Geometry Group 16-layer), DenseNet (Densely Connected Convolutional Network), or other AI models.

[0035] First, the first classification unit 31b in the classification unit 31 classifies the eardrum image D' into two categories. Specifically, it classifies whether the eardrum image D' is acute otitis media or other than acute otitis media. This two-category classification is determined based on the ratio of red pixels among all the pixels of the eardrum image D', the size of the area of brown pixels (also called turbidity), and other characteristics.

[0036] Second, the second classification unit 31c in the classification unit 31 classifies the eardrum image D' other than acute otitis media into two categories. Specifically, it classifies whether the eardrum image D' other than acute otitis media is a normal eardrum or otitis media with effusion. This two-category classification is determined based on the ratio of brown pixels among all the pixels and other characteristics. Through such two-stage classification, the eardrum image D' is classified into three categories with a higher correct answer rate than before.

[0037] FIG. 2 is a diagram illustrating the classification results executed by the eardrum classification device of the present embodiment. The classification results output by the eardrum classification device 30 are transmitted to the terminal 20 via the network means NW and displayed on the screen 21. Also, the probability [%] calculated by the eardrum classification device 30 for each classification result is displayed. The probability [%] here is a numerical value calculated by the classification unit 31.

[0038] The terminal 20 can communicate with the medical institution 40 via wireless communication or wired communication such as the network means NW. The user of the terminal 20 can question and answer with the staff of the medical institution 40 about the classification results of the eardrum image D´, the validity of the classification, future diagnoses and measures, etc.

[0039] When the classification result is a normal eardrum, the probability X [%] of being a normal eardrum is also displayed together with the classification result. When the classification result is acute otitis media, the probability Y [%] of being acute otitis media is also displayed together with the classification result. When the classification result is secretory otitis media, the probability Z [%] of being secretory otitis media is also displayed together with the classification result. The probabilities X, Y, and Z [%] are based on the previous classification results executed by the classification unit 31 and the verification of the labeling performed by the doctor for the classification results, in other words, the correct answer rate.

[0040] Here, it should be noted that in FIG. 2, numerical values or degrees such as changing the periphery of image B, the color of the background surrounding image B, changing the thickness or color of the characters, or standardizing indicators such as scales of different sizes, or displaying symbols and markers together may be used to issue warnings. For example, for the image B classified as normal in FIG. 2(a), image B is outlined in white. Also, for the image B classified as acute otitis media in FIG. 2(b), instead of displaying the probability Y [%], when acute otitis media is suspected (probability Y [%]<50 [%]), image B is outlined in a light color such as pink. Also, when acute otitis media is almost certain (probability Y [%]≧50 [%]), image B is outlined in a dark color such as bright red. The same applies to the exudative otitis media in FIG. 2(c). Thus, it is possible to surely inform the viewer of the screen 21 that the symptoms of the eardrum D are urgent. Note that the probabilities X, Y, Z [%] may be displayed together with the indicators and markers.

[0041] The correct answer rates of the eardrum classification device 30 of the present embodiment and the eardrum classification device of the proportionality were investigated. The results of the correct answer rates of these eardrum classification devices are shown in Table 1.

[0042]

Table 1

[0043] First, a plurality of pieces (for a plurality of people and the number of ears) of teacher data were machine-learned. Next, a plurality of eardrum images D´ were classified by the eardrum classification device 30 (this time). Further, a plurality of pieces (for a plurality of people and the number of ears) of teacher data were machine-learned. Next, a plurality of eardrum images D´ were classified by the eardrum classification device 30 (the second time). If only the classification of the first time (this time) is executed without executing the classification of the second time, the correct answer rate of the second time is not displayed.

[0044] The accuracy rates of the classification results of this embodiment are shown in Nos. 3 and 4 of Table 1. According to the two-stage binary classification of this embodiment, it was found that higher accuracy rates than before could be obtained, such as 99.98% in the first stage and 99.50% in the second stage. This is presumably because the eardrum D of acute otitis media is redder than the eardrums D other than acute otitis media, and the eardrum D of secretory otitis media has a more prominent turbidity area than the eardrums D other than secretory otitis media.

[0045] The accuracy rate of the proportional ratio was also investigated. The classification process of the proportional ratio is shown in FIG. 5.

[0046] The eardrum classification device for the proportional ratio is equipped with an algorithm for one-stage multi-classification (three-classification) that classifies the input eardrum image into a normal eardrum, acute otitis media, or secretory otitis media in one stage. According to the one-stage three-classification of the proportional ratio, as shown in No. 1 of Table 1, the accuracy rates were 68.26% in the first time (this time) and 91.12% in the second time. Thus, the accuracy rate (No. 1) of the eardrum classification device for the proportional ratio was inferior to the accuracy rates (Nos. 3 and 4) of the eardrum classification device of this embodiment. Thereby, it was found that the method of performing binary classification in multiple stages as in this embodiment improves the accuracy rate compared to before.

[0047] Next, a modified example of this embodiment will be described. The eardrum classification device 30 of the modified example trims approximately the lower half of the eardrum D as shown in FIG. 3. Then, the trimmed semi-circular image D´´ is classified as described above. Secretory otitis media exhibits a prominent turbidity area in the lower half of the eardrum D. Therefore, by trimming approximately the lower half of the eardrum D (for example, the lower half of the central region of the eardrum D), the accuracy rate of the classification result is improved. Also, by labeling the image of the lower half of the eardrum D and making the classification unit 31 perform machine learning as teacher data, the accuracy rate of the classification unit 31 itself can be improved.

[0048] As shown in No. 5, the accuracy rate of the modified example of this embodiment is 96.07% in the first time (this time), which is superior to the case of the eardrum image D´ (No. 4). Also, it is 100% in the second time, and the accuracy rate has improved.

[0049] As shown in No.2, the correct answer rate of the modification example of the comparative example was 62.41[%] in the first time (this time), which was rather inferior to the case of the eardrum image D´ (No.1). Also, it was 65.74[%] in the second time. Although the correct answer rate improved, it did not exceed the correct answer rate (No.5) of the present embodiment.

[0050] Referring to FIG. 1, the eardrum classification device 30 of the present embodiment includes a communication unit 33 to which an eardrum image D´ including the eardrum D is input from the outside, a first classification unit 31b (FIG. 3) that extracts characteristics from the eardrum image D´ by an algorithm and classifies whether the eardrum D is acute otitis media or other than acute otitis media, and a second classification unit 31c that extracts characteristics from the eardrum image D´ classified as other than acute otitis media by the first classification unit 31b by an algorithm and classifies whether the eardrum D is normal or otitis media with effusion. Based on the classification results of these first classification unit 31b and second classification unit 31c, acute otitis media or normal or otitis media with effusion is output from the communication unit 33 to the outside. According to such an eardrum classification device 30, the correct answer rate of classification is improved compared to the conventional case by the two-stage two-classification algorithm shown in FIG. 4.

[0051] The operation and effect of such an eardrum classification device 30 will be described for each of the following phases.

[0052] In the first phase, the eardrum classification device 30 classifies the eardrum D by the above-described unique algorithm of AI. As a result, it is possible to have the symptoms of the eardrum D judged without directly visiting a medical institution 40 to receive a diagnosis from an otolaryngologist. For example, in an ordinary household not proficient in medical technology, a family takes a picture of a child's eardrum D and uses it for disease determination of the eardrum D.

[0053] In the second phase, the medical institution 40 is equipped with the ear scope 10 and the terminal 20 as instruments used for examinations, and is interconnected with the eardrum classification device 30. As a result, the eardrum classification device 30 is utilized by an otolaryngologist at the medical institution 40 as a diagnostic support tool for diagnosing the eardrum image D' captured by the ear scope 10. Also, the image B is used by the otolaryngologist. Alternatively, for example, in an ordinary household not proficient in medical technology, a family member can take a picture of a child's eardrum D and receive support equivalent to an examination, such as advice on seeing a doctor from an otolaryngologist at the medical institution 40, and can be aware of symptoms at an appropriate timing.

[0054] In the third phase, the eardrum classification device 30 in the first and second phases described above is used either as a diagnostic support tool or as a medical device. By accumulating AI machine learning, the eardrum classification device 30 can classify at the same level as an otolaryngologist. For example, a prescription program for medicine is incorporated into an AI algorithm to enable the prescription of prescription drugs based on the diagnostic support tool of this embodiment. For example, the eardrum classification device 30 is interconnected with a pharmacy (computer) not shown in the figure.

[0055] Also, referring to FIGS. 3(a) and 3(b), the eardrum classification device 30 of this embodiment includes a trimming unit that trims the image portion of the eardrum D from the image B input to the communication unit 33 and provides it to the first classification unit 31b. When the light emitting unit 14 (FIG. 1) illuminates the ear canal H, a bright light F or a foreign object G is likely to be reflected on the exit side of the ear canal H. According to this embodiment, since the image on the exit side of the ear canal H is removed, the bright light F and the foreign object G are also removed from the image. Therefore, machine learning can be effectively performed.

[0056] Also, referring to FIGS. 3(a) and 3(c), the eardrum classification device 30 of a modified example of this embodiment includes a trimming unit that trims the lower half of the image of the eardrum D from the image B input to the communication unit 33 and provides it to the first classification unit 31b. As a result, cases with prominent lower half lesions such as exudative otitis media can be correctly classified, and furthermore, machine learning can be effectively performed.

[0057] Also, the eardrum classification device 30 of the present embodiment stores the image B to be classified in the memory unit 32, and later, it is correctly labeled by the medical institution 40. And at least one of the first classification unit 31b and the second classification unit 31c takes in the labeled teacher data and performs machine learning with a teacher. As a result, the eardrum classification device 30 accumulates machine learning and the correct answer rate is continuously improved.

[0058] Also, the eardrum classification system of the present embodiment includes the eardrum classification device 30, an ear scope 10 for photographing the eardrum D, and a terminal 20 that receives the image B of the eardrum D from the ear scope 10 and outputs it to the communication unit 33 of the eardrum classification device 30. Thereby, in an ordinary household that is not proficient in medical techniques, by operating the ear scope 10 and the terminal 20, the symptoms of the eardrum D can be known.

[0059] Also, the terminal 20 of the present embodiment includes a screen 21 for displaying the image photographed by the ear scope 10, and displays a frame C that guides the eardrum D to be displayed at the center of the screen on the screen 21, and inputs the image B in which the eardrum D is photographed into the communication unit 33 within the frame C at the center. Thereby, even a user who is not proficient in medical techniques can appropriately photograph the eardrum image D'.

[0060] Also, the photographing unit 15 of the present embodiment is a digital camera provided in the ear canal cleaning structure 12. Thereby, by using the digital camera mounted on the ear canal cleaning structure as a daily necessity for regular use, the symptoms of the eardrum can be known.

[0061] Also, since the communication unit 33 and the terminal 20 of the present embodiment can communicate with the medical institution 40, even if they cannot visit a medical institution, they can receive support equivalent to a diagnosis and know the symptoms at an appropriate timing.

[0062] Next, a supplementary explanation will be given regarding the screen 21 provided on the user's terminal 20. The terminal 20 communicates with the ear scope 10 and projects the image B photographed by the ear scope 10 onto the screen 21 in real time.

[0063] FIG. 6 is a diagram illustrating a terminal screen of the present embodiment, and represents a screen 21 as a guiding means. An image B of the external auditory canal H taken when the user inserts the earscope 10 into the ear of the subject is output in real time from the terminal 20 to the eardrum classification device 30 via the network NW (FIG. 1). The classification unit 31 of the eardrum classification device 30 includes a program (imaging guidance program) as a guiding means for determining in real time whether the image B includes the entire eardrum D and can be used for eardrum classification. Specifically, it determines whether or not to include the eardrum image D' shown in FIG. 3 described above. When the earscope 10 starts to be inserted into the ear of the subject and the image B does not include the eardrum image D' and the image B is not available for eardrum classification, a blue ring 22 is displayed on the screen 21. The ring 22 has a radius that is half of the frame C of the image B. Also, in the screen 21, the outer region J of the frame C has a plain color such as light gray.

[0064] In the imaging guidance program into which the image B is input, when it is determined that the region surrounded by the blue circular ring 22 displays the eardrum D, the imaging guidance program determines that the image B is available for eardrum classification and converts the color of the ring 22 from blue to green. Then, a determination operation button 23 is displayed in the outer region J. Thereby, the appropriately taken eardrum image is outlined by the ring 22.

[0065] According to the imaging guidance program of the present embodiment, it is possible to assist the user who is not used to taking pictures of the eardrum D in taking pictures of the eardrum D. Note that the size and circular shape of the ring 22 described above, and its color are merely examples, and any visual change display indicating whether or not it can be used for eardrum classification may be used.

[0066] When the user touches the determination operation button 23 with a finger, the terminal 20 outputs a classification command to the communication unit 33 of the eardrum classification device 30. The eardrum classification device 30 classifies the image data input from the terminal 20 in two stages and two classifications, and displays the classification result on the screen 21.

[0067] FIG. 7 is a diagram illustrating the probability [%] as each classification result in the two-stage two-classification in the classification unit 31.

[0068] FIG. 8 is a diagram illustrating the classification result displayed on the screen 21.

[0069] More specifically, the binary classification by the first classification unit 31b shown in FIG. 4 described above calculates the probability (numerical value) [%] of each of the two classifications as shown in FIG. 7 by an algorithm. The same applies to the second classification unit 31c.

[0070] Exemplarily, when the classification result of the first classification unit 31b is 10 [%] for acute otitis media and 90 [%] for not acute otitis media, it further proceeds to the second classification unit 31c, and the classification result is 70 [%] for normal and 30 [%] for otitis media with effusion, and the result is output. As shown in Table 2, such a result apportions the 90 [%] for not acute otitis media in the classification result of the first classification unit 31b according to the classification result of the first classification unit 31b, and as the probability of the final classification result, the probability of normal X = 63 [%], the probability of acute otitis media Y = 10 [%], and the probability of otitis media with effusion Z = 27 [%] are calculated. The screen 21 as the classification result display unit indexes and displays the magnitudes of these probabilities with indicators for each of the three classifications so that the probabilities X, Y, Z [%] total 100 [ %].

[0071]

Table 2

[0072] Furthermore, as shown in FIG. 8, the screen 21 as the classification result display unit indexes and displays the probability [%] for each classification with three different colored indicators in the circular frame C surrounding the image B.

[0073] In the tympanic membrane image D' of a normal tympanic membrane exemplified in FIG. 8, in the circular ring of the frame C that displays the classification result of the tympanic membrane image D', the blue indicator Cx (63 [%]) representing normal is displayed at the highest ratio, the orange indicator Cz (27 [%]) representing otitis media with effusion is displayed at a lower ratio, and the red indicator Cy (10 [%]) representing acute otitis media is displayed slightly. The indicators Cx, Cy, Cz can be appropriately rearranged in the order of the magnitudes of the probabilities.

[0074] According to this embodiment, since the classification result is displayed by an indicator in the frame C of the screen 21 and the image B is bordered by the indicators Cx, Cy, and Cz, it is possible to draw the user's attention.

[0075] Note that the above-described three different colors are examples, and each color can be appropriately changed or selected. Optionally, the date and time regarding the image used for classification and the identification of the left and right ears may be displayed.

[0076] As described above, the embodiments of the present invention have been described with reference to the drawings. However, the present invention is not limited to the illustrated embodiments. Various modifications and variations can be made within the same scope or an equivalent scope as the present invention with respect to the illustrated embodiments.

[0077] As a modification not shown in the drawings, instead of the configuration in which the tympanic membrane classification device 30 and the terminal 20 are communicatively connected by the network means NW shown in FIG. 1, the tympanic membrane classification device 30 shown in FIG. 1 may be a program installed in the terminal 20. The program is, for example, application software stored in the terminal 20, and executes the above-described algorithm processing within the terminal 20.

[0078] As a modification not shown in the drawings, various options can be added to and substituted for the screen 21 of the terminal 20 as a classification result display unit. Various displays such as the above-described color coding and shapes may be by sound or light.

Industrial Applicability

[0079] The present invention is advantageously used in the medical industry.

Explanation of Signs

[0080] 10 Earscope, 11 Main body, 12 Ear canal cleaning structure, 14 Light emitting unit, 15 Photographing unit, 20 Terminal, 21 Screen, 22 Annular ring (inductive means), 30 Tympanic membrane classification device, 31 (first and second) classification units, 32 memory unit, 33 communication unit, 40 medical institution, B image, C frame (inductive means), Indicators of classification results of Cx, Cy, Cz, D eardrum, D' eardrum image, H external auditory canal.

Claims

1. a communication unit to which image data including the eardrum is input; a first classification unit that extracts characteristics from the image data by an algorithm and classifies the eardrum as having acute otitis media or other than acute otitis media; a second classification unit that extracts characteristics from the image data classified as other than acute otitis media by an algorithm and classifies the tympanic membrane as a normal tympanic membrane or serous otitis media, The tympanic membrane classification device outputs acute otitis media, normal tympanic membrane, or serous otitis media from the communication unit based on the classification results of the first and second classification units.

2. The tympanic membrane classification device according to claim 1 , further comprising a trimming unit that trims the tympanic membrane from the image data input to the communication unit and provides the trimmed image to the first classification unit.

3. The tympanic membrane classification device according to claim 1 , further comprising a trimming unit that trims a predetermined portion of the tympanic membrane from the image data input to the communication unit and provides the trimmed portion to the first classification unit.

4. The eardrum classification device according to claim 1 , wherein at least one of the first classifying unit and the second classifying unit takes in labeled teacher data and performs supervised machine learning.

5. The eardrum classification device according to claim 1 ; An earscope to photograph the eardrum, A tympanic membrane classification system comprising: a terminal that receives image data of the tympanic membrane from the earscope and outputs the image data to the tympanic membrane classification device.

6. The tympanic membrane classification system according to claim 5, wherein the terminal has a screen for displaying an image captured by the earscope, and a guidance means for guiding the terminal so that the tympanic membrane is displayed in the center of the screen, and outputs image data in which the tympanic membrane is photographed in the center to the communication unit.

7. The tympanic membrane classification system of claim 5, wherein the ear scope is a digital camera mounted on an ear canal cleaning structure.

8. The tympanic membrane classification system according to claim 5 , further comprising a medical institution terminal capable of communicating with the tympanic membrane classification device and the terminal.

9. The tympanic membrane classification system according to claim 5 , wherein the earscope or the terminal has a trimming unit that creates the image data by trimming the tympanic membrane from an image including the tympanic membrane.

10. The tympanic membrane classification system according to claim 6 , wherein the guidance means displays on the screen whether or not the image displayed on the screen is suitable for the classification.

11. The tympanic membrane classification system according to claim 5 , wherein the terminal displays probabilities regarding the classification results of the first and second classification units on the screen.

12. A program for causing a computer to classify image data including an eardrum, a first classification step of extracting features from the image data by an algorithm and classifying the eardrum as having acute otitis media or other than acute otitis media; A second classification step of extracting features from the image data classified as other than acute otitis media by an algorithm and classifying the tympanic membrane as a normal tympanic membrane or serous otitis media; The tympanic membrane classification program causes the computer to execute a step of outputting acute otitis media, normal tympanic membrane, or serous otitis media based on the classification results of the first and second classification steps.

Images