Processing apparatus, processing program, processing method, and processing system

Abstract

The present invention provides a processing device and the like suitable for processing images obtained by photographing the inside of the oral cavity for use in diagnosing oral conditions. [Solution] A processing device is provided which includes at least one processor, the at least one processor being configured to acquire one or more judgment images of a subject via a camera for capturing an image of the subject including at least a portion of the user's oral cavity, to determine the possibility of contracting a predetermined disease based on a trained judgment model stored in memory for determining the possibility of contracting a predetermined disease and the acquired one or more judgment images, and to output information indicating the determined possibility of contracting the disease.

Description

[Technical Field]

[0001] The present disclosure relates to a processing apparatus, a processing program, a processing method, and a processing system for processing an image of a subject captured by a camera. [Background Art]

[0002] Conventionally, it has been known that a doctor observes changes in the condition of a user's oral cavity to make a diagnosis of, for example, a viral cold. Non-Patent Document 1 reports that lymphoid follicles appearing in the deepest part of the pharynx located in the oral cavity have a pattern specific to influenza. Lymphoid follicles having this specific pattern are called influenza follicles, which are signs characteristic of influenza and are said to appear approximately 2 hours after onset. However, such a pharyngeal region has been diagnosed by a doctor through direct visual inspection, and diagnosis using images has not been performed. [Prior Art Literature] [Non-Patent Literature]

[0003] [Non-Patent Document 1] Miyamoto · Watanabe, "Consideration on the Meaning and Value of Pharyngeal Examination Findings (Influenza Follicles)", Nihon Daigaku Igaku Zasshi (Journal of Nihon University Medical Association), 72(1):11-18(2013) [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] Therefore, based on the above-described technology, an object of the present disclosure is to provide, according to various embodiments, a processing apparatus, a processing program, a processing method, or a processing system for determining the possibility of contracting a predetermined disease using a determination image of a subject obtained by imaging a user's oral cavity. [Means for Solving the Problem]

[0005] According to one aspect of the present disclosure, a processing device is provided which includes at least one processor, the at least one processor configured to perform processing for acquiring one or more judgment images of a subject via a camera for capturing an image of the subject including at least a portion of the user's oral cavity, determining the possibility of contracting a predetermined disease based on a trained judgment model stored in memory for determining the possibility of contracting a predetermined disease and the acquired one or more judgment images, and outputting information indicating the determined possibility of contracting the disease.

[0006] According to one aspect of the present disclosure, a processing program is provided which, executed by at least one processor, acquires one or more judgment images of a subject via a camera for capturing images of the subject including at least a portion of the user's oral cavity, determines the possibility of contracting a predetermined disease based on a trained judgment model stored in memory for determining the possibility of contracting a predetermined disease, and the acquired one or more judgment images, and outputs information indicating the determined possibility of contracting the disease.

[0007] According to one aspect of the present disclosure, a processing method is provided which is performed by at least one processor and includes the steps of: acquiring one or more judgment images of a subject via a camera for taking an image of the subject including at least a portion of the user's oral cavity; a trained judgment model stored in memory for determining the possibility of contracting a predetermined disease; determining the possibility of contracting the predetermined disease based on the acquired one or more judgment images; and outputting information indicating the determined possibility of contracting the disease.

[0008] According to one aspect of the present disclosure, a processing system is provided that includes "a photographing device equipped with a camera for taking an image of a subject which includes at least a portion of the user's oral cavity, and the processing device described above which is connected to the photographing device via a wired or wireless network." [Effects of the Invention]

[0009] According to this disclosure, it is possible to provide a processing device, a processing program, a processing method, and a processing system suitable for processing images obtained by photographing the inside of the oral cavity for use in oral cavity diagnosis.

[0010] The effects described above are merely illustrative for the sake of explanation and are not limiting. In addition to, or in lieu of, any other effects described herein or that would be obvious to those skilled in the art may be achieved. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 shows the usage state of processing system 1 according to one embodiment of the present disclosure. [Figure 2] Figure 2 shows the usage state of processing system 1 according to one embodiment of the present disclosure. [Figure 3] Figure 3 is a schematic diagram of a processing system 1 according to one embodiment of the present disclosure. [Figure 4] Figure 4 is a block diagram showing the configuration of a processing system 1 according to one embodiment of the present disclosure. [Figure 5] Figure 5 is a schematic diagram showing the top view configuration of a photographing device 200 according to one embodiment of the present disclosure. [Figure 6] Figure 6 is a schematic diagram showing the cross-sectional configuration of a photographic device 200 according to one embodiment of the present disclosure. [Figure 7A] Figure 7A is a conceptual diagram showing an image management table stored in a processing device 100 according to one embodiment of the present disclosure. [Figure 7B] Figure 7B is a conceptual diagram showing a user table stored in a processing device 100 according to one embodiment of the present disclosure. [Figure 8] Figure 8 shows a processing sequence performed between a processing device 100 and an imaging device 200 according to one embodiment of the present disclosure. [Figure 9]FIG. 9 is a diagram showing a processing flow executed in a processing apparatus 100 according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a diagram showing a processing flow executed in an imaging apparatus 200 according to an embodiment of the present disclosure. [Figure 11] FIG. 11 is a diagram showing a processing flow executed in a processing apparatus 100 according to an embodiment of the present disclosure. [Figure 12] FIG. 12 is a diagram showing a processing flow executed in a processing apparatus 100 according to an embodiment of the present disclosure. [Figure 13] FIG. 13 is a diagram showing a processing flow related to generation of a trained model according to an embodiment of the present disclosure. [Figure 14] FIG. 14 is a diagram showing a processing flow executed in a processing apparatus 100 according to an embodiment of the present disclosure. [Figure 15] FIG. 15 is a diagram showing a processing flow related to generation of a trained model according to an embodiment of the present disclosure. [Figure 16] FIG. 16 is a diagram showing a processing flow related to generation of a trained model according to an embodiment of the present disclosure. [Figure 17] FIG. 17 is a diagram showing a processing flow related to generation of a trained model according to an embodiment of the present disclosure. [Figure 18] FIG. 18 is a diagram showing an example of a screen displayed on a processing apparatus 100 according to an embodiment of the present disclosure. [Figure 19] FIG. 19 is a diagram showing an example of a screen displayed on a processing apparatus 100 according to an embodiment of the present disclosure. [Figure 20] FIG. 20 is a schematic diagram of a processing system 1 according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF EMBODIMENTS

[0012] Various embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that common constituent elements in the drawings are denoted by the same reference numerals.

[0013] <First Embodiment> 1. Overview of Processing System 1 The processing system 1 described herein is primarily used to photograph the inside of a user's oral cavity and obtain subject images. In particular, the processing system 1 is used to photograph the area around the back of the throat, specifically the pharynx. Therefore, the following description will mainly focus on the case in which the processing system 1 described herein is used to photograph the pharynx. However, the pharynx is just one example of a photographic site, and naturally, the processing system 1 described herein can be suitably used for other parts of the oral cavity, such as the tonsils.

[0014] The processing system 1 described herein is used to determine the likelihood of a user contracting a predetermined disease from a subject image obtained by photographing a subject that includes at least the pharyngeal region of the user's oral cavity, and to diagnose or assist in the diagnosis of the predetermined disease. An example of a disease that can be determined by the processing system 1 is influenza. Normally, the likelihood of contracting influenza is diagnosed by examining the user's pharynx and tonsil region, or by determining the presence or absence of findings such as follicles in the pharyngeal region. However, by using the processing system 1 to determine the likelihood of contracting influenza and outputting the result, it becomes possible to diagnose or assist in the diagnosis. Note that the determination of the likelihood of contracting influenza is just one example. The processing system 1 can be suitably used to determine any disease in which a difference in oral findings appears when the user is infected. Note that the difference in findings is not limited to those discovered by a physician or other medical professional and whose existence is medically known. For example, any difference that can be recognized by a person other than a physician, or any difference that can be detected by artificial intelligence or image recognition technology, can be suitably applied to the processing system 1. Examples of such diseases include, in addition to influenza, infections such as streptococcal infection, adenovirus infection, EB virus infection, mycoplasma infection, hand-foot-and-mouth disease, herpangina, and candidiasis; diseases that present with vascular disorders or mucosal disorders such as arteriosclerosis, diabetes, and hypertension; and tumors such as tongue cancer and pharyngeal cancer.

[0015] In this disclosure, terms such as "judgment" and "diagnosis" are used in relation to diseases, but these do not necessarily mean a definitive judgment or diagnosis by a physician. For example, it is certainly possible that the user themselves or an operator other than a physician may use the processing system 1 of this disclosure, and the processing device 100 included in the processing system 1 may make a judgment or diagnosis.

[0016] Furthermore, in this disclosure, the users who are the subjects of imaging by the imaging device 200 may include any person, such as patients, subjects, diagnostic users, and healthy individuals. Also, in this disclosure, the operators who hold the imaging device 200 and perform imaging operations may include any person, such as the user themselves, not limited to medical professionals such as doctors, nurses, and laboratory technicians. The processing system 1 related to this disclosure is typically envisioned to be used in a medical institution. However, it is not limited to this case, and its use location may be any other place, such as the user's home, school, or workplace.

[0017] Furthermore, as stated above, in this disclosure, the subject only needs to include at least a portion of the user's oral cavity. The disease being assessed can also be any disease that shows a difference in oral findings. However, the following description will focus on the case where the subject includes the pharynx or surrounding area, and the disease being assessed is the possibility of contracting influenza.

[0018] Furthermore, in this disclosure, the subject image and judgment image may be one or more videos or one or more still images. As an example of operation, when the power button is pressed, the camera captures a through image, and the captured through image is displayed on the display 203. Then, when the operator presses the shooting button, one or more still images are captured by the camera and the captured images are displayed on the display 203. Alternatively, when the user presses the shooting button, video recording begins, and the images being captured by the camera during this time are displayed on the display 203. Then, when the shooting button is pressed again, video recording ends. In this way, in a series of operations, various images such as through images, still images, and videos are captured by the camera and displayed on the display, but the subject image does not mean only a specific image among these, but may include all images captured by the camera.

[0019] Figure 1 shows the state of use of a processing system 1 according to one embodiment of the present disclosure. According to Figure 1, the processing system 1 according to the present disclosure includes a processing device 100 and an imaging device 200. The operator attaches the auxiliary device 300 so as to cover the tip of the imaging device 200 and inserts the imaging device 200 together with the auxiliary device 300 into the user's oral cavity 710. Specifically, first, the operator (which may be the user 700 themselves or someone other than the user 700) attaches the auxiliary device 300 so as to cover the tip of the imaging device 200. Then, the operator inserts the imaging device 200 with the auxiliary device 300 attached into the oral cavity 710. At this time, the tip of the auxiliary device 300 is inserted past the incisors 711 to the vicinity of the soft palate 713. In other words, the imaging device 200 is also inserted to the vicinity of the soft palate 713. At this time, the tongue 714 is pushed downward by the auxiliary device 300 (which functions as a tongue depressor), and the movement of the tongue 714 is restricted. Furthermore, the tip of the auxiliary device 300 pushes the soft palate 713 upward. This allows the operator to secure a good field of view for the imaging device 200 and to take good images of the pharynx 715 located in front of the imaging device 200.

[0020] The captured subject image (typically an image including the pharynx 715) is transmitted from the imaging device 200 to the processing device 100, which is connected via a wired or wireless network. The processor of the processing device 100, upon receiving the subject image, processes a program stored in memory to select a judgment image to be used for the determination from the subject image, and also determines the likelihood of the subject suffering from a predetermined disease. The results are then output to a display or the like.

[0021] Figure 2 shows the state of use of a processing system 1 according to one embodiment of the present disclosure. Specifically, Figure 2 shows the state in which the operator 600 is holding the imaging device 200 of the processing system 1. According to Figure 2, the imaging device 200 consists of a main body 201, a grip 202, and a display 203, from the side inserted into the oral cavity. The main body 201 and grip 202 are formed in a substantially columnar shape of a predetermined length along the insertion direction H into the oral cavity. The display 203 is positioned on the opposite side of the grip 202 from the main body 201 side. Therefore, the imaging device 200 as a whole is formed in a substantially columnar shape and is held by the operator 600 in a manner similar to holding a pencil. In other words, in the state of use, the display panel of the display 203 faces the operator 600, making it possible to easily handle the imaging device 200 while checking the subject image captured by the imaging device 200 in real time.

[0022] Furthermore, when the operator 600 holds the grip 202 with the subject image displayed on the display 203 in the normal orientation, the shooting button 220 is positioned on the upper side of the grip. Therefore, when the operator 600 holds the grip, they can easily press the shooting button 220 with their index finger or the like.

[0023] 2. Configuration of Processing System 1 Figure 3 is a schematic diagram of a processing system 1 according to one embodiment of the present disclosure. According to Figure 3, the processing system 1 includes a processing device 100 and an imaging device 200 that is communicably connected to the processing device 100 via a wired or wireless network. The processing device 100 receives operation input from the operator and controls imaging by the imaging device 200. The processing device 100 also processes the subject image captured by the imaging device 200 to determine the possibility that the user has contracted influenza. Furthermore, the processing device 100 outputs the determined result and notifies the user, operator, or doctor of the result.

[0024] The imaging device 200 has its tip inserted into the user's oral cavity to photograph the inside of the oral cavity, particularly the pharynx. The specific imaging process will be described later. The captured subject image is transmitted to the processing device 100 via a wired or wireless network.

[0025] Furthermore, the processing system 1 may include a mounting platform 400 as needed. The mounting platform 400 can stably support the imaging device 200. In addition, the mounting platform 400 can be connected to a power supply via a wired cable, allowing power to be supplied from the power supply terminal of the mounting platform 400 to the imaging device 200 through the power supply port of the imaging device 200.

[0026] Figure 4 is a block diagram showing the configuration of a processing system 1 according to one embodiment of the present disclosure. According to Figure 4, the processing system 1 includes a processing unit 100 including a processor 111, a memory 112, an input interface 113, an output interface 114, and a communication interface 115, and an imaging device 200 including a camera 211, a light source 212, a processor 213, a memory 214, a display panel 215, an input interface 210, and a communication interface 216. These components are electrically connected to each other via control lines and data lines. Note that the processing system 1 does not need to include all the components shown in Figure 4; it is possible to omit some components or add other components. For example, the processing system 1 may include a battery for driving each component.

[0027] First, among the processing units 100, the processor 111 functions as a control unit that controls other components of the processing system 1 based on a program stored in the memory 112. Based on the program stored in the memory 112, the processor 111 controls the driving of the camera 211 and the light source 212, and also stores subject images received from the imaging device 200 in the memory 112 and processes the stored subject images. Specifically, the processor 111 executes processes such as acquiring subject images of a subject from the camera 211, inputting the acquired subject images into a judgment image selection model to acquire candidate judgment images, acquiring a judgment image from the acquired candidate judgment images based on the similarity between each image, acquiring at least one of the user's medical history information and attribute information, determining the possibility of contracting influenza based on a trained judgment model stored in the memory 112, one or more acquired judgment images, and, if necessary, at least one of the user's medical history information and attribute information, and outputting information indicating the possibility of contracting a predetermined disease in order to diagnose or assist in such diagnosis, based on a program stored in the memory 112. The processor 111 is mainly composed of one or more CPUs, but may be combined with a GPU, FPGA, etc. as appropriate.

[0028] Memory 112 is composed of RAM, ROM, non-volatile memory, HDD, etc., and functions as a storage unit. Memory 112 stores instruction commands for various controls of the processing system 1 according to this embodiment as programs. Specifically, Memory 112 stores programs for the processor 111 to execute, such as the process of acquiring subject images of a subject from the camera 211, the process of inputting the acquired subject images into a judgment image selection model to acquire candidate judgment images, the process of acquiring a judgment image from the acquired candidate judgment images based on the similarity between each image, the process of acquiring at least one of the user's medical interview information and attribute information, the process of determining the possibility of contracting influenza based on the trained judgment model stored in Memory 112, one or more acquired judgment images, and at least one of the user's medical interview information and attribute information as needed, and the process of outputting information indicating the possibility of contracting a predetermined disease in order to diagnose or assist in such diagnosis. In addition to the program, memory 112 also stores subject images captured by the camera 211 of the imaging device 200, an image management table for managing such images, a user table for storing user attribute information, medical history information, and judgment results. Memory 112 also stores various trained models, such as a trained judgment image selection model used to select judgment images from subject images, and a trained judgment model for determining the likelihood of disease onset from judgment images.

[0029] The input interface 113 functions as an input unit that receives operator instructions for the processing unit 100 and the imaging device 200. Examples of physical keys for the input interface 113 include a "shoot button" for instructing the start and end of recording by the imaging device 200, a "confirm button" for making various selections, a "back / cancel button" for returning to the previous screen or canceling a confirmed operation, a directional key for moving pointers output to the output interface 114, an on / off key for turning the processing unit 100 on and off, and character input keys for entering various characters. It is also possible to use a touch panel for the input interface 113 that is superimposed on the display which functions as the output interface 114 and has an input coordinate system corresponding to the display coordinate system of the display. In this case, icons corresponding to the above physical keys are displayed on the display, and the operator makes a selection for each icon by giving instructions via the touch panel. The method for detecting the user's instructions via the touch panel may be any method, such as capacitive or resistive. The input interface 113 does not always need to be physically provided on the processing unit 100 and may be connected as needed via a wired or wireless network.

[0030] The output interface 114 functions as an output unit for outputting subject images captured by the imaging device 200 or for outputting results determined by the processor 111. An example of the output interface 114 is a display composed of a liquid crystal panel, an organic EL display, or a plasma display. However, the processing unit 100 itself does not necessarily need to be equipped with a display. For example, an interface for connecting to a display that can be connected to the processing unit 100 via a wired or wireless network can also function as the output interface 114 for outputting display data to the display.

[0031] The communication interface 115 functions as a communication unit for sending and receiving various commands related to the start of shooting and image data captured by the shooting device 200 to and from the shooting device 200, which is connected via a wired or wireless network. Examples of the communication interface 115 include wired communication connectors such as USB and SCSI, wireless communication transceivers such as wireless LAN, Bluetooth®, and infrared, and various connection terminals for printed circuit boards and flexible circuit boards.

[0032] Next, the camera 211 of the imaging device 200 functions as an imaging unit that detects reflected light reflected from the oral cavity, which is the subject, and generates an image of the subject. To detect this light, the camera 211 includes, for example, a CMOS image sensor, and a lens system and drive system to realize the desired function. The image sensor is not limited to a CMOS image sensor; other sensors such as a CCD image sensor can also be used. Although not specifically shown in the figures, the camera 211 may have an autofocus function, and it is preferable that the focus be set to a specific area in front of the lens, for example. The camera 211 may also have a zoom function, and it is preferable that it is set to image at an appropriate magnification depending on the size of the pharynx or influenza follicle.

[0033] It is known that lymphoid follicles appearing in the deepest part of the pharynx, located within the oral cavity, exhibit a pattern specific to influenza. These lymphoid follicles with a specific pattern are called influenza follicles, and they are a characteristic sign of influenza, appearing approximately two hours after the onset of symptoms. As described above, the processing system 1 of this embodiment is used to determine the likelihood of a user contracting influenza by, for example, imaging the pharynx in the oral cavity and detecting the above-mentioned follicles. Therefore, when the imaging device 200 is inserted into the oral cavity, the distance between the camera 211 and the subject becomes relatively close. Accordingly, it is preferable that the camera 211 has a field of view (2θ) such that the value calculated by [(distance from the tip of the camera 211 to the posterior wall of the pharynx) * tanθ] is 20 mm or more vertically and 40 mm or more horizontally. By using a camera with such a field of view, it becomes possible to image a wider range even when the camera 211 and the subject are in close proximity. That is, while it is possible to use a normal camera for the camera 211, it is also possible to use a camera known as a wide-angle camera or an ultra-wide-angle camera.

[0034] Furthermore, in this embodiment, the main subject imaged by the camera 211 is the pharynx or influenza follicles formed in the pharyngeal region. Generally, the pharynx is formed in a recessed direction, so if the depth of field is shallow, the focus will shift between the anterior and posterior parts of the pharynx, making it difficult to obtain a subject image suitable for use in the processing device 100. Therefore, the camera 211 has a depth of field of at least 20 mm, preferably 30 mm or more. By using a camera with such a depth of field, it is possible to obtain a subject image that is in focus at any point from the anterior to the posterior part of the pharynx.

[0035] The light source 212 is driven by instructions from the processing device 100 or the imaging device 200 and functions as a light source unit for irradiating light into the oral cavity. The light source 212 includes one or more light sources. In this embodiment, the light source 212 is composed of one or more LEDs, and light having a predetermined frequency band is irradiated from each LED toward the oral cavity. The light source 212 uses light having a desired band from the ultraviolet light band, visible light band, infrared light band, or a combination thereof. When the processing device 100 determines the possibility of influenza infection, it is preferable to use light in the short wavelength band of the ultraviolet light band.

[0036] The processor 213 functions as a control unit that controls other components of the imaging device 200 based on a program stored in the memory 214. Based on the program stored in the memory 214, the processor 213 controls the operation of the camera 211 and the light source 212, and controls the storage of subject images captured by the camera 211 in the memory 214. The processor 213 also controls the output of subject images and user information stored in the memory 214 to the display 203 and their transmission to the processing unit 100. The processor 213 is mainly composed of one or more CPUs, but may be combined with other processors as appropriate.

[0037] Memory 214 consists of RAM, ROM, non-volatile memory, HDD, etc., and functions as a storage unit. Memory 214 stores instruction commands for various control of the imaging device 200 as programs. In addition to these programs, memory 214 also stores subject images captured by the camera 211, various user information, and so on.

[0038] The display panel 215 is provided on the display 203 and functions as a display unit for displaying subject images captured by the imaging device 200. The display panel 215 is made of a liquid crystal panel, but is not limited to a liquid crystal panel; it may also be made of an organic EL display, a plasma display, or the like.

[0039] The input interface 210 functions as an input unit that receives user instructions for the processing unit 100 and the imaging device 200. Examples of input interface 210 include physical keys such as a "shoot button" for instructing the start and end of recording by the imaging device 200, a "power button" for turning the power of the imaging device 200 on and off, a "confirm button" for making various selections, a "back / cancel button" for returning to the previous screen or canceling an entered confirmation operation, and a directional pad for moving icons displayed on the display panel 215. These various buttons and keys may be physically provided, or they may be displayed as icons on the display panel 215 and made selectable using a touch panel or the like superimposed on the display panel 215 as the input interface 210. The method for detecting user instructions via the touch panel may be any method, such as capacitive or resistive touch.

[0040] The communication interface 216 functions as a communication unit for sending and receiving information with the imaging device 200 and / or other devices. Examples of the communication interface 216 include wired communication connectors such as USB and SCSI, wireless communication transceivers such as wireless LAN, Bluetooth®, and infrared, and various connection terminals for printed circuit boards and flexible circuit boards.

[0041] Figure 5 is a top view showing the configuration of an imaging device 200 according to one embodiment of the present disclosure. Specifically, Figure 5 shows the imaging device 200, including the main body 201, grip 202, and display 203, viewed from above, from the side that is inserted into the oral cavity. According to Figure 5, the main body 201 is made up of a columnar body having a base end 225 and a tip end 222, and having a predetermined length in a direction substantially parallel to the direction in which light is irradiated from the light source 212, i.e., the direction H in which it is inserted into the oral cavity. At least the tip end 222 of the main body 201 is inserted into the oral cavity.

[0042] The main body 201 is formed in a hollow cylindrical column shape with a perfectly circular cross-section. Its wall portion 224 can be made of any material capable of guiding light into its interior, and one example is that it can be made using a thermoplastic resin. Examples of thermoplastic resins include polyolefin resins such as chain polyolefin resins (polypropylene resins, etc.) and cyclic polyolefin resins (norbornene resins, etc.), cellulose ester resins such as triacetylcellulose and diacetylcellulose, polyester resins, polycarbonate resins, (meth)acrylic resins, polystyrene resins, or mixtures or copolymers thereof. In other words, the wall portion 224 of the main body 201 functions as a light guide for guiding light irradiated from a light source towards the oral cavity or the diffuser plate.

[0043] Since the main body 201 is formed hollow, a housing space 223 is formed on its inner surface by the wall portion 224. The camera 211 is housed in this housing space 223. The main body 201 only needs to be formed in a columnar shape having the housing space 223. Therefore, the housing space 223 does not need to have a cylindrical cross-section with a perfect circle; its cross-section may be elliptical or polygonal. Also, the main body 201 does not necessarily need to be formed hollow inside.

[0044] The grip 202 has its tip connected to the base end 225 of the main body 201. The user grasps the grip 202 to perform operations such as inserting and removing the imaging device 200. The grip 202 is made of a columnar body of a predetermined length, positioned substantially parallel to the direction H in which it is inserted into the oral cavity, that is, along the longitudinal direction of the main body 201, and is arranged on the same straight line as the main body 201 in direction H. In this embodiment, the cross-section in the vertical direction is formed to be substantially oval, but it does not have to be oval; it may be a perfect circle, ellipse, or polygon.

[0045] The grip 202 has a connecting portion 230 formed at the position closest to the base end 225 of the main body 201, and is connected to the main body 201 via this connecting portion 230. The outer circumference of the connecting portion 230 has engaging projections 217 (217-1 to 217-4) and positioning projections 218 for positioning the auxiliary device 300. The engaging projections 217 engage with the engaging projections 318 (318-1 to 318-4) provided on the auxiliary device 300. The positioning projections 218 are inserted into insertion holes 321 provided on the auxiliary device 300 to position the imaging device 200 and the auxiliary device 300 relative to each other. In this embodiment, the engaging projections 217 of the main body 201 consist of a total of four engaging projections (engaging projections 217-1 to 217-4), which are arranged at equal intervals on the surface of the grip 202 near the base end 225 of the main body 201. Furthermore, one positioning projection 218 is positioned between the engaging projections 217 on the surface of the grip 202, near the base end 225 of the main body 201. However, this is not the only option; either the engaging projection 217 or the positioning projection 218 may be provided. The number of each of the engaging projections 217 and positioning projections 218 may also be any number, as long as there is one or more.

[0046] Furthermore, the grip 202 includes a capture button 220 on its upper surface, near the base end 225 of the main body 201, that is, near the tip in the insertion direction H of the grip 202 into the oral cavity. This allows the operator 600 to easily press the capture button 220 with their index finger or the like when holding the device. Additionally, the power button 221 is located on the upper surface of the grip 202, near the display 203, that is, on the opposite side of the grip 202 from the capture button 220. This prevents the operator 600 from accidentally pressing the power button 221 while holding the device and taking images.

[0047] The display 203 has a roughly rectangular parallelepiped shape overall and is positioned on the same straight line as the main body 201 in direction H. The display 203 also includes a display panel 215 on the side opposite to the direction H in which it is inserted into the oral cavity (i.e., the direction toward the user). Therefore, the display 203 is formed such that the side including the display panel is approximately perpendicular to the longitudinal direction of the main body 201 and grip 202, which are formed to be approximately parallel to the direction H in which it is inserted into the oral cavity. The display 203 is connected to the grip 202 on the side opposite to the side of the grip 202 that is not toward the oral cavity. Note that the shape of the display is not limited to a roughly rectangular parallelepiped shape, but may be any shape, such as a cylinder.

[0048] The diffuser plate 219 is positioned at the tip 222 of the main body 201 and diffuses the light that has been irradiated from the light source 212 and passed through the main body 201 towards the inside of the oral cavity. The diffuser plate 219 has a shape that corresponds to the cross-sectional shape of the portion of the main body 201 that is configured to guide light. In this embodiment, the main body 201 is formed in a hollow cylindrical shape. Therefore, the cross-section of the diffuser plate 219 is also formed in a hollow shape corresponding to its shape.

[0049] Camera 211 is used to generate an image of a subject by detecting the reflected light that is diffused from the diffuser plate 219, irradiated into the oral cavity, and reflected back to the subject. Camera 211 is positioned in a housing space 223 formed on the inner surface of the wall portion 224 of the main body 201, i.e., inside the main body 201, so as to be on the same straight line as the main body 201 in direction H. In this embodiment, only one camera 211 is described, but the imaging device 200 may include multiple cameras. By generating an image of a subject using multiple cameras, the image of the subject will include information about its three-dimensional shape. In this embodiment, camera 211 is positioned in the housing space 223 of the main body 201, but it may also be positioned at the tip 222 of the main body 201 or on the main body 201 (either inside the main body 201 or on the outer circumference of the main body 201).

[0050] Figure 6 is a schematic diagram showing the cross-sectional configuration of an imaging device 200 according to one embodiment of the present disclosure. According to Figure 6, the light source 212 consists of a total of four light sources 212-1 to 212-4 arranged on a substrate 231 located on the tip side of the grip 202. Each light source 212 is, for example, composed of an LED, and light having a predetermined frequency band is emitted from each LED toward the oral cavity. Specifically, the light emitted from the light source 212 enters the base end 225 of the main body 201 and is guided toward the diffuser plate 219 by the wall portion 224 of the main body 201. The light that reaches the diffuser plate 219 is diffused into the oral cavity by the diffuser plate 219. The light diffused by the diffuser plate 219 is then reflected by the pharynx 715, etc., which is the subject. When this reflected light reaches the camera 211, a subject image is generated.

[0051] Furthermore, the light sources 212-1 to 212-4 may be configured to be controlled independently. For example, by illuminating some of the light sources 212-1 to 212-4, the shadow of a three-dimensional object (such as an influenza follicle) can be included in the object image. This allows the object image to contain information about the object's three-dimensional shape, making object identification clearer and enabling the determination algorithm to more accurately determine the likelihood of influenza infection.

[0052] Furthermore, in this embodiment, the light sources 212-1 to 212-4 are arranged on the base end 225 side of the main body 201, but they may also be arranged on the tip 222 of the main body 201 or on the main body 201 (either inside the main body 201 or on the outer circumference of the main body 201).

[0053] In this embodiment, the diffuser plate 219 is used to prevent the light emitted from the light source 212 from illuminating only a part of the oral cavity and to generate uniform light. Therefore, as an example, a lens-shaped diffuser plate with an arbitrary diffusion angle is used, by forming a fine lens array on the surface of the diffuser plate 219. Alternatively, a diffuser plate capable of diffusing light by other methods may be used, such as a diffuser plate that achieves a light diffusion function by randomly arranged fine irregularities on its surface. Furthermore, the diffuser plate 219 may be configured integrally with the main body 201. For example, this can be achieved by forming fine irregularities on the tip portion of the main body 201.

[0054] Furthermore, in this embodiment, the diffuser plate 219 is positioned on the tip 222 side of the main body 201. However, it is not limited to this, and may be positioned anywhere between the light source 212 and the oral cavity that is to be irradiated, for example, on the tip 222 of the main body 201 or on the main body 201 (either inside the main body 201 or on the outer circumference of the main body 201).

[0055] 3. Information stored in the memory 112 of the processing unit 100 Figure 7A is a conceptual diagram showing an image management table stored in a processing unit 100 according to one embodiment of the present disclosure. The information stored in the image management table is updated as needed in accordance with the progress of processing by the processor 111 of the processing unit 100.

[0056] According to Figure 7A, the image management table stores subject image information, candidate information, and judgment image information, etc., associated with user ID information. "User ID information" is information unique to each user and is used to identify each user. User ID information is generated each time a new user is registered by the operator. "Subject image information" is information that identifies the subject image captured by the operator for each user. The subject image is one or more images including a subject captured by the camera of the imaging device 200, and is stored in memory 112 after being received from the imaging device 200. "Candidate information" is information that identifies an image that is a candidate for selection as the judgment image from one or more subject images. "Judgment image information" is information that identifies the judgment image used to determine the possibility of influenza infection. Such a judgment image is selected from the candidate images identified by the candidate information based on similarity. As described above, information for identifying each image is stored as subject image information, candidate information, and judgment image information. Thus, the information used to identify each image is typically identification information for each image, but it may also be information indicating the storage location of each image, or the image data itself for each image.

[0057] Figure 7B is a conceptual diagram showing a user table stored in a processing unit 100 according to one embodiment of the present disclosure. The information stored in the user table is updated as needed in accordance with the progress of processing by the processor 111 of the processing unit 100.

[0058] According to Figure 7B, the user table stores attribute information, medical history information, QR code information, and judgment result information, etc., associated with user ID information. "User ID information" is information unique to each user and is used to identify each user. User ID information is generated each time a new user is registered by the operator. "Attribute information" is information entered by the operator or user, for example, and is information related to the individual user, such as the user's name, gender, age, and address. "Medical history information" is information entered by the operator or user, for example, and is information used by doctors, etc., as reference for diagnosis, such as the user's medical history and symptoms. Examples of such medical information include patient background such as weight, allergies, and underlying diseases; body temperature, peak body temperature since onset, time elapsed since onset, heart rate, pulse rate, oxygen saturation, blood pressure, medication use, contact with other influenza patients, presence or absence of subjective symptoms and physical findings such as rash on hands and feet, redness or white coating on the throat, tonsil swelling, history of tonsillectomy, strawberry tongue, and swelling of the anterior cervical lymph nodes with tenderness, influenza vaccination history, and vaccination timing. "Two-dimensional code information" is information for identifying a recording medium on which user ID information, information to identify it, attribute information, medical information, and at least one combination thereof is recorded. Such a recording medium does not necessarily have to be a two-dimensional code. Instead of two-dimensional codes, various other forms of code can be used, such as one-dimensional barcodes, other multi-dimensional codes, text information such as specific numbers or characters, and image information. "Judgment result information" is information that indicates the likelihood of influenza infection based on the judgment image. An example of such judgment result information is the positive rate for influenza. However, it is not limited to the positive rate; any information that indicates the possibility, such as whether it is positive or negative, is acceptable. Furthermore, the judgment result does not need to be a specific numerical value; the format can be anything, such as a classification according to the level of the positive rate or a classification indicating whether it is positive or negative.

[0059] Furthermore, attribute information and medical history information do not need to be entered by the user or operator each time; they may be received, for example, from an electronic medical record system or other terminal device connected via a wired or wireless network. Alternatively, they may be obtained by analyzing subject images captured by the imaging device 200. In addition, although not specifically shown in Figures 7A and 7B, it is also possible to store in memory 112 current epidemic information on infectious diseases that are the target of diagnosis or assistance in diagnosis, such as influenza, as well as external factor information such as the judgment results and disease status of other users regarding these infectious diseases.

[0060] 4. Processing sequence executed by the processing device 100 and the imaging device 200 Figure 8 is a diagram showing a processing sequence performed between a processing device 100 and an imaging device 200 according to one embodiment of the present disclosure. Specifically, Figure 8 shows a processing sequence performed from the selection of the imaging mode in the processing device 100, to the capture of the subject image in the imaging device 200, and the output of the determination result from the processing device 100.

[0061] As shown in Figure 8, the processing unit 100 outputs a mode selection screen via the output interface 114 and accepts the operator's mode selection via the input interface 113 (S11). Once the shooting mode selection is accepted, the processing unit 100 outputs an attribute information input screen via the output interface 114. The processing unit 100 accepts input from the operator or user via the input interface 113, acquires attribute information, and stores it in the user table in association with user ID information (S12). Once the attribute information is acquired, the processing unit 100 outputs a medical interview information input screen via the output interface 114. The processing unit 100 accepts input from the operator or user via the input interface 113, acquires medical interview information, and stores it in the user table in association with user ID information (S13). Note that the acquisition of attribute information and medical interview information does not need to be performed at this timing and can be performed at other times, such as before the judgment process. Furthermore, this information may be obtained not only by receiving input via the input interface 113, but also by receiving it from an electronic medical record system or other terminal devices connected via a wired or wireless network. Alternatively, this information may be recorded on a recording medium such as a two-dimensional code after being input by an electronic medical record system or other terminal device, and then captured by a camera or imaging device 200 connected to the processing unit 100. Alternatively, this information may be obtained by having users, operators, patients, medical personnel, etc., fill out a paper document such as a medical questionnaire, and then capturing the paper document with a scanner or imaging device 200 connected to the processing unit 100 and performing optical character recognition.

[0062] The processing unit 100 generates a two-dimensional code containing pre-generated user ID information under the control of the processor 111 and stores it in the memory 112 (S14). Then, the processing unit 100 outputs the generated two-dimensional code via the output interface 114 (S15).

[0063] Next, the imaging device 200 activates the camera 211, etc., when it receives input from the operator to the input interface 210 (e.g., the power button) (S21). Then, by having the activated camera 211 capture the two-dimensional code output via the output interface 114, the imaging device 200 reads the user ID information recorded in the two-dimensional code (S22).

[0064] Next, the operator covers the tip of the imaging device 200 with the auxiliary tool 300 and inserts the imaging device 200 into the user's mouth to a predetermined position. When the imaging device 200 receives input from the operator via the input interface 210 (e.g., the capture button), it starts capturing a subject image of the subject that includes at least a portion of the mouth (S23). When the subject image capture is complete, the imaging device 200 stores the captured subject image in the memory 214 in association with the user ID information read from the two-dimensional code, and also outputs the captured subject image to the display panel 215 of the display (S24). Then, when the imaging device 200 receives input from the operator via the input interface 210 indicating the end of the capture, it transmits the stored subject image (T21) in association with the user ID information to the processing unit 100 via the communication interface 216.

[0065] Next, when the processing unit 100 receives a subject image via the communication interface 115, it stores it in the memory 112 and registers it in the image management table based on the user ID information. The processing unit 100 selects a judgment image from the stored subject images to be used to determine the possibility of influenza infection (S31). Once a judgment image is selected, the processing unit 100 performs the process of determining the possibility of influenza infection using the selected judgment image (S32). Once a judgment result is obtained, the processing unit 100 stores the obtained judgment result in association with the user ID information in the user table and outputs it via the output interface 114 (S33). This completes the processing sequence.

[0066] 5. Processing flow executed by the processing unit 100 (mode selection process, etc.) Figure 9 is a diagram showing the processing flow executed in a processing apparatus 100 according to one embodiment of the present disclosure. Specifically, Figure 9 is a diagram showing the processing flow executed at a predetermined cycle for the processing related to S11 to S15 in Figure 8. This processing flow is mainly performed by the processor 111 of the processing apparatus 100 reading and executing a program stored in the memory 112.

[0067] According to Figure 9, the processor 111 outputs a mode selection screen via the output interface 114 (S111). Here, Figure 18 is a diagram showing an example of a screen displayed in the processing unit 100 according to one embodiment of the present disclosure. Specifically, Figure 18 shows an example of a mode selection screen output in S111 and S112 of Figure 9. According to Figure 18, approximately in the center of the display which functions as the output interface 114, there is a shooting mode icon 11 for switching to a shooting mode for capturing an image of a subject, and a judgment result confirmation mode icon 12 for switching to a judgment result confirmation mode for outputting the result of the determination of the possibility of having influenza to the display. The user can select which mode to switch to by operating the input interface 113.

[0068] Returning to Figure 9, the processor 111 determines whether or not it has received a mode selection from the operator via the input interface 113 (S112). If the processor 111 determines that no input has been received for either the shooting mode icon 11 or the determination result confirmation mode icon 12 shown in Figure 18, and therefore no mode selection has been received, the processing flow ends.

[0069] On the other hand, when the processor 111 receives input for either the shooting mode icon 11 or the judgment result confirmation mode icon 12 shown in Figure 18 and determines that a mode has been selected, the processor 111 determines whether or not the shooting mode has been selected (S113). Then, if the processor 111 determines that the judgment result confirmation mode icon 12 shown in Figure 18 has been selected, it displays the desired judgment result via the output interface 114 (S118).

[0070] On the other hand, when the processor 111 determines that the shooting mode icon 11 shown in Figure 18 has been selected, it displays a screen on the output interface 114 that accepts input of user attribute information (not shown). This screen includes fields for the user's name, gender, age, address, etc., which are required to be entered as attribute information, and input boxes for entering answers to each field. The processor 111 then acquires the information entered into each input box via the input interface 113 as attribute information (S114). The processor 111 then generates new user ID information corresponding to the user newly stored in the user table, and stores the attribute information in the user table in association with the user ID information. Note that if user ID information has been selected in advance before the input of attribute information, the generation of new user ID information can be omitted.

[0071] Next, the processor 111 displays a screen on the output interface 114 that accepts input of the user's medical questionnaire information (not shown). This screen includes items that need to be entered as medical questionnaire information, such as the user's body temperature, heart rate, medication status, and presence or absence of subjective symptoms, as well as input boxes for entering answers to each item. The processor 111 then acquires the information entered into each input box via the input interface 113 as medical questionnaire information and stores it in the user table in association with the user ID information (S115).

[0072] The above explanation describes how attribute information and medical history information are input by the processing device 100. However, the information may also be obtained by receiving it from an electronic medical record device or other terminal device connected via a wired or wireless network.

[0073] Next, the processor 111 refers to the user table, reads the user ID information corresponding to the user for whom this information was entered, and generates a two-dimensional code that records this information (S116). The processor 111 stores the generated two-dimensional code in the user table, associating it with the user ID information, and also outputs it via the output interface 114 (S117). With this, the processing flow is completed.

[0074] Here, Figure 19 shows an example of a screen displayed in a processing device 100 according to one embodiment of the present disclosure. Specifically, Figure 19 shows an example of a display screen for the two-dimensional code output in S117 of Figure 9. According to Figure 19, above the display which functions as an output interface 114, user ID information of the user, in which attribute information and the like have been entered, is displayed. In addition, a two-dimensional code generated in S116 of Figure 19 is displayed approximately in the center of the display. By photographing the two-dimensional code with the imaging device 200, it is possible to read the user ID information recorded in the two-dimensional code.

[0075] 6. Processing flow executed by the imaging device 200 (imaging process, etc.) Figure 10 is a diagram showing the processing flow executed in a processing apparatus 100 according to one embodiment of the present disclosure. Specifically, Figure 10 is a diagram showing the processing flow executed at a predetermined cycle for the processing related to S21 to S24 in Figure 8. This processing flow is mainly performed by the processor 213 of the imaging device 200 reading and executing a program stored in the memory 214.

[0076] As shown in Figure 10, the processor 213 determines whether or not input from the operator has been received via the input interface 210 (for example, the power button) (S211). If the processor 213 determines that no input from the operator has been received, the processing flow terminates.

[0077] On the other hand, when the processor 213 determines that it has received input from the operator, it outputs a standby screen to the display panel 215 (S212). This standby screen (not shown) includes a through image captured via the camera 211. The operator then moves the imaging device 200 so that the two-dimensional code output to the output interface 114 of the processing unit 100 is included in the field of view of the camera 211, causing the processor 213 to capture the two-dimensional code with the camera 211 (S213). Once the two-dimensional code is captured, the processor 213 reads the user ID information recorded in the two-dimensional code and stores the read user ID information in the memory 214 (S214). The processor 213 then outputs the standby screen to the display panel 215 again (S215).

[0078] Next, the operator covers the tip of the imaging device 200 with the auxiliary tool 300 and inserts the imaging device 200 into the user's mouth to a predetermined position. When the processor 213 receives a shooting start operation from the operator via the input interface 210 (for example, the shooting button), the processor 213 controls the camera 211 to start capturing images of the subject (S216). This subject image capture is performed by taking a fixed number of images (for example, 30 images) in a continuous shooting sequence at regular intervals when the shooting button is pressed. When the subject image capture is finished, the processor 213 stores the captured subject image in the memory 214 in association with the user ID information read from it. Then, the processor 213 outputs the stored subject image to the display panel 215 (S217).

[0079] Here, the operator can remove the imaging device 200 along with the auxiliary device 300 from inside the oral cavity, check the subject image output on the display panel 215, and if the desired image is not obtained, input a command to retake the image. Therefore, the processor 213 determines whether or not it has received a command to retake the image from the operator via the input interface 210 (S218). If a command to retake the image has been received, the processor 213 displays the standby screen of S215 again and enables the capture of the subject image.

[0080] On the other hand, if no instruction for reshooting has been received, and the operator returns the imaging device 200 to the mounting table 400 and receives an instruction from the processing unit 100 to end the shooting, the processor 213 transmits the subject image stored in the memory 214 and the user ID information associated with the subject image to the processing unit 100 via the communication interface 216 (S219). With this, the processing flow ends.

[0081] 7. Processing flow (decision processing, etc.) executed by the processing unit 100 Figure 11 is a diagram showing the processing flow executed in a processing apparatus 100 according to one embodiment of the present disclosure. Specifically, Figure 11 is a diagram showing the processing flow executed for the processing related to S31 to S33 in Figure 8. This processing flow is mainly performed by the processor 111 of the processing apparatus 100 reading and executing a program stored in the memory 112.

[0082] As shown in Figure 11, when the processor 111 receives a subject image and its associated user ID information from the imaging device 200, it stores them in the memory 112 and registers them in the image management table (S311). The processor 111 then outputs the received user ID information or its corresponding attribute information (e.g., name) via the output interface 114 and accepts the selection of a user to be assessed for the possibility of contracting influenza (S312). If the imaging device 200 receives multiple user ID information and their associated subject images, it is possible to output multiple user ID information or their corresponding attribute information and select one of the users.

[0083] When the processor 111 receives the selection of a user to be evaluated via the input interface 113, it reads attribute information associated with the user's user ID information from the user table in memory 112 (S313). Similarly, the processor 111 also reads medical history information associated with the user's user ID information from the user table in memory 112 (S314).

[0084] Next, the processor 111 reads the subject image associated with the user ID information of the selected user from the memory 112 and performs a selection process for the judgment image to be used to determine the possibility of contracting influenza (S315: details of this selection process will be described later). Then, the processor 111 performs a judgment process to determine the possibility of contracting influenza based on the selected judgment image (S316: details of this judgment process will be described later). When the judgment result is obtained through the judgment process, the processor 111 stores it in the user table associated with the user ID information and outputs the judgment result via the output interface 114 (S317). This completes the processing flow.

[0085] Figure 12 is a diagram showing the processing flow performed in a processing apparatus 100 according to one embodiment of the present disclosure. Specifically, Figure 12 is a diagram showing the details of the selection process of a judgment image performed in S315 of Figure 11. This processing flow is mainly performed by the processor 111 of the processing apparatus 100 reading and executing a program stored in the memory 112.

[0086] As shown in Figure 12, processor 111 reads a subject image from memory 112 that is associated with the user ID information of the selected user (S411). Next, processor 412 selects a candidate image to be used as the judgment image from the read subject images (S412). This selection is performed, for example, using a trained judgment image selection model.

[0087] Here, Figure 13 is a diagram showing a processing flow for generating a trained model according to one embodiment of the present disclosure. Specifically, Figure 13 is a diagram showing a processing flow for generating a trained judgment image selection model used in S412 of Figure 12. This processing flow may be executed by the processor 111 of the processing unit 100, or by the processor of another processing unit.

[0088] As shown in Figure 13, the processor performs the step of acquiring a subject image of a subject that includes at least a part of the pharynx as a training subject image (S511). Next, the processor performs the processing step of assigning label information to the acquired training subject image to indicate whether or not it is an image that can be used as a judgment image (S512). Then, the processor performs the step of storing the assigned label information in association with the training subject image (S513). Note that the label assignment process and the label information storage process may be performed by having a human determine in advance whether or not the training subject image is a judgment image and then storing the result in association with the training subject image, or the processor may perform an analysis to determine whether or not it is a judgment image using a known image analysis process and store the result in association with the training subject image. Furthermore, the label information is assigned based on whether or not at least a part of the oral cavity of the subject is captured, and whether or not the image quality is good, such as whether or not there is camera shake, out of focus, or blur.

[0089] Once training subject images and their associated label information are obtained, the processor performs a step of machine learning to determine the selection pattern of judgment images using them (S514). This machine learning is performed, for example, by providing the training subject images and label information pairs to a neural network composed of combinations of neurons, and repeatedly adjusting the parameters of each neuron so that the output of the neural network matches the label information. Then, a step of obtaining the trained judgment image selection model (e.g., a neural network and parameters) is performed (S515). The obtained trained judgment image selection model may be stored in the memory 112 of the processing unit 100 or in another processing unit connected to the processing unit 100 via a wired or wireless network.

[0090] Returning to Figure 12, the processor 111 inputs the subject image read in S411 to the trained judgment image selection model, thereby obtaining candidate images that can be used as judgment images as output. This allows for the selection of images that show at least a portion of the subject's oral cavity, and images with good image quality, such as those free from camera shake, focus errors, motion blur, exposure issues, and cloudiness. Furthermore, it enables the stable selection of images with good image quality regardless of the operator's skill in taking photographs. The processor 111 then registers the acquired candidate images that can be used as judgment images into the image management table.

[0091] Next, the processor 111 performs a process to select a judgment image based on similarity from the selected candidate images (S413). Specifically, the processor 111 compares the obtained candidate images with each other and calculates the similarity between each candidate image. Then, the processor 111 selects the candidate image that is judged to have a low similarity to the other candidate images as the judgment image. The similarity between each candidate image is calculated using methods such as the Bag-of-Keypoints method, the Earth Mover's Distance (EMD) method, the Support Vector Machine (SVM) method, the Hamming distance method, and the cosine similarity method.

[0092] In this way, by calculating the similarity between the obtained candidate images and selecting candidate images that are judged to have low similarity to other candidate images, subject images from different fields of view are selected as judgment images. This makes it possible to perform judgment processing based on more diverse information compared to using subject images obtained from the same field of view as judgment images, and thus improves the accuracy of the judgment. Specifically, even if a part of the pharynx is obscured by the uvula in one judgment image, the obscured part of the pharynx is visible in another judgment image from a different field of view, thus preventing the oversight of important features such as influenza follicles.

[0093] Next, the processor 111 registers the candidate images selected based on similarity as judgment images in the image management table (S414).

[0094] Here, the subject image, candidate image, and judgment image may each be one or multiple images. However, as an example, it is preferable to select a group of candidate images from a group of, for example, 5 to 30 subject images, and finally obtain a group of about 5 judgment images. This is because selecting judgment images from a large number of subject images increases the likelihood of obtaining better judgment images. Furthermore, by using multiple groups of judgment images in the judgment process described later, the judgment accuracy can be further improved compared to using only one judgment image. Another example is that each time a subject image is captured, the captured subject image is sent to the processing device 100, and then the selection of candidate images and judgment images is performed, or the selection of candidate images and judgment images is performed in the shooting device 200, and the shooting is terminated when a predetermined number of judgment images (for example, about 5) have been acquired. In this way, the time required for capturing subject images can be minimized while maintaining the improved judgment accuracy as described above. In other words, discomfort to the user, such as the gag reflex, can be reduced.

[0095] Figure 14 is a diagram showing the processing flow performed in a processing apparatus 100 according to one embodiment of the present disclosure. Specifically, Figure 14 is a diagram showing the details of the process for determining the possibility of contracting influenza, which is performed in S316 of Figure 11. This processing flow is mainly performed by the processor 111 of the processing apparatus 100 reading and executing a program stored in memory 112.

[0096] As shown in Figure 14, the processor 111 obtains the judgment result by ensemble processing the first positive rate, second positive rate, and third positive rate, which were obtained using different methods.

[0097] First, the process for obtaining the first positive rate will be explained. The processor 111 reads the judgment image associated with the user ID information of the user to be judged from the memory 112 (S611). Then, the processor 111 performs predetermined preprocessing on the read judgment image. Such preprocessing may include filtering processes such as bandpass filters including high-pass filters and low-pass filters, averaging filters, Gaussian filters, Gabor filters, Canny filters, Sobel filters, Laplacian filters, median filters, and bilateral filters; vascular extraction processing using Hessian matrices, segmentation processing of specific regions (e.g., follicles) using machine learning; trimming processing of the segmented regions; de-haze processing; super-resolution processing; and combinations thereof, selected according to the purpose of high resolution, region extraction, noise reduction, edge enhancement, image correction, and image transformation. By performing preprocessing in this way, it is possible to improve the accuracy of the judgment by pre-extracting or highlighting important regions for disease diagnosis, such as follicles in influenza.

[0098] Here, we will specifically describe, as an example, the case in which preprocessing such as haze removal, super-resolution processing, and segmentation processing is performed. First, for haze removal, as an example, a trained haze-removed image model obtained by machine learning is used by providing a trainer with a set of training subject images and training degraded images obtained by applying a haze-adding filter to the training subject images. The processor 111 takes the read judgment image as input and inputs it to the trained haze-removed image model stored in memory 112, and obtains the judgment image with the haze removed as output. For super-resolution processing, a trained super-resolution image model obtained by machine learning is used by providing a trainer with a set of high-resolution images of the subject and low-resolution images obtained by performing degradation processing such as scaling down or blurring on the high-resolution images as training images. The processor 111 takes the judgment image with the haze removal processing as input and inputs it to the trained super-resolution image model stored in memory 112, and obtains the judgment image with the super-resolution processing as output. Furthermore, the segmentation process uses a trained segmentation image model obtained by machine learning, which is created by providing a learning model with a set of training subject images and the positional information of labels obtained by assigning labels to areas of interest (e.g., follicles) based on operational input from physicians to the training subject images. The processor 111 takes the super-resolution processed judgment image as input, inputs the trained segmentation image model stored in memory 112, and obtains a judgment image in which areas of interest (e.g., follicles) are segmented. The processor 112 then stores the pre-processed judgment image in memory 112. Note that, although the processes were performed in the order of de-haze processing, super-resolution processing, and segmentation processing, the order may be any, or at least one of the processes may be performed. Also, although the examples of each process use a trained model, processes such as de-haze filters, scaling processing, and sharpening processing may be used.

[0099] The processor 111 then provides the pre-processed judgment image as input to the feature extractor (S613) and obtains the image features of the judgment image as output (S614). Furthermore, the processor 111 provides the obtained features of the judgment image as input to the classifier (S615) and obtains the first positive rate, which indicates the first possibility of contracting influenza, as output (S616). The feature extractor can obtain a predetermined number of features, such as the presence or absence of follicles and the presence or absence of redness in the judgment image, as vectors. For example, 1024-dimensional feature vectors are extracted from the judgment image and stored as the features of the judgment image.

[0100] Here, Figure 15 is a diagram showing a processing flow for generating a trained model according to one embodiment of the present disclosure. Specifically, Figure 15 is a diagram showing a processing flow for generating a trained positive rate determination selection model that includes the feature extractor S613 and the classifier S615 in Figure 14. This processing flow may be executed by the processor 111 of the processing unit 100, or by the processor of another processing unit.

[0101] As shown in Figure 15, the processor performs a step of acquiring an image of a subject that includes at least a portion of the pharynx, pre-processed in the same manner as in S612 of Figure 14, as a training judgment image (S711). Next, the processor performs a processing step of assigning a correct label to the user who is the subject of the acquired training judgment image, based on the results of an influenza rapid test by immunochromatography, PCR test, virus isolation culture test, etc., which have been assigned in advance (S712). Then, the processor performs a step of storing the assigned correct label information as judgment result information, associating it with the training judgment image (S713).

[0102] Once training images and their corresponding ground truth label information are obtained, the processor performs a step of machine learning to create a positive rate judgment pattern using them (S714). This machine learning is performed, for example, by providing a pair of training images and ground truth label information to a feature extractor composed of a convolutional neural network and a classifier composed of a neural network, and repeatedly adjusting the parameters of each neuron so that the output from the classifier is the same as the ground truth label information. Then, a step of obtaining a trained positive rate judgment model is performed (S715). The obtained trained positive rate judgment model may be stored in the memory 112 of the processing unit 100 or in another processing unit connected to the processing unit 100 via a wired or wireless network.

[0103] Returning to Figure 14, the processor 111 takes the pre-processed judgment image from S612 as input to the trained positive rate judgment model, and outputs the feature quantities of the judgment image (S614) and the first positive rate (S616) indicating the first possibility of contracting influenza, which are then stored in memory 112 in association with the user ID information.

[0104] Next, the process for obtaining the second positive rate will be described. The processor 111 reads at least one of the medical interview information and attribute information associated with the user ID information of the user to be judged from memory 112 (S617). The processor 111 also reads the feature quantities of the judgment image, which were calculated in S614 and stored in memory 112 associated with the user ID information, from memory 112 (S614). Then, the processor 111 provides at least one of the read medical interview information and attribute information and the feature quantities of the judgment image as input to the trained positive rate judgment model (S618) and obtains the second positive rate, which indicates the second possibility of contracting influenza, as output (S619).

[0105] Here, Figure 16 is a diagram showing a processing flow for generating a trained model according to one embodiment of the present disclosure. Specifically, Figure 16 is a diagram showing a processing flow for generating a trained positive rate determination selection model as shown in S618 of Figure 14. This processing flow may be executed by the processor 111 of the processing unit 100, or by the processor of another processing unit.

[0106] As shown in Figure 16, the processor performs a step of acquiring training features from a judgment image that has undergone preprocessing similar to that in Figure 14, S612, for an image of a subject that includes at least a part of the pharynx (S721). The processor also performs a step of acquiring medical history information and attribute information that has been stored in advance and associated with the user ID information of the user who is the subject of the judgment image (S721). Next, the processor performs a processing step of assigning a correct label to the user who is the subject of the judgment image, which has been assigned in advance based on the results of an immunochromatographic rapid influenza test, PCR test, virus isolation culture test, etc. (S722). Then, the processor performs a step of storing the assigned correct label information as judgment result information, associating it with the training features of the judgment image, as well as the medical history information and attribute information (S723).

[0107] Once the learning features for the judgment image, as well as the medical history information, attribute information, and corresponding correct label information are obtained, the processor performs a step of machine learning to create a positive rate judgment pattern using these (S724). This machine learning is performed, for example, by providing these sets of information to a neural network made up of neurons and repeatedly learning while adjusting the parameters of each neuron so that the output from the neural network is the same as the correct label information. Then, a step of obtaining the trained positive rate judgment model is performed (S725). The obtained trained positive rate judgment model may be stored in the memory 112 of the processing unit 100 or in another processing unit connected to the processing unit 100 via a wired or wireless network.

[0108] Returning to Figure 14, the processor 111 inputs the feature quantities of the judgment image read in S614 and at least one of the medical history information and attribute information read in S617 to the trained positive rate judgment model, and outputs a second positive rate (S619) indicating a second possibility of contracting influenza, which is then stored in memory 112 in association with the user ID information.

[0109] Next, the process for obtaining the third positive rate will be described. The processor 111 reads at least one of the questionnaire information and attribute information associated with the user ID information of the user to be judged from memory 112 (S617). The processor 111 also reads the first positive rate calculated in S616 and stored in memory 112 associated with the user ID information from memory 112. Then, the processor 111 provides at least one of the read questionnaire information and attribute information, along with the first positive rate, as input to the trained positive rate judgment model (S620), and obtains the third positive rate, which indicates the third possibility of contracting influenza, as output (S621).

[0110] Here, Figure 17 is a diagram showing a processing flow for generating a trained model according to one embodiment of the present disclosure. Specifically, Figure 17 is a diagram showing a processing flow for generating a trained positive rate determination selection model in S620 of Figure 14. This processing flow may be executed by the processor 111 of the processing unit 100, or by the processor of another processing unit.

[0111] As shown in Figure 17, the processor performs the step of obtaining first positive rate information by inputting a judgment image, which has undergone preprocessing similar to S612 in Figure 14, onto an image of a subject that includes at least a part of the pharynx, into a trained positive rate judgment selection model that includes a feature extractor (S613 in Figure 14) and a classifier (S615 in Figure 14) (S731). The processor also performs the step of obtaining medical history information and attribute information that has been stored in advance and associated with the user ID information of the user who is the subject of the judgment image (S731). Next, the processor performs the processing step of assigning a correct label to the user who is the subject of the judgment image, which has been previously assigned based on the results of an immunochromatographic rapid influenza test, PCR test, virus isolation culture test, etc. (S732). Then, the processor performs the step of storing the assigned correct label information as judgment result information, associated with the first positive rate information, as well as the medical history information and attribute information (S733).

[0112] Once the first positive rate information, as well as the medical history information, attribute information, and corresponding correct label information, are obtained, the processor performs a step of machine learning to create a positive rate determination pattern using these (S734). This machine learning is performed, for example, by providing these sets of information to a neural network made up of neurons and repeatedly learning while adjusting the parameters of each neuron so that the output from the neural network is the same as the correct label information. Then, a step of obtaining the trained positive rate determination model is performed (S735). The obtained trained positive rate determination model may be stored in the memory 112 of the processing unit 100 or in another processing unit connected to the processing unit 100 via a wired or wireless network.

[0113] Returning to Figure 14, the processor 111 takes the first positive rate information read in S616 and at least one of the medical history information and attribute information read in S617 as input to the trained positive rate determination model to obtain a third positive rate (S621) indicating a third possibility of contracting influenza as output, and stores it in memory 112 in association with the user ID information.

[0114] In this way, once the first, second, and third positive rates are calculated, the processor 111 reads each positive rate from the memory 112 and performs ensemble processing (S622). As an example of this ensemble processing, the obtained first, second, and third positive rates are given as input to a ridge regression model, and an ensemble result of each positive rate is obtained as the determination of the possibility of contracting influenza (S623).

[0115] The ridge regression model used in S622 is generated by machine learning performed by the processor 111 of the processing unit 100 or the processor of another processing unit. Specifically, the processor obtains the first positive rate, second positive rate, and third positive rate from the training judgment images. The processor also assigns a correct label to the user who was the subject of the training judgment image, based on the results of an immunochromatographic rapid influenza test, PCR test, virus isolation culture test, etc. The processor then provides each positive rate and its corresponding correct label pair to the ridge regression model and repeats the learning process while adjusting the parameters given to each positive rate so that the output is the same as the correct label information of the ridge regression model. This results in a ridge regression model used for ensemble processing, which is stored in the memory 112 of the processing unit 100 or in another processing unit connected to the processing unit 100 via a wired or wireless network.

[0116] Furthermore, while we have described the use of a ridge regression model as an example of ensemble processing, any other method may be used, such as processing to obtain the mean of each positive rate, processing to obtain the maximum value, processing to obtain the minimum value, processing to add weights, or processing using other machine learning techniques such as bagging, boosting, stacking, lasso regression, and linear regression.

[0117] The processor 111 stores the determination result obtained in this manner in the user table of memory 112, associating it with the user ID information (S624). This terminates the processing flow.

[0118] In Figure 14, an ensemble process was performed on the first, second, and third positive rates to obtain the final judgment result. However, this is not the only option; each positive rate can be used as the final judgment result, or an ensemble process using any two positive rates can be performed to obtain the final judgment result. Furthermore, other positive rates obtained by other methods can be added and further processed in an ensemble to obtain the final judgment result.

[0119] Furthermore, as shown in Figure 11, the obtained judgment results are output via the output interface 114. However, only the final judgment result may be output, or each positive rate may also be output.

[0120] Thus, this embodiment makes it possible to provide a processing device, a processing program, a processing method, and a processing system suitable for processing images obtained by photographing the inside of the oral cavity for use in oral cavity diagnosis.

[0121] 8. Variations The example in Figure 14 illustrates a case where information indicating the possibility of contracting influenza is output using at least one of the medical interview information and attribute information. However, instead of these pieces of information, or in addition to these pieces of information, information indicating the possibility of contracting influenza may be output using external factor information related to influenza. Such external factor information includes the results of judgments made for other users, the results of a doctor's diagnosis, and information on influenza outbreaks in the region to which the user belongs. The processor 111 can obtain such external factor information from other processing devices via the communication interface 115 and provide the external factor information as input to the trained positive rate determination model, thereby enabling the acquisition of a positive rate that takes the external factor information into account.

[0122] In the example shown in Figure 14, the case where medical history information and attribute information are pre-entered by the operator or user, or received from an electronic medical record device connected to a wired or wireless network, was explained. However, this information may also be obtained from captured subject images instead of, or in addition to, these methods. A trained information estimation model is obtained by providing attribute information and medical history information associated with training subject images as correct labels, and using machine learning on these pairs in a neural network. Then, the processor 111 can obtain the desired medical history information and attribute information by providing the subject images as input to the trained information estimation model. Examples of such medical history information and attribute information include gender, age, degree of pharyngeal redness, degree of tonsil swelling, and presence or absence of white coating. This eliminates the need for the operator to manually input medical history information and attribute information.

[0123] The following describes a specific example of obtaining characteristic features of pharyngeal follicles as part of the medical history information. As mentioned above, follicles appearing in the pharynx are a characteristic sign of influenza and are confirmed by visual examination during diagnosis by physicians. Therefore, physicians perform a labeling process on training subject images to identify areas of interest such as follicles through manual input. The positional information (shape information) of the labels in the training subject images is then acquired as training positional information, and a trained region extraction model is obtained by machine learning the training subject images and the pair of training subject images and their labeled training positional information using a neural network. The processor 111 then provides the subject images as input to the trained region extraction model, outputting positional information (shape information) of the areas of interest (i.e., follicles). Subsequently, the processor 111 stores the obtained follicle positional information (shape information) as medical history information.

[0124] Furthermore, as another specific example of medical information, let's explain how to obtain heart rate. First, the processor 111 receives a predetermined period of time for capturing subject images. Then, the processor 111 extracts each RGB color component from each frame that makes up the received video and obtains the brightness of the G (green) component. The processor 111 generates a brightness waveform of the G component in the video from the brightness of the G component of each obtained frame, and estimates the heart rate from its peak value. This method utilizes the fact that hemoglobin in the blood absorbs green light, but of course, the heart rate may be estimated by other methods. The processor 111 then stores the estimated heart rate as medical information.

[0125] In the example shown in Figure 14, the case where the judgment image read from memory 112 is preprocessed in S612 and then given as input to the feature extractor was explained. However, preprocessing is not always necessary. For example, the processor 111 may read the judgment image from memory 112 and give the read judgment image as input to the feature extractor without preprocessing. Even when preprocessing is performed, the processor 111 may give both the preprocessed judgment image and the unprocessed judgment image as input to the feature extractor.

[0126] Furthermore, in the generation of each trained model shown in Figures 15 to 17, the same preprocessed judgment images as in S612 of Figure 14 were used as training data. However, considering cases where no preprocessing is performed in Figure 14 as described above, or where both the preprocessed and unprocessed judgment images are used as judgment images, unprocessed judgment images may also be used as training data.

[0127] The trained models described in Figures 13, 15-17, etc., were generated using neural networks or convolutional neural networks. However, they can also be generated using machine learning methods other than these, such as nearest neighbor, decision trees, regression trees, and random forests.

[0128] In the example shown in Figure 8, the processing unit 100 acquires attribute information and medical history information, selects judgment images, performs judgment processing, and outputs judgment results, while the imaging device 200 captures subject images. However, these various processes can be appropriately distributed and processed by the processing unit 100, the imaging device 200, and other devices. Figure 20 is a schematic diagram of a processing system 1 according to one embodiment of the present disclosure. Specifically, Figure 20 is a diagram showing an example of the connection of various devices that may constitute the processing system 1. According to Figure 20, the processing system 1 includes a processing unit 100, an imaging device 200, a terminal device 810 such as a smartphone, tablet, or laptop PC, an electronic medical record device 820, and a server device 830, which are connected to each other via a wired or wireless network. Note that each device listed in Figure 20 does not necessarily have to be provided, and may be provided as appropriate according to the example of processing distribution shown below.

[0129] Instead of the example in Figure 8, in the processing system 1 illustrated in Figure 20, various processes can be distributed as follows. (1) All processes, including capturing images of the subject, acquiring attribute information and medical interview information, selecting images for judgment, judgment processing, and outputting the judgment results, are performed by the imaging device 200. (2) The camera 200 captures images of the subject and outputs the judgment results, while the server 830 (cloud server) performs machine learning-based processing such as selecting judgment images and judgment processing. (3) The terminal device 810 inputs medical interview information and attribute information, the processing device 100 selects the judgment image, performs the judgment processing and outputs the judgment result, and the shooting device 200 captures the subject image. (4) The imaging device 200 performs the input of medical interview information and attribute information, and the capture of subject images, while the processing device 100 performs the selection of judgment images, judgment processing, and output of judgment results. (5) The terminal device 810 performs input of medical interview information and attribute information and output of judgment results, the processing device 100 performs selection of judgment images and judgment processing, and the shooting device 200 performs capturing of subject images. (6) The electronic medical record device 820 inputs the medical history information and attribute information, the processing device 100 selects the judgment image and performs the judgment processing, the imaging device 200 captures the subject image, and the terminal device 810 outputs the judgment result. (7) The electronic medical record device 820 performs input of medical history information and attribute information, and output of judgment results, the processing device 100 performs selection of judgment images and judgment processing, and the imaging device 200 performs capturing of subject images. (8) The terminal device 810 performs input of medical interview information and attribute information and output of judgment results, the server device 830 performs selection of judgment images and judgment processing, and the shooting device 200 performs capturing of subject images. (9) The terminal device 810 and the electronic medical record device 820 perform input of medical history information and attribute information, and output of judgment results, the server device 830 performs selection of judgment images and judgment processing, and the imaging device 200 performs capturing of subject images. (10) The electronic medical record device 820 performs input of medical history information and attribute information, and output of judgment results, the server device 830 performs selection of judgment images and judgment processing, and the imaging device 200 performs capturing of subject images.

[0130] Let's specifically explain one example of the distributed processing described above. The processing related to S11 to S15 shown in Figure 8 is executed on terminal devices 810, such as a smartphone held by a patient, a tablet used in a medical institution, or a laptop PC used by a doctor. After that, when the processing related to S21 to S24 is executed on the imaging device 200, the subject image is transmitted to the server device 830 via terminal device 810 or directly. Upon receiving the subject image, the server device 830 executes the processing related to S31 to S32, and the judgment result is output from the server device 830 to the terminal device 810. Upon receiving the judgment result, the terminal device 810 stores the result in memory and displays it on the display.

[0131] It should be noted that the above is merely one example of processing distribution. Furthermore, in this disclosure, the processing unit 100 is referred to as a processing unit. However, this is simply because the processing unit 100 performs various processes related to judgment processing, etc. For example, if various processes are performed by the imaging device 200, terminal device 810, electronic medical record device 820, server device 830, etc., these also function as processing units and may be referred to as processing units.

[0132] In the example shown in Figure 3, a roughly cylindrical imaging device 200 was used to capture images of the subject. However, this is not the only option; for example, a terminal device 810 can be used as the imaging device, and images of the subject can be captured using the camera provided in the terminal device 810. In such a case, the camera would not be inserted into the oral cavity near the pharynx, but would be positioned outside the incisors (outside the body) to capture images of the oral cavity.

[0133] These modifications are similar in configuration, processing, and procedure to the embodiment described in Figures 1 to 19, except for the points specifically described above. Therefore, a detailed explanation of those matters will be omitted. It is also possible to configure the system by appropriately combining or substituting the elements described in each modification and embodiment.

[0134] The processes and procedures described herein can be implemented not only by those explicitly described in the embodiments, but also by software, hardware, or a combination thereof. Specifically, the processes and procedures described herein can be implemented by implementing the logic corresponding to the process on a medium such as an integrated circuit, volatile memory, non-volatile memory, magnetic disk, or optical storage. Furthermore, the processes and procedures described herein can be implemented as computer programs and executed by various computers, including processing units and server devices.

[0135] Even if it is stated that the processes and procedures described herein are performed by a single device, software, component, or module, such processes or procedures may be performed by multiple devices, multiple software programs, multiple components, and / or multiple modules. Similarly, even if it is stated that the various types of information described herein are stored in a single memory or storage unit, such information may be distributed and stored in multiple memories within a single device or in multiple memories distributed across multiple devices. Furthermore, the software and hardware elements described herein may be implemented by integrating them into fewer components or by decomposing them into more components. [Explanation of Symbols]

[0136] 1. Processing System 100 Processing Units 200 imaging devices 300 assistive devices 400 mounting platform 600 Operator 700 users 810 Terminal device 820 Electronic medical record system 830 Server Equipment

Claims

1. Includes at least one processor, The at least one processor, One or more determination images of a subject are obtained via a camera used to capture images of the subject that include at least a portion of the user's oral cavity. Based on a trained judgment model stored in memory to determine the likelihood of contracting a predetermined disease, and the acquired judgment images, the likelihood of contracting the predetermined disease is determined. Output information indicating the likelihood of the determined disease. A processing device configured to perform processing for the purpose of

2. The apparatus according to claim 1, wherein the subject includes at least the pharynx.

3. The apparatus according to claim 1, wherein the subject includes at least the tonsils.

4. The processing device according to any one of claims 1 to 3, wherein the one or more judgment images are obtained by inputting the one or more subject images into a trained judgment image selection model stored in the memory for selecting the one or more judgment images from the one or more subject images captured by the camera.

5. The apparatus according to claim 4, wherein the trained judgment image selection model is obtained by training using training subject images of the subject and label information indicating whether or not the training subject images can be used to determine the possibility of disease.

6. The processing apparatus according to any one of claims 1 to 3, wherein the one or more determination images are obtained from a plurality of subject images captured by the camera based on the similarity between each subject image.

7. The aforementioned processor, The determination of the likelihood of contracting the aforementioned predetermined disease is performed by inputting the one or more determination images into a trained feature extractor for extracting predetermined features from the one or more determination images and calculating the predetermined features, Based on the predetermined features and the trained judgment model, the likelihood of contracting the predetermined disease is determined. The apparatus according to any one of claims 1 to 6.

8. The aforementioned processor, The user's medical interview information and attribute information are obtained, In addition to the trained judgment model and the one or more judgment images, the likelihood of contracting the predetermined disease is determined based on at least one of the medical interview information and the attribute information. The apparatus according to any one of claims 1 to 7.

9. The aforementioned processor, The user's medical interview information and attribute information are obtained, Without using either the aforementioned medical interview information or attribute information, the first probability of contracting the predetermined disease is determined based on the trained judgment model and the one or more judgment images. In addition to the trained judgment model and the one or more judgment images, the second possibility of contracting the predetermined disease is determined based on at least one of the medical interview information and the attribute information. Based on the first possibility and the second possibility, information indicating the possibility of the disease is obtained. The apparatus according to any one of claims 1 to 7.

10. The processing apparatus according to claim 8 or 9, wherein the attribute information is information obtained from one or more subject images captured by the camera.

11. The aforementioned processor, Obtain external factor information related to the aforementioned predetermined disease, In addition to the trained judgment model and the one or more judgment images, the possibility of contracting the predetermined disease is determined based on the external factor information. The apparatus according to any one of claims 1 to 10.

12. The processing apparatus according to any one of claims 1 to 11, wherein the processor determines the possibility of contracting the predetermined disease based on at least two or more judgment images.

13. By being executed by at least one processor, One or more determination images of a subject are obtained via a camera used to capture images of the subject that include at least a portion of the user's oral cavity. Based on a trained judgment model stored in memory to determine the likelihood of contracting a predetermined disease, and the acquired judgment images, the likelihood of contracting the predetermined disease is determined. Output information indicating the likelihood of the determined disease. A processing program that causes the aforementioned at least one processor to function in this manner.

14. A processing method that is performed by at least one processor, A step of acquiring one or more determination images of a subject via a camera for taking images of the subject that include at least a portion of the user's oral cavity, A step of determining the likelihood of contracting a predetermined disease based on a trained judgment model stored in memory for determining the likelihood of contracting a predetermined disease and one or more judgment images acquired, A step of outputting information indicating the likelihood of the determined disease, A processing method that includes this.

15. A photographing device equipped with a camera for taking an image of a subject that includes at least a portion of the user's oral cavity, A processing device according to any one of claims 1 to 12, connected to the aforementioned imaging device via a wired or wireless network, A processing system that includes this.

Images