Imaging apparatus, program and method

Abstract

[Problem] To provide an imaging device more suitable for capturing an image of a subject that includes at least a portion of a natural opening of a subject person. [Solution] Provided is an imaging device which is configured such that at least one processor, when receiving, via a communication interface, subject person information including first subject person information and associated with each of one or more subject persons including a first subject person, from an external device communicably connected via a network to the imaging device, and outputting a list of the subject person information regarding a subject person among the one or more subject persons who has not been imaged, performs processing for outputting the list without including the first subject person information associated with the first subject person therein before the first subject person information is registered in the external device, and outputting the list by including the first subject person information therein after the first subject person information is registered in the external device.

Description

[Technical Field]

[0001] The present disclosure relates to an imaging apparatus, a program, and a method that include a camera configured to capture an image of a subject.

[0002] Conventionally, it has been known that doctors observe changes in the oral condition of a subject to diagnose conditions such as viral colds. Non-Patent Document 1 reports that lymphoid follicles appearing in the deepest part of the pharynx located in the oral cavity have a pattern specific to influenza. Lymphoid follicles having this specific pattern are called influenza follicles, which are characteristic signs of influenza and are said to appear approximately 2 hours after onset. Therefore, acquiring an image capturing the oral condition of a subject is extremely important. [Prior Art Literature] [Non-Patent Literature]

[0003] [Non-Patent Document 1] Miyamoto and Watanabe, "Consideration on the Meaning and Value of Pharyngeal Examination Findings (Influenza Follicles)", Nihon Daigaku Igaku Zasshi 72(1): 11-18 (2013) [Summary of the Invention] [Problem to be Solved by the Invention]

[0004] Accordingly, based on the above techniques, the present disclosure aims to provide an imaging apparatus, a program, and a method that are more suitable for capturing an image of a subject including at least a part of the subject's natural opening, according to various embodiments. [Means for Solving the Problem]

[0005] According to one aspect of the present disclosure, a photographing device is provided, comprising a camera configured to capture an image of a subject including at least a portion of the natural aperture of one or more subjects, including a first subject, and at least one processor, wherein the at least one processor is configured to receive subject information, including first subject information associated with each of one or more subjects, including the first subject, via a communication interface from an external device that is communicably connected to the photographing device via a network, and to output the subject information as a list for subjects among the one or more subjects that have not been photographed, outputting the list without including the first subject information before the first subject information associated with the first subject is registered in the external device, and outputting the list including the first subject information after the first subject information is registered in the external device.

[0006] According to one aspect of the present disclosure, a program is provided that causes a camera configured to capture an image of a subject including at least a portion of the natural aperture of one or more subjects, including a first subject, to function as a processor for receiving subject information, including first subject information associated with each of one or more subjects including the first subject, via a communication interface from an external device that is communicably connected to the camera via a network, and outputting the subject information as a list for subjects among the one or more subjects that have not been photographed, wherein the list is output without including the first subject information before the first subject information associated with the first subject is registered in the external device, and the list is output with the first subject information included after the first subject information is registered in the external device.

[0007] According to one aspect of the present disclosure, a method is provided for a photographing device including a camera configured to capture an image of a subject including at least a portion of the natural aperture of one or more subjects, including a first subject, and at least one processor, the method being performed by the at least one processor, the method being to receive subject information, including first subject information associated with each of one or more subjects, including the first subject, via a communication interface from an external device that is communicably connected to the photographing device via a network; and when outputting the subject information as a list for subjects among the one or more subjects that have not been photographed, the method being to output the list without including the first subject information before the first subject information associated with the first subject is registered in the external device, and the method being to output the list including the first subject information after the first subject information is registered in the external device. [Effects of the Invention]

[0008] This disclosure provides a more suitable imaging apparatus, program, and method for capturing images of a subject that include at least a portion of the subject's natural aperture.

[0009] The effects described above are merely illustrative for the sake of explanation and are not limiting. In addition to, or in lieu of, any other effects described herein or that would be obvious to those skilled in the art may be achieved. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 shows the state of use of the imaging device 200 according to one embodiment of the present disclosure. [Figure 2] Figure 2 shows the state of use of the imaging device 200 according to one embodiment of the present disclosure. [Figure 3] Figure 3 is a block diagram showing the configuration of a processing system 1 according to one embodiment of the present disclosure. [Figure 4]Figure 4 is a block diagram showing the configuration of a processing apparatus 100 and an imaging apparatus 200 according to one embodiment of the present disclosure. [Figure 5] Figure 5 is a block diagram showing the configuration of a server device 300 according to one embodiment of the present disclosure. [Figure 6A] Figure 6A is a conceptual diagram showing a target management table stored in a server device 300 according to one embodiment of the present disclosure. [Figure 6B] Figure 6B is a conceptual diagram showing an image management table stored in a server device 300 according to one embodiment of the present disclosure. [Figure 7] Figure 7 shows a processing sequence executed between a processing device 100, an imaging device 200, and a server device 300 according to one embodiment of the present disclosure. [Figure 8A] Figure 8A is a diagram showing the processing flow performed in the processing apparatus 100 according to one embodiment of the present disclosure. [Figure 8B] Figure 8B is a diagram showing the processing flow performed in the processing apparatus 100 according to one embodiment of the present disclosure. [Figure 8C] Figure 8C is a diagram showing the processing flow performed in the processing apparatus 100 according to one embodiment of the present disclosure. [Figure 9A] Figure 9A is a diagram showing the processing flow executed in a server device 300 according to one embodiment of the present disclosure. [Figure 9B] Figure 9B is a diagram showing the processing flow executed in a server device 300 according to one embodiment of the present disclosure. [Figure 9C] Figure 9C is a diagram showing the processing flow for generating a trained model according to one embodiment of the present disclosure. [Figure 9D] Figure 9D is a diagram showing the processing flow for generating a trained model according to one embodiment of this disclosure. [Figure 10] Figure 10 is a diagram showing the processing flow performed in a photographing device 200 according to one embodiment of the present disclosure. [Figure 11A] Figure 11A is a diagram showing an example of a screen displayed in a processing apparatus 100 according to one embodiment of the present disclosure. [Figure 11B] FIG. 11B is a diagram illustrating an example of a screen displayed on a processing apparatus 100 according to an embodiment of the present disclosure. [Figure 11C] FIG. 11C is a diagram illustrating an example of a screen displayed on the processing apparatus 100 according to an embodiment of the present disclosure. [Figure 12A] FIG. 12A is a diagram illustrating an example of a screen displayed on an imaging apparatus 200 according to an embodiment of the present disclosure. [Figure 12B] FIG. 12B is a diagram illustrating an example of a screen displayed on the imaging apparatus 200 according to an embodiment of the present disclosure. [Figure 12C] FIG. 12C is a diagram illustrating an example of a screen displayed on the imaging apparatus 200 according to an embodiment of the present disclosure. [Figure 12D] FIG. 12D is a diagram illustrating an example of a screen displayed on the imaging apparatus 200 according to an embodiment of the present disclosure. [Figure 12E] FIG. 12E is a diagram illustrating an example of a screen displayed on the imaging apparatus 200 according to an embodiment of the present disclosure. [Figure 12F] FIG. 12F is a diagram illustrating an example of a screen displayed on the imaging apparatus 200 according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF EMBODIMENTS

[0011] <First Embodiment> 1. Overview of Processing System 1 A processing system 1 according to the present disclosure is mainly used for imaging the inside of the oral cavity of a subject to obtain a subject image. In particular, the processing system 1 is used for imaging the area around the deep throat of the oral cavity, specifically the pharynx. Accordingly, the following description mainly focuses on the case where the processing system 1 according to the present disclosure is used for imaging the pharynx. However, the pharynx is merely an example of an imaging site, and the processing system 1 according to the present disclosure can naturally be suitably used for other sites in the oral cavity such as the tonsils and larynx, as well as other natural openings such as the external auditory canal, vagina, rectum, and nasal cavity.

[0012] The processing system 1 described herein is used, for example, to determine the likelihood of a subject having a predetermined disease from a subject image obtained by photographing a subject that includes at least the pharyngeal region of the subject's oral cavity, and to diagnose or assist in the diagnosis of the predetermined disease. An example of a disease that can be determined by the processing system 1 is influenza. Normally, the likelihood of having influenza is diagnosed by examining the subject's pharynx and tonsil region, or by determining the presence or absence of findings such as follicles in the pharyngeal region. However, by using the processing system 1 to determine the likelihood of having influenza and outputting the result, it becomes possible to diagnose or assist in the diagnosis. The determination of the likelihood of having influenza is just one example. The processing system 1 can be suitably used to determine any disease in which a difference in the findings of the natural opening appears when the subject is infected. Note that the difference in findings is not limited to those discovered by a physician or other medical professional and whose existence is medically known. For example, any difference that can be recognized by a person other than a physician, or any difference that can be detected by artificial intelligence or image recognition technology, can be suitably applied to the processing system 1.

[0013] Examples of such diseases, primarily those diagnosed from images of the oral cavity, pharynx, and larynx among the natural openings, include, in addition to influenza, infectious diseases such as streptococcal infection, adenovirus infection, EB virus infection, mycoplasma infection, hand-foot-and-mouth disease, herpangina, and candidiasis; diseases exhibiting vascular or mucosal disorders such as arteriosclerosis, diabetes, and hypertension; tumors such as oral cancer, tongue cancer, and pharyngeal cancer; and trauma. Diseases diagnosed from images of the external auditory canal among the natural openings include tumors such as middle ear cancer and auditory organ cancer; inflammation such as otitis media and tympaniitis; tympanic membrane diseases such as tympanic membrane perforation; and trauma. Diseases diagnosed from images of the nasal cavity among the natural openings include inflammation such as rhinitis, infectious diseases such as sinusitis, tumors such as nasal cavity cancer, trauma, epistaxis, and diseases exhibiting vascular or mucosal disorders such as granulomatosis with polyangiitis. Furthermore, diseases that can be identified from images of the vagina, which is a natural opening, include tumors such as cervical cancer, diseases that cause dryness such as Sjögren's syndrome, diseases that cause mucosal damage or bleeding such as vaginal erosion, trauma, inflammation such as vaginitis, infections such as vaginal candidiasis, and diseases that cause ulcers or skin lesions such as Behçet's disease. Similarly, diseases that can be identified from images of the rectum, which is a natural opening, include tumors such as rectal cancer, diseases that cause mucosal damage or bleeding such as ulcerative colitis, infections such as enteritis, and trauma.

[0014] In this disclosure, terms such as "judgment" and "diagnosis" are used in relation to diseases, but these do not necessarily mean a definitive judgment or diagnosis by a physician. For example, it is certainly possible that the subject may use the processing system 1 of this disclosure themselves, or that an operator other than a physician may use it, and the processing device 100 included in the processing system 1 may make a judgment or diagnosis.

[0015] Furthermore, in this disclosure, the subjects of imaging by the imaging device 200 may include any person, such as patients, subjects of examination, people undergoing diagnosis, and healthy individuals. Also, in this disclosure, the operator who holds the imaging device 200 and performs the imaging operation may include any person, not limited to medical professionals such as doctors, nurses, and laboratory technicians, but including the subject themselves. The processing system 1 related to this disclosure is typically envisioned to be used in a medical institution. However, it is not limited to this case, and its use location may be any other place, such as the subject's home, school, or workplace.

[0016] Furthermore, as stated above, in this disclosure, the subject only needs to include at least a portion of the subject's natural openings. The disease being assessed can also be any disease that shows a difference in findings in the natural openings of the subject. However, the following description will focus on a case where the subject includes at least a portion of the oral cavity, particularly the pharynx or surrounding area, and the disease being assessed is the possibility of contracting influenza.

[0017] Furthermore, in this disclosure, the subject image may be one or more videos or one or more still images. As an example of operation, when the power button is pressed, the camera captures a through image, and the captured through image is displayed on the display 203. Then, when the operator presses the shutter button, one or more still images are taken by the camera and the taken images are displayed on the display 203. Alternatively, when the subject presses the shutter button, video recording begins, and the images being taken by the camera are displayed on the display 203. Then, when the shutter button is pressed again, video recording ends. In this way, in a series of operations, various images such as through images, still images, and videos are taken by the camera and displayed on the display, but the subject image does not mean only a specific image among these, but may include all images taken by the camera.

[0018] Figure 1 is a diagram showing the state of use of an imaging device 200 according to one embodiment of the present disclosure. According to Figure 1, the operator attaches the auxiliary device 400 to the tip of the imaging device 200 so as to cover it, and inserts the imaging device 200 together with the auxiliary device 400 into the oral cavity 712 of the subject 700. Specifically, first, the operator (which may be the subject 700 themselves or someone other than the subject 700) attaches the auxiliary device 400 to the tip of the imaging device 200 so as to cover it. Then, the operator inserts the imaging device 200 with the auxiliary device 400 attached into the oral cavity 712. At this time, the tip of the auxiliary device 400 passes through the incisors 711 and is inserted to the vicinity of the soft palate 713. In other words, the imaging device 200 is also inserted to the vicinity of the soft palate 713. At this time, the tongue 714 is pushed downward by the auxiliary device 400 (which functions as a tongue depressor), and the movement of the tongue 714 is restricted. This allows the operator to ensure a good field of view for the imaging device 200 and to take good images of the pharynx 715 located in front of the imaging device 200.

[0019] The captured subject image (typically an image including the pharynx 715) is transmitted from the imaging device 200 to a server device 300 that is connected via a wired or wireless network. The processor of the server device 300, upon receiving the subject image, processes a program stored in memory to determine whether the subject image is suitable for determining the possibility of contracting a predetermined disease, and the possibility of contracting the predetermined disease is determined from the subject image. The result is then transmitted to the processing device 100 and output to a display or the like via the output interface of the processing device 100.

[0020] Figure 2 shows a state of use of an imaging device 200 according to one embodiment of the present disclosure. Specifically, Figure 2 shows a state in which an operator 600 is holding the imaging device 200. According to Figure 2, the imaging device 200 consists of a main body 201, a grip 202, and a display 203, from the side inserted into the oral cavity. The main body 201 and grip 202 are formed in a substantially columnar shape of a predetermined length along the insertion direction H into the oral cavity. The display 203 is positioned on the opposite side of the grip 202 from the main body 201 side. Therefore, the imaging device 200 as a whole is formed in a substantially columnar shape and is held by the operator 600 in a manner similar to holding a pencil. In other words, in the state of use, the display panel of the display 203 faces the operator 600, making it possible to easily handle the imaging device 200 while checking the subject image captured by the imaging device 200 in real time.

[0021] Furthermore, when the operator 600 holds the grip 202 with the subject image displayed on the display 203 in the normal orientation, the shooting button 220 is positioned on the upper side of the grip. Therefore, when the operator 600 holds the grip, they can easily press the shooting button 220 with their index finger or the like.

[0022] 2. Configuration of Processing System 1 Figure 3 is a schematic diagram of a processing system 1 according to one embodiment of the present disclosure. According to Figure 3, the processing system 1 includes a processing unit 100, an imaging device 200, and a server device 300, and each device is connected to communicate via a wired or wireless network. The processing unit 100 receives input such as subject information, medical history information, and findings information. The processing unit 100 also outputs subject images captured by the imaging device 200. Furthermore, the processing unit 100 receives and outputs the results of a determination of the possibility of contracting a predetermined disease based on the subject images captured by the imaging device 200 from the server device 300.

[0023] The imaging device 200 has its tip inserted into the subject's oral cavity to photograph the inside of the oral cavity, particularly the pharynx. The specific imaging process will be described later. The captured subject images are transmitted to the server device 300 via a wired or wireless network.

[0024] The server device 300 receives and manages subject information, medical history information, diagnostic information, etc., input to the processing device 100, and also receives and manages images of subjects captured by the imaging device 200. Furthermore, the server device 300 determines whether the received images of subjects are suitable for subsequent processing and transmits the result to the imaging device 200. It also determines the possibility of contracting a predetermined disease based on the subject images, medical history information, and findings information and transmits the result to the processing device 100.

[0025] In this disclosure, "external device" means the processing unit 100 or other processing units, the server unit 300 or other server units, or a combination thereof. In other words, unless otherwise specified, "external device" may include the processing unit 100, the server unit 300, or any combination thereof.

[0026] Figure 4 is a block diagram showing the configuration of a processing unit 100 and an imaging device 200 according to one embodiment of the present disclosure. Specifically, Figure 4 is a diagram specifically showing the configuration of the processing unit 100 and the imaging device 200 of System 1. According to Figure 4, the processing unit 100 includes a processor 111, a memory 112, an input interface 113, an output interface 114, and a communication interface 115. The imaging device 200 includes a camera 211, a light source 212, a processor 213, a memory 214, an output interface 215, an input interface 210, and a communication interface 216. Each of these components is electrically connected to each other via control lines and data lines. Note that the processing unit 100 and the imaging device 200 do not need to include all the components shown in Figure 4; they can be configured with some components omitted, or other components can be added. For example, the processing unit 100 or the imaging device 200 can include a battery to power each component. Furthermore, as explained in Figure 3, the processing unit 100 and the imaging device 200 only need to be connected in a way that allows them to communicate via a wired or wireless network; they do not need to be configured to communicate directly with each other.

[0027] 2-1. Configuration of the processing unit 100 First, among the processing units 100, the processor 111 functions as a control unit that controls other components of the processing system 1 based on a program stored in the memory 112. Based on the program stored in the memory 112, the processor 111 performs processing related to the input of subject information, medical history information, findings information, etc., the output of subject images taken by the imaging device 200, and the output of the determination result of the possibility of contracting a predetermined disease. Specifically, the processor 111 performs the following: "processing to receive input of subject information related to the subject by the operator via the input interface 113", "processing to transmit the received subject information to the server device 300 via the communication interface 115", "processing to receive input of medical history information of the subject by the operator or the subject via the input interface 113", "processing to transmit the received medical history information together with the subject information to the server device 300 via the communication interface 115", "processing to receive input of subject information related to the subject by the operator via the input interface 113", "communication interface 115 The following processes are executed based on a program stored in memory 112: "a process to transmit the received subject information to the server device 300 via the input interface 113," "a process to receive input of subject observation information from the operator via the input interface 113," "a process to transmit the received observation information together with the subject information to the server device 300 via the communication interface 115," and "a process to receive from the server device 300 via the communication interface 115 a judgment result indicating the possibility of contracting a predetermined disease determined based on the subject image of the subject, and the subject image, and output the judgment result and the subject image together via the output interface 114." The processor 111 is mainly composed of one or more CPUs, but a GPU or FPGA may be combined as appropriate.

[0028] Memory 112 is composed of RAM, ROM, non-volatile memory, HDD, etc., and functions as a storage unit. Memory 112 stores instruction commands for various controls of the processing system 1 according to this embodiment as programs. Specifically, memory 112 performs the following processes: "receiving input of subject information related to the subject by the operator via the input interface 113", "transmitting the received subject information to the server device 300 via the communication interface 115", "receiving input of subject medical interview information by the operator or subject via the input interface 113", "transmitting the received medical interview information together with the subject information to the server device 300 via the communication interface 115", "receiving input of subject information related to the subject by the operator via the input interface 113", and "transmitting the communication interface 115 The memory 112 stores programs for the processor 111 to execute, such as "a process to transmit the received subject information to the server device 300 via the input interface 113", "a process to receive input of subject observation information from the operator via the input interface 113", "a process to transmit the received observation information together with the subject information to the server device 300 via the communication interface 115", and "a process to receive a judgment result indicating the possibility of contracting a predetermined disease determined based on the subject image, etc., from the server device 300 via the communication interface 115, and output the judgment result and subject image together via the output interface 114". In addition to these programs, the memory 112 also stores the subject's information, subject image, medical history information, observation information, etc.

[0029] The input interface 113 functions as an input unit that receives instructions from an operator to the processing unit 100. Examples of physical keys for the input interface 113 include a "confirm button" for making various selections, a "back / cancel button" for returning to the previous screen or canceling a confirmed operation, a directional pad for moving a pointer output to the output interface 114, an on / off key for turning the processing unit 100 on and off, and character input keys for entering various characters. It is also possible to use a touch panel for the input interface 113, which is superimposed on the display that functions as the output interface 114 and has an input coordinate system corresponding to the display coordinate system of the display. In this case, icons corresponding to the above physical keys are displayed on the display, and the operator makes a selection for each icon by giving instructions via the touch panel. The method for detecting the user's instructions via the touch panel can be any method, such as capacitive or resistive touch. In addition to the above, a mouse or keyboard can also be used as the input interface 113. The input interface 113 does not always need to be physically provided on the processing unit 100 and may be connected as needed via a wired or wireless network.

[0030] The output interface 114 functions as an output unit for outputting information such as judgment results received from the server device 300. An example of the output interface 114 is a display composed of an LCD panel, an organic EL display, or a plasma display. However, the processing unit 100 itself does not necessarily need to be equipped with a display. For example, an interface for connecting to a display that can be connected to the processing unit 100 via a wired or wireless network can also function as the output interface 114 for outputting display data to the display.

[0031] The communication interface 115 functions as a communication unit for sending and receiving subject information, medical interview information, subject images, findings information, etc., with the server device 300 connected via a wired or wireless network. Examples of the communication interface 115 include wired communication connectors such as USB and SCSI, wireless communication transceivers such as wireless LAN, Bluetooth®, and infrared, and various connection terminals for printed circuit boards and flexible circuit boards.

[0032] 2-2. Configuration of the imaging device 200 Of the imaging device 200, the camera 211 functions as an imaging unit that detects reflected light reflected from the oral cavity, which is the subject, and generates an image of the subject. To detect this light, the camera 211 includes, for example, a CMOS image sensor, and a lens system and drive system to realize the desired function. The image sensor is not limited to a CMOS image sensor; other sensors such as a CCD image sensor can also be used. Although not specifically shown in the figures, the camera 211 may have an autofocus function, and it is preferable that the focus be set to a specific area in front of the lens, for example. The camera 211 may also have a zoom function, and it is preferable that it is set to capture images at an appropriate magnification depending on the size of the pharynx or influenza follicles.

[0033] It is known that lymphoid follicles appearing in the deepest part of the pharynx, located within the oral cavity, exhibit a pattern specific to influenza. These lymphoid follicles with a specific pattern are called influenza follicles, and they are a characteristic sign of influenza, appearing approximately two hours after the onset of symptoms. As described above, the processing system 1 of this embodiment is used to determine the likelihood of a subject contracting influenza by, for example, photographing the pharynx in the oral cavity and detecting the above-mentioned follicles. Therefore, when the imaging device 200 is inserted into the oral cavity, the distance between the camera 211 and the subject becomes relatively close. Accordingly, it is preferable that the camera 211 has a field of view (2θ) such that the value calculated by [(distance from the tip of the camera 211 to the posterior pharyngeal wall) * tanθ] is 20 mm or more vertically and 40 mm or more horizontally. By using a camera with such a field of view, it becomes possible to photograph a wider range even when the camera 211 and the subject are in close proximity. That is, while it is possible to use a normal camera for the camera 211, it is also possible to use a camera known as a wide-angle camera or an ultra-wide-angle camera.

[0034] Furthermore, in this embodiment, the main subject captured by the camera 211 is the pharynx or influenza follicles formed in the pharyngeal region. Generally, the pharynx is formed in a recessed direction, so if the depth of field is shallow, the focus will shift between the anterior and posterior parts of the pharynx, making it difficult to obtain a subject image suitable for use in the processing device 100. Therefore, the camera 211 has a depth of field of at least 20 mm, preferably 30 mm or more. By using a camera with such a depth of field, it is possible to obtain a subject image that is in focus at any point from the anterior to the posterior part of the pharynx.

[0035] The light source 212 is driven by instructions from the processor 213 of the imaging device 200 and functions as a light source unit for irradiating light into the oral cavity. The light source 212 includes one or more light sources. In this embodiment, the light source 212 is composed of one or more LEDs, and light having a predetermined frequency band is irradiated from each LED toward the oral cavity. The light source 212 uses light having a desired band from the ultraviolet light band, visible light band, infrared light band, or a combination thereof. When the processing device 100 determines the possibility of influenza infection, it is preferable to use light in the visible light band.

[0036] The processor 213 functions as a control unit that controls other components of the imaging device 200 based on a program stored in the memory 214. Based on the program stored in the memory 214, the processor 213 performs the following processes: "receiving subject information, including first subject information associated with each of one or more subjects, including a first subject, from an external device (processing device 100 or server device 300) that is communicably connected to the imaging device 200 via a network, via the communication interface 216"; "when outputting subject information as a list for subjects that have not been photographed among one or more subjects, outputting the list without including the first subject information before the first subject information associated with the first subject is registered in the external device, and outputting the list with the first subject information after the first subject information is registered in the external device"; and "outputting the list with the first subject information included". The following processes are executed based on a program stored in memory 214: "When a selection for the first subject information is received from the generated list, output the attribute information of the first subject"; "After a selection for the first subject information is received from the output list including the first subject information, when a first subject image including at least a part of the oral cavity of the first subject is captured by the camera, transmit the first subject information and the first subject image in association to an external device"; "Receive discrimination information from the external device indicating whether or not the first subject image is suitable for use in the aforementioned determination, and output the received discrimination information"; "Output an attachment indicator prompting the attachment of the auxiliary device 400 which covers at least a part of the shooting device and is inserted into the oral cavity together with a part of it." The processor 213 is mainly composed of one or more CPUs, but a GPU or FPGA may be combined as appropriate.

[0037] Memory 214 is composed of RAM, ROM, non-volatile memory, HDD, etc., and functions as a storage unit. Memory 214 stores instruction commands for various controls of the processing system 1 according to this embodiment as programs. Specifically, memory 214 performs the following processes: "receiving subject information, including first subject information associated with each of one or more subjects, including a first subject, via the communication interface 216 from an external device (processing device 100 or server device 300) that is communicably connected to the imaging device 200 via a network"; "when outputting subject information as a list for subjects that have not been photographed among one or more subjects, outputting the list without including the first subject information before the first subject information associated with the first subject is registered in the external device, and outputting the list including the first subject information after the first subject information is registered in the external device"; and "from the list output including the first subject information, The memory 213 stores programs for execution, such as: "When a selection for subject information is received, the process outputs attribute information of the first subject"; "After receiving a selection for the first subject information from a list output including the first subject information, when the camera captures a first subject image that includes at least a part of the oral cavity of the first subject, the process associates the first subject information with the first subject image and transmits it to an external device"; "The process receives discrimination information from an external device indicating whether or not the first subject image is suitable for use in the aforementioned determination, and outputs the received discrimination information"; and "The process outputs an attachment indicator prompting the attachment of an auxiliary device 400 that covers at least a part of the imaging device and is inserted into the oral cavity together with a part of it." In addition to these programs, the memory 214 also stores subject information, subject images, etc.

[0038] The output interface 215 functions as an output unit for outputting subject images and subject information captured by the imaging device 200. An example of the output interface 215 is a display 203, but it is not limited to this and may consist of other liquid crystal panels, organic EL displays, plasma displays, etc. Furthermore, it is not necessary for the display 203 to be provided; for example, an interface for connecting to a display that can be connected to the processing device 100 via a wired or wireless network can also function as the output interface 114 for outputting display data to the said display.

[0039] The input interface 210 functions as an input unit that receives operator instructions for the imaging device 200. Examples of input interface 210 include physical keys such as a "shoot button" for instructing the start and end of recording by the imaging device 200, a "power button" for turning the imaging device 200 on and off, a "confirm button" for making various selections, a "back / cancel button" for returning to the previous screen or canceling a confirmed operation, and a directional pad for moving icons displayed on the output interface 215. These various buttons and keys may be physically provided, or they may be displayed as icons on the output interface 215 and made selectable using a touch panel or the like superimposed on the output interface 215 as the input interface 210. The method for detecting the subject's instruction input via the touch panel may be any method, such as capacitive or resistive touch.

[0040] The communication interface 216 functions as a communication unit for sending and receiving information with the server device 300 and / or other devices. Examples of the communication interface 216 include wired communication connectors such as USB and SCSI, wireless communication transceivers such as wireless LAN, Bluetooth®, and infrared, and various connection terminals for printed circuit boards and flexible circuit boards.

[0041] 2-3. Configuration of Server 300 Figure 5 is a block diagram showing the configuration of a server device 300 according to one embodiment of the present disclosure. According to Figure 5, the server device 300 includes a memory 311, a processor 312, and a communication interface 313. These components are electrically connected to each other via control lines and data lines. Note that the server device 300 does not need to include all the components shown in Figure 5; it is possible to omit some components or add other components. For example, it is possible to connect it to another server device and configure the server device 300 as a single unit. It is also possible to connect it to another database device and configure the server device 300 as a single unit.

[0042] Memory 311 is composed of RAM, ROM, non-volatile memory, HDD, etc., and functions as a storage unit. Memory 311 stores instruction commands for various controls of the processing system 1 according to this embodiment as programs. Specifically, memory 311 performs the following processes: "receiving subject information received by the processing device 100 from the processing device 100 via the communication interface 313," "storing the received subject information in the subject management table in association with subject ID information," "receiving a subject information request from the imaging device 200 via the communication interface 313, and extracting subject information for subjects whose oral cavity imaging has not yet been completed by referring to the subject management table," "transmitting the extracted subject information for un-photographed subjects to the imaging device 200 via the communication interface 313," "receiving subject images taken by the imaging device 200 from the imaging device 200 via the communication interface 313, and storing them in the image management table in association with subject ID information," "determining whether the received subject image is suitable for determining the possibility of suffering from a predetermined disease," and "if it is determined to be appropriate, storing it in the image management table and sending the determination result to the communication interface 3 The processor 312 stores programs for execution, such as: "a process of transmitting to the processing unit 100 and the imaging device 200 via 13"; "a process of receiving a request from the processing unit 100 and transmitting to the processing unit 100 via the communication interface 113 the subject image and information identifying the subject image to be used as the judgment image"; "a process of receiving the medical interview information and findings information entered by the processing unit 100 from the processing unit 100 via the communication interface 313 and storing it in the subject management table in association with the subject ID information"; "a process of reading the subject image associated with the subject from the image management table and the medical interview information and subject information associated with the subject from the subject management table when the processing unit 100 receives a request from the processing unit 100 via the communication interface 313 to determine the possibility of a subject being affected by a predetermined disease for a subject selected by the processing unit 100, and determining the possibility"; and "a process of transmitting the determined result to the processing unit 100 via the communication interface 313".In addition to the program, memory 311 also stores various information stored in the subject management table (Figure 6A) and the image management table (Figure 6B), as well as image data itself, such as subject images.

[0043] The processor 312 functions as a control unit that controls other components of the server device 300 based on a program stored in the memory 311. Based on the program stored in the memory 311, the processor 312 performs processing to determine whether an image is appropriate for determining the possibility of contracting a predetermined disease, and processing to determine the possibility of contracting a predetermined disease.Specifically, the processor 312 performs the following processes: "receiving subject information received by the processing unit 100 from the processing unit 100 via the communication interface 313," "storing the received subject information in the subject management table in association with subject ID information," "receiving a subject information request from the imaging device 200 via the communication interface 313 and extracting subject information for subjects whose oral cavity imaging has not yet been completed by referring to the subject management table," "transmitting the extracted subject information for un-photographed subjects to the imaging device 200 via the communication interface 313," "receiving subject images taken by the imaging device 200 from the imaging device 200 via the communication interface 313 and storing them in the image management table in association with subject ID information," "determining whether the received subject image is suitable for determining the possibility of suffering from a predetermined disease," and "if the determination is deemed appropriate, storing it in the image management table and sending the determination result to the communication interface 313." The following processes are executed based on a program stored in memory 311: "a process of transmitting to the processing unit 100 and the imaging device 200 via 13," "a process of receiving a request from the processing unit 100 and transmitting to the processing unit 100 via the communication interface 113 the subject image and information identifying the subject image to be used as the image for determination," "a process of receiving the medical interview information and findings information entered in the processing unit 100 via the communication interface 313 and storing it in the subject management table in association with the subject ID information," "a process of reading the subject image associated with the subject from the image management table and the medical interview information and subject information associated with the subject from the subject management table when the processing unit 100 receives a request from the processing unit 100 via the communication interface 313 to determine the possibility of a subject being affected by a predetermined disease, and determining the possibility," and "a process of transmitting the determined result to the processing unit 100 via the communication interface 313." The processor 312 is mainly composed of one or more CPUs, but a GPU or FPGA may be combined as appropriate.

[0044] 3. Information stored in the memory 311 of the server device 300 Figure 6A is a conceptual diagram showing a target management table stored in a server device 300 according to one embodiment of this disclosure. The information stored in the target management table is updated as needed in accordance with the progress of processing by the processor 312 of the server device 300.

[0045] According to Figure 6A, the subject management table stores subject name information, gender information, date of birth information, medical history information, findings information, judgment result information, and status information, associated with subject ID information. "Subject ID information" is information unique to each subject and is used to identify each subject. Subject ID information is generated each time a new subject is registered by the operator or the subject themselves. "Subject name information" is information indicating the name of each subject output by the processing device 100 or imaging device 200, and is information indicating an arbitrary string entered by the operator or the subject themselves. "Gender information" is information indicating the gender of the subject and is information selected by the operator or the subject themselves. Preferably, the biological sex of each subject is stored. "Date of birth information" is information indicating the date of birth of the subject. This information is not limited to year, month, and day, but may be information of the year only, or age information may be used instead of date of birth information.

[0046] "Medical history information" is information entered by, for example, the operator or the subject, and is used as reference for diagnosis by doctors, etc., regarding the subject's medical history and symptoms. Examples of such medical history information include patient background such as weight, allergies, and underlying diseases; body temperature, peak body temperature since onset, time elapsed since onset, heart rate, pulse rate, oxygen saturation, blood pressure, medication use, contact with other influenza patients, presence or absence of subjective symptoms and physical findings such as joint pain, muscle pain, headache, fatigue, loss of appetite, chills, sweating, cough, sore throat, runny nose / nasal congestion, tonsillitis, gastrointestinal symptoms, rash on the hands and feet, redness or white coating on the throat, swollen tonsils, history of tonsillectomy, strawberry tongue, swelling of the anterior cervical lymph nodes with tenderness, influenza vaccination history, and vaccination timing. "Findings information" is information entered by operators such as physicians, and it is information that indicates abnormalities obtained through various examinations and tests to assist in diagnosis of the subject, such as visual inspection, interview, palpation, and auscultation. Examples of such findings information include redness or white coating of the pharynx, swelling of the tonsils, presence or absence of tonsillitis, and redness or white coating of the tonsils.

[0047] "Judgment result information" is information that indicates the determination of the possibility of influenza infection based on the medical interview information, findings information, and images for determination. An example of such judgment result information is the positive rate for influenza. However, it is not limited to the positive rate; any information that indicates possibility, such as whether it is positive or negative, is acceptable. Also, the judgment result does not need to be a specific numerical value; the format can be anything, such as a classification according to the level of positive rate or a classification indicating whether it is positive or negative. "Status information" is information that indicates the current status of each subject. Such status information may include "Medical interview incomplete" indicating that the input of medical interview information has not yet been completed, "Not taken" indicating that the subject image has not been taken (i.e., images for determination have not been acquired), "Taken" indicating that the subject image has been taken (i.e., images for determination have been acquired), but the input of findings by a doctor, etc. has not been completed, and "Determined" indicating that the input of findings has been completed and the possibility of infection with the specified disease has been determined. Note that this status information is merely an example, and it is possible to define the status in more detail or in a more general way.

[0048] The information illustrated in Figure 6A is stored as subject information. Of this information, all information except for the subject ID is used as attribute information. In Figure 6A, attribute information is stored in association with the subject ID, but it is not necessary to store all of the information stored here. For example, personal information such as the subject's address, telephone number, and family structure, or information on the current prevalence of infectious diseases such as influenza, may also be included.

[0049] Furthermore, the medical history information and findings information do not need to be input each time by the subject or operator via the processing device 100, etc., but may be received, for example, from an electronic medical record system or other terminal device connected via a wired or wireless network. Alternatively, they may be obtained by analyzing subject images captured by the imaging device 200. In addition, although not specifically shown in Figures 6A and 6B, it is also possible to store in memory 112 current epidemic information of infectious diseases that are the target of diagnosis or assistance in diagnosis, such as influenza, as well as external factor information such as the judgment results and disease status of other subjects regarding these infectious diseases.

[0050] Figure 6B is a conceptual diagram showing an image management table stored in a server device 300 according to one embodiment of this disclosure. The information stored in the image management table is updated as needed in accordance with the progress of processing by the processor 312 of the server device 300.

[0051] According to Figure 6B, the image management table stores subject image information, discrimination result information, and judgment image information, etc., associated with the subject ID information. "Subject ID information" is information unique to each subject and is used to identify each subject. "Subject image information" is information that identifies the subject image taken by the operator for each user. The subject image is one or more images including the subject taken by the camera of the shooting device 200, and is stored in the memory 112 after being received from the shooting device 200. "Discrimination result information" is information indicating the result of the discrimination of whether the received subject image is suitable for determining the possibility of contracting a predetermined disease. "Judgment image information" is information that identifies the judgment image used to determine the possibility of contracting a predetermined disease. As described above, information for identifying each image is stored as subject image information. Thus, the information for identifying the image is typically identification information for identifying each image, but it may also be information indicating the storage location of each image.

[0052] 4. Processing sequence executed by the processing device 100 and the imaging device 200 Figure 7 shows a processing sequence performed between a processing unit 100, an imaging device 200, and a server device 300 according to one embodiment of the present disclosure. Of these, S11 to S14 mainly show a processing sequence related to input processing of subject information etc. performed between the processing unit 100 and the server device 300, S21 to S30 mainly show a processing sequence related to imaging processing performed between the imaging device 200 and the server device 300, and S41 to S46 mainly show a processing sequence related to determination processing performed between the processing unit 100 and the server device 300. For the sake of explanation, in the following, we will use the example of registering a "first subject" as one of the subjects and determining the possibility of disease.

[0053] 4-1. Input processing of target information, etc. As shown in Figure 7, the processing unit 100 accepts the selection of a new registration mode for registering a new subject via the input interface 113 on the subject list screen output via the output interface 114 (S11). Note that the various screens output before the subject list screen are not explained here. When the processing unit 100 outputs the new registration screen to the display via the output interface 114, it outputs the newly generated subject ID information and accepts input of subject name information, gender information, and date of birth information for the first subject via the input interface 113 (S12). The processing unit 100 then associates the accepted subject information (T11) with the subject ID information and transmits it to the server device 300 via the communication interface 115. When the server device 300 receives the subject information for the first subject via the communication interface 313, it associates it with the subject ID information received together and stores it in the subject management table of memory 311 (S13).

[0054] Next, the processing unit 100 outputs the medical interview information input screen for the first subject to the display via the output interface 114, and accepts input of various medical interview information via the input interface 113, such as symptoms for a predetermined disease, weight, allergies, and patient background information such as underlying diseases (S14). The processing unit 100 associates the received medical interview information (T12) with the subject ID information and transmits it to the server device 300 via the communication interface 115. When the server device 300 receives the medical interview information for the first subject via the communication interface 313, it associates it with the subject ID information received together and stores it in the subject management table of the memory 311 (S15). At this time, the server device 300 stores "Not Captured" as status information, indicating that the subject image capture has not been completed (i.e., the image for judgment has not been acquired) because the subject information and medical interview information have been received. This completes the processing sequence related to the input of subject information, etc. In this way, the processing unit 100 inputs and transmits the medical interview information in association with the subject ID information that identifies the subject, and this information is then stored in the server device 200. Therefore, it is possible to accurately associate and manage the relationship between the input medical interview information and the subject.

[0055] In Figure 7, the medical interview information is entered at the time of S14, but this input can be performed at any time. For example, it can be done after the imaging process but before the input of the findings information. Also, although the input of subject information (S12) and the input of medical interview information (S14) are accepted as separate processes, the medical interview input screen for the subject may be displayed immediately after the subject information has been entered, and the input of the medical interview information may be accepted at that time.

[0056] 4-2. Image Processing According to Figure 7, pressing the power button or the like turns on the power of the imaging device 200, and the imaging device 200 is started up (S21). Once started up, the imaging device 200 sends a subject information request (T21) to the server device 300 via the communication interface 216.

[0057] When the server device 300 receives a subject information request (T21), it refers to the status information in the subject management table and searches for subjects marked "not yet photographed," which indicates that the subject image capture has not been completed. The server device 300 then retrieves subject information for each subject, including the first subject, whose status information is recorded as "not yet photographed" (S22). The server device 300 transmits the subject information (T22) for the subjects not yet photographed, including the first subject, to the imaging device 200 via the communication interface 313.

[0058] When the imaging device 200 receives subject information via the communication interface 216, it outputs a list of unphotographed subjects, including the first subject, to the display via the output interface 215 (S23). When the imaging device 200 receives a selection of subject information for a subject to be photographed (in this case, the first subject is used as an example) from the output list via the input interface 210 (S24), it outputs the attribute information of the first subject to the display via the output interface 215. By outputting the attribute information of the first subject to be photographed in this way, it becomes possible to reliably associate the subject ID information of the first subject with the subject image that is subsequently photographed, thereby preventing misidentification of subjects.

[0059] The imaging device 200 determines whether the assistive device 400 is attached, and if it is not yet attached, it displays an attachment indicator via the output interface 215 to prompt attachment (S25). Note that this indicator is merely an example, and attachment may also be prompted by sound, flashing lights, vibration, etc. When the imaging device 200 detects that the assistive device is attached (S26), the imaging device 200 takes a picture of the subject (S27). Once the subject image has been taken, the imaging device 200 transmits the captured subject image (T23) along with the subject ID information of the first subject via the communication interface 216 to the server device 300. Note that although S26 is used to detect the attachment of the assistive device 400, this process itself may be skipped. Alternatively, instead of detecting the attachment of the assistive device 400, a confirmation display for the operator to confirm the attachment of the assistive device 400 may be output via the output interface 215, and a predetermined operation input (e.g., a tap operation) in response to the confirmation display may be accepted by the operator, thereby allowing the operator to input that they have attached the assistive device 400.

[0060] When the server device 300 receives a subject image via the communication interface 313, it stores the received subject image in the image management table in association with the subject ID information received along with the image, and also stores the image data of the received subject image in the memory 311. The server device 300 then determines whether the received subject image is suitable for determining the possibility of having a predetermined disease (S28). If no suitable image is found, the server device 300 sends a notification to the imaging device 200 prompting it to retake the image (not shown). On the other hand, if there is a suitable image (for example, one or more subject images with the highest similarity to the appropriate image), the server device 300 stores the image as a usable image for determination in the determination image information of the image management table in association with the subject ID information of the first subject (S29). Furthermore, the server device 300 refers to the subject management table and updates the status information associated with the subject ID information of the first subject to "Taken," and transmits information (T24) indicating that a judgment image has been obtained as a result of the discrimination, along with the subject ID information of the first subject, to the processing unit and the imaging device 200 via the communication interface 313. Although not specifically shown in the figures, as shown in Figure 12E, the discrimination result may be transmitted to the imaging device 200, and the operator may be prompted to select whether or not to finalize the judgment image. That is, the imaging device 200 may preview the image once, and the server device 300 may finalize the judgment image upon receiving confirmation that the finalization operation has been performed, and then store it in the image management table.

[0061] When the imaging device 200 receives the discrimination result via the communication interface 216, it outputs the result via the output interface 215. It also receives subject information of subjects that have not yet been photographed along with the discrimination result, and outputs it as a list on the display via the output interface 215, similar to S23 (S30). At this time, the status information of the first subject is "Photographed," so it is not included in the subject information of the subjects that have not yet been photographed. Therefore, the subject information of the first subject is not included in the list. This completes the processing sequence related to the series of imaging processes.

[0062] 4-3. Judgment Process As shown in Figure 7, the processing unit 100 receives a selection of a subject to be diagnosed (here, the first subject is used as an example) from the subject information whose status is displayed as "Photographed" on the subject list screen output via the output interface 114, via the input interface 113, and accepts a selection of a diagnostic mode for performing the diagnosis on that subject (S41). Note that the various screens output before the subject list screen are not explained here. The processing unit 100 then sends a subject information request (T41) for the selected first subject to the server device 300 via the communication interface 115.

[0063] When the server device 300 receives a subject information request for the first subject via the communication interface 313, it refers to the subject management table to obtain attribute information of the first subject based on the subject ID information of the first subject, and also refers to the image management table to obtain subject images associated with the subject ID information of the first subject, and judgment image information that identifies the image to be used as the judgment image (S42). Then, the server device 300 transmits the acquired subject images, judgment image information and attribute information, along with the subject ID information of the first subject, to the processing device 100 via the communication interface 313 (T42).

[0064] When the processing unit 100 receives subject images, judgment image information, etc. via the communication interface 115, it outputs the received subject images, judgment image information, and attribute information to the display via the output interface 114. The processing unit 100 then accepts input of findings information from an operator such as a physician via the input interface 113 (S43). When findings information is input, the processing unit 100 transmits a request for determination of the possibility of contracting a predetermined disease (T43) via the communication interface 115, along with the subject ID information of the first subject, the medical history information, findings information, and judgment images.

[0065] In this manner, the server device 300 acquires attribute information, subject images, and judgment image information of the first subject, associated with subject ID information that identifies the subject, and the processing device 100 outputs this information, associated with the subject ID information, before inputting observation information or sending judgment requests. This reduces the risk of misidentifying subjects, such as inputting observation information or sending judgment requests to the wrong subject. Furthermore, the server device 300 can reliably associate each subject with the observation information and judgment results.

[0066] When the server device 300 receives a judgment request from the processing unit 100 via the communication interface 313, it stores the received findings information in the findings information of the subject management table, associating it with the subject ID information of the first subject, and updates the status information. Then, the server device 300 performs a judgment process to determine the possibility of contracting a predetermined disease based on the stored medical history information, findings information, and judgment image (S44). Details of this judgment process will be described later. Note that in this judgment process, the judgment may be performed using only the judgment image without using the medical history information or findings information. Once the judgment process is completed, the server device 300 stores the judgment result information in the subject management table based on the subject ID information of the first subject, and updates the status information to "Judged". The server device 300 transmits the judgment result information (T44) stored in the subject management table, along with the subject ID information of the first subject, to the processing unit 100 via the communication interface 313.

[0067] When the processing unit 100 receives the judgment result information via the communication interface 115, it outputs the received judgment result to the display via the output interface 114 (S45). This completes the processing sequence related to the series of imaging processes. According to this processing sequence, subject information, medical history information, findings information, subject images, and judgment results are acquired by different devices (processing unit 100, imaging device 200, or server device 300, etc.) at different times. However, in S12, new registration of subject information is performed first, and then this information is acquired, and at the time of acquisition, subject ID information is associated with the information and processed. Therefore, it is possible to accurately associate each subject with subject information, medical history information, findings information, subject images, and judgment results, and the risk of mistaking a subject for this information can be reduced.

[0068] Similarly, in the processing flow described in Figure 8A and later, although the input of medical interview information was performed during the input process for subject information, etc., in Figure 7, it is not limited to this timing and can be performed at any time, such as during the shooting process or during the judgment process. Furthermore, it is not necessary for the information to be input by the processing device 100; for example, the medical interview information may be input via a website in advance or during the process of a series of processes on the subject's terminal device. Also, although the subject image is transmitted to the processing device 100 during the judgment process, only the image used as the judgment image may be transmitted and output by the processing device 100. In addition, although the example in Figure 7 describes the case where the first subject is registered and judged, it is possible to process subjects other than the first subject in the same way. Furthermore, in the example in Figure 7, a discrimination process was performed on the subject image to obtain a judgment image, but the possibility of disease determination processing may be performed using all or part of the acquired subject image without performing such a discrimination process.

[0069] 5. Processing flow executed by the processing unit 100 Figure 8A is a diagram showing the processing flow executed in a processing device 100 according to one embodiment of the present disclosure. Specifically, Figure 8A is a diagram showing the processing flow executed in S11 and S12, the new registration processing of subject information, among the input processing of subject information, etc., shown in Figure 7. This processing flow is mainly performed by the processor 111 of the processing device 100 reading and executing a program stored in memory 112. In the following explanation, the first subject will be used as an example, but the processing can be done similarly for other subjects.

[0070] According to Figure 8A, the processor 111 accesses the server device 300 at predetermined intervals and receives subject information stored in the subject management table from the server device 300 (S111). Then, based on the received subject information, the processor 111 updates and outputs the subject list screen that could be output to the display via the output interface 114 (S112). At this time, the first subject is not output to the subject list screen because the subject information has not yet been registered. Note that the reception of subject information in S111 is not limited to predetermined intervals, but may also be performed by accepting instructions from the operator.

[0071] Here, Figure 11A shows an example of a screen displayed on a processing device 100 according to one embodiment of the present disclosure. Specifically, Figure 11A shows an example of a target list screen output from the processing device 100. According to Figure 11A, the target list screen includes a list area 11 in which the received target information is output one line for each target. Each line in the list area 11 includes status information, target name information, target ID information, and attribute information for each target. In addition, although not specifically shown, the device accepts input from the operator via an input interface 113, and when a desired target to be processed is selected, the line in which the target information of that target is output is highlighted and displayed. For example, if the input interface 113 has a touch sensor, the target is selected as a target for processing by accepting input of a tap operation on the target name of the desired target. Note that the above highlighting display is for notifying the target information of the selected target, so it is sufficient that the selected target can be identified, and of course, notification may be done by other methods.

[0072] Furthermore, as shown in Figure 11A, the subject list screen includes a medical questionnaire input icon 12, a diagnosis icon 13, and a new registration icon 14 at the bottom of the list area 11. When a selection is received for any of the icons via the input interface 113, the mode corresponding to the selected icon is activated. Specifically, if the medical questionnaire input icon 12 is selected, the system switches to the medical questionnaire input mode; if the diagnosis icon 13 is selected, the system switches to the diagnosis mode; and if the new registration icon 14 is selected, the system switches to the new subject registration mode.

[0073] Returning to Figure 8A, when the processor 111 receives the selection of the new registration icon 14 shown in Figure 11A via the input interface 113 (S113), it outputs the new registration screen to the display via the output interface 114.

[0074] Here, Figure 11B is a diagram showing an example of a screen displayed in a processing device 100 according to one embodiment of the present disclosure. Specifically, Figure 11B is a diagram showing an example of a new registration screen for a subject that is output from the processing device 100. According to Figure 11B, the new registration screen outputs subject ID information (Un) that has been pre-assigned to the subject to be newly registered. Below the subject ID information, input boxes are output for entering subject name information, gender information, and date of birth information that are required to be registered as subject information. When either the operator or the subject selects an input box, it becomes possible to input desired characters or other information or select information to the input box via the input interface 113.

[0075] Furthermore, as shown in Figure 11B, a registration icon 15 and a cancel icon 16 are displayed at the bottom of each input box. When a selection for the registration icon 15 is received via the input interface 113, the system confirms the subject information entered in each input box and sends it to the server device 300. On the other hand, when a selection for the cancel icon 16 is received via the input interface 113, the new input of subject information is terminated at this point, and the system returns to the subject list screen.

[0076] Returning to Figure 8A, the processor 111 receives input of the subject information of the first subject shown in Figure 11B via the input interface 113 (S114). Then, when the processor 111 receives a selection for the registration icon 15 shown in Figure 11B via the input interface 113, it associates the input subject information with the subject ID information and transmits it to the server device 300 via the communication interface 115 (S115). With this, the processor 111 completes the process related to the new registration of subject information for the first subject.

[0077] Figure 8B is a diagram showing the processing flow executed in a processing device 100 according to one embodiment of the present disclosure. Specifically, Figure 8B is a diagram showing the processing flow executed in the input processing of medical interview information (S13) among the input processing of subject information, etc., shown in Figure 7. This processing flow is mainly performed by the processor 111 of the processing device 100 reading and executing a program stored in the memory 112. In the following explanation, the first subject will be used as an example, but the processing can be performed similarly for other subjects.

[0078] As shown in Figure 8B, the processor 111 accepts the selection of a desired subject (the first subject) for whom medical questionnaire information is to be entered on the subject list screen (S112 in Figure 8A) via the input interface 113 (S121). Since the first subject has been newly registered as shown in Figure 8A, the subject list screen includes and displays the subject information of the first subject. When the subject name of the first subject is selected on the subject list screen, the processor 111 highlights and displays the row containing the subject information of the first subject. The above highlighting is to notify the user of the subject information of the selected first subject, so it is sufficient that the selected subject can be identified, and of course, notification can be done by other means. Next, when the processor 111 accepts the selection of the medical questionnaire input icon 12 via the input interface 113, it switches to medical questionnaire input mode and outputs the medical questionnaire input screen to the display via the output interface 114 (S122). A detailed explanation of the medical questionnaire input screen is omitted here.

[0079] The processor 111 accepts the input of medical information from the operator or subject on the medical interview input screen via the input interface 113 (S123). Then, the processor 111 associates the input medical information with the subject ID information of the first subject and transmits it to the server device 300 via the communication interface 115 (S124). With this, the processor 111 completes the input processing of the medical interview information of the first subject.

[0080] Figures 8A and 8B illustrate the case where the input of subject information (S114) and the input of medical interview information (S123) are accepted as separate processes. However, after the subject information of a subject is newly entered, the medical interview input screen for that subject may be displayed immediately, and the input of medical interview information may be accepted.

[0081] Figure 8C is a diagram showing the processing flow executed in a processing apparatus 100 according to one embodiment of the present disclosure. Specifically, Figure 8C is a diagram showing the processing flow executed in the determination process S41 to S45 of Figure 7. This processing flow is mainly performed by the processor 111 of the processing apparatus 100 reading and executing a program stored in the memory 112. In the following explanation, the first target person will be used as an example, but the processing can be performed similarly for other target persons.

[0082] According to Figure 8C, the processor 111, after receiving the selection of a desired subject (first subject) to be assessed on the subject list screen (S112 in Figure 8A) via the input interface 113 (S121 in Figure 8B), and then receiving the selection of the diagnostic icon 13, switches to diagnostic mode and outputs a diagnostic screen (S131). The processor 111 also sends a subject information request for the first subject to the server device 300 via the communication interface 115, along with the subject ID information of the first subject selected in S121 in Figure 8B (S132).

[0083] When the processor 111 receives subject information (attribute information), subject images, and judgment image information that identifies the image to be used as the judgment image from the subject images from the server device 300 via the communication interface 115 (S134), it outputs a findings input screen to the display via the output interface 114 and accepts the input of findings information (S135).

[0084] Here, Figure 11C is a diagram showing an example of a screen displayed in a processing device 100 according to one embodiment of the present disclosure. Specifically, Figure 11C is a diagram showing an example of a subject's findings input screen output from the processing device 100. According to Figure 11C, the findings input screen includes an image display area 31 for displaying the transmitted subject images along with the subject ID information of the subject for whom findings are input. In addition to displaying the subject images captured by the imaging device 200, the image display area 31 also displays a display 32 for identifying the image to be used as a judgment image superimposed on each subject image. This display 32 is provided based on the judgment image information received in S134, which identifies the image to be used as a judgment image. This makes it possible to confirm which of the subject images is the judgment image. The operator, such as a doctor, can refer to the subject images displayed in the image display area 31 to make a judgment regarding the findings for the subject. Although this explanation describes the case where subject images are displayed, the displayed subject images may include all subject images captured by the imaging device 200, or only some images may be displayed, excluding, for example, some images identified as judgment images by judgment image information or defective images with focus errors, etc.

[0085] Furthermore, as shown in Figure 11C, the findings input screen includes a findings input area 30 for operators such as physicians to input findings. The findings input area 30 displays judgment items for representative findings related to the disease being judged, and options for selecting the result of the judged finding. Operators such as physicians can input findings by selecting the result of the judgment for these findings via the input interface 113.

[0086] Furthermore, the findings input screen displays a judgment icon 28 and a cancel icon 29 at the bottom of the findings input area 30. When a selection for the judgment icon 28 is received via the input interface 113, the entered findings information is sent to the server device 300 as findings information, and the process of determining the possibility of contracting a predetermined disease is executed. On the other hand, when a selection for the cancel icon 29 is received, the process ends at that point and the system returns to the subject list screen.

[0087] Returning to Figure 8C, when the processor 111 receives the input of findings information on the findings input screen in Figure 11C, it sends a judgment request to the server device 300 via the communication interface 115 along with the input findings information and the subject ID information of the first subject (S136). Then, when the processor 111 receives the judgment result regarding the possibility of contracting a predetermined disease and the judgment image used for the judgment from the server device 300 (S137), it outputs the judgment result and the judgment image to the display via the output interface 114 (S138). With this, the processor 111 completes the judgment process for the first subject.

[0088] 6. Processing flow executed on server device 300 Figure 9A is a diagram showing the processing flow executed in a server device 300 according to one embodiment of the present disclosure. Specifically, Figure 9A is a diagram showing the processing flow executed in the shooting process S22 to S29 of the input processing of subject information etc. in Figure 7. This processing flow is mainly performed by the processor 312 of the server device 300 reading and executing a program stored in memory 311. In the following explanation, the first subject will be used as an example, but the processing can be done similarly for other subjects.

[0089] As shown in Figure 9A, when the processor 312 receives a request for subject information from the imaging device 200 via the communication interface 313 (S211), it refers to the status information of the subject management table in the memory 311, searches for subjects whose status information is recorded as "not yet photographed," and extracts the subject information of those subjects (S212). Here, it is assumed that the first subject is included as a subject that has not been photographed. The processor 312 then transmits the subject information of the subjects that have not been photographed, including the first subject, to the imaging device 200 via the communication interface 313 (S213).

[0090] Next, the processor 312 waits until it receives a subject image from the imaging device 200 via the communication interface 313. Upon receiving the subject image (S214), it determines whether the received subject image is suitable for determining the possibility of having a predetermined disease (S215). One example of such determination is inputting the received subject image into a trained image selection model for determination, as shown below. However, this method is not the only one that can be used; any method can be employed, such as a method that determines based on the degree of agreement with appropriate images prepared in advance through image analysis processing. Furthermore, if multiple images are received as subject images, the system may determine whether each subject image is suitable. In addition, a score may be calculated for each subject image to determine whether it is suitable, and a predetermined number of images with high scores may be selected, or images with scores exceeding a predetermined threshold may be selected.

[0091] Here, Figure 9C is a diagram showing the processing flow for generating a trained model according to one embodiment of the present disclosure. Specifically, Figure 9C is a diagram showing the processing flow for generating a trained image selection model for judgment used in the judgment processing of S215 in Figure 9A. This processing flow may be executed by the processor 312 of the server device 300, or by the processor of another processing device.

[0092] As shown in Figure 9C, the processor 312 performs the step of acquiring a subject image of a subject that includes at least a part of the pharynx as a training subject image (S241). Next, the processor 312 performs the processing step of assigning label information to the acquired training subject image to indicate whether or not it is an image that can be used as a judgment image (S242). Then, the processor performs the step of storing the assigned label information in association with the training subject image (S243). Note that the label assignment process and label information storage process may be performed by having a human determine in advance whether or not the training subject image is a judgment image and then storing it in association with the training subject image, or the processor 312 may perform an analysis to determine whether or not it is a judgment image using a known image analysis process and store the result in association with the training subject image. Furthermore, the label information is assigned based on aspects such as whether or not at least a part of the oral cavity of the subject is captured, and whether or not the image quality is good, such as whether or not there is camera shake, out of focus, or blur.

[0093] Once training subject images and their associated label information are obtained, the processor 312 performs a step of machine learning to create a selection pattern for judgment images using them (S244). This machine learning is performed, for example, by providing the training subject image and label information pair to a neural network composed of neurons, and repeatedly adjusting the parameters of each neuron so that the output of the neural network matches the label information. Then, a step is performed to obtain a trained judgment image selection model (e.g., a neural network and parameters) (S245). The obtained trained judgment image selection model may be stored in the memory 311 of the server device 300 or in another device connected to the server device 300 via a wired or wireless network.

[0094] Returning to Figure 9A, the processor 312 inputs the received subject image into the trained image selection model for judgment generated in Figure 9C, and determines whether each image is suitable for determining the possibility of having a predetermined disease (S215). Then, it stores the information identifying the image usable for determining the possibility of having the disease as judgment image information, associating it with the subject ID information of the first subject in the image management table (S216). In addition, although not specifically shown in the figures, the processor 312 also refers to the subject management table and updates the status information associated with the subject ID information of the first subject to "taken," and transmits information indicating that a judgment image has been obtained as a result of the judgment, along with the subject ID information of the first subject, to the processing unit 100 and the imaging device 200 via the communication interface 313. On the other hand, if the processor 312 does not obtain a specified number of judgment images during the judgment process, it sends a notification to the imaging device 200 prompting reshooting. It should be noted that while such judgment processing is important for improving the accuracy of determining the possibility of having the disease, it is not always necessary to perform it. Furthermore, although not specifically illustrated, as shown in Figure 12E, the system may transmit the discrimination result to the imaging device 200, accept operation input from the operator, and allow the operator to choose whether or not to finalize the image for judgment. In other words, the image determined to be usable as a judgment image may be transmitted to the imaging device 200 for a preview display, and the server device 300 may finalize the judgment image upon receiving confirmation that the finalization operation has been performed. At this point, the processor 312 terminates the imaging process.

[0095] The images selected for judgment in this manner may be one or multiple images. However, as an example, it is preferable to obtain a group of about 5 judgment images from a group of 5 to 30 subject images. This is because selecting judgment images from a large number of subject images increases the likelihood of obtaining better judgment images. Furthermore, by using multiple judgment image groups in the judgment process described later, the judgment accuracy can be further improved compared to using only one judgment image. As another example, each time a subject image is captured, the captured subject image may be sent to the processing unit 100 for selection of judgment images, or the capture device 200 may select the judgment images and terminate the capture once a predetermined number of judgment images (for example, about 5) have been acquired. In this way, the time required for capturing subject images can be minimized while maintaining the improved judgment accuracy as described above. In other words, discomfort to the subject, such as the gag reflex, can be reduced.

[0096] Figure 9B is a diagram showing the processing flow executed in a server device 300 according to one embodiment of the present disclosure. Specifically, Figure 9B is a diagram showing the processing flow executed in the determination processing S42 to S44 of the input processing of subject information etc. in Figure 7. This processing flow is mainly performed by the processor 312 of the processing device 100 reading and executing a program stored in memory 311. In the following explanation, the first subject will be used as an example, but the processing can be done similarly for other subjects.

[0097] According to Figure 9B, when the processor 312 receives a request for subject information of a desired subject (in this case, the first subject) from the processing unit 100 via the communication interface 313 (S221), it refers to the subject management table to obtain attribute information of the first subject based on the subject ID information of the first subject, and also refers to the image management table to obtain subject images associated with the subject ID information of the first subject, and judgment image information that identifies the image to be used as the judgment image from among the subject images. The processor 312 transmits the acquired subject images, etc., along with the attribute information and subject ID information of the first subject to the processing unit 100 via the communication interface 313 (S222).

[0098] Next, the processor 312 receives from the processing unit 100 via the communication interface 313 the subject ID information of the first subject, along with the findings information entered by an operator such as a physician and a request for determination of the possibility of having a predetermined disease (S223). The processor 312 stores the received findings information in the subject management table, associating it with the subject ID information (S224). Next, the processor 312 reads the findings information and interview information associated with the subject ID information of the first subject by referring to the subject management table. The processor 312 also reads the judgment image based on the judgment image information associated with the subject ID information of the first subject by referring to the image management table. Then, the processor 312 executes the judgment process based on this read information (S225). As an example of such a judgment process, it is conceivable to input this information into a trained judgment model as shown below and make a judgment. However, of course, it is not limited to this method, and any method can be adopted, such as a method that makes a judgment based on the degree of agreement with an image showing the diseased state through image analysis processing.

[0099] Here, Figure 9D is a diagram showing a processing flow for generating a trained model according to one embodiment of the present disclosure. Specifically, Figure 9D is a diagram showing a processing flow for generating a trained decision model used in the decision processing of S245 in Figure 9A. This processing flow may be executed by the processor 312 of the server device 300, or by the processor of another processing device.

[0100] As shown in Figure 9D, the processor 312 performs the step of acquiring a judgment image selected from images of a subject that include at least a portion of the pharynx (S261). The processor 312 also performs the step of acquiring interview information and findings information that have been stored in advance and associated with the subject ID information of the subject who is the subject of the judgment image (S261). Next, the processor 312 performs a processing step of assigning a correct label to the subject who is the subject of the judgment image, which has been assigned in advance based on the results of an immunochromatographic rapid influenza test, PCR test, virus isolation culture test, etc. (S262). Then, the processor 312 performs the step of storing the assigned correct label information as judgment result information, associating it with the judgment image, as well as the interview information and findings information (S263). Note that although the judgment image itself is used here, feature quantities obtained from the judgment image may also be used.

[0101] Once the judgment image, as well as the medical history information, findings information, and corresponding correct label information are obtained, the processor performs a step of machine learning to determine disease incidence patterns using these (S264). This machine learning is performed, for example, by providing these sets of information to a neural network made up of neurons and repeatedly learning while adjusting the parameters of each neuron so that the output from the neural network is the same as the correct label information. Then, a step of acquiring the trained judgment model is performed (S265). The acquired trained judgment model may be stored in the memory 311 of the server device 300 or in another device connected to the server device 300 via a wired or wireless network.

[0102] Returning to Figure 9B, the processor 312 inputs the findings information, interview information, and assessment images of the first subject into the trained assessment model generated in Figure 9D to determine the likelihood of the subject having a predetermined disease (S225). Once the likelihood of the subject having the disease has been determined, the processor 312 stores the assessment result information in the subject management table based on the subject ID information of the first subject and updates the status information to "Assessed". The processor 312 also transmits the assessment result information stored in the subject management table and the assessment images used for the assessment, along with the subject ID information of the first subject, to the processing unit 100 via the communication interface 313 (S226). With this, the processor 312 terminates the assessment process.

[0103] 7. Processing flow executed by the imaging device 200 Figure 10 is a diagram showing the processing flow performed in an imaging device 200 according to one embodiment of the present disclosure. Specifically, Figure 10 is a diagram showing the processing flow performed in the imaging process S21 to S30 of the input processing of subject information etc. in Figure 7. This processing flow is mainly performed by the processor 213 of the imaging device 200 reading and executing a program stored in the memory 214. In the following explanation, the first subject will be used as an example, but the processing can be performed similarly for other subjects.

[0104] According to Figure 10, when the imaging device 200 is started up, the processor 213 sends a request for subject information to the server device 300 via the communication interface 216 (S311). The processor 213 then receives subject information of unphotographed subjects extracted by the server device 300 via the communication interface 216 (S312). Upon receiving subject information of unphotographed subjects, the processor 213 outputs the received subject information to the display via the output interface 215 (S313).

[0105] Here, Figures 12A and 12B show examples of screens displayed in the imaging device 200 according to one embodiment of the present disclosure. Specifically, Figure 12A shows an example of a screen displayed before the subject information of the first subject is newly registered and stored in the subject management table of the server device 300. On the other hand, Figure 12B shows an example of a screen displayed after the subject information of the first subject has been newly registered and stored in the subject management table of the server device 300.

[0106] According to Figure 12A, the display shows a list of unphotographed subjects via the output interface 215. The list of unphotographed subjects includes a subject list area 17 in which the subject information of unphotographed subjects transmitted from the server device 300 is arranged in a list format, row by row. The subject list area 17 includes subject name information and time information (not shown in Figure 6A, but stored as one of the attribute information) for each subject.

[0107] Here, Figure 12A shows an example of a screen that is output before the subject information of the first subject is newly registered and stored in the subject management table of the server device 300, as described above. Therefore, the subject information of the first subject (subject name information: N1, registration time information: T1) is not output on this unphotographed subject list screen.

[0108] Next, according to Figure 12B, a screen displaying a list of unphotographed subjects, similar to that in Figure 12A, is output. Here, Figure 12B shows an example of a screen output after the subject information of the first subject has been newly registered and stored in the subject management table of the server device 300, as described above. Therefore, by receiving the subject information of the first subject at S312 in Figure 10, the subject information of the first subject (subject name information: N1, registration time information: T1) is output at the top of the subject list area 17 on the unphotographed subject list screen.

[0109] In Figures 12A and 12B, the subject information is arranged from top to bottom in order of closest registration time. However, it is possible to arrange the information in various other orders, such as by earliest registration time or by name. Also, in Figures 12A and 12B, subjects who have not yet been photographed are displayed as a list. However, it is also possible to sequentially switch between screens displaying the subject information of each subject who has not yet been photographed (or a predetermined number of subjects at a time), and accept selection when the desired subject is displayed.

[0110] Returning to Figure 10, the processor 213 accepts the selection of a desired subject to be photographed via the input interface 210 on the list of unphotographed subjects displayed in S313 (S314). For example, on the screen in Figure 12A or Figure 12B, a subject to be photographed is selected by tapping on the subject name information of the desired subject to be photographed on the list of unphotographed subjects. Once a subject is selected, the processor 213 outputs the attribute information of the selected subject to the display via the output interface 215.

[0111] Here, Figure 12C shows an example of a screen displayed in a shooting device 200 according to one embodiment of the present disclosure. Specifically, Figure 12C shows an example of an attribute information display screen displayed when a first subject is selected in S314 of Figure 10. According to Figure 12C, the attribute information display screen includes an attribute display area 18. In addition to the subject ID information of the first subject, the attribute display area 18 displays gender information and date of birth information as attribute information of the first subject received from the server device 300. This allows the operator to confirm detailed information about the first subject, and by verifying the identity of the subject based on the information displayed here, the possibility of misidentifying the subject can be reduced. In other words, it becomes possible to reliably associate the subject image of the subject to be captured later with the subject ID information of the first subject.

[0112] Furthermore, as shown in Figure 12C, a "take a picture icon 19" and a "back icon 20" are displayed at the bottom of the attribute display area 18. When a selection is made for the "take a picture icon 19" via the input interface 210, the camera 211 is activated and the system proceeds to taking a picture of the subject. On the other hand, when a selection is made for the "back icon 20" via the input interface 210, the system returns to the list of unphotographed subjects.

[0113] Returning to Figure 10, the processor 213 determines whether the auxiliary device 400 is properly attached to the imaging device 200. If it is determined that the device is not attached, it outputs an attachment indicator to prompt the user to attach the auxiliary device 400 (S315). This determination can be made by any method, but examples include detecting that a predetermined switch is turned on when the auxiliary device 400 is attached, detecting the presence or absence of a pattern specific to images taken with the auxiliary device 400 attached, or a combination of these. In S315, the system determines whether the auxiliary device 400 is attached, but this process itself may be skipped. Alternatively, instead of determining whether the auxiliary device 400 is attached, for example, a confirmation indicator for the operator to confirm that the auxiliary device 400 is attached may be output via the output interface 215, and the system may accept a predetermined operation input (e.g., a tap operation) from the operator in response to this confirmation indicator, allowing the operator to input that the auxiliary device 400 has been attached.

[0114] Here, Figure 12D is a diagram showing an example of a screen displayed in the imaging device 200 according to one embodiment of the present disclosure. Specifically, Figure 12D is a diagram showing an example of the attachment display screen displayed in S315 of Figure 10. According to Figure 12D, the attachment display screen includes a message prompting the user to attach the auxiliary device 400, along with images of the auxiliary device 400 and the imaging device 200. For example, the screen transitions so that both images, which are models of the auxiliary device 400 and the imaging device 200, gradually move closer together over time, making it possible to more intuitively prompt the user to attach the auxiliary device 400. A "back icon 21" is displayed on the screen, and it is possible to return to the attribute information display screen by selecting this icon.

[0115] Returning to Figure 10, with the auxiliary device attachment display screen displayed, the processor 213 determines whether the auxiliary device 400 is attached at predetermined intervals. If it is determined that it is attached correctly, the processor 213 starts taking images of the oral cavity, including the pharynx, with the camera 211 (S316). Specifically, the light source 212 is turned on to illuminate the subject, and the camera 211 detects the reflected light from the subject, such as the pharynx, to perform the image capture.

[0116] Here, Figure 12E is a diagram showing an example of a screen displayed in a shooting device 200 according to one embodiment of the present disclosure. Specifically, Figure 12E is a diagram showing an example of a shooting screen output in S316 of Figure 10. According to Figure 12E, the shooting screen includes a through-image area 32, and the current subject image detected by the camera 211 is output to the through-image area 32 (specific images are not shown). At this time, a guide 22 drawn to conform to the general shape of the pharynx, which is the subject, is superimposed on the through-image and output to the through-image area 32. By superimposing such a guide 22 on the through-image and outputting it, it becomes possible to adjust the left-right position and depth-direction size of the subject currently being photographed by referring to the guide 22.

[0117] Furthermore, as shown in Figure 12E, a capture icon 23 and a back icon 24 are output at the bottom of the through image area 32. By receiving an operation input for the capture icon 23 via the input interface 210, the system starts capturing the image taken by the camera 211 as a subject image, either as a still image or a video. Note that the operation input for the capture icon 23 is just one example of how to start shooting; it may also be started by other methods, such as detecting the press of the capture button. Additionally, by receiving an operation input for the back icon 24 via the input interface 210, it is possible to return to the attribute display screen.

[0118] Returning to Figure 10, when the processor 213 acquires one or more subject images as described above, it transmits the acquired subject images to the server device 300 via the communication interface 216 along with the subject ID information of the first subject (S317). The processor 213 waits until it receives the subject image discrimination result from the server device 300, and when it receives information indicating that an image suitable for determining the possibility of disease has been acquired, it outputs the result to the display via the output interface 215 (S318).

[0119] Here, Figure 12F is a diagram showing an example of a screen displayed in a shooting device 200 according to one embodiment of the present disclosure. Specifically, Figure 12F is a diagram showing an example of a discrimination result screen output in S318 of Figure 10. According to Figure 12F, the discrimination result screen includes a preview area 25 for the judgment image. In the preview area 25, the subject image selected as the judgment image as a result of discrimination by the server device 300 is output to the operator. The operator can check the judgment image output in the preview area 25 and choose whether to take another shot or to confirm with the judgment image.

[0120] Furthermore, information indicating the discrimination result by the server device 300 is displayed at the bottom of the preview area 25. Here, the message "Good" is displayed, indicating that a suitable image for determining the likelihood of having a predetermined disease has been obtained. This allows the operator to confirm that a subject image usable for discrimination has been captured.

[0121] Furthermore, a reshoot icon 26 and a confirmation icon 27 are output at the bottom of the information indicating the discrimination result. When the reshoot icon 26 is selected via the input interface 210, the processor 213 sends information to the server device 300 indicating that a reshoot will be performed, and also starts the camera 211 and returns to the shooting process in S316. On the other hand, when the confirmation icon 27 is selected via the input interface 210, the processor 213 sends information to the server device 300 indicating that the confirmation operation for the judgment image has been performed.

[0122] In other words, by outputting the result of the determination of whether or not an image suitable for determining the possibility of disease has been acquired in S318 of Figure 10, as illustrated in Figure 12F, it becomes possible to notify the operator immediately after capturing the subject image (after processing in S316) whether or not reshooting is necessary, enabling smoother shooting. For example, normally, the process of determining the possibility of contracting a certain disease is computationally intensive and time-consuming. Therefore, by first executing a determination process to determine whether or not the image is suitable, which is less computationally intensive and can be processed in a shorter time, and outputting the result, the time until reshooting can be reduced. Also, even if it is discovered that reshooting is necessary immediately before determining the possibility of contracting a certain disease, the assistive device 400, etc., may have already been removed and discarded, requiring the attachment of a new assistive device 400. As described above, by notifying the operator immediately after capturing the subject image, such inconveniences can be eliminated.

[0123] Figure 12F describes the case where the discrimination result and the preview area 25 of the image for judgment are displayed together on the discrimination result screen. However, the case is not limited to this; the discrimination result and the preview area 25 of the image for judgment may be displayed on separate screens, and the operator may switch between them.

[0124] Returning to Figure 10, when the input interface 210 receives a selection for the confirmation icon 27 shown in Figure 12F, the processor 213 transmits a message to the server device 300 via the communication interface 216 indicating that the confirmation operation has been performed. The processor 213 then receives the subject information of the unphotographed subjects again via the communication interface 216 and updates the unphotographed subject list screen on the display via the output interface 215 (S320). At this time, the status information of the first subject is "photographed" and is therefore not included in the subject information of the unphotographed subjects mentioned above. As a result, the subject information of the first subject is not included in the unphotographed subject list screen. Note that it is possible to output a list of photographed subjects by selecting the photographed list icon on the unphotographed subject list screen, and the subject information of the first subject will be output in that list.

[0125] In this embodiment, we can provide a shooting device, program, and method that are more suitable for capturing images of subjects that include at least a portion of the subject's oral cavity.

[0126] 8. Variations In the above embodiment, a case was described in which information indicating the possibility of contracting influenza is output using subject images, medical interview information, and findings information. However, instead of medical interview information and findings information, or in addition to this information, information indicating the possibility of contracting influenza may be output using external factor information related to influenza. Such external factor information includes judgment results made for other subjects, diagnosis results by a doctor, and information on influenza outbreaks in the region to which the subject belongs. The processor 312 can obtain such external factor information from other processing devices via the communication interface 313 and provide the external factor information as input to the trained judgment model, thereby enabling the acquisition of a positive rate that takes the external factor information into account.

[0127] Furthermore, the above embodiment described a case in which the possibility of contracting influenza is determined using not only the subject image but also the medical history information and findings information. However, it is not limited to this, and it is also possible to make the determination using only the subject image.

[0128] In the above embodiment, the cases in which the medical history information and findings information are input in advance by the operator or subject, or received from an electronic medical record device connected to a wired or wireless network, were described. However, instead of these, or in addition to these, this information may also be obtained from captured subject images. A trained information estimation model is obtained by providing the findings information and medical history information associated with the training subject images as correct labels to the training subject images, and using machine learning on these pairs in a neural network. Then, the processor 111 can obtain the desired medical history information and attribute information by providing the subject images as input to the trained information estimation model. Examples of such medical history information and attribute information include gender, age, degree of pharyngeal redness, degree of tonsil swelling, presence or absence of white coating, etc. This eliminates the need for the operator to input the medical history information and findings information.

[0129] Each of the pre-trained models described in the above embodiments was generated using neural networks or convolutional neural networks. However, it is not limited to these; it is also possible to generate them using machine learning methods such as nearest neighbor, decision trees, regression trees, and random forests.

[0130] In the above embodiment, the case in which the discrimination and judgment processing of the judgment image is performed in the server device 300 was described. However, these various processes can be appropriately distributed and processed by the processing device 100, the imaging device 200, and other devices (including cloud server devices, etc.).

[0131] These modifications are similar in configuration, processing, and procedures to the embodiment described in Figures 1 to 12F, except for the points specifically described above. Therefore, a detailed explanation of those matters will be omitted. It is also possible to configure the system by appropriately combining or substituting the elements described in each modification and embodiment.

[0132] The processes and procedures described herein can be implemented not only by those explicitly described in the embodiments, but also by software, hardware, or a combination thereof. Specifically, the processes and procedures described herein can be implemented by implementing the logic corresponding to the process on a medium such as an integrated circuit, volatile memory, non-volatile memory, magnetic disk, or optical storage. Furthermore, the processes and procedures described herein can be implemented as computer programs and executed by various computers, including processing units and server devices.

[0133] Even if it is stated that the processes and procedures described herein are performed by a single device, software, component, or module, such processes or procedures may be performed by multiple devices, multiple software programs, multiple components, and / or multiple modules. Similarly, even if it is stated that the various types of information described herein are stored in a single memory or storage unit, such information may be distributed and stored in multiple memories within a single device or in multiple memories distributed across multiple devices. Furthermore, the software and hardware elements described herein may be implemented by integrating them into fewer components or by decomposing them into more components. [Explanation of Symbols]

[0134] 1. Processing System 100 Processing Units 200 imaging devices 300 Server Devices 400 assistive devices 600 Operator 700 target individuals

Claims

1. A camera comprising a camera configured to capture an image of a subject that includes at least a portion of the natural aperture of one or more subjects, including a first subject, and at least one processor, The at least one processor, The camera receives subject information, including first subject information associated with each of one or more subjects, including the first subject, via a communication interface from an external device that is communicably connected to the camera via a network. When outputting a list of subject information for subjects who have not been photographed among the one or more subjects mentioned above, the list is output without including the first subject information before the first subject information associated with the first subject is registered in the external device, and after the first subject information is registered in the external device, the list is output with the first subject information included. After receiving a selection for the first subject information from the list output, which includes the first subject information, and when the camera captures a first subject image that includes at least a portion of the natural aperture of the first subject, the first subject information selected from the list and the first subject image are associated with each other and transmitted to the external device for storage. A photographic device configured to perform processing for the purpose of [doing something].

2. The imaging apparatus according to claim 1, wherein at least one processor is configured to process for outputting attribute information of the first subject when it receives a selection for the first subject information from the list output, which includes the first subject information.

3. The imaging apparatus according to claim 1, wherein the first subject image is used by the external device to determine the possibility of contracting a predetermined disease.

4. A shooting device comprising a camera configured to capture an image of a subject that includes at least a portion of the natural aperture of one or more subjects, including a first subject, and at least one processor, The at least one processor, The camera receives subject information, including first subject information associated with each of one or more subjects, including the first subject, via a communication interface from an external device that is communicably connected to the camera via a network. When outputting a list of subject information for subjects who have not been photographed among the one or more subjects mentioned above, the list is output without including the first subject information before the first subject information associated with the first subject is registered in the external device, and after the first subject information is registered in the external device, the list is output with the first subject information included. The external device receives discrimination information from the external device indicating whether a first subject image used to determine the possibility of contracting a predetermined disease is suitable for the determination, and outputs the received discrimination information. A photographic device configured to perform processing for the purpose of [doing something].

5. The imaging apparatus according to any one of claims 1 to 4, wherein at least one processor is configured to perform processing for outputting an attachment indicator that prompts the attachment of an auxiliary device which covers at least a portion of the imaging apparatus and is inserted into a natural opening together with the portion.

6. A photographic device including a camera configured to capture an image of a subject that includes at least a portion of the natural aperture of one or more subjects, including the first subject, The camera receives subject information, including first subject information associated with each of one or more subjects, including the first subject, via a communication interface from an external device that is communicably connected to the camera via a network. When outputting a list of subject information for subjects who have not been photographed among the one or more subjects mentioned above, the list is output without including the first subject information before the first subject information associated with the first subject is registered in the external device, and after the first subject information is registered in the external device, the list is output with the first subject information included. After receiving a selection for the first subject information from the list output, which includes the first subject information, and when the camera captures a first subject image that includes at least a portion of the natural aperture of the first subject, the first subject information selected from the list and the first subject image are associated with each other and transmitted to the external device for storage. A program that makes it function as a processor.

7. A photographing apparatus comprising a camera configured to capture an image of a subject that includes at least a portion of the natural aperture of one or more subjects, including a first subject, and at least one processor, wherein a method is performed by the at least one processor, The steps include receiving subject information, including first subject information associated with each of one or more subjects, including the first subject, via a communication interface from an external device that is communicably connected to the imaging device via a network, When outputting a list of subject information for subjects who have not been photographed among the one or more subjects mentioned above, the list is output without including the first subject information before the first subject information associated with the first subject is registered in the external device, and after the first subject information is registered in the external device, the list is output with the first subject information included. After receiving a selection for the first subject information from the list output including the first subject information, and when the camera captures a first subject image that includes at least a portion of the natural aperture of the first subject, the first subject information selected from the list and the first subject image are transmitted to the external device for storage in the external device, A method that includes this.

8. A photographic device including a camera configured to take an image of a subject that includes at least a portion of the natural aperture of one or more subjects, including a first subject, The camera receives subject information, including first subject information associated with each of one or more subjects, including the first subject, via a communication interface from an external device that is communicably connected to the camera via a network. When outputting a list of subject information for subjects who have not been photographed among the one or more subjects mentioned above, the list is output without including the first subject information before the first subject information associated with the first subject is registered in the external device, and after the first subject information is registered in the external device, the list is output with the first subject information included. The external device receives discrimination information from the external device indicating whether a first subject image used to determine the possibility of contracting a predetermined disease is suitable for the determination, and outputs the received discrimination information. A program that makes it function as a processor.

9. A photographing apparatus comprising a camera configured to capture an image of a subject that includes at least a portion of the natural aperture of one or more subjects, including a first subject, and at least one processor, wherein a method is performed by the at least one processor, The steps include receiving subject information, including first subject information associated with each of one or more subjects, including the first subject, via a communication interface from an external device that is communicably connected to the imaging device via a network, When outputting a list of subject information for subjects who have not been photographed among the one or more subjects mentioned above, the list is output without including the first subject information before the first subject information associated with the first subject is registered in the external device, and after the first subject information is registered in the external device, the list is output with the first subject information included. The steps include receiving discrimination information from the external device indicating whether a first subject image used in the external device to determine the possibility of contracting a predetermined disease is suitable for use in the determination, and outputting the received discrimination information, A method that includes this.

Images