Processing device, processing program, processing method, and processing system

Abstract

[Problem] To provide a more reliable processing device, processing program, processing method, and processing system. [Solution] Provided is a processing device that, for an assessment result that represents the possibility of morbidity of one or a plurality of diseases and is generated by acquiring, via a communication interface, at least one among a subject image acquired by an imaging device configured to capture an image of a subject who is a target person, medical interview information of the target person, and observation information of the target person and inputting the at least one among the subject image, the medical interview information, and the observation information which was acquired into a trained evaluation model, executes processing for generating degree-of-effect information representing the extent of the effect exerted by the at least one among the subject image, the medical interview information, and the observation information input into the learned evaluation model.

Description

Processing device, processing program, processing method, and processing system

[0001] The present disclosure relates to a processing device, a processing program, a processing method, and a processing system configured to execute processing related to a determination result indicating the possibility of being affected by a disease.

[0002] Conventionally, a technology for capturing an image of a subject and diagnosing a predicted disease has been known. Patent Document 1 describes an information processing system including: an acquisition unit that acquires input information including 3D data related to an MRI image of the brain of a subject; and an output unit that uses the input information as input data and outputs disease name information or disease diagnosis support information related to Parkinson's syndrome for the subject based on a machine learning model. However, in this technology, the reliability of the output information obtained from the machine learning model based on the input information is unknown.

[0003] Japanese Patent Application Laid-Open No. 2023-161222

[0004] In light of the above-described techniques, the present disclosure aims to provide a more reliable processing device, processing program, processing method, and processing system through various embodiments.

[0005] According to one aspect of the present disclosure, there is provided a processing device having at least one processor, wherein the at least one processor is configured to acquire, via a communication interface, at least one of a subject image acquired by an imaging device configured to image a subject, medical interview information of the subject, and finding information of the subject, and to execute processing to generate impact information indicating the degree of influence of at least one of the subject image, the medical interview information, and the finding information input to a trained judgment model on a judgment result indicating the possibility of contracting one or more diseases, the judgment result being generated by inputting at least one of the acquired subject image, the medical interview information, and the finding information into the trained judgment model.

[0006] According to one aspect of the present disclosure, there is provided a processing program that, when executed by at least one processor, causes the at least one processor to function to acquire, via a communication interface, at least one of a subject image acquired by an imaging device configured to image a subject, medical interview information about the subject, and finding information about the subject, and to generate impact information indicating the degree of influence of at least one of the subject image, medical interview information, and finding information input into a trained judgment model on a judgment result indicating the possibility of contracting one or more diseases, the judgment result being generated by inputting at least one of the acquired subject image, medical interview information, and finding information into the trained judgment model.

[0007] According to one aspect of the present disclosure, there is provided a processing method executed by at least one processor, the processing method including: a step of acquiring, via a communication interface, at least one of a subject image acquired by an imaging device configured to image a subject of a subject, medical interview information of the subject, and finding information of the subject; and a step of generating impact information indicating the degree of influence of at least one of the subject image, the medical interview information, and the finding information inputted into a trained judgment model on a judgment result indicating the possibility of contracting one or more diseases, the judgment result being generated by inputting at least one of the acquired subject image, the medical interview information, and the finding information into the trained judgment model.

[0008] According to one aspect of the present disclosure, the processing system is "a processing system including a terminal device and a processing device communicatively connected to the terminal device, wherein at least one processor included in the terminal device selects at least one of a subject image acquired by an imaging device configured to image a subject of a subject, medical interview information of the subject, and finding information of the subject, and at least one processor included in the processing device acquires at least one of the selected subject image, medical interview information, and finding information via a communication interface, and is configured to execute processing to generate impact information indicating the degree of influence of at least one of the subject image, medical interview information, and finding information input into a trained judgment model on a judgment result indicating the possibility of contracting one or more diseases, the judgment result being generated by inputting at least one of the acquired subject image, medical interview information, and finding information into the trained judgment model."

[0009] According to the present disclosure, it is possible to provide a more reliable processing device, processing program, processing method, and processing system.

[0010] It should be noted that the above effects are merely illustrative for the sake of convenience and are not limiting. In addition to or instead of the above effects, any effect described in this disclosure or an effect obvious to a person skilled in the art may be achieved.

[0011] FIG. 1 is a block diagram showing a configuration of a processing system 1 according to an embodiment of the present disclosure. FIG. 2A is a block diagram showing a configuration of a server device 100 according to an embodiment of the present disclosure. FIG. 2B is a block diagram showing a configuration of a terminal device 300 according to an embodiment of the present disclosure. FIG. 3A is a diagram conceptually showing a subject management table stored in the server device 100 according to an embodiment of the present disclosure. FIG. 3B is a diagram conceptually showing a contribution management table stored in the server device 100 according to an embodiment of the present disclosure. FIG. 3C is a diagram conceptually showing a region-of-interest management table stored in the server device 100 according to an embodiment of the present disclosure. FIG. 3D is a diagram conceptually showing a subject image stored in the server device 100 according to an embodiment of the present disclosure. FIG. 4 is a diagram showing a processing sequence executed in the processing system 1 according to an embodiment of the present disclosure. FIG. 5 is a diagram showing a processing flow executed in the server device 100 according to an embodiment of the present disclosure. FIG. 6 is a diagram showing a processing flow executed in the server device 100 according to an embodiment of the present disclosure. FIG. 7 is a diagram showing a processing flow executed in the server device 100 according to an embodiment of the present disclosure. FIG. 8A is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 8B is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 9A is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 9B is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 9C is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 9D is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 9E is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 10 is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. FIG. 11 is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure.

[0012] 1. Overview of Processing System 1 The processing system 1 according to the present disclosure generates influence information indicating the degree of influence of at least one of the subject image, medical interview information, and finding information input to a trained determination model on a determination result indicating the possibility of a disease for at least one of one or more diseases based on at least one of the subject image, medical interview information, and finding information of the subject. In this way, the basis for the determination result determined by the processing system 1 can be confirmed based on the influence information, thereby making it possible to increase the reliability of the generated determination result.

[0013] In the present disclosure, the one or more diseases may be any disease for which a diagnosis result indicating the possibility of affliction can be obtained based on at least one of the subject image, medical interview information, and finding information, as described above. Among such diseases, diseases in which findings are observed in natural orifices are preferred. Furthermore, among such diseases, diseases in which findings are observed in the oral cavity, pharynx, and larynx are preferred, such as infectious diseases such as influenza, coronavirus infection, streptococcal infection, adenovirus infection, Epstein-Barr virus infection, mycoplasma infection, hand, foot, and mouth disease, herpangina, and candidiasis; diseases presenting vascular or mucosal disorders such as arteriosclerosis, diabetes, and hypertension; tumors such as tongue cancer and pharyngeal cancer; periodontal diseases such as dental caries, gingivitis, and periodontal disease; and combinations thereof. In particular, among such diseases, influenza, which exhibits a unique pattern in lymphoid follicles appearing in the deepest part of the pharynx located in the oral cavity, is preferred. The following description will be given using influenza as an example of a disease, but the present invention is not limited to this.

[0014] In addition, in the present disclosure, the term "determination result" broadly includes the result of determining the possibility of a disease based on at least one of the subject image, the medical interview information, and the finding information. Such a determination result is obtained, for example, by analysis using a trained determination model. That is, the determination result includes not only a definitive determination by a medical professional or the like, but also a determination result obtained using a trained determination model or a result used to assist a medical professional in making a definitive determination.

[0015] In addition, in the present disclosure, the subject, who is the subject of imaging by the imaging device, includes all people, such as patients, test subjects, persons to be evaluated, and healthy individuals. Furthermore, in the present disclosure, the operator who holds the imaging device and performs imaging operations is not limited to medical professionals such as doctors, nurses, and laboratory technicians, but also includes all people, such as the subject themselves and their guardians. That is, although the subject and operator are given names based on the actions they perform for convenience of explanation, they may be different persons or the same person. Hereinafter, these persons will be collectively referred to as users.

[0016] 2. Configuration of Processing System 1 FIG. 1 is a block diagram showing the configuration of a processing system 1 according to an embodiment of the present disclosure. According to FIG. 1 , the processing system 1 includes a server device 100, an imaging device 200, and a terminal device 300, which are communicatively connected via a wired or wireless network. The server device 100 uses a trained determination model to determine the likelihood of a patient suffering from a disease based on at least one of a subject image, medical interview information, and finding information acquired from at least one of the imaging device 200 and the terminal device 300, and generates influence information indicating the degree of influence of at least one of the subject image, medical interview information, and finding information input to the trained determination model on the determination result. The imaging device 200 has its tip inserted into the oral cavity of a subject to capture images of the oral cavity, particularly the pharynx, and transmits the captured subject images to the server device 100 via a wired or wireless network. The terminal device 300 inputs subject information, medical interview information, findings information, etc. required for processing in the server device 100, and receives and outputs the determination results of the possibility of contracting one or more diseases and impact information from the server device 100.

[0017] In this disclosure, the processing device refers to the server device 100, the terminal device 300, or a combination thereof. In other words, although the following describes a case where the server device 100 functions as a processing device, the terminal device 300 can also function as a processing device. In addition, in this disclosure, the storage and processing performed by the processing device may be distributed to other terminal devices, other server devices, etc. In other words, the processing device is not limited to those configured in a single housing, but includes the server device 100, the imaging device 200, the terminal device 300, other terminal devices, other server devices, or a combination thereof.

[0018] 1 illustrates only one terminal device 300 and one imaging device 200. However, for example, in a large-scale medical institution, it is possible that multiple terminal devices 300 or multiple imaging devices 200 may be managed and operated.

[0019] FIG. 2A is a block diagram showing the configuration of a server device 100 according to an embodiment of the present disclosure. According to FIG. 2A , the server device 100 includes a processor 111, a memory 112, and a communication interface 113. These components are electrically connected to one another via control lines and data lines. The server device 100 does not need to include all of the components shown in FIG. 2A ; some components may be omitted, or other components may be added. For example, an external memory, a database device, a server device, or the like connected in a communicable manner as memory may be used. Furthermore, some processing may be distributed and executed among processing devices, including other server devices. In other words, the server device 100 is not limited to a single device, but may be distributed across multiple devices depending on the information handling and processing load.

[0020] The processor 111 functions as a control unit that controls other components of the processing system 1 based on a processing program stored in the memory 112. Based on the processing program stored in the memory 112, the processor 111 executes processing for generating influence information indicating the degree of influence that at least one of the subject image, medical interview information, and finding information input to the trained judgment model has on the judgment result, mainly based on the processing program stored in the memory 112. Specifically, the processor 111 executes, based on the processing program stored in the memory 112, "processing for acquiring, via the communication interface 113, at least one of the subject image, medical interview information, and finding information of the subject acquired via at least one of the imaging device 200 configured to capture an image of the subject or the terminal device 200" and "processing for generating influence information indicating the degree of influence that at least one of the subject image, medical interview information, and finding information input to the trained judgment model has on a judgment result indicating the possibility of affliction with one or more diseases, which is generated by inputting at least one of the acquired subject image, medical interview information, and finding information to the trained judgment model." The processor 111 is mainly composed of one or more CPUs, but may also be appropriately combined with a GPU, FPGA, etc.

[0021] The memory 112 is composed of RAM, ROM, non-volatile memory, HDD, SSD, etc., and functions as a storage unit. The memory 112 stores instructions and commands for various controls of the processing system 1 according to this embodiment as processing programs. Specifically, the memory 112 stores programs to be executed by the processor 111, such as "a process of acquiring, via the communication interface 113, at least one of a subject image, medical interview information, and finding information of the subject, acquired via at least one of the imaging device 200 configured to capture an image of a subject or the terminal device 200" and "a process of generating influence information indicating the degree of influence of at least one of the subject image, medical interview information, and finding information input to the learned determination model on a determination result indicating the possibility of afflicting one or more diseases, generated by inputting at least one of the acquired subject image, medical interview information, and finding information into the learned determination model." In addition to the programs, the memory 112 also stores various information stored in a subject management table, a contribution management table, a region of interest management table, etc. It should be noted that this information does not need to be stored constantly in the memory 112 within the server device 100, but may be stored in a database device installed remotely. In this case, the database device is also included in the memory 112.

[0022] The communication interface 113 functions as a notification unit for transmitting and receiving various information to and from the image capture device 200 and the terminal device 300 connected via a wired or wireless network. Examples of the communication interface 113 include a wired communication connector such as a USB or SCSI, a wireless communication transmitting / receiving device for broadband wireless communication such as wireless LAN, Bluetooth (registered trademark), or LTE, an infrared or other wireless communication, and various connection terminals for printed circuit boards or flexible circuit boards. The communication interface 113 receives subject images from and to the image capture device 200 and transmits impact level information to and from the terminal device 300, for example.

[0023] FIG. 2B is a block diagram showing the configuration of a terminal device 300 according to an embodiment of the present disclosure. According to FIG. 2B , the terminal device 300 includes a processor 311, a memory 312, an input interface 313, an output interface 314, and a communication interface 315. These components are electrically connected to one another via control lines and data lines. The terminal device 300 does not need to include all of the components shown in FIG. 2B ; some components may be omitted, or other components may be added. The terminal device 300 may be any device capable of communicating with the server device 100 via a wired or wireless network. Examples of the terminal device 300 include a smartphone, a tablet device, a laptop PC, a desktop PC, and a photographing device. While it is not essential that the terminal device 300 be communicatively connected to the photographing device 200, it is preferable that they be communicatively connected. Such terminal devices 300 include medical institution terminal devices installed in medical institutions that diagnose the possibility of disease morbidity and used by medical professionals, subject terminal devices used by subjects and their associates (such as guardians), user terminal devices used by users who operate the imaging device 200, and information provider terminal devices used by information providers who provide information to medical professionals and the like based on impact information. However, the terminal devices exemplified here are merely names given to distinguish them according to their functions and the attributes of the main users. In other words, one terminal device can function as multiple terminal devices.

[0024] The processor 311 functions as a control unit that controls other components of the processing system 1 based on a processing program stored in the memory 312. The processor 311 transmits a request to generate impact information and outputs impact information generated by the server device 100 based on the processing program stored in the memory 312. Specifically, the processor 311 executes the following processes based on the processing program stored in the memory 312: "selecting, via the input interface 313, at least one of a subject image acquired via the imaging device 200 configured to image a subject, the subject's medical interview information, and the subject's findings information," "transmitting, via the communication interface 315, at least one of the selected subject image, the subject's medical interview information, and the subject's findings information to the server device 100," "acquiring impact information from the server device 100 via the communication interface 315," and "outputting impact information acquired via the output interface 314." The processor 111 is primarily composed of one or more CPUs, but may also be combined with a GPU, FPGA, or the like as appropriate.

[0025] The memory 312 is composed of RAM, ROM, non-volatile memory, HDD, SSD, etc., and functions as a storage unit. The memory 312 stores instructions and commands for various controls of the processing system 1 according to this embodiment as processing programs. Specifically, the memory 312 stores processing programs to be executed by the processor 311, such as "a process of selecting, via the input interface 313, at least one of a subject image acquired via the imaging device 200 configured to image a subject, the subject's medical interview information, and the subject's finding information," "a process of transmitting, via the communication interface 315, at least one of the selected subject image, the subject's medical interview information, and the subject's finding information to the server device 100," "a process of acquiring impact information from the server device 100 via the communication interface 315," and "a process of outputting the impact information acquired via the output interface 314."

[0026] The input interface 313 functions as an input unit that accepts user operation inputs to the terminal device 300. Examples of the input interface 313 include physical key buttons and a touch panel having an input coordinate system corresponding to the display coordinate system of the display. In the case of a touch panel, icons are displayed on the display, and the operator selects each icon by inputting instructions via the touch panel. The method for detecting the instruction input by the subject using the touch panel may be any method, such as a capacitive method or a resistive method. The input interface 313 does not always need to be physically provided on the terminal device 300, and may be connected as needed via a wired or wireless network. Therefore, in addition to the above, a mouse, a keyboard, etc. can also be used as the input interface 313.

[0027] The output interface 314 functions as an output unit for outputting information such as impact information received from the server device 100. An example of the output interface 314 is a display configured with a liquid crystal panel, an organic EL display, a plasma display, or the like. However, the terminal device 300 itself does not necessarily need to be equipped with a display. For example, an interface for connecting to a display or the like connectable to the terminal device 300 via a wired or wireless network can also function as the output interface 314 for outputting display data to the display or the like.

[0028] The communication interface 315 functions as a communication unit for transmitting and receiving determination requests, generation requests, determination results, impact information, etc., to and from the server device 100 and the image capturing device 200 connected via a wired or wireless network. Examples of the communication interface 315 include a wired communication connector such as a USB or SCSI, a wireless communication transmitting / receiving device for broadband wireless communication such as wireless LAN, Bluetooth (registered trademark), or LTE, or an infrared wireless communication, and various connection terminals for a printed circuit board or a flexible circuit board.

[0029] 3. Various Information Used in Processing in the Processing System 1 Figures 3A to 3C show various tables that store information that is stored in the server device 100 and provided to the imaging device 200 and terminal device 300 as the processing progresses. This information is updated and stored as the processing progresses. Note that the information shown in Figures 3A to 3C may be stored in the memory 112 of the server device 100, or may be stored in another remotely installed database device, electronic medical record device, or the like, and read out as the processing progresses.

[0030] FIG. 3A is a diagram conceptually illustrating a subject management table stored in the server device 100 according to an embodiment of the present disclosure. According to FIG. 3A , the subject management table stores, in association with subject ID information, assessment result information, medical interview information, findings information, subject image information, and history information. The "subject ID information" is information unique to each subject for identifying each subject. For example, the subject ID information is generated each time a new subject is registered by a user. However, the subject ID information may be any information that can identify the subject as described above. It is not limited to such information, but may also be unique identification information assigned by institutions such as Japan or other foreign countries, their local governments, schools, and workplaces, or a composite key generated by combining multiple types of identification information. The "assessment result information" is information indicating the result of an assessment of the subject's likelihood of suffering from one or more diseases. For example, such assessment result information is generated by inputting at least one of the subject image, medical interview information, and findings information into a trained assessment model in the server device 100.

[0031] The "medical interview information" is information input by, for example, a subject or a medical professional, etc., and is used as a reference for a doctor's diagnosis, such as the subject's medical history and symptoms. Examples of such medical interview information include patient background information such as weight, allergies, and underlying diseases, body temperature, peak body temperature since onset, time elapsed since onset, heart rate, pulse rate, oxygen saturation, blood pressure, medication status, contact with other infectious disease patients, subjective symptoms and physical findings such as joint pain, muscle pain, headache, fatigue, loss of appetite, chills, sweating, cough, sore throat, runny nose / nasal congestion, tonsillitis, gastrointestinal symptoms, rash on the hands and feet, redness or white coating of the pharynx, swollen tonsils, history of tonsillectomy, strawberry tongue, and swelling of tender anterior cervical lymph nodes, history of infectious disease vaccination, and timing of vaccination. The medical interview information is selected or input by accepting instruction input from the subject or a medical professional via the input interface 313 in the terminal device 300. The information is then transmitted from the terminal device 300 to the server device 100 and stored in the subject management table.

[0032] "Finding information" is information input by a medical professional such as a doctor, and indicates a condition that is different from normal obtained by various examinations of the subject, such as visual examination, questioning, palpation, auscultation, or percussion, or by tests to assist in diagnosis. Examples of such finding information include redness or white coating of the pharynx, swelling of the tonsils, the presence or absence of tonsillitis, redness or white coating of the tonsils, etc. The finding information is selected or input by accepting an instruction input by the medical professional via the input interface 313 in the terminal device 300. The finding information is then transmitted from the terminal device 300 to the server device 100, where it is stored in the subject management table.

[0033] "Subject image information" refers to image data of a subject's subject image captured using the imaging device 200 or the like, including at least the subject's pharynx as a subject. The image data may be one or more still images, one or more videos, or a combination thereof. The subject image information is stored by being received from at least one of the imaging device 200 and the terminal device 300 via the communication interface 113. The subject image information may be the image data itself captured by the imaging device 200, or may be data obtained by performing image processing such as sharpening on the image data. The subject image information is typically useful for diagnosing the possibility of infection with an infectious disease in which findings are found in the oral cavity, including the pharynx. Specifically, the subject image information may be used by a medical professional such as a doctor to make a diagnosis, or may be used in a processing device to diagnose the possibility of infection with one or more diseases or to generate information to assist in such diagnosis. Note that such subject image information may be various information analyzed from the image data instead of or together with the image data. For example, it may include values, classifications, or categories obtained by inputting image data into a trained analysis model to quantify redness of the throat, swelling of the tonsils, etc.

[0034] 3D is a diagram conceptually illustrating subject images stored in the server device 100 according to an embodiment of the present disclosure. According to FIG. 3D , subject image M1 is an image captured of the pharynx, which is the subject of the imaging device 200, and the surrounding area. Subject image M1 shows that follicles, which are characteristic findings of influenza, are observed in regions 37 and 38. Thus, subject images can be suitably used for determining the possibility of contracting one or more diseases based on visual findings or a trained determination model.

[0035] Returning to FIG. 3A again, the "history information" includes subject images previously captured of the subject, previously acquired findings, previously acquired interview information, previously acquired assessment result information, and previously acquired impact information, as well as information indicating the time at which each was acquired. Such history information is preferably used for comparison when outputting the current assessment result or impact information. Note that, as described above, such history information stores information acquired at multiple points in the past. Therefore, a history information management table may be provided for each subject ID information, separate from the subject management table, and the history information may be managed in that table.

[0036] Although not specifically shown, the information may also include attribute information such as the subject's name, age, sex, and date of examination.

[0037] 3B is a diagram conceptually illustrating a contribution management table stored in the server device 100 according to an embodiment of the present disclosure. According to FIG. 3B , the contribution management table stores contribution information in association with judgment item information. This contribution management table is generated in association with subject ID information of a specified subject each time a request for generating an impact level is received from the terminal device 300. When processing based on the generated contribution information is completed, the information stored in the contribution management table is stored as history information in association with the associated subject ID information.

[0038] The "determination item information" includes at least one of the subject image, medical interview information, and finding information input as input information in the trained determination model to generate a determination result. The information may be automatically acquired from information input as input information when the server device 100 performs a determination process to obtain a determination result indicating the possibility of one or more diseases. Alternatively, the information may be acquired based on an operator's selection from information input as input information when the server device 100 performs a determination process to obtain a determination result indicating the possibility of one or more diseases. The "contribution information" is a type of influence information. The contribution information indicates the degree of influence that at least one of the subject image, medical interview information, and finding information input to the trained determination model, i.e., each piece of information acquired as determination item information, has on the determination result acquired by the trained determination model. More specifically, the contribution information indicates the degree to which each piece of information acquired as determination item information, when input to the trained determination model, deviates from the average predicted value of the trained determination model. An example of such contribution information is the Shapley value. Details of the process of generating contribution information will be described later.

[0039] As an example, the contribution information is expressed as a specific numerical value, where a higher value indicates a higher contribution, i.e., an item that the trained judgment model paid more attention to when generating the judgment result, and a lower value indicates a lower contribution, i.e., an item that the trained judgment model paid less attention to when generating the judgment result. Note that the contribution information may be a classification or category such as "high," "medium," and "low" in addition to a numerical value.

[0040] 3B, the assessment item information includes, for example, pulse rate, body temperature, contact with the patient, joint pain, and loss of appetite, but may include other assessment information and findings information not shown here. Also, while FIG. 3B shows an example of a single subject image, the assessment item information may include multiple subject images.

[0041] 3C is a diagram conceptually illustrating a region-of-interest management table stored in the server device 100 according to an embodiment of the present disclosure. According to FIG. 3C , the region-of-interest management table stores region-of-interest information in association with subject image ID information. Each time a request for generating an influence level is received from the terminal device 300, the region-of-interest management table is generated in association with the subject ID information of the specified subject. When processing based on the generated region-of-interest information is completed, the information stored in the region-of-interest management table is stored as history information in association with the associated subject ID information.

[0042] The "subject image ID information" is information for identifying a subject image that includes at least the pharynx of a subject as a subject, which is photographed using the photographing device 200, etc. As an example, the subject image information is generated in association with the subject image each time the subject image is photographed by the photographing device 200.

[0043] "Area of ​​interest information" is one type of influence information. The area of ​​interest information is information indicating the degree of influence that a subject image input into a trained judgment model has on the judgment result obtained by the trained judgment model. More specifically, the area of ​​interest information is information indicating an area in the subject image that has a large influence on the judgment result when the judgment result is generated by inputting the subject image into the trained judgment model. In such influence information, the degree of influence on the judgment result is indicated in correspondence with the position information of the subject image. The process of generating contribution information will be described in detail later.

[0044] As an example, the attention area information is expressed as a specific numerical value corresponding to the coordinates of the subject image, where a higher value indicates a higher degree of attention, i.e., the area is more focused on by the trained judgment model when generating the judgment result, and a lower value indicates a lower degree of attention, i.e., the area is less focused on by the trained judgment model when generating the judgment result. Note that the contribution information may be a classification or category such as "high," "medium," or "low" in addition to a numerical value. Also, an attention area image may be generated in which an image indicating a numerical value is arranged in association with the position information of the subject image, and the image may be used as the attention area information.

[0045] 4. Processing Sequence Executed in Processing System 1 Fig. 4 is a diagram showing a processing sequence executed in the processing system 1 according to an embodiment of the present disclosure. Specifically, Fig. 4 shows a processing sequence executed between the server device 100, the image capture device 200, and the terminal device 300. Of these, S11 to S16 mainly represent processing for acquiring a subject image from at least one of the image capture device 200 and the terminal device 300, S21 to S24 represent processing for generating a determination result in the server device 100, and S31 to S38 represent processing for generating impact information in the server device 100.

[0046] (1) Subject Image Acquisition Process First, the subject image acquisition process will be described. Referring to FIG. 4 , the power button or the like is pressed to turn on the image capture device 200, and the image capture device 200 is activated (S11). Then, the image capture device 200 selects subject information of a subject to be captured from the subject information of uncaptured subjects received from the server device 100, based on the operator's instruction input received via the input interface 210 (S12). Next, the image capture device 200 determines whether or not an auxiliary tool is attached to the tip of the image capture device 200 to assist in inserting the image capture device into the oral cavity. If the auxiliary tool is not yet attached, the image capture device 200 outputs a display prompting the user to attach the auxiliary tool via the output interface (S13). Note that this display is merely an example, and the user may also be prompted to attach the auxiliary tool by sound, flashing light, vibration, or other means. When the image capture device 200 detects that the auxiliary tool has been attached (S14), the image capture device 200 captures a subject image based on the operator's instruction input received via the input interface (S15). Although the wearing of the assistive device is detected here, this process may be skipped.

[0047] When the subject image is captured, the photographing device 200 transmits the captured subject image (T11) together with the subject ID information of the photographed subject to the server device 100 via the communication interface. When the server device 100 receives the subject image via the communication interface 113, it stores the subject image in the subject management table in association with the subject ID information received together.

[0048] In Figure 4, the server device 100 has been described as acquiring a subject image from the photographing device 200, but when a subject image is captured by the photographing device 200, it may be transmitted to the terminal device 200, and the server device 100 may acquire the subject image from the terminal device 200.

[0049] (2) Processing for Generating Assessment Results Next, the processing for generating assessment results will be described. This processing is performed, for example, each time an assessment request is generated by the operator. As shown in FIG. 4 , the processor 311 of the terminal device 300 accepts selection of subject information for the subject to be assessed via the input interface 313 on a screen displaying a list of subject information for the subjects (S21). At this time, although not specifically shown, the terminal device 300 generates subject ID information, medical interview information, and findings information for the subject based on the input by the operator, and transmits the information to the server device 100 via the communication interface 315. Upon receiving the subject ID information, medical interview information, and findings information via the communication interface 115, the server device 100 stores the information in a subject management table.

[0050] Furthermore, when determining the possibility of affliction with one or more diseases, the processor 311 of the terminal device 300 accepts an operation input from the operator via the input interface 313 and selects a judgment threshold (S22). When the possibility of affliction with one or more diseases is obtained as output information in the trained judgment model, the judgment threshold is used to determine that there is a possibility of affliction when the probability exceeds the judgment threshold, and to determine that there is no possibility of affliction when the probability is equal to or less than the judgment threshold. In other words, S22 indicates that the operator can change such a judgment threshold to any value.

[0051] Here, Fig. 8A is a diagram illustrating an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. Specifically, it is a diagram illustrating an example of a determination instruction screen 20 output on the terminal device 300 when selecting a determination threshold performed in S22 of Fig. 4. According to Fig. 8A, the determination instruction screen 20 includes, at the top of the screen, attribute information of the subject selected in S21 of Fig. 4 , such information as the subject's name, age, sex, and examination date. The determination instruction screen 20 also includes, at the bottom, a medical interview information display area 21 and a finding information display area 22 that display the medical interview information and finding information generated in S21 of Fig. 4 . Furthermore, the determination instruction screen 20 includes a subject image display area 23 that displays the subject image captured in S15 of Fig. 4 obtained from the server device 100 or the imaging device 200.

[0052] Furthermore, the judgment instruction screen 20 has a judgment threshold adjustment area 25 for adjusting the judgment threshold. The judgment threshold adjustment area 25 includes a slide bar and slide button 26 that allow the operator to select a desired judgment threshold between thresholds T1 and T2. When the processor 311 of the terminal device 300 receives an operation input from the operator using the slide button 26, it moves the slide button 26 on the slide bar in accordance with the operation input. This allows the operator to select a desired judgment threshold between judgment threshold T1 (a lower threshold value) and judgment threshold T2 (a higher threshold value).

[0053] The judgment instruction screen 20 also includes, below the judgment threshold adjustment area 25, a sensitivity display area 27 and a specificity display area 28, which are display areas for other judgment parameters that are adjusted in conjunction with adjusting the judgment threshold. In this embodiment, a trained judgment model is typically used to determine the possibility of a specific disease. The sensitivity display area 27 displays a slide bar and slide button indicating the sensitivity, which indicates the probability that the trained judgment model will correctly determine a positive result. Similarly, the specificity display area 28 displays a slide bar and slide button indicating the sensitivity, which indicates the probability that the trained judgment model will correctly determine a negative result. The processor 311 of the terminal device 300 controls the display so that these slide buttons also move in accordance with the movement of the slide button 26 in the judgment threshold adjustment area 25. For example, in the example of Figure 8A, when the slide button 26 in the judgment threshold adjustment area 25 is moved to the left (toward a lower threshold), the slide button in the sensitivity display area 27 moves in the direction of increasing sensitivity, and the slide button in the specificity display area 28 moves in the direction of decreasing specificity.

[0054] When the adjustment of the judgment thresholds and the like is completed, the processor 311 of the terminal device 300 receives the operator's input to the judgment button 24 via the input interface 314, and transmits a judgment request shown as T21 in Figure 4 to the server device 100.

[0055] 8A allows the user to more intuitively adjust the judgment threshold using the slide button in the judgment threshold adjustment area 25. Furthermore, the slide buttons in the sensitivity display area 27 and the specificity display area 28 also move in response to adjustment of the slide button, allowing the operator to more accurately grasp the relationship between the judgment threshold and the sensitivity and specificity. This allows the operator to adjust the judgment threshold to a lower value to increase sensitivity, for example, when determining the possibility of contracting a certain disease, for example, when screening infected patients or when more aggressively preventing the spread of infection during an epidemic. Furthermore, when more reliable diagnosis is desired during a non-epidemic period, for example, the judgment threshold can be adjusted to a higher value to increase specificity.

[0056] Note that while a slide bar and slide button are used to select the judgment threshold in FIG. 8A , this is merely an example. For example, a box may be provided in which a desired numerical value can be input or selected as the judgment threshold. Furthermore, while the judgment parameters adjusted in accordance with the adjustment of the judgment threshold are described using sensitivity and specificity as examples in FIG. 8A , other judgment parameters such as a positive agreement rate and a negative agreement rate may also be used in addition to or instead of these.

[0057] Furthermore, a different threshold value may be selected for each disease to be determined, or the same threshold value may be selected for all diseases to be determined.

[0058] Returning to FIG. 4 , the processor 311 of the terminal device 300 then references the subject management table to read the subject ID information of the subject, and transmits the read information, along with a request (T21) for determining the possibility of the subject having one or more diseases and a determination threshold, to the server device 100 via the communication interface 315. Upon receiving the determination request, the server device 100 references the subject image in the subject management table based on the subject ID information received together, and reads the subject image, medical interview information, and findings information associated with the subject ID information. The server device 100 then determines the possibility of the subject having each disease based on the read subject image, and stores the determination result in the subject management table (S23). Details of the determination process will be described later.

[0059] Once the determination is made, the server device 100 outputs the stored determination result (T22) to the terminal device 300 that sent the determination request. The processor 311 of the terminal device 300 displays the determination result received via the communication interface 315 on the display via the output interface 314.

[0060] (3) Impact Information Generation Process Next, the process for generating impact information will be described. This process is performed, for example, each time an impact generation request is generated by an operator. As shown in FIG. 4 , the processor 311 of the terminal device 300 accepts an operator's input via the input interface 313 on the impact request screen for a specific subject and selects a judgment item for which a contribution level, which is one of the impact levels, is to be generated (S31). The processor 311 of the terminal device 300 also accepts an operator's input via the input interface 313 on the impact request screen and selects a subject image for which a region of interest, which is one of the impact levels, is to be generated (S32). The processor 311 of the terminal device 300 then accepts an operator's input via the input interface 313 on the impact request screen and selects an execute button to generate impact information (S33).

[0061] Then, the processor 311 of the terminal device 300 reads out the subject ID information of the subject on whose impact request screen the impact request screen is displayed, and transmits it to the server device 100 via the communication interface 315 together with a request to generate impact information (T31) and information identifying the judgment items and subject image selected in S31 and S32.

[0062] Here, Fig. 8B is a diagram showing an example of a screen output on the terminal device 300 according to an embodiment of the present disclosure. Specifically, Fig. 8B is a diagram showing an example of an impact request screen 10 output on the terminal device 300. According to Fig. 8B, the subject's name, age, sex, and consultation date are output at the top of the screen as the subject's attribute information.

[0063] The impact request screen 10 also includes an interview information selection area 11, a finding information selection area 12, and a subject image selection area 13. The interview information selection area 11 receives an operation input from an operator via an input interface 313, and outputs interview information (determination items) selected as a target for generating an impact from among the interview information input as input information to the determined learning model. The selection may be made by any of the following methods: the operator inputs the desired interview information as text via the input interface 313; a check box or the like is prepared in association with each piece of interview information and the operator selects the check box corresponding to the desired interview information; or a method in which the operator copies and pastes the desired interview information from a list of interview information.

[0064] Note that selection of medical interview information is not necessarily required; for example, medical interview information input into the learned judgment model may be obtained in advance, and the medical interview information may be automatically output to the medical interview information selection area 11.

[0065] The finding information selection area 12 receives an operation input from the operator via the input interface 313, and outputs finding information (determination items) selected as targets for generating an influence degree from the finding information input as input information to the determined learning model. The selection may be made by any method, such as a method in which the operator inputs the desired finding information as text via the input interface 313, a method in which check boxes or the like are prepared in association with each finding information and the operator selects the check box corresponding to the desired finding information, or a method in which the operator copies and pastes the desired finding information from a list of finding information.

[0066] Note that selection of the finding information is not necessarily required; for example, the finding information input to the learned judgment model may be acquired in advance, and the finding information may be automatically output to the finding information selection area 12.

[0067] The subject image selection area 13 receives an operation input from the operator via the input interface 313, and outputs a subject image selected as a target for generating an influence from among the subject images input as input information to the determined learning model. The selection may be made by any of the following methods: a method in which the operator prepares check boxes or the like in association with each subject image via the input interface 313 and selects a check box corresponding to a desired subject image; a method in which the operator copies and pastes a desired subject image from a list of subject images; or a method in which the operator copies and pastes a desired subject image from a folder in which subject images are stored.

[0068] Note that selection of a subject image is not necessarily required; for example, the subject image input to the learned judgment model may be acquired in advance, and the subject image may be automatically output to the subject image selection area 13.

[0069] The impact request screen 10 also includes an execute button 14. The execute button 14 serves as a trigger for transmitting an impact generation request to the server device 100. That is, by receiving an operation input from the operator to the execute button 14 via the input interface 313, an impact generation request is transmitted to the server device 100 together with the subject ID information and each piece of information selected in the interview information selection area 11, the finding information selection area 12, and the subject image selection area 13.

[0070] Returning to FIG. 4 , when the processor 111 of the server device 100 receives the impact generation request, it reads the subject ID information received together with the request and information identifying the assessment items and subject image selected in S31 and S32 from the subject management table. The processor 111 of the server device 100 then executes a contribution analysis process based on the subject image, medical interview information, and findings information read as assessment items (S34). The server device 100 also executes a region-of-interest analysis process based on the read subject image (S35) and a segmentation process based on the subject image (S36). Details of each process will be described later.

[0071] When the processes of S34 to S36 are completed, the processor 111 of the server device 100 generates impact information (S37) and transmits the generated impact information (T32) via the communication interface 113 to the terminal device 300 that sent the impact generation request. The processor 311 of the terminal device 300 displays the impact information received via the communication interface 315 on the display via the output interface 314.

[0072] 9A to 11 are diagrams showing examples of screens output on the terminal device 300 according to an embodiment of the present disclosure. Specifically, FIGS. 9A to 11 are diagrams showing examples of impact information screens 30a to 30e, an impact information screen 60, and an impact information screen 70 output on the terminal device 300.

[0073] 9A is an example of an impact information screen, and on the impact information screen 30a, the subject's name, age, sex, and consultation date are displayed at the top of the screen as the subject's attribute information. Also, below the screen, the determination result indicating the possibility of the disease generated in S23 is displayed. Note that while influenza is shown as an example of a disease here, it may of course be a disease other than influenza, or multiple diseases. Also, while the positivity rate is shown here, it may simply be "yes" or "no" regarding the possibility of the disease based on a determination threshold.

[0074] The influence information screen 30a also includes a contribution display area 31 and a focus area display area 32. The contribution display area 31 includes a bar graph corresponding to the contribution (influence) of each of the subject image, medical interview information, and finding information selected as the judgment items in S31 to the judgment result. Specifically, the contribution of each of the input subject image, medical interview information, and finding information to the judgment result obtained by a trained judgment model (e.g., the second trained model in FIG. 6 ) using the subject image, medical interview information, and finding information as input is shown.

[0075] 9A, for example, the value of the contribution of the pulse rate generated by the process of S34 (K －２ From K－３ 9A, a bar graph corresponding to the contribution of the pulse rate to the determination result (values ​​between K and K) is displayed. This indicates that the contribution of the pulse rate to the determination result is relatively low compared to other determination items (for example, body temperature). On the other hand, in FIG. 9A, for example, the contribution of the body temperature is displayed by the process of S34. ３ This indicates that the contribution of body temperature to the assessment result is relatively high compared to other assessment items (e.g., contact with other patients).

[0076] It should be noted that the above-described bar graph showing the degree of influence in the contribution degree display area 31 is merely an example. For example, it is of course possible to show the degree of influence as a numerical value or in other ways.

[0077] The attention area display area 32 also includes an attention area image 34 that indicates an area that affected the determination result for the selected subject image, together with the subject image 40. Specifically, the attention area in the input subject image is displayed for the determination result obtained by a trained determination model (e.g., the first trained model in FIG. 6 ) using the subject image as input.

[0078] 9A , the attention area image 34 is an image in which the numerical values ​​indicating the attention level generated by the processing of S35 are distinguished by color schemes or shades of color so as to correspond to the position information of the subject image. In the example of Fig. 9A , dark colors are displayed near areas 38 and 37 where follicular findings are observed, and gradually lighter colors are displayed as the distance from these areas increases, with no color outside the attention area image 34. This suggests that the attention level is particularly high near areas 38 and 37, and that the vicinity of the follicles was focused on when generating the judgment result using the trained judgment model.

[0079] Furthermore, in the attention area display area 32, a frame 39 corresponding to the posterior pharyngeal wall and a frame 33 corresponding to the uvula, which were identified as segments by the processing of S36, are displayed, and names 35 and 36 are displayed corresponding to the identified segments. This allows an operator, etc., who refers to the attention area display area 32, to more intuitively identify the area of ​​interest.

[0080] 9A , the contribution (influence) of the medical interview information and the finding information to the determination result is displayed side by side for comparison in the contribution display area 31, and the region of interest of the subject image is displayed in the region of interest display area 32. Therefore, by associating the contribution (influence) of the medical interview information and the finding information with the region of interest of the subject image, it becomes possible to infer in more detail the basis for the determination result.

[0081] It should be noted that the above-described heat map representation of the attention level in the attention area display area 32 is merely an example. For example, it is of course possible to represent the attention level as a numerical value or in other ways.

[0082] Furthermore, although only the subject image 40 is displayed in the attention area display area 32, similar images may be displayed in sequence for the other subject images selected in S32 by accepting an operator's operation input to an icon (not shown) via the input interface 313. Alternatively, a certain number of subject images and their attention area images may be displayed side by side so that they can be compared.

[0083] 9B is another example of the influence information screen. The influence information screen 30b in FIG. 9B includes a contribution display area 43, a first attention area display area 45, and a second attention image display area 46. Note that, except for the portions specifically mentioned, the influence information screen 30b is the same as the influence information screen 30a in FIG. 9A, and therefore, a description of the points common to the influence information screen 30a will be omitted.

[0084] 6, in this embodiment, a plurality of subject images are input to each trained determination model to obtain a determination result (S215 in FIG. 6, etc.). Therefore, for each subject image input to the trained determination model, it is possible to detect the contribution (influence) of the subject image to the determination result and the region of interest in the subject image.

[0085] 9B , the first attention area display area 45 and the second attention image display area 46 display information related to the judgment results obtained by inputting the subject image A and the subject image B into the trained judgment model, respectively. Specifically, the first attention area display area 45 displays the attention area in the input subject image A for the judgment result obtained by the trained judgment model (e.g., the first trained model of FIG. 6 ) using the subject image A as input. Similarly, the second attention area display area 46 displays the attention area in the input subject image B for the judgment result obtained by the trained judgment model (e.g., the first trained model of FIG. 6 ) using the subject image B as input. In addition to these, the contribution display area 43 displays the contributions of the input subject image A, subject image B, medical interview information, and finding information to the judgment result obtained by the trained judgment model (e.g., the second trained model of FIG. 6 ) using the subject image A, subject image B, medical interview information, and finding information as input.

[0086] In this way, by outputting the contribution (influence) of each object image input to the trained judgment model to the judgment result and the region of interest in the object image, it is possible to estimate the influence each object image had on the judgment result. In other words, it is possible to estimate the object image that had a significant influence on the judgment result, and further to understand the basis for which region of that object image the trained judgment model focused on.

[0087] 9C is another example of an influence information screen. The influence information screen 30c in FIG. 9C includes a contribution display area 47, a first attention area display area 48a, and a second attention area display area 48b. Note that, except for portions specifically mentioned, the influence information screen 30c is the same as the influence information screen 30a in FIG. 9A, and therefore, a description of the points common to the influence information screen 30a will be omitted.

[0088] As described above, in this embodiment, multiple subject images are input to each trained judgment model to obtain judgment results (e.g., S215 in FIG. 6 ). Therefore, for each subject image input to the trained judgment model, it is possible to detect the contribution (influence) of the subject image to the judgment result and the area of ​​interest in the subject image. As shown in FIG. 9C , the first area of ​​interest display area 48a displays the area of ​​interest in the input subject image C for the judgment result obtained by the trained judgment model (e.g., the first trained model in FIG. 6 ) using subject image C as input. The second area of ​​interest display area 48b displays the area of ​​interest in the input subject image D for the judgment result obtained by the trained judgment model (e.g., the first trained model in FIG. 6 ) using subject image D as input. Furthermore, the contribution display area 47 displays the contribution of the subject images C and D, the medical interview information, and the findings information to the judgment result obtained by the trained judgment model (e.g., the second trained model in FIG. 6 ), as a result of summing the judgment results obtained from the subject images C and D.

[0089] In this way, by indicating the area of ​​interest for each subject image, it is possible to understand the degree of contribution that the subject image made to the judgment result, and then estimate which subject image contributed more and which area the trained judgment model paid more attention to in the judgment.

[0090] 9D is another example of an influence information screen. The influence information screen 30d of FIG. 9D includes a contribution display area 47, a first attention area display area 49a, a second attention area display area 49b, a third attention area display area 49d, and a fourth attention area display area 49d. Note that, except for portions specifically mentioned, the influence information screen 30d is the same as the influence information screen 30a of FIG. 9A, and therefore, a description of the points common to the influence information screen 30a will be omitted.

[0091] As described above, in this embodiment, multiple subject images are input to each trained judgment model to obtain a judgment result (e.g., S215 in FIG. 6 ). Therefore, for each subject image input to the trained judgment model, it is possible to detect the contribution (influence) of the subject image to the judgment result and the region of interest in the subject image. As shown in FIG. 9C , the contribution display area 47 displays the contribution of subject images E to H, medical interview information, and findings information to the judgment result obtained from the trained judgment model (e.g., the second trained model in FIG. 6 ) as a result of adding up the judgment results obtained from subject images E to H.

[0092] The first attention area display area 49a shows the attention area of ​​the trained determination model (for example, the first trained model in FIG. 6) of the subject image E, the second attention area display area 49b shows the attention area of ​​the subject image F, the third attention area display area 49d shows the attention area of ​​the trained determination model (for example, the first trained model in FIG. 6) of the subject image G, and the fourth attention area display area 49d shows the attention area of ​​the trained determination model (for example, the first trained model in FIG. 6) of the subject image H. Here, the contribution of the subject image E to the determination result is K. ３ , the contribution of the subject image F to the determination result is K ２ , the contribution of the subject image G to the determination result is K １ , the contribution of the subject image H to the determination result is K －１ , that is, the degree of contribution to the determination structure is assumed to be greatest in the order of subject image E, subject image F, subject image G, and subject image H. In such a case, as shown in Fig. 9D, the images can be displayed in order of decreasing degree of contribution in the first attention region display region 49a, the second attention region display region 49b, the third attention region display region 49d, and the fourth attention region display region 49d.

[0093] In this way, by indicating the area of ​​interest for each subject image, it is possible to understand the degree of contribution that the subject image made to the judgment result, and then estimate which subject image contributed more and which area the trained judgment model paid more attention to in the judgment.

[0094] 9D also shows the regions of interest for four subject images, subject image E, subject image F, subject image G, and subject image H, side by side. Follicles can be identified in the areas surrounded by dashed lines in each image in FIG. 9D . For example, referring to subject image E, subject image F, and subject image H in FIG. 9D , it can be seen that the degree of attention increases in the region of interest seen on the center right side in the order of subject image H, subject image F, and subject image E. This trend coincides with the range of follicles identified as findings. Therefore, it is possible to estimate the correlation between the region of interest and findings in the judgment of a trained judgment model (e.g., the first trained model in FIG. 6 ).

[0095] Note that various modifications can be made to the first attention area display area 49a, the second attention area display area 49b, the third attention area display area 49d, and the fourth attention area display area 49d, as follows: While four areas are depicted in FIG. 9D , there may be any number of areas, such as one to three or five or more. While the subject images displayed in each area are displayed in descending order of contribution, they may be displayed in any order, such as descending order of contribution, order of capture, or order of attention (e.g., the calculated cumulative value of attention), or a combination of these. A small number of attention area display areas may be set relative to the number of subject images input to the trained model, and the images may be selected and displayed in descending order of contribution or attention.

[0096] 9E is another example of an influence information screen. The influence information screen 30e in FIG. 9E includes a contribution display area 50 and a focus area display area 51. Note that, except for portions specifically mentioned, the influence information screen 30e is the same as the influence information screen 30a in FIG. 9A, and therefore, a description of the points common to the influence information screen 30a will be omitted.

[0097] 9E, instead of displaying numerical values ​​indicating the degree of attention in colors or shades of colors corresponding to the position information of the subject image in attention area display area 51, the degree of attention is shown as a graph for each segment (for example, attention area image 34 in FIG. 9A). Specifically, in FIG. 9E, for the uvula, posterior pharyngeal wall, and soft palate identified as segments by the process of S36 in FIG. 4, the degree of attention area (degree of attention) included in each segment is added up, and the obtained numerical value is shown as a graph for each segment.

[0098] In this way, by displaying the integrated value of the degree of attention for each segment as a graph, the level of attention for each segment can be more intuitively grasped, and it becomes easier to compare the degree of attention between segments.

[0099] 9E shows the integrated value for each segment, but it is also possible to obtain the area of ​​each segment that shows an attention level equal to or greater than a predetermined threshold, and to display this area value as a graph. Also, while Fig. 9E does not weight the attention level, it is also possible to weight the attention level of each area included in a segment according to its size, and to integrate the weighted values.

[0100] 10 is another example of an impact information screen, and the impact information screen 60 of FIG. 10 includes a contribution display area 61 and a contribution display area 62, each of which includes a bar graph corresponding to the contribution (impact) that each judgment item has made to the judgment result. Note that, except for the parts that are particularly mentioned, the impact information screen 60 is the same as the impact information screen 30a of FIG. 9A, and therefore a description of the points that are common to the impact information screen 30a will be omitted.

[0101] The contribution display area 61 displays a bar graph corresponding to the current numerical value for each judgment item generated by the process of S34, and the judgment result (influenza positivity rate) generated by the process of S23. On the other hand, the contribution display area 62 displays a bar graph corresponding to the past numerical value (e.g., three days ago) for each judgment item generated by a process similar to the process of S34, and the past judgment result (influenza positivity rate) generated by the process of S23. The numerical values ​​shown in the contribution display area 62 are obtained by the processor 111 of the server device 100 by reading out past numerical values ​​for a specific date specified by the operator, for example, from the history information in the subject management table.

[0102] In this way, by outputting the current and past bar graphs as well as the current and past influenza positivity rates side by side, it is possible to understand the progression of symptoms and also to estimate the impact that changes in symptoms have had on the influenza positivity rate determination results.

[0103] Although only two areas, the contribution display area 61 and the contribution display area 62, are shown in FIG. 10, it goes without saying that there may be three or more areas.

[0104] Fig. 11 is yet another example of the impact information screen, and the impact information screen 70 in Fig. 11 includes an attention area display area 71 and an attention area display area 72. Note that, except for the parts that are particularly mentioned, the impact information screen 70 is the same as the impact information screen 30a in Fig. 9A, and therefore a description of the points that are common to the impact information screen 30a will be omitted.

[0105] The attention area display area 71 includes an attention area image 73, which is an image corresponding to a numerical value indicating the current attention level generated by the processing of S35, and the determination result (influenza positivity rate) generated by the processing of S23, so as to correspond to the position information of the subject image. On the other hand, the attention area display area 72 includes an attention area image 74, which is an image corresponding to a numerical value indicating a past attention level generated for each predetermined pixel of the subject image by the same processing as S35, and the past determination result (influenza positivity rate) generated by the processing of S23. The numerical values ​​shown in the attention area display area 72 are obtained by the processor 111 of the server device 100 by reading out past numerical values ​​for a specific date specified by the operator, for example, from the history information in the subject management table.

[0106] In this way, by outputting the current influenza positivity rate and the past influenza positivity rate side by side in addition to the current attention area image 73 and the past attention area image 74, it is also possible to estimate the influence that the attention area has had on the influenza positivity rate determination result.

[0107] Although only two regions, the attention region display region 71 and the attention region display region 72, are shown in FIG. 11, it goes without saying that there may be three or more regions.

[0108] In addition, each of the impact information screens of Figures 9A to 9E, the impact information screen 60 of Figure 10, and the impact information screen 70 of Figure 11 may be selected according to the operator's wishes by accepting the operator's operational input at the input interface 313 of the terminal device 300.

[0109] 5. Processing Flow Executed by Server Device 100 (1) Determination Result Generation Processing Fig. 5 is a diagram showing a processing flow executed by the server device 100 according to an embodiment of the present disclosure. Specifically, Fig. 5 is a diagram showing a processing flow executed by the server device 100 in the determination result generation processing of S21 to S24 in the processing sequence of Fig. 4. This processing flow is mainly performed by the processor 111 of the server device 100 reading and executing a program stored in the memory 112.

[0110] 5, the processor 111 receives a request for determining the possibility of the subject having one or more diseases, along with the subject ID information of the subject selected as the subject of determination, from the terminal device 300 via the communication interface 113 (S111). Then, the medical interview information and findings information entered by the operator in the terminal device 300 or the like are stored in the subject management table in association with the subject ID information. Therefore, based on the received subject ID information, the processor 111 references the medical interview information and findings information in the subject management table and reads out the medical interview information and findings information associated with the subject ID information of the subject (S112 and S113). Based on the same received subject ID information, the processor 111 references the subject image in the subject management table and reads out the subject image associated with the subject ID information of the subject (S114).

[0111] Next, the processor 111 executes a process of determining the possibility of contracting one or more diseases using the readout medical interview information, finding information, and subject image (S115).

[0112] Here, one example of the above-mentioned determination process is to input this information into a trained determination model as shown below and make a determination. However, it is not limited to this processing method, and any processing method can be adopted, such as a processing method in which a determination is made based on the degree of coincidence with an image showing the diseased state by image analysis processing.

[0113] Fig. 6 is a diagram showing a processing flow executed in the server device 100 according to an embodiment of the present disclosure. Specifically, Fig. 6 is a diagram showing a detailed processing flow of the determination processing executed in S115 of Fig. 5. This processing flow is mainly performed by the processor 111 of the server device 100 reading and executing a program stored in the memory 112.

[0114] According to Figure 6, the processor obtains the first positive rate, the second positive rate, and the third positive rate, which indicate the possibility of contracting one or more diseases, using different processes, and then obtains a judgment result by ensemble processing them.

[0115] <Subject Image Screening Process> First, the subject image screening process will be described. The processor 111 acquires multiple subject images associated with the subject ID information of the subject. The processor 111 then executes a process of screening the acquired multiple subject images to be used to obtain the determination result (S211). The screening process is performed by labeling the training images in advance based on whether they can be used effectively for determination, and using a trained image selection model obtained by training based on the training images and the label information. Specifically, the processor inputs the received multiple subject images into the trained image selection model and acquires the multiple subject images to be used to obtain the determination result as output information.

[0116] <Preprocessing of Subject Image> Next, preprocessing performed on the subject image will be described. The processor 111 acquires the subject image screened in S211 and performs predetermined preprocessing on the acquired subject image (S212). The processor 111 stores the preprocessed subject image in a subject management table. Examples of such preprocessing include filtering such as bandpass filters (including high-pass filters and low-pass filters), averaging filters, Gaussian filters, Gabor filters, Canny filters, Sobel filters, Laplacian filters, median filters, and bilateral filters; vascular extraction using Hessian matrices; segmentation of specific regions (e.g., follicles) using machine learning; trimming of the segmented region; dehazing; super-resolution processing; and combinations thereof, including high-definition processing, region extraction, noise removal, edge enhancement, image correction, and image conversion. These processes can be appropriately selected depending on the purpose. By performing preprocessing in this way, it is possible to improve the accuracy of diagnosis by extracting or enhancing regions of interest that are important in diagnosing one or more diseases.

[0117] <Processing for Obtaining the Determination Result of the First Positive Rate> Next, the processing for obtaining the determination result of the first positive rate will be described. The processor 111 accesses the subject management table and obtains preprocessed subject images. The processor 111 then provides the obtained preprocessed subject images as input to a feature extractor (S213), obtains image features of the plurality of subject images as output, and stores the output in the subject management table (S114). The processor 111 then provides the plurality of subject images or the features of each subject image as input to a first trained determination model (S215), and obtains a first positive rate indicating a first possibility of contracting one or more diseases as output (S216). The processor 111 stores the obtained first positive rate in the subject management table.

[0118] The first trained judgment model is generated by providing a learning device with a pair of a subject image or a feature of the subject image and label information labeled with the presence or absence of one or more diseases determined in the subject of the subject image, and performing machine learning. Such a trained judgment model can also be generated using machine learning such as a neural network, a convolutional neural network, a multi-layer Hercepton (MLP), a long short term memory (LSTM), a gated recurrent unit (GRU), a graph neural network (GNN), a transformer, or the like, a gradient boosting decision tree (GBDT) method such as LightGradientBoostingMachine (LightGBM), XGBoost, or CatBoost, a ridge regression, a logistic regression, a support vector regression (SVR), a nearest neighbor method, a decision tree, a regression tree, or a random forest. Among these, it is preferable to use a convolutional neural network as the trained judgment model from the viewpoint of processing the subject image and identifying the region of interest.

[0119] <Processing for Acquiring the Determination Result of the Second Positive Rate> Next, the processing for acquiring the determination result of the second positive rate will be described. The processor 111 accesses the subject management table and acquires at least one of the medical interview information and the findings information associated with the subject ID information of the subject to be determined (S217). The processor 111 also accesses the subject management table and acquires the feature values ​​of the subject image calculated in S114 and stored in memory in association with the subject ID information (S214). The processor 111 then provides at least one of the acquired medical interview information and the findings information, the subject image, the features of the subject image, or the first positive rate obtained as the determination result by the first trained determination model as input to the second trained determination model (S218), and acquires as output a second positive rate indicating a second possibility of contracting one or more diseases (S219). The processor 111 stores the acquired second positive rate in the subject management table.

[0120] The second trained judgment model is generated by providing a learning device with a set of a subject image or at least one of the subject image's features, medical interview information, and findings information, and label information labeled with the presence or absence of one or more diseases determined in the subject who is the subject of the subject image, and performing machine learning. Such a trained decision model can also be generated using machine learning such as a neural network, a convolutional neural network, a multi-layer Herceptron (MLP), a long short term memory (LSTM), a gated recurrent unit (GRU), a graph neural network (GNN), a transformer, or the like; a gradient boosting decision tree (GBDT) method such as Light Gradient Boosting Machine (LightGBM), XGBoost, or CatBoost; ridge regression, logistic regression, support vector regression (SVR), a nearest neighbor method, a decision tree, a regression tree, or a random forest.

[0121] <Processing for Acquiring the Determination Result of the Third Positive Rate> Next, the processing for acquiring the determination result of the third positive rate will be described. The processor 111 accesses the subject management table and acquires at least one of the medical interview information and the finding information associated with the subject ID information of the subject to be assessed (S217). The processor 111 then provides at least one of the read medical interview information and the finding information as input to the third trained determination model (S220), and acquires as output a third positive rate indicating a third possibility of contracting one or more diseases (S221). The processor 111 stores the acquired third positive rate in the subject management table.

[0122] The third trained judgment model is generated by providing a learning device with a pair of at least one of the medical interview information and the finding information, and label information labeled with the presence or absence of one or more diseases determined in the subject for whom the medical interview information was input, and performing machine learning. Such a trained decision model can also be generated using machine learning such as a neural network, a convolutional neural network, a multi-layer Herceptron (MLP), a long short term memory (LSTM), a gated recurrent unit (GRU), a graph neural network (GNN), a transformer, or the like; a gradient boosting decision tree (GBDT) method such as Light Gradient Boosting Machine (LightGBM), XGBoost, or CatBoost; ridge regression, logistic regression, support vector regression (SVR), a nearest neighbor method, a decision tree, a regression tree, or a random forest.

[0123] <Post-processing (ensemble)> Next, post-processing (ensemble) will be described. As described above, once the first positive rate, second positive rate, and third positive rate have been calculated, the processor 111 accesses the subject management table and acquires each positive rate. The processor 111 then performs ensemble processing using each acquired positive rate (S222). As an example of this ensemble processing, the acquired first positive rate, second positive rate, and third positive rate are input to a ridge regression model, and a result in which each positive rate is ensembled is acquired as a determination result of the possibility of contracting one or more diseases.

[0124] The ridge regression model used in S222 is generated by machine learning by the processor 111 of the server device 100 or the processor 111 of another processing device. Specifically, the processor 111 acquires the first positive rate, the second positive rate, and the third positive rate from the training images. The processor 111 also assigns correct labels to the subjects who are the subjects of the training images based on the diagnosis results of doctors in advance. The processor 111 then provides the ridge regression model with a set of each positive rate and the corresponding correct label, and repeats learning while adjusting the parameters provided for each positive rate so that the output is the same as the correct label information of the ridge regression model. This results in a ridge regression model to be used in ensemble processing.

[0125] Furthermore, although the use of a ridge regression model has been described as an example of ensemble processing, any method may be used, such as a process for obtaining the average value of each positive rate, a process for obtaining the maximum value, a process for obtaining the minimum value, a weighted addition process, or a process using other machine learning methods such as bagging, boosting, stacking, lasso regression, or linear regression.

[0126] In this embodiment, as shown in S22 of Fig. 4, a judgment threshold is selected by an operator's input. Then, at T21 of Fig. 4, information on the judgment threshold is sent to the server device 100 along with the judgment supply. Therefore, the processor 111 reads the received judgment threshold (S223) and compares it with the post-ensemble positivity rate acquired in S222. As a result of the comparison with the judgment threshold, if the positivity rate exceeds the threshold, the processor 111 generates a judgment result of "positive" for one or more diseases, and if the positivity rate is equal to or less than the threshold, generates a judgment result of "negative" for one or more diseases (S224).

[0127] The processor 111 stores the determination result thus obtained in the subject management table in association with the user ID information (S224), and then ends the processing flow.

[0128] 5, the judgment process in S115 is, for example, performed according to the processing flow shown in Fig. 6. That is, the processor 111 reads the medical interview information, finding information, and subject image acquired in S112 to S114 from the memory 112, and provides them as input information to the first trained judgment model, the second trained judgment model, and the third trained judgment model, respectively. Then, the processor 111 acquires, from each trained judgment model, a judgment result indicating the possibility of the subject having one or more diseases based on the input information.

[0129] The processor 111 stores the determination result information indicating the possibility of contracting each disease obtained by the determination process in the subject management table in association with the subject ID information. The processor 111 then outputs the determination result information to the terminal device 300 that sent the determination request via the communication interface 115 (S116). This completes the processing flow for generating the determination result.

[0130] (2) Impact Information Generation Process Fig. 7 is a diagram showing a processing flow executed by the server device 100 according to an embodiment of the present disclosure. Specifically, Fig. 7 is a diagram showing a processing flow executed by the server device 100 in the impact information generation processing of S31 to S38 in the processing sequence of Fig. 4. This processing flow is mainly performed by the processor 111 of the server device 100 reading and executing a program stored in the memory 112.

[0131] 7, the processor 111 receives an impact generation request from the terminal device 300 via the communication interface 113 (S311). Then, the processor 111 reads out the medical interview information, the findings information, and the subject image from the subject management table based on the subject ID information received together with the impact generation request and the information specifying the assessment items and the subject image selected in S31 and S32 of FIG. 4 (S312).

[0132] The processor 111 executes a contribution analysis process to acquire contribution information, which is one type of influence information, based on the read information (S313). Here, the contribution information is information regarding the directionality of the influence of local input information on the predicted value and the amount of influence. Examples of such contribution information include a Shapley value, which indicates the degree of deviation from the expected value of the prediction of the trained judgment model when certain judgment item information is input, or an approximation of the Shapley value, and the output value of LIME (Local Interpretable Model-agnostic Explanations).

[0133] That is, as an example, the contribution information is generated by the processor 111 executing the following process. Note that, for ease of explanation, a case in which the contribution of "pulse rate" is calculated will be described here. First, the processor 111 uses the selected judgment items to generate all possible combinations of judgment items (e.g., pulse rate, body temperature, loss of appetite, pulse rate and body temperature, pulse rate and loss of appetite, and pulse rate, body temperature and loss of appetite, etc.). Then, the processor 111 provides each combination as input information to a trained judgment model (e.g., the second trained judgment model) shown in FIG. 6 and obtains the expected value of the trained judgment model (a judgment result (positive rate) indicating the possibility of contracting one or more diseases).

[0134] Next, processor 111 provides each combination that does not include "pulse rate" as input information to the trained judgment model, and obtains a first output value (a judgment result (positive rate) indicating the possibility of contracting one or more diseases). Processor 111 then calculates the difference between the first output value and the expected value. Processor 111 also provides each combination that includes "pulse rate" as input information to the trained judgment model, and obtains a second output value (a judgment result (positive rate) indicating the possibility of contracting one or more diseases). Processor 111 then calculates the difference between the second output value and the expected value.

[0135] The processor 111 calculates the marginal contribution of the "pulse rate" to the expected value of the trained judgment model by subtracting the difference calculated from the first output value from the difference calculated from the second output value. The processor 111 acquires the average value of all the calculated marginal contributions as contribution information. The processor 111 then performs the same process for other judgment items other than the "pulse rate" and acquires each average value as contribution information.

[0136] Note that, when the judgment results of multiple trained judgment models (first trained judgment model, second trained judgment model, and third trained judgment model) are ensembled, as in the judgment process described in FIG. 6 , the contribution information can be further calculated as follows. For example, when trained judgment models with the same structure are used as each trained judgment model, the following method can be adopted. The processor 111 calculates the contribution (e.g., Shapley value) for each judgment item for each trained judgment model for each judgment result obtained from multiple trained judgment models (e.g., multiple second trained judgment models obtained by cross-validation) obtained by training judgment models with the same structure using training information from different subject groups. Then, the processor 111 adds up the calculated contributions and sets the resulting value as contribution information for each judgment item. Note that this method is not limited to this, and other methods are also possible for obtaining contribution information.

[0137] Furthermore, for example, when trained judgment models with different structures are used as the respective trained judgment models, the following method can be adopted. The processor 111 calculates the contribution (e.g., Shapley value) for each judgment item obtained from multiple trained judgment models with different structures (e.g., in the second trained judgment model, a trained judgment model using LightGBM and a trained judgment model using CatBoost). The processor 111 then weights the contribution for each trained judgment model based on the weighting coefficients used when ensembling the judgment results of each trained model to calculate a weighted average. The processor 111 regards the weighted average contribution of each judgment item as contribution information. Naturally, contribution information may be obtained by other methods as well, not limited to this method.

[0138] Next, the processor 111 executes an analysis process of the area of ​​interest based on the read object image to obtain area of ​​interest information, which is one type of influence information (S314). Here, the area of ​​interest information is information indicating an area in the object image that has a large influence on a judgment result when the object image is input into a trained judgment model (e.g., the first trained judgment model) to generate a judgment result. Such area of ​​interest information is calculated, for example, by Gradient-weighted Class Activation Mapping (Grad-CAM), SHAP Deep Explainer, Local Interpretable Model-agnostic Explanations (LIME), or a combination thereof.

[0139] Note that, when the judgment results of multiple trained judgment models (first trained judgment model and second trained judgment model) are ensembled, as in the judgment process described in FIG. 6 , the attention area information can be further calculated as follows. For example, when multiple trained judgment models with the same structure obtained by training using training information from different subject groups (e.g., multiple first trained judgment models obtained by cross-validation) are used as the trained judgment models, the following method can be adopted. For each trained model, the processor 111 generates an attention level (e.g., an attention level image output by the Grad-CAM exemplified above) for the subject image input to each trained model. Then, the processor 111 averages the attention level images generated for each trained model and uses the averaged attention level image as attention level information. Note that, of course, attention level information can be obtained by other methods as well, not limited to this method.

[0140] Furthermore, for example, when trained judgment models with different structures are used as the trained models (specifically, when the input judgment images have different sizes, or when the sizes of the output attention images are different due to different structures of the trained judgment models), the following method can be adopted. (A) The processor 111 generates an image indicating the attention level for each trained model (for example, an attention image output by the Grad-CAM exemplified above) for each subject image input to each trained model. Since the attention images obtained here are of different sizes, the processor 111 enlarges them to the same size. Then, the processor 111 weights each of the attention images enlarged or reduced to the same size based on the weighting coefficient used when ensembling the judgment results of each trained model, and calculates a weighted average. The processor 111 regards the weighted averaged attention image as attention information.

[0141] Next, the processor 111 executes a segmentation process for identifying regions included in the read-out subject image based on the read-out subject image (S315). This segmentation process uses a trained segmentation image model obtained by machine learning, which is provided to a learner with a set of training subject images and position information of the labels obtained by labeling each region (e.g., position information of the posterior pharyngeal wall, uvula, etc.) based on operation input by a doctor or other personnel for the training subject image. The processor 111 inputs the subject image to the trained segmentation model and obtains position information of each segmented region (e.g., the posterior pharyngeal wall, uvula, etc.).

[0142] Next, the processor 111 stores the contribution information acquired in S313 in the contribution management table, and transmits it as impact information to the terminal device 300 that sent the impact generation request via the communication interface 113 (S316). Also, the processor 111 stores the attention area information acquired in S314 in the attention area management table, and transmits it as impact information to the terminal device 300 that sent the impact generation request via the communication interface 113 (S316). At this time, the attention area information can also include segmentation information acquired by the segmentation process of S315.

[0143] Here, as in S316 and S317, the contribution information and the attention area information may be transmitted at different times. Specifically, the attention area analysis process of S314, which involves analyzing the subject image, imposes a relatively greater processing load than the contribution analysis process of S313. Therefore, it may take time to acquire the attention area information. Therefore, the processor 111 transmits the contribution information to the terminal device 300 in advance as in S316 when the contribution information is acquired, and then transmits the attention area information to the terminal device 300 as in S317 when the attention area information is acquired. By processing in this manner, at least a portion of the influence information can be output before the terminal device 300 receives the attention area information, allowing the operator to more quickly confirm the analysis results. This concludes the processing flow for generating influence information.

[0144] As shown in the processing flow, by generating influence information indicating the degree of influence of at least a portion of the subject image, medical interview information, and finding information input to the trained judgment model on the judgment result for one or more diseases, it is possible to increase the reliability of the judgment result. In particular, by using contribution information as influence information, it is possible to grasp the contribution of each item selected as a judgment item and accurately understand the basis for the judgment made by the trained judgment model. Furthermore, by using attention area information as influence information, it is possible to grasp the area focused on when the trained judgment model obtains the judgment result, which, in addition to the results of their own visual examination, allows, for example, medical professionals and the like to feel more reassured.

[0145] As described above, in this embodiment, it is possible to increase the reliability of the judgment results by generating influence information that indicates the degree of influence that at least a portion of the subject image, medical interview information, and finding information input into the trained judgment model has on the judgment results for one or more diseases.

[0146] 6. Modifications As described above, one embodiment according to the present disclosure has been described with reference to FIGS. 1 to 11. However, various modifications can be applied without being limited to those described above. Note that, although each modification will be described below, it is also possible to use each modification in appropriate combination. Furthermore, although details of the modification will be described below, the remaining parts can be implemented in the same manner as the embodiment described with reference to FIGS. 1 to 11.

[0147] (1) Influence Information Screen In S316 and S317 of Fig. 7, the contribution information and the attention area information are transmitted, respectively, and based on the information, the influence information screens shown in Figs. 9A to 11 are output by the terminal device 300. However, in addition to this, various other information may be transmitted and output on the influence information screen.

[0148] For example, by using the contribution information, it is possible to identify information on the judgment items that had a significant impact on the judgment result. Therefore, the processor 111 may generate recommended information on prescription drugs or countermeasures based on the contribution information and output the recommended information on the impact information screen. The processor 111 extracts judgment items from the generated contribution information according to a predetermined criterion (e.g., the top three or judgment items with a contribution level equal to or greater than a predetermined threshold), and generates a prompt from the judgment items and a pre-prepared template. The processor 111 then provides the generated prompt to a large-scale language model and obtains the recommended information as output. Instead of using a large-scale language model, a recommendation management table indicating the correspondence between judgment items extracted according to a predetermined criterion and the recommended information generated accordingly may be stored and used to generate the recommended information.

[0149] The processor 111 may also generate explanatory text and diagrams explaining the basis of the determination result based on the contribution information and attention area information, and transcribe the generated explanatory text and diagrams into the subject's electronic medical record information stored in the electronic medical record device, for example. For such explanatory text and diagrams, the processor 111 generates a prompt from the generated contribution information and attention area information and a pre-prepared template. The processor 111 then provides the generated prompt to a large-scale language model, and obtains the explanatory text and diagrams as output. Furthermore, the processor 111 may also generate an explanatory text of the treatment plan based on prescription information entered by the operator in addition to the contribution information and attention area information, and transcribe the explanation into the subject's electronic medical record information stored in the electronic medical record device, for example. For such explanatory text, the processor 111 generates a prompt from the generated contribution information, attention area information, prescription information, and a pre-prepared template. The processor 111 then provides the generated prompt to a large-scale language model, and obtains the explanatory text as output. The prescription information may be input by an operator as described above, may be obtained from an electronic medical record device, or may be generated by a large-scale language model based on the judgment results of a trained judgment model or the diagnosis results of a medical professional.

[0150] (2) Process for Generating Determination Results and Process for Generating Impact Information In the example of FIG. 4 , the process for generating determination results indicating the possibility of affliction with one or more diseases and the process for generating impact information were all executed by the same server device 100. Also, the case was described in which the determination request, which triggers the process for generating determination results, and the impact generation request, which triggers the process for generating impact information, are both transmitted from the terminal device 300. However, it is also conceivable that the process for generating determination results is transmitted from the terminal device of a medical professional, and the impact generation request is transmitted from the terminal device of a provider of the determination service. Therefore, the terminal device 300 in the process for generating determination results and the terminal device 300 in the process for generating impact information may be separate terminal devices. Similarly, the server device 100 in the process for generating determination results and the server device 100 in the process for generating impact information may be separate terminal devices.

[0151] 5 to 7, the processes performed by the server device 100 can be distributed to multiple server devices as appropriate depending on the processing load. For example, in FIG. 6, the screening process, preprocessing, and judgment processes using each trained judgment model can be performed by different server devices.

[0152] (3) Processing Device In the above embodiment, the server device 100 mainly functions as a processing device and executes the process of generating the judgment result and the process of generating the impact information. However, this is not limited to this, and the terminal device 300 may execute the process of generating the judgment result and the process of generating the impact information. Furthermore, the server device 100 may execute the process of generating the judgment result and the terminal device 300 may execute the process of generating the impact information, or the terminal device 300 may execute the process of generating the judgment result and the server device 100 may execute the process of generating the impact information.

[0153] (6) Subject Image In the above embodiment, the subject image was an image captured by the imaging device 200 that can be inserted into the oral cavity and capture images of the oral cavity as a subject. However, the subject image is not limited to this, and subject images captured by a CT device, MRI device, X-ray device, ultrasound diagnostic device, endoscopic device, angiography device, or a combination thereof may also be used. Furthermore, the subject image is merely one type of medical information used in the subject image determination process, and medical information measured by other devices such as a blood pressure monitor device or an electrocardiogram device may be used instead of or in combination with the subject image.

[0154] The processes and procedures described herein can be realized not only by those explicitly described in the embodiments, but also by software, hardware, or a combination thereof. Specifically, the processes and procedures described herein can be realized by implementing logic corresponding to the processes in a medium such as an integrated circuit, volatile memory, non-volatile memory, magnetic disk, or optical storage. Furthermore, the processes and procedures described herein can be implemented as computer programs and executed by various computers, including processing devices and server devices.

[0155] Although processes and procedures described herein are described as being performed by a single device, software, component, or module, such processes or procedures may be performed by multiple devices, multiple software, multiple components, and / or multiple modules. Furthermore, although various information described herein is described as being stored in a single memory or storage unit, such information may be stored in multiple memories within a single device or multiple memories distributed across multiple devices. Furthermore, software and hardware elements described herein may be realized by integrating them into fewer components or by decomposing them into more components.

[0156] 1 Processing system 100 Server device 200 Imaging device 300 Terminal device

Claims

1. A processing device having at least one processor, wherein the at least one processor is configured to execute a process of acquiring, via a communication interface, at least one of a subject image acquired by an imaging device configured to image a subject of interest, medical interview information about the subject, and finding information about the subject, and generating impact information indicating the degree of influence of at least one of the subject image, medical interview information, and finding information input to a trained judgment model on a judgment result indicating the possibility of contracting one or more diseases, the judgment result being generated by inputting at least one of the acquired subject image, medical interview information, and finding information into the trained judgment model.

2. The processing device described in claim 1, wherein the judgment result is generated by inputting at least two or more pieces of information, namely the subject image, the medical interview information, and the finding information, into the trained judgment model, the impact information is generated for each of the at least two or more pieces of input information, and each of the generated impact information is output in a manner that allows it to be compared with each other.

3. The processing device described in claim 1, wherein the influence information is information indicating the degree to which the input of at least one of the subject image, the medical interview information, and the finding information deviates from the average predicted value of the trained judgment model.

4. The processing device according to claim 1, wherein the influence information is a Shapley value or an approximation of the Shapley value.

5. The processing device described in claim 1, wherein the influence information is information indicating an area in the subject image that had a large influence on the judgment result when the judgment result was generated by inputting the subject image into the learned judgment model.

6. The processing device according to claim 4, wherein the influence information is information indicating the degree of influence exerted on the determination result in correspondence with the position information of the subject image.

7. The processing device according to claim 1, wherein the generated impact information is output to a terminal device connected via a communication interface.

8. The processing device described in claim 7, wherein the influence information includes both deviation degree information indicating the degree of deviation from the average predicted value of the trained judgment model when at least one of the subject image, the interview information, and the finding information is input, and area information indicating areas in the subject image that had a large influence on the judgment result when the judgment result was generated by inputting the subject image into the trained judgment model, and the deviation degree information is output to the terminal device before the area information.

9. A processing program that, when executed by at least one processor, causes the at least one processor to function as follows: acquire, via a communication interface, at least one of a subject image acquired by an imaging device configured to photograph a subject, the subject's medical interview information, and the subject's findings information; and generate impact information indicating the degree of influence of at least one of the subject image, the medical interview information, and the findings information inputted into the trained judgment model on a judgment result indicating the possibility of contracting one or more diseases, generated by inputting at least one of the acquired subject image, the medical interview information, and the findings information into the trained judgment model.

10. A processing method executed by at least one processor, comprising: a step of acquiring, via a communication interface, at least one of a subject image acquired by an imaging device configured to image a subject of a subject, medical interview information of the subject, and finding information of the subject; and a step of generating impact information indicating the degree of influence of at least one of the subject image, medical interview information, and finding information inputted into a trained judgment model on a judgment result indicating the possibility of contracting one or more diseases, the judgment result being generated by inputting at least one of the acquired subject image, medical interview information, and finding information into the trained judgment model.

11. A processing system including a terminal device and a processing device communicatively connected to the terminal device, wherein at least one processor of the terminal device selects at least one of a subject image acquired by an imaging device configured to capture an image of a subject, medical interview information of the subject, and finding information of the subject, and at least one processor of the processing device acquires at least one of the selected subject image, medical interview information, and finding information via a communication interface, and generates impact information indicating the degree of influence of at least one of the subject image, medical interview information, and finding information input to a trained judgment model on a judgment result indicating the possibility of contracting one or more diseases, the judgment result being generated by inputting at least one of the acquired subject image, medical interview information, and finding information into the trained judgment model.

Images