Information processing apparatus, information processing method, computer readable medium and computer program product

Abstract

The present invention provides an information processing device, an information processing method, a computer readable medium, and a computer program product capable of monitoring the health state of a subject during imaging for image diagnosis. An information processing device (10) is provided with a processor that acquires color information of at least a part of the body surface of a subject during imaging for image diagnosis, and monitors the health state of the subject during imaging using an estimation model for estimating a change in the color information when the health state of the subject changes.

Description

Technical Field

[0001] This invention relates to an information processing device, an information processing method, an information processing program, and a learning completion model. Background Technology

[0002] Previously, various technologies were known for monitoring users' health status.

[0003] For example, Patent Document 1 discloses a method for identifying a patient's facial expressions based on continuously recorded facial information, generating an index representing the time-series changes in facial expressions using the facial information and the patient's medical data, and displaying the index and facial expressions together with the time-series changes in the medical data. Furthermore, Patent Document 2 discloses a method for identifying a specified user from an image of an analysis object generated by photographing the front of a reflector, and analyzing the user's physical state based on the user's judgment criterion feature and the analysis time feature derived from the analysis object image.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2022-188611

[0005] Patent Document 2: Japanese Patent Application Publication No. 2018-183509

[0006] In recent years, with advancements in medical devices such as CT (Computed Tomography) and MRI (Magnetic Resonance Imaging), higher-quality, higher-resolution three-dimensional images have been utilized for diagnostic imaging. In diagnostic imaging using CT and MRI, contrast agents are sometimes used for more detailed diagnosis, but these agents can sometimes cause discomfort to the patient. Furthermore, even without contrast agents, sudden changes in the patient's condition or discomfort due to stress can occur. Therefore, there is a need for a technology that can monitor the health status of patients during diagnostic imaging. Summary of the Invention

[0007] This invention provides an information processing device, information processing method, information processing program, and learning completion model capable of monitoring the health status of subjects in imaging for diagnostic purposes.

[0008] The first aspect of the present invention is an information processing apparatus comprising a processor that performs the following processing: acquiring color information of at least a portion of the body surface of a subject in an image for diagnostic purposes, and monitoring the health status of the subject in the image using an estimation model for estimating changes in the color information when the health status of the subject changes.

[0009] The estimation model can be a learning model that has been pre-learned with color information that is a reference for the health state of the subject, that is, reference color information, as input and color information when the health state of the subject is abnormal, that is, abnormal color information, as output. The processor can perform processing of acquiring color information of the body surface of the subject before the start of photography as reference color information, generating abnormal color information by inputting the reference color information to the estimation model, and estimating that the health state of the subject is abnormal when a comparison result of the color information in the photography and the abnormal color information satisfies a condition set in advance.

[0010] The estimation model can output a plurality of abnormal color information when the health state of the subject becomes a different type of abnormal state, respectively. The processor can estimate that the health state of the subject is abnormal when a comparison result of the color information in the photography and at least one of the plurality of abnormal color information satisfies a condition set in advance.

[0011] The estimation model can be learned by machine learning using a combination of color information when the health state is normal and color information when the health state is abnormal as learning data.

[0012] The processor can issue a warning when it is estimated that the health state of the subject is abnormal.

[0013] The processor can extract color information from a visible light image obtained by visible light imaging of the subject.

[0014] The processor can extract color information in a region set in advance in the body surface of the subject from the visible light image.

[0015] The color information can be color information of at least one of the skin and the mucous membrane of the subject.

[0016] The color information can be color information of the face of the subject.

[0017] The photography can be photography accompanied by injection of a contrast medium.

[0018] The estimation model can generate a plurality of abnormal color information from color information when the health state of the subject is normal.

[0019] A second aspect of the present application is an information processing method in which a computer performs processing of acquiring color information of at least a part of the body surface of the subject in photography for image diagnosis and monitoring the health state of the subject in the photography using an estimation model for estimating a change in color information when the health state of the subject changes.

[0020] A third aspect of the present application is an information processing program for causing a computer to execute processing of acquiring color information of at least a part of a body surface of a subject in photography for image diagnosis, and monitoring a health state of the subject in the photography using an estimation model for estimating a change in color information when the health state of the subject changes.

[0021] A fourth aspect of the present application is a learning completion model for causing a computer to function as an input of reference color information that is a reference of a health state of a subject and an output of abnormal color information that is color information of the subject when the health state is abnormal, and the color information is color information of at least a part of a body surface of the subject in photography for image diagnosis, wherein

[0022] The learning completion model is pre-learned by machine learning using a combination of color information when the health state is normal and color information when the health state is abnormal as learning data.

[0023] Effects of Invention

[0024] According to the above-described aspects, the information processing apparatus, the information processing method, the information processing program, and the learning completion model of the present application can monitor a health state of a subject in photography for image diagnosis. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a schematic perspective view showing a CT apparatus.

[0026] Figure 2 is a block diagram showing an example of a hardware structure of an information processing apparatus.

[0027] Figure 3 is a block diagram showing an example of a functional structure of an information processing apparatus.

[0028] Figure 4 is a diagram for explaining processing of an information processing apparatus.

[0029] Figure 5 is a diagram for explaining processing of an information processing apparatus.

[0030] Figure 6 is a diagram showing an example of a screen displayed on a display.

[0031] Figure 7 is a flowchart showing an example of information processing.

[0032] Explanation of Symbols

[0033] 1-CT device, 2-rack, 3-bed, 3A-bed section, 3B-base section, 3C-driving section, 4-control console, 5-opening section, 7-camera, 10-information processing device, 21-CPU, 22-storage section, 23-memory, 24-display, 25-input section, 26-network I / F, 27-information processing program, 28-bus, 30-acquisition section, 32-estimation section, 34-control section, 40-estimation model, 50, 50C-visible light image, 52-face region, 54-cheek region, 56-lip region, 60A, 60B-sample, 90-warning text, D1-screen, H-subject. DETAILED DESCRIPTION

[0034] Hereinafter, an example of an embodiment of the disclosed technology will be described with reference to the drawings. In addition, in each drawing, the same reference signs are attached to the same or equivalent constituent elements and portions, and repetitive explanation will be omitted. Also, the dimensional ratios of the drawings are exaggerated for the sake of explanation, and there are cases where the actual ratios are different.

[0035] Reference Figure 1 The structure of a CT (Computed Tomography) device 1 will be described. As shown in FIG. 1, the CT device 1 according to the present embodiment is provided with a rack 2, a bed 3, and a control console 4. The rack 2, the bed 3, and the control console 4 are configured to be connectable via a wired or wireless network (not shown). In addition, the control console 4 can be disposed in a remote state (for example, in another room or the like) from the rack 2 and the bed 3. Figure 1

[0036] The rack 2 has a tunnel-like structure with an opening section 5 in the center thereof. Inside the rack 2, a ray source section that radiates X-rays and a detection section that detects X-rays and generates a radiograph are provided (both not shown). The ray source section and the detection section are each rotatable along the annular shape of the rack 2 in a state in which the positional relationship of the ray source section and the detection section is maintained. Also, inside the rack 2, a control section that controls the operation of the CT device 1 is provided.

[0037] Also, the rack 2 has a camera 7 for visible light photographing of a subject H in photography for image diagnosis. The camera 7 is a camera that can photograph a color image of R (red), G (green), and B (blue) by detecting reflected light of the subject H. The camera 7 has a lens and a CCD (Charge Coupled Device) or the like, acquires a visible light image as a moving image by photographing the subject H on the bed 3 at a predetermined frame rate, and outputs to the control console 4.

[0038] ​Specifically, the camera 7 can be provided near the radiation source section so as to be able to take an image of the face of the subject H carried into the opening section 5 of the gantry 2 (refer to Figure 1 ). Also, the camera 7 can be provided at the ceiling or wall surface of the photographic room, or the frame of the gantry 2, etc. so as to be able to take an image of the face of the subject H in a state in which the face is exposed from the opening section 5. Also, the camera 7 can be provided at each of a plurality of positions as described above.

[0039] The subject H is placed on the table 3. The table 3 is provided with a table section 3A on which the subject H lies, a base section 3B that supports the table section 3A, and a drive section 3C that reciprocally moves the table section 3A in the arrow A direction. The table section 3A is slidable with respect to the base section 3B in the arrow A direction by the drive section 3C. When imaging of a CT image is performed, the subject H lying on the table section 3A is carried into the opening section 5 of the gantry 2 by the table section 3A being slid.

[0040] In the CT apparatus 1, the drive of the gantry 2 and the drive of the table 3 are performed by input by an operator such as a technician in the console 4, whereby imaging of a CT image is performed. The console 4 includes the information processing apparatus of the present application. The information processing apparatus related to the present embodiment has a function of monitoring the health state of the subject H in imaging for image diagnosis. Hereinafter, one example of the structure of the information processing apparatus 10 related to the present embodiment will be described.

[0041] First, reference will be made to Figure 2 One example of the hardware structure of the information processing apparatus 10 related to the present embodiment will be described. The information processing apparatus 10 includes a CPU (Central Processing Unit) 21, a nonvolatile storage section 22, and a memory 23 as a temporary storage area. Also, the information processing apparatus 10 includes a display 24, an input section 25, and a network I / F (Interface) 26. The CPU 21, the storage section 22, the memory 23, the display 24, the input section 25, and the network I / F 26 are connected via a bus 28 such as a system bus and a control bus in a manner that enables them to mutually transmit and receive various information.

[0042] The storage section 22 is realized by, for example, a storage medium such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), and a flash memory. The information processing program 27 in the information processing apparatus 10 is stored in the storage section 22. The CPU 21 reads out the information processing program 27 from the storage section 22, expands it to the memory 23, and executes the expanded information processing program 27. The CPU 21 is one example of the processor of the present application.

[0043] The display 24 is, for example, a liquid crystal display that displays various information. The input section 25 includes a pointing device such as a mouse and a keyboard and the like, and is used for various inputs to the apparatus. In addition, the display 24 can be constituted by a touch panel and used as the input section 25. The display 24 and the input section 25 are also illustrated in FIG. 1. Figure 1 .

[0044] The network I / F 26 is an interface for communicating with various apparatuses including the CT apparatus 1 via a network. In this communication, a standard of wired communication such as Ethernet (registered trademark) or FDDI (Fiber Distributed Data Interface) or a standard of wireless communication such as 4G, 5G, or Wi-Fi (registered trademark) is used, for example. As the information processing apparatus 10, a server computer, a personal computer, a smartphone, a tablet terminal, a wearable terminal, or the like can be appropriately applied, for example.

[0045] Next, a configuration example of the information processing apparatus 10 according to the present embodiment will be described with reference to Figure 3 to FIG. 1. As shown in FIG. 1, the information processing apparatus 10 includes a CPU 21, a ROM 22, a RAM 23, a display 24, an input section 25, a storage 26, and a communication interface (I / F) 27. The information processing apparatus 10 is connected to the CT apparatus 1 via the communication I / F 27. Figure 3 The CPU 21 functions as an acquisition section 30, an estimation section 32, and a control section 34 by executing an information processing program 27. The ROM 22 stores the information processing program 27 and the like. The RAM 23 temporarily stores data and the like. The display 24 is, for example, a liquid crystal display that displays various information. The input section 25 includes a pointing device such as a mouse and a keyboard and the like, and is used for various inputs to the apparatus. In addition, the display 24 can be constituted by a touch panel and used as the input section 25. The display 24 and the input section 25 are also illustrated in FIG. 1.

[0046] The acquisition section 30 acquires color information of at least a part of a body surface of the subject H in photography for image diagnosis. The photography for image diagnosis refers to CT photography based on the CT apparatus 1, and not visible light photography based on the camera 7, which is hereinafter referred to as "CT photography". In addition, the photography for image diagnosis can be photography with injection of a contrast medium. The body surface refers to, for example, the skin of the whole body including the face, and the mucosa of the lips and the eyes and the like. In addition, the nails are also included in the skin. As the color information, a combination of R components, G components, and B components can be applied, a combination of hue, lightness, and chroma can be applied, or both of them can be applied, for example.

[0047] Specifically, the acquisition section 30 can extract color information from a visible light image obtained by visible light photography of the subject H by the camera 7. Also, the acquisition section 30 can extract color information in a region set in advance in the body surface of the subject H from the visible light image. This is because, even if the health status changes, the color of the body surface does not change uniformly, and there are many cases where only a part of the region changes locally or in a spot-like manner.

[0048] As the region of the body surface that is set in advance, a region in which the color is likely to change depending on the health state of the subject H is preferably used. For example, a region including at least one of the skin of the subject H and the mucous membranes of the lips and eyes is preferable. That is, the color information is preferably color information of at least one of the skin and the mucous membranes of the subject H. Also, among the entire body, the face is particularly preferable. That is, the color information is preferably color information of the face of the subject.

[0049] Also, the acquisition unit 30 repeatedly acquires the color information of at least a part of the body surface of the subject H over time from before the CT imaging is started to when the CT imaging is performed. Here, the color information that is to be a reference for the health state of the subject H is referred to as reference color information. For example, the acquisition unit 30 can acquire the color information of the body surface of the subject H before the CT imaging is started as the reference color information. Also, for example, the acquisition unit 30 can acquire the color information of the body surface of the subject H that is acquired immediately after the CT imaging is started (i.e., immediately after the start) as the reference color information.

[0050] The estimation unit 32 monitors the health state of the subject H during the CT imaging using the estimation model 40 for estimating a change in the color information when the health state of the subject H changes. Specifically, the estimation unit 32 generates color information when the health state of the subject H is abnormal (hereinafter referred to as abnormal color information) by inputting the reference color information into the estimation model 40.

[0051] The estimation model 40 is a learning-completed model for causing a computer to function as an estimation model that inputs the reference color information and outputs the abnormal color information. As the estimation model 40, for example, a machine learning model such as a convolutional neural network and a recurrent neural network can be used.

[0052] The learning data of the estimation model 40 is a combination of color information when the health state is normal and color information when the health state is abnormal. Since there are individual differences in the color of the body surface, it is preferable to perform learning using a combination of color information of the normal state and color information of the abnormal state of the body surface of various people. Through machine learning using such learning data, the estimation model 40 is learned in such a manner that it can output abnormal color information that is appropriate for the input reference color information regardless of the color of the body surface of the subject H.

[0053] Furthermore, the estimation model 40 can output multiple abnormal color information for different types of abnormal states in the health status of the subject H. For example, suppose that in CT imaging, the patient experiences discomfort due to the contrast agent and cyanosis due to hypoxia. In this case, the estimation model 40 can output abnormal color information for the patient experiencing discomfort due to the contrast agent and abnormal color information for the patient experiencing cyanosis, respectively. Moreover, multiple estimation models 40 can be generated according to the types of these abnormal states.

[0054] Next, whenever color information is acquired in a CT scan, the estimation unit 32 compares the color information in the CT scan with abnormal color information. Then, if the comparison result meets a preset condition, the health status of the subject H is estimated to be abnormal. The preset condition, for example, refers to a situation where the difference between the color information in the CT scan and the abnormal color information is below a preset threshold. The difference between the color information is calculated, for example, using Euclidean distance.

[0055] Furthermore, when multiple abnormal color information are generated according to the type of each abnormal state, the estimation unit 32 can estimate that the health status of the examinee H is abnormal if the comparison result between the color information in the CT scan and at least one of the multiple abnormal color information meets a preset condition. For example, if only one of the abnormal color information indicating physical discomfort due to the effect of contrast agent and the abnormal color information indicating cyanosis meets a preset condition, the health status of the examinee H can be estimated as abnormal.

[0056] refer to Figure 4 and Figure 5 An example of the processing of the acquisition unit 30 and the estimation unit 32 will be explained. Figure 4 This refers to the visible light image 50 acquired before the start of CT imaging. Figure 5 This represents a visible light image 50C acquired during a CT scan when the patient's health condition becomes abnormal. Here, the CT scan is performed with the injection of a contrast agent, allowing for estimation of whether the patient is experiencing discomfort due to the contrast agent based on the degree of flushing and pallor of the cheeks. Furthermore, it allows for estimation of whether the patient is cyanotic due to hypoxia based on the color of their lips.

[0057] Acquisition Department 30 Acquisition in Figure 4 The image shown is a visible light image 50 acquired before the start of a CT scan. The acquisition unit 30 then detects a facial region 52 from the visible light image 50. Next, the acquisition unit 30 detects a cheek region 54 and a lip region 56 from the facial region 52. Furthermore, known image recognition techniques can be appropriately applied as a method for detecting each region.

[0058] Next, the acquisition unit 30 extracts color information such as R, G, and B components for the detected cheek region 54 and lip region 56, respectively. This color information extracted from the visible light image 50 acquired before the start of CT imaging becomes the reference color information for each of the cheek region 54 and lip region 56.

[0059] The estimation unit 32 generates abnormal color information for each of the cheek region 54 and lip region 56 by inputting the reference color information for each into the estimation model 40. As an example, in Figure 4 and Figure 5 The diagram shows samples 60A and 60B, which contain abnormal color information. Sample 60A represents abnormal color information in the cheek region 54, and is estimated to be in an abnormal state when the color information changes towards the flushed or pale side. Sample 60B represents abnormal color information in the lip region 56, and is estimated to be in an abnormal state when the color information changes towards the cyanotic side.

[0060] If CT imaging begins, the acquisition unit 30 acquires visible light images over time. Similarly to the visible light image 50, the acquisition unit 30 extracts color information for the cheek region 54 and the lip region 56 respectively. Furthermore, the acquisition unit 30 can extract color information each time a visible light image is acquired (i.e., corresponding to the frame rate of the camera 7), or it can extract color information at preset time intervals, such as every 1 second or every 30 frames.

[0061] The estimation unit 32 compares the color information in the CT scan with abnormal color information whenever color information is acquired, monitoring whether the health status of the subject H is abnormal. Assume that in the CT scan, color information is acquired... Figure 5 The visible light image 50C and its color information are shown. In this case, the difference between the current color information and the abnormal color information in the cheek region 54 is below a preset threshold, so the estimation unit 32 estimates it as an abnormal state.

[0062] Figure 6 The image shown is an example of screen D1 displayed on display 24 via control unit 34. For example... Figure 6 As shown, the control unit 34 can be controlled to display the visible light image 50 with acquired reference color information and the visible light image 50C at the current time. Furthermore, the control unit 34 can be controlled to display samples 60A and 60B of abnormal color information for user reference.

[0063] Furthermore, the preferred control unit 34 issues a warning when it estimates that the health status of the subject H is abnormal. Figure 6In the example of Fig. 9, a warning text 90 indicating that the health state of the subject H is abnormal is displayed. The warning can be, for example, a simple content that conveys only that it is an abnormal state without including a reason and detailed state, or a specific content such as a flushing state, a pale state, and a cyanotic state. Also, the method of the warning is not limited to text, and, for example, sound and light, etc. can be used.

[0064] In addition, when the reference color information as an input to the estimation model 40 greatly differs for each subject H, it is preferable to prepare a plurality of estimation models 40 corresponding to the reference color information. That is, it is preferable to generate a plurality of estimation models 40 from color information when the health state of the subject H is normal. For example, if learning is performed by mixing learning data on different races such as Caucasians, black people, and Mongolians, there is a possibility that the learning is difficult to perform or the accuracy of the estimation decreases because of a large bias. In this case, by dividing the learning data by race to generate the estimation model 40, these problems can be avoided.

[0065] Next, the information processing performed by the information processing apparatus 10 will be described with reference to the flowchart of Fig. 10. Figure 7 The operation of the information processing apparatus 10 will be described. In the information processing apparatus 10, the information processing illustrated in Fig. 11 is performed by the CPU 21 executing the information processing program 27. Figure 7 This processing is performed, for example, when an instruction to start the execution is given by the user.

[0066] In step S10, the acquisition unit 30 acquires a visible light image obtained by the camera 7 performing visible light imaging of the subject H before the start of the CT imaging. In step S12, the acquisition unit 30 extracts color information from the visible light image acquired in step S10 as reference color information. In step S14, the estimation unit 32 generates abnormal color information when the health state of the subject H is abnormal by inputting the reference color information extracted in step S12 to the estimation model 40.

[0067] In step S16, the user starts the CT imaging. In the case of using a contrast medium, the contrast medium is injected into the subject H. In step S18, the acquisition unit 30 acquires a visible light image obtained by the camera 7 performing visible light imaging of the subject H in the CT imaging. In step S20, the acquisition unit 30 extracts color information from the visible light image acquired in step S18.

[0068] In step S22, the estimation unit 32 compares the color information extracted from the CT scan in step S20 with the abnormal color information generated in step S14, and determines whether the comparison result meets a preset condition. If the comparison result meets the preset condition in step S22, the health status of the examinee H is estimated to be abnormal, and the process proceeds to step S24. In step S24, the control unit 34 issues a warning that the health status of the examinee H is abnormal, and the process proceeds to step S26.

[0069] On the other hand, if step S22 is rejected (i.e., the comparison result does not meet the preset conditions), the health status of the examinee H is estimated to be normal, and the process proceeds to step S26. In step S26, the control unit 34 determines whether the CT scan has ended. If step S26 is rejected (i.e., the CT scan has not ended), steps S18 to S26 are repeated. On the other hand, if step S26 is affirmed (i.e., the CT scan has ended), this information processing ends.

[0070] As explained above, the information processing apparatus 10 according to this embodiment includes a processor. The processor acquires color information of at least a portion of the body surface of a subject H in an image diagnostic photograph, and uses an estimation model 40 for estimating changes in color information when the health status of the subject H changes to monitor the health status of the subject H in the photograph. Therefore, according to the information processing apparatus 10 according to this embodiment, the health status of a subject H in an image diagnostic photograph can be monitored.

[0071] Furthermore, while the method for extracting color information from a visible light image has been described in the above embodiments, it is not limited thereto. Color information of at least a portion of the subject H's body surface can be measured, for example, using a color sensor employing a photoelectric sensor.

[0072] Furthermore, at least one of the functional units of the information processing device 10 in the above embodiments may be provided by other devices such as the control device provided by the frame 2 and the control device provided by the drive unit 3C.

[0073] Furthermore, while the above embodiment was described using a CT device 1, the technology of the present invention can also be applied to various medical imaging devices other than the CT device 1 that perform imaging for diagnostic purposes. Examples of various medical imaging devices include conventional X-ray imaging devices, MRI (Magnetic Resonance Imaging) devices, PET (Positron Emission Tomography) devices, mammography devices, ultrasound diagnostic devices, endoscopes, and fundus cameras.

[0074] Furthermore, in the above embodiments, for example, the hardware structure of the processing unit that performs various processes such as the acquisition unit 30, the estimation unit 32, and the control unit 34 can use various processors as shown below. These various processors include general-purpose processors (CPUs) that execute software (programs) and function as various processing units, as described above, as well as processors such as FPGAs (Field Programmable Gate Arrays) whose circuit structure can be changed after manufacturing (i.e., Programmable Logic Devices (PLDs)) and processors such as ASICs (Application Specific Integrated Circuits) with circuit structures specifically designed for performing specific processes (i.e., dedicated circuits).

[0075] A processing unit can consist of one of these various processors, or it can consist of a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs or a combination of a CPU and an FPGA). Furthermore, multiple processing units can also be composed of a single processor.

[0076] As an example of a single processor comprising multiple processing units, the first type comprises a single processor consisting of one or more CPUs and software, as exemplified by computers such as client machines and servers, and this processor functions as multiple processing units. The second type comprises a processor, as exemplified by a System-on-Chip (SoC), which uses a single Integrated Circuit (IC) chip to implement the overall system functionality including multiple processing units. Thus, various processing units are configured using one or more of the aforementioned processors as hardware structures.

[0077] Moreover, the hardware architecture of these various processors, more specifically, can utilize circuits composed of circuit elements such as semiconductor elements.

[0078] Furthermore, while the above embodiment describes the information processing program 27 being pre-stored (installed) in the storage unit 22, it is not a limitation. The information processing program 27 may also be provided as a recording medium such as a CD-ROM (Compact Disc Read Only Memory), DVD-ROM (Digital Versatile Disc Read Only Memory), or USB (Universal Serial Bus) memory. Additionally, the information processing program 27 may be configured to be downloaded from an external device via a network.

[0079] Furthermore, the present invention is also applicable to programs and program articles. Specifically, the information processing program 27 described in the above embodiments can be provided as a program article. A program article includes all forms of articles for providing programs. For example, a program article includes programs provided via networks such as the Internet, and computer-readable recording media for non-transitory storage of programs.

[0080] The technology of the present invention can also be appropriately combined with the above-described embodiments and modifications. The descriptions and illustrations above are detailed explanations of the parts involved in the technology of the present invention, and are merely one example of the technology of the present invention. For example, the descriptions related to the above-described structure, function, effect, and effect are descriptions related to one example of the structure, function, effect, and effect of the parts involved in the technology of the present invention. Therefore, without departing from the spirit of the present invention, unnecessary parts may be deleted or new elements may be added or replaced in the descriptions and illustrations above.

[0081] The following notes further disclose the above-described embodiments.

[0082] [Postscript 1]

[0083] An information processing device comprising a processor,

[0084] The processor performs the following processing:

[0085] Obtain color information of at least a portion of the subject's body surface from photographs used for image diagnosis.

[0086] The health status of the subject in the photograph is monitored using an estimation model that estimates changes in color information as the subject's health status changes.

[0087] [Postscript 2]

[0088] According to the information processing apparatus described in Appendix 1, wherein,

[0089] The estimation model is a pre-learned model that takes the reference color information (which serves as the benchmark for the health status of the examinee) as input and the abnormal color information (which indicates that the examinee's health status is abnormal) as output.

[0090] The processor performs the following processing:

[0091] The color information of the subject's body surface before the start of the photography is obtained as the reference color information.

[0092] The anomalous color information is generated by inputting the baseline color information into the estimation model.

[0093] If the comparison result between the color information and the abnormal color information in the photograph meets the preset conditions, the health status of the subject is estimated to be abnormal.

[0094] [Postscript 3]

[0095] According to the information processing apparatus described in Appendix 2, wherein,

[0096] The estimation model outputs multiple abnormal color information when the health status of the examinee becomes different types of abnormal states.

[0097] If the comparison result between the color information in the photograph and at least one of the plurality of abnormal color information satisfies a preset condition, the processor estimates that the health status of the subject is abnormal.

[0098] [Postscript 4]

[0099] According to the information processing apparatus described in Appendix 2 or 3, wherein,

[0100] The estimation model learns by using a combination of color information when the health status is normal and color information when the health status is abnormal as learning data through machine learning.

[0101] [Postscript 5]

[0102] The information processing apparatus according to any one of Appendices 1 to 4, wherein,

[0103] The processor issues a warning if it estimates that the subject's health condition is abnormal.

[0104] [Postscript 6]

[0105] The information processing apparatus according to any one of Appendices 1 to 5, wherein,

[0106] The processor extracts the color information from a visible light image obtained by taking a visible light picture of the subject.

[0107] [Postscript 7]

[0108] According to the information processing apparatus described in Appendix 6, wherein...

[0109] The processor extracts the color information from a pre-defined region on the subject's body surface from the visible light image.

[0110] [Postscript 8]

[0111] The information processing apparatus according to any one of Appendices 1 to 7, wherein,

[0112] The color information refers to the color information of at least one of the subject's skin and mucous membranes.

[0113] [Postscript 9]

[0114] The information processing apparatus according to any one of Appendices 1 to 8, wherein,

[0115] The color information refers to the color information of the subject's face.

[0116] [Postscript 10]

[0117] The information processing apparatus according to any one of Appendices 1 to 9, wherein,

[0118] The photography was conducted with the injection of contrast agent.

[0119] [Postscript 11]

[0120] The information processing apparatus according to any one of Appendices 1 to 10, wherein,

[0121] The estimation model generates multiple models based on the color information when the subject's health status is normal.

[0122] [Postscript 12]

[0123] An information processing method, wherein a computer performs the following processing:

[0124] Obtain color information of at least a portion of the subject's body surface from photographs used for image diagnosis.

[0125] The health status of the subject in the photograph is monitored using an estimation model that estimates changes in color information as the subject's health status changes.

[0126] [Postscript 13]

[0127] An information processing program that causes a computer to perform the following processing:

[0128] Obtain color information of at least a portion of the subject's body surface from photographs used for image diagnosis.

[0129] The health status of the subject in the photograph is monitored using an estimation model that estimates changes in color information as the subject's health status changes.

[0130] [Postscript 14]

[0131] A learning completion model enables a computer to perform the following functions: taking reference color information, which serves as a benchmark of a subject's health status, as input, and taking abnormal color information, i.e., color information when the subject's health status is abnormal, as output, wherein the color information is the color information of at least a portion of the subject's body surface during imaging for image diagnosis, wherein...

[0132] The learning completion model was pre-learned by using a combination of color information when the health status is normal and color information when the health status is abnormal as learning data for machine learning.

Claims

1. An information processing apparatus comprising a processor, the processor performs processing of: acquiring color information of at least a part of a body surface of a subject in photography for image diagnosis, monitoring a health state of the subject in the photography using an estimation model for estimating a change in the color information when the health state of the subject changes. 2.The information processing apparatus according to claim 1, wherein the estimation model is a learned model that is pre-learned in a manner that the color information that is a reference of the health state of the subject, that is, reference color information is input and the color information that is abnormal when the health state of the subject is abnormal, that is, abnormal color information is output, the processor performs processing of: acquiring color information of the body surface of the subject before the photography starts as the reference color information, generating the abnormal color information by inputting the reference color information to the estimation model, estimating that the health state of the subject is abnormal in a case where a comparison result of the color information in the photography and the abnormal color information satisfies a condition that is set in advance. 3.The information processing apparatus according to claim 2, wherein the estimation model outputs a plurality of the abnormal color information when the health state of the subject becomes different types of abnormal states, respectively, the processor estimates that the health state of the subject is abnormal in a case where a comparison result of the color information in the photography and at least one of the plurality of the abnormal color information satisfies a condition that is set in advance. 4.The information processing apparatus according to claim 2 or 3, wherein the estimation model is learned by machine learning that uses a combination of the color information when the health state is normal and the color information when the health state is abnormal as learning data. 5.The information processing apparatus according to claim 1 or 2, wherein the processor issues a warning in a case where it is estimated that the health state of the subject is abnormal. 6.The information processing apparatus according to claim 1 or 2, wherein the processor extracts the color information from a visible light image obtained by taking a visible light photograph of the subject. 7.The information processing apparatus according to claim 6, wherein the processor extracts the color information in a region that is set in advance in the body surface of the subject from the visible light image. 8.The information processing apparatus according to claim 1 or 2, wherein the color information is color information of at least one of skin and mucous membrane of the subject. 9.The information processing apparatus according to claim 1 or 2, wherein the color information is color information of a face of the subject. 10.The information processing apparatus according to claim 1 or 2, wherein the photography is photography that accompanies injection of a contrast medium. 11.The information processing apparatus according to claim 1 or 2, wherein the estimation model is generated to have a plurality according to the color information when the health state of the subject is normal.

12. An information processing method, wherein, a computer performs processing of: acquiring color information of at least a part of a body surface of a subject in photography for image diagnosis, monitoring a health state of the subject in the photography using an estimation model for estimating a change in the color information when the health state of the subject changes.

13. A computer-readable medium recording an information processing program which causes a computer to execute processing of: acquiring color information of at least a part of a body surface of a subject in photography for image diagnosis, monitoring a health state of the subject in the photography using an estimation model for estimating a change in the color information when the health state of the subject changes.

14. A computer-readable medium recording a learning completion model for causing a computer to function as an input of reference color information which is a reference of a health state of a subject and an output of color information when the health state of the subject is abnormal, the color information being color information of at least a part of a body surface of the subject in photography for image diagnosis, wherein the learning completion model is pre-learned by machine learning using a combination of the color information when the health state is normal and the color information when the health state is abnormal as learning data.

15. A computer program product containing an information processing program which causes a computer to execute processing of: acquiring color information of at least a part of a body surface of a subject in photography for image diagnosis, monitoring a health state of the subject in the photography using an estimation model for estimating a change in the color information when the health state of the subject changes.

16. A computer program product containing a learning completion model for causing a computer to function as an input of reference color information which is a reference of a health state of a subject and an output of color information when the health state of the subject is abnormal, the color information being color information of at least a part of a body surface of the subject in photography for image diagnosis, wherein the learning completion model is pre-learned by machine learning using a combination of the color information when the health state is normal and the color information when the health state is abnormal as learning data.

Images