Information processing device, control method, and program

Abstract

To make it possible to display an image and a user interface without delay or disturbance and perform smooth communication, in telemedicine involving photographing.SOLUTION: A physician-side information processing apparatus communicably connected to a patient-side information processing apparatus includes: receiving means for receiving, from the patient-side information processing apparatus, a second image in which a patient has been captured in response to an instruction from the physician-side information processing apparatus and whose data amount has been reduced compared to a first image in which the patient has been captured; processing means for performing processing for cutting out, from the second image, a partial image; and transmitting means for transmitting, to the patient-side information processing apparatus, information that has been set by performing the processing on the second image by the processing means. The receiving means receives, from the patient-side information processing apparatus, a processed image that is the first image on which the processing has been performed based on the information in the patient-side information processing apparatus.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a system for supporting remote medical treatment.

Background Art

[0002] In a conventional remote medical treatment system, while the doctor side and the patient side view each other's images and talk, the doctor conducts a medical examination while considering the patient's expression, complexion, and content of questions. Further, in a conventional remote medical treatment system, a UI (user interface) mainly for one-on-one conversation between a doctor and a patient is provided, but a UI suitable for medical examinations other than one-on-one conversation is not provided. Patent Document 1 describes that when performing remote medical treatment, in addition to the doctor's image and the patient's image, an image of a specialist is displayed on the UI.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in Patent Document 1, in remote medical treatment involving photography, a UI in accordance with the medical treatment process, such as photographing a patient (affected part) from the start to the end of the medical treatment and explaining the photographed image, is not provided. Further, there is a possibility that smooth communication cannot be achieved because the doctor is distracted by UI operations, or the display of images and UI is delayed or disturbed due to insufficient capacity during communication.

[0005] The present invention has been made in view of the above problems, and an object thereof is to realize a technology that can display images and UI without delay or disturbance in remote medical treatment involving photography and enable smooth communication.

Means for Solving the Problems

[0006] To solve the above problems and achieve the objective, the present invention provides a physician-side information processing device that is communicatively connected to a patient-side information processing device, comprising: a receiving means for receiving from the patient-side information processing device a second image, which is an image of the patient taken in accordance with the instructions of the physician-side information processing device, and which has a reduced data volume compared to a first image of the patient; Display means for displaying a user interface including the second image, The system includes a processing means for extracting a portion of an image from the second image, and a transmitting means for transmitting information set by the processing means to the second image to the patient-side information processing device, wherein the receiving means receives the processed image from the patient-side information processing device, which has performed the processing on the first image based on the information. Furthermore, the physician's information processing device is capable of switching between multiple modes, the display means is a user interface including the second image, and displays a user interface corresponding to one of the multiple modes, the multiple modes include a first mode in which the patient and the physician converse, a second mode in which the patient is photographed, a third mode in which the patient's image is processed, and a fourth mode in which the processed image is used to explain to the patient, in the second mode the transmission means transmits an instruction to the patient's information processing device to photograph the patient, and in the third mode the transmission means transmits the information to the patient's information processing device. .

[0007] Furthermore, the present invention relates to a patient-side information processing device that is communicatively connected to a physician-side information processing device, A user interface including a first image generated by photographing a patient in accordance with the instructions of the physician's information processing device, comprising a display means for displaying a user interface corresponding to the mode of the physician's information processing device, before Record number The system comprises: a generation means for generating a second image with a reduced data size compared to the first image; a transmission means for transmitting the second image to the physician's information processing device; and a receiving means for receiving information set by the physician's information processing device after performing a process to extract a portion of the second image. The transmission means transmits the processed image, which has been processed on the first image based on the information, to the physician's information processing device. The modes of the physician's information processing device include a first mode in which the patient and the physician converse, a second mode in which the patient is photographed, a third mode in which the patient's image is processed, and a fourth mode in which the processed image is used to explain to the patient, wherein in the second mode, the transmitting means transmits the second image to the physician's information processing device, and in the third mode, the transmitting means transmits the processed image to the physician's information processing device. . [Effects of the Invention]

[0008] According to the present invention, images and UI can be displayed without delay or distortion in telemedicine involving imaging, enabling smooth communication. [Brief explanation of the drawing]

[0009] [Figure 1] System configuration diagram of this embodiment. [Figure 2] A block diagram showing the device configuration of this embodiment. [Figure 3]A diagram illustrating the flow of telemedicine in this embodiment and a UI that follows that flow. [Figure 4] A diagram illustrating the flowchart and UI in the trimming mode of this embodiment. [Figure 5] A diagram showing a modified example of the system configuration of this embodiment. [Figure 6] A diagram illustrating a flowchart and UI in a shooting mode in a modified system configuration of this embodiment. [Figure 7] A flowchart illustrating the operation of the physician-side information processing device and the patient-side information processing device in the telemedicine system of this embodiment. [Figure 8] A flowchart showing the save range specification process in S712 of Figure 7. [Modes for carrying out the invention]

[0010] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0011] [Embodiment 1] Hereinafter, an embodiment of the present invention will be described in which the present invention is applied to a system that supports telemedicine (online medical consultation) in which an information processing device on the physician's side and an information processing device on the patient's side are connected to communicate via a network.

[0012] <System Configuration> First, the system configuration of this embodiment will be described with reference to Figure 1.

[0013] The system of this embodiment (hereinafter, the telemedicine system) includes a doctor-side information processing device used by a doctor in a medical facility such as a hospital, and a patient-side information processing device used by a patient in a residential facility such as a home or a nursing facility. The two are communicably connected via a network such as the Internet, and communication can be carried out by voice or chat using the functions of applications operating on the information processing devices.

[0014] In addition, the telemedicine system of this embodiment includes a doctor-side imaging device that generates doctor-side images and a patient-side imaging device that captures patient-side images. The doctor-side information processing device and the patient-side information processing device can communicate more smoothly by transmitting and receiving doctor-side images and patient-side images via a network.

[0015] Note that the telemedicine system of this embodiment will describe an example of a configuration in which the doctor-side imaging device and the patient-side imaging device generate images, and the doctor-side information processing device and the patient-side information processing device distribute the images, but it is not limited to this. For example, the doctor-side information processing device and the patient-side information processing device may have the functions of imaging devices. The communication form may be either wired or wireless. Also, although an example of connecting a medical facility and a residential facility is described for the network, it may be a closed intranet within the medical facility.

[0016] As shown in FIG. 1, the telemedicine system 100 of this embodiment includes a doctor-side information processing device 101, a doctor-side imaging device 102, a patient-side information processing device 103, and a patient-side imaging device 104. The doctor-side information processing device 101 and the patient-side information processing device 103 are communicably connected via a network 105.

[0017] The physician-side information processing device 101 is, for example, a desktop or notebook PC (Personal Computer), or a tablet PC. The patient-side information processing device 103 is, for example, a desktop or notebook PC (Personal Computer), a tablet PC, a stick PC, or a smartphone. The physician-side and patient-side imaging devices 102 and 104 are, for example, web cameras, compact cameras, or smartphones. The network 105 is, for example, the Internet, a WAN (Wide Area Network), a LAN (Local Area Network), or a public wireless communication network such as 4G or 5G. The communication method between the physician-side information processing device 101 and the patient-side information processing device 103 is, for example, a P2P (Peer to Peer) type communication method, and communication is performed either wired or wirelessly. The communication method between the physician-side information processing device 101 and the physician-side imaging device 102 is a wireless communication method such as wireless LAN, infrared communication, Bluetooth (registered trademark), WirelessUSB, etc., a public wireless communication method such as 4G or 5G, or a wired communication method such as a USB cable, HDMI (registered trademark), IEEE1394, etc. The communication method between the patient-side information processing device 103 and the patient-side imaging device 104 is a wireless communication method such as wireless LAN, infrared communication, Bluetooth (registered trademark), WirelessUSB, etc., a public wireless communication method such as 4G or 5G, or a wired communication method such as a USB cable, HDMI (registered trademark), IEEE1394, etc.

[0018] The physician-side information processing device 101 transmits physician-side data to the patient-side information processing device 103 via the network 105. The physician-side imaging device 102 generates image data including a live view or other moving images and still images on the physician side, and transmits it to the physician-side information processing device 101. Also, the physician-side information processing device 101 receives patient-side data from the patient-side information processing device 103 via the network 105.

[0019] The physician-side data is, for example, data such as characters and voices generated by the physician-side information processing device 101, or image data generated by the physician-side imaging device 102.

[0020] The patient-side information processing device 103 transmits patient-side data to the physician-side information processing device 101 via the network 105. The patient-side imaging device 104 generates image data, including live views and other videos and still images of the patient, and transmits it to the patient-side information processing device 103. The patient-side information processing device 103 also receives physician-side data from the physician-side information processing device 101 via the network 105.

[0021] Patient-side data includes, for example, data such as text and voice generated by the patient-side information processing device 103, and image data generated by the patient-side imaging device 104.

[0022] With the above configuration, doctors and patients can communicate remotely by sending and receiving data to each other. <Device Configuration> Next, with reference to Figure 2, the configuration of the information processing device and imaging device that constitute the remote medical consultation system of this embodiment will be described.

[0023] As shown in Figure 2, the physician-side information processing device 101, the patient-side information processing device 103, the physician-side imaging device 102, and the patient-side imaging device 104 each have a display unit 201, VRAM 202, BMU 203, keyboard 204, PD 205, CPU 206, ROM 207, RAM 208, HDD 209, memory I / F 210, microphone 211, speaker 212, communication I / F 213, and bus 214.

[0024] The display unit 201 displays, for example, live view and other images, icons, messages, menus and other user interface information.

[0025] VRAM202 is a video memory where data for display on the display unit 201 is drawn. The data generated in VRAM202 is transferred to the display unit 201 according to predetermined rules, thereby displaying an image on the display unit 201.

[0026] The BMU (Bit Move Unit) 203 is a circuit that controls, for example, data transfer between memories (e.g., between VRAM 202 and other memories) and data transfer between memory and each I / O device (e.g., communication interface 213).

[0027] The keyboard 204 is an operating unit that has various keys for inputting characters and other information.

[0028] The PD (pointing device) 205 is used, for example, to point to icons, menus, or other content displayed on the display unit 201, or for dragging and dropping objects.

[0029] The CPU206 is a control unit that controls each device based on the OS (operating system) and control programs (applications) stored in the ROM207 and HDD209.

[0030] ROM207 is a non-volatile memory that stores control programs and data executed by CPU206.

[0031] RAM208 is a work memory that contains the work area for CPU206, a data storage area for error handling, and a control program load area.

[0032] HDD209 is an auxiliary storage device that stores control programs, content, and data executed by CPU206.

[0033] The memory interface 210 is a memory control unit that controls the reading and writing of data to an external storage medium such as a memory card.

[0034] The audio input unit 211 is one or more microphones built into the device or connected via an audio input terminal, which collect sounds (voices) from the surrounding area of ​​the device and generate an audio signal.

[0035] The audio output unit 212 is one or more speakers built into the device or connected via an audio output terminal, which emit voices from the doctor's side and the patient's side, and sounds electronic sounds or other sounds according to the operating status or instructions of the device.

[0036] Communication I / F213 is a communication unit that communicates with external devices such as information processing devices and imaging devices. Furthermore, if the external device is a communication device such as a smartphone, communication I / F213 can perform public wireless communication such as 4G or 5G.

[0037] Bus 214 includes an address bus, a data bus, and a control bus.

[0038] The control program executed by CPU206 can be provided from ROM207, HDD209, or memory card, or from external devices connected via communication I / F213.

[0039] The control program includes the operating system (OS), which is the basic software executed by the CPU 206, and applications that work in cooperation with the OS to realize advanced functions. The applications include those that allow the physician-side information processing device 101 and the patient-side information processing device 103 of this embodiment to perform the flowchart processing described later.

[0040] The flowchart processing in this embodiment is realized by loading software provided by an application. The application is assumed to have software for utilizing the basic functions of the operating systems installed on the physician-side information processing device 101 and the patient-side information processing device 103. The operating systems of the physician-side information processing device 101 and the patient-side information processing device 103 may also have software for realizing the processing in this embodiment.

[0041] <Remote Medical Consultation Flow and UI (User Interface)> Next, with reference to Figure 3, the remote medical consultation flow and the UI that follows the remote medical consultation flow of this embodiment will be described.

[0042] In telemedicine involving imaging, for example, the process begins with a conversation between the doctor and the patient, followed by imaging of the affected area, and then diagnosis and explanation to the patient using the captured images. The telemedicine ends with another conversation to conclude the consultation. Figure 3 illustrates the flow of telemedicine and the UI that follows this flow.

[0043] In the following explanation, the physician-side information processing device 101 or the patient-side information processing device 103 will be described as the main processing unit. However, in reality, this is achieved by the CPU 206 of the physician-side information processing device 101 or the patient-side information processing device 103 executing a program of the telemedicine application (or the application's functions, or the OS or OS services, etc.) stored in the memory unit. The same applies to Figures 4, 6, 7, and 8 described later. The memory unit includes ROM 207, HDD 209, memory card, and external devices connected via communication I / F 213.

[0044] In S301, the physician-side information processing device 101 sends an instruction to the patient-side information processing device 103 to switch to conversation mode. The physician-side information processing device 101 also sends the physician's live view to the patient-side information processing device 103, and the patient-side information processing device 103 sends the patient's live view to the physician-side information processing device 101.

[0045] The conversation mode is a mode for communication between a doctor and a patient, such as for icebreakers. In conversation mode, the doctor's information processing device 101 displays UI310 on the display 201. The UI310 displays a large image of the patient acquired from the patient's information processing device 103. The doctor's image 311 acquired by the doctor's imaging device 102 is superimposed on the lower left of the UI310. This allows the doctor to check how their image is displayed to the patient. In conversation mode, the patient's information processing device 103 also displays UI312 on the display 201. The UI312 displays a large image of the doctor acquired from the doctor's information processing device 101. The patient's image 313 acquired by the patient's imaging device 104 is superimposed on the lower left of the UI312.

[0046] In conversation mode, if the physician inputs a voice command to start imaging, for example, the physician-side information processing device 101 determines that the voice acquired by the voice input unit 211 is a command to start imaging and transitions to imaging mode in S302. Alternatively, as a method other than voice, a mode change button may be placed on the UI displayed on the display 201, and the system may transition to imaging mode in S302 when the PD205 determines that the mode change button has been operated.

[0047] In S302, the physician-side information processing device 101 sends an instruction to the patient-side information processing device 103 to switch to imaging mode. The patient-side information processing device 103 also sends a thumbnail of the patient (affected area) image taken by the patient-side imaging device 104 to the physician-side information processing device 101. Imaging mode is a mode for, for example, when a physician takes a high-resolution image of the patient (affected area) necessary for diagnosis. In imaging mode, the physician-side information processing device 101 displays UI320 on the display 201. UI320 displays the patient (affected area) image (thumbnail) acquired from the patient-side information processing device 103. In addition, an imaging start button 321 for executing the image acquisition process of the patient (affected area) is superimposed on the lower right of UI320. Also in imaging mode, the patient-side information processing device 103 displays UI322 on the display 201. On UI322, the image of the patient (affected area) captured by the patient-side imaging device 104 (original image) and the image 323 acquired from the physician-side information processing device 101 are superimposed on the lower right. This allows the physician to capture the affected area they want to photograph, or the affected area the patient wants examined, while both parties can view a larger image of the area.

[0048] In shooting mode, the patient (affected area) image is displayed largely on both the physician-side information processing device 101 and the patient-side information processing device 103's displays 201. However, the thumbnail displayed on the physician-side information processing device 101's display 201 may have degraded image quality compared to the original image displayed on the patient-side information processing device 103's display 201. This is thought to be a result of compression processing during streaming. To counteract this image quality degradation, it is useful for the physician-side information processing device 101 to acquire the original image from the patient-side information processing device 1103.

[0049] In shooting mode, the physician-side information processing device 101, for example, if it determines that the shooting start button 321 has been operated by PD205, will switch to the trimming mode in S303. Alternatively, the system may also switch to the trimming mode in S303 if the patient-side information processing device 103 determines that the shutter button of the patient-side imaging device 104 has been operated, as opposed to button operation.

[0050] In S303, the physician-side information processing device 101 sends an instruction to the patient-side information processing device 103 to switch to trimming mode. Trimming mode is a mode in which the physician processes a portion of the patient's (affected area) image to extract the area necessary for diagnosis. Details of trimming mode will be described later in Figures 4 and 8. In trimming mode, the physician-side information processing device 101 displays UI330 on the display 201. UI330 displays a thumbnail of the patient's (affected area) image taken in S302. The thumbnail is acquired from the patient-side information processing device 103 in the process described later in Figure 7. In addition, UI330 displays a cursor 332 indicating the operation position of PD205 and a trimming area 331 set by the drag operation of PD205. Also in trimming mode, the patient-side information processing device 103 displays UI333 on the display 201. UI333 displays the patient's (affected area) image (original image) taken by the patient-side imaging device 104. Furthermore, UI333 displays a message 334 superimposed in the upper left corner indicating that trimming is in progress on the physician's information processing device 101, and a physician's image 335 obtained from the physician's information processing device 101 is superimposed in the lower right corner. The physician's image 335 shows that the physician is concentrating on the trimming work. This clearly indicates in the UI that the physician is working on a medical examination and that communication is temporarily difficult.

[0051] Furthermore, the reason for cropping the patient's (affected area) image (thumbnail) is not only to simplify patient understanding by limiting the image area explained to the patient in the explanation mode of S305, but also to reduce the file size of the cropped image transmitted to the patient-side information processing device 103, thereby minimizing data transfer time and suppressing image and sound distortion caused by bandwidth congestion.

[0052] Once the trimming process is completed in the physician-side information processing device 101, the physician-side information processing device 101 and the patient-side information processing device 103 begin the image transmission process in S304.

[0053] In S304, the physician-side information processing device 101 transmits the information of the trimming region 331 set in S303 to the patient-side information processing device 103. Based on the information of the trimming region 331 received from the physician-side information processing device 101, the patient-side information processing device 103 generates a processed image (trimmed original image) by performing a trimming process on the original image and transmits it to the physician-side information processing device 101. The physician-side information processing device 101 displays the UI 340 on the display 201 until it receives the trimmed original image from the patient-side information processing device 103. The UI 340 displays a trimmed thumbnail 341. When the physician-side information processing device 101 receives the trimmed original image from the patient-side information processing device 103, it replaces the trimmed thumbnail 341 displayed on the UI 340 with the trimmed original image 342. This allows the telemedicine to continue without interruption even during the waiting time while the trimmed original image is being received. Furthermore, UI340 displays the patient's image 343 acquired from the patient-side information processing device 103. In addition, during the image transmission process in S304, the patient-side information processing device 103 displays UI344 on the display 201. UI344 displays the trimmed original image 345, which is the original image trimmed by the patient-side information processing device 103 based on the trimming region 331 information received from the physician-side information processing device 101. In addition, a message 346 indicating that the trimming work in the physician-side information processing device 101 has been completed is displayed in the upper left corner of UI344.

[0054] The image transmission process in S304 corresponds to the process of sending the trimmed original image generated by the patient-side information processing device 103 to the physician-side information processing device 101, and the explanation mode in S305 starts in the UI after a few seconds.

[0055] In S305, the physician-side information processing device 101 sends an instruction to the patient-side information processing device 103 to switch to explanation mode. The physician-side information processing device 101 also sends the physician's live view to the patient-side information processing device 103, and the patient-side information processing device 103 sends the patient's live view to the physician-side information processing device 101.

[0056] The explanation mode is a mode in which the doctor explains the examination results, etc., to the patient while referring to the trimmed original image generated by the patient-side information processing device 103. In explanation mode, the doctor-side information processing device 101 displays UI350 on the display 201 so that the doctor and the patient can refer to the same trimmed original image. UI350 displays the trimmed original image 351 acquired from the patient-side information processing device 103, the doctor-side image 352 acquired by the doctor-side imaging device 102, and the patient-side image 353 acquired from the patient-side information processing device 103. In addition, in explanation mode, the patient-side information processing device 103 displays UI354 on the display 201 so that the doctor and the patient can refer to the same trimmed original image. UI354 displays the trimmed original image 355 generated by the patient-side information processing device 103, the patient-side image 356 acquired by the patient-side imaging device 104 is displayed in the lower left, and the doctor-side image 357 acquired from the doctor-side information processing device 101 is displayed in the lower right. This allows both the doctor and the patient to refer to the same cropped original image, enabling the doctor to explain the condition of the affected area to the patient, and the patient to hear the explanation from the doctor.

[0057] In explanation mode, if the physician informs the patient verbally that the explanation is over, for example, the physician-side information processing device 101 determines that the voice acquired by the voice input unit 211 is an instruction to end the explanation and transitions to conversation mode in S306. Alternatively, instead of voice, an explanation end button may be placed on the UI displayed on the display 201, and when the PD205 determines that the explanation end button has been operated, the device may transition to conversation mode in S306.

[0058] In the conversation mode of S306, the physician-side information processing device 101 and the patient-side information processing device 103 switch to the same conversation mode as in S301. In the conversation mode of S306, the physician and patient converse about the prescribed medication, questions and answers throughout the consultation, and the next consultation, and then the telemedicine session ends.

[0059] Furthermore, the timing, position, and size of the images 311, 321, 323, 335, 352, and 356 superimposed on UI 310, 320, 340, and 350 of the physician-side information processing device 101, as well as the images 313, 343, 353, and 355 superimposed on UI 312, 322, 333, 344, and 354 of the patient-side information processing device 103, can be controlled by pre-determining them in a database or settings for each mode.

[0060] <Trimming Mode> Next, with reference to Figure 4, the processing and UI of the trimming mode in S303 of Figure 3 will be described.

[0061] Figure 4 illustrates the flowchart and UI showing the trimming mode processing in S303 of Figure 3. Further details of the trimming process are explained in Figures 7 and 8.

[0062] When trimming mode is started, the physician-side information processing device 101 displays a UI 410 on the display 201, which includes the thumbnail 411 acquired from the patient-side information processing device 103 in the shooting mode of S302 in Figure 3. Below the thumbnail 411, the UI 410 also displays a checkbox 414, a transfer time display 415, a reshoot button 416, and a save button 417. The checkbox 414 allows setting the timing for acquiring the trimmed original image from the patient-side information processing device 103; check this box when you want to acquire them all at once after the telemedicine consultation in Figure 3. The transfer time display 415 shows the time until the trimmed original image is received from the patient-side information processing device 103. The reshoot button 416 is operated when you want to discard the image of the patient (affected area) and reshoot due to camera shake or other reasons during shooting with the patient-side imaging device 104 in the shooting mode of S302 in Figure 3. The save button 417 is used to confirm the trimming area and send the trimming area information to the patient-side information processing device 103. Additionally, UI410 displays message 418, indicating that the trimming area can be specified by dragging PD205.

[0063] In S401, the physician-side information processing device 101 determines the drag operation position of PD205 relative to UI410 as an operation to specify the trimming area of ​​the thumbnail 411. Then, the physician-side information processing device 101 overlays a cursor 413 indicating the operation position of PD205 and the trimming area 412 corresponding to the drag of PD205 onto the thumbnail 411. This allows the physician to quickly specify the area of ​​the affected area necessary for examination. The patient-side information processing device 103 displays UI419 on the display 201. UI419 is the same as UI333 in Figure 3, so its explanation is omitted.

[0064] In S401, when the physician-side information processing device 101 detects a drag operation on the thumbnail 411 by PD205, it starts processing in S402. Also, when the physician-side information processing device 101 detects that the re-shoot button 416 has been operated by PD205, it starts processing the shooting mode in S302 as shown in Figure 3.

[0065] In S402, the physician-side information processing device 101 determines whether the save button 417 has been operated using PD205. If the physician-side information processing device 101 determines that the save button 417 has not been operated, it returns to S401 and waits until the specification of the trimming area is complete. If the physician-side information processing device 101 determines that the save button 417 has been operated, it proceeds to S403. The transfer time display 415 displays the approximate time required to acquire the trimmed original image, which is used by the physician to decide whether or not to operate the save button 417. The data transfer time varies depending on the size of the trimming area 412. The data transfer time is calculated based on the data transfer speed between the physician-side information processing device 101 and the patient-side information processing device 103, as will be described later in Figure 8.

[0066] In S403, the physician-side information processing device 101 determines the setting of checkbox 414. If the physician-side information processing device 101 determines that checkbox 414 is checked, it proceeds to S404; if it determines that checkbox 414 is not checked, it terminates the process. Checking checkbox 414 allows for settings such as sending trimmed original images in batches after the completion of telemedicine. This is useful, for example, when trimmed original images are not used in the explanation mode of S305 in Figure 3, but are used for purposes such as recording in the electronic medical record.

[0067] In S404, the patient-side information processing device 103 generates a trimmed original image based on the trimming area information received from the physician-side information processing device 101 and sends it to the physician-side information processing device 101. The physician-side information processing device 101 receives the trimmed original image generated by the patient-side information processing device 103. The processing in S404 is the same as the image transmission processing in S304 in Figure 3. In S404, the patient-side information processing device 103 displays a dialog 420 on the display 201, which includes a message 421 and a close button 422. Message 421 is a string of characters that notifies the patient that there is a risk of the streaming image or sound being disrupted. The close button 422 allows the patient-side information processing device 103 to hide the dialog 420 by operating the close button 422 via PD205. Once the trimmed original image is sent from the patient-side information processing device 103 to the physician-side information processing device 101, the processing in S405 begins.

[0068] In S405, the physician-side information processing device 101 displays the UI430, which includes the trimmed original image 431 acquired from the patient-side information processing device 103, on the display 201 and terminates processing. The UI430 is the same as the UI350 in Figure 3, so its explanation is omitted.

[0069] <Modified System Configurations> Next, modified system configurations of this embodiment will be described with reference to Figures 5 and 6.

[0070] The system configuration in Figure 5 differs from that in Figure 1 in that an assistant-side imaging device 501 is added to the patient-side information processing device 103. The assistant-side imaging device 501 is intended to be held by a third party other than the patient, such as an assistant (caregiver, nurse, etc.). This allows for the capture of images of the back and other areas that the patient cannot capture on their own, and these images can be shared with a doctor to assist in telemedicine.

[0071] The communication method between the patient-side information processing device 103 and the assistant-side imaging device 501 is the same as the communication method between the patient-side information processing device 103 and the patient-side imaging device 104.

[0072] Figure 6 illustrates a flowchart and UI showing a modified version of the shooting mode shown in Figure 3.

[0073] In S601, the physician-side information processing device 101 sends an instruction to the patient-side information processing device 103 to switch to shooting mode. S601 is the same as the shooting mode in S302 in Figure 3. In the shooting mode of S601, for example, the physician-side information processing device 101 displays UI610 on the display 201. The patient-side information processing device 103 displays UI612 on the display 201. UI610 of the physician-side information processing device 101 has a camera switching button 611 added to UI320 of the physician-side information processing device 101 in Figure 3. The camera switching button 611 is a button that allows the physician to select whether to photograph the patient with the patient-side shooting device 104 or with the assistant-side shooting device 501. UI612 of the patient-side information processing device 103 is the same as UI322 of the patient-side information processing device 103 in Figure 3. When the physician's information processing device 101 determines that the camera switching button 611 has been operated by PD205, it starts processing S602.

[0074] In S602, the physician-side information processing device 101 sends a camera switching instruction to the patient-side information processing device 103. The patient-side information processing device 103 determines, in response to the camera switching instruction received from the physician-side information processing device 101, which of the patient-side imaging device 104 and the assistant-side imaging device 501 is currently configured to take images, and whether there is a connected imaging device that is not configured to take images. In the example in Figure 5, if the patient-side imaging device 104 is the configured imaging device, then the assistant-side imaging device 501 is the connected imaging device. The physician-side information processing device 101 displays UI620 on the display 201. The patient-side information processing device 103 displays UI623 on the display 201. On UI620 and 624, messages 621 and 624, indicating that the patient-side information processing device 103 is switching cameras, are superimposed on the upper left of the patient (affected area) image. In addition, the physician-side image 622 is superimposed on the lower left of the patient (affected area) image on UI620. In UI624, the doctor's image 625 is superimposed on the lower right of the patient's (affected area) image. Doctor's images 622 and 625 show the doctor concentrating on camera switching operations. This clearly indicates in the UI that the doctor is in the process of switching cameras and that communication is temporarily difficult.

[0075] In S603, the patient-side information processing device 103 determines whether there is a connected imaging device other than the one currently set, in response to the camera switching instruction received in S602. If the patient-side information processing device 103 determines that there is a connected imaging device, it switches the currently set imaging device to the connected imaging device; if it determines that there is no connected imaging device, it terminates the process. In the example in Figure 5, the currently set imaging device is switched from the patient-side imaging device 104 to the assistant-side imaging device 501. The physician-side information processing device 101 displays UI630 on the display 201. The patient-side information processing device 103 displays UI631 on the display 201. UI630 and 631 display images of the patient (affected area) taken by the imaging device after the currently set imaging device was switched. In the example in Figure 5, images of the patient (affected area) taken by the assistant-side imaging device 501 are displayed. This allows for examination by switching from an image of the patient taken from the front to an image taken from the assistant's perspective, for example, from behind, and sharing it with the physician.

[0076] The camera switching button 611 may be located in the UI of the patient-side information processing device 103, and the connected imaging device may be switched each time the camera switching button 611 is operated. Furthermore, the camera switching button 611 may be in the form of a menu display that allows the user to select the name or unique identification information (such as a MAC address) of the connected imaging device.

[0077] <Operation Flow> Next, with reference to Figure 7, the operation of the physician-side information processing device and the patient-side information processing device in the telemedicine system of this embodiment will be described.

[0078] The storage units of the physician-side information processing device 101 and the patient-side information processing device 103 store data related to the UI corresponding to each mode (conversation, photography, trimming, explanation) in the telemedicine flow shown in Figure 3.

[0079] In S701, the physician-side information processing device 101 sends an instruction to the patient-side information processing device 103 to switch to conversation mode. The physician-side information processing device 101 displays, for example, the UI310 shown in Figure 3 on the display 201.

[0080] In S702, the patient-side information processing device 103 receives an instruction from the physician-side information processing device 101 to switch to conversation mode. The patient-side information processing device 103 displays, for example, the UI312 shown in Figure 3 on the display 201.

[0081] In S703, the physician-side information processing device 101 determines whether the conversation mode has ended. As explained in S301 in Figure 3, in conversation mode, if the physician gives a voice command to start imaging, for example, the physician-side information processing device 101 determines that the voice acquired by the voice input unit 211 is a command to start imaging. Alternatively, a mode change button may be placed on the UI displayed on the display 201, and the PD 205 may determine that the mode change button has been operated. While the conversation mode in S703 is ongoing, the physician and patient are communicating, for example, as explained in S301 in Figure 3.

[0082] In S704, the physician-side information processing device 101 sends an instruction to the patient-side information processing device 103 to switch to imaging mode. The physician-side information processing device 101 displays, for example, UI320 in Figure 3 or UI610 and UI630 in Figure 6 on the display 201.

[0083] In S705, the patient-side information processing device 103 receives an instruction from the physician-side information processing device 101 to switch to imaging mode. The patient-side information processing device 103 displays, for example, UI322 or UI631 in Figure 3 on the display 201.

[0084] In S706, the physician-side information processing device 101 sends an imaging instruction to the patient-side information processing device 103 in order to capture an image of the patient (affected area) using the patient-side imaging device 104. The imaging instruction can be given, for example, by operating the imaging start button 321 of the UI320 in Figure 3 using the PD205.

[0085] In S707, the patient-side information processing device 103 receives imaging instructions from the physician-side information processing device 101. In response to the imaging instructions, the patient-side information processing device 103 uses the patient-side imaging device 104 to image the patient (affected area) and stores the generated image data in the storage unit.

[0086] In S708, the patient-side information processing device 103 generates thumbnail data from the image data (original image data) saved in S707, saves it to the memory unit, and transmits it to the physician-side information processing device 101. In this case, the file size of the image data (original image) is also transmitted. The file size is the amount of data in the image file. For example, a JPEG image file with 3648 x 2736 pixels (10MP, 4:3) will have a file size of approximately 2.9MB depending on the compression ratio. When generating thumbnail data from this image file, it can be reduced to, for example, a JPEG image of 640 x 480 pixels (0.3MP, 4:3). The file size is used to calculate the file size of the cropped original image in S804, which will be described later in Figure 8.

[0087] In S709, the physician-side information processing device 101 stores the file sizes of the thumbnail data and original image data received from the patient-side information processing device 103 in its storage unit.

[0088] In S710, the physician-side information processing device 101 sends an instruction to the patient-side information processing device 103 to switch to trimming mode. The physician-side information processing device 101 displays, for example, UI330 in Figure 3 or UI410 in Figure 4 on the display 201.

[0089] In S711, the patient-side information processing device 103 receives an instruction from the physician-side information processing device 101 to switch to trimming mode. The patient-side information processing device 103 displays, for example, the UI333 shown in Figure 3 on the display 201.

[0090] In S712, the physician-side information processing device 101 performs trimming area specification processing based on the thumbnail data saved in S709 and the file size of the original image data. The physician-side information processing device 101 can obtain information on the trimming area of ​​the original image data by converting the coordinate information of the trimming area set for the thumbnail data to the file size of the original image data. Details of the trimming area specification processing will be described later in Figure 8.

[0091] In S713, the physician-side information processing device 101 determines whether to save or discard (re-shoot) the file size of the thumbnail data and original image data received in S709 and the trimming area information acquired in S712. For example, if the re-shoot button 416 in Figure 4 is operated, it is determined to re-shoot, and if the save button 417 in Figure 4 is operated, it is determined to save. If the physician-side information processing device 101 determines to re-shoot, it starts the process in S704, and if it determines to save, it starts the process in S714.

[0092] In S714, the physician-side information processing device 101 transmits the trimmed area information saved in S713 to the patient-side information processing device 103. In this case, flag information indicating whether or not to immediately retrieve the trimmed original image is also transmitted, depending on whether or not the checkbox 414 of UI410 in Figure 4 is checked. Once transmission is complete, the physician-side information processing device 101 displays, for example, UI340 in Figure 3 on the display 201. The trimmed thumbnail 341 displayed in UI340 is generated in the process of saving the trimmed area information in S803, which will be described later.

[0093] In S715, the patient-side information processing device 103 performs trimming of the image data (original image) based on the trimming area information received from the physician-side information processing device 101. Note that the coordinate information of the trimming area specified in S712 is calculated based on the thumbnail, so it needs to be converted to the size of the image data (original image). In the example of file size explained in S708, the image data (original image) is a JPEG image with 3648 x 2736 pixels (10MP, 4:3), while the thumbnail data is a JPEG image with 640 x 480 pixels (0.3MP, 4:3). Therefore, the trimming area of ​​the image data (original image) can be calculated by multiplying the top-left and bottom-right coordinates of the trimming area received from the physician-side information processing device 101 in S714 by 5.7 both vertically and horizontally.

[0094] In S716, the patient-side information processing device 103 determines whether to immediately send the trimmed original image to the physician-side information processing device 101 based on the flag information received from the physician-side information processing device 101 in S714. If the patient-side information processing device 103 determines to immediately send the trimmed original image, it proceeds to S717; if it determines not to immediately send it, it proceeds to S722. If it determines not to immediately send it, the trimmed original image is stored in the memory unit as it will be sent in S727, as described later.

[0095] In S717, the patient-side information processing device 103 transmits the trimmed original image generated in S715 to the physician-side information processing device 101. The patient-side information processing device 103 displays, for example, the UI344 shown in Figure 3 on the display 201.

[0096] In S718, the physician-side information processing device 101 stores the trimmed original image received from the patient-side information processing device 103 in its storage unit.

[0097] In S719, the physician-side information processing device 101 sends an instruction to the patient-side information processing device 103 to switch to explanation mode. The physician-side information processing device 101 displays, for example, the UI350 shown in Figure 3 on the display 201.

[0098] In S720, the patient-side information processing device 103 receives an instruction from the physician-side information processing device 101 to switch to explanation mode. The patient-side information processing device 103 displays, for example, the UI354 shown in Figure 3 on the display 201.

[0099] In S721, the physician-side information processing device 101 determines whether the explanation mode has ended. As explained in S305 of Figure 3, in explanation mode, if the physician verbally informs the patient that the explanation is ending, the physician-side information processing device 101 determines that the voice acquired by the voice input unit 211 is an instruction to end the explanation. Alternatively, as a method other than voice, an explanation end button may be placed on the UI displayed on the display 201, and the explanation may be determined to have ended when the explanation end button is operated by PD205.

[0100] In S722, the physician-side information processing device 101 determines whether or not to continue imaging. This allows the physician to continue imaging and explanation from S706 onward. In imaging mode, for example, if the physician verbally inputs that imaging of the affected area should be stopped, the physician-side information processing device 101 determines that the voice acquired by the voice input unit 211 is an instruction to stop imaging. Alternatively, instead of voice, an imaging stop button may be placed on the UI displayed on the display 201, and imaging may be stopped when the imaging stop button is operated by the PD205.

[0101] In S723, the physician-side information processing device 101 sends an instruction to the patient-side information processing device 103 to switch to conversation mode. The physician-side information processing device 101 displays, for example, the UI310 shown in Figure 3 on the display 201.

[0102] In S724, the patient-side information processing device 103 receives an instruction from the physician-side information processing device 101 to switch to conversation mode. The patient-side information processing device 103 displays, for example, the UI312 shown in Figure 3 on the display 201.

[0103] In S725, the physician-side information processing device 101 determines whether or not to terminate the conversation mode (remote consultation). While the remote consultation is ongoing, for example, as explained in S306 of Figure 3, the physician and patient converse about the prescribed medication, questions and answers throughout the consultation, and the next consultation. In conversation mode, for example, if the physician inputs a voice command to terminate the remote consultation, the voice input unit 211 determines that the voice acquired is a command to terminate the remote consultation. Alternatively, as a method other than voice, a remote consultation termination button may be placed on the UI displayed on the display 201, and the PD205 may determine that the remote consultation has been terminated when the remote consultation termination button is operated.

[0104] In S726, the patient-side information processing device 103 determines, based on the flag information received from the physician-side information processing device 101 in S714, whether or not there is a trimmed original image saved for later transmission in S716. If the patient-side information processing device 103 determines that there is a trimmed original image saved for later transmission, it proceeds to S727; otherwise, it terminates the process.

[0105] In S727, the patient-side information processing device 103 transmits the trimmed original image saved in S716 to the physician-side information processing device 101, similar to S717.

[0106] In S728, the physician-side information processing device 101 stores the trimmed original image received from the patient-side information processing device 103 in its storage unit, similar to S718.

[0107] In S707, the explanation was based on the premise that the patient-side imaging device 104 is capable of taking images. However, if it is not possible to take images, such as with a webcam, then, according to the imaging instruction, image data can be generated and saved by taking a screenshot of the live view.

[0108] <Cropping Area Specification Process> Next, with reference to Figure 8, the cropping area specification process in S712 of Figure 7 will be explained.

[0109] In S712, the physician's information processing device 101 displays, for example, the UI410 shown in Figure 4 on the display 201.

[0110] In S801, the physician-side information processing device 101 continues processing until it detects a drag operation by PD205 on the thumbnail 411 of UI410, which is the operation to start specifying the trimming area of ​​the thumbnail 411 of UI410 in Figure 4. If the physician-side information processing device 101 detects a drag operation by PD205, it proceeds to S802. In this case, the starting point of the drag operation is the coordinate of the top left of the rectangle of the trimming area. The physician-side information processing device 101 stores the coordinate information of the starting point of the drag operation in its storage unit.

[0111] In S802, the physician-side information processing device 101 continues processing until it detects the end of the drag operation of PD205 on the thumbnail 411 of UI410, which is the operation to terminate the specification of the trimming area of ​​the thumbnail 411 of UI410 in Figure 4. If the physician-side information processing device 101 detects the end of the drag operation of PD205, it proceeds to S803. In this case, the endpoint of the drag operation is the coordinate of the lower right corner of the rectangle of the trimming area. The physician-side information processing device 101 stores the coordinate information of the endpoint of the drag operation in its storage unit.

[0112] In S803, the physician-side information processing device 101 temporarily stores the coordinate information of the trimming area saved in S801 and S802, along with the trimmed thumbnail data, in its storage unit. The trimmed thumbnail data temporarily stored here corresponds to the trimmed thumbnail 341 of the UI340 in Figure 3, which is displayed on the display 201 of the physician-side information processing device 101 in S714 of Figure 7.

[0113] In S804, the physician-side information processing device 101 converts the coordinate information of the trimmed area of ​​the trimmed thumbnail data saved in S803 to the size of the original image, which is also used in the calculation of data transfer time in S805. The calculation method is explained using the example of S708 in Figure 7, where the original image is a 2.9MB JPEG image with 3648 x 2736 pixels (10MP, 4:3), and the thumbnail is an 87KB JPEG image with 640 x 480 pixels (0.3MP, 4:3). For example, if the coordinates of the trimmed area of ​​the thumbnail saved in S803 are a rectangle with a starting point of (100,50) and an ending point of (500,300), then by multiplying both the vertical and horizontal dimensions by 5.7 as the ratio for conversion to the size of the original image, the trimmed original image will have a starting point of (570,285) and an ending point of (2850,1710), and the file size can be calculated to be approximately 944KB.

[0114] In S805, the physician-side information processing device 101 calculates the data transfer time of the trimmed original image and displays it as the transfer time display 415 in UI410 in Figure 4. The calculation method can be calculated, for example, from the time required to send the thumbnail data in S708 in Figure 7 and the file size of the original image data. For example, if the original image is a 2.9MB JPEG image with 3648 x 2736 pixels (10MP, 4:3) and the thumbnail is an 87KB JPEG image with 640 x 480 pixels (0.3MP, 4:3), then if the transfer time of the thumbnail (87KB) is 0.1 seconds, the transfer time of the original image (944KB) can be calculated to be approximately 1.1 seconds.

[0115] As described above, this embodiment enables the display of images and UI without delay or distortion during telemedicine involving photography, facilitating smooth communication.

[0116] [Other embodiments] The present invention can also be realized by supplying a program that implements one or more functions of each embodiment to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. Furthermore, the present invention can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0117] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]

[0118] 101...Physician-side information processing device, 102...Physician-side imaging device, 103...Patient-side information processing device, 104...Patient-side imaging device, 105...Network, 501...Assistant-side imaging device

Claims

1. A physician's information processing device that is communicatively connected to a patient's information processing device, A receiving means that receives from the patient-side information processing device a second image, which is an image of the patient taken in accordance with the instructions of the physician-side information processing device, and which has a reduced data size compared to the first image of the patient. Display means for displaying a user interface including the second image, Processing means for extracting a portion of an image from the second image, The processing means includes a transmission means that transmits the information set by performing the processing on the second image to the patient's information processing device. The receiving means receives the processed image from the patient-side information processing device, which has performed the processing on the first image based on the information in the patient-side information processing device. The physician's information processing device can be switched between multiple modes. The display means is a user interface including the second image, which displays a user interface corresponding to one of the plurality of modes. The aforementioned multiple modes include a first mode in which the patient and the doctor converse, a second mode in which the patient is photographed, a third mode in which the patient's image is processed, and a fourth mode in which the processed image is used to explain to the patient. In the second mode, the transmitting means transmits an instruction to the patient-side information processing device to photograph the patient. An information processing apparatus characterized in that, in the third mode, the transmitting means transmits the information to the patient-side information processing apparatus.

2. In the second mode, the images of the shooting instruction means and the doctor's side are superimposed on the second image. The information processing apparatus according to claim 1, characterized in that, in the third mode, the patient's image and the doctor's image are superimposed on the processed image.

3. The system has means for calculating the time required to receive the processed image from the file size and transfer speed when the second image is received from the patient's information processing device, The information processing apparatus according to claim 1 or 2, characterized in that the display means displays the time in the third mode.

4. The display means displays means for specifying the timing for acquiring the processed image from the patient-side information processing device in the third mode. The information processing device according to any one of claims 1 to 3, characterized in that the timing includes the timing at which the third mode ends or the timing at which the fourth mode ends in the information processing device on the physician's side.

5. The display means includes means for returning to the second mode in the third mode and re-capturing the patient's image, An information processing apparatus according to any one of claims 1 to 4, characterized by displaying means for saving the information set in the third mode and terminating the third mode.

6. The display means, in the second mode, displays a means that allows the patient's information processing device to select whether to photograph the patient using a first photographing device that takes images of the patient, or whether a third party other than the patient photographs the patient using a second photographing device that takes images of the patient. The information processing device according to claim 1, characterized in that it is possible to switch between displaying an image of the patient taken by the first imaging device and an image of the patient taken by the second imaging device.

7. The information processing apparatus according to claim 1, characterized in that the display means replaces the processed second image displayed in the third mode with the processed first image received from the patient-side information processing apparatus.

8. The second image is a thumbnail corresponding to the first image. The aforementioned process is cropping, which involves cutting out a portion of the image. The information processing apparatus according to any one of claims 1 to 7, characterized in that the information is information of the region to be trimmed.

9. A patient-side information processing device that is communicatively connected to a physician-side information processing device, A user interface including a first image generated by photographing a patient in accordance with the instructions of the physician's information processing device, comprising a display means for displaying a user interface corresponding to the mode of the physician's information processing device, A generation means for generating a second image with a reduced data size compared to the first image, A transmission means for transmitting the second image to the physician's information processing device, The physician's information processing device has a receiving means that receives information set by performing a process to extract a portion of the image from the second image, The transmission means transmits the processed image, which has undergone the processing on the first image based on the information, to the physician's information processing device. The modes of the physician's information processing device include a first mode in which the patient and the physician converse, a second mode in which the patient is photographed, a third mode in which the patient's image is processed, and a fourth mode in which the processed image is used to explain to the patient. In the second mode, the transmitting means transmits the second image to the physician's information processing device. In the third mode, the information processing apparatus is characterized in that the transmitting means transmits the processed image to the physician's information processing apparatus.

10. In the second and third modes, the doctor's image is superimposed on the first image. In the third mode, information indicating that the processing is being performed and the doctor's image are superimposed on the first image. The information processing apparatus according to claim 9, characterized in that, in the fourth mode, the patient's image and the doctor's image are superimposed on the processed image.

11. Processing means for extracting a portion of an image from the first image, The system includes a calculation means that converts the aforementioned information to the size of the first image and calculates information about the region in which the processing means performs processing on the first image, The processing means performs the processing on the first image based on the region information calculated by the calculation means. The transmission means transmits the processed image, which has undergone the processing on the first image, to the physician's information processing device. The information processing apparatus according to claim 9, characterized in that the display means displays the processed image.

12. The information processing device according to any one of claims 9 to 11, characterized in that the transmission means transmits the processed image at a timing specified by the physician's information processing device.

13. A first imaging device that captures images of the patient and a second imaging device that allows a third party other than the patient to capture images of the patient are connected in a manner that enables communication between them. The information processing apparatus according to any one of claims 9 to 12, characterized in that the transmission means transmits an image of the patient taken by the first imaging device or an image of the patient taken by the second imaging device to the information processing apparatus on the physician's side.

14. A method for controlling a physician's information processing device that is communicatively connected to a patient's information processing device, The steps include receiving from the patient-side information processing device a second image, which is an image of the patient taken in accordance with the instructions of the physician-side information processing device, and which has a reduced data size compared to the first image of the patient, The steps include displaying the user interface including the second image on the display means, The steps include: performing a process to extract a portion of an image from the second image displayed on the display means; The steps include: transmitting the information set by performing the above processing on the second image to the patient-side information processing device; The process includes receiving a processed image from the patient-side information processing device, which has performed the processing on the first image based on the information. The physician's information processing device can be switched between multiple modes. In the display step, a user interface including the second image, corresponding to one of the plurality of modes, is displayed on the display means. The aforementioned multiple modes include a first mode in which the patient and the doctor converse, a second mode in which the patient is photographed, a third mode in which the patient's image is processed, and a fourth mode in which the processed image is used to explain to the patient. In the second mode described above, the transmission step involves sending an instruction to the patient-side information processing device to photograph the patient, In the third mode, the method is characterized in that the transmission step involves transmitting the information to the patient-side information processing device.

15. A method for controlling a patient-side information processing device that is connected to a physician-side information processing device in a communicative manner, A user interface including a first image generated by photographing a patient in accordance with the instructions of the physician's information processing device, comprising the step of displaying the user interface corresponding to the mode of the physician's information processing device on a display means, The steps include generating a second image with a reduced data size compared to the first image, A first transmission step of transmitting the second image to the physician's information processing device, The process of extracting a portion of an image from the second image in the physician's information processing device and receiving the set information from the physician's information processing device, The process includes a second transmission step of transmitting the processed image, which has been processed on the first image based on the aforementioned information, to the physician's information processing device. The modes of the physician's information processing device include a first mode in which the patient and the physician converse, a second mode in which the patient is photographed, a third mode in which the patient's image is processed, and a fourth mode in which the processed image is used to explain to the patient. In the second mode, the first transmission step involves transmitting the second image to the physician's information processing device. The method is characterized in that, in the third mode, the second transmission step involves transmitting the processed image to the physician's information processing device.

16. A program for causing a computer to function as each of the means of an information processing device described in any one of claims 1 to 13, excluding the display means.

Images