Information processing device, control method and program for the information processing device, and telemedicine support system

Abstract

To change a streaming condition transmitted from a patient-side information processing apparatus according to the details of medical treatment to preferentially avoid deterioration of information necessary for the medical treatment.SOLUTION: An information processing apparatus is available for a doctor and a patient to transmit and receive voices and videos in a remote medical treatment assistance system, and the apparatus has: an acquisition unit that acquires information indicating the type of medical treatment; a determination unit that, according to the type of medical treatment acquired by the acquisition unit, determines a streaming condition for performing transmission from a patient-side information processing apparatus to a doctor-side information processing apparatus; and a changing unit that changes streaming according to the streaming condition determined by the determination unit.SELECTED DRAWING: Figure 4

Description

Technical Field

[0005] , , , , , ,

[0004] , , , , ,

[0001] The present invention relates to an information processing apparatus, The information processing device performs a control method and program, and a remote medical assistance system.

Background Art

[0002] In a remote medical system, diagnosis may be performed using streaming with video. For example, when conducting a medical interview, there is a case where a doctor talks while observing the patient's complexion. In such a case, a live view video stream or the like is transmitted and received by connecting an imaging device such as a camera to the PCs (Personal Computers, hereinafter referred to as PCs) of both the doctor and the patient.

[0003] Also, when using a special modality during remote medical treatment, there is a case where both the video captured by that modality and the video of a normal camera are displayed on the remote diagnosis system. In such a case, since the normal camera always has a constant frame rate, while the frame rate of the modality video changes depending on the diagnosis mode, there was a problem when mixing and distributing them. As a means for solving such a problem, for example, Patent Document 1 is known. In Patent Document 1, even if the frame rates of the camera video capturing a diagnostic scene or the like and the ultrasonic image are different, by storing them mixed in the order of acquisition time series, a technique is disclosed that can make the synchronization relationship between the ultrasonic image and the patient video and each frame rate the same as when acquired on the patient side.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In conventional telemedicine systems, if the specifications of the PC used were insufficient or the network bandwidth was insufficient, it was sometimes impossible to achieve optimal resolution and frame rate for the video used for streaming.

[0006] However, conventional technologies have the problem that, because they are controlled specifically for a particular modality, they cannot cope with the various conditions that arise and change depending on the content of the consultations conducted remotely, and therefore cannot avoid the risks mentioned above. [Means for solving the problem]

[0007] To solve this problem, for example, the information processing apparatus of the present invention has the following configuration. That is, An information processing device that can be used for sending and receiving audio and video between doctors and patients in a remote medical support system, A means of obtaining information indicating the type of medical treatment, A determination means for determining streaming conditions to be transmitted from the patient's information processing device to the physician's information processing device according to the type of medical treatment acquired by the acquisition means, The system includes a modification means for changing the streaming to conform to the streaming conditions determined by the determination means. [Effects of the Invention]

[0008] According to the present invention, it becomes possible to change the streaming conditions transmitted from the patient-side information processing device according to the content of the examination, thereby prioritizing the avoidance of degradation of information necessary for diagnosis. [Brief explanation of the drawing]

[0009] [Figure 1] Overall configuration diagram of the telemedicine support system in the embodiment. [Figure 2] (a) is a hardware configuration diagram of the information processing device, and (b) is a hardware configuration diagram of the user equipment. [Figure 3] A diagram showing an example of the consultation flow for telemedicine in the embodiment. [Figure 4] A diagram showing an example of the streaming condition changing unit of the first embodiment. [Figure 5] A diagram showing an example of a streaming condition correspondence table in an embodiment. [Figure 6] A diagram showing an example of the streaming condition modification unit of the second embodiment. [Figure 7] A diagram showing an example of the streaming condition modification unit of the third embodiment. [Figure 8] A diagram showing an example of a correspondence table between modalities and streaming conditions in an embodiment. [Figure 9] A diagram showing an example of the streaming condition modification unit of the fourth embodiment. [Figure 10] A diagram showing an example of the streaming condition changing unit of the fifth embodiment. [Figure 11] A diagram showing an example of the streaming condition changing unit of the seventh embodiment. [Modes for carrying out the invention]

[0010] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0011] [First Embodiment] Figure 1 is an overall diagram of the telemedicine support system in an embodiment.

[0012] The remote medical assistance system according to this embodiment is composed of an information processing device used by a doctor in the hospital, an information processing device used by patients in each household or nursing facility, and a network (e.g., the Internet) that connects them. And through an application operating on the information processing device, doctors and patients can communicate with each other by voice or chat. Furthermore, cameras capable of projecting themselves are prepared for both the doctor and the patient side. By distributing and receiving the videos taken by this camera to each other, communication can be made smoother.

[0013] Note that in the information processing devices used by doctors and patients that constitute the remote medical assistance system according to this embodiment, it is assumed that a user device for photographing oneself as a distribution target is connected as an external device. The information processing device will distribute the video acquired by the above user device to the information processing device on the other side via the network. However, the information processing devices used by doctors and patients may have a built-in imaging function for the video to be distributed. Also, the connection form between the information processing device and the user device may be either wired or wireless. In the following embodiments, the network will be described by taking the case of connecting a hospital to each household or nursing facility as an example, but it may also be an intranet closed within the hospital.

[0014] As shown in FIG. 1, the remote medical assistance system is composed of a doctor-side information processing device 102 and user devices 101 connected thereto, a patient-side information processing device 104 and user devices 105 connected thereto, and a network 103 that communicably connects the doctor-side information processing device 102 and the patient-side information processing device 104.

[0015] The doctor-side information processing device 102 performs speech recognition on the voice signal of the doctor obtained from the microphone connected to itself, and acquires the live view video data of the doctor from the user device 101. Then, the doctor-side information processing device 102 transmits the live view video data with the doctor's voice to the patient-side information processing device 104. Note that, so that the doctor can check his / her own video, the doctor-side information processing device 102 may display the video to be transmitted on a part of the display screen. Also, here it is assumed that the user device 101 acquires the video and the doctor-side information processing device 102 acquires the voice. However, for example, the user device 101 may acquire both the video and the voice, or the doctor-side information processing device 102 may acquire both the video and the voice. In the latter case, the user device 101 may be made unnecessary.

[0016] Similarly, the patient-side information processing device 104 performs speech recognition on the voice signal of the doctor obtained from the microphone connected to itself, acquires data such as the live view video of the patient from the user device 105, and transmits it to the doctor-side information processing device 102. At this time, so that the patient can check his / her own video, the patient-side information processing device 104 may display the video to be transmitted on a part of the display screen. Also, here it is assumed that the user device 105 acquires the video and the patient-side information processing device 104 acquires the voice. However, for example, the user device 105 may acquire both the video and the voice, or the patient-side information processing device 104 may acquire both the video and the voice. In the latter case, the user device 101 may be made unnecessary.

[0017] In addition, the doctor-side information processing device 102 transmits the data obtained from the keyboard and the microphone, and the data (live view video data) obtained from the user device 101, to the patient-side information processing device 104 via the network 103. Also, the doctor-side information processing device 102 receives various data from the patient-side information processing device 104 via the network 103.

[0018] The patient-side information processing device 104 transmits data acquired from the keyboard and microphone, as well as data acquired from the user device 105 (live view video data), to the physician-side information processing device 102 via the network 103. The patient-side information processing device 104 also receives various data from the physician-side information processing device 102 via the network 103.

[0019] In this way, by processing data exchanged between the physician-side information processing device 102, operated by a doctor within the hospital, and the patient-side information processing device 104, operated by a patient at their home or nursing facility, this system can assist in telemedicine.

[0020] Although the applications running on the physician-side information processing device 102 and the patient-side information processing device 104 differ, their hardware configurations can be the same as those of a general-purpose personal computer. Similarly, the user devices 101 and 105 can be the same, consisting of devices such as digital cameras that have imaging capabilities and the ability to output captured live video data externally. Therefore, the physician-side information processing device 102 and the user device 101 will be described below, while the patient-side information processing device 104 and the user device 105 will be assumed to be the same, and their descriptions will be omitted.

[0021] Figure 2(a) shows the block diagram of the physician-side information processing device 102, and Figure 2(b) shows the block diagram of the user device 101. As already explained, the patient-side information processing device 104 also has the configuration shown in Figure 2(a), and the user device 105 also has the configuration shown in Figure 2(b).

[0022] As shown in Figure 2(a), the information processing device includes a display unit 201, VRAM 202, BMU 203, keyboard 204, PD 205, CPU 206, HDD 207, RAM 208, ROM 209, flexible disk 210, microphone 211, speaker 212, network I / F 213, bus 214, and interface 215.

[0023] The display unit 201 displays, for example, live view images, icons, messages, menus, and other user interface information. The VRAM 202 is used to draw moving images for display on the display unit 201. The data generated in the VRAM 202 is transferred to the display unit 201 according to predetermined rules, and is then displayed on the display unit 201. The BMU (Bit Move Unit) 203 controls, for example, data transfer between memories (for example, between VRAM 202 and other memories) and data transfer between memory and each I / O device (for example, network I / F 213).

[0024] The keyboard 204 has various keys for inputting characters, etc. The PD (pointing device) 205 is used, for example, to point to icons, menus, or other content displayed on the display unit 201, or for dragging and dropping objects.

[0025] The CPU 206 controls each device based on control programs such as the OS or other programs described later, which are stored in the HDD 207, ROM 209, or memory card 210.

[0026] HDD207 stores various control programs and temporarily stored data. RAM208 contains the CPU206's work area, a data backup area for error handling, and a control program load area. ROM209 stores data such as control programs executed within the information processing unit and temporarily stored data.

[0027] The memory card 210 stores various control programs, temporary data, etc. The microphone 211 acquires various sounds (voices). The microphone 211 may be a separate device from the physician-side information processing unit 102. The speaker 212 emits various sounds (voices). The speaker 212 may be a separate device from the physician-side information processing unit 102. The network interface 213 communicates with the patient-side information processing unit 104 via the network.

[0028] Bus 214 includes an address bus, a data bus, and a control bus. Control programs for the CPU 206 can be provided from the HDD 207, ROM 209, memory card 210, or from other information processing devices via the network interface 213. Interface 215 is an interface for connecting user devices 101, and is typically a USB interface.

[0029] As shown in Figure 2(b), the user device 101 has an interface 221, a CPU 222, a ROM 223, a RAM 224, an imaging unit 225, and a bus 226. Interface 221 is an interface for connecting to the interface 215 of the physician's information processing device 102, and is typically a USB interface. The CPU 222 is a processor that controls the user device 101 by executing a program stored in the ROM 223. At this time, the CPU 223 uses the RAM 224 as a work area. This RAM 224 is also used to temporarily store video data obtained by imaging by the imaging unit 225. The main processing of the CPU 222 is the control of the imaging unit 225 and the processing of transmitting the images obtained by imaging to the physician's information processing device 102 via the interface 221.

[0030] In the above configuration, the physician's information processing device 102 processes the transmission of the physician's video and audio to the patient's information processing device 104. The physician's information processing device 102 also receives the patient's video and audio from the patient's information processing device 104, and displays it on the display unit 201 and outputs audio from the speaker 212. The patient's information processing device 104 also processes the transmission of the patient's video and audio to the physician's information processing device 102. The patient's information processing device 104 also receives the physician's video and audio from the physician's information processing device 102, and displays it on the display unit 201 and outputs audio from the speaker 212. The patient's information processing device 104 further receives the patient's medical record from the physician's information processing device 102 and processes the display of it on the display unit 201 (or prints it using a printer not shown).

[0031] Figure 3 shows an example of a telemedicine consultation flow.

[0032] When the physician-side information processing device 102, used by a physician within the hospital, and the patient-side information processing device 104, used by a patient at their home or nursing facility, become able to communicate via a network 103 such as an internet connection, remote medical consultation begins (S301). It is also possible to use the remote medical consultation support system in cases where a large hospital and a small hospital are connected via the internet, with the patient located at the small hospital. In such cases, the physician-side environment shown in Figure 1 is assumed to be at the large hospital, and the patient-side environment at the small hospital. These assumptions are not limited to these.

[0033] When telemedicine begins, the doctor and patient first engage in small talk (S302). As mentioned earlier, the telemedicine support system provides each doctor and patient with a camera capable of displaying themselves, and they communicate by sharing video via an application connected to the internet.

[0034] Next, the doctor will interview the patient about their symptoms and current condition based on the pre-consultation questionnaire (S303). The pre-consultation questionnaire should be filled out on the telemedicine appointment form when the telemedicine appointment is scheduled. Telemedicine appointments can be made by telephone by hospital staff, or the patient can fill out a questionnaire on a designated form via the internet, but these are not the only methods. If a pre-consultation questionnaire has not been conducted, the doctor may interview the patient directly during the telemedicine session.

[0035] Then, the doctor begins the consultation (S304). The content of the consultation varies depending on the individual case. Once the consultation is complete, the doctor enters the results into a designated system such as an electronic medical record and explains them to the patient (S305). Although not shown in Figure 1, the doctor's information processing device 102 is capable of connecting to external services of the hospital system (for example, a server that manages the patient's electronic medical record). Once the communication between the doctor and the patient is complete, the telemedicine consultation ends (S306).

[0036] Here, in S304, when the physician begins the consultation, the streaming condition change unit 401 in the physician-side information processing device 102 of the telemedicine support system starts operating. This streaming condition change unit 401 is also a function of the medical application executed by the CPU 206 in the physician-side information processing device 102. It is assumed that the streaming condition change unit 401 can be notified of the start of a consultation by having the physician press a consultation start button on the application of the telemedicine support system, but it is not limited to this. For example, the streaming condition change unit 401 may start operating from the time of the start of the telemedicine (S301).

[0037] Referring to Figure 4, the streaming condition change unit 401 will be explained. As mentioned above, the ROM 209 of the physician-side information processing device 102 contains various control programs that are executed within the information processing device, and this includes the control program for the streaming condition change unit. Each change process is realized by the CPU 206 controlling various parts of the physician-side information processing device 102 according to the input signals and programs.

[0038] The streaming condition modification unit 401 includes a medical treatment content acquisition unit 402, a streaming condition determination unit 403, and a streaming condition reflection unit 404.

[0039] The medical information acquisition unit 402 acquires medical information from the operator's instructions for the keyboard 204 or PD205. However, it may also acquire medical information from the information used when making a remote consultation reservation from a server of the in-house system (not shown), or from the patient's electronic medical record in the in-house system, and the source of the medical information is not particularly limited.

[0040] The streaming condition determination unit 403 determines the streaming conditions based on the medical information acquired by the medical information acquisition unit 402. Figure 5 shows an example of a table that the streaming condition determination unit 403 refers to when determining the streaming conditions. This table is assumed to be stored on the HDD 207, but the storage location may be a suitable server in the hospital system. Here, if the medical information acquired by the medical information acquisition unit 402 is "visual examination, walking," the streaming condition determination unit 403 determines "frame rate priority" because the main focus is on confirming the patient's posture while walking. Also, if the medical information acquired by the medical information acquisition unit 402 is "visual examination, magnification of the affected area," it determines "resolution priority." Note that the table shown in Figure 5 is merely an example, and the contents are not limited to this. Furthermore, the system should be structured to use wording that matches the information source of the medical information acquired by the medical information acquisition unit 402. In addition, the streaming condition determination unit 403 may show the physician the correspondence table shown in Figure 5, allowing the physician to select or edit the conditions. In this case, if a medical treatment does not exist, the physician may add it.

[0041] Once the streaming conditions are determined by the streaming condition determination unit 403, the streaming condition reflection unit 404 reflects the conditions as settings in the telemedicine support system. Specifically, the streaming condition determination unit 403 sends a request command to the patient-side information processing device 104 to transmit video according to the determined conditions. The patient-side information processing device 104 sets the imaging resolution and imaging rate and transmits the information to the physician-side information processing device 102 according to the conditions indicated in the received request command (normal state, resolution priority, or frame rate priority).

[0042] For example, suppose that at the start of telemedicine, the streaming conditions (normal state) were a resolution of 1280x720 and a frame rate of 30fps. If the streaming condition is now set to "frame rate priority," the streaming condition reflection unit 404 sends a request command to the patient-side information processing device 104 to set the resolution to 640x480 and the frame rate to 60fps. On the other hand, if the condition is "resolution priority," the streaming condition reflection unit 404 sends a request command to the patient-side information processing device 104 to set the resolution to 1920x1080 and the frame rate to 15fps.

[0043] The above settings are just examples and are not limited to these values. They may be changed depending on the performance of the physician's information device 102 and user device 101, the patient's information processing device 104 and user device 101, or the bandwidth of the communication path via the network 103.

[0044] As explained above, according to this embodiment, even if the specifications of the PC being used are insufficient or the bandwidth is insufficient, the streaming condition changing unit 401 can change the streaming conditions according to the medical treatment content, thereby prioritizing the avoidance of degradation of information necessary for diagnosis and reducing the risk of being unable to make an accurate diagnosis.

[0045] Furthermore, the streaming condition modification unit 401 shown in Figure 4 is implemented by processing performed by the CPU 206. Therefore, the medical treatment content acquisition unit 402, streaming condition determination unit 403, and streaming condition reflection unit 404 in the same figure can also be viewed as flowcharts representing the processing procedures executed by the CPU 206.

[0046] In the above example, the streaming condition change unit 401 was described as being implemented by the CPU 206 in the physician-side information processing device 102 executing a program, but it may also be implemented by the CPU 206 in the patient-side information processing device 104 executing a program. In the latter case, during telemedicine, when the physician selects a medical item of type, for example, "visual examination, walking," from the medical menu (not shown) displayed on the physician-side information processing device 102, the CPU 206 of the physician-side information processing device 102 transmits information indicating the selected medical item "visual examination, walking" to the patient-side information processing device 104. As a result, the medical content acquisition unit 402 in the streaming condition change unit 401 of the patient-side information processing device 104 successfully acquires "visual examination, walking" as a medical item. Subsequently, the streaming condition determination unit 403 refers to a table (Figure 5) stored on the HDD of the patient-side information processing device 104 to determine the streaming conditions, and the streaming condition reflection unit 404 sets the imaging resolution and imaging rate according to the determined conditions. The patient-side information processing device 104 then transmits the acquired video data to the physician-side information processing device 102 according to the set conditions.

[0047] The streaming condition determination unit 403 may also determine the streaming conditions by taking into account the transmission bandwidth (transmission speed) of the network 103 in use. For example, when the CPU 206 of the physician-side information processing device 102 is communicating with the patient-side information processing device 104, it determines or detects the bandwidth of the network 103 transmission between the two. The determination / detection of the network 103 transmission bandwidth may be done using known techniques. For example, it can be determined by measuring the time from when the physician-side information processing device 102 sends a command of an appropriate size to the patient-side information processing device 104 until it receives an acknowledgment from the patient-side information processing device 104 that the command has been received. Now, if the transmission bandwidth of the network 103 is B (bits / sec), the frame rate of the video is F, the amount of data in one frame of the image is M, and the average compression ratio when using an available codec is R, then the following equation should be satisfied in order to transfer data without difficulty. F × {M × R} ≤ B - α Here, "α" is a positive value and is a correction factor to allow for fluctuations in network traffic.

[0048] For example, if "frame rate priority" is selected as the streaming condition, and the target frame rate is set to 30fps, the data size M of one frame of an image should satisfy the following equation, which can be obtained by rearranging the above equation. M ≤ {B - α} / {30 × R} If we assume that the vertical number of pixels in a single frame of a color image (an image with three components: R, G, and B) is H, the aspect ratio is A (if the aspect ratio is 16:9, then A is 16 / 9, so the horizontal number of pixels is A × H), and each color component is 8 bits, then the amount of data M per frame can be expressed by the following formula. M = H × (H × A) × 3 × 8 Therefore, H×(H×A)×3×8≦{B-α} / {30×R} By finding the maximum value H that satisfies the given conditions and capturing an image at a resolution corresponding to this value, it is possible to capture an image at a resolution that minimizes image quality degradation while prioritizing frame rate. Similarly, the maximum frame rate can be set for "resolution priority" as well.

[0049] Through the above processing, even if the specifications of the PC being used are insufficient or the bandwidth is insufficient, the streaming condition modification unit 401 can change the streaming conditions according to the medical treatment content, thereby prioritizing the avoidance of degradation of information necessary for diagnosis and reducing the risk of being unable to make an accurate diagnosis.

[0050] [Second Embodiment] In addition to the first embodiment described above, a second embodiment will be described in which the streaming conditions are changed by detecting the patient's condition during remote medical consultation. The system configuration is shown in Figure 1, and the information processing devices and user equipment at each location are shown in Figures 2(a) and (b). Their descriptions will be omitted, and the differences will be explained.

[0051] Figure 6 is a diagram showing the configuration of the streaming condition change unit 601 in this second embodiment. In this second embodiment as well, the streaming condition change unit 601 will be described as being realized by the CPU 206 of the physician-side information processing device 102 executing an application program stored in the ROM 208 or loaded from the HDD into the RAM 209.

[0052] The streaming condition change unit 601 includes a medical treatment content acquisition unit 602, a streaming condition determination unit 603, a streaming condition reflection unit 604, and a patient status detection unit 605.

[0053] The patient status detection unit 605 detects the user's status by analyzing the video (video captured by the user device 105) received from the patient-side information processing device 104 in real time. For example, it analyzes the patient's movements using a machine learning model to detect whether the patient's movements are fast or not. A predetermined threshold is set in advance to determine whether the patient's movements are fast or not. For example, to detect whether a patient is bedridden or not, a weight sensor or temperature sensor can be placed on the patient's bed, and the patient-side information processing device 104 can make a judgment using the threshold based on the sensor values. These methods are not limited to these. Note that the processes for detecting the patient's movements and detecting whether they are bedridden used here are general technologies, and therefore a detailed explanation is omitted here. The patient status detection unit 605 notifies the streaming condition determination unit 603 of the detected results.

[0054] The streaming condition determination unit 603 determines the streaming conditions based on the information notified by the patient information detection unit 605. For example, if the streaming condition determination unit 603 is notified by the patient information detection unit 605 that the patient is moving quickly, it will prioritize the frame rate. For example, if the streaming condition determination unit 603 is notified by the patient information detection unit 605 that the patient is bedridden, it will prioritize the resolution.

[0055] The streaming condition determination unit 603 can make these determinations by creating and maintaining a correspondence table in advance between the notification content notified by the patient information detection unit 605 and the streaming conditions, but the method is not limited to this.

[0056] Here, the detection method and content of the patient status detection unit 605 are not limited to those described above. For example, it may detect whether the patient's eyes are visible and, if it notifies the streaming condition determination unit 603 that the eyes are visible, it may prioritize the frame rate. For example, when diagnosing skin, it may detect whether the patient's inflamed area is visible and, if it notifies the streaming condition determination unit 603 that the patient's inflamed area is visible, it may prioritize the resolution. The control processing required for such detection is assumed to use general techniques and will not be explained here.

[0057] Furthermore, the function of the patient information detection unit 605 may be set to OFF or ON by the user. If the function of the patient information detection unit 605 is OFF, the behavior will be the same as in the first embodiment.

[0058] Through the above process, the streaming condition modification unit 601 can change the streaming conditions not only according to the medical treatment content but also according to the patient's condition.

[0059] In the above description, the streaming condition change unit 601 was explained as a function of the physician-side information processing unit 102, but it may also be implemented in the patient-side information processing unit 104. In this case, the CPU 206 of the patient-side information processing unit 104 will execute the application program stored in the ROM 208 or loaded from the HDD into the RAM 209.

[0060] [Third Embodiment] In addition to the first and second embodiments described above, a third embodiment will be described in which the streaming conditions are changed by detecting the physician's operating status during remote medical consultation. The system configuration, the information processing device at each location, and the user equipment configuration will be the same as in the first embodiment, and their description will be omitted.

[0061] Figure 7 shows the streaming condition changing unit 701 in this third embodiment. In this third embodiment as well, the streaming condition changing unit 701 is described as being realized by the CPU 206 of the physician-side information processing device 102 executing an application program stored in the ROM 208 or loaded from the HDD into the RAM 209.

[0062] The streaming condition change unit 701 includes a medical treatment content acquisition unit 702, a streaming condition determination unit 703, a streaming condition reflection unit 704, a patient status detection unit 705, and an operation status detection unit 706.

[0063] The operation status detection unit 706 detects operations performed by a physician. Examples of operations performed by physicians are shown below.

[0064] If a physician manually changes the streaming conditions, it is assumed that the conditions have been changed to those the physician wishes to view, and therefore those settings should take precedence. For this reason, the operation status detection unit 706 detects the physician's operation of the PC (including operation of the telemedicine support system). In this case, the streaming condition determination unit 703 notifies the streaming condition reflection unit 704 of the notified content as is, and the streaming condition reflection unit 704 also reflects the settings. The operation of the streaming condition reflection unit 704 is as described in the first embodiment.

[0065] Furthermore, physicians may use modalities appropriate to the diagnosis for diagnostic purposes. As mentioned above, the telemedicine support system can be used not only when a physician in the hospital and a patient in their home or nursing home are connected via the internet using a pre-prepared information processing device, but also in cases where a large hospital and a small hospital are connected via the internet, with the patient located at the small hospital. In the latter case, various cases are conceivable, such as using a dermoscopy device to enlarge the affected area, using thermography to check skin temperature, using an X-ray device to check the condition of a fracture, or using ultrasound to check for abdominal tenderness. Examples of modalities appropriate to the diagnosis are not limited to these. Here, the operation status detection unit 706 detects that these modalities have been used and notifies the streaming condition determination unit 703 of the results. The method by which the operation status detection unit 706 detects the use of modalities may be input by the physician, or the system may detect that a modality has been connected to the patient-side information processing device 104, but these are not the only methods. The streaming condition determination unit 703 determines conditions that match the modality. For example, as shown in Figure 8, the streaming condition determination unit 703 maintains a pre-created correspondence table between modalities and streaming conditions, and uses this to determine the conditions and notify the streaming condition reflection unit. Alternatively, for example, the patient-side information processing device 104 may detect the resolution and frame rate conditions of the streaming output by the modality, and the streaming condition reflection unit 704 may directly inherit these resolution and frame rate values, but this is not the only option. The streaming condition determination unit 703 prioritizes notifications from the operation status detection unit 706 over those from the medical content acquisition unit 702 and the patient information detection unit 705 when determining streaming conditions.

[0066] Through the above process, the streaming condition modification unit 701 can change the streaming conditions according to the medical treatment content, patient condition, and the physician's operating status.

[0067] In the above explanation, the streaming condition change unit 701 was described as a function of the physician-side information processing unit 102. However, it will now be explained as being implemented by the CPU 206 in the patient-side information processing unit 104 executing an application program stored in the ROM 208 or loaded from the HDD into the RAM 209. In this case, the operation state detection unit 706 in the patient-side information processing unit 104 only needs to detect the operation state from the received physician's video.

[0068] [Fourth Embodiment] In addition to the first to third embodiments described above, a fourth embodiment will be described as an example of reducing the amount of streaming data when PC or network conditions are poor. The system configuration, the information processing device at each location, and the user equipment configuration will be the same as in the first embodiment, and their description will be omitted.

[0069] Figure 9 shows the streaming condition changing unit 901 in this fourth embodiment. In this fourth embodiment as well, the streaming condition changing unit 701 is described as being realized by the CPU 206 of the physician-side information processing device 102 executing an application program stored in the ROM 208 or loaded from the HDD into the RAM 209.

[0070] The streaming condition modification unit 901 includes a medical treatment content acquisition unit 902, a streaming condition determination unit 903, a streaming condition reflection unit 904, a patient status detection unit 905, an operation status detection unit 906, a communication status detection unit 907, and a streaming data reduction unit 908.

[0071] The communication status detection unit 907 detects whether the specifications of the information processing device on which the streaming condition change unit 901 is executed, the network bandwidth, etc., are in a state where there are no problems in sending and receiving the stream set by the stream condition reflection unit 904, or it determines whether the communication status is good or bad. The method of detection by the communication status detection unit 907 is as follows: In remote diagnosis, doctors and patients communicate by distributing video of themselves, but a UI (User Interface) button is provided for the doctor or patient to press if the video is distorted based on visual judgment. If the button is pressed, the communication status detection unit 907 determines that the communication status is bad. Conversely, if the button is not pressed, the communication status detection unit 907 determines that the communication status is good. The detection method of the communication status detection unit 907 is not limited to this, and for example, it may also be determined using the specifications of the information processing device. Specifically, this is a method using the link speed of the network adapter. The communication status detection unit 907 obtains values ​​such as 1000Mbps transmission and 1000Mbps reception from the information processing device, compares them with the amount of streaming data calculated from the conditions determined by the streaming condition reflection unit 904, and determines that the communication status is good if the network adapter link speed is greater, and poor if it is smaller. Another method is to detect the network bandwidth. As mentioned above, when telemedicine begins, the telemedicine support system communicates by connecting to the internet and distributing video via an application. At that time, the communication status detection unit 907 obtains the amount of data actually transmitted and received in real time, compares it with the amount of streaming data calculated from the conditions determined by the streaming condition reflection unit 904, and determines that the communication status is good if the amount of data actually transmitted and received is greater than a threshold, and poor if it is below the threshold. Note that the method for obtaining the amount of data actually transmitted and received may be to obtain values ​​from an external application that detects general communication volume. Thus, there are multiple detection methods for the communication status detection unit 907, but it is sufficient if detection can be performed by any of these methods. The communication status detection unit 907 notifies the streaming data reduction unit 908 of the detection result, indicating whether the communication status is good or bad.

[0072] The streaming data reduction unit 908 performs processing only when the communication status detection unit 907 indicates that the communication status is poor. An example of data reduction by the streaming data reduction unit 908 is shown below.

[0073] If the medical information obtained by the medical information acquisition unit 902 is a visual examination or walking, the streaming condition determination unit 903 and the streaming condition reflection unit 904 set the settings prioritizing the frame rate. In such cases, it is determined that color information does not affect the diagnosis and is converted to grayscale. If the stream data is RGB, it is converted to grayscale, or only the R component, G component, or B component is transmitted as stream data. Alternatively, if the stream data is YUV or YCbCr, only the Y component is transmitted as stream data. As a result, compared to a normal color image (3 components), only one component is needed, so the amount of data can be simply reduced to 1 / 3.

[0074] If the medical information obtained by the medical information acquisition unit 902 is a visual examination or skin examination, the streaming condition determination unit 903 and the streaming condition reflection unit 904 set the settings prioritizing resolution. In such cases, the system determines that the affected area, such as the inflamed area, is important, and reduces the data by extracting an ROI (Region of Interest) before transmitting the stream data. The ROI should be specified by the physician or patient, or it may be automatically detected by the system. Note that the method of determining the region during data reduction is not limited to this, and for example, the coordinates of the area in focus from the camera may be obtained and the surrounding area may be extracted.

[0075] If the operation status detection unit 906 detects that the X-ray device is in use, the streaming condition determination unit 903 and the streaming condition reflection unit 904 set the settings with resolution priority. In this case, it is determined that color information is important due to the characteristics of the X-ray device modality, and that it does not affect the diagnosis, so it is converted to grayscale. The conversion to grayscale is as described above.

[0076] If the operation status detection unit 906 detects that the ultrasound device is in use, the streaming condition determination unit 903 and the streaming condition reflection unit 904 set the settings prioritizing the frame rate. In this case as well, considering the characteristics of the ultrasound device modality, it is determined that color information is important and does not affect the diagnosis, so it is converted to grayscale. The conversion to grayscale is as described above.

[0077] Although an example of the processing by the streaming data reduction unit 908 has been described above, any data reduction method that allows a physician to make a diagnosis is acceptable, and the reduction method does not limit the present invention.

[0078] If the streaming data reduction unit 908 receives a message from the communication status detection unit 907 indicating that the communication status is good, it will revert to a state where it does not reduce data.

[0079] Through the above process, streaming conditions can be changed according to the detection results of the communication status, in addition to the medical treatment content, patient condition, and physician's operating status.

[0080] [Fifth Embodiment] In addition to the first to fourth embodiments, a fifth embodiment will be described that uses the previous settings. The system configuration, the information processing device at each location, and the user equipment configuration will be the same as in the first embodiment, and their description will be omitted.

[0081] Figure 10 shows the streaming condition changing unit 1001 of the fifth embodiment. In this fourth embodiment as well, the streaming condition changing unit 1001 will be described as being realized by the CPU 206 of the physician-side information processing device 102 executing an application program stored in the ROM 208 or loaded from the HDD into the RAM 209.

[0082] The streaming condition modification unit 1001 includes a medical treatment content acquisition unit 1002, a streaming condition determination unit 1003, a streaming condition reflection unit 1004, a patient status detection unit 1005, an operation status detection unit 1006, a communication status detection unit 1007, a streaming data reduction unit 1008, and a streaming condition history retention unit 1009.

[0083] The streaming condition history storage unit 1009 stores the streaming conditions determined by the streaming condition determination unit 1003, linked to the information of the user, who is a physician. The physician's information may be an ID and password used in an external system such as an electronic medical record, or it may be information that the physician enters specifically for this system.

[0084] The processing of the streaming condition history storage unit 1009 can be enabled or disabled. If the processing of the streaming condition history storage unit 1009 is enabled, the streaming condition determination unit 1003 retrieves the previous streaming conditions linked to the physician's information from the streaming condition history storage unit 1009. Based on this, the streaming condition determination unit 1003 determines the streaming conditions.

[0085] Furthermore, the data in the streaming condition history retention unit 1009 is not limited to this; it may also link streaming conditions and physician information with the results of the medical treatment content acquisition unit 1002. Additionally, the entire history, not just the previous value, may be retained, allowing the user to select from the available history.

[0086] Through the above process, the streaming condition change unit 1001 can create an environment equivalent to that when a diagnosis was previously made by reflecting the physician's past settings again using the streaming condition determination unit 1003 and the streaming condition history retention unit 1009.

[0087] [Sixth Embodiment] In addition to the first to fifth embodiments, a sixth embodiment will be described that includes an example of detecting special user operations. The system configuration, the information processing device at each location, and the user equipment configuration will be the same as in the first embodiment, and their description will be omitted.

[0088] The streaming condition changing unit in this sixth embodiment is the same as in Figure 10. In this sixth embodiment as well, the streaming condition changing unit 1001 will be described as being realized by the CPU 206 of the physician-side information processing device 102 executing an application program stored in the ROM 208 or loaded from the HDD into the RAM 209.

[0089] Figure 10 shows the streaming condition changing unit 1001 in this sixth embodiment. The operation of the streaming condition determination unit 1003 and the operation state detection unit 1006 will now be explained.

[0090] In the fourth embodiment described above, the operation status detection unit 1006 detected an operation by a physician, the streaming condition determination unit 703 notified the stream condition reflection unit 704 of the notified content, and the streaming condition reflection unit 704 also reflected the setting content. In this sixth embodiment, the operation status detection unit 1006 detects whether the physician has performed a ZOOM operation (enlarge or reduce operation) on the displayed streaming video. When the operation status detection unit 1006 detects an enlargement operation, it notifies the streaming condition determination unit 1003, and the streaming condition determination unit 1003 determines that the physician is looking at the affected area and prioritizes resolution. That is, when the physician performs an enlargement operation, the streaming condition determination unit 1003 transmits the enlargement request and resolution priority to the patient-side information processing device 104. The patient-side information processing device 104, in response to the received request, gives an enlargement instruction and a high-resolution imaging instruction to the user device 105 and transmits the obtained information to the physician-side information processing device 102.

[0091] Conversely, when the operation state detection unit 1006 detects a zoom operation, it notifies the streaming condition determination unit 1003, and the streaming state determination unit 1003 prioritizes the frame rate, assuming that the doctor is viewing the movement from an overhead perspective. Although the above describes an example of ZOOM operation, any operation performed by a doctor to obtain information necessary for diagnosis is acceptable, and the method and content of the operation do not limit the present invention.

[0092] In the above description, the streaming condition change unit 1001 was explained as a function of the physician-side information processing device 102, but it may also be a function of the patient-side information processing device 104. In the latter case, the physician-side information processing device 102 receives input for zoom operation from the physician and transmits information indicating the degree of enlargement or reduction as request information to the patient-side information processing device 104. The streaming condition change unit 1001 of the patient-side information processing device 104 controls the resolution and frame rate of the user device 105 based on the received request information and transmits the video acquired under that control to the physician-side information processing device 102.

[0093] Through the above process, the streaming condition determination unit 103 and the operation state detection unit 1006 can detect the physician's actions and change the streaming conditions.

[0094] [Seventh Embodiment] In addition to the first to sixth embodiments, a seventh embodiment is shown that presents an example in which the trigger for starting the operation of the streaming condition change unit is different. The system configuration, the information processing device at each location, and the user equipment configuration are the same as in the first embodiment, and their description is omitted.

[0095] Figure 11 shows the streaming condition changing unit 1101 in this seventh embodiment. In this seventh embodiment, the streaming condition changing unit 1101 is described as being realized by the CPU 206 of the physician-side information processing device 102 executing an application program stored in the ROM 208 or loaded from the HDD into the RAM 209.

[0096] The streaming condition change unit 1101 includes a medical treatment content acquisition unit 1102, a streaming condition determination unit 1103, a streaming condition reflection unit 1104, a patient status detection unit 1105, an operation status detection unit 1106, a communication status detection unit 1107, a streaming data reduction unit 1108, a streaming condition history retention unit 1109, and a condition change processing start detection unit 1110.

[0097] In the first embodiment described above, the streaming condition change unit 401 is described in an example where processing is started when a doctor presses a consultation start button provided on the application of the telemedicine support system, or in an example where the streaming condition change unit 401 is always operating.

[0098] In contrast, in this seventh embodiment, the condition change processing start detection unit 1110 detects whether or not to start processing from the medical treatment content acquisition unit 1102 onwards within the streaming condition change unit 1101. An example of the processing start detection process in the condition change processing start detection unit 1110 is shown below.

[0099] For example, if the system recognizes the doctor's speech and detects keywords such as "Shall we examine you?", "I'll examine the affected area," or "Let's begin the diagnosis," the condition change processing start detection unit 1110 detects that the subsequent processing should begin and starts processing with the streaming condition change unit 1101. Note that the keywords for speech recognition are not limited to these, and processing can be performed with any keyword indicating the start of a medical consultation. Also, for example, the condition change processing start detection unit 1110 detects whether the electronic medical record window on the doctor's information processing device 102 is active or not. If the condition change processing start detection unit 1110 detects that the state has changed from active to inactive after the start of remote medical consultation, it determines that the doctor is in a diagnosis state rather than an input state and starts processing with the streaming condition change unit 1101. The method for determining the active state of the window is not limited to this, and for example, it may be detected that the application used to distribute video in the remote medical consultation support system has become active.

[0100] Through the above process, the condition change processing start detection unit 1110 detects the start timing of the processing by the streaming condition change unit 1101, thereby enabling the processing of the present invention to start at an appropriate timing.

[0101] In the above description, the streaming condition change unit 1101 was explained as a function of the physician-side information processing unit 102, but it may also be a function of the patient-side information processing unit 104. In the latter case, the CPU 206 of the patient-side information processing unit 104 only needs to recognize the physician's voice received from the physician-side information processing unit 102. The process from the recognition process onward is the same as described above.

[0102] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0103] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]

[0104] 101, 105...User equipment, 102...Doctor-side information processing device, 104...Patient-side information processing device, 103...Network

Claims

1. An information processing device that can be used for doctors and patients to send and receive audio and video in a telemedicine support system, A means of obtaining information indicating the type of medical treatment, A determination means for determining streaming conditions to be transmitted from the patient's information processing device to the physician's information processing device according to the type of medical treatment acquired by the acquisition means, A modification means for changing the streaming to conform to the streaming conditions determined by the determination means. An information processing device characterized by having the following features.

2. The information processing apparatus according to claim 1, characterized in that the streaming determined by the determination means includes prioritizing video resolution and prioritizing video frame rate.

3. The information processing device according to claim 1 or 2, characterized in that the acquisition means acquires the type of medical treatment from an electronic medical record, a telemedicine appointment slip, and a doctor's operation.

4. It further includes a patient status detection means for detecting the patient's condition, The determination means further determines the streaming conditions according to the patient's condition detected by the patient condition detection means. The information processing apparatus according to any one of claims 1 to 3.

5. A communication state detection means detects the quality of the communication state by comparing the transmission speed and threshold of the communication between the physician's information processing device and the patient's information processing device, If the communication status detected by the communication status detection means is found to be negative, a reduction means is provided to reduce the amount of data transmitted from the patient's information processing device to the physician's information processing device. The information processing apparatus according to any one of claims 1 to 4, further comprising the above.

6. The reduction means converts the video transmitted from the patient's information processing device to the physician's information processing device into single-component video data. The information processing apparatus according to feature 5.

7. The information processing apparatus according to any one of claims 1 to 6, further comprising a start detection means that provides a trigger for the start of the change processing by the change means.

8. The information processing device according to claim 7, wherein the start detection means uses the detection of a predetermined voice message indicating the start of medical treatment by a physician as a trigger to start the modification process by the modification means.

9. A control method for an information processing device that can be used by doctors and patients to send and receive audio and video in a remote medical support system, The acquisition process involves obtaining information indicating the type of medical treatment, A determination step, which determines the streaming conditions to be transmitted from the patient's information processing device to the physician's information processing device according to the type of medical treatment acquired in the acquisition step, A modification step to change the streaming to conform to the streaming conditions determined in the determination step. A control method performed by an information processing device characterized by having the following features.

10. A program that, when read and executed by a computer, causes the computer to perform each step of the control method described in claim 9.

11. A telemedicine support system in which a physician's information processing device and a patient's information processing device send and receive video and audio to each other for the purpose of telemedicine, A means of obtaining information indicating the type of medical treatment, A determination means for determining streaming conditions to be transmitted from the patient-side information processing device to the physician-side information processing device according to the type of medical treatment acquired by the acquisition means, A modification means for changing the streaming to conform to the streaming conditions determined by the determination means. A telemedicine system characterized by having the following features.

Images