Information presentation system and information presentation program
The information presentation system addresses the challenge of guiding imaging adjustments in remote nursing by using a terminal-based method to correct video capture areas, ensuring precise and intuitive image capture without symbols, thus reducing cognitive load.
Patent Information
- Application Number
- PCT/JP2025/016775
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-05-07
- Publication Date
- 2026-11-12
AI Technical Summary
Existing remote nursing care methods struggle to accurately guide patients in adjusting the position and angle of imaging devices for fine body parts due to the difficulty in conveying subtle movements verbally, leading to potential discrepancies and increased cognitive load from using symbols like signs and icons.
An information presentation system and program that guides the imaging area without using symbols, utilizing a first terminal with a camera and a second terminal to transmit and correct video capture areas based on instructions, adjusting the central region of the observed video to align with the nurse's expectations.
Effectively guides the patient's shooting area to capture images as intended by the nurse, reducing cognitive load and ensuring accurate observation without the need for symbol interpretation.
Smart Images

Figure JP2025016775_12112026_PF_FP_ABST
Abstract
Description
Information presentation system and information presentation program
[0001] The present invention relates to an information presentation system and an information presentation program.
[0002] Against the backdrop of a rapidly growing number of care recipients amid a super-aging society, there are limits to implementing nursing care in medical facilities such as hospitals where nurses and care recipients are always in face-to-face contact. Based on this, the Ministry of Health, Labour and Welfare has promoted home-based nursing care centered on visiting nursing care under the concept of a "Community Comprehensive Care System". This initiative aims to reduce the burden by having the local community handle tasks other than nursing care, but the shortage of nursing care workers has become a barrier to expanding the implementation scope and opportunities. Therefore, in addition to conventional embodiments premised on face-to-face interaction, there is an urgent need to establish new remotely implemented embodiments.
[0003] Accordingly, with the aim of implementing nursing care for a large number of care recipients with a small number of nurses, nursing care via remote communication has been trialed. In this approach, a nurse at a hospital conducts nursing observation while interacting remotely with a care recipient at a separate location such as their home via a video call system. The nurse interviews the care recipient about their condition through the interaction, and visually observes the state of the care recipient's facial image displayed on the video.
[0004] In such remote nursing care, for the purpose of improving the quality of visual observation, when visually observing fine body parts such as the "nape of the neck" or "fingertips", it is necessary for the nurse to instruct the care recipient to move so that the corresponding observation site can be properly captured. When prompting the care recipient to perform imaging at the position, angle and size expected by the nurse, it is required that the nurse accurately describe the accompanying fine movements and ensure that the care recipient understands the instructions. However, it is difficult to convey these fine movements solely verbally. Therefore, creating a mechanism to support the transmission of these fine movements that are difficult to verbalize has become an issue.
[0005] Therefore, an information presentation method that supports the transmission of subtle movements using non-verbal visual information has been proposed, as disclosed in Non-Patent Document 1. In this method, the body movements required to achieve the desired posture for the patient are presented and communicated using visual information called "signs" or "icons." Signs indicate the desired whole-body posture using symbols such as pictograms or reference lines. Icons indicate the positioning of the imaging device relative to the body part being photographed using symbols such as arrows. This makes it possible to intuitively express and understand subtle movements that are difficult to verbalize, contributing to the implementation of high-quality visual observation.
[0006] Soichiro Uchida et al., "An Interface Design Utilizing Nonverbal Information to Support Smooth Visual Observation in Remote Nursing," [Online], [Retrieved April 18, 2025], Internet <URL: https: / / ken.ieice.org / ken / paper / 20240126BcaY / >
[0007] In the method disclosed in Non-Patent Document 1, icons (symbols such as arrows) are used to guide the position and angle of the imaging device during imaging, enabling intuitive operation. On the other hand, the range and speed of moving the imaging device based on these symbols are left to the interpretation of the patient.
[0008] Therefore, especially when making fine adjustments to the position of the imaging device, a discrepancy may arise between the distance the nurse "intended to move" and the distance the patient "actually moved." This can result in the area the nurse expects not to be properly photographed, potentially hindering the observation process.
[0009] Furthermore, in this method, visual symbols with different meanings, such as signs and icons, are simultaneously and continuously presented on the observation video. Therefore, if the patient is unfamiliar with the task, there is a risk that the cognitive load during the action may increase.
[0010] Therefore, there is a need for a technology that guides the shooting area without using symbols.
[0011] This invention has been made in view of the above circumstances, and its purpose is to provide an information presentation system and information presentation program that guides the shooting area without using symbols.
[0012] One aspect of the present invention is an information presentation system. The information presentation system includes a first terminal and a second terminal. The first terminal includes a camera. The second terminal receives captured video from the first terminal and transmits presentation information to the first terminal. The second terminal performs the following processes: displaying the entire area of the captured video, receiving instructions for correcting the area of the captured video, and transmitting the correction instructions to the first terminal. The first terminal performs the following processes: displaying a portion of the captured video, receiving correction instructions, and defining the portion of the video based on the correction instructions.
[0013] One aspect of the present invention is an information presentation program. The information presentation program causes a computer having a processor to execute at least a part of the functions of at least one of the first terminal and the second terminal described above.
[0014] According to the present invention, an information presentation system and information presentation program are provided that guide the imaging area without using symbols.
[0015] Figure 1 is a block diagram showing the functional configuration of the information presentation system according to the embodiment. Figure 2 is a block diagram showing the configuration of a computer capable of configuring a remote nurse terminal and a patient / observation terminal, respectively. Figure 3 is a flowchart showing an example of the flow of visual observation of a patient's body in remote nursing using the information presentation system according to the embodiment. Figure 4 is a diagram showing the observation video, the display video on the nurse terminal of the remote nurse terminal, and the display videos on the observation terminal and patient terminal of the patient / observation terminal. Figure 5 is a schematic diagram showing an example of communicating observation areas using signs. Figure 6 is a schematic diagram showing an example of inputting correction instructions on the correction instruction information input screen. Figure 7 is a schematic diagram showing another example of inputting correction instructions on the correction instruction information input screen. Figure 8 is a schematic diagram showing an example of changing the drawing range of the central area. Figure 9 is a schematic diagram showing another example of changing the drawing range of the central area.
[0016] Embodiments of the present invention will be described below with reference to the drawings. In these embodiments, the visual observation work in remote nursing will be used as an example. Here, we envision its use in work that involves a nurse and a patient in a remote location communicating via a video call system, and the patient moving their body to a specific posture in accordance with the nurse's instructions (for example, work that facilitates the visual observation of the patient's body).
[0017] (Information Presentation System) First, the functional configuration of the information presentation system 100 according to the embodiment will be described with reference to Figure 1. Figure 1 is a block diagram showing an example of the functional configuration of the information presentation system 100 according to the embodiment. For convenience, only the main blocks in the description of the information presentation system 100 are shown in Figure 1, and the illustration of blocks that are common in video call systems is omitted.
[0018] The information presentation system 100 includes a remote nurse terminal 200 and a patient / observation terminal 300. The remote nurse terminal 200 and the patient / observation terminal 300 are capable of video calls via a network such as the internet. The remote nurse terminal 200 and the patient / observation terminal 300 each have functions such as camera capture, call, and communication.
[0019] The remote nurse terminal 200 is used by the nurse. The patient / observation terminal 300 is used by the patient. For example, the patient uses the patient / observation terminal 300 to receive instructions from the nurse, takes a picture of a part of their body (observation area) according to the nurse's instructions, and sends it to the remote nurse terminal 200. The nurse uses the remote nurse terminal 200 to receive the captured video (observation video), views the observation video, creates presentation information, creates presentation information for the patient, and sends it to the patient / observation terminal 300.
[0020] The remote nurse terminal 200 is an information display device that presents information to the patient / observation terminal 300. The patient / observation terminal 300 is an information display device that receives and displays the presented information from the remote nurse terminal 200.
[0021] For example, the remote nurse terminal 200 and the patient / observation terminal 300 are each composed of computers. These computers can be, for example, PCs (personal computers), tablets, smartphones, etc.
[0022] The remote nurse terminal 200 and the patient / observation terminal 300 are not limited to these configuration examples, and may have other configurations as long as they have a display screen and an input UI and can communicate via a network. The display screen may be provided as a native application of each OS or as a web application.
[0023] The remote nurse terminal 200 and the patient / observation terminal 300 do not necessarily have to be composed of a single piece of hardware. That is, the remote nurse terminal 200 and the patient / observation terminal 300 may each be composed of a single piece of hardware or multiple pieces of hardware.
[0024] For convenience, the remote nurse terminal 200 is assumed to include a PC-based nurse terminal, and the patient / observation terminal 300 is assumed to include a tablet-based patient terminal and a smartphone-based observation terminal. For example, the patient terminal is mainly used for displaying information, and the observation terminal is mainly used for photographing the patient. In other words, the observation terminal can essentially be considered a photographic device.
[0025] The observation terminal does not necessarily have to be a separate device such as a smartphone; it may also be a standalone camera connected to the patient's terminal via wired or wireless connection. In that case, the video feed from the camera, which serves as a substitute for the observation terminal, can always be viewed on the patient's terminal.
[0026] The camera can be a small camera such as a webcam or endoscope. Observation devices such as smartphones have drawbacks such as being difficult to insert into narrow spaces like the inside of the mouth and being heavy and difficult to handle, but small cameras such as webcams or endoscopes can mitigate these problems.
[0027] For convenience, in the following explanation, nurses will be described as performing tasks such as checking observation images and presenting various information through their nursing terminals, and patients will be described as performing tasks such as taking pictures of the observation area and checking the presented information through their patient terminals and observation terminals.
[0028] In other words, the nurse terminal is described as if it were the UI of the remote nurse terminal 200, and the patient terminal and observation terminal are described as if they were the UI of the patient / observation terminal 300.
[0029] These are assumptions for the sake of explanation; for example, the nurse terminal constitutes the remote nurse terminal 200, and the patient terminal and observation terminal constitute the patient / observation terminal 300.
[0030] (Remote Nurse Terminal) First, the remote nurse terminal 200 will be described. The remote nurse terminal 200 has an observation video receiving unit 211, an observation video rendering unit 212, a correction instruction information input unit 221, a correction instruction information transmission unit 222, a sign information selection unit 231, and a sign information transmission unit 232.
[0031] The observation video receiving unit 211 receives observation videos transmitted from the patient / observation terminal 300.
[0032] The observation video rendering unit 212 renders the observation video received by the observation video receiving unit 211 on the display screen.
[0033] The correction instruction information input unit 221 receives correction instruction information for the imaging area entered by the nurse.
[0034] The correction instruction information transmission unit 222 transmits the correction instruction information received by the correction instruction information input unit 221 to the patient / observation terminal 300.
[0035] The signature information selection unit 231 allows nurses to select signature information via the input UI and accepts the selection of signature information.
[0036] The signature information transmission unit 232 transmits the signature information selected by the signature information selection unit 231 to the patient / observation terminal 300.
[0037] (Care Recipient / Observation Terminal) Next, the care recipient / observation terminal 300 will be described. The care recipient / observation terminal 300 includes an observed video acquisition unit 311, an observed video transmission unit 312, a correction instruction information receiving unit 321, a central region defining unit 322, a central region correction unit 323, an observed video rendering unit 324, a sign information receiving unit 331, a sign information correction unit 332, and a sign information rendering unit 333.
[0038] The observed video acquisition unit 311 acquires an observed video captured by the observation terminal.
[0039] The observed video transmission unit 312 transmits the observed video acquired by the observed video acquisition unit 311 to the remote nurse terminal 200.
[0040] The correction instruction information receiving unit 321 receives correction instruction information transmitted from the remote nurse terminal 200.
[0041] The central region defining unit 322 defines a central region for the entire region of the observed video, based on the correction instruction information received by the correction instruction information receiving unit 321.
[0042] The central region correction unit 323 performs correction on the reference central region based on initial settings for the observed video acquired by the observed video acquisition unit 311, based on the central region defined by the central region defining unit 322. The correction includes correction of the position and size of the central region relative to the entire region of the observed video.
[0043] The reference central region is set to a predetermined position and a predetermined size relative to the entire region of the observed video based on initial settings. For example, the predetermined position is a position where the center coincides with the center of the entire region of the observed video. The predetermined size is a size of a predetermined ratio that is smaller than the entire region of the observed video.
[0044] In other words, it can also be said that the central region correction unit 323 applies the setting of the central region for the entire region defined by the central region defining unit 322 to the observed video acquired by the observed video acquisition unit 311.
[0045] The observation video rendering unit 324 renders the video of the set central region in the observation video acquired by the observation video acquisition unit 311 on the display screens of the care recipient terminal and the observation terminal, based on the central region corrected by the central region correction unit 323.
[0046] The sign information receiving unit 331 receives the sign information transmitted from the remote nurse terminal 200.
[0047] The sign information correction unit 332 corrects the sign information received by the sign information receiving unit 331, based on the central region defined by the central region defining unit 322. The correction includes correction of the display position and size of the sign information.
[0048] The sign information rendering unit 333 renders the sign information corrected by the sign information correction unit 332 on the display screens of the observation terminal and the care recipient terminal.
[0049] (Hardware Configuration of Remote Nurse Terminal, Care Recipient Terminal and Observation Terminal) Next, the hardware configuration of each of the remote nurse terminal 200 and the care recipient / observation terminal 300 will be described with reference to Figure 2. Figure 2 is a block diagram showing the configuration of a computer 400 that can constitute each of the remote nurse terminal 200 and the care recipient / observation terminal 300.
[0050] The computer 400 includes a processor 401, a ROM (Read Only Memory) 402, a RAM (Random Access Memory) 403, an auxiliary storage device 404, an output device 405, an input device 406, and a communication device 407.
[0051] The processor 401, ROM 402, RAM 403, auxiliary storage device 404, output device 405, input device 406, and communication device 407 are electrically connected to one another via a bus 408, and exchange data via the bus 408.
[0052] The processor 401 is composed of a general-purpose hardware processor, such as a CPU (Central Processing Unit) or a GPU (Graphical Processing Unit). The processor 401 controls the entire system, including the ROM 402, RAM 403, auxiliary storage device 404, output device 405, input device 406, and communication device 407.
[0053] ROM 402 is a non-volatile memory that constitutes part of the main memory. ROM 402 non-temporarily stores the startup program required when the processor 401 starts up. The processor 401 starts up by executing the program in ROM 402. ROM 402 is, for example, composed of EPROM (Erasable Programmable Read Only Memory) and stores various startup settings in addition to the startup program.
[0054] RAM 403 is a volatile memory that constitutes part of the main memory. RAM 403 temporarily stores the program necessary for processing by processor 401 and the data necessary for executing the program. By executing the program in RAM 403, processor 401 performs calculations on the data in RAM 403 and stores the calculation results in RAM 403.
[0055] The auxiliary storage device 404 consists of non-volatile memory such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). The auxiliary storage device 404 non-temporarily stores programs executed by the processor 401 and data necessary for program execution. The processor 401 reads the programs and data from the auxiliary storage device 404 into the RAM 403 and executes various functions by executing the programs.
[0056] The output device 405 is a device that outputs information in order to provide information to the user. The output device 405 includes a display device that displays characters, images, etc. The display device is, for example, a liquid crystal display, an organic EL display, a plasma display, etc. The output device 405 also includes an audio output device that outputs sound.
[0057] The input device 406 is a device for receiving input such as information and instructions from the user. The input device 406 includes a keyboard, a pointing device, etc. The pointing device includes a mouse, a trackpad, a touchscreen, etc. The input device 406 also includes sensors. The sensors include a camera, a position sensor, etc.
[0058] The output device 405 and the input device 406 may be composed of an input / output device having both functions. The input / output device may be composed of, for example, a touch panel display.
[0059] Furthermore, the output device 405 may also include devices that output information or data to the outside. For example, the output device 405 may include wired or wireless interfaces or transmitting devices.
[0060] Furthermore, the input device 406 may also include devices that take in information or data from an external source. For example, the input device 406 may include wired or wireless interfaces and receiving devices.
[0061] Furthermore, the input device 406 may also include a device that reads data from a computer-readable recording medium 500 on which data such as programs is stored non-temporarily. For example, the recording medium 500 includes disks such as flexible disks, optical disks (CD-ROM, CD-R, DVD-ROM, DVD-R, etc.), magneto-optical disks (MO, etc.), and semiconductor memory. The input device 406 includes drives and readers for these.
[0062] The communication device 407 is a device for communicating with external devices. The communication device 407 is configured to communicate with external devices by wire or wireless connection.
[0063] A program stored non-temporarily in the auxiliary storage device 404 is provided to the computer 400, for example, via a recording medium 500 that is readable by the computer 400 on which the program was stored non-temporarily. Such a recording medium 500 is called a non-temporarily computer-readable recording medium. Non-temporarily computer-readable recording media include disks such as flexible disks, optical disks (CD-ROM, CD-R, DVD-ROM, DVD-R, etc.), magneto-optical disks (MO, etc.), and semiconductor memory.
[0064] The program to be temporarily stored in the auxiliary storage device 404 is read into the auxiliary storage device 404 via the disk drive, which is the input device 406, if the recording medium 500 is a disk, and stored temporarily in the auxiliary storage device 404 via the connection port, which is the input device 406, if the recording medium 500 is semiconductor memory. Alternatively, the program may be stored on a server on a network, downloaded from the server, and temporarily stored in the auxiliary storage device 404.
[0065] The program stored non-temporarily in the auxiliary storage device 404 of the computer 400 that constitutes the remote nursing terminal 200 includes an information presentation program. The information presentation program is a program that causes the computer 400 that constitutes the remote nursing terminal 200 to execute at least a part of the functions of the remote nursing terminal 200. The information presentation program is also a program that causes the computer 400 that constitutes the patient / observation terminal 300 to execute at least a part of the functions of the patient / observation terminal 300.
[0066] When the computer 400 starts up, the processor 401 executes the program in the ROM 402 and loads the OS into the RAM 403 to start up. The processor 401 monitors instruction inputs and the connection of external devices under the control of the OS. The processor 401 also sets up a program area and a data area in the RAM 403 under the control of the OS. In response to an instruction input to start up a device (remote nursing terminal 200 or patient / observation terminal 300), the processor 401 loads a program (information presentation program) from the auxiliary storage device 404 into the program area of the RAM 403, and loads the data necessary for program execution from the auxiliary storage device 404 into the data area of the RAM 403. The processor 401 performs calculations on the data in the data area according to the program and writes the calculation results to the data area. Through this operation, the processor 401, RAM 403, auxiliary storage device 404, output device 405, input device 406, and communication device 407 work together to perform at least some of the functions of the device (remote nursing terminal 200 or patient / observation terminal 300).
[0067] (Example of using the information presentation system) Next, we will explain an example of using the information presentation system 100. Here, we will explain an example of its use in visual observation of a patient's body in telenursing. Referring to Figure 3, we will briefly explain the flow of visual observation in telenursing. Figure 3 is a flowchart showing an example of the flow of visual observation of a patient's body in telenursing using the information presentation system 100.
[0068] First, prior to visually observing the patient's body, a video call is established between the remote nurse terminal 200 of the information presentation system 100 and the patient / observation terminal 300. This allows the nurse and the patient to communicate with video.
[0069] When a nurse visually observes a patient's body, the nurse informs the patient of the areas to be observed (S10). Here, the areas to be observed are broadly defined to include not only the body parts the nurse wishes to observe, but also the posture the patient should adopt.
[0070] The patient receives information about the area to be observed and, for example, uses an observation terminal to photograph the area (S20). The patient moves the observation terminal to photograph the area to be observed while checking the observation image displayed on the terminal.
[0071] Observational video captured by the observation terminal is transmitted from the patient / observation terminal 300 to the remote nurse terminal 200. The nurse, for example, checks the observational video through the nurse terminal and, if the observational video does not meet the nurse's expectations, gives appropriate instructions for correcting the shooting area (S30).
[0072] Correction instructions for the imaging area are transmitted from the remote nurse terminal 200 to the patient / observation terminal 300. The patient / observation terminal 300 changes the drawing range of the central area of the observation image displayed on the patient terminal, for example, according to the correction instructions for the imaging area (S40).
[0073] The process of photographing the observation area (S20), giving instructions to correct the imaging area (S30), and changing the drawing range of the central area (S40) continues until the observation image meets the nurse's expectations.
[0074] Subsequently, if the nurse wishes to observe other body parts (S50: YES), they communicate the new areas to be observed (S10), and then the above actions (S20-S40) are repeated.
[0075] If the patient does not wish to be examined in other body parts (S50: NO), the nurse will end the visual examination of the patient's body.
[0076] (Details of visual observation of the patient's body) First, prior to the visual observation of the patient's body, a video call is established between the remote nurse terminal 200 of the information presentation system 100 and the patient / observation terminal 300. This allows the nurse and the patient to communicate with video.
[0077] For example, the patient starts taking pictures using the camera function of the observation terminal. While the photography may begin after the nurse has indicated the area to be observed, for the sake of explanation, we will assume it begins at this point.
[0078] The observation video acquisition unit 311 acquires observation videos captured by the observation terminal. The observation video rendering unit 324 renders the observation videos acquired by the observation video acquisition unit 311 on the display screen of the observation terminal. The observation video rendering unit 324 also renders the observation videos on the display screen of the patient's terminal.
[0079] The observation video transmission unit 312 transmits the observation video acquired by the observation video acquisition unit 311 to the remote nurse terminal 200. In the remote nurse terminal 200, the observation video receiving unit 211 receives the observation video transmitted from the patient / observation terminal 300. The observation video rendering unit 212 renders the observation video received by the observation video receiving unit 211 on the display screen of the nurse terminal.
[0080] The nurse terminal of the remote nurse terminal 200 displays the entire area of the observation video acquired by the observation video acquisition unit 311. In contrast, the observation terminal and patient terminal of the patient / observation terminal 300 display a central area that is narrower than the entire area of the observation video acquired by the observation video acquisition unit 311.
[0081] Referring to Figure 4, the displayed images on the remote nursing terminal 200 and the displayed images on the patient / observation terminal 300 will be explained. Figure 4 is a diagram showing the observation images, the displayed images on the remote nursing terminal 200, and the displayed images on the patient / observation terminal 300.
[0082] For each area of the observed image, a central area slightly smaller than the entire area is defined. The entire area of the observed image is the shooting area defined by the shooting function of the observation terminal. For example, in the initial settings, the central area is the area whose center coincides with the entire area and whose contour is similar. Various settings for the central area, such as its relative relationship to the entire area, are set in advance. Furthermore, the area between the entire area and the central area is referred to as the peripheral area for convenience.
[0083] The entire area of the observation video is displayed on the nurse terminal of the remote nurse terminal 200. That is, both the central and peripheral areas of the observation video are displayed on the nurse terminal of the remote nurse terminal 200. For example, the central area of the observation video is displayed along with a border or other indicator to help users recognize its boundaries. On the other hand, the entire area of the observation video is displayed on the observation terminal and the patient terminal of the patient / observation terminal 300.
[0084] The central region is defined by the central region defining unit 322. Furthermore, the central region is appropriately corrected by the central region correction unit 323. These will be described later.
[0085] (Communication of Observation Areas) When a nurse visually observes the patient's body, the nurse communicates the observation areas to the patient (S10). In one example, the communication of observation areas may be done verbally or through gestures and images via video call on the remote nurse terminal 200 or the patient / observation terminal 300. In another example, the communication of observation areas may be done using signs.
[0086] Here, with reference to Figure 5, an example of communicating observation areas using signs will be explained. Figure 5 is a schematic diagram illustrating an example of communicating observation areas using signs. In this example, silhouette-type signs are used.
[0087] If a nurse wishes to communicate the area being observed using signs, they instruct the remote nursing terminal 200 to perform the communication of the area being observed using signs via their nursing terminal.
[0088] Upon receiving instructions, the remote nurse terminal 200 displays a silhouette selection screen on the nurse terminal using the sign information selection unit 231. The silhouette selection screen is a UI screen for silhouette selection and includes multiple selection windows for silhouettes and a start button to instruct the transmission of sign information. Any of the multiple selection windows can be selected, for example, by clicking.
[0089] For example, the multiple selection windows correspond to the desired postures of the entire palm of the right hand, the area around the right wrist, and the entire back of the right hand, and silhouette images of the right hand corresponding to the desired postures are pre-registered.
[0090] The signature information selection unit 231, for example, when one of multiple selection windows is selected by clicking and the start button is pressed, acquires information about the selected selection window as signature information. The signature information is a silhouette image corresponding to the selected selection window.
[0091] The signature information transmission unit 232 transmits the signature information (silhouette image) acquired by the signature information selection unit 231 to the patient / observation terminal 300.
[0092] The patient / observation terminal 300 receives the signature information (silhouette image) via the signature information receiving unit 331.
[0093] The signature information correction unit 332 corrects the signature information (silhouette image) received by the signature information receiving unit 331 based on the central region defined by the central region defining unit 322.
[0094] The signature information drawing unit 333 draws the signature information (silhouette image) corrected by the signature information correction unit 332 on, for example, the display screen of an observation terminal.
[0095] As a result, the observation terminal's display screen shows sign information (silhouette images) superimposed on the subject being photographed (observation area). The sign information (silhouette images) displayed on the observation terminal represent the observation area in the posture desired by the nurse.
[0096] The patient moves the observation terminal and changes the posture of the area being observed so that the area being photographed (observation area) displayed on the terminal matches the sign information (silhouette image), and then takes a photograph. As a result, the observation area is photographed by the patient in the posture desired by the nurse.
[0097] Thus, in communicating observation areas using signs, icons are not used in conjunction, eliminating the burden of simultaneously interpreting symbols that have different meanings from the observation area. Therefore, the cognitive load on the patient is less likely to increase.
[0098] The sign information drawing unit 333 may also draw the sign information (silhouette image) on the display screen of the patient terminal, in addition to or instead of the observation terminal.
[0099] (Photography of the area to be observed) The patient receives information about the area to be observed and then photographs it (S20). For example, the patient moves the observation terminal while watching the observation image displayed on the terminal so that the area to be observed is within the shooting area. In the case of communication of the area to be observed using a silhouette sign, the patient moves the observation terminal and changes the posture of the area to be observed so that the area to be observed matches the silhouette.
[0100] Observational video captured by the observation terminal is transmitted from the patient / observation terminal 300 to the remote nurse terminal 200. The nurse performs visual observation of the observation area while checking the entire area of the observational video displayed on the nurse terminal (see Figure 4). If the observational video does not meet the nurse's expectations, the nurse gives appropriate instructions for correcting the shooting area (S30).
[0101] To receive correction instructions for the imaging area, the remote nurse terminal 200 displays a correction instruction information input screen on the nurse terminal using the correction instruction information input unit 221. The correction instruction information input screen is a UI screen for receiving correction instructions, and it displays the observed image and allows the receipt of correction instructions.
[0102] Correction instructions for the shooting area primarily include information about the direction the nurse wants to view the observation image displayed on the nurse's terminal. This direction includes not only up, down, left, and right, but also depth directions such as back (magnification) and front (reduction). The desired viewing direction is the direction in which the shooting area should be moved. In other words, correction instructions include information about the direction in which the shooting area should be corrected.
[0103] For example, a correction instruction includes not only the direction but also information about the strength of the movement of the shooting area—how much the shooting area should be moved in that direction. In other words, a correction instruction includes information about the strength of the correction applied to the shooting area. This information about the strength of the correction applied to the shooting area may be optional.
[0104] The correction instruction information input screen can receive information on the direction and strength of the correction. In other words, the correction instruction information input screen allows input of information on the direction and strength of the correction as a correction instruction. This allows nurses to input correction instructions for the imaging area in response to their observation intentions, such as "I want to see a little more to the right," for the captured image being observed, via the correction instruction information input screen.
[0105] In the following, for convenience, the information regarding the direction of correction and the strength of correction may be referred to as "correction information." In other words, correction information refers to information regarding the direction of correction and the strength of correction.
[0106] Figure 6 schematically shows an example of inputting a correction instruction on the correction instruction information input screen. In this input example, for instance, the correction instruction information input screen includes six selection buttons corresponding to the six directions mentioned above, and the direction to be corrected can be selected by clicking any of the selection buttons. In this example, the right is selected as the direction to be viewed. In addition to the selection buttons, the correction instruction information input screen also includes a slider, and the strength of the correction can be specified by dragging the slider knob.
[0107] Figure 7 schematically shows another example of inputting correction instructions on the correction instruction information input screen. In this input example, for example, clicking on the observation image displayed on the correction instruction information input screen allows the user to specify the direction of correction (up, down, left, or right) based on the relative position of the click to the center of the observation image. The up, down, left, and right directions are determined by the relative position from the center of the click. Furthermore, regarding the depth direction of the correction, for example, scrolling up with the mouse allows the user to specify the direction towards the back, and scrolling down allows the user to specify the direction towards the front. In addition, the strength of the correction can be specified by the distance from the center of the observation image to the click position. The strength of the correction is calculated from the distance from the center of the click position.
[0108] The correction instruction information input unit 221 acquires the correction instruction information entered via the correction instruction information input screen in this manner. The correction instruction information transmission unit 222 transmits the correction instruction information acquired by the correction instruction information input unit 221 to the patient / observation terminal 300.
[0109] (Changing the drawing range of the central region) The patient / observation terminal 300 receives correction instruction information transmitted from the remote nurse terminal 200 and changes the drawing range of the central region of the observation image displayed on the patient terminal according to the correction instruction information of the imaging region (S40).
[0110] The patient / observation terminal 300 receives correction instruction information via the correction instruction information receiving unit 321. The central area definition unit 322 defines the central area for the entire area of the observation video based on the correction instruction information received by the correction instruction information receiving unit 321.
[0111] The central region defining unit 322 defines the central region so that it changes across the entire area of the observed image, specifically so that at least one of the position and size of the central region changes, according to the direction of correction included in the correction instruction information.
[0112] The central area defining unit 322 defines the central area such that, when the correction direction is up, down, left, or right, the central area gradually moves in the opposite direction to the up, down, left, or right direction relative to the entire area. For example, when the correction direction is to the right, the central area defining unit 322 defines the central area such that the central area gradually moves to the left relative to the entire area. The same applies to other directions besides up, down, left, and right, except that the directional relationship is different.
[0113] Furthermore, if the correction direction included in the correction instruction information is the depth direction, the central area defining unit 322 defines the central area so that it expands or contracts relative to the entire area according to the depth direction. For example, if the correction direction is the inward direction, approaching the observation area, the central area defining unit 322 defines the central area so that it gradually shrinks relative to the entire area. Also, if the correction direction is the forward direction, moving away from the observation area, the central area defining unit 322 defines the central area so that it gradually expands relative to the entire area.
[0114] The central area defining unit 322 sets the degree of movement or enlargement / reduction of the central area relative to the entire area of the observed image based on the correction strength included in the correction instruction information. Based on the correction strength included in the correction instruction information, the central area defining unit 322 sets the movement distance of the central area when the correction direction is up, down, left, or right, and sets the enlargement / reduction ratio of the central area when the correction direction is in the depth direction. Furthermore, the central area defining unit 322 may set the time required for movement and enlargement / reduction of the central area based on the correction strength included in the correction instruction information.
[0115] The central region correction unit 323 corrects the reference central region of the observation video acquired by the observation video acquisition unit 311, based on the central region defined by the central region definition unit 322. The correction includes changing the position and size of the central region relative to the entire area of the observation video.
[0116] Alternatively, it can be said that the central region correction unit 323 applies the central region setting defined by the central region defining unit 322 to the entire region of the observation image acquired by the observation image acquisition unit 311.
[0117] As a result, the central region corrected by the central region correction unit 323 moves or expands / contracts across the entire area of the observed image according to the direction of correction indicated by the correction instruction information.
[0118] The observation image rendering unit 324 renders the image of the central region, which has been set for the observation image acquired by the observation image acquisition unit 311, on the display screens of the observation terminal and the patient terminal, based on the central region corrected by the central region correction unit 323.
[0119] As a result, the central region of the observation image displayed on the observation terminal and the patient terminal moves or shrinks. For example, the observation area displayed on the observation terminal and the patient terminal moves or shrinks up, down, left, and right relative to the display screen.
[0120] Furthermore, when using signs to transmit the observed area, the sign information (silhouette image) is corrected based on the central area defined by the central area defining unit 322.
[0121] In other words, the signature information correction unit 332 corrects the signature information (silhouette image) received by the signature information receiving unit 331 based on the central region defined by the central region defining unit 322. The signature information drawing unit 333 then draws the signature information (silhouette image) corrected by the signature information correction unit 332 on the display screen of the observation terminal or the patient terminal.
[0122] As a result, the sign information (silhouette images) displayed on the observation terminal and the patient terminal move or resize in sync with the displayed video. For example, the sign information (silhouette images) displayed on the observation terminal and the patient terminal move or resize up, down, left, and right along with the observed area on the display screen.
[0123] Figure 8 schematically shows an example of changing the drawing range of the central region. In this example, a nurse requests to view the right side of the observation area in the observation video, and the correction instruction specifies a movement of the imaging area to the right. As a result of processing by the central region definition unit 322 and the central region correction unit 323 in response to this correction instruction information, the central region is gradually moved to the left across the entire area of the observation video. Consequently, on the display screens of the observation terminal and the patient terminal, the observation area gradually moves to the right.
[0124] The patient moves the observation terminal to the right so that the area being observed, which is gradually moving to the right, remains centered on the observation image displayed on the terminal. In other words, the patient moves the observation terminal to the right so that the area being observed, which is moving to the right, remains centered on the display screen.
[0125] As a result, the imaging area captured by the observation terminal shifts to the right of the observation site compared to before the correction instruction was given. Consequently, the nurse's terminal displays the observation image shifted to the right, as desired by the nurse.
[0126] Figure 9 schematically shows another example of a change in the drawing range of the central region. In this example, a nurse requests to move closer to the area being observed in the observation video, and the correction instruction specifies a movement of the shooting area in the background direction. As a result of processing by the central region definition unit 322 and the central region correction unit 323 in response to this correction instruction information, the central region is gradually expanded vertically and horizontally across the entire area of the observation video. Consequently, the observed area gradually becomes smaller on the display screens of the observation terminal and the patient terminal.
[0127] The patient moves the observation device closer to the area being observed so that the gradually shrinking area remains the same size as the observation image displayed on the device. In other words, the patient moves the observation device closer to the area being observed so that the shrinking area remains the same size.
[0128] As a result, the imaging area captured by the observation terminal becomes closer to the observation site compared to before the correction instructions were given. This allows the nurse's terminal to display an observation image that is closer to the observation site, as desired by the nurse.
[0129] For example, how to move the observation terminal in response to changes in the displayed image may be instructed through prior lectures. However, it is quite reasonable to expect that the aforementioned movements of the observation terminal will be performed naturally by the medical practitioner even without such lectures.
[0130] For example, if a patient is viewing an area of observation in the center of the observation terminal's display screen and witnesses the area of observation moving to the right on the screen, it is natural to perceive the observation terminal as having moved to the left and to move the terminal to the right so that the area of observation is centered on the screen.
[0131] (Effect) According to the embodiment, by changing the observation area displayed on the observation terminal, it becomes possible to guide the patient's shooting area so that the observation image is taken as desired by the nurse. In other words, in the embodiment, the correction instructions, which are the presented information, are presented to the patient in the form of changes in the observation area displayed on the observation terminal.
[0132] Patients simply view the observation images displayed on the imaging device and receive the correction instructions, which are presented as information, without consciously thinking about them. Therefore, patients do not have the burden of simultaneously viewing and interpreting symbols or other elements that have different meanings from the observed area in order to receive the presented information. Consequently, it is expected that the cognitive load on patients will be kept low.
[0133] In other words, according to the embodiment, a technique is provided for effectively guiding the imaging area without using symbols.
[0134] In this embodiment, we have described the visual observation work in remote nursing as an example, but the method of this embodiment is applicable to all remote collaborative work involving physical movements, including photography work in accordance with the intentions of the supervisor in a remote location. Furthermore, it is effective in various tasks where photography itself is not the direct purpose.
[0135] For example, by correcting the image displayed on a video-see-through head-mounted display using the method of this embodiment, it can be effectively applied to situations such as guiding a worker wearing the head-mounted display to move in a direction different from the direction the worker originally intended to go.
[0136] It should be noted that the present invention is not limited to the embodiments described above, and can be modified in various ways during implementation without departing from its essence. Furthermore, each embodiment may be combined as appropriate, and in that case, the combined effects can be obtained. Moreover, the above embodiments include various inventions, and various inventions can be extracted by selecting combinations from the multiple constituent elements disclosed. For example, if the problem can be solved and effects obtained even if some constituent elements are deleted from all the constituent elements shown in the embodiment, then the configuration with these deleted constituent elements can be extracted as an invention.
[0137] 100...Information display system 200...Remote nurse terminal 211...Observation video receiving unit 212...Observation video rendering unit 221...Correction instruction information input unit 222...Correction instruction information transmission unit 231...Sign information selection unit 232...Sign information transmission unit 300...Patient / observation terminal 311...Observation video acquisition unit 312...Observation video transmission unit 321...Correction instruction information receiving unit 322...Central area definition unit 323...Central area correction unit 324...Observation video rendering unit 331...Sign information receiving unit 332...Sign information correction unit 333...Sign information rendering unit 400...Computer 401...Processor 402...ROM 403...RAM 404...Auxiliary storage device 405...Output device 406...Input device 407...Communication device 408...Bus 500...Recording medium
Claims
1. An information presentation system comprising a first terminal including a camera, and a second terminal that receives captured video from the first terminal and transmits presentation information to the first terminal, wherein the second terminal performs the following: a process of displaying the entire area of the captured video; a process of receiving a correction instruction for the captured area of the captured video; and a process of transmitting the correction instruction to the first terminal; and the first terminal performs the following: a process of displaying a partial area of the captured video; a process of receiving the correction instruction; and a process of defining the partial area based on the correction instruction.
2. The information presentation system according to claim 1, wherein the process for defining the partial region defines the partial region such that the partial region changes with respect to the entire region according to the direction of correction of the imaging region included in the correction instruction.
3. The information presentation system according to claim 1, wherein the second terminal performs a process to transmit the area to be observed to the first terminal using a sign, and the first terminal performs a process to display the sign superimposed on the captured video.
4. An information presentation program that causes a computer having a processor to execute at least a part of the functions of at least one of the first terminal and the second terminal of the information presentation system according to claim 1.
