Information presentation system, information presentation method, and information presentation program

Abstract

An information presentation system according to an embodiment of the present invention comprises an image capturing device, a first terminal, a second terminal, a presentation unit, and an addition unit. The image capturing device captures an image of an observation part, which is a body part of a first user, to acquire a captured video. The first terminal presents information to the first user. The presentation unit causes the first terminal to present an observation video obtained by superimposing, on the captured video, a visual silhouette which is set by a second user who is different from the first user and located remotely from the image capturing device and the first terminal, and with which the first user is to match the location and posture of the observation part. The addition unit adds a visual effect to the observation video presented to the first terminal so as to distinctly display an observation target area centered on the location of a key observation point set by the second user for observation by the second user in the observation part of the first user.

Description

Information Presentation System, Information Presentation Method, and Information Presentation Program

[0001] One aspect of the present invention relates to an information presentation system, an information presentation method, and an information presentation program.

[0002] Amid the rapid increase in the number of care recipients due to the super-aging society, there is a limit to continuously providing nursing care while nurses and care recipients interact face-to-face at medical facilities such as hospitals. Based on this situation, the Ministry of Health, Labour and Welfare of Japan promotes home-based nursing care centered on visiting nursing in its "Community-Based Integrated Care System" initiative. This initiative aims to reduce the burden by having local communities handle tasks other than nursing, but the shortage of nursing professionals has become a barrier to expanding the scope and opportunities of implementation. Therefore, in addition to the conventional implementation form premised on face-to-face interaction, the establishment of a new remotely implemented form is an urgent need.

[0003] Accordingly, with the aim of enabling a small number of nurses to provide nursing care to a large number of care recipients, nursing care through remote communication has been trialed. In this approach, a nurse at a hospital conducts nursing observation while remotely communicating with a care recipient located in another location such as their home via a video call system. The nurse interviews the care recipient about their condition through the conversation, and visually observes the appearance of the patient's face image captured in the video.

[0004] In such remote nursing care, when aiming to improve the quality of visual observation, for example when visually observing minute body parts such as the "nape of the neck" or "fingertips", it is necessary for the nurse to instruct the care recipient on how to move so that the relevant observation site can be properly captured. When prompting the care recipient to capture the image at the position, angle, and size expected by the nurse, it is required that the nurse accurately express the fine movements involved so that the care recipient can understand the instructions. However, it is difficult to convey these fine movements only verbally. Therefore, creating a mechanism to support the transmission of these fine movements, which are difficult to verbalize, has become an issue.

[0005] Therefore, an information presentation method that supports the transmission of subtle movements using non-verbal visual information has been proposed, as disclosed in Non-Patent Document 1. In this method, the physical movements required to achieve the desired posture for the patient are presented and communicated using visual information called "signs." Specifically, the body is guided by representing and instructing the desired overall posture with abstract icons such as pictograms. Alternatively, the body is guided by representing and instructing the desired posture of individual body parts with reference lines, etc. This makes it possible to intuitively express and understand subtle movements that are difficult to verbalize, contributing to the implementation of high-quality visual observation.

[0006] Soichiro Uchida et al., "An Interface Design Utilizing Nonverbal Information to Support Smooth Visual Observation in Remote Nursing," [Online], [Retrieved February 7, 2025], Internet <URL: https: / / ken.ieice.org / ken / paper / 20240126BcaY / >

[0007] While the method disclosed in Non-Patent Document 1 can roughly guide the patient's body position and angle toward the posture desired by the nurse, it is difficult to express more subtle postures at the joint level, such as the degree to which the fingers are spread or the degree to which the neck is tilted.

[0008] Furthermore, from the perspective of what nurses observe, while patients can understand things at the level of body parts such as their hands or neck, nurses cannot obtain detailed information about which specific areas within those areas they want to focus on.

[0009] As a result, patients may be forced to perform actions without understanding the strictness of the posture they should adopt or which areas within the observation range require particular precision. Nurses may struggle to maintain a precise posture in areas other than those they originally intended to observe, potentially unnecessarily increasing the workload on the patient.

[0010] Therefore, a system is desired that can represent the observed area down to the finest details of its posture while visualizing the areas of particular importance within that representation.

[0011] This invention was made in view of the above circumstances, and aims to provide an information presentation method that can achieve more detailed representation of posture and visualization of areas of importance in observation.

[0012] To solve the above problems, one aspect of the present invention provides an information presentation system comprising a shooting device, a first terminal, a second terminal, a presentation unit, and an additional unit. The shooting device photographs an observation area, which is a body part of a first user, and acquires a captured image. The first terminal presents information to the first user. The presentation unit causes the first terminal to display an observation image in which a visual silhouette, set by a second user who is located remotely from the shooting device and the first terminal and is different from the first user, is superimposed on the captured image, where the first user should align the position and posture of the observation area. The additional unit adds a visual effect to the observation image presented to the first terminal so as to identify and display the observation target area, centered on the position of key points of observation in the observation area of ​​the first user that the second user should observe, as set by the second user.

[0013] According to one aspect of this invention, when a patient, the first user, places their body against the silhouette, they are naturally guided to align themselves around the observation points that the nurse, the second user, wants to emphasize. This reduces the workload on the patient and makes it easier for the nurse to adopt the posture they expect, thus providing an information presentation method that can achieve more precise representation of posture and visualization of areas of interest during observation.

[0014] Figure 1 is a block diagram showing an example of the configuration of an information presentation system according to the first embodiment of this invention. Figure 2 is a block diagram showing an example of the hardware configuration of a nurse terminal included in the information presentation system according to the first embodiment. Figure 3 is a block diagram showing an example of the software configuration of a nurse terminal. Figure 4 is a diagram showing an example of the stored contents of the silhouette information storage unit of a nurse terminal. Figure 5 is a block diagram showing an example of the hardware configuration of a patient terminal included in the information presentation system according to the first embodiment. Figure 6 is a block diagram showing an example of the software configuration of a patient terminal. Figure 7 is a sequence diagram for explaining the operation of the information presentation system according to the first embodiment. Figure 8 is a diagram showing an example of an observation area selection screen presented on the nurse terminal. Figure 9 is a diagram showing an example of a silhouette selection screen presented on the nurse terminal. Figure 10 is a diagram showing an example of a method for specifying observation points on the observation screen presented on the nurse terminal. Figure 11 is a diagram showing a blurring effect as an example of a visual effect. Figure 12 is a diagram showing a brightness adjustment effect as an example of a visual effect. Figure 13 is a diagram showing an example of an observation screen with a blurring effect applied, presented on the nurse terminal and the patient terminal when a blurring effect is adopted. Figure 14 is a diagram showing an example of an observation screen with brightness adjustment applied, as presented on the nurse terminal and the patient terminal when brightness adjustment effect is adopted. Figure 15 is a schematic diagram showing how the patient positions their body according to the silhouette and visual effects. Figure 16 is a flowchart showing an example of the processing procedure and content of the nurse terminal. Figure 17 is a flowchart showing an example of the processing procedure and content of the patient terminal. Figure 18 is a schematic diagram illustrating a group of observation images for wound progress monitoring. Figure 19 is a diagram showing an example of a progress observation screen when emphasis is placed on image matching. Figure 20 is a diagram showing an example of a progress observation screen when emphasis is placed on matching observation points. Figure 21 is a block diagram showing an example of the configuration of an information presentation system according to the second embodiment of this invention. Figure 22 is a block diagram showing an example of the hardware configuration of an information presentation server included in the information presentation system according to the second embodiment. Figure 23 is a block diagram showing an example of the software configuration of an information presentation server.

[0015] Embodiments of this invention will be described below with reference to the drawings.

[0016] [First Embodiment] Figure 1 is a block diagram showing an example of the configuration of an information presentation system SYS according to the first embodiment of the present invention. The information presentation system SYS according to the first embodiment includes a nurse terminal 1, a patient terminal 2, and an observation terminal 3, all connected to a network NW such as the Internet or a LAN (Local Area Network). The connection between the nurse terminal 1, patient terminal 2, and observation terminal 3 and the network NW may be a wired connection or a wireless connection.

[0017] The observation terminal 3 is, for example, a smartphone, and uses its camera as a shooting device to photograph the body parts of the patient that the nurse wishes to observe. The observation terminal 3 continuously transmits the captured video in real time. For example, the observation terminal 3 uses WebRTC (Web Real-Time Communication) to send and receive streams with the patient terminal 2 via the network NW. The observation terminal 3 may also be directly connected to the patient terminal 2, as shown by the dashed line in Figure 1. In this case, the connection between the patient terminal 2 and the observation terminal 3 can be either wired or wireless. The patient can directly view the captured images by checking the screen of the smartphone, which is the observation terminal 3.

[0018] Furthermore, the observation terminal 3 may be a camera such as a webcam, endoscope, or general video camera, connected to the patient terminal 2 by wire or wireless connection. In this case, the images captured by the camera acting as the observation terminal 3 are always viewed on the patient terminal 2. For example, an observation terminal 3 such as a smartphone is difficult to insert into a limited space such as the inside of the mouth, and is also heavy and difficult to handle, but a webcam or similar device can mitigate these problems.

[0019] Nurse terminal 1 is an information processing device such as a PC (Personal Computer), smartphone, or tablet. Nurse terminal 1 is used by nurses and displays images captured by observation terminal 3 to the nurse, and allows nurses to input and configure information to be displayed on patient terminal 2. Details of nurse terminal 1 will be described later.

[0020] Patient terminal 2 is also an information processing device such as a PC, smartphone, or tablet. Patient terminal 2 is used by the patient and displays images captured by observation terminal 3 to the patient, as well as various information in response to instructions from nurse terminal 1. Details of patient terminal 2 will be described later.

[0021] Note that while Figure 1 only shows one of each of the nurse terminal 1, patient terminal 2, and observation terminal 3, the number of each is arbitrary. What is necessary is that when a nurse observes a patient, one nurse terminal 1 is provided for the nurse, and one patient terminal 2 and one observation terminal 3 are provided for each patient. This does not prevent one nurse terminal 1 from being shared by multiple nurses, or one patient terminal 2 and one observation terminal 3 from being shared by multiple patients.

[0022] Figure 2 is a block diagram showing an example of the hardware configuration of the nurse terminal 1. As shown in Figure 2, the nurse terminal 1 includes a control unit 11, to which a storage unit having a program storage unit 12 and a data storage unit 13, a communication interface unit 14, and an input / output interface unit 15 are connected via a bus 16. In Figure 2, "interface" is abbreviated as "IF".

[0023] The control unit 11 is a hardware processor such as a CPU (Central Processing Unit). For example, by using a multi-core and multi-threaded CPU, multiple information processing operations can be performed simultaneously. The control unit 11 may also be equipped with multiple hardware processors.

[0024] The communication interface unit 14, under the control of the control unit 11, transmits and receives information with other devices using a communication protocol defined by the network NW.

[0025] The input / output interface unit 15 is connected to an input device 171, an audio input device 172, a video input device 173, an audio output device 174, a video output device 175, and the like. Note that the nurse terminal 1 does not necessarily have to be equipped with all of these devices; some of them may be connected as external devices.

[0026] The input device 171 is used by the nurse to input instructions and information necessary for the operation of the nurse terminal 1. If the nurse terminal 1 is a PC, the input device 171 may include, for example, a keyboard, a mouse, a touchpad or other pointing device. If the nurse terminal 1 is a smartphone or tablet, the input device 171 may include, for example, a touch panel, operation buttons, etc., located on the display screen of the video output device 175. The input device 171 may also include a reader device for reading information from a recording medium that stores various information such as programs and data.

[0027] The voice input device 172 is equipped with a microphone and acquires the voice spoken by the nurse.

[0028] The video input device 173 is equipped with a camera such as a webcam or a general video camera, and captures images of the nurse.

[0029] The video output device 175 includes a display such as an LCD monitor, an OLED (Electro-Luminescence) monitor, a projector screen, or a head-mounted display, and displays various information on the nurse terminal 1.

[0030] The audio output device 174 is equipped with a speaker and transmits various information to the nurse by sound.

[0031] Although not shown in Figure 2, the input / output interface unit 15 may also include a writer device for writing the information generated by the nurse terminal 1 to a recording medium.

[0032] The program storage unit 12 is configured by combining, for example, a non-volatile memory that can be written to and read at any time, such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or EEPROM (Registered Trademark) (Electric Erasable Programmable Read-Only Memory), with a non-volatile memory such as ROM (Read Only Memory). In addition to middleware such as an OS (Operating System), the program storage unit 12 stores information presentation programs, which are application programs necessary for the nurse terminal 1 to perform actions for observing patients. Hereafter, the OS and each application program will be collectively referred to as a program.

[0033] The data storage unit 13 combines, for example, a non-volatile memory that can be written to and read at any time, such as an HDD, SSD, EEPROM, or memory card, with a volatile memory such as RAM (Random Access Memory), as a storage medium.

[0034] In addition to pre-storing the above-mentioned information presentation program in the program storage unit 12, it may also be read from the storage medium containing the information presentation program and stored in the program storage unit 12 when needed. Alternatively, the information presentation program may be downloaded from an application provider connected to the network NW (not shown in Figure 1) and stored in the program storage unit 12.

[0035] Figure 3 is a block diagram showing an example of the software configuration of Nurse Terminal 1. Note that basic functions such as the login section and the session management database that links the screens of Nurse Terminal 1 and Patient Terminal 2, which are common in online information exchange systems and not specific to this information presentation system SYS, have been omitted.

[0036] The control unit 11, as a processing function unit according to the first embodiment of this invention, includes a silhouette information registration unit 111, a silhouette information selection unit 112, a silhouette information drawing unit 113, a silhouette information transmission unit 114, a captured video receiving unit 115, a captured video drawing unit 116, an observation point information selection unit 117, an observation point information reflection unit 118, an observation point information transmission unit 119, and a video call unit 1110. All of these processing function units 111 to 1110 are realized by causing the hardware processor of the control unit 11 to execute an information presentation program according to the first embodiment of this invention, which is stored in the program storage unit 12. Furthermore, at least one or at least a part of the processing functions among the above-mentioned processing function units 111 to 1110 may be implemented by integrated circuits such as ASIC (Application Specific Integrated Circuit), DSP (Digital Signal Processor), FPGA (field-programmable gate array), GPU (Graphics Processing Unit), instead of being implemented by the information presentation program and the hardware processor of the control unit 11.

[0037] Furthermore, the data storage unit 13 includes a silhouette information storage unit 131 in its non-volatile memory, which is necessary for carrying out the first embodiment.

[0038] The silhouette information registration unit 111 stores silhouette information in the silhouette information storage unit 131. In this embodiment, "silhouette" is a line drawing illustration that represents the observation area down to the minute posture expected by the nurse, and "silhouette information" is the data of that line drawing illustration. Figure 4 is a diagram showing an example of the contents stored in the silhouette information storage unit 131. As shown in Figure 4, the silhouette information registration unit 111 stores each of the multiple silhouette information prepared for each observation area and each posture expected by the nurse, linking them to the observation area.

[0039] The silhouette information registration unit 111 can store pre-created silhouette information in the silhouette information storage unit 131 by having the reader device, which is the input device 171, read the information from the recording medium, or by receiving it from an information providing device connected to the network NW via the communication interface unit 14. Alternatively, the silhouette information registration unit 111 may dynamically generate silhouettes by having the nurse assume the desired posture and taking a picture of her with the camera of the video input device 173, and then applying image processing to the acquired image. For example, by taking a picture against a fixed-color background such as a green screen, or by extracting the human body from the background using machine learning, an image of only the observed parts can be created, and this image can be converted into a line drawing of the outline using readily available technology such as OpenCV to generate a silhouette. A method for extracting human bodies from a background using machine learning is disclosed in Hirokazu Kakinuma et al., "Real-time Subject Extraction Technology for Arbitrary Backgrounds Using Machine Learning," [Online], [Retrieved February 7, 2025], Internet <URL: https: / / journal.ntt.co.jp / wp-content / uploads / 2020 / 06 / JN20181016.pdf>. The silhouette information registration unit 111 can store the silhouette information of the silhouettes thus generated in the silhouette information storage unit 131, linking it to the observation area instructed by the operation of the nurse's input device 171, such as a keyboard or mouse.

[0040] The silhouette information selection unit 112 receives a selection operation from the nurse using the input device 171 from among multiple silhouette information stored in the silhouette information storage unit 131, and reads the selected silhouette information from the silhouette information storage unit 131. When the nurse visually observes the patient's body, they select the silhouette that they feel is closest to the posture they most expect. If no suitable silhouette exists, the nurse creates silhouette information using the silhouette information registration unit 111.

[0041] The silhouette information drawing unit 113 presents the silhouette to the nurse by drawing the silhouette information read by the silhouette information selection unit 112 on the display screen of the video output device 175.

[0042] The silhouette information transmitting unit 114 transmits the silhouette information read out by the silhouette information selecting unit 112 to the patient terminal 2 via the network NW by the communication interface unit 14.

[0043] The captured video receiving unit 115 acquires the captured video that has been transmitted from the observation terminal 3 directly or via the patient terminal 2 via the network NW and received by the communication interface unit 14.

[0044] The captured video drawing unit 116 inserts and draws the captured video acquired by the captured video receiving unit 115 below the silhouette drawn by the silhouette information drawing unit 113 on the display screen of the video output device 175, thereby presenting the captured video to the nurse. In the present embodiment, the video obtained by superimposing the silhouette on the captured video is referred to as observation video.

[0045] The key observation information selecting unit 117 accepts an arbitrary coordinate designation from among the observation sites of the patient via an operation of the nurse on the input device 171, and selects the coordinate position as key observation information. The selected key point is that the arbitrary coordinate position designated by the nurse is the observation target site of the body part that the nurse wants to observe, that is, the position that the nurse particularly focuses on.

[0046] The key observation information reflecting unit 118 causes the silhouette information drawing unit 113 and the captured video drawing unit 116 to reflect the key observation information selected by the key observation information selecting unit 117. Details of this reflection will be described later.

[0047] The key observation information transmitting unit 119 transmits the key observation information selected by the key observation information selecting unit 117 to the patient terminal 2 via the network NW by the communication interface unit 14.

[0048] The video call unit 1110 provides a general video call function that enables verbal (linguistic) conversation between a nurse and a care recipient. Specifically, regarding audio, the communication interface unit transmits, via the network NW, the nurse's audio input from the audio input device 172 to the care recipient terminal 2, and receives the care recipient's audio transmitted from the care recipient terminal 2, and causes the audio output device 174 to output the audio. Also, regarding video, the communication interface unit transmits, via the network NW, the nurse's video input from the video input device 173 to the care recipient terminal 2, and receives the care recipient's video transmitted from the care recipient terminal 2, and causes the care recipient's video to be superimposed and drawn in the form of a small window on a part of the observation video of the video output device 175.

[0049] Figure 5 is a block diagram showing an example of the hardware configuration of the care recipient terminal 2. As shown in Figure 5, the care recipient terminal 2 includes, similarly to the control unit 11, program storage unit 12, data storage unit 13, communication interface unit 14, input / output interface unit 15, bus 16, input device 171, audio input device 172, video input device 173, audio output device 174, and video output device 175 of the nurse terminal 1, a control unit 21, a program storage unit 22, a data storage unit 23, a communication interface unit 24, an input / output interface unit 25, a bus 26, an input device 271, an audio input device 272, a video input device 273, an audio output device 274, and a video output device 275. The care recipient terminal 2 may further include an external input / output interface unit 28. The external input / output interface unit 28 may be a wired or wireless communication interface for connecting the observation terminal 3.

[0050] Figure 5 is a block diagram showing an example of the software configuration of the care recipient terminal 2. Note that, in Figure 5, similarly to Figure 3, blocks that are not specific to the information presentation system SYS are omitted.

[0051] The control unit 21, as a processing function unit according to the first embodiment of this invention, includes a silhouette information receiving unit 211, a silhouette information drawing unit 212, a captured image acquisition unit 213, a captured image drawing unit 214, a captured image transmission unit 215, an observation point information receiving unit 216, an observation point information reflection unit 217, and a video call unit 218. All of these processing function units 211 to 218 are realized by having the hardware processor of the control unit 21 execute an information presentation program according to the first embodiment of this invention, which is stored in the program storage unit 22. In addition, at least one or at least a part of the processing functions of the above processing function units 211 to 218 may be realized by an integrated circuit such as an ASIC, DSP, FPGA, GPU, instead of being realized by an information presentation program and the hardware processor of the control unit 21, similar to the nurse terminal 1.

[0052] The silhouette information receiving unit 211 acquires silhouette information from the nurse terminal 1, which is received via the network NW by the communication interface unit 24.

[0053] The silhouette information drawing unit 212 presents the silhouette to the patient by drawing the silhouette information acquired by the silhouette information receiving unit 211 on the display screen of the video output device 275.

[0054] The captured video acquisition unit 213 acquires captured video transmitted from the observation terminal 3 via the network NW and received by the communication interface unit 24. Alternatively, the captured video acquisition unit 213 can also acquire captured video from the observation terminal 3 via the external input / output interface unit 28.

[0055] The captured video rendering unit 214 presents the captured video to the patient by inserting the captured video acquired by the captured video acquisition unit 213 onto the display screen of the video output device 275, below the silhouette drawn by the silhouette information rendering unit 212. In this way, the patient terminal 2 also presents the same observational video as the nurse terminal 1, in which a silhouette is superimposed on the captured video.

[0056] The captured video transmission unit 215 transmits the captured video acquired by the captured video acquisition unit 213 to the nurse terminal 1 via the network NW using the communication interface unit 24.

[0057] The observation point information receiving unit 216 acquires observation point information from the nurse terminal 1, which is received via the network NW by the communication interface unit 24.

[0058] The observation point information reflection unit 217 reflects the observation point information acquired by the observation point information receiving unit 216 to the silhouette information drawing unit 212 and the captured image drawing unit 214.

[0059] The video call unit 218 provides general video call functionality to enable patients and nurses to communicate verbally (verbally).

[0060] Next, an example of the operation of the information presentation system SYS according to the first embodiment, configured as described above, will be explained. Figure 7 is a sequence diagram illustrating the operation of the information presentation system SYS according to the first embodiment.

[0061] Before beginning observation of the patient, the nurse terminal 1 prepares silhouette information as needed and stores it in the silhouette information storage unit 131 (ST1).

[0062] Then, on the nurse terminal 1, the nurse selects the body part they want to observe and the silhouette that best matches the desired posture (ST2).

[0063] Figure 8 shows an example of an observation area selection screen 1751 presented on the nurse terminal 1. When the silhouette information selection unit 112 of the nurse terminal 1 receives an instruction from the input device 171 to select predetermined silhouette information, it first displays this observation area selection screen 1751 on the video output device 175. As shown in Figure 8, the observation area selection screen 1751 includes a schematic diagram of the human body 17511 and a next selection button 17512, and a mouse pointer 17513 is displayed on this observation area selection screen 1751.

[0064] When the mouse on the input device 171 is right-clicked, the silhouette information selection unit 112 determines that the part of the human body diagram 17511 corresponding to the coordinate position pointed to by the mouse pointer 17513 at that moment has been selected, and switches the identification display 17514 to the selected part. When the mouse pointer 17513 is moved and the mouse is right-clicked again, the silhouette information selection unit 112 switches the identification display 17514 to the part corresponding to the coordinate position of the mouse pointer 17513. When the next selection button 17512 is selected, for example, when the mouse pointer 17513 is moved to the position of the next selection button 17512 and the mouse is right-clicked, the silhouette information selection unit 112 determines the selected part displayed by the identification display 17514 as the target part for silhouette selection, and switches the display screen of the video output device 175 to the silhouette selection screen. Furthermore, by moving the mouse pointer 17513 to the position of an arbitrary part and performing a predetermined operation such as a double-click of the right mouse button, the silhouette information selection unit 112 can determine that part as the part to be selected for silhouette selection and switch the display screen of the video output device 175 to the silhouette selection screen.

[0065] Figure 9 shows an example of a silhouette selection screen 1752 presented on the nurse terminal 1. As shown in Figure 9, the silhouette selection screen 1752 includes a silhouette list 17521 and a start command button 17522, and a mouse pointer 17523 is displayed on this silhouette selection screen 1752. The silhouette list 17521 is a list of silhouettes corresponding to the selected target area selected on the observation area selection screen 1751. The observation area selection screen 1751 reads the silhouette information associated with the selected target area from the silhouette information storage unit 131, creates a reduced-size silhouette, and places this silhouette list 17521 on the silhouette selection screen 1752. Alternatively, the silhouette information storage unit 131 may store a list of images corresponding to each observation area, and the silhouette information selection unit 112 may read them and place them on the silhouette selection screen 17521 as the silhouette list 17521. However, in this case, when a nurse creates a new silhouette of their choice in the silhouette information registration unit 111 and stores it in the silhouette information storage unit 131, it is also necessary to update this list image.

[0066] When the mouse of the input device 171 is right-clicked, the silhouette information selection unit 112 selects the silhouette corresponding to the coordinate position pointed to by the mouse pointer 17523 at that time, and overlays the selection frame 17524 surrounding that silhouette. When the mouse pointer 17513 is moved and the mouse is right-clicked again, the silhouette information selection unit 112 switches the overlaid display of the selection frame 17524 to the silhouette corresponding to the coordinate position of the mouse pointer 17513. When the start instruction button 17522 is selected, for example, when the mouse pointer 17523 is moved to the position of the start instruction button 17522 and the mouse is right-clicked, the silhouette information selection unit 112 determines that the silhouette overlaid in the selection frame 17524 is the silhouette to be presented to the patient terminal 2. Furthermore, the silhouette information selection unit 112 can also determine which silhouette to present to the patient terminal 2 in response to a predetermined operation, such as moving the mouse pointer 17513 to the position of an arbitrary silhouette and double-clicking the right mouse button.

[0067] Returning to the explanation of Figure 7, once a silhouette is selected, the nurse terminal 1 begins communication with the patient terminal 2 using a known method. The nurse terminal 1 then transmits the silhouette information of the selected silhouette to the patient terminal 2 (ST3).

[0068] Furthermore, the nurse terminal 1 displays the silhouette information on the display screen of the video output device 175, presenting the silhouette to the nurse (ST4).

[0069] Meanwhile, the patient terminal 2 displays the silhouette information transmitted from the nurse terminal 1 on the display screen of the video output device 275, presenting the silhouette to the patient (ST5).

[0070] Furthermore, patient terminal 2 begins acquiring the video footage captured by observation terminal 3 (ST6). Since observation terminal 3 continuously transmits the video footage in real time, patient terminal 2 continues to acquire the video footage thereafter.

[0071] Furthermore, the video footage captured by the patient terminal 2 is also transmitted to the nurse terminal 1, and the nurse terminal 1 begins to acquire the video footage transmitted from the patient terminal 2 (ST7).

[0072] In the patient terminal 2, the captured video is displayed on the display screen of the video output device 275 by overlaying it below the silhouette drawn on the screen, thereby presenting the observed video to the patient (ST8).

[0073] Similarly, in the nurse terminal 1, the captured video transmitted from the patient terminal 2 is displayed on the display screen of the video output device 175 by inserting it below the silhouette drawn on the screen, thereby presenting the observed video to the nurse (ST9).

[0074] Then, on the nurse terminal 1, the nurse specifies the center coordinates of the observation points they want to focus on within the observation area represented by the silhouette. Figure 10 is a diagram showing an example of how to specify observation points OP on the observation screen 1753 presented on the nurse terminal 1. As shown in Figure 10, the observation screen 1753 draws silhouettes 17532 in the observation video 17531. At the start of observation, the patient's body parts in the observation video 17531 are not aligned with silhouettes 17532. Therefore, the nurse specifies a position on silhouette 17532 that corresponds to the position they want to observe, rather than the patient's body parts shown in the observation video 17531. The nurse can set the center C of the observation points by specifying an arbitrary position on this silhouette, for example, using mouse operations.

[0075] Returning to the explanation of Figure 7, when observation points are specified or updated on the nurse terminal 1 (ST10: YES), the observation point information is transmitted from the nurse terminal 1 to the patient terminal 2.

[0076] When the patient terminal 2 receives this observation point information (ST11: YES), it reflects a weighted representation based on that observation point information in the observation video displayed on the display screen of the video output device 275 (ST12). This weighted representation is an identification display that allows the patient to easily identify the observation points, i.e., the areas to be observed. It provides a visual effect by applying image processing to the observation video according to the distance from the observation point center C, which is set by the nurse. Figures 11 and 12 show examples of visual effects as weighted representations indicating areas to be observed. As a visual effect, for example, by applying image processing to the observation video 27511 displayed on the observation screen 2751, gradually increasing the blur filter according to the distance from the observation point center C, a blurring effect can be considered, as shown in Figure 11, where the focus is on the observation point center C, and the blur increases so that the focus shifts as the distance from the observation point center C increases. Furthermore, as a visual effect, for example, by applying image processing to the observation video 27511 presented on the observation screen 2751, gradually lowering the brightness according to the distance from the observation point center C, a brightness adjustment effect can be considered, as shown in Figure 12, which makes it appear as if a spotlight is shining on the observation point center C. The weighting representation can be determined by default, or it can be specified or switched at any point in time on the nurse terminal 1, for example, when selecting a silhouette.

[0077] In the nurse terminal 1, the same weighted representation based on the observation points information is reflected in the observation video (ST13).

[0078] Figure 13 shows an example of observation screens 1753 and 2751 with a blurring effect applied, as presented on the nurse terminal 1 and patient terminal 2. As shown in Figure 13, the observation images 17531 and 27511 presented on the observation screens 1753 and 2751 include silhouettes 17532 and 27512 and observation point marks 17533 and 27513. In addition to the observation images 17531 and 27511, the observation screens 1753 and 27511 also include observation terminal information 17534 and 27514. Furthermore, only on the observation screen 1753 of the nurse terminal 1, a weighting expression application toggle button 17535, a capture button 17536, a mouse pointer 17537, and a target circle 17538 are included.

[0079] Observation point marks 17533 and 27513 indicate the location of the observation point center C designated by the nurse.

[0080] The observation terminal information 17534 and 27514 indicates the observation terminal 3 from which the observation video was acquired.

[0081] The weighted expression application toggle button 17535 is a button that is operated to instruct the user to switch between applying and not applying, i.e., deactivating, the weighted expression. This button operation may be performed, for example, by moving the mouse pointer 17537 over the weighted expression application toggle button 17535 and right-clicking the mouse. Furthermore, the weighted expression application toggle button 17535 can present the nurse with the applied / deactivated status of the weighted expression by changing its display depending on whether the weighted expression is currently applied or not.

[0082] The capture button 17536 is operated to instruct the recording of the observation video 17531. This button operation can be, for example, by moving the mouse pointer 17537 over the capture button 17536 and performing a right-click operation with the mouse. Although not specifically shown in Figure 3, when the capture button 17536 is operated, the control unit 11 of the nurse terminal 1 stores the captured video that formed the basis of the observation video as a still image in the data storage unit 13, linked to the coordinates of the observation point center C, the current time, and patient identification information. Patient identification information is identification information such as the identification number of the patient or patient terminal.

[0083] The target circle 17538 is a mark that surrounds the mouse pointer 17537 to improve its visibility. The target circle 17538 moves in conjunction with the movement of the mouse pointer 17537.

[0084] As shown in Figure 13, in the observation images 17531 and 27511 presented on the observation screens 1753 and 2751, the captured body parts of the patient become blurred as they move away from the observation point marks 17533 and 27513, which indicate the observation point center C.

[0085] Figure 14 shows an example of observation screens 1753 and 2751 with the brightness adjustment effect applied, as presented on the nurse terminal 1 and patient terminal 2 when the brightness adjustment effect is adopted. As shown in Figure 14, in the observation images 17531 and 27511 presented on the observation screens 1753 and 2751, the images become darker as the captured body parts of the patient move away from the observation point marks 17533 and 27513, which indicate the observation point center C.

[0086] Returning to the explanation of Figure 7, the patient, after viewing the silhouette 27512 and observation point marks 27513 drawn on the observation video 27511 which reflects the weighted representation, performs an adjustment action to align the position and posture of their body parts with the silhouette 27512 (ST14). The patient can easily understand whether their movements are correct or not through the observation video which is updated in real time. At this time, the observation video 27511 which reflects the weighted representation, that is, the visual effect, allows the patient to understand the importance of the precision of the posture they take and the areas within the observation range that require particular precision. In other words, it is important to understand that it is more important to prioritize aligning areas near the observation points than to uniformly align with the entire silhouette. Therefore, it is possible to reduce the unnecessary movement burden on the patient, such as when the nurse struggles to take a precise posture in an area other than the area they originally wanted to observe.

[0087] Figure 15 is a schematic diagram showing how a patient positions their body according to the silhouette 27512 and its visual effects. In the observation screen 2751 on the left side of Figure 15, as indicated by arrow A, the patient twists their wrist to align with the silhouette 27512. At this time, rather than aligning with the fingertips or wrist of the silhouette, the patient can focus on aligning the back of their hand near the observation point center C indicated by the observation point mark 27513 with the silhouette 27512. This allows the nurse to obtain an observation image showing the observation points they wish to observe, as shown in the observation screen 2751 on the right side of Figure 15.

[0088] Returning to the explanation of Figure 7, on the patient terminal 2, the observation screen 2751 is updated based on the video footage captured by the observation terminal 3 from ST8 onwards until the nurse terminal 1 instructs it to end (ST15: NO).

[0089] Furthermore, the same image as the observation screen 2751 presented on the patient terminal 2 is also presented as the observation screen 1753 on the nurse terminal 1. At this time, the nurse can, if necessary, operate the weighted expression application switch button 17535 on the observation screen 1753 to remove the weighted expression and observe, or operate the capture button 17536 to record the observation image 17531. The observation screen 1753 is updated based on the captured image from the observation terminal 3 from ST9 onwards until the desired observation target area is observed by the nurse (ST16:NO).

[0090] Furthermore, even if the desired observation area is observed by the nurse (ST16: YES), other observation points may also exist in that observation area. In this case (ST17: YES), the nurse moves the mouse pointer 17537 to the location of the new observation point on the observation screen 1753 to specify the observation point, that is, to update the observation point. In response to this operation, the nurse terminal 1 updates the observation screen 1753 to correspond to this new observation point using ST10 and ST13.

[0091] Similarly, the weighted representation on the observation screen 2751 in the patient terminal 2 is updated according to these new observation points. The patient then adjusts their settings to reflect the new observation points.

[0092] Furthermore, even if observation of one or more points of interest in the observation area is completed (ST17: NO), there may be other body parts that the nurse wishes to observe. In this case (ST18: YES), the nurse will finish this observation screen 1753 and repeat the process from selecting the observation area and silhouette in ST2 above.

[0093] Once observation of all desired observation sites is complete (ST18: NO), Nurse Terminal 1 instructs Patient Terminal 2 to finish. This instruction to finish may also be communicated via video call.

[0094] Next, we will explain the processing procedures for the nurse terminal 1 and the patient terminal 2, respectively, for performing these operations.

[0095] First, let's explain Nurse Terminal 1. Figure 16 is a flowchart showing an example of the processing procedure and processing content of Nurse Terminal 1.

[0096] As shown in Figure 16, the control unit 11 of the nurse terminal 1 first operates as a silhouette information registration unit 111, preparing silhouette information corresponding to the body part and posture to be observed in advance, and registering it, i.e., storing it, in the silhouette information storage unit 131 reserved in the non-volatile memory of the data storage unit 13 (step S101). As mentioned above, this silhouette information registration process is not required every time the nurse terminal 1 is used.

[0097] Next, the control unit 11 operates as a silhouette information selection unit 112, receiving input from the nurse using the input device 171, and selecting the silhouette information to be used from the silhouette information stored in the silhouette information storage unit 131 (step S102). The control unit 11 reads the selected silhouette information from the silhouette information storage unit 131 and stores the read silhouette information in the work memory reserved in the volatile memory of the data storage unit 13.

[0098] Then, the control unit 11 operates as a silhouette information transmission unit 114 and transmits the silhouette information stored in the work memory to the patient terminal 2 via the communication interface unit 14 (step S103).

[0099] Furthermore, the control unit 11 operates as a silhouette information drawing unit 113 and draws the silhouette information temporarily stored in the work memory onto the screen of the video output device 175 (step S104).

[0100] Subsequently, the control unit 11 operates as a captured video receiving unit 115 and receives captured video from the observation terminal 3 transmitted from the patient terminal 2 via the communication interface unit 14, and records the received captured video in the work memory of the data storage unit 13 (step S105).

[0101] Then, the control unit 11 operates as a captured image rendering unit 116 and inserts the captured image recorded in the work memory below the silhouette drawn on the screen of the video output device 175 (step S106).

[0102] Subsequently, the control unit 11 operates as an observation point information selection unit 117 to determine whether or not the coordinates of an observation point have been specified by operating the input device 171 (step S107). This operation to specify the coordinates of an observation point may, for example, be a right-click operation of the mouse after moving the mouse pointer 17537 to the desired observation point position, as described above.

[0103] If the control unit 11 determines that no coordinates of the points of observation have been specified, it determines whether or not a shooting instruction has been given by operating the input device 171 (step S108). This shooting instruction operation may be, for example, the operation of the shooting button 17536, as described above.

[0104] If the control unit 11 determines that there is no instruction to take a photograph, it further determines whether or not there has been a request to end the observation of the observation points by operating the input device 171 (step S109). This operation to end the observation can be, for example, the operation of an observation end button (not shown) located on the observation screen 1753.

[0105] If the control unit 11 determines that there is no indication to end the observation of the observation points, it repeats the process described above from step S105.

[0106] In step S107, if it is determined that the coordinates of the observation point have been specified, the control unit 11, which is operating as the observation point information selection unit 117, stores the coordinates of the observation point center C as observation point information in the work memory of the data storage unit 13. Then, the control unit 11 operates as the observation point information reflection unit 118 and reflects the observation point information stored in the work memory onto the observation video (step S110). Specifically, when the control unit 11, acting as the silhouette information drawing unit 113, draws the silhouette information on the screen of the video output device 175, it draws the observation point mark 17533. Also, when the control unit 11, acting as the captured video drawing unit 116, draws the captured video on the screen of the video output device 175, it adds a pre-specified blurring effect or brightness adjustment effect as a visual effect. Here, in order not to interfere with the visibility of the silhouette 17532, the visual effect is added only to the captured video. Of course, visual effects may be added to the observation image in which the silhouette 17532 and the captured video are superimposed. In that case, the visual effects may be added to the observation image that is a composite of the silhouette 17532, the observation point marks 17533, and the captured video.

[0107] Subsequently, the control unit 11 operates as an observation point information transmission unit 119 and transmits the observation point information stored in the work memory to the patient terminal 2 via the communication interface unit 14 (step S111). The control unit 11 then repeats the above process from step S104.

[0108] Furthermore, if the control unit 11 determines in step S108 that a shooting instruction has been given, it saves the observed video and key point information (step S112). Specifically, the control unit 11 converts the captured video that formed the basis of the observed video recorded in the work memory at that time into a still image. The control unit 11 then links this still image with the key point information, which is the coordinates of the observation point center C, the current time, and patient identification information stored in the work memory, and stores it in a history storage unit (not shown) reserved in the data storage unit 13.

[0109] Furthermore, if the control unit 11 determines in step S109 that the observation of the observation point has been terminated, it then determines whether or not an instruction has been given to observe other observation areas (step S113). This instruction can be performed, for example, by displaying a selection screen on the video output device screen in response to the operation of the observation termination button on the observation screen 1753, and then operating an observation button for another area (not shown) located thereon. Alternatively, the operation of the observation completion button located on that selection screen can be used as an instruction not to observe other observation areas.

[0110] If the control unit 11 determines that it is time to observe other observation areas, it repeats the process described above from step S102.

[0111] If, in response, the control unit 11 determines that it will not perform observation of other areas, it communicates the termination to the patient terminal 2 via the communication interface unit 14 (step S114). Alternatively, the control unit 11 may operate as a video call unit 1110 and transmit an audio message from the voice input device 172 indicating the termination of the nurse's observation to the patient terminal 2 via the communication interface unit 14. The control unit 11 then completes the processing procedure shown in this flowchart.

[0112] Next, we will explain patient terminal 2. Figure 17 is a flowchart showing an example of the processing procedure and processing content of patient terminal 2.

[0113] As shown in Figure 17, the control unit 21 of the patient terminal 2 first operates as a silhouette information receiving unit 211 and receives silhouette information transmitted from the nurse terminal 1 via the communication interface unit 24 (step S201). The control unit 21 stores the received silhouette information in the work memory reserved in the volatile memory of the data storage unit 23.

[0114] Then, the control unit 21 operates as a silhouette information drawing unit 212 and draws the silhouette information temporarily stored in the work memory onto the screen of the video output device 275 (step S202).

[0115] Subsequently, the control unit 21 operates as a captured video acquisition unit 213 and acquires the captured video sent from the observation terminal 3 via the communication interface unit 24 or the external input / output interface unit 28, and records the received captured video in the work memory of the data storage unit 23 (step S203).

[0116] Then, the control unit 21 operates as a captured image drawing unit 214 and inserts the captured image recorded in the work memory below the silhouette drawn on the screen of the video output device 275 (step S204).

[0117] Furthermore, the control unit 21 operates as a captured video transmission unit 215 and transmits the captured video recorded in the work memory to the nurse terminal 1 via the communication interface unit 24 (step S205).

[0118] Subsequently, the control unit 21 operates as an observation point information receiving unit 216 and determines whether or not it has received the coordinates of the observation point center C, i.e., the observation point information, transmitted from the nurse terminal 1 via the communication interface unit 24 (step S206).

[0119] If the control unit 21 determines that it has not received the observation points information, it then operates as a silhouette information receiving unit 211 and determines whether or not it has received the silhouette information transmitted from the nurse terminal 1 via the communication interface unit 24 (step S207).

[0120] If the control unit 21 determines that it has not received silhouette information, it further determines, via the communication interface unit 24, whether or not a termination message has been received from the nurse terminal 1 (step S208).

[0121] If the control unit 21 determines that there is no notification of completion from the nurse terminal 1, it repeats the process described above from step S203.

[0122] In step S206 above, if it is determined that observation point information has been received, the control unit 21, which is operating as an observation point information receiving unit 216, stores the received observation point information in the work memory of the data storage unit 23. Then, the control unit 21 operates as an observation point information reflecting unit 217 and reflects the observation point information stored in the work memory in the observation video (step S209). Specifically, when the control unit 21, acting as a silhouette information drawing unit 212, draws silhouette information on the screen of the video output device 275, it draws observation point marks 27513. Also, when the control unit 21, acting as a captured video drawing unit 214, draws the captured video on the screen of the video output device 275, it adds a pre-specified blurring effect or brightness adjustment effect as a visual effect. Here, as explained for the nurse terminal 1, the visual effect is added only to the captured video so as not to interfere with the visibility of the silhouette 27512.

[0123] Subsequently, the control unit 21 repeats the process from step S202 onwards.

[0124] Furthermore, if it is determined in step S207 that silhouette information has been received, the process is repeated from step S202, assuming that an instruction has been given to observe a new observation area.

[0125] Then, in step S208, if it is determined that the nurse terminal 1 has communicated that the process is finished, the control unit 21 terminates the processing procedure shown in this flowchart.

[0126] Furthermore, in the nurse terminal 1, the still images of the captured video as observational footage (hereinafter referred to as observational images) and key information stored in the history storage unit (not shown) of the data storage unit 13 in step S112 above can be utilized, for example, as follows.

[0127] Figure 18 is a schematic diagram illustrating a group of observation images used for monitoring the progress of a wound. The history storage unit of the data storage unit 13 records observation images P1, P2, P3, ... and observation points specified at the time of shooting in chronological order. When monitoring the progress of a specific wound, it is necessary to compare observation images P1, P2, P3, ... of the same area at different observation times.

[0128] Figure 19 shows an example of a progress observation screen 1754 when emphasis is placed on image matching. For example, when examining the healing of a wound by displaying two observation images superimposed, the superimposed image 17541, with the four corners of both images aligned, can be displayed as shown in Figure 19.

[0129] Figure 20 shows an example of a progress observation screen 1754 when emphasis is placed on the matching of observation points. As shown in Figure 20, the two observation images can be displayed as a superimposed image 17541 with the positions of the observation points aligned. In this way, by using the information of the observation points to superimpose the observation images, positional discrepancies when comparing the observed parts can be reduced, and the quality of observation can be improved. In other words, by utilizing the observation points, it becomes possible to display multiple observation images for comparison. However, in this case, the size difference between the observation images cannot be reduced, so a mechanism to change the size of each observation image may be provided separately.

[0130] Furthermore, as shown in Figures 19 and 20, it is also conceivable to allow the degree to which the display approaches the matching of observation points to be switched in stages using the slider 17542.

[0131] As described above, the information presentation system SYS according to the first embodiment includes an observation terminal 3 that photographs an observation area, which is a body part of the patient, and acquires the captured image, and a patient terminal 2 for presenting the captured image acquired by the observation terminal 3 to the patient. Thus, the patient is an example of a first user, the patient terminal 2 is an example of a first terminal, the observation terminal 3 is an example of a camera, and the nurse is an example of a second user. Furthermore, the processor constituting the control unit 21 of the patient terminal 2 of the information presentation system SYS according to the first embodiment operates as a silhouette information drawing unit 212 and a captured image drawing unit 214, causing the video output device 275 of the patient terminal 2 to display an observation image on the video output device 275 of the patient terminal 2, which superimposes a visual silhouette 27512, set by the nurse, onto the captured image from the observation terminal 3, so as to reflect observation point information 217, and adds a visual effect to the observation image displayed on the video output device 275 so as to identify and display the observation target area centered on the observation point center C in the patient's observation area that the nurse should observe, as set by the nurse. Thus, the observation point center C is an example of the location of an observation point, and the processor constituting the control unit 21 of the patient terminal 2 is an example of a presentation unit and an addition unit. According to the information presentation system SYS of this first embodiment, based on line-drawing illustration-style visual information called a "silhouette," which can represent even finer details of posture of the observed area, the system communicates to the patient which part of the silhouette should be observed with emphasis through visual effects. As a result, when the patient places their body against the silhouette, the system can guide the patient to naturally align themselves with the observation points that the nurse wants to emphasize, thereby reducing the workload on the patient and making it easier for the nurse to assume the desired posture. Specifically, the nurse selects a silhouette showing the appearance of a particular observation area, such as the "back of the right hand," and superimposes it onto a video of the patient. Then, by specifying a particular area to be observed (center of the observation point), such as the "center of the back of the hand," the system applies a visual effect to the silhouette that changes gradually depending on the distance from that area. Therefore, the information presentation system SYS of the first embodiment provides an information presentation method that can achieve both the representation of finer details of posture and the visualization of areas that should be emphasized during observation.

[0132] In the information presentation system SYS according to the first embodiment, a nurse terminal 1 is further provided, which allows the nurse to specify the visual silhouette and the location of points of observation, and to display the observation video on a video output device 175. Thus, the nurse terminal 1 is an example of a second terminal. The processor constituting the control unit 11 of the nurse terminal 1 sets the center of the points of observation C by receiving a position specification operation by the nurse on the silhouette 17532 in the observation video presented on the video output device 175. In this way, according to the information presentation system SYS according to the first embodiment, the center of the points of observation C for the silhouette 27512 presented on the patient terminal 2 can be set on the silhouette 17532 presented on the nurse terminal 1.

[0133] Furthermore, in the information presentation system SYS according to the first embodiment, the processor constituting the control unit 21 of the patient terminal 2 adds observation point marks 17533 and 27513 indicating the center of the observation point C to the silhouettes 17532 and 27512 in the observation video presented to the nurse terminal 1 and the patient terminal 2. Therefore, according to the information presentation system SYS according to the first embodiment, the patient can clearly identify where the observation point the nurse wants to observe is, and the nurse can easily confirm whether the set observation point is correct or not.

[0134] Furthermore, in the information presentation system SYS according to the first embodiment, the nurse terminal 1 is further equipped with a silhouette information storage unit 131 that stores multiple visual silhouettes corresponding to multiple positions and postures for each of the multiple body positions, and the processor constituting the control unit 11 of the nurse terminal 1 accepts selection operations by the nurse using the input device 171 and sets the silhouette to be used for observation video. Therefore, according to the information presentation system SYS according to the first embodiment, the silhouette that the nurse wants to use can be easily prepared.

[0135] In the information presentation system SYS according to the first embodiment, the visual effect is a blurring effect, where the degree of blurring increases as the distance from the observation point center C increases. Therefore, with the information presentation system SYS according to the first embodiment, the patient can identify the areas that are not blurred as observation points, making it possible for the observer to easily grasp the observation points.

[0136] Alternatively, in the information presentation system SYS according to the first embodiment, the visual effect is a brightness adjustment effect, where the area becomes darker the further it is from the observation point center C. Therefore, according to the information presentation system SYS according to the first embodiment, the patient can identify the bright areas as observation points, making it possible to easily grasp the observation points for the patient.

[0137] [Second Embodiment] In the first embodiment described above, the control unit 11 of the nurse terminal 1 and the control unit 21 of the patient terminal 2 worked in cooperation. However, the information presentation system may be configured such that the nurse terminal 1 and the patient terminal 2 are merely input / output terminals with information viewing and instruction input functions, and the actual operation is performed by a server device on the network NW. This will be described below as the second embodiment.

[0138] Figure 21 is a block diagram showing an example of the configuration of an information presentation system SYS according to a second embodiment of the present invention. In this embodiment, an information presentation server 4 is connected to a network NW. The observation terminal 3 transmits captured video to the information presentation server 4.

[0139] Figure 22 is a block diagram showing an example of the hardware configuration of the information presentation server 4. The information presentation server 4 may be an information processing device such as a server computer. As shown in Figure 22, the information presentation server 4 includes a control unit 41, to which a storage unit having a program storage unit 42 and a data storage unit 43, and a communication interface unit 44 are connected via a bus 45.

[0140] The control unit 41 is a hardware processor such as a CPU. For example, by using a multi-core and multi-threaded CPU, multiple information processing operations can be performed simultaneously. The control unit 41 may also be equipped with multiple hardware processors.

[0141] The program storage unit 42 is configured by combining, for example, a non-volatile memory that can be written to and read at any time, such as an HDD, SSD, or EEPROM, as a storage medium, and a non-volatile memory such as ROM. In addition to middleware such as the OS, the program storage unit 42 stores an information presentation program, which is an application program necessary to cause the nurse terminal 1 to perform operations to observe the patient, and to cause the patient terminal 2 to perform operations to allow the patient to align the area being observed with the silhouette.

[0142] The data storage unit 43 combines, for example, a non-volatile memory that can be written to and read at any time, such as an HDD, SSD, EEPROM, or memory card, with a volatile memory such as RAM, as a storage medium.

[0143] The communication interface unit 44, under the control of the control unit 41, transmits and receives information with other devices using a communication protocol defined by the network NW.

[0144] Figure 23 is a block diagram showing an example of the software configuration of the information presentation server 4. As with Figures 3 and 6 of the first embodiment, blocks that are not specific to this information presentation system SYS have been omitted.

[0145] The control unit 41, as a processing function unit according to the second embodiment of this invention, includes a silhouette / image receiving unit 411, a silhouette information registration unit 412, a silhouette information selection unit 413, a silhouette information drawing unit 414, a captured image receiving unit 415, a captured image drawing unit 416, an image synthesis / transmission unit 417, an observation point information selection unit 418, an observation point information reflection unit 419, and a video call relay unit 4110. All of these processing function units 411 to 4110 are realized by having the hardware processor of the control unit 41 execute an information presentation program according to the second embodiment of this invention, which is stored in the program storage unit 42. At least one or at least a part of the processing functions of the above processing function units 411 to 4110 may be realized by an integrated circuit such as an ASIC, DSP, FPGA, GPU, instead of being realized by the information presentation program and the hardware processor of the control unit 11.

[0146] Furthermore, the data storage unit 43 includes a silhouette information storage unit 431 in its non-volatile memory, which is necessary for carrying out the second embodiment.

[0147] The silhouette / image receiving unit 411 receives pre-created silhouette information from an information providing device connected to the network NW via the communication interface unit 44. The silhouette / image receiving unit 411 also receives images for silhouette creation and body part information to be registered from the nurse terminal 1 via the communication interface unit 44.

[0148] The silhouette information registration unit 412 stores the silhouette information received by the silhouette / image receiving unit 411 in the silhouette information storage unit 431. The silhouette information registration unit 412 also generates a silhouette by applying image processing to the silhouette creation image received by the silhouette / image receiving unit 411, and stores the silhouette information linked to the observation area indicated by the received area information in the silhouette information storage unit 131.

[0149] The silhouette information selection unit 413 transmits screen data of a list screen showing a list of multiple silhouette information stored in the silhouette information storage unit 431 to the nurse terminal 1 via the communication interface unit 44. The silhouette information selection unit 413 then receives the silhouette information selection result from the nurse terminal 1 via the communication interface unit 44 and reads the silhouette information from the silhouette information storage unit 431 according to the selection result.

[0150] The silhouette information drawing unit 414 creates silhouette screen data by drawing the silhouette information read by the silhouette information selection unit 413.

[0151] The captured video receiving unit 415 receives captured video transmitted from the observation terminal 3 via the network NW using the communication interface unit 14.

[0152] The captured video rendering unit 416 creates captured video screen data by rendering the captured video acquired by the captured video receiving unit 415.

[0153] The video synthesis and transmission unit 417 creates observation video screen data by combining the silhouette screen data created by the silhouette information drawing unit 414 and the captured video screen data created by the captured video drawing unit 416. During this synthesis, the video synthesis and transmission unit 417 combines the two screen data so that the silhouette screen data is superimposed on the captured video screen data. The video synthesis and transmission unit 417 transmits the created observation video screen data to the nurse terminal 1 and the patient terminal 2 via the communication interface unit 44.

[0154] Upon receiving this observation video screen data, the nurse terminal 1 displays the observation video screen via the video output device 175. When observation information indicating the coordinates of observation points is specified for this observation video screen by operating the input device 171, the observation information is transmitted from the nurse terminal 1.

[0155] The observation point information selection unit 418 receives observation point information from the nurse terminal 1 via the communication interface unit 44.

[0156] The observation point information reflection unit 419 reflects the observation point information received by the observation point information selection unit 418 to the silhouette information drawing unit 414 and the captured image drawing unit 416. This reflection is as described in the first embodiment.

[0157] As a result, the observation video screen data created by the video synthesis and transmission unit 417 includes observation point markers and weighted representations indicating the areas to be observed. Therefore, when the patient terminal 2 receives this observation video screen data, the video output device 275 displays the observation video screen, and the patient can adjust the position and posture of their own observation area to match the silhouette on this observation video screen.

[0158] The video call relay unit 4110 provides video call functionality between the nurse terminal 1 and the patient terminal 2.

[0159] As described above, the information presentation system SYS according to the second embodiment includes an observation terminal 3 that photographs an observation area, which is a body part of the patient, and acquires the captured image; a patient terminal 2 for presenting the captured image acquired by the observation terminal 3 to the patient; and an information presentation server 4 that is located remotely from the patient terminal 2 and the observation terminal 3 and communicates with these terminals. The processor constituting the control unit 41 of the information presentation server 4 of the information presentation system SYS according to the first embodiment operates as a silhouette information drawing unit 414 and a captured image drawing unit 416, causing the video output device 275 of the patient terminal 2 to display an observation image on the video output device 275 of the patient terminal 2, which is formed by superimposing a visual silhouette 27512, set by the nurse, onto the captured image from the observation terminal 3, where the patient should align the position and posture of the observation area. It also operates as an observation point information reflection unit 419, adding a visual effect to the observation image presented on the video output device 275 so as to identify and display the observation target area centered on the observation point center C in the patient's observation area, which is set by the nurse and should be observed by the nurse. Thus, the observation point center C is an example of the location of an observation point, and the processor constituting the control unit 41 of the information presentation server 4 is an example of a presentation unit and an additional unit. Therefore, in the information presentation system SYS according to the second embodiment, as in the first embodiment, the patient is informed by visual effects which part of the silhouette should be observed in detail, thus providing an information presentation method that can achieve a more detailed representation of posture and visualization of the parts that should be emphasized in observation.

[0160] [Other Embodiments] However, this invention is not limited to the embodiments described above.

[0161] For example, while the first and second embodiments have been described using visual observation work in remote nursing as an example, the present invention is applicable to any task that involves transmitting instructions for physical movements in accordance with the intentions of a remote supervisor (nurse) to one or more workers (collaborators such as patients or their families), and can cover all remote collaborative work involving physical movements.

[0162] Furthermore, the flow of each process described with reference to the sequence diagram and flowchart is not limited to the procedures described. For example, although the process was described as drawing the silhouette first and then the captured video, either can be started first or simultaneously. Also, the processes of step S204 and step S205 shown in Figure 17 can be performed in reverse order or simultaneously. In this way, the order of some steps can be changed, and some steps can be performed simultaneously. Moreover, the processing content of some steps can be modified.

[0163] Furthermore, the information presentation programs stored in the program storage units 12, 22, and 42 may be transferred to each computer constituting the nurse terminal 1, patient terminal 2, and information presentation server while stored in them, or they may be transferred without being stored in each computer. In the latter case, the information presentation programs may be transferred via a network NW, or they may be transferred while recorded on a recording medium. The recording medium is a non-temporary tangible medium. The recording medium is a computer-readable medium. The recording medium may be any medium capable of storing programs and readable by a computer, such as a CD-ROM or memory card, and its form is not limited.

[0164] Although embodiments of this invention have been described in detail above, the above description is merely illustrative in all respects. It goes without saying that various improvements and modifications can be made without departing from the scope of this invention. In other words, when implementing this invention, specific configurations may be adopted as appropriate depending on the embodiment.

[0165] In short, this invention is not limited to the embodiments described above, and in the implementation stage, the components can be modified and implemented without departing from the gist of the invention. Furthermore, various inventions can be formed by appropriately combining the multiple components disclosed in the embodiments. For example, some components may be deleted from all the components shown in the embodiments. Moreover, components from different embodiments may be appropriately combined.

[0166] 1...Nurse terminal 2...Patient terminal 3...Observation terminal 4...Information display server 11, 21, 41...Control unit 12, 22, 42...Program storage unit 13, 23, 43...Data storage unit 14, 24, 44...Communication interface unit 15, 25...Input / output interface unit 16, 26, 45...Bus 28...External input / output interface unit 111, 412...Silhouette information registration unit 112, 413...Silhouette information selection unit 113, 212, 414...Silhouette information drawing unit 114...Silhouette information transmission unit 115, 415...Captured video reception unit 116, 214, 416...Captured video drawing unit 117, 418...Observation key point information selection unit 118, 217, 419...Observation key point information reflection unit 119...Observation key point information transmission unit 1110, 218...Video call unit 131, 431... Silhouette information storage unit 171, 271... Input device 172, 272... Audio input device 173, 273... Video input device 174, 274... Audio output device 175, 275... Video output device 1751... Observation area selection screen 17511... Human body diagram 17512... Next selection button 17513, 17523, 17537... Mouse pointer 17514... Identification display 1752... Silhouette selection screen 17521... Silhouette list 17522... Start instruction button 17524... Selection frame 1753, 2751... Observation screen 17531, 27511... Observation video 17532, 27512... Silhouette 17533, 27513... Observation point marker 17534, 27514... Observation terminal information 17535... Weighted representation application switch button 17536... Capture button 17538... Target circle 1754... Progress observation screen 17541... Overlay image 17542... Slider 211... Silhouette information receiving unit 213... Captured video acquisition unit 215... Captured video transmission unit 216... Observation key point information receiving unit 411... Silhouette / video receiving unit 417... Video synthesis / transmission unit 4110... Video call relay unit C... Observation key point center NW... Network OP... Observation key point P1, P2, P3... Observation image SYS... Information presentation system

Claims

1. An information presentation system comprising: a shooting device that photographs an observation area which is a body part of a first user and acquires a captured image; a first terminal for presenting the captured image acquired by the shooting device to the first user; a presentation unit that causes the first terminal to present an observation image in which a visual silhouette, set by a second user different from the first user and located remotely from the shooting device and the first terminal, is superimposed on the captured image, in which the first user should align the position and posture of the observation area; and an additional unit that adds a visual effect to the observation image presented to the first terminal so as to identify and display an observation target area centered on the position of key points of observation in the observation area of ​​the first user which the second user should observe, as set by the second user.

2. The information presentation system according to claim 1, further comprising a second terminal for which the second user specifies the location of the visual silhouette and the points of observation, and presents the observation video to the second user, wherein the second terminal sets the location of the points of observation upon receiving a location specification operation on the silhouette by the second user.

3. The information presentation system according to claim 2, wherein the additional part adds key point marks indicating the key points of observation to the silhouette in the observation video presented on the first and second terminals.

4. The information presentation system according to claim 2, further comprising a storage unit that stores multiple visual silhouettes corresponding to multiple positions and postures for each of multiple body positions, wherein the second terminal receives a selection operation by the second user and sets the silhouette to be used for the observation video.

5. The information presentation system according to any one of claims 1 to 4, wherein the visual effect is a blurring effect in which the degree of blurring increases as the point of observation moves further away.

6. The information presentation system according to any one of claims 1 to 4, wherein the visual effect is a brightness adjustment effect, where the image becomes darker the further away it is from the point of observation.

7. An information presentation method in an information presentation system having a processor, comprising: a shooting device for shooting an observation area which is a body part of a first user and acquiring a captured image; and a first terminal for presenting the captured image acquired by the shooting device to the first user, the information presentation method comprising: the processor causing the first terminal to present an observation image in which a visual silhouette, set by a second user different from the first user and located remotely from the shooting device and the first terminal, onto the captured image, in which the first user should align the position and posture of the observation area; and adding a visual effect to the observation image presented to the first terminal so as to identify and display an observation target area centered on the position of key points of observation in the observation area of ​​the first user which the second user should observe, as set by the second user.

8. An information presentation program that causes a processor in an information presentation system comprising: a shooting device that photographs an observation area which is a body part of a first user and acquires a captured image; and a first terminal for presenting the captured image acquired by the shooting device to the first user, to present an observation image to the first terminal, which is formed by superimposing a visual silhouette on the captured image, set by a second user who is located remotely from the shooting device and the first terminal and is different from the first user, onto which the first user should align the position and posture of the observation area; and an information presentation program that adds a visual effect to the observation image presented to the first terminal so as to identify and display an observation target area centered on the position of key points of observation in the observation area of ​​the first user which the second user should observe, set by the second user.

Images