Information processing system, information processing method, and program
The information processing system addresses setup costs and operational complexity by enabling mode-switchable remote scene display and user image control, enhancing visibility and security in remote presence applications.
Patent Information
- Application Number
- JP2023084570
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-23
- Publication Date
- 2024-12-05
- Estimated Expiration
- 2043-05-23
AI Technical Summary
Existing information processing systems for remote presence at construction sites are costly and time-consuming to set up, and require complex screen operations that can be challenging based on the IT literacy of supervisors, leading to potential failures in capturing required image data.
An information processing system that allows switching between remote scene modes, enabling easy ON/OFF control for displaying images from imaging devices, suppressing processing loads, and simplifying operations by allowing or disallowing the display of user images based on mode settings.
The system effectively reduces processing loads and simplifies operations, ensuring visibility of construction site images while preventing unauthorized data capture and server load increases, thus meeting remote presence requirements efficiently.
Smart Images

Figure 2024168137000001_ABST
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an information processing system and an information processing method, and further to a program for causing a computer to execute the information processing method. [Background technology]
[0002] In recent years, with the involvement of the Ministry of Land, Infrastructure, Transport and Tourism and other organizations, "remote presence" at construction sites is becoming more widespread. "Remote on-site inspection" refers to the inspection of a construction site from a remote location by installing fixed cameras or wearable cameras for workers at the construction site, and having supervisors check the images of the construction site captured through these cameras on information terminals, etc. According to the definition of the Ministry of Land, Infrastructure, Transport and Tourism, "remote on-site inspection" refers to the inspection of materials, stages, and attendance being carried out remotely.
[0003] The Ministry of Land, Infrastructure, Transport and Tourism has established basic matters regarding the implementation of remote on-site inspections, but the details are left to the public agencies and local governments that are the clients of the construction work. For example, some construction clients' implementation guidelines require that "screens showing both the site and the witnesses shall be saved as a record of confirmation, inspection and witnessing," making it urgent to have a camera system that can meet this requirement.
[0004] For example, Patent Document 1 discloses an information processing system that supports remote presence. [Prior art documents] [Patent documents]
[0005] [Patent Document 1] Publication No. 3234061 Summary of the Invention [Problem to be solved by the invention]
[0006] However, since the information processing system disclosed in Patent Document 1 is a dedicated system made up of a dedicated terminal for remote attendance and the like, there is a concern that it will take a lot of time and cost to build the system.
[0007] Therefore, for example, if there is an existing system that displays images captured by a wearable camera carried by a worker in a viewer on a supervisor's terminal, it is possible to use that existing system to meet the above-mentioned demand for remote presence.
[0008] Specifically, one possible method would be to start up the viewer and web conferencing system on the supervisor's terminal, display the image captured by the wearable camera carried by the worker on the viewer, and then display that screen via screen sharing on the web conferencing system, while also capturing a photo of the supervisor's face with the camera on the supervisor's terminal and displaying it as a conference participant on the web conferencing system. By taking a screenshot in this state, image data can be obtained that includes both the image of the construction site and the supervisor's face.
[0009] However, this method requires the viewer and the Web conferencing system to be launched at the same time, which places a heavy processing load on the supervisor's terminal.In addition, because it requires complicated screen operations, depending on the supervisor's IT literacy level, it may not be possible to obtain the desired image data.
[0010] An object of the present disclosure is to provide an information processing system, an information processing method, and a program that can appropriately meet the demand for remote presence by displaying video captured by an imaging device on a viewer in an information processing device. [Means for solving the problem]
[0011] An information processing system according to an aspect of the present disclosure displays an image captured by an imaging device in a viewer of an information processing device. The information processing system is capable of switching ON / OFF of a remote presence mode for remotely attending a construction site, and when the remote presence mode is ON, displays an object for permitting a user of the information processing device to capture an image in the viewer, and when the user is permitted to capture an image via the object, displays an image of the user in the viewer.
[0012] According to the above configuration, the image captured by the imaging device is displayed in the viewer of the information processing device, and the remote presence mode can be easily switched ON / OFF. This reduces the processing load of the information processing device, and allows the remote presence request to be appropriately handled with simple screen operations.
[0013] Furthermore, when the remote presence mode is OFF, the information processing system may not display the object.
[0014] According to the above configuration, when the remote presence mode is OFF, images of the object and the user are not displayed, so that a decrease in visibility of the image captured by the imaging device in the viewer can be suppressed.
[0015] In addition, the remote presence mode may be OFF in an initial setting of the information processing system.
[0016] According to the above configuration, by initially setting the remote presence mode to OFF and not displaying images of objects and users, it is possible to suppress a decrease in visibility of images captured by the imaging device in the viewer.
[0017] The information processing system may be capable of switching the remote presence mode on / off for each of the imaging devices.
[0018] According to the above configuration, it is possible to turn on the remote presence mode only for imaging devices that need to be in remote presence mode, and for imaging devices that do not need to be in remote presence mode, the remote presence mode can be turned off and images of objects and users will not be displayed, thereby preventing a decrease in visibility of the images captured by the imaging devices in the viewer.
[0019] The information processing system may be capable of setting whether or not to display a button for switching ON / OFF the remote presence mode.
[0020] According to the above configuration, for imaging devices that do not need to be in remote presence mode, a remote presence mode ON / OFF switching button is not displayed, thereby preventing a reduction in visibility of the image captured by the imaging device in the viewer.
[0021] Furthermore, when the remote presence mode is ON, the information processing system may display at least one of time information of the video captured by the imaging device and position information of the imaging device on the viewer.
[0022] According to the above configuration, in the remote presence mode, it is possible to specify on the viewer the time and place of the video captured by the imaging device.
[0023] Furthermore, the information processing system may transmit the image captured by the imaging device to the information processing device via a server, and may not transmit the image of the user to the server or the information processing device.
[0024] According to the above configuration, it is possible to respond to requests for remote presence by transmitting the image captured by the imaging device to the information processing device via the server. Also, since it is sufficient to display the user's image on the information processing device in order to respond to requests for remote presence, it is possible to prevent the leakage of personal information and the increase in the load on the server by not transmitting the user's image to the server or the information processing device.
[0025] The information processing system may be configured such that the remote presence mode cannot be switched ON when a built-in camera of the information processing device is unavailable.
[0026] According to the above configuration, when the built-in camera of the information processing device is unavailable, the object is not displayed, thereby making it possible to suppress a decrease in visibility of the image captured by the imaging device in the viewer.
[0027] In addition, in the information processing system, the imaging device may be operated by a worker at the construction site.
[0028] In addition, in the information processing system, the user may be a supervisor at the construction site.
[0029] An information processing method executed by an information processing device, the information processing device being communicatively connected to a server via a communication network and capable of switching ON / OFF of a remote presence mode for remotely attending a construction site. The server is communicatively connected to an imaging device via the communication network. The information processing method includes the steps of: when the remote presence mode is ON, displaying an object for allowing a user of the information processing device to take pictures in the viewer; and when the user is permitted to take pictures via the object, displaying an image of the user in the viewer.
[0030] According to the above configuration, the image captured by the imaging device is displayed in the viewer of the information processing device, and the remote presence mode can be switched ON / OFF. This reduces the processing load of the information processing device, and allows the remote presence request to be appropriately handled with simple screen operations.
[0031] Further, a program for causing the information processing device to execute the information processing method may be provided. Effect of the Invention
[0032] According to the present disclosure, it is possible to provide an information processing system, an information processing method, and a program capable of appropriately meeting the demand for remote presence by displaying an image captured by an imaging device on a viewer in an information processing device. [Brief description of the drawings]
[0033] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a diagram illustrating an example of a hardware configuration of a camera. [Diagram 3] FIG. 2 illustrates an example of a hardware configuration of a server. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a user terminal. [Diagram 5] FIG. 2 is a sequence diagram for explaining a series of processes executed by the information processing system according to the present embodiment. [Figure 6] 13 is a diagram showing an example of a video display screen displayed on a user terminal when a remote presence mode is set to OFF. FIG. [Figure 7] 13 is a diagram showing an example of a video display screen displayed on a user terminal when a remote presence mode is set to ON and a user's video is not displayed. FIG. [Figure 8] 13 is a diagram showing an example of a video display screen displayed on a user terminal when a remote presence mode is set to ON and a user's video is displayed. FIG. [Figure 9] FIG. 13 is a diagram illustrating an example of a setting screen of a user terminal. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0034] Hereinafter, the present embodiment will be described with reference to the drawings. For the sake of convenience, the description of the members having the same reference numbers as those already described in the description of the embodiment will be omitted. For the sake of convenience, the dimensions of each member shown in the drawings may differ from the actual dimensions of each member.
[0035] FIG. 1 is a diagram showing an example of a configuration of an information processing system according to an embodiment of the present disclosure. As shown in FIG. 1, the information processing system 1 includes a camera 2, which is an imaging device, a server 3, and a user terminal 4. These are connected to a communication network 5. For example, the camera 2 is a live camera communicatively connected to the server 3 via the communication network 5. The camera 2 is also a wearable camera attached to a part of the body of a worker O at a construction site. The camera 2 may be a mobile camera mounted on a moving object such as a vehicle or an unmanned aerial vehicle (drone), or may be a fixed camera installed at a construction site. A plurality of cameras 2 may be communicatively connected to the server 3 via the communication network 5. The user terminal 4 is communicatively connected to the server 3 via the communication network 5.
[0036] The communication network 5 is configured by at least one of, for example, a local area network (LAN), a wide area network (WAN), the Internet, and a wireless core network.
[0037] Next, the hardware configuration of camera 2 will be described below with reference to Fig. 2. Camera 2 is configured to capture an image showing a construction site, and transmit the captured image to server 3 via communication network 5 shown in Fig. 1. As shown in Fig. 2, camera 2 includes control unit 20, storage device 21, position information acquisition unit 22, RTC (Real Time Clock) 23, communication unit 24, input operation unit 25, imaging unit 26, and audio input / output unit 27. These elements are connected to communication bus 28. Also, camera 2 may have a built-in battery (not shown).
[0038] The control unit 20 includes a memory and a processor. The memory is configured to store computer-readable instructions (programs). For example, the memory is configured with a ROM (Read Only Memory) in which various programs and the like are stored, a RAM (Random Access Memory) in which various programs and the like to be executed by the processor are stored, and the like. The processor is configured with at least one of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit). The processor may be configured to load a program designated from various programs incorporated in the storage device 21 or the ROM onto the RAM, and to execute various processes in cooperation with the RAM.
[0039] The storage device 21 is, for example, a storage device (storage) such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory, and is configured to store programs and various data (video data, position information, etc.). The position information acquisition unit 22 is configured to acquire information related to the current position (longitude θ, latitude φ) of the camera 2, and is, for example, a GPS (Global Positioning System) receiver. For example, the position information acquisition unit 22 may acquire information related to the current position of the camera 2 at a predetermined time interval (for example, 10 seconds).
[0040] The RTC 23 is configured to acquire information indicating the current time. The control unit 20 associates the time information indicating the current time t acquired by the RTC 23 with the position information indicating the current position X (longitude θ, latitude φ) of the camera 2 acquired by the position information acquisition unit 22.
[0041] The communication unit 24 is configured to connect the camera 2 to the communication network 5. The communication unit 24 includes a wireless communication module for wirelessly communicating with an external device such as a base station or a wireless LAN router. The wireless communication module includes a transmitting / receiving antenna and a signal processing circuit. The wireless communication module may be a wireless communication module compatible with a short-range wireless communication standard such as Wi-Fi (registered trademark) or Bluetooth (registered trademark), or may be a wireless communication module compatible with an X-generation mobile communication system (for example, a fourth-generation mobile communication system such as LTE) using a SIM (Subscriber Identity Module).
[0042] The input operation unit 25 is configured to receive an input operation by an operator (worker O) and generate an operation signal according to the input operation by the operator. The imaging unit 26 is configured to capture an image showing the surrounding environment of the camera 2 (for example, a construction site). In particular, the imaging unit 26 is configured to generate a video signal showing the surrounding environment of the camera 2, and includes an optical system, an image sensor, and an analog processing circuit. The optical system includes, for example, an optical lens and a color filter. The image sensor includes a CCD (Charge-Coupled Device) or a CMOS (Complementary MOS), etc. The analog processing circuit is configured to process a video signal (analog signal) photoelectrically converted by the image sensor, and includes, for example, an amplifier and an AD converter. The control unit 20 generates video data (an example of image data) based on the video signal (digital signal) received from the imaging unit 26. The frame rate of the video data is, for example, 30 fps.
[0043] The audio input / output unit 27 is configured to receive audio from the surroundings of the camera 2, generate an audio signal, and convert an audio signal received from a communication destination such as the user terminal 4 into analog audio data and output it. The audio input / output unit 27 includes, for example, a microphone and a speaker.
[0044] The camera 2 can transmit images (image data stream) showing the surrounding environment of the camera 2 in real time to the server 3 via the communication network 5. The camera 2 may also transmit time information and position information of the camera 2 to the server 3 via the communication network 5.
[0045] Next, the hardware configuration of the server 3 will be described below with reference to Fig. 3. The server 3 functions as a management server that manages a camera information table related to the camera 2, video data, and a user information table related to the user. The camera information table may include the network address (e.g., IP address), identification information, setting information, and location information of one or more cameras 2. The user information table may include attribute information and contract information of the user, and identification information of the camera associated with the user.
[0046] Furthermore, the server 3 also functions as a web server that transmits data (described later) for displaying a video display screen (viewer) in response to a request from the user terminal 4. In this way, the server 3 functions as a server for providing SaaS (System as a Service) and may be composed of multiple servers. Also, the server 3 may be constructed on-premise or may be a cloud server.
[0047] The server 3 is communicatively connected to the camera 2 and the user terminal 4 via a communication network 5, and is configured to transmit video data of the camera 2 in response to a request from the user terminal 4. The server 3 includes a control unit 30, a storage device 31, a communication unit 32, an input operation unit 33, and a display unit 34. These elements are connected to a communication bus 35.
[0048] The control unit 30 includes a memory and a processor. The memory is configured to store computer-readable instructions (programs). For example, the memory is configured with a ROM and a RAM. For example, the processor is configured with at least one of a CPU, an MPU, and a GPU.
[0049] The storage device 31 is, for example, a storage device (storage) such as an HDD, SSD, or flash memory, and is configured to store programs and various data. Various databases are constructed in the storage device 31. In particular, the storage device 31 may store a camera information table related to each camera, video data captured by the camera, and a user information table related to a user.
[0050] The communication unit 32 is configured to connect the server 3 to the communication network 5. Specifically, the communication unit 32 may include various wired connection terminals for communicating with external terminals on the communication network 5. The input operation unit 33 is, for example, a touch panel, a mouse, and / or a keyboard arranged overlaid on the display unit 34, and is configured to receive an input operation by an operator and generate an operation signal according to the input operation by the operator. The display unit 34 is, for example, a liquid crystal panel or an organic EL panel.
[0051] Next, the hardware configuration of the user terminal 4 will be described below with reference to Fig. 4. The user terminal 4 may be, for example, a personal computer, a smartphone, a tablet, or a wearable device (for example, AR glasses or a head-mounted display) worn by a user U of the user terminal 4. The user terminal 4 may have a web browser for displaying data received from a web server. The user terminal 4 includes a control unit 40, a storage device 41, an imaging unit 42, a communication unit 43, an input operation unit 44, a display unit 45, and an audio input / output unit 46. These elements are connected to a communication bus 47.
[0052] The control unit 40 includes a memory and a processor. The memory is configured to store computer-readable instructions (programs). For example, the memory is configured with a ROM and a RAM. For example, the processor is configured with at least one of a CPU, an MPU, and a GPU.
[0053] The storage device 41 is, for example, a storage device (storage) such as an HDD, SSD, or flash memory, and is configured to store programs and various data. The imaging unit 42 is built into the user terminal 4 and configured to capture an image showing the user U of the user terminal 4. In particular, the imaging unit 42 is configured to generate a video signal showing the user U of the user terminal 4, and includes an optical system, an image sensor, and an analog processing circuit. The optical system includes, for example, an optical lens and a color filter. The image sensor is configured by a CCD (Charge-Coupled Device) or a CMOS (Complementary MOS), etc. The analog processing circuit is configured to process the video signal (analog signal) photoelectrically converted by the image sensor, and includes, for example, an amplifier and an AD converter. The control unit 40 generates video data (an example of image data) based on the video signal (digital signal) received from the imaging unit 42. The frame rate of the video data is, for example, 30 fps. The communication unit 43 is configured to connect the user terminal 4 to the communication network 5. The communication unit 43 includes a wireless communication module and / or a wired connection terminal for wireless communication with a base station or a wireless LAN router. The input operation unit 44 is, for example, a touch panel, a mouse, and / or a keyboard arranged on the display unit 45, and is configured to receive an input operation of the user U and generate an operation signal according to the input operation of the user U. The display unit 45 is, for example, configured of a liquid crystal panel or an organic EL panel. In this embodiment, an image display screen 50 (see FIG. 6) including an image of the camera 2 is displayed on the display unit 45. The audio input / output unit 46 is configured to receive an input of an audio signal from the surroundings of the user terminal 4, convert an audio signal received from a communication destination such as the camera 2 into analog audio data, and output the analog audio data. The audio input / output unit 46 includes, for example, a microphone and a speaker.
[0054] (A series of processes executed by the information processing system 1) Next, a series of processes executed by the information processing system 1 according to this embodiment will be described below with reference to Fig. 5. In this example, the camera 2 is associated with the user U. In other words, the user U has concluded a contract for a video providing service of the camera 2 via the communication network 5. Each process of the server 3 is executed by the control unit 30 of the server 3. Similarly, each process of the user terminal 4 is executed by the control unit 40 of the user terminal 4.
[0055] 5, in step S1, the server 3 receives video (video data) from the camera 2 through the communication network 5. The server 3 may also receive time information and location information from the camera 2 through the communication network 5. The server 3 stores the received video, time information, and location information in the storage device 31. Here, the camera 2 may continue to transmit a video data stream to the server 3 in real time, while transmitting time information and location information to the server 3 at predetermined time intervals.
[0056] Next, in step S2, the user terminal 4 transmits to the server 3 login information (e.g., a login ID and a login password) that the user U enters when logging in to the user terminal 4. In step S3, the server 3 authenticates the user U based on the login information received from the user terminal 4. Note that the authentication of the user U is not limited to login authentication, and a biometric authentication method such as face authentication or fingerprint authentication may also be applied. After authenticating the user U, the server 3 identifies the camera 2 associated with the user U by referring to the user information table.
[0057] In step S4, the server 3 transmits data for displaying the video display screen 50 shown in FIGS. 6 to 8 (hereinafter referred to as video display data) and the video data of the camera 2 to the user terminal 4 via the communication network 5. At this time, time information and position information of the camera 2 may also be transmitted to the user terminal 4.
[0058] The video display data may be a file (e.g., an HTML file, a CSS file, a program file, etc.) for displaying the video display screen 50 on the WEB browser of the user terminal 4. Here, the program file may be a JavaScript file. Note that the video display data (HTML file, etc.) may be transmitted only once. Meanwhile, the video data stream of the camera 2 continues to be transmitted in real time.
[0059] In step S5, the user terminal 4 judges whether the remote presence mode is ON or OFF. Incidentally, immediately before this, a group call may be started via a call object 51C described later, and the process may proceed to step S5. If the remote presence mode is OFF (No in step S5), in step S10, the user terminal 4 displays the video display screen 50 based on the video display data received from the server 3, and displays the video 51 of the camera 2 on the video display screen 50. Incidentally, the user terminal 4 may display at least one of the time information 51A and the position information 51B received from the server 3 on the video display screen 50.
[0060] Here, an example of the video display screen 50 displayed on the user terminal 4 when the remote presence mode is set to OFF will be described with reference to FIG. 6. As shown in FIG. 6, the video display screen 50 displays the video 51 of the camera 2, the name and device ID (not shown) of the camera 2, time information 51A, and position information 51B, a call object 51C (not shown) for starting a group call with a communication partner (for example, a worker O wearing the camera 2), and a shooting object 51D (not shown) for taking a snapshot of the video 51. At this time, the objects related to the remote presence mode and the video of the user are not displayed because they are unnecessary. In this way, the objects related to the remote presence mode and the video of the user are not displayed in the video display screen 50, so that the reduction in visibility of the video 51 in the video display screen 50 can be suppressed. Details of the objects related to the remote presence mode will be described later.
[0061] Returning to the description of Fig. 5, if the remote presence mode is set to ON in step S5 (Yes in step S5), in step S6, the user terminal 4 displays the video display screen 50 based on the video display data received from the server 3, and displays the video 51 of the camera 2 on the video display screen 50. The user terminal 4 also displays objects related to the remote presence mode on the video display screen 50. The user terminal 4 may display the video 51 of the camera 2, time information 51A, and position information 51B received from the server 3 on the video display screen 50.
[0062] Next, in step S7, the user terminal 4 judges whether the photographing object 53B is ON / OFF. If the photographing object 53B is set to ON (Yes in step S7), in step S8, the user terminal 4 displays the image 52 of the user U on the image display screen 50. In this state, for example, by clicking the photographing object 51D to take a snapshot of the image 51, it is possible to obtain a "screen showing both the site and the witnesses" required by the implementation guidelines of the construction work client. If the photographing object 53B is set to OFF (No in step S7), in step S9, the user terminal 4 does not display the image 52 of the user U on the image display screen 50, and displays the photographing object 53B in gray to indicate that the photographing object 53B is set to OFF.
[0063] Here, an example of the video display screen 50 displayed on the user terminal 4 when the remote presence mode is set to ON will be described with reference to Figs. 7 and 8. Fig. 7 is a diagram showing an example of a video display screen displayed on the user terminal when the remote presence mode is set to ON and the user's video is not displayed. Fig. 8 is a diagram showing an example of a video display screen displayed on the user terminal when the remote presence mode is set to ON and the user's video is displayed.
[0064] As shown in Figs. 7 and 8, the video display screen 50 displays the video 51 of the camera 2, the name and device ID (not shown) of the camera 2, time information 51A, and location information 51B, a call object 51C (not shown) for starting a group call with a communication partner (e.g., worker O wearing the camera 2), and a shooting object 51D (not shown) for taking a snapshot of the video 51 (including video 52 in Fig. 8). In addition, a group of objects 53 related to the remote presence mode are displayed so as to be superimposed on the video 51 of the camera 2. The group of objects 53 includes a voice output object 53A for turning on / off the voice input of the user in the user terminal 4, a shooting object 53B for turning on / off the permission of the user to shoot in the user terminal 4, and a call object 53C for ending the group call with the communication partner (e.g., worker O wearing the camera 2).
[0065] When the audio output mute function of the audio input / output unit 46 is set to ON via the audio output object 53A, audio around the user terminal 4 is not output to the group call destination, and when it is set to OFF, audio around the user terminal 4 is output to the group call destination. In the examples of Figures 7 and 8, the audio output mute function of the audio input / output unit 46 is set to ON. Note that the audio output mute function may be set to OFF as an initial setting.
[0066] When the user U is permitted to be photographed through the photographing object 53B, the user U is photographed by the imaging unit 42, and the image 52 of the user U photographed by the imaging unit 42 is displayed so as to be superimposed on the image 51 of the camera 2. Also, when the user is not permitted to be photographed, the image 52 of the user U is not displayed. In the example of FIG. 7, the photographing object 53B is set to OFF and photographing of the user U is not permitted, so the user U is not photographed by the imaging unit 42 and the image 52 of the user U is not displayed. At this time, the photographing object 53B may be displayed in gray to indicate that it is set to OFF as shown in FIG. 7. In the example of FIG. 8, the photographing object 53B is set to ON and photographing of the user U is permitted, so the user U is photographed by the imaging unit 42, and the image 52 of the user U is displayed so as to be superimposed on the image 51 of the camera 2. In this case, the name of the user U may be displayed at the bottom of the image 52 as shown in FIG. 8 to indicate that the user U is a supervisor at a construction site. The name of the user U may be a user ID stored in the storage device 41 in Fig. 4 and used for user authentication when the user U logs in to the user terminal 4. Note that the remote presence mode is initially set to OFF.
[0067] When the group call function is set to ON via the call object 51C, for example, a group call with the worker O wearing the camera 2, who is the communication partner, becomes possible. When the call function is set to OFF via the call object 53C, for example, a group call with the worker O wearing the camera 2, who is the communication partner, becomes impossible. When a group call with the camera 2 is started on a terminal other than the user terminal 4, a group call is started between three devices including the camera 2. When a device in this state ends the group call, the device is removed from the group call. In the examples of FIGS. 7 and 8, the call function is set to ON. The call function may be set to OFF in the initial setting.
[0068] In this way, when the remote presence mode is set to ON and the user U is permitted to be photographed, the video 52 of the supervisor user U can be displayed in the video display screen 50 displayed on the user terminal 4. This eliminates the need to simultaneously start multiple display screens (viewer and Web conference system), suppresses the processing load of the user terminal 4, and allows the request for remote presence to be appropriately handled with simple screen operations. In addition, when the remote presence mode is set to OFF, the photographing object 53B and the video 51 of the user U are not displayed, and when the remote presence mode is set to OFF and the user U is not permitted to be photographed, the video of the user U is not displayed. This prevents a decrease in visibility of the video 51 in the video display screen 50. In addition, the video 52 of the user U does not need to be transmitted to the server 3 or the camera 2. In order to respond to a request for remote presence, it is sufficient to display the video 52 of the user U on the user terminal 4, and by not transmitting the video 52 of the user U to the server 3 or the camera 2, it is possible to prevent the leakage of personal information and the increase in the load on the server.
[0069] Next, an example of the setting screen 60 of the user terminal 4 will be described with reference to Fig. 9. The setting screen 60 is a user interface for making various settings related to the video display screen 50. As shown in Fig. 9, the setting screen 60 includes a device setting area 70 and a remote mode setting area 80.
[0070] The device setting area 70 is an area for the user U to make settings related to various devices and systems of the user terminal 4, and specifically includes a streaming setting tab 71, a microphone setting tab 72, a microphone volume setting slider 73, a video quality setting tab 74, a live distribution mode setting tab 75, an image rotation setting tab 76, a speaker volume setting slider 77, and a time display setting tab 78. The remote mode setting area 80 is an area for the user U to make ON / OFF settings for the remote presence mode, and includes a remote presence setting tab 81.
[0071] For example, when the user U sets the streaming setting tab 71 to NO, the image 51 of the camera 2 is displayed in the image display screen 50 in a data stream format, and when the user U sets it to OFF, the image 51 of the camera 2 is displayed in the image display screen 50 in a download format. Also, for example, when the user U sets the microphone setting tab 72 to ON, the microphone of the audio input / output unit 46 of the user terminal 4 is set to ON, and the surrounding audio of the user terminal 4 input from the audio input / output unit 46 is output from the communication destination such as the camera 2. Also, for example, when the user U specifies a desired microphone volume with the microphone volume setting slider 73, the surrounding audio of the user terminal 4 is output from the communication destination such as the camera 2 at the specified volume. Also, for example, when the user U specifies a desired image quality with the image quality setting tab 74, the image 51 of the camera 2 is displayed in the image display screen 50 in the specified image quality. Also, for example, when the user U specifies a normal mode or a low latency mode of LIVE distribution with the LIVE distribution mode setting tab 75, the image 51 of the camera 2 is displayed in the specified mode on the image display screen 50. Also, for example, when the user U specifies a desired rotation angle of the image with the image rotation setting tab 76, the image 51 of the camera 2 is displayed at the specified rotation angle within the image display screen 50. Also, for example, when the user U specifies a desired volume of the speaker with the speaker volume setting slider 77, audio is output from the speaker of the audio input / output unit 46 at the specified volume. Also, for example, when the user U sets the time display setting tab 78 to NO, the current time is displayed within the image display screen 50.
[0072] For example, when the user U sets the remote presence setting tab 81 to ON, the remote presence mode is set to ON, and the object group 53 is displayed so as to be superimposed on the image 51 of the camera 2 within the image display screen 50. Also, when the user U sets the remote presence setting tab 81 to OFF, the remote presence mode is set to OFF, and the object group 53 is not displayed within the image display screen 50.
[0073] In addition, the remote presence mode is set to OFF in the initial setting. By setting the remote presence mode to OFF and not displaying the images of the object and the user in the initial setting, it is possible to suppress a decrease in the visibility of the image 51 in the image display screen 50.
[0074] In addition, when there are multiple cameras 2 capable of communicating with the user terminal 4, the remote presence mode may be switchable between ON and OFF for each camera. This makes it possible to turn on the remote presence mode only for the cameras 2 that need to be in the remote presence mode, and for the cameras 2 that do not need to be in the remote presence mode, the remote presence mode is turned off and the object group 53 and the video 52 of the user U are not displayed, thereby preventing a decrease in visibility of the video 51 of the camera 2 in the video display screen 50.
[0075] Also, it may be possible to set whether or not to display the remote presence setting tab 81, which is a button for switching the remote presence mode ON / OFF, for each camera. In this way, if the camera is not scheduled to be used for remote presence, not displaying the remote presence setting tab 81 not only suppresses a decrease in the visibility of the device setting area 70 in the setting screen 60 and the visibility of the video 51 in the video display screen 50, but also prevents the remote presence mode from being turned ON by mistake (especially effective when a camera is shared by multiple users).
[0076] Furthermore, when the imaging unit 42 of the user terminal 4 is unavailable, the remote presence mode may not be switched ON. In this way, when the imaging unit 42 of the user terminal 4 is unavailable, the object group 53 is not displayed, thereby preventing a decrease in visibility of the video 51 of the camera 2 on the video display screen 50.
[0077] In the above description, the display of the object group 53 is controlled to be turned on or off simply according to the remote presence mode being on or off. In contrast, the display of the object group 53 may be controlled further considering whether or not a group call is in progress. Specifically, when not in a group call, the object group 53 is not displayed regardless of whether the remote presence mode is on or off (for example, a screen similar to that of FIG. 6 is displayed). On the other hand, when in a group call, if the remote presence mode is on, the object group 53 is displayed (for example, a screen similar to that of FIG. 7 is displayed), and if the remote presence mode is off, the object group 53 is displayed but the photographed object 53B is removed (for example, the screen of FIG. 7 is displayed). In this way, the object group 53 and the photographed object 53B are displayed only when necessary, thereby suppressing a decrease in the visibility of the video 51 of the camera 2 in the video display screen 50.
[0078] In the above description, the display of the object group 53 is controlled to be turned on or off simply depending on whether the remote presence mode is turned on or off. In contrast, the display of the object group 53 may be controlled by further considering the authority of the user U. Specifically, when the user U has the authority of a site supervisor, the control is performed in the same manner as described above. On the other hand, when the user U does not have the authority of a site supervisor, the photographed object 53B and the video 52 of the user U are not displayed regardless of whether the remote presence mode is turned on or off. In this way, the photographed object 53B is displayed only when it is necessary, thereby suppressing a decrease in the visibility of the video 51 of the camera 2 in the video display screen 50.
[0079] In the above description, the display of the user U's video 52 is controlled to be turned on or off simply in response to the ON / OFF state of the photographing object 53B. In contrast, the display of the user U's video 52 may be controlled to be turned on or off in consideration of whether the video 51 is a live video or not. Specifically, when the video 51 is a live video, the control is performed in the same manner as described above. On the other hand, when the video 51 is not a live video (for example, it is a past video recorded in the cloud), the video 52 of the user U is not displayed. In this case, the photographing object 53B may be grayed out and the reason may be displayed instead of the video 52. This makes it impossible to display the video 52 of the user U by superimposing it on the past video after the fact, and therefore it is possible to prevent the illegal acquisition of a "screen showing both the site and the witnesses" required by the implementation guidelines of the client of the construction work.
[0080] Furthermore, in the above description, there is no particular mention of whether or not to include user U's video 52 when storing video 51 in the cloud or the like. In this regard, video 51 may be stored without including user U's video 52, or may be stored as displayed in video 51 (including user U's video 52 when it is displayed), or it may be possible to switch between these by setting.
[0081] According to this embodiment, the video of user U, who is the supervisor, captured by camera 2 is displayed in the video display screen 50 displayed on the user terminal 4. Therefore, there is no need to start multiple display screens (viewer and Web conference system) at the same time, the processing load of the user terminal 4 is reduced, and a request for remote presence can be appropriately handled with simple screen operations. In addition, since the remote presence mode can be switched ON / OFF, when the user U is not permitted to be photographed, the video of user U and the photographed object 53B are not displayed, thereby preventing a decrease in visibility of the video 51 of camera 2 in the video display screen 50.
[0082] Although the present embodiment has been described above, it goes without saying that the technical scope of the present embodiment should not be interpreted as being limited by the description of the present embodiment. The present embodiment is merely an example, and it is understood by those skilled in the art that various modifications of the embodiment are possible within the scope of the invention described in the claims. The technical scope of the present embodiment should be determined based on the scope of the invention described in the claims and its equivalents. [Explanation of symbols]
[0083] 1: Information processing system 2: Camera 3: Server 4: User terminal 5: Communication network 20: Control unit 21: Storage device 22: Location information acquisition unit 23:RTC 24: Communications Department 25: Input operation section 26: Imaging unit 27: Audio input / output section 28: Communication bus 30: Control unit 31: Storage device 32: Communications Department 33: Input operation section 34: Display section 35: Communication bus 40: Control unit 41:Storage device 43: Communications Department 44: Input operation section 45: Display section 46: Audio input / output section 47: Communication bus 50: Video display screen 60: Setting screen 70: Device setting area 80: Remote mode setting area
Claims
1. An information processing system for displaying an image captured by an imaging device on a viewer in an information processing device, The remote presence mode for remotely observing a construction site can be switched ON / OFF. When the remote presence mode is ON, an object for permitting a user of the information processing device to take a photograph is displayed on the viewer; If the user is permitted to be photographed via the object, an image of the user is displayed on the viewer. Information processing system.
2. The information processing system includes: When the remote presence mode is OFF, the object is not displayed. The information processing system according to claim 1 .
3. The information processing system includes: The remote presence mode is OFF in the initial setting. The information processing system according to claim 1 .
4. The information processing system includes: The remote presence mode can be switched ON / OFF for each of the imaging devices. The information processing system according to claim 1 .
5. The information processing system includes: It is possible to set whether or not to display a button for switching the remote presence mode ON / OFF. The information processing system according to claim 1 .
6. The information processing system includes: When the remote presence mode is ON, at least one of time information of the video captured by the imaging device and position information of the imaging device is displayed on the viewer. The information processing system according to claim 1 .
7. The information processing system includes: The image captured by the imaging device is transmitted to the information processing device via a server; The image of the user is not transmitted to the server or the information processing device. The information processing system according to claim 1 .
8. The information processing system includes: When the built-in camera of the information processing device is unavailable, the remote presence mode cannot be switched to ON. The information processing system according to claim 1 .
9. The information processing system includes: The imaging device is operated by a worker at the construction site. The information processing system according to claim 1 .
10. The information processing system includes: The user is a supervisor at the construction site. The information processing system according to claim 1 .
11. The information processing system includes: A group call including the imaging device and the information processing device is possible, When the group call is not in progress, the object is not displayed in the viewer regardless of whether the remote presence mode is ON or OFF. The information processing system according to claim 1 .
12. The information processing system includes: It is possible to set permissions for each user. If the user does not have the authority of a site supervisor, the object is not displayed in the viewer regardless of whether the remote presence mode is ON or OFF. The information processing system according to claim 1 .
13. The information processing system includes: It is possible to display live and past footage, When the past video is being displayed, the video of the user is not displayed on the viewer. The information processing system according to claim 1 .
14. An information processing method executed by an information processing device, The information processing device is communicably connected to a server via a communication network, and is capable of switching ON / OFF of a remote presence mode for remotely attending a construction site; the server is communicatively connected to the imaging device via a communication network; The information processing method includes: displaying an object for permitting a user of the information processing device to take a photograph in a viewer when the remote presence mode is ON; displaying an image of the user on the viewer when the user is permitted to be photographed via the object; Including, Information processing methods.
15. A program for causing the information processing device to execute the information processing method according to claim 14.
Citation Information
Patent Citations
Image acquisition program, image acquisition device, and image acquisition method
JP2015070483A
Safety management system
JP2022040981A
Remote support system
JP2022042303A
Remote on-site inspection supervision system and device
JP3234061U
