Communication system, communication device, server device, communication method, communication program, and storage medium
Patent Information
- Application Number
- JP2021211288
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2021-12-24
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-12-24
AI Technical Summary
Existing communication systems face inefficiencies and privacy concerns when attempting to connect an in-vehicle device with an external terminal, as connection attempts are wasted if the driver is not present, and there is a lack of control over the display of the driver's status information without restriction.
A communication system that includes a connection information providing unit and a display control unit to manage connection information and status information display, ensuring that the driver's privacy is protected by not displaying status information after communication ends and preventing unnecessary connection attempts.
The system enables efficient and smooth communication by preventing wasteful connection attempts and ensuring the driver's privacy is maintained by controlling the display of status information after communication is completed.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
Technical Field
[0004] , , , , , , , , , ,
[0005] , , , , , ,
[0001] The present invention relates to a communication system, a communication device, a server device, a communication method, a communication program, and a storage medium, and for example, relates to a communication system, a communication device, a server device, a communication method, a communication program, and a storage medium capable of providing video and audio from a mobile body to a user.
Background Art
[0002] There is a communication system that performs communication between an in-vehicle device mounted on a vehicle and an out-vehicle terminal located outside the vehicle.
[0003] For example, Patent Document 1 discloses a system in which a voice call is made between a driver of a vehicle and an operator of an out-vehicle terminal, and when the voice call is being made, video data representing a video in front of the vehicle is transmitted from the in-vehicle device to the out-vehicle terminal and the video is displayed on the out-vehicle terminal. In Patent Document 1, the in-vehicle device starts a process of performing communication and transmitting data such as images based on receiving a call start notification, transmits a voice signal of the speech content of the driver of the vehicle to the out-vehicle terminal, and also transmits vehicle information including front image data to the out-vehicle terminal.
Prior Art Documents
Patent Documents
[0004]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0005] In a system like the one described in Patent Document 1, for example, when a call start notification is sent from an external terminal to an in-vehicle unit, if the driver is not in the vehicle equipped with the in-vehicle unit, the connection attempt by sending the call start notification may be wasted. To prevent this, it is conceivable to inform the user of the external terminal of the driver's presence before attempting to connect. However, if the presence status is known without restriction, one of the challenges is that it raises concerns from the perspective of driver privacy.
[0006] The present invention has been made in view of the above-mentioned points, and one of its objectives is to provide a communication system, communication device, server device, communication method, communication program, and storage medium that enable efficient and smooth communication between a first device inside a mobile body and a second device outside the mobile body, while giving consideration to the privacy of the operator of the mobile body and suppressing unnecessary connection attempts from the second device to the first device. [Means for solving the problem]
[0007] The invention described in claim 1 is a communication system for performing audio-video communication, which transmits audio and video from a mobile body acquired by a first device moving together with the mobile body to a second device, comprising: a connection information providing unit that provides connection information used for connecting the audio-video communication; and a display control unit that, upon receiving access to the connection information from the second device, causes the second device to display status information indicating the status of the operator of the mobile body, wherein the display control unit does not display the status information on the second device after the completion of the audio-video communication initiated using the connection information.
[0008] The invention described in claim 6 is a communication method performed by a communication system that performs audio-video communication, transmitting audio and video of a mobile body acquired by a first device moving together with the mobile body to a second device, comprising: a connection information provision step of providing connection information used for connecting the audio-video communication; and a display control step of causing the second device to display status information indicating the status of the operator of the mobile body when access to the connection information from the second device is received, characterized in that, in the display control step, the status information is not displayed on the second device after the termination of the audio-video communication initiated using the connection information.
[0009] The invention described in claim 7 is a communication program executed by a communication system that performs audio-video communication, which includes a computer and transmits audio and video from a mobile body acquired by a first device that moves together with the mobile body to a second device, the communication program causing the computer to perform a connection information provision step of providing connection information used for connecting the audio-video communication, and a display control step of causing the second device to display status information indicating the status of the operator of the mobile body when it receives access to the connection information from the second device, and in the display control step, the communication program causes the computer to control the system so that the status information is not displayed on the second device after the completion of the audio-video communication which was started using the connection information.
[0010] The invention described in claim 8 is a computer-readable storage medium that stores a communication program for causing a communication system, which includes a computer and performs audio-video communication, transmitting audio and video acquired from a mobile body by a first device moving together with the mobile body to a second device, to perform connection information used for connecting the audio-video communication, and, upon receiving access to the connection information from the second device, to display status information indicating the status of the operator of the mobile body on the second device, wherein in the display control step, after the completion of the audio-video communication initiated using the connection information, the system stores a communication program for causing the system to control the system so that the status information is not displayed on the second device.
[0011] The invention described in claim 9 is a communication device that moves with a mobile body and performs audio-video communication, acquiring audio and video from the mobile body and transmitting them to another device, comprising: a connection information providing unit that provides connection information used for connecting the audio-video communication; and a display control unit that, upon receiving access to the connection information from the other device, causes the other device to display status information indicating the status of the operator of the mobile body, wherein the display control unit does not display the status information to the other device after the termination of the audio-video communication initiated using the connection information.
[0012] The invention described in claim 10 is a server device that performs audio-video communication for transmitting audio and video from a mobile body to a second device, which is acquired by a first device moving together with the mobile body, the server device comprising: a connection information providing unit that provides connection information used for connecting the audio-video communication; and a display control unit that, upon receiving access to the connection information from the second device, causes the second device to display status information indicating the status of the operator of the mobile body, wherein the display control unit does not display the status information on the second device after the completion of the audio-video communication initiated using the connection information. [Brief explanation of the drawing]
[0013] [Figure 1] This is a schematic diagram showing an overview of a communication system according to an embodiment of the present invention. [Figure 2] This diagram shows the configuration of the front seat area of a vehicle according to the embodiment. [Figure 3] This is a block diagram showing an example of the configuration of an in-vehicle device according to the embodiment. [Figure 4] This block diagram shows an example of the configuration of a server device according to the embodiment. [Figure 5] This is a front view of the external device according to the embodiment. [Figure 6] This is a block diagram showing an example of the configuration of an external device according to the embodiment. [Figure 7] This is a sequence diagram showing an example of communication performed in the communication system according to the embodiment. [Figure 8] It is a flowchart showing an example of a routine executed by the server device according to the embodiment. [Figure 9] It is a flowchart showing an example of a routine executed by the server device according to the embodiment. [Figure 10] It is a diagram showing an example of a display screen displayed on the external device according to the embodiment. [Figure 11] It is a diagram showing an example of a display screen displayed on the external device according to the embodiment. [Figure 12] It is a diagram showing an example of a display screen displayed on the external device according to the embodiment. [Figure 13] It is a block diagram showing an example of the configuration of the in-vehicle device according to a modified example of the embodiment.
Embodiments of the Invention
[0014] Hereinafter, embodiments of the present invention will be described in detail. In the following description and the accompanying drawings, substantially the same or equivalent parts are denoted by the same reference numerals.
Embodiment
[0015] [1. Configuration of the System] The configuration of the communication system 100 including the in-vehicle device 10 according to the embodiment will be described with reference to the accompanying drawings.
[0016] FIG. 1 shows a communication system 100 which is an embodiment of the present invention. As shown in FIG. 1, the communication system 100 includes an in-vehicle device 10, a server 30, and an external device 40. In FIG. 1, a case where the in-vehicle device 10 is mounted on an automobile M as an example of a moving body is shown. Also, in FIG. 1, a smartphone is shown as an example of the external device 40.
[0017] The in-vehicle device 10, the server 30, and the external device 40 can send and receive data from each other via a network NW using communication protocols such as TCP / IP or UDP / IP. The network NW can be constructed using, for example, a mobile communication network, wireless communication such as Wi-Fi (registered trademark), and internet communication including wired communication.
[0018] In the communication system 100, audio-video communication is possible, in which audio and video acquired from the vehicle M by the in-vehicle device 10, which is a first device that moves together with the vehicle M, is transmitted to the external device 40, which is a second device.
[0019] In the communication system 100 of this embodiment, after a voice call is established between the in-vehicle device 10 and the external device 40, the video captured in the vehicle M is distributed from the in-vehicle device 10 to the external device 40.
[0020] In the following description, the communication method described above, which establishes voice communication between the in-vehicle device 10 and the external device 40 while simultaneously distributing video footage captured in the vehicle M from the in-vehicle device 10 to the external device 40, is referred to as video communication as a form of voice-video communication. In this embodiment, the in-vehicle device 10 and the external device 40 communicate via the server 30.
[0021] Through this type of video communication, the user of the external device 40, who is viewing the video transmitted from the in-vehicle device 10, can get the feeling that they are riding in the car M with the driver of the vehicle M, who is the operator of the moving object. In other words, video communication enables the user of the external device 40 to virtually ride in the car M. A system like the communication system 100 of this embodiment, which is capable of realizing such video communication, is also referred to as a virtual ride-along system.
[0022] In the following examples, the case in which the in-vehicle device 10 is a car navigation system will be described as an example. Furthermore, in the examples, the case in which the in-vehicle device 10 is a terminal device for a so-called cloud-type car navigation system, which receives a destination that the user wishes to be guided to from the user, transmits the destination to the server 30, and the server 30 generates a route to the destination will be described as an example.
[0023] Furthermore, in this embodiment, as an example of how video communication is initiated, we will describe a case in which an invitation for video communication is first made from the in-vehicle device 10 to the external device 40, and connection information for video communication is provided at the time of the invitation. In this embodiment, when the external device 40 attempts to connect using the said connection information, the process of initiating video communication proceeds.
[0024] Furthermore, in this embodiment, it will be explained that when terminating video communication, it is possible to terminate the communication from the in-vehicle device 10 by an operation to terminate video communication received by the in-vehicle device 10, and from the external device 40 by an operation to terminate video communication received by the external device 40.
[0025] Figure 2 is a perspective view showing the area around the front seats of a car M equipped with an in-vehicle device 10, which functions as a video transmission or video generation device. Figure 1 shows an example of installation where the in-vehicle device 10 is mounted inside the dashboard DB of the front seats of car M.
[0026] The GPS receiver 11 is a device that receives signals (GPS signals) from GPS (Global Positioning System) satellites. The GPS receiver 11 is located, for example, on the dashboard DB. However, the GPS receiver 11 can be located anywhere as long as it can receive GPS signals. The GPS receiver 11 is capable of transmitting the received GPS signals to the in-vehicle device 10.
[0027] The external camera 13 is an imaging device that captures images of the area in front of the vehicle M. In this embodiment, the external camera 13 is positioned on the dashboard DB so that the forward direction is the shooting direction. For example, the external camera 13 is a wide-angle camera and can capture a wide area in front of the vehicle M through the windshield. The external camera 13 may also be provided on the rearview mirror RM or mounted on the inside of the windshield FG.
[0028] The in-vehicle camera 15 is an imaging device that captures images of the interior of the automobile M. In this embodiment, the in-vehicle camera 15 is installed at the upper edge of the windshield FG or on the ceiling near the upper edge, and is capable of capturing images of the driver of the automobile M.
[0029] During video communication, images captured by the external camera 13 or the internal camera 15 are distributed to the external device 40.
[0030] The touch panel 17 is a touch panel monitor that combines a display, such as an LCD capable of displaying images, with a touchpad. The touch panel 17 is located, for example, on the center console of the dashboard DB. The touch panel 17 only needs to be located in a place that is visible to the driver and within the driver's reach. For example, the touch panel 17 may be mounted on the dashboard DB.
[0031] The touch panel 17 can display screen information based on the control of the in-vehicle device 10. The touch panel 17 can also transmit signals to the in-vehicle device 10 representing input operations received from the user. For example, the touch panel 17 may display navigation instructions. Furthermore, it may be possible to perform operations related to the navigation function, such as setting a destination, via the touch panel 17.
[0032] Furthermore, the touch panel 17 may display information related to video communication, and may also display a screen for accepting operations to start or end a connection for video communication (hereinafter also referred to as a video connection). The occupant of the vehicle M may perform the video communication connection operation by inputting information on the touch panel 17.
[0033] The speaker 19 is installed, for example, on the interior side of the A-pillar AP. The speaker 19 is capable of emitting sounds such as music and voices based on the control of the in-vehicle device 10. During video communication, the speaker 19 emits voice from the external device 40 for voice calls.
[0034] Microphone 21 is a microphone device that receives sounds from inside the vehicle and is located, for example, on the dashboard DB. Microphone 21 may be installed anywhere, such as the rearview mirror RM or the steering wheel, as long as it can receive sounds from inside the vehicle. During video communication, the audio picked up by microphone 21 is transmitted to the external device 40 as the audio for the voice call. In addition, operation inputs related to starting or ending video communication may be made by voice via microphone 21.
[0035] Figure 3 is a block diagram showing the configuration of the in-vehicle device 10. For example, the in-vehicle device 10 is a device in which a storage unit 24, a control unit 25, and a communication unit 27 cooperate via a system bus.
[0036] The seating sensor 22 is capable of actually detecting whether or not a driver is seated in (hereinafter also referred to as "occupied") the vehicle M, and is a sensor capable of outputting a signal indicating the presence or absence of the detected driver. The seating sensor 22 is, for example, installed on the seat surface or back of the driver's seat of the vehicle M, and is a sensor that detects load. The driver may be detected by image recognition of the video from the in-vehicle camera 15, or by directly detecting the driver himself using other photoelectric sensors such as infrared sensors.
[0037] Furthermore, the vehicle M is equipped with an acceleration sensor 23. The acceleration sensor 23 is capable of measuring the acceleration of the vehicle M and outputting a signal indicating the measured acceleration. For example, it may be a MEMS acceleration sensor. The acceleration sensor 23 may also be an inertial measurement unit (IMU). In addition, the acceleration sensor 23 may be built into the in-vehicle device 10.
[0038] The storage unit 24 is a storage device composed of, for example, a hard disk drive, an SSD (solid state drive), or flash memory. The storage unit 24 stores various programs executed in the in-vehicle device 10, such as the operating system and terminal software.
[0039] Various programs may be acquired, for example, from other server devices via a network, or they may be recorded on a recording medium and read via various drive devices. In other words, the various programs stored in the storage unit 24 can be transmitted via a network, and they can also be recorded on a computer-readable recording medium and transferred.
[0040] Furthermore, the memory unit 24 also stores map information, including road maps. The memory unit 24 also stores information indicating the contact details of the external device 40, which the in-vehicle device 10 uses for video communication, such as a telephone number or email address. The information indicating these contact details may also be associated with an identifier, such as a nickname, that can identify the virtual passenger who is the user of the external device 40.
[0041] The control unit 25 is composed of a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and functions as a computer. The control unit 25 realizes various functions by having the CPU read and execute various programs stored in the ROM and memory unit 24. In this embodiment, the control unit 25 provides functions such as presenting information to the driver regarding the start and end of video communication, video distribution during video communication, and car navigation.
[0042] The control unit 25 is connected to the GPS receiver 11, exterior camera 13, interior camera 15, touch panel 17, speaker 19, microphone 21, seat sensor 22, and acceleration sensor 23, which are located within the in-vehicle device 10, in a manner that allows for communication.
[0043] The control unit 25 sequentially acquires video images captured by the exterior camera 13 and the interior camera 15. The control unit 25 also sequentially acquires information indicating whether or not a driver is in the vehicle, as detected by the seat sensor 22. Furthermore, the control unit 25 sequentially acquires signals from the GPS receiver 11 and signals indicating acceleration measured by the acceleration sensor 23.
[0044] The communication unit 27 is a communication device that sends and receives data with external devices according to instructions from the control unit 25. The communication unit 27 is, for example, a NIC (Network Interface Card) for connecting to a network NW.
[0045] The communication unit 27 is connected to the network NW described above and sends and receives various data to and from the server 30. The communication unit 27 also sends and receives various data to and from the external device 40 via the server 30.
[0046] The communication unit 27 transmits information to the server 30, for example, video of the outside or inside of the automobile M, information indicating input operations performed on the in-vehicle device 10, current location information, and travel route. The communication unit 27 also receives information from the server 30, for example, information to be displayed on the touch panel 17 regarding video communication.
[0047] During video communication, the control unit 25 sequentially transmits the video captured by the external camera 13 or the internal camera 15 to the server 30 via the communication unit 27.
[0048] The control unit 25 receives display information regarding the start and end of video communication from the server 30 via the communication unit 27 and displays it on the touch panel 17. The control unit 25 may also output information regarding the start and end of video communication as audio from the speaker 19.
[0049] The control unit 25 can transmit information indicating instructions regarding the start or end of video communication, received, for example, via the touch panel 17 or microphone 21, to the server 30 via the communication unit 27.
[0050] For example, the control unit 25 of the in-vehicle device 10 can transmit location identification information, which is location data that can identify the current location of the in-vehicle device 10, to the server 30 via the communication unit 27. In this embodiment, this information pertains to the current location of the automobile M. Also, for example, the control unit 25 can transmit information including a destination entered by the user to the server 30 via the communication unit 27, and receive route information or navigation information to that destination from the server 30.
[0051] The control unit 25 can transmit audio data of voice picked up by the microphone 21 to an external device 40 via the communication unit 27 for voice communication in video communication. The control unit 25 can also receive audio data of voice input to the external device 40 via the communication unit 27 for voice communication in video communication.
[0052] Figure 4 is a block diagram showing the configuration of server 30. For example, server 30 is a device in which a storage unit 31, a communication unit 33, and a control unit 35 cooperate via a system bus.
[0053] Server 30 has functions similar to a SIP server, which establishes a voice call between the in-vehicle device 10 and the external device 40 during video communication and transfers the data of the voice call.
[0054] Furthermore, the server 30 has the function of receiving location information of the vehicle M and destination information set by the user, who is an occupant of the vehicle M, from the in-vehicle device 10, and generating a route to the destination based on the location information and destination information.
[0055] The storage unit 31 is composed of, for example, a hard disk drive and an SSD (solid state drive), and stores various programs such as the operating system and software for the server 30.
[0056] Various programs may be acquired, for example, from other server devices via a network, or they may be recorded on a recording medium and read via various drive devices. In other words, the various programs stored in the storage unit 31 can be transmitted via a network, and they can also be recorded on a computer-readable recording medium and transferred.
[0057] The memory unit 31 stores, for example, a communication program that the server 30 uses to control the connection of video communication in the communication system 100.
[0058] Furthermore, the memory unit 31 stores a map information database (not shown) in which map information, including road maps, is stored. The map information in the map information database is a database that contains information equivalent to, for example, the map information used in a navigation device.
[0059] The communication unit 33 is connected to the network NW described above and transmits and receives various data between the in-vehicle device 10 and the external device 40.
[0060] The control unit 35 is composed of a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and functions as a computer. The CPU then reads and executes various programs stored in the ROM and memory unit 31 to realize various functions.
[0061] The control unit 35 executes the above-described communication program to realize control over the video communication connection in the communication system 100.
[0062] The control unit 35 includes a connection information provision unit 36, a boarding information acquisition unit 37, and a display control unit 38.
[0063] The connection information provision unit 36 is a functional unit that generates and manages URLs (Uniform Resource Locators) as connection information used when an external device 40 performs video communication with the in-vehicle device 10.
[0064] The connection information provision unit 36 performs a connection information provision step in response to a request from the in-vehicle device 10, providing the external device 40 with a URL as connection information used for connection when the external device 40 performs video communication with the in-vehicle device 10.
[0065] For example, when the connection information providing unit 36 receives a request from the in-vehicle device 10 to send a URL to invite the external device 40 to video communication, it generates a URL that can be used for video communication with the external device 40 and provides the URL to the external device 40 by sending the URL to the external device 40. The URL is sent, for example, by sending an invitation email as described above.
[0066] The URL usable for video communication with the external device 40 is, for example, a URL that provides access to a start screen, which is an operation screen that accepts input commands to initiate video communication. In this embodiment, the start screen includes a display showing the driver's status. In other words, the start screen displayed on the external device 40 when the external device 40 accesses the URL is an operation screen that accepts commands to initiate video communication, and is also a status information display screen that shows the status of the operator of the mobile device.
[0067] In this embodiment, the connection information providing unit 36 sets an expiration date for the URL when generating it. The connection information providing unit 36 generates a URL used for video communication connections, for example, and stores the URL and its expiration date in a database. This database is stored, for example, in the storage unit 31. The connection information providing unit 36 may also obtain URLs with expiration dates from an external server or other device.
[0068] Furthermore, for example, the connection information provision unit 36 will refuse access to the page via the URL when the URL's expiration date arrives.
[0069] The expiration date of the above URL is set to a predetermined time after a request is made from the in-vehicle device 10. For example, the expiration date may be a few hours to 24 hours from the time of the request, or it may be several days.
[0070] Furthermore, the expiration date of the URL may be set by the driver for each user of the external device 40. This allows, for example, the driver of car M to set a longer expiration date for users with whom they frequently communicate via video, such as close friends, thus saving the trouble of sending an invitation each time they communicate via video. Conversely, a shorter expiration date can be set for users who do not communicate via video very often, preventing unnecessary access from those users to the driver.
[0071] Furthermore, when the video communication initiated using the URL generated as described above is completed, the connection information provision unit 36 invalidates the URL. If the URL is invalid, the external device 40 will not be able to access the URL, and the start screen, including the display showing the driver's status as described above, will not be displayed on the external device 40.
[0072] Therefore, after the video communication ends, the driver's status cannot be known to the user of the external device 40, thus protecting the driver's privacy. In addition, after the video communication ends, it becomes impossible to perform input operations to start video communication, thus preventing unwanted video communication connections by the driver.
[0073] Furthermore, when a video communication initiated using the above URL is terminated based on an operation performed on the in-vehicle device 10, the connection information provision unit 36 invalidates the URL. As described above, if the URL is invalid, the start screen, including the display showing the driver's status, will not be displayed on the external device 40.
[0074] Therefore, after video communication is terminated at the driver's discretion, the driver's status can be kept secret from the user of the external device 40. If video communication is terminated against the driver's intent, such as due to a connection failure or termination by the user of the external device 40, a display indicating the driver's status can be shown on the start screen. Thus, the driver's privacy can be appropriately protected when necessary. Furthermore, video communication can be prevented when it is highly likely that the driver does not wish to connect.
[0075] The passenger information acquisition unit 37 is a functional unit that acquires passenger status information indicating whether or not a driver is in the vehicle M, as status information indicating the driver's situation.
[0076] The passenger status information is, for example, information based on the actual detection results of whether or not a driver is in the vehicle M. For example, the passenger information acquisition unit 37 acquires video footage from the in-vehicle device 10, which includes the driver's seat area of the vehicle M, captured by the in-vehicle camera 15, and actually detects whether or not a driver is in the driver's seat by performing image recognition on the video footage. Based on this detection result, the passenger information acquisition unit 37 acquires passenger status information indicating whether or not a driver is in the vehicle.
[0077] Furthermore, for example, the passenger information acquisition unit 37 actually detects whether or not a driver is present in the driver's seat by acquiring the detection result from the seat sensor 22 installed in the automobile M, and based on the detection result, acquires passenger status information indicating whether or not a driver is in the vehicle.
[0078] Furthermore, the passenger status information is based, for example, on whether or not the in-vehicle device 10 is connected to the network NW, which is a communication network used for video communication. The passenger information acquisition unit 37 acquires the passenger status information by detecting whether or not the in-vehicle device 10 is connected to the network NW.
[0079] Furthermore, the passenger status information may also be based on, for example, whether or not the accessory power of vehicle M is turned on.
[0080] For example, a monitoring device capable of detecting whether the accessory power is ON or OFF may be installed inside the vehicle M, and the passenger information acquisition unit 37 may acquire information from the monitoring device indicating whether or not the accessory power is turned on.
[0081] Furthermore, the passenger status information may include, for example, information indicating the driver's level of composure while driving. This composure level, in other words, is determined by the busyness or workload of the driving. The passenger information acquisition unit 37 acquires passenger status information indicating the driver's level of composure while driving, for example, by using information from the in-vehicle camera 15, the exterior camera 13, the acceleration sensor 23, etc. To put it another way, this composure level can be said to be the state of whether the driver has enough composure to communicate via video while driving.
[0082] The display control unit 38 is a functional unit that, upon receiving access from an external device 40 to a URL provided by the connection information provision unit 36, executes a display control step to display boarding status information, which is status information indicating the status of the operator of the mobile vehicle, on the external device 40. In this display control step, the display control unit 38 performs control to prevent the status information from being displayed on the external device 40 after the video communication initiated using the URL provided by the connection information provision unit 36 has finished.
[0083] For example, the display control unit 38 controls the external device 40 to display a start screen including status information if the URL generated by the connection information provision unit 36 for video communication between the external device 40 and the in-vehicle device 10 is valid, and prevents the start screen including status information from being displayed on the external device 40 if the URL is invalid.
[0084] For example, the display control unit 38 determines whether the URL that was accessed is valid, and if it is valid, it causes the external device 40 to display a start screen containing boarding status information.
[0085] For example, if the above URL is valid, the display control unit 38 refers to the passenger status information acquired by the passenger information acquisition unit and determines whether or not a driver is in the vehicle M.
[0086] If the determination indicates that a driver is in the vehicle M, the display control unit 38 causes the external device 40 to display, for example, an indication that a driver is in the vehicle. If the determination indicates that a driver is not in the vehicle M, the display control unit 38 causes the external device 40 to display, for example, an indication that a driver is getting out of the vehicle.
[0087] As described above, when a video communication initiated using a URL generated by the connection information provision unit 36 ends, the connection information provision unit 36 invalidates the URL. Also, as described above, after a video communication initiated using the above URL ends based on an operation performed on the in-vehicle device 10, the connection information provision unit 36 invalidates the URL.
[0088] For example, if the URL that was accessed is invalid, the display control unit 38 will not display the start screen on the external device 40, but will instead display a message on the external device 40 indicating that the URL is invalid.
[0089] Therefore, until the video communication is completed, the user of the external device 40 can know whether or not the driver is in the vehicle, that is, whether or not video communication between the in-vehicle device 10 and the external device 40 is impossible, before attempting to connect. Thus, if the driver is out of the vehicle, it is possible to avoid situations where a connection attempt fails or the unnecessary hassle of repeatedly attempting to connect.
[0090] The driver of vehicle M can prevent situations where their passenger status information is revealed to the virtual passenger even after the video communication has ended. Therefore, the driver's privacy is protected.
[0091] Furthermore, the display control unit 38 can control the system so that the URL is invalidated after the video communication has ended, preventing the start screen for accepting start requests from being displayed on the external device 40. Therefore, for example, it is possible to prevent unnecessary video communication start requests from being made by the driver after the video communication has ended.
[0092] Furthermore, if video communication is terminated based on an operation performed on the in-vehicle device 10, it is possible that the driver intentionally terminated the video communication and does not intend to resume it afterward. On the other hand, in cases where the driver did not intentionally terminate the video communication, such as when a virtual passenger terminates the video communication or when the video communication terminates due to a connection problem, the driver may wish to resume the video communication.
[0093] Therefore, by preventing the display of status information after the video communication ends based on an operation performed on the in-vehicle device 10, it is possible to prevent the display of the start screen including status information only when the video communication is terminated by the driver. In other cases, for example, when a virtual passenger terminates the video communication or when the video communication ends due to a connection problem, the start screen including status information can be displayed again. Thus, the driver's privacy is appropriately protected when necessary.
[0094] Furthermore, in this embodiment, disabling the URL prevents access to the start screen for initiating video communication. Therefore, by disabling the URL after video communication has ended based on an operation performed on the in-vehicle device 10, it is possible to prevent the external device 40 from requesting to start video communication again only if the video communication was terminated by the driver, while allowing the driver to request to start video communication again in all other cases. Thus, it is possible to prevent the driver from unintentionally restarting video communication.
[0095] Furthermore, the control unit 35 acquires location information indicating the current location of the vehicle M from the in-vehicle device 10 via the communication unit 33. The control unit 35 also acquires destination information entered into the in-vehicle device 10 by the occupants of the vehicle M via the communication unit 33. Based on the location information and destination information, the control unit 35 generates a route to the destination and transmits information indicating the route to the in-vehicle device 10.
[0096] Furthermore, the control unit 35 transmits the video and audio received from the in-vehicle device 10 to the external device 40 via the communication unit 33. The control unit 35 also transmits the audio received from the external device 40 to the in-vehicle device 10 via the communication unit 33.
[0097] Figure 5 is a front view of the external device 40. As described above, in this embodiment, the external device 40 is a smartphone.
[0098] The touch panel 41 is a touch panel monitor that combines a display, such as a liquid crystal display capable of displaying images, with a touchpad. The touch panel 41 is capable of generating signals that represent input operations received from the user. In this embodiment, the touch panel 41 displays images distributed from the in-vehicle device 10.
[0099] Furthermore, the touch panel 41 displays information related to video communication and shows a screen for accepting operations to initiate a video communication connection. The user of the external device 40 can perform the necessary input operations to initiate video communication by inputting information to the touch panel 41.
[0100] Speaker 43 is capable of emitting sounds such as music and voice. During video communication, speaker 43 emits voice from the in-vehicle device 10 for voice communication.
[0101] Microphone 45 is a microphone device that receives sound emitted toward the external device 40. During video communication, the audio picked up by microphone 45 is transmitted to the in-vehicle device 10 via server 30 as audio for the voice call.
[0102] Figure 6 is a block diagram showing an example of the configuration of the external device 40. For example, the external device 40 is a device in which a storage unit 47, a control unit 49, and a communication unit 51 cooperate via a system bus (not shown).
[0103] The storage unit 47 is composed of, for example, a hard disk drive, an SSD (solid state drive), flash memory, etc., and stores various programs such as the operating system and software for external devices 40.
[0104] Furthermore, various programs may be acquired, for example, from other server devices via a network, or they may be recorded on a recording medium and read via various drive devices. In other words, the various programs stored in the storage unit 47 can be transmitted via a network, and they can also be recorded on a computer-readable recording medium and transferred.
[0105] The control unit 49 is composed of a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and functions as a computer. The CPU then reads and executes various programs stored in the ROM and memory unit 47 to realize various functions.
[0106] The control unit 49 is communicatively connected to the touch panel 41, speaker 43, and microphone 45.
[0107] The control unit 49 is capable of receiving signals indicating input operations to the touch panel 41 and audio input signals from the microphone 45. For example, the control unit 49 can receive requests from the user to start or end video communication via the touch panel 41 or the microphone 45.
[0108] Furthermore, the control unit 49 can transmit video or image signals to the touch panel 41 for display, or transmit audio signals to the speaker 43 for sound output.
[0109] For example, the control unit 49 causes the touch panel 41 to display a start screen for accepting a request to attempt a video communication connection, a screen indicating whether or not the driver of the vehicle M is in the vehicle, or a screen indicating that the URL is invalid.
[0110] Furthermore, for example, the control unit 49 displays the video during video communication on the touch panel 41 and outputs the audio recorded in the automobile M during video communication to the speaker 43.
[0111] The communication unit 51 is connected to the network NW described above and sends and receives various data to and from the server 30. The communication unit 51 also sends and receives various data, including video transmitted from the in-vehicle device 10, to and from the in-vehicle device 10 via the server 30.
[0112] The control unit 49 of the external device 40 can receive information via the communication unit 51 to display a start screen for accepting a request to attempt a video communication connection, a screen indicating whether or not the driver of the vehicle M is in the vehicle, or a screen indicating that the URL is invalid.
[0113] Furthermore, the control unit 49 can transmit audio data of voices picked up by the microphone 45 to the in-vehicle device 10 via the communication unit 51 for voice communication in video communication. Also, the control unit 49 can receive audio data transmitted from the in-vehicle device 10 for voice communication in video communication via the communication unit 51.
[0114] For example, the control unit 49 of the external device 40 can receive location information that can identify the current location of the vehicle M transmitted from the server 30 to the in-vehicle device 10 via the communication unit 51. Also, for example, the control unit 49 can receive route information or navigation information of the vehicle M from the server 30 via the communication unit 51.
[0115] [2. Sequence when terminating from the in-vehicle device side] Figure 7 is a sequence diagram showing an example of communication from the start of video communication between the in-vehicle device 10 and the external device 40 via the server 30 in the communication system 100, when the URL is valid, to its termination by an operation on the in-vehicle device 10.
[0116] The following describes a case where a driver, who is a user of the in-vehicle device 10, conducts video communication with a virtual passenger, who is a user of the external device 40, while driving the automobile M. For example, the driver sends an invitation email for video communication to the virtual passenger's email address in advance via the server 30. The invitation email contains a valid URL used for connecting for video communication between the in-vehicle device 10 and the external device 40, and an identifier that can identify the in-vehicle device 10.
[0117] As shown in Figure 7, when a virtual passenger taps the URL included in the invitation email using the external device 40, the external device 40 accesses the URL (step S11).
[0118] When an external device 40 accesses the URL, the server 30 checks whether the URL is valid. If the URL is determined to be valid (step S12), a browser is launched on the touch panel 41 of the external device 40, and a start screen including a display showing the driver's occupancy status is displayed (step S13).
[0119] For example, if the driver is in the vehicle, the start screen will display a message indicating that the driver is in the vehicle, along with a start button, which is a display area for accepting requests to initiate video communication from a virtual passenger.
[0120] When the virtual passenger taps the start button, a request to start video communication is received (step S14), and the request to start video communication is sent to the server 30 (step S15).
[0121] When the server 30 receives a request to start video communication, it notifies the in-vehicle device 10 that there has been a request to start video communication from the external device 40 (step S16).
[0122] In step S16, for example, when the in-vehicle device 10 receives a notification from the server 30, a selection button is displayed on the touch panel 17 indicating whether or not to consent to the start of video communication. For example, if the driver taps the consent button, a consent notification indicating that the driver consents to the start of video communication is sent to the server 30 (step S17).
[0123] When server 30 receives the acceptance notification, it executes a video communication start process to establish an audio call between the in-vehicle device 10 and the external device 40, while simultaneously distributing the video captured in the vehicle M from the in-vehicle device 10 to the external device 40 (step S18). In this way, video communication is started.
[0124] Subsequently, when an input operation is performed to instruct the in-vehicle device 10 to terminate the video communication, the in-vehicle device 10 accepts the request to terminate the video communication (step S19). This input operation to terminate the video communication may be performed via the touch panel 17 of the in-vehicle device 10, or by voice via the microphone 21.
[0125] When the termination request is received, the in-vehicle device 10 sends a video communication termination notification to the server 30 (step S20), and the server 30 sends a video communication termination notification to the external device 40 (step S21).
[0126] Furthermore, upon receiving the termination notification, the server 30 executes a video communication termination process to stop the voice call between the in-vehicle device 10 and the external device 40, and to stop the distribution of video captured in the vehicle M from the in-vehicle device 10 to the external device 40 (step S22). The in-vehicle device 10 stops transmitting images and audio. In this way, video communication is terminated by an operation on the in-vehicle device 10.
[0127] When video communication is terminated by an operation on the in-vehicle device 10, the server 30 invalidates the URL used for the terminated video communication (step S23). However, if video communication is terminated without an operation on the in-vehicle device 10, the server 30 retains the URL used for the terminated video communication as a valid URL.
[0128] [3. Server Operation] The following describes the control routines of the server 30 for realizing the operation of the communication system 100 in this embodiment. Hereinafter, the boarding information acquisition unit 37 will be described as, for example, acquiring boarding status information as needed after a video communication invitation email is sent from the server 30 to the external device 40, and maintaining the latest boarding status information.
[0129] Figure 8 is a flowchart showing the URL management routine RT1 executed by the connection information provision unit 36 of the server 30. The connection information provision unit 36 starts the URL management routine RT1, for example, when video communication is started.
[0130] When the connection information provision unit 36 starts the URL management routine RT1, it determines whether or not video communication has ended (step S101). In step S101, for example, if communication between the in-vehicle device 10 and the external device 40 is interrupted, or if the distribution of video captured in the vehicle M from the in-vehicle device 10 to the external device 40 stops, it is determined that video communication has ended.
[0131] In step S101, if the connection information providing unit 36 determines that the video communication has not ended (step S101: NO), it repeats step S101 and determines again whether or not the video communication has ended.
[0132] In step S101, the connection information providing unit 36 determines that the video communication has ended (step S101: YES), and then determines whether the termination was initiated by the in-vehicle device 10, that is, whether the termination was caused by an operation performed on the in-vehicle device 10 (step S102).
[0133] In step S102, if video communication ends due to, for example, a termination notification being sent from the in-vehicle device 10 to the server 30 (for example, step S20 in Figure 7), it is determined that the termination was initiated by the in-vehicle device.
[0134] In step S102, if video communication ends, for example, because an termination notification is sent from the external device 40 to the server 30, it is determined that the termination was not initiated by the in-vehicle device.
[0135] Furthermore, in step S102, if the video communication connection is disconnected due to, for example, a communication failure, it is determined that the termination was not initiated by the in-vehicle device.
[0136] In step S102, the connection information provision unit 36 determines that the termination was initiated by the in-vehicle device 10 (step S102: YES), and invalidates the URL used in the terminated video communication (step S103).
[0137] In step S103, for example, the connection information providing unit 36 may invalidate the URL by deleting it from the database that holds the URL.
[0138] The connection information provision unit 36 terminates the URL management routine RT1 after the execution of step S103, or if it determines in step S102 that the termination is not initiated by the in-vehicle device 10 (step S102: NO).
[0139] Figure 9 is a flowchart showing the display control routine RT2 executed by the display control unit 38 of the server 30. For example, when an invitation email is sent from the server 30 to the external device 40, the display control routine RT2 is started.
[0140] When the display control unit 38 starts the display control routine RT2, it determines whether or not there has been access to the URL included in the invitation email (step S201). In step S201, for example, if a virtual passenger opens the invitation email using the external device 40, taps the URL included in the invitation email, and the external device 40 accesses the URL, it is determined that there has been access to the URL included in the invitation email.
[0141] In step S201, if the display control unit 38 determines that there is no access to the URL (step S201: NO), it terminates the display control routine RT2, starts the next display control routine RT2, and executes step S201 again.
[0142] In step S201, the display control unit 38 determines that there has been an access to a URL (step S201: YES), and then determines whether the URL is valid or not (step S202). In step S202, for example, the display control unit 38 refers to the database where the URLs are stored and determines whether the accessed URL is valid or not.
[0143] In step S202, the display control unit 38 determines that the URL is valid (step S202: YES) and then refers to the passenger status information that the passenger information acquisition unit 37 has acquired and holds sequentially (step S203). In step S203, for example, passenger status information indicating whether or not the driver is on board is referred to.
[0144] Furthermore, for example, in step S203, information indicating whether or not the in-vehicle device 10 is connected to the network NW may be referenced, or information indicating the ON / OFF state of the seat occupancy sensor provided on the driver's seat of the automobile M may be referenced.
[0145] After step S203 is executed, the display control unit 38 determines whether or not the driver is in the vehicle based on the passenger status information referenced in step S203 (step S204).
[0146] In step S204, the display control unit 38 determines that the driver is in the vehicle (step S204: YES), and causes the external device 40 to display a screen containing a message indicating that the driver is in the vehicle (step S205).
[0147] Figure 10 shows an example of the in-ride screen displayed on the touch panel 41 of the external device 40 in step S205. As shown in Figure 10, the in-ride screen displays a message indicating that the driver is in the vehicle.
[0148] Furthermore, the in-ride screen shown in Figure 10 displays a message indicating that the driver is in the vehicle, as well as a message prompting the user to start a video call and a start button. The start button may be displayed on a different screen from the in-ride screen; for example, the in-ride screen may display a button such as "Next" or "Go to Start Screen" to proceed to the screen for starting video communication, and when this button is pressed, the start screen including the start button may be displayed.
[0149] Furthermore, a field for entering a PIN for identity verification may be displayed on the boarding screen or the start screen. For example, identity verification may be performed by entering the last four digits of a phone number into the input field.
[0150] Furthermore, the terms of service and privacy policy regarding video communication may be available on the in-ride screen or the start screen. For example, a message may be displayed stating that pressing the start button constitutes agreement to the said terms of service and privacy policy.
[0151] In step S204, if the display control unit 38 determines that the driver is not in the vehicle (step S204: NO), it causes the external device 40 to display a screen containing a message indicating that the driver is getting out of the vehicle (step S206).
[0152] Figure 11 shows an example of the "Disembarking" screen displayed on the touch panel 41 of the external device 40 in step S206. As shown in Figure 11, the "Disembarking" screen displays a message indicating that the driver is disembarking. Alternatively, as shown in Figure 11, the "Disembarking" screen may also display a message indicating that video communication cannot be started.
[0153] In the example shown in Figure 11, the display related to starting video communication, such as the start button, is shaded, and the start button is not functional. Note that the display related to starting video communication, including the start button, does not necessarily need to be shown at all on the screen while the passenger is disembarking.
[0154] If the display control unit 38 determines in step S202 that the URL is invalid (step S202: NO), it causes the external device 40 to display a screen containing a message indicating that the URL is invalid (step S207).
[0155] Figure 12 shows an example of the URL invalidation screen displayed on the touch panel 41 of the external device 40 in step S207. As shown in Figure 12, in step S207, the screen related to video communication is inaccessible, so the URL invalidation screen displays the message "URL is invalid" as a message indicating that the URL is invalid.
[0156] For example, the invalid URL screen may also display a message indicating that video communication cannot be started, or a message indicating that a new invitation email must be received in order to start video communication.
[0157] In steps S204 to S206, when the display control unit 38 receives access to the URL used for video communication from the external device 40, it executes a display control step that causes the external device 40 to display a start screen showing status information of the operator of the mobile body.
[0158] Furthermore, in step S207, the display control unit 38 executes a control to prevent the status information from being displayed on the external device 40 after the video communication initiated using the URL provided by the connection information provision unit 36 has finished.
[0159] After step S205, step S206, or step S207 is executed, the display control unit 38 terminates the display control routine RT2 and starts the next display control routine RT2.
[0160] According to the display control routine RT2, the in-car screen shown in Figure 10 or the alighting screen shown in Figure 11 is displayed, allowing the virtual passenger to know whether the driver is in the vehicle M before attempting to connect to video communication. Therefore, the virtual passenger can avoid unnecessary connection attempts.
[0161] Furthermore, the display control routine RT2 can control the system so that the boarding status information is displayed on the external device 40 only when the URL is valid, and not displayed on the external device 40 when the URL is invalid.
[0162] Furthermore, as described above, in this embodiment, the URL management routine RT1 is executed, and the URL is invalidated when the in-vehicle device 10 terminates video communication.
[0163] Therefore, after the in-vehicle device 10 terminates video communication, the passenger status information cannot be indefinitely disclosed to the virtual passenger, thus protecting the driver's privacy. For example, it is possible to prevent the passenger status information from being repeatedly disclosed even after the scheduled video communication has ended.
[0164] Furthermore, according to this embodiment, since the start screen is not displayed after the in-vehicle device 10 terminates video communication, it is possible to prevent unintended connection attempts by the driver.
[0165] Furthermore, if the URL is invalid, a message indicating that the URL is invalid, as shown in Figure 12, will be displayed, preventing the virtual passenger from accessing unnecessary connection information.
[0166] In addition, in step S203 of the above-mentioned display control routine RT2, the driver's occupancy status, or alternatively, the driver's available time while driving, may be referenced, and if the driver is in a situation where there is no available time even if they are in the vehicle, a message indicating that video communication cannot be started may be displayed.
[0167] In this embodiment, the display control unit 38 may, after the video communication has ended based on an operation performed on the in-vehicle device 10, control the system so as not to display status information on the external device 40 without invalidating the URL. For example, in step S202 of the display control routine RT2, it may be determined whether the video communication previously performed using the URL accessed in step 201 has ended based on an operation performed on the in-vehicle device 10.
[0168] In that case, if the display control unit 38 determines in step S202 that video communication has ended based on the operation performed on the in-vehicle device 10, it may proceed to step S207 and have the external device 40 display a message on the external device 40 indicating that the start screen, including the driver's occupancy status, cannot be displayed, instead of a message indicating that the URL is invalid. For example, the display control unit 38 may make the URL inaccessible by deleting the page identified by the URL.
[0169] As described above, the communication system 100 of this embodiment is capable of establishing voice communication between the in-vehicle device 10, which is the first device, and the external device 40, which is the second device, via the server 30, while simultaneously performing video communication to transmit video from the automobile M on which the in-vehicle device 10 is installed to the external device 40.
[0170] The communication system 100 of this embodiment includes a connection information providing unit used for connecting audio-video communications, including video communications, and providing connection information, and a display control unit that, upon receiving access to the connection information from an external device 40, causes the external device 40 to display status information indicating the status of the operator of the mobile body. After the completion of the audio-video communications initiated using the connection information, the display control unit performs control to prevent the status information from being displayed on the external device 40.
[0171] Therefore, for a virtual passenger, if audio and video communication has not ended, they can know the pilot's status before attempting to connect to audio and video communication, thus avoiding unnecessary connection attempts when the pilot is not on board or is not available.
[0172] For the operator of a mobile vehicle, this prevents virtual passengers from knowing the operator's status, availability, and other conditions indefinitely after the end of audio and video communication. Therefore, the operator's privacy is protected.
[0173] For example, according to this embodiment, after the termination of the audio-video communication initiated using the connection information, control can be performed to prevent the status information from being displayed on the second device by invalidating the connection information.
[0174] Furthermore, according to this embodiment, after the audio-video communication initiated using the connection information has ended based on an operation performed on the first device, the connection information can be invalidated to prevent the status information from being displayed on the second device.
[0175] Therefore, for example, if the pilot intentionally terminates video communication, the pilot's privacy is protected, while in other cases, situational information is displayed on the second device, thus reducing the workload for the virtual passenger, thus enabling smooth audio and video communication.
[0176] Furthermore, according to this embodiment, if the connection information is invalidated after the termination of audio-video communication initiated using the connection information, the second device will not display a start screen, which is an operation screen that accepts input operations to start audio-video communication, and audio-video communication using the connection information cannot be started. Therefore, it is possible to prevent, for example, a request to start audio-video communication that the driver does not want to receive after the termination of the audio-video communication.
[0177] Accordingly, according to this embodiment, it is possible to provide a communication system, communication device, server device, communication method, communication program, and storage medium that enable efficient and smooth communication between a first device inside a mobile body and a second device outside the mobile body, while giving consideration to the privacy of the operator of the mobile body and suppressing unnecessary connection attempts from the second device to the first device.
[0178] [Differentiation] Figure 13 is a block diagram showing an example of the configuration of an in-vehicle device 60 in a modified version of the communication system 100 of the embodiment. In this modified version, the in-vehicle device 60 has the functions of the connection information provision unit 36, the boarding information acquisition unit 37, and the display control unit 38 that are located in the server 30 in the above embodiment.
[0179] Specifically, the control unit 61 of the in-vehicle device 60 includes a connection information provision unit 63, a passenger information acquisition unit 64, and a display control unit 65.
[0180] The memory unit 67 stores a communication program for the control unit 61 to execute. This communication program provides a URL used for connecting to video communication, and when it receives access to the URL from the external device 40, it causes the external device 40 to display status information indicating the status of the driver of the automobile M, and after the video communication initiated using the URL ends, the control unit 61 executes a control to prevent the status information from being displayed on the external device 40.
[0181] For example, when the connection information providing unit 63 of the in-vehicle device 60 receives an operation input from the driver instructing it to send an invitation email to the external device 40, it generates a URL with an expiration date and provides connection information by sending an invitation email containing the URL to the external device 40 via the communication unit 27.
[0182] For example, the connection information providing unit 63 of the in-vehicle device 60 invalidates the provided URL after the video communication initiated using that URL has finished.
[0183] Furthermore, for example, the connection information providing unit 63 of the in-vehicle device 60 invalidates the provided URL when a video communication initiated using that URL is terminated based on an operation performed on the in-vehicle device 10.
[0184] The passenger information acquisition unit 64 sequentially acquires passenger status information, such as whether or not the driver is in the vehicle M, as situation information, based on detection results from, for example, the in-vehicle camera 15 or the seating sensor 22.
[0185] When the display control unit 65 receives access to the above URL from the external device 40, it controls the external device 40 to display occupancy status information indicating whether or not a driver is in the vehicle M. Furthermore, after the video communication initiated using the said URL has ended, the display control unit 65 controls the external device 40 so that the status information is not displayed.
[0186] For example, the display control unit 65 can perform control to prevent the start screen containing status information from being displayed on the external device 40 after the video communication has ended based on an operation performed on the in-vehicle device 10.
[0187] For example, the display control unit 65 performs control to prevent the status information from being displayed on the external device 40 if the URL is invalid.
[0188] According to this modified example, with regard to video communication between the external device 40 and the in-vehicle device 60, it is possible to control the display of status information on the external device 40 without going through the server 30, and to prevent the display of such status information on the external device 40 after the video communication has ended. For example, the control regarding the display of boarding status information and video communication in this modified example may be performed directly between the in-vehicle device 10 and the external device 40 by P2P (Peer to Peer) communication or the like, without going through the server 30.
[0189] The configurations, routines, etc., of the in-vehicle device 10, in-vehicle device 60, server 30, and external device 40 in the above-described embodiments and modifications are merely illustrative examples and can be appropriately selected or modified depending on the application.
[0190] For example, in the above embodiment and modified examples, the situation information may include not only passenger status information indicating whether or not the driver is in the vehicle, but also information indicating whether or not the driver is in a situation where video communication cannot be initiated.
[0191] For example, if the driver is communicating via video with a device other than the external device 40, if there is a passenger in the vehicle M, if the driver is under heavy driving load, or if the driver is in conversation with a passenger or the person they are communicating with, it is presumed that the driver is in a situation where they cannot respond even if video communication is initiated, or do not want to transmit video from inside or outside the vehicle or audio from inside the vehicle M to the external device 40.
[0192] In such cases, the driver's status may be treated as being unable to initiate video communication, even if they are in the vehicle. In this case, for example, the in-vehicle screen, which indicates that the driver is in the vehicle, may display a message indicating that video communication cannot be initiated.
[0193] For example, the automobile M may be equipped with seat occupancy sensors in seats other than the driver's seat, and the presence or absence of passengers used to determine the driver's occupancy status may be detected by these seat occupancy sensors.
[0194] For example, the magnitude of the driving load used to determine the driver's occupancy status may be calculated by workload estimation using images from an external camera 13 or an internal camera 15.
[0195] Furthermore, although the above embodiment describes displaying a screen on the external device 40 indicating that the URL is invalid, the invention is not limited to this, and access may also be made impossible by deleting the page identified by the URL.
[0196] In the above embodiment, the case in which the connection information provider sets an expiration date for the URL when generating the URL was described, but it is not limited to this. The connection information provider does not have to set an expiration date for the URL.
[0197] Furthermore, in the above embodiments and modifications, an example was described in which a request to start video communication from the external device 40 is received by tapping the start button displayed on the touch panel 41, but this is not limited to this. The request to start may also be received, for example, by voice input.
[0198] The above embodiments and modifications describe examples in which video communication is performed in the communication system 100, but the invention is not limited thereto. The present invention is applicable when the communication system 100 performs audio-video communication in which audio and video from the automobile M, acquired by an in-vehicle device 10 which is a first device that moves with the automobile M, are transmitted to an external device 40 which is a second device. In other words, the present invention is also applicable to communication in which no audio is transmitted from the external device 40.
[0199] Furthermore, the in-vehicle device 10 and the external device 40 do not necessarily have to be able to communicate via voice. For example, the communication between the in-vehicle device 10 and the external device 40 may be such that video and audio are simply distributed from the in-vehicle device 10 to the external device 40 without involving two-way voice communication, like YouTube® or Nico Nico Live Broadcasting®.
[0200] Furthermore, in the above embodiment, the in-vehicle device 10 and the external device 40 perform video communication via the server 30, but video communication may also be performed directly between the in-vehicle device 10 and the external device 40.
[0201] Furthermore, while the video is being distributed from the in-vehicle device 10 to the external device 40 via the communication system 100, a voice call may also be established in parallel between the in-vehicle device 10 and the external device 40 via a separate route from the communication system 100.
[0202] In the above embodiments and modifications, the in-vehicle device 10 is described as an in-vehicle navigation device, but the in-vehicle device 10 only needs to be capable of distributing video from the external camera 13 or the internal camera 15.
[0203] For example, the in-vehicle device 10 may be a configuration in which a terminal device having a similar configuration to the in-vehicle device 10, an external camera 13, and a touch panel 17 are integrated. Specifically, for example, the in-vehicle device 10 may be a terminal device such as a smartphone, tablet, or PC with a camera that has an application that performs the same functions as the in-vehicle device 10. In this case, the in-vehicle device 10 may be mounted on the dashboard DB, for example, by a cradle, so that the built-in camera can capture images of the front of the vehicle M through the windshield of the vehicle M.
[0204] Furthermore, the in-vehicle device 10 may be configured not to display a screen for the driver of the vehicle M. For example, the in-vehicle device 10 may have a configuration similar to a drive recorder, or it may be a device integrated with an external camera 13. Specifically, the in-vehicle device 10 may be a device in which the hardware performing the video communication function of the in-vehicle device 10 described above is built into the housing of the external camera 13. In this case, the in-vehicle device 10 may not perform the various display outputs described above.
[0205] In the above embodiments and modifications, the case where the external device 40 is a smartphone has been described, but it is not limited to this. The external device 40 is a terminal device that a virtual passenger can use for video communication, and it is sufficient that it is configured to display or present messages related to video communication, receive operation inputs necessary for performing video communication, send and receive audio data, and receive and display video. For example, the external device 40 may be a terminal device such as a tablet, PC, or wearable device.
[0206] In the above embodiments and modifications, examples were described in which the in-vehicle device 10 and the in-vehicle device 60 are mounted on an automobile M, but the in-vehicle device 10 may be mounted on other mobile devices such as bicycles, motorcycles, or ships. Alternatively, the in-vehicle device 10 may be a terminal device with a built-in camera, such as a smartphone, which is held by a person and used to transmit video while walking, for example. In that case, instead of indicating whether the driver is on board the mobile device, information indicating whether the user of the terminal device is holding the terminal device may be displayed on the external device 40. [Explanation of Symbols]
[0207] 10 Onboard equipment 13. Exterior car camera 15 In-car camera 17 Touch panel 19 speakers 21 Mike 23 Memory section 25 Control Unit 27 Communications Department 30 servers 31 Storage section 33 Communications Department 35 Control Unit 36, 63 Connection Information Provision Department 37, 64 Boarding Status Information Acquisition Unit 38, 65 Display Control Unit 40 External device 41 Touch panel 43 speakers 45 Mike 47 Memory section 49 Control Unit 51 Communications Department
Claims
1. A communication system for performing audio-visual communication in which audio and video of a moving object captured by a first device moving together with the moving object is transmitted to a second device, a connection information providing unit that provides connection information used for connecting the audiovisual communication; a display control unit that, when an access to the connection information from the second device is accepted, causes the second device to display status information indicating a status of a driver of the moving object; A communication system characterized in that the display control unit displays the status information on the second device when the connection information is valid, and does not display the status information on the second device when the connection information is invalid.
2. The connection information providing unit sets expiration information indicating a period during which the connection information is valid when providing the connection information, The communication system according to claim 1 , wherein the expiration date information is set by a passenger of the mobile body.
3. 3. The communication system according to claim 1, wherein the connection information providing unit invalidates the connection information after the audiovisual communication started using the connection information is terminated.
4. The communication system described in claim 3, characterized in that the connection information providing unit does not invalidate the connection information when the audio-visual communication started using the connection information is terminated without any operation performed on the first device.
5. the audiovisual communication is started by accepting an operation on an operation screen displayed on the second device in response to access to the connection information; The communication system according to claim 3 or 4, characterized in that, when the display control unit accepts access to the connection information from the second device, if the connection information is valid, the display control unit causes the second device to display the operation screen, and if the connection information is invalid, the display control unit does not cause the second device to display the operation screen.
6. 6. The communication system according to claim 1, wherein the status information includes boarding status information indicating whether or not the pilot is on board the moving object.
7. A communication method executed by a communication system that performs audio-video communication in which audio and video of a moving object captured by a first device moving together with the moving object are transmitted to a second device, the method comprising: a connection information providing step of providing connection information used for connecting the audiovisual communication; a display control step of, when access to the connection information from the second device is accepted, causing the second device to display status information indicating a status of a driver of the moving object; A communication method characterized in that, in the display control step, when the connection information is valid, the status information is displayed on the second device, and when the connection information is invalid, the status information is not displayed on the second device.
8. A communication program executed by a communication system for performing audio-video communication, the communication system including a computer, the first device moving together with a mobile object, and transmitting audio and video captured by the first device at the mobile object to a second device, the communication program comprising: a connection information providing step of providing connection information used for connecting the audiovisual communication; a display control step of causing the second device to display status information indicating a status of a driver of the moving object when the second device receives access to the connection information from the second device; A communication program for controlling, in the display control step, to display the status information on the second device when the connection information is valid, and to not display the status information on the second device when the connection information is invalid.
9. A communication system for performing audio-video communication, comprising: a first device that moves together with a mobile object, and transmits audio and video captured by the first device at the mobile object to a second device; a connection information providing step of providing connection information used for connecting the audiovisual communication; a display control step of causing the second device to display status information indicating a status of a driver of the moving object when the second device receives access to the connection information from the second device; A computer-readable storage medium that stores a communication program for controlling, in the display control step, to display the status information on the second device when the connection information is valid, and not to display the status information on the second device when the connection information is invalid.
10. A communication device that moves with a moving object, acquires audio and video in the moving object, and transmits the audio and video to another device, the communication device performing audio and video communication comprising: a connection information providing unit that provides connection information used for connecting the audiovisual communication; a display control unit that, when an access to the connection information from the other device is accepted, causes the other device to display status information indicating a status of a driver of the moving body; A communication device characterized in that the display control unit causes the other device to display the status information when the connection information is valid, and does not cause the other device to display the status information when the connection information is invalid.
11. A server device that performs audio / video communication to transmit audio and video captured by a first device moving together with a moving object to a second device, the server device comprising: a connection information providing unit that provides connection information used for connecting the audiovisual communication; a display control unit that, when an access to the connection information from the second device is accepted, causes the second device to display status information indicating a status of a driver of the moving object; A server device characterized in that the display control unit displays the status information on the second device when the connection information is valid, and does not display the status information on the second device when the connection information is invalid.
