COMMUNICATION SYSTEM, COMMUNICATION DEVICE, SERVER DEVICE, COMMUNICATION METHOD, COMMUNICATION PROGRAM, AND STORAGE MEDIUM

Abstract

To provide a communication system, a communication device, a server device, a communication method, a communication program, and a storage medium, for making it possible, in communication between a first device in a mobile body and a second device outside the mobile body, to implement efficient smooth communication by suppressing a wasteful connection trial from the second device to the first device while considering privacy of an operator of the mobile body.SOLUTION: A communication system 100 for performing voice video communication of transmitting voice and video at a mobile body acquired by a first device moving together with the mobile body to a second device comprises: a connection information provision unit that provides connection information to be used for connection of the voice video communication; and a display control unit that when receiving access from the second device to the connection information, makes the second device display state information showing a state of an operator of the mobile body. After termination of the voice video communication started using the connection information, the display control unit inhibits the second device from displaying the state information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a communication system, a communication device, a server device, a communication method, a communication program, and a storage medium, and more particularly to a communication system, a communication device, a server device, a communication method, a communication program, and a storage medium that can provide a user with video and audio from a mobile object, for example. [Background technology]

[0002] 2. Description of the Related Art There is a communication system that performs communication between an on-board device mounted in a vehicle and an external terminal located outside the vehicle.

[0003] For example, Patent Document 1 discloses a system in which a voice call is made between a driver of a vehicle and an operator of an external terminal, and while the voice call is being made, image data showing an image of the area in front of the vehicle is transmitted from an in-vehicle device to the external terminal, and the image is displayed on the external terminal. Patent Document 1 discloses that, upon receiving a call start notification, the in-vehicle device starts processing for transmitting data such as a call and an image, transmits an audio signal of the speech of the driver of the vehicle to the external terminal, and also transmits vehicle information including image data of the area in front of the vehicle to the external terminal. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-213791 Summary of the Invention [Problem to be solved by the invention]

[0005] In a system such as that disclosed in Patent Document 1, for example, when a call start notification is sent from an external terminal to an in-vehicle device, if the driver is not in the vehicle in which the in-vehicle device is installed, the connection attempt by sending the call start notification may be in vain. To prevent this, it is conceivable to inform the user of the external terminal of the driver's riding status before the connection attempt. In such a case, one of the issues is that if the riding status is made known without restriction, it will be problematic from the perspective of the driver's privacy.

[0006] The present invention has been made in consideration of the above-mentioned points, and one of its objectives is to provide a communication system, communication device, server device, communication method, communication program, and storage medium that enable efficient and smooth communication between a first device inside a mobile body and a second device outside the mobile body by suppressing unnecessary connection attempts from the second device to the first device while taking into consideration the privacy of the operator of the mobile body. [Means for solving the problem]

[0007] The invention described in claim 1 is a communication system for performing audio-visual communication in which audio and video of a mobile object acquired by a first device moving together with the mobile object is transmitted to a second device, the system including a connection information providing unit that provides connection information used for connecting the audio-visual communication, and a display control unit that, upon receiving access to the connection information from the second device, causes the second device to display status information indicating the status of the operator of the mobile object, wherein the display control unit: If the connection information is valid, the status information is displayed on the second device, and if the connection information is invalid, The status information is not displayed on the second device.

[0008] Claim 7The invention described in is a communication method executed by a communication system that performs audio-visual communication in which audio and video of a mobile object acquired by a first device that moves together with the mobile object is transmitted to a second device, and includes a connection information providing step of providing connection information used for connecting the audio-visual communication, and a display control step of, when access to the connection information from the second device is accepted, causing the second device to display status information indicating the status of the operator of the mobile object, wherein in the display control step: If the connection information is valid, the status information is displayed on the second device, and if the connection information is invalid, The status information is not displayed on the second device.

[0009] Claim 8 The invention described in is a communication program executed by a communication system that includes a computer and performs audio-visual communication in which audio and video of a mobile body acquired by a first device moving together with the mobile body is transmitted to a second device, the program causing the computer to execute a connection information providing step of providing connection information used for connecting the audio-visual communication, and a display control step of causing the second device to display status information indicating the status of the operator of the mobile body when access to the connection information from the second device is accepted, and in the display control step: If the connection information is valid, the status information is displayed on the second device, and if the connection information is invalid, The communication program is for controlling the second device so that the status information is not displayed on the second device.

[0010] Claim 9 The invention described in the item (1) is a communication system that includes a computer and performs audio-visual communication in which audio and video of a mobile body acquired by a first device moving together with the mobile body is transmitted to a second device. The system executes a connection information providing step of providing connection information used for connecting the audio-visual communication, and a display control step of displaying status information indicating a status of a driver of the mobile body on the second device when access to the connection information from the second device is accepted, and in the display control step, If the connection information is valid, the status information is displayed on the second device, and if the connection information is invalid, A computer-readable storage medium stores a communication program for controlling the second device not to display the status information.

[0011] Claim10 The invention described in is a communication device that moves together with a moving body and performs audio-visual communication by acquiring audio and video from the moving body and transmitting the audio and video to another device, the communication device having a connection information providing unit that provides connection information used for connecting the audio-visual communication, and a display control unit that, when receiving access to the connection information from the other device, causes the other device to display status information indicating the status of the operator of the moving body, the display control unit If the connection information is valid, the status information is displayed on the other device, and if the connection information is invalid, The status information is not displayed on the other device.

[0012] Claim 11 The invention described in is a server device that performs audio-visual communication by transmitting audio and video of a mobile object acquired by a first device that moves together with the mobile object to a second device, the server device having a connection information providing unit that provides connection information used for connecting the audio-visual communication, and a display control unit that, upon receiving access to the connection information from the second device, causes the second device to display status information indicating the status of the operator of the mobile object, wherein the display control unit: If the connection information is valid, the status information is displayed on the second device, and if the connection information is invalid, The status information is not displayed on the second device. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a schematic diagram illustrating an overview of a communication system according to an embodiment of the present invention. [Figure 2] 1 is a diagram showing a configuration of a front seat portion of a vehicle according to an embodiment; [Figure 3] 1 is a block diagram showing an example of a configuration of an in-vehicle device according to an embodiment; [Figure 4] FIG. 2 is a block diagram illustrating an example of a configuration of a server device according to an embodiment. [Figure 5] FIG. 2 is a front view of an external device according to an embodiment. [Figure 6] FIG. 2 is a block diagram illustrating an example of a configuration of an external device according to an embodiment. [Figure 7] FIG. 2 is a sequence diagram illustrating an example of communication performed in a communication system according to an embodiment. [Figure 8]10 is a flowchart illustrating an example of a routine executed by a server device according to an embodiment. [Figure 9] 10 is a flowchart illustrating an example of a routine executed by a server device according to an embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a display screen displayed on an external device according to an embodiment. [Figure 11] FIG. 10 is a diagram illustrating an example of a display screen displayed on an external device according to an embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of a display screen displayed on an external device according to an embodiment. [Figure 13] FIG. 10 is a block diagram showing an example of the configuration of an in-vehicle device according to a modified example of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following description and accompanying drawings, substantially the same or equivalent parts are designated by the same reference numerals. [Example]

[0015] [1. System Configuration] The configuration of a communication system 100 including an in-vehicle device 10 according to an embodiment will be described with reference to the accompanying drawings.

[0016] Fig. 1 shows a communication system 100 according to an embodiment of the present invention. As shown in Fig. 1, the communication system 100 includes an in-vehicle device 10, a server 30, and an external device 40. Fig. 1 shows a case in which the in-vehicle device 10 is mounted on an automobile M, which is an example of a moving body. Fig. 1 also shows a smartphone as an example of the external device 40.

[0017] The in-vehicle device 10, the server 30, and the external device 40 can transmit and receive data to and from each other via a network NW using a communication protocol such as TCP / IP or UDP / IP. The network NW can be constructed using, for example, a mobile communication network, wireless communication such as Wi-Fi (registered trademark), and internet communication including wired communication.

[0018] In the communication system 100, audio and video communication is possible in which audio and video captured by an in-vehicle device 10 as a first device traveling together with the automobile M is transmitted to an external device 40 as a second device.

[0019] In the communication system 100 of this embodiment, after a voice call is established between the in-vehicle device 10 and the external device 40, the in-vehicle device 10 distributes a video image taken in the automobile M to the external device 40.

[0020] In the following description, the communication mode in which a voice call is established between the in-vehicle device 10 and the external device 40 and video captured in the automobile M is distributed from the in-vehicle device 10 to the external device 40 as described above will be referred to as video communication as one mode of audio / video communication. In this embodiment, the in-vehicle device 10 and the external device 40 perform video communication via the server 30.

[0021] By performing such video communication, the user of the external device 40 watching the video transmitted from the in-vehicle device 10 can feel as if he or she is riding in the vehicle M with the driver of the vehicle M who is operating the moving object. In other words, the video communication can realize a virtual ride-along with the user of the external device 40 in the vehicle M. A system such as the communication system 100 of this embodiment that can realize such video communication is also called a virtual ride-along system.

[0022] In the following embodiments, the in-vehicle device 10 is described as a car navigation device. Also, the in-vehicle device 10 is described as a terminal device of a so-called cloud-based car navigation device, which receives a destination to which the user wants guidance from the user, transmits the destination to the server 30, and the server 30 generates a route to the destination.

[0023] In this embodiment, as an example of how video communication is started, a case will be described in which the in-car device 10 first invites the external device 40 to start video communication, and connection information for the video communication is provided at the time of the invitation. In this embodiment, when a connection attempt is made from the external device 40 using the connection information, the process of starting video communication proceeds.

[0024] In addition, in this embodiment, when terminating video communication, it is possible to terminate the video communication from the in-vehicle device 10 side by an operation to instruct the in-vehicle device 10 to terminate the video communication, or from the external device 40 side by an operation to instruct the in-vehicle device 10 to terminate the video communication, which is accepted by the external device 40.

[0025] Fig. 2 is a perspective view showing the vicinity of the front seat of an automobile M equipped with an in-vehicle device 10 as a video transmission device or a video generation device. Fig. 1 shows, as an example of installation, a case where the in-vehicle device 10 is installed inside a dashboard DB of the front seat of the automobile M.

[0026] The GPS receiver 11 is a device that receives signals (GPS signals) from GPS (Global Positioning System) satellites. The GPS receiver 11 is disposed, for example, on a dashboard DB. The GPS receiver 11 may be disposed anywhere as long as it can receive GPS signals. The GPS receiver 11 is capable of transmitting the received GPS signals to the in-vehicle device 10.

[0027] The exterior camera 13 is an imaging device that captures images in front of the automobile M. In this embodiment, the exterior camera 13 is disposed on the dashboard DB so that the imaging direction faces forward. For example, the exterior camera 13 is a wide-angle camera that can capture images of a wide area in front of the automobile M through the windshield. The exterior camera 13 may be provided on the rearview mirror RM or attached to the inside of the windshield FG.

[0028] The in-vehicle camera 15 is an imaging device that captures images of the interior of the automobile M. In this embodiment, the in-vehicle camera 15 is provided on the upper end of the windshield FG or on the ceiling near the upper end, and is capable of capturing images of the driver of the automobile M.

[0029] During video communication, the video captured by the vehicle exterior camera 13 or the vehicle interior camera 15 is distributed to the external device 40.

[0030] The touch panel 17 is, for example, a touch panel monitor that combines a display, such as a liquid crystal display, capable of displaying images, with a touchpad. The touch panel 17 is disposed, for example, in the center console of the dashboard DB. The touch panel 17 may be disposed in a location that is visible to the driver and within the driver's reach. For example, the touch panel 17 may be attached to the dashboard DB.

[0031] The touch panel 17 can display a screen based on the control of the in-vehicle device 10. The touch panel 17 can also transmit a signal representing an input operation to the touch panel 17 received from a user to the in-vehicle device 10. For example, the touch panel 17 may display car navigation guidance. Furthermore, operations related to the car navigation function, such as setting a destination, may be possible via the touch panel 17.

[0032] Furthermore, the touch panel 17 may display information about video communication, or may display a screen for accepting operations to start or end a connection for video communication (hereinafter also referred to as a video connection). The occupant of the automobile M may perform an operation to connect video communication by inputting an operation to the touch panel 17.

[0033] The speaker 19 is provided, for example, on the interior side of the A-pillar AP. The speaker 19 is capable of emitting sounds such as music and voices under the control of the in-vehicle device 10. During video communication, the speaker 19 emits sound from the external device 40 during a voice call.

[0034] The microphone 21 is a microphone device that picks up sounds inside the vehicle, and is arranged, for example, on the dashboard DB. The microphone 21 may be arranged anywhere, such as on the rearview mirror RM or the steering wheel, as long as it can pick up sounds inside the vehicle. During video communication, the sound picked up by the microphone 21 is transmitted to the external device 40 as the sound of the voice call. Furthermore, an operation input for starting or ending the video communication may be made by voice via the microphone 21.

[0035] 3 is a block diagram showing the configuration of the in-vehicle device 10. For example, the in-vehicle device 10 is a device in which a storage unit 24, a control unit 25, and a communication unit 27 cooperate with each other via a system bus.

[0036] The seating sensor 22 is a sensor that can actually detect whether or not the driver of the automobile M is in the car (hereinafter also referred to as "on board"), and is capable of outputting, for example, a signal indicating the presence or absence of the detected driver. The seating sensor 22 is provided, for example, on the seat surface or back of the driver's seat of the automobile M, and is a sensor that detects load. Note that the driver may be detected by image recognition of the image from the in-vehicle camera 15, or by directly detecting the driver himself using another photoelectric sensor such as an infrared sensor.

[0037] The automobile M is also equipped with an acceleration sensor 23. The acceleration sensor 23 may be, for example, a MEMS acceleration sensor that is capable of measuring the acceleration of the automobile M and outputting a signal indicating the measured acceleration. The acceleration sensor 23 may also be an inertial measurement unit (IMU). The acceleration sensor 23 may also be built into the in-vehicle device 10.

[0038] The storage unit 24 is a storage device configured by, for example, a hard disk drive, a solid state drive (SSD), a flash memory, etc. The storage unit 24 stores various programs executed in the in-vehicle device 10, such as an operating system and terminal software.

[0039] The various programs may be acquired, for example, from another server device or the like via a network, or may be recorded on a recording medium and read via various drive devices. In other words, the various programs stored in the storage unit 24 can be transmitted via a network, or can be recorded on a computer-readable recording medium and transferred.

[0040] The storage unit 24 also stores map information including road maps. The storage unit 24 also stores information indicating contact information of the external device 40 as a party with which the in-vehicle device 10 will perform video communication, such as a telephone number or an email address. The contact information may be associated with an identifier such as a nickname that can identify the virtual passenger who is the user of the external device 40.

[0041] The control unit 25 is configured with a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and functions as a computer. The control unit 25 realizes various functions by the CPU reading and executing various programs stored in the ROM and the storage unit 24. In this embodiment, the control unit 25 performs functions such as a function of presenting information regarding the start and end of video communication to the driver, a function of distributing video during video communication, and a car navigation function.

[0042] The control unit 25 is communicatively connected to the GPS receiver 11, the exterior camera 13, the interior camera 15, the touch panel 17, the speaker 19, the microphone 21, the seating sensor 22, and the acceleration sensor 23 provided within the in-vehicle device 10.

[0043] The control unit 25 sequentially acquires images captured by the exterior camera 13 and the interior camera 15. The control unit 25 also sequentially acquires information indicating whether or not a driver is in the vehicle, which is detected by the seating sensor 22. The control unit 25 also sequentially acquires a signal from the GPS receiver 11 and a signal indicating acceleration measured by the acceleration sensor 23.

[0044] The communication unit 27 is a communication device that transmits and receives data to and from external devices in accordance with instructions from the control unit 25. The communication unit 27 is, for example, a network interface card (NIC) for connecting to the network NW.

[0045] The communication unit 27 is connected to the above-mentioned network NW, and transmits and receives various data to and from the server 30. The communication unit 27 also transmits and receives various data to and from the external device 40 via the server 30.

[0046] The communication unit 27 transmits, for example, images of the exterior or interior of the automobile M, information indicating input operations performed on the in-vehicle device 10, current location information, travel route, and other information to the server 30. The communication unit 27 also receives, for example, information to be displayed on the touch panel 17 regarding video communication from the server 30.

[0047] During the video communication, the control unit 25 sequentially transmits the images captured by the exterior camera 13 or the interior camera 15 to the server 30 via the communication unit 27 .

[0048] The control unit 25 receives display information relating to the start and end of video communication from the server 30 via the communication unit 27 and displays it on the touch panel 17. The control unit 25 may also output information relating to the start and end of video communication by sound from the speaker 19.

[0049] The control unit 25 can transmit, to the server 30 via the communication unit 27, information indicating an instruction regarding the start or end of video communication received via the touch panel 17 or the microphone 21, for example.

[0050] For example, the control unit 25 of the in-vehicle device 10 can transmit, via the communication unit 27, to the server 30, location identification information as location data that can identify the current location of the in-vehicle device 10, that is, in this embodiment, information on the current location of the automobile M. Furthermore, for example, the control unit 25 can transmit, via the communication unit 27, to the server 30, information including a destination input by a user, and receive, from the server 30, route information or navigation information to the destination.

[0051] The control unit 25 can transmit audio data of the audio picked up by the microphone 21 to the external device 40 via the communication unit 27 for an audio call in the video communication. The control unit 25 can also receive audio data of the audio input to the external device 40 via the communication unit 27 for an audio call in the video communication.

[0052] 4 is a block diagram showing the configuration of the server 30. For example, the server 30 is a device in which a storage unit 31, a communication unit 33, and a control unit 35 cooperate with each other via a system bus.

[0053] The server 30 has a function similar to that of a SIP server that establishes a voice call between the in-car device 10 and the external device 40 during video communication and transfers data of the voice call.

[0054] The server 30 also has the function of receiving from the in-vehicle device 10 the location identification information of the automobile M and information on the destination set by the user who is an occupant of the automobile M, and generating a route to the destination based on the location identification information and the destination information.

[0055] The storage unit 31 is configured with, for example, a hard disk drive and an SSD (solid state drive), and stores various programs such as an operating system and software for the server 30.

[0056] The various programs may be acquired, for example, from another server device or the like via a network, or may be recorded on a recording medium and read via various drive devices. That is, the various programs stored in the storage unit 31 can be transmitted via a network, or can be recorded on a computer-readable recording medium and transferred.

[0057] The storage unit 31 stores, for example, a communication program that the server 30 uses to control video communication connections in the communication system 100.

[0058] The storage unit 31 also stores a map information database (not shown) that stores map information including road maps. The map information in the map information database is a database that contains information equivalent to the map information used in, for example, a navigation device.

[0059] The communication unit 33 is connected to the network NW, and transmits and receives various data between the in-vehicle device 10 and the external device 40.

[0060] The control unit 35 is configured with a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and functions as a computer. The CPU reads and executes various programs stored in the ROM and the storage unit 31, thereby realizing various functions.

[0061] The control unit 35 executes the above-mentioned communication program to realize control relating to video communication connections in the communication system 100.

[0062] The control unit 35 includes a connection information providing unit 36, a boarding information acquiring unit 37, and a display control unit 38.

[0063] The connection information providing unit 36 ​​is a functional unit that generates and manages a URL (Uniform Resource Locator) as connection information used for connection when the external device 40 performs video communication with the in-car device 10.

[0064] The connection information providing unit 36 ​​executes a connection information providing step in response to a request from the in-vehicle device 10, in which the connection information providing unit 36 ​​provides the external device 40 with a URL as connection information used for connecting when the external device 40 performs video communication with the in-vehicle device 10.

[0065] For example, when the connection information providing unit 36 ​​receives a request from the in-car device 10 to send a URL for inviting the external device 40 to video communication, the connection information providing unit 36 ​​generates a URL that can be used for video communication with the external device 40 and transmits the URL to the external device 40, thereby providing the URL to the external device 40. The URL is transmitted, for example, by transmitting an invitation email as described above.

[0066] A URL that can be used for video communication with external device 40 is, for example, a URL that can access a start screen that serves as an operation screen that accepts an input operation for starting video communication. In this embodiment, the start screen includes a display showing the driver's status. In other words, the start screen that is displayed on external device 40 when external device 40 accesses the URL is an operation screen that accepts an operation for starting video communication, and also serves as a status information display screen that shows the status of the operator of the moving object.

[0067] In this embodiment, when generating a URL, the connection information providing unit 36 ​​sets an expiration date for the URL. The connection information providing unit 36 ​​generates, for example, a URL used for connecting for video communication, and stores the URL and its expiration date in a database in association with each other. The database is stored in, for example, the storage unit 31. The connection information providing unit 36 ​​may also obtain a URL with an expiration date from an external device such as a server.

[0068] Furthermore, for example, when the expiration date of a URL arrives, the connection information providing unit 36 ​​denies access to the page using that URL.

[0069] The expiration date of the above URL is set, for example, to the time when a predetermined time has elapsed since the request was made from the in-vehicle device 10. For example, the expiration date may be a few hours to 24 hours from the time of the request, or may be a few days.

[0070] The driver may also be able to set the URL expiration date for each user of the external device 40. This allows the driver of the automobile M, for example, to set a longer expiration date for users with whom he or she frequently conducts video communications, such as close friends, thereby eliminating the need to send an invitation every time a video communication is conducted. Also, for users with whom he or she does not conduct video communications very frequently, the driver may set a shorter expiration date, thereby preventing unnecessary access from such users to the driver.

[0071] Furthermore, when the video communication started using the URL generated as described above ends, the connection information providing unit 36 ​​invalidates the URL. If the URL is invalid, the external device 40 will no longer be able to access the URL, and the start screen including the display indicating the driver's status as described above will not be displayed on the external device 40.

[0072] Therefore, after the video communication ends, the driver's situation can be kept secret from the user of external device 40, thereby protecting the driver's privacy. Furthermore, after the video communication ends, input operations for starting video communication are no longer possible, which prevents video communication connections that the driver does not want.

[0073] Furthermore, when the video communication started using the above URL is ended based on an operation performed on the in-vehicle device 10, the connection information providing unit 36 ​​invalidates the URL. As described above, when the URL is invalid, the start screen including the display showing the driver's status is not displayed on the external device 40.

[0074] Therefore, after the video communication is ended at the driver's will, the driver's status can be hidden from the user of external device 40. If the video communication is ended unintentionally by the driver, such as due to a connection failure or when the video communication is ended by the user of external device 40, a display indicating the driver's status can be displayed on the start screen. Therefore, the driver's privacy can be appropriately protected when necessary. Also, the video communication can be prevented from being connected when there is a high possibility that the driver does not want it.

[0075] The boarding information acquisition unit 37 is a functional unit that acquires boarding status information indicating whether or not the driver is in the automobile M as status information indicating the driver's status.

[0076] The boarding status information is, for example, information based on the result of actually detecting whether or not the driver is in the automobile M. For example, the boarding information acquisition unit 37 acquires from the in-vehicle device 10 an image of an area including the driver's seat of the automobile M captured by the in-vehicle camera 15, and actually detects whether or not the driver is in the driver's seat by performing image recognition on the image. Based on the detection result, the boarding information acquisition unit 37 acquires boarding status information indicating whether or not the driver is in the automobile M.

[0077] Furthermore, for example, the boarding information acquisition unit 37 actually detects whether or not a driver is present in the driver's seat by acquiring the detection results from the seating sensor 22 provided in the automobile M, and acquires boarding status information indicating whether or not the driver is on board based on the detection results.

[0078] The boarding status information is information based on whether the in-vehicle device 10 is connected to a network NW, which is a communication network used for video communication. The boarding information acquisition unit 37 acquires the boarding status information by detecting whether the in-vehicle device 10 is connected to the network NW.

[0079] Furthermore, the boarding status information may be information based on whether or not the accessory power supply of the automobile M is turned on, for example.

[0080] For example, a monitoring device capable of detecting whether the accessory power supply is on or off may be installed inside the automobile M, and the boarding information acquisition unit 37 may acquire information indicating whether the accessory power supply is on or off from the monitoring device.

[0081] The boarding status information may also include information indicating the driver's level of composure while driving. In other words, the level of composure is determined by how busy the driver is or how high the workload is. The boarding information acquisition unit 37 acquires boarding status information indicating the driver's level of composure while driving, using information from the in-vehicle camera 15, the outside-vehicle camera 13, the acceleration sensor 23, etc. In other words, the level of composure can be said to be a state in which the driver has the composure to have a video call and talk while driving.

[0082] The display control unit 38 is a functional unit that executes a display control step of causing the external device 40 to display boarding status information as status information indicating the status of the operator of the mobile object, when the display control unit 38 receives access from the external device 40 to the URL provided by the connection information providing unit 36. In the display control step, the display control unit 38 performs control so that the status information is not displayed on the external device 40 after the video communication started using the URL provided by the connection information providing unit 36 ​​is completed.

[0083] For example, the display control unit 38 controls the external device 40 to display a start screen including a display of status information when the URL generated by the connection information providing unit 36 ​​for video communication between the external device 40 and the in-vehicle device 10 is valid, and controls the external device 40 not to display a start screen including a display of status information when the URL is invalid.

[0084] For example, the display control unit 38 determines whether the accessed URL is valid, and if it is valid, causes the external device 40 to display a start screen including boarding status information.

[0085] For example, if the above URL is valid, the display control unit 38 refers to the boarding status information acquired by the boarding information acquisition unit and determines whether the driver is in the automobile M or not.

[0086] If the result of the determination is that a driver is in the automobile M, the display control unit 38 causes the external device 40 to display, for example, a display indicating that the driver is in the automobile M. If the result of the determination is that a driver is not in the automobile M, the display control unit 38 causes the external device 40 to display, for example, a display indicating that the driver is getting out of the automobile M.

[0087] As described above, when the video communication started using the URL generated by the connection information providing unit 36 ​​is terminated, the connection information providing unit 36 ​​invalidates the URL. Also, as described above, after the video communication started using the URL is terminated based on an operation performed on the in-car device 10, the connection information providing unit 36 ​​invalidates the URL.

[0088] For example, when the URL for which access has been accepted is invalid, the display control unit 38 does not cause the external device 40 to display the start screen, but causes the external device 40 to display a message indicating that the URL is invalid.

[0089] Therefore, until the video communication is finished, the user of the external device 40 can know before attempting a connection whether the driver is in the vehicle or not, i.e., whether video communication between the in-vehicle device 10 and the external device 40 is impossible or not. Therefore, when the driver is out of the vehicle, it is possible to avoid a situation where a connection attempt is not successful or unnecessary trouble such as having to retry the connection multiple times.

[0090] The driver of the automobile M can prevent the boarding status information from being revealed to the virtual passengers even after the video communication has ended, thereby protecting the privacy of the driver.

[0091] Furthermore, the display control unit 38 can perform control such that the URL is invalidated after the video communication has ended, so that the start screen for accepting a start request is not displayed on the external device 40. Therefore, for example, it is possible to prevent a request to start a video communication that is unnecessary for the driver from being made after the video communication has ended.

[0092] Furthermore, when video communication is ended based on an operation performed on the in-vehicle device 10, it is conceivable that the driver intentionally ended the video communication and does not intend to resume the video communication thereafter. On the other hand, when the driver did not intentionally end the video communication, for example, when a virtual passenger ended the video communication or when the video communication was ended due to a poor connection, the driver may wish to resume the video communication.

[0093] Therefore, by not displaying the status information after the video communication is ended based on an operation performed on the in-vehicle device 10, the display of the start screen including the status information is prevented only when the video communication is ended by the driver, and in other cases, such as when the virtual passenger ends the video communication or when the video communication is ended due to a poor connection, the start screen including the status information can be displayed again. Therefore, the driver's privacy is appropriately protected when necessary.

[0094] Furthermore, in this embodiment, if the URL is invalidated, the start screen for starting the video communication cannot be accessed. Therefore, by invalidating the URL after the video communication has ended based on an operation performed on the in-vehicle device 10, a request to start the video communication again from the external device 40 can be prevented only when the video communication has been ended by the driver, and a request to start the video communication again can be made in other cases. Therefore, it is possible to prevent the video communication from being restarted unintentionally by the driver.

[0095] The control unit 35 also acquires location information indicating the current location of the automobile M from the in-vehicle device 10 via the communication unit 33. The control unit 35 also acquires destination information input to the in-vehicle device 10 by an occupant of the automobile M from the in-vehicle device 10 via the communication unit 33. The control unit 35 generates a route to the destination based on the location information and the destination information, and transmits information indicating the route to the in-vehicle device 10.

[0096] The control unit 35 also transfers the video and audio received from the in-vehicle device 10 to the external device 40 via the communication unit 33. The control unit 35 also transfers the audio received from the external device 40 to the in-vehicle device 10 via the communication unit 33.

[0097] 5 is a front view of the external device 40. As described above, in this embodiment, the external device 40 is a smartphone.

[0098] The touch panel 41 is, for example, a touch panel monitor that combines a display, such as a liquid crystal display, capable of displaying images, with a touch pad. The touch panel 41 is capable of generating signals that represent input operations to the touch panel 41 received from a user. In this embodiment, images distributed from the in-vehicle device 10 are displayed on the touch panel 41.

[0099] Furthermore, information about the video communication and a screen for accepting an operation to start a video communication connection are displayed on the touch panel 41. The user of the external device 40 can perform an input operation to start the video communication by inputting information to the touch panel 41.

[0100] The speaker 43 is capable of emitting sounds such as music and voices. During video communication, the speaker 43 emits the sound from the in-car device 10 during voice communication.

[0101] The microphone 45 is a microphone device that receives sounds emitted toward the external device 40. During video communication, the sounds collected by the microphone 45 are transmitted to the in-car device 10 via the server 30 as voice calls.

[0102] 6 is a block diagram showing an example of the configuration of the external device 40. For example, the external device 40 is a device in which a storage unit 47, a control unit 49, and a communication unit 51 cooperate with each other via a system bus (not shown).

[0103] The storage unit 47 is configured by, for example, a hard disk drive, a solid state drive (SSD), a flash memory, etc., and stores various programs such as an operating system and software for the external device 40.

[0104] The various programs may be acquired, for example, from another server device or the like via a network, or may be recorded on a recording medium and read via a drive device. That is, the various programs stored in the storage unit 47 can be transmitted via a network, or can be recorded on a computer-readable recording medium and transferred.

[0105] The control unit 49 is configured with a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and functions as a computer. The CPU reads and executes various programs stored in the ROM and the storage unit 47, thereby realizing various functions.

[0106] The control unit 49 is communicably connected to the touch panel 41, the speaker 43, and the microphone 45.

[0107] The control unit 49 can receive signals indicating input operations on the touch panel 41 and audio input signals from the microphone 45. For example, the control unit 49 can accept a request to start or end video communication made by the user via the touch panel 41 or the microphone 45.

[0108] Furthermore, the control unit 49 can transmit a video or image signal to the touch panel 41 to display it, and can transmit an audio signal to the speaker 43 to output sound.

[0109] For example, the control unit 49 causes the touch panel 41 to display a start screen for accepting a request to attempt a video communication connection, a screen indicating whether the driver of the automobile M is in the car, or a screen indicating that the URL is invalid.

[0110] Furthermore, for example, the control unit 49 causes the touch panel 41 to display an image during the video communication, and causes the speaker 43 to output audio collected in the automobile M during the video communication.

[0111] The communication unit 51 is connected to the above-mentioned network NW, and transmits and receives various data to and from the server 30. The communication unit 51 also transmits and receives various data, including video transmitted from the in-vehicle device 10, to and from the in-vehicle device 10 via the server 30.

[0112] The control unit 49 of the external device 40 can receive information via the communication unit 51 to display a start screen for accepting a request to attempt a video communication connection, a screen indicating whether the driver of the automobile M is in the car, or a screen indicating that the URL is invalid.

[0113] Furthermore, the control unit 49 can transmit audio data of the audio picked up by the microphone 45 to the in-vehicle device 10 via the communication unit 51 for audio communication in video communication. The control unit 49 can also receive audio data transmitted from the in-vehicle device 10 via the communication unit 51 for audio communication in video communication.

[0114] For example, the control unit 49 of the external device 40 can receive, via the communication unit 51, location identification information capable of identifying the current location of the automobile M transmitted from the in-vehicle device 10 from the server 30. Furthermore, for example, the control unit 49 can receive, via the communication unit 51, route information or navigation information for the automobile M from the server 30.

[0115] [2. Sequence when terminating from the in-vehicle device] FIG. 7 is a sequence diagram showing an example of communication from the start of video communication between the in-vehicle device 10 and the external device 40 via the server 30 in the communication system 100 when the URL is valid, to the end of the communication by operating the in-vehicle device 10.

[0116] The following describes a case where a driver, who is a user of the in-vehicle device 10, performs video communication with a virtual passenger, who is a user of the external device 40, while driving the automobile M. For example, the driver sends an invitation email for video communication to the email address of the virtual passenger in advance via the server 30. The invitation email includes a valid URL used for connection for video communication between the in-vehicle device 10 and the external device 40 and an identifier that can identify the in-vehicle device 10.

[0117] As shown in FIG. 7, when the virtual passenger taps the URL included in the invitation email using the external device 40, the external device 40 accesses the URL (step S11).

[0118] When the URL is accessed from the external device 40, the server 30 checks whether the URL is valid. If it is determined that the URL is valid (step S12), a browser is launched on the touch panel 41 of the external device 40, and a start screen including a display showing the driver's riding status is displayed (step S13).

[0119] For example, when the driver is in the car, the start screen displays a message indicating that the driver is in the car, along with a start button, which is a display area for accepting a request to start video communication from a virtual passenger.

[0120] When the virtual passenger taps the start button, a request to start video communication is accepted (step S14), and the request to start video communication is transmitted to the server 30 (step S15).

[0121] When the server 30 receives the request to start video communication, it notifies the in-car device 10 that there has been a request to start video communication from the external device 40 (step S16).

[0122] In step S16, for example, when the in-vehicle device 10 receives a notification from the server 30, a selection button for whether to accept or decline the start of video communication is displayed on the touch panel 17. For example, when the driver taps the accept button, an acceptance notification indicating that the driver has accepted the start of video communication is transmitted to the server 30 (step S17).

[0123] Upon receiving the acceptance notification, the server 30 executes a video communication start process to establish a voice call between the in-vehicle device 10 and the external device 40, and causes the in-vehicle device 10 to distribute the video captured in the automobile M to the external device 40 (step S18). In this way, the video communication is started.

[0124] Thereafter, when an input operation to instruct the in-vehicle device 10 to end the video communication is performed, the in-vehicle device 10 accepts the request to end the video communication (step S19). The input operation to instruct the in-vehicle device 10 to end the video communication may be performed via the touch panel 17 of the in-vehicle device 10, or may be performed by voice via the microphone 21.

[0125] When the end request is accepted, the in-car device 10 transmits a video communication end notification to the server 30 (step S20), and the server 30 transmits a video communication end notification to the external device 40 (step S21).

[0126] Furthermore, upon receiving the end notification, the server 30 executes a video communication end process to terminate the voice call between the in-vehicle device 10 and the external device 40, and stops the distribution of the video captured in the automobile M from the in-vehicle device 10 to the external device 40 (step S22). The in-vehicle device 10 stops transmitting images and audio. In this way, the video communication is terminated by operating the in-vehicle device 10.

[0127] When the video communication is terminated by an operation on the in-vehicle device 10, the server 30 invalidates the URL used in the terminated video communication (step S23). Note that when the video communication is terminated without an operation on the in-vehicle device 10, the server 30 holds the URL used in the terminated video communication as a valid URL.

[0128] [3. Server Operation] The following describes a control routine of the server 30 for realizing the operation of the communication system 100 of the embodiment. The following description is based on the assumption that the boarding information acquisition unit 37 acquires boarding status information as needed and stores the latest boarding status information after, for example, an invitation email for video communication is sent from the server 30 to the external device 40.

[0129] 8 is a flowchart showing a URL management routine RT1 executed by the connection information providing unit 36 ​​of the server 30. The connection information providing unit 36 ​​starts the URL management routine RT1 when, for example, video communication is started.

[0130] When the URL management routine RT1 starts, the connection information providing unit 36 ​​determines whether the video communication has ended (step S101). In step S101, it is determined that the video communication has ended, for example, when the communication between the in-vehicle device 10 and the external device 40 is disconnected or when the distribution of the video captured in the automobile M from the in-vehicle device 10 to the external device 40 is stopped.

[0131] In step S101, if the connection information providing unit 36 ​​determines that the video communication has not ended (step S101: NO), it repeats step S101 to determine again whether or not the video communication has ended.

[0132] In step S101, when the connection information providing unit 36 ​​determines that the video communication has ended (step S101: YES), it determines whether the end was initiated from the in-vehicle device 10, i.e., whether the end was due to an operation performed on the in-vehicle device 10 (step S102).

[0133] In step S102, if the video communication is ended by, for example, an end notification from the in-car device 10 to the server 30 (for example, step S20 in FIG. 7), it is determined that the end was initiated by the in-car device.

[0134] In step S102, if the video communication is ended by, for example, an end notification from the external device 40 to the server 30, it is determined that the end was not initiated by the in-vehicle device.

[0135] Furthermore, in step S102, if the video communication connection is cut off due to a communication failure, for example, it is determined that the termination was not initiated by the in-vehicle device.

[0136] In step S102, when the connection information providing unit 36 ​​determines that the termination was initiated by the in-vehicle device 10 (step S102: YES), it invalidates the URL used in the terminated video communication (step S103).

[0137] In step S103, for example, the connection information providing unit 36 ​​may invalidate the URL by deleting the URL from a database that holds the URL.

[0138] After executing step S103, or when it is determined in step S102 that the termination was not initiated by the vehicle-mounted device 10 (step S102: NO), the connection information providing unit 36 ​​terminates the URL management routine RT1.

[0139] 9 is a flowchart showing a display control routine RT2 executed by the display control unit 38 of the server 30. For example, when an invitation email is sent from the server 30 to the external device 40, the display control routine RT2 starts.

[0140] When the display control routine RT2 starts, the display control unit 38 determines whether or not a URL included in the invitation email has been accessed (step S201). In step S201, for example, if the virtual passenger opens the invitation email using the external device 40, taps the URL included in the invitation email, and the external device 40 accesses the URL, it is determined that the URL included in the invitation email has been accessed.

[0141] In step S201, when the display control unit 38 determines that the URL has not been accessed (step S201: NO), it ends the display control routine RT2, starts the next display control routine RT2, and executes step S201 again.

[0142] In step S201, when the display control unit 38 determines that a URL has been accessed (step S201: YES), it determines whether the URL is valid (step S202). In step S202, for example, the display control unit 38 refers to a database in which URLs are stored, and determines whether the accessed URL is valid.

[0143] In step S202, when the display control unit 38 determines that the URL is valid (step S202: YES), the display control unit 38 refers to the boarding status information sequentially acquired and stored by the boarding information acquisition unit 37 (step S203). In step S203, for example, the boarding status information indicating whether or not the driver is on board is referred to.

[0144] Also, for example, in step S203, information indicating whether the in-vehicle device 10 is connected to the network NW may be referenced, or information indicating the ON / OFF state of a seating sensor provided in the driver's seat of the automobile M may be referenced.

[0145] After executing step S203, the display control unit 38 determines whether or not the driver is currently in the vehicle based on the boarding status information referred to in step S203 (step S204).

[0146] In step S204, when the display control unit 38 determines that the driver is in the vehicle (step S204: YES), it causes the external device 40 to display a screen including a message indicating that the driver is in the vehicle (step S205).

[0147] 10 is a diagram showing an example of a riding screen displayed in step S205 on the touch panel 41 of the external device 40. As shown in Fig. 10, the riding screen displays a message indicating that the driver is riding.

[0148] 10 displays a message stating that the driver is in the vehicle, as well as a message prompting the user to start a video communication call and a start button. The start button may be displayed on a screen different from the in-vehicle screen. For example, a button such as "Next" or "Go to start screen" for proceeding to a screen for starting video communication may be displayed on the in-vehicle screen, and pressing the button may display a start screen including the start button.

[0149] The in-boarding screen or the start screen may display a PIN entry field for identity verification. For example, identity verification may be performed by entering the last four digits of a phone number in the entry field.

[0150] In addition, the terms of use and privacy policy for video communication may be viewable from the in-ride screen or start screen, and for example, a message may be displayed indicating that pressing the start button constitutes agreement to the terms of use and privacy policy.

[0151] In step S204, when the display control unit 38 determines that the driver is not in the vehicle (step S204: NO), it causes the external device 40 to display a screen including a message indicating that the driver is getting off the vehicle (step S206).

[0152] Fig. 11 is a diagram showing an example of a dismounting screen displayed on the touch panel 41 of the external device 40 in step S206. As shown in Fig. 11, the dismounting screen displays a message indicating that the driver is dismounting. Also, as shown in Fig. 11, the dismounting screen may display a message indicating that video communication cannot be started.

[0153] 11, the display related to the start of video communication, such as the start button, is shaded, and the start button is disabled. Note that the display related to the start of video communication, including the start button, may not be displayed at all on the dismounting screen.

[0154] If the display control unit 38 determines in step S202 that the URL is invalid (step S202: NO), it causes the external device 40 to display a screen including a message indicating that the URL is invalid (step S207).

[0155] 12 is a diagram showing an example of a URL invalid screen displayed on the touch panel 41 of the external device 40 in step S207. As shown in Fig. 12, in step S207, since the screen related to video communication cannot be accessed, the URL invalid screen displays a message saying "URL is invalid" as a message indicating that the URL is invalid.

[0156] For example, the URL invalid screen may further display a message indicating that video communication cannot be started, or a message indicating that a new invitation email must be received in order to start video communication.

[0157] In steps S204 to S206, when the display control unit 38 receives access from the external device 40 to a URL used for video communication, it executes a display control step of causing the external device 40 to display a start screen showing status information of the operator of the moving object.

[0158] Furthermore, in step S207, the display control unit 38 executes control to prevent the external device 40 from displaying the status information after the video communication started using the URL provided by the connection information providing unit 36 ​​has ended.

[0159] After executing step S205, step S206, or step S207, the display control unit 38 ends the display control routine RT2 and starts the next display control routine RT2.

[0160] According to the display control routine RT2, the in-vehicle screen shown in Fig. 10 or the out-of-vehicle screen shown in Fig. 11 is displayed, so that the virtual passenger can know whether the driver is in the vehicle M before attempting to connect to video communication. Therefore, the virtual passenger can avoid making unnecessary connection attempts.

[0161] In addition, according to the display control routine RT2, the boarding status information is displayed on the external device 40 only when the URL is valid, and the boarding status information is not displayed on the external device 40 when the URL is invalid.

[0162] As described above, in this embodiment, the URL management routine RT1 is executed, and the URL is invalidated when the video communication is terminated from the vehicle-mounted device 10 side.

[0163] Therefore, the driver's privacy can be protected without limiting the availability of the boarding status information to the virtual passenger after the video communication is ended from the in-vehicle device 10. For example, it is possible to prevent the boarding status information from being repeatedly disclosed even after the scheduled end of the video communication.

[0164] Furthermore, according to this embodiment, after the video communication is ended from the in-vehicle device 10 side, the start screen is not displayed either, so that it is possible to prevent connection attempts unintentional by the driver.

[0165] Furthermore, if the URL is invalid, a message indicating that the URL is invalid, as shown in FIG. 12, is displayed, thereby preventing the virtual passenger from accessing connection information unnecessarily.

[0166] In step S203 of the above-mentioned display control routine RT2, the driver's state of ease while driving may be referenced in addition to, or instead of, the driver's riding status, and if the driver is in a riding position but is in a state where he or she is not at ease, a message may be displayed indicating that video communication cannot be started.

[0167] In this embodiment, the display control unit 38 may perform control so as not to display the situation information on the external device 40, without invalidating the URL, after the video communication has ended based on an operation performed on the in-vehicle device 10. For example, in step S202 of the display control routine RT2, it may be determined whether the video communication previously performed using the URL accessed in step 201 has ended based on an operation performed on the in-vehicle device 10.

[0168] In this case, when the display control unit 38 determines in step S202 that the video communication has ended based on the operation performed on the in-vehicle device 10, the process proceeds to step S207, and may cause the external device 40 to display a message indicating that the start screen including the driver's riding status cannot be displayed, instead of a message indicating that the URL is invalid. For example, the display control unit 38 may make the URL inaccessible by deleting the page identified by the URL.

[0169] As described above, the communication system 100 of this embodiment is capable of establishing a voice call between the in-vehicle device 10 as the first device and the external device 40 as the second device via the server 30, while also performing video communication in which images of the automobile M in which the in-vehicle device 10 is installed are transmitted to the external device 40.

[0170] The communication system 100 of this embodiment includes a connection information providing unit that provides connection information used for connecting audiovisual communication including video communication, and a display control unit that, upon receiving access to the connection information from the external device 40, causes the external device 40 to display status information indicating the status of the operator of the mobile object. After the audiovisual communication started using the connection information is completed, the display control unit controls the external device 40 not to display the status information.

[0171] Therefore, if the audiovisual communication has not ended, the virtual passenger can know the status of the pilot before attempting to connect the audiovisual communication, and can avoid making unnecessary connection attempts when the pilot is not on board or when the pilot is not available.

[0172] For the driver of a vehicle, it is possible to prevent the virtual passenger from knowing the driver's riding status, availability, etc., unlimitedly after the audio / video communication ends, thereby protecting the driver's privacy.

[0173] For example, according to this embodiment, after the audiovisual communication started using the connection information is terminated, the connection information can be invalidated, thereby controlling the second device not to display the status information.

[0174] Furthermore, according to this embodiment, after the audiovisual communication started using the connection information is terminated based on an operation performed on the first device, the connection information can be invalidated, thereby controlling the second device to not display the status information.

[0175] Therefore, for example, smooth audio-visual communication can be achieved by protecting the privacy of the pilot when the pilot intentionally ends the video communication, while in other cases displaying the situation information on the second device, thereby reducing the burden on the virtual passenger.

[0176] Furthermore, according to this embodiment, if the connection information is invalidated after the audiovisual communication started using the connection information has ended, the second device will not display a start screen as an operation screen for accepting input operations to start the audiovisual communication, and audiovisual communication using the connection information cannot be started. Therefore, it is possible to prevent, for example, a request to start audiovisual communication that the driver does not want to make after the audiovisual communication has ended.

[0177] Therefore, according to this embodiment, it is possible to provide a communication system, a communication device, a server device, a communication method, a communication program, and a storage medium that enable efficient and smooth communication between a first device inside a mobile body and a second device outside the mobile body by suppressing unnecessary connection attempts from the second device to the first device while taking into consideration the privacy of the operator of the mobile body.

[0178] [Variations] 13 is a block diagram showing an example of the configuration of an in-vehicle device 60 in a modified example of the communication system 100 of the embodiment. In this modified example, the in-vehicle device 60 has the functions of the connection information providing unit 36, the boarding information acquiring unit 37, and the display control unit 38 that the server 30 has in the above embodiment.

[0179] Specifically, the control unit 61 of the in-vehicle device 60 has a connection information providing unit 63 , a boarding information acquiring unit 64 , and a display control unit 65 .

[0180] The storage unit 67 stores a communication program to be executed by the control unit 61. The communication program is a communication program that provides a URL used for connecting to video communication, and when access to the URL from the external device 40 is accepted, causes the external device 40 to display situation information indicating the situation of the driver of the automobile M, and causes the control unit 61 to execute control not to display the situation information on the external device 40 after the video communication started using the URL is completed.

[0181] For example, when the connection information providing unit 63 of the in-vehicle device 60 receives an operational input from the driver instructing the sending of an invitation email to the external device 40, the connection information providing unit 63 generates a URL with an expiration date and provides connection information by sending an invitation email including the URL to the external device 40 via the communication unit 27.

[0182] For example, the connection information providing unit 63 of the in-vehicle device 60 invalidates the provided URL after the video communication that was started using the URL has ended.

[0183] Furthermore, for example, when video communication started using a provided URL is ended based on an operation performed on the vehicle-mounted device 60, the connection information providing unit 63 of the vehicle-mounted device 60 invalidates the URL.

[0184] The boarding information acquisition unit 64 sequentially acquires boarding status information indicating whether the driver is boarding the automobile M as status information based on the detection results of the in-vehicle camera 15 or the seating sensor 22, for example.

[0185] When the display control unit 65 receives access to the URL from the external device 40, it controls the external device 40 to display boarding status information indicating whether or not a driver is in the automobile M. Furthermore, after the video communication started using the URL ends, the display control unit 65 controls the external device 40 not to display the status information.

[0186] For example, the display control unit 65 can perform control so that after the video communication is ended based on an operation performed on the in-car device 10, the start screen including the situation information is not displayed on the external device 40.

[0187] For example, if the URL is invalid, the display control unit 65 performs control so that the external device 40 does not display the status information.

[0188] According to this modification, with regard to video communication between the external device 40 and the in-vehicle device 60, it is possible to control so that status information is displayed on the external device 40 without going through the server 30, and after the video communication ends, the status information is not displayed on the external device 40. For example, the control regarding the display of the boarding status information and the video communication in this modification may be performed directly between the in-vehicle device 10 and the external device 40 by P2P (Peer to Peer) communication or the like, without going through the server 30.

[0189] The configurations, routines, etc. of the in-vehicle device 10, the in-vehicle device 60, the server 30, and the external device 40 in the above-described embodiment and modified examples are merely examples, and can be appropriately selected or changed depending on the application, etc.

[0190] For example, in the above embodiments and variations, the status information may include, in addition to onboard status information indicating whether the driver is on board or not, information indicating whether video communication cannot be started even if the driver is on board.

[0191] For example, if the driver is in video communication with another device other than the external device 40, if there are passengers in the automobile M, if the driving load is high, or if the driver is in the middle of a conversation with a passenger or the other party, it is assumed that the driver will be in a situation where he or she will not be able to respond even if video communication is started, or will not want to transmit images inside or outside the automobile or audio from inside the automobile M to the external device 40.

[0192] In such a case, the driver's riding status may be treated as a situation in which video communication cannot be started even if the driver is riding in the vehicle. In this case, for example, a message indicating that video communication cannot be started may be displayed on the riding screen indicating that the driver is riding in the vehicle.

[0193] For example, the automobile M may be provided with seating sensors in seats other than the driver's seat, and the presence or absence of a passenger, which is used to determine the driver's riding status, may be detected by the seating sensors.

[0194] For example, the magnitude of the driving load used to determine the driver's riding condition may be calculated by workload estimation using images from the exterior camera 13 or the interior camera 15.

[0195] In addition, in the above embodiment, a screen indicating that the URL is invalid is displayed on the external device 40, but this is not limited to this, and the page identified by the URL may be made inaccessible by deleting it.

[0196] In the above embodiment, the connection information providing unit sets the expiration date of the URL when generating the URL, but this is not limited to this. The connection information providing unit does not have to set the expiration date of the URL.

[0197] In the above embodiment and modified example, the request to start video communication from the external device 40 is received by tapping the start button displayed on the touch panel 41, but this is not limiting. The request to start may be received by, for example, a voice input operation.

[0198] In the above embodiment and modified example, an example of performing video communication in the communication system 100 has been described, but the present invention is not limited to this. The present invention is applicable to a case where audio-video communication is performed in the communication system 100, in which audio and video in the automobile M acquired by an in-vehicle device 10 as a first device traveling together with the automobile M is transmitted to an external device 40 as a second device. In other words, the present invention is also applicable to communication in a mode in which audio is not transmitted from the external device 40.

[0199] Furthermore, voice communication is not necessarily possible between the in-vehicle device 10 and the external device 40. For example, the communication between the in-vehicle device 10 and the external device 40 may be a communication mode in which video and audio are simply distributed from the in-vehicle device 10 to the external device 40 without two-way voice communication, such as YouTube (registered trademark) or Niconico Live Broadcast (registered trademark).

[0200] In the above embodiment, the in-vehicle device 10 and the external device 40 perform video communication via the server 30. However, the video communication may be performed directly between the in-vehicle device 10 and the external device 40.

[0201] In addition, while the above-mentioned video distribution is being performed from the in-vehicle device 10 to the external device 40 via the communication system 100, a voice call may be established between the in-vehicle device 10 and the external device 40 in parallel via a path separate from the communication system 100.

[0202] In the above embodiment and modified example, the in-vehicle device 10 is an in-vehicle navigation device. However, the in-vehicle device 10 may simply be capable of distributing video from the exterior camera 13 or the interior camera 15.

[0203] For example, the in-vehicle device 10 may be configured such that a terminal device having a configuration similar to that of the in-vehicle device 10, the exterior camera 13, and the touch panel 17 are integrated together. Specifically, the in-vehicle device 10 may be a terminal device such as a smartphone, tablet, or PC with a camera that is equipped with an application that performs the same functions as the in-vehicle device 10. In this case, the in-vehicle device 10 may be attached to the dashboard DB by, for example, a cradle or the like so that the built-in camera can capture an image of the area ahead of the automobile M through the windshield of the automobile M.

[0204] Furthermore, the in-vehicle device 10 may be configured not to display a screen to be presented to the driver of the automobile M. For example, the in-vehicle device 10 may have a configuration similar to that of a drive recorder, and may be a device integrated with the exterior camera 13. Specifically, the in-vehicle device 10 may be a device in which hardware that performs the video communication function of the in-vehicle device 10 is built into the housing of the exterior camera 13. In this case, the in-vehicle device 10 may not perform the various display outputs described above.

[0205] In the above embodiment and modified example, the external device 40 is described as a smartphone, but is not limited thereto. The external device 40 is a terminal device that can be used by a virtual passenger for video communication, and is configured to be able to display or present messages related to video communication, accept operation inputs required for video communication, send and receive audio data, and receive and display video. For example, the external device 40 may be a terminal device such as a tablet, PC, or wearable device.

[0206] In the above embodiment and modified example, the in-vehicle device 10 and the in-vehicle device 60 are mounted on an automobile M. However, the in-vehicle device 10 may be mounted on other moving objects such as a bicycle, a motorcycle, or a boat. The in-vehicle device 10 may be a terminal device with a built-in camera, such as a smartphone, which may be held by a person and used for video communication while walking, for example, to distribute video. In this case, for example, information indicating whether the user of the terminal device is holding the terminal device may be displayed on the external device 40, instead of whether the driver is in the moving object. [Explanation of symbols]

[0207] 10 Onboard equipment 13. Exterior camera 15 In-car camera 17 Touch Panel 19 Speaker 21. Mike 23 Memory section 25 Control Unit 27 Communications Department 30 servers 31 Storage section 33 Communications Department 35 Control Unit 36, 63 Connection Information Department 37, 64 Boarding status information acquisition unit 38, 65 Display control unit 40 External device 41 Touch Panel 43 Speaker 45 Mike 47 Memory section 49 Control Unit 51 Communications Department

Claims

1. A communication system for performing audio-video communication in which audio and video captured by a first device moving together with a mobile object are transmitted to a second device, a connection information providing unit that provides connection information used for connecting the audiovisual communication; a display control unit that, when receiving access to the connection information from the second device, causes the second device to display status information indicating a status of the operator of the moving body; A communication system characterized in that the display control unit displays the status information on the second device when the connection information is valid, and does not display the status information on the second device when the connection information is invalid.

2. The connection information providing unit sets expiration date information indicating a period during which the connection information is valid when providing the connection information, The communication system according to claim 1 , wherein the expiration date information is set by a passenger of the mobile body.

3. 3. The communication system according to claim 1, wherein the connection information providing unit invalidates the connection information after the audiovisual communication started using the connection information is completed.

4. The communication system described in claim 3, characterized in that the connection information providing unit does not invalidate the connection information if the audio-visual communication started using the connection information is terminated without any operation performed on the first device.

5. the audiovisual communication is initiated by accepting an operation on an operation screen displayed on the second device in response to access to the connection information; The communication system described in claim 3 or 4, characterized in that when the display control unit accepts access to the connection information from the second device, if the connection information is valid, it displays the operation screen on the second device, and if the connection information is invalid, it does not display the operation screen on the second device.

6. 6. The communication system according to claim 1, wherein the status information includes boarding status information indicating whether the operator is on board the moving object.

7. 1. A communication method executed by a communication system for performing audio-video communication in which audio and video captured by a first device moving together with a mobile object are transmitted to a second device, the method comprising: a connection information providing step of providing connection information used for connecting the audiovisual communication; a display control step of, when receiving access to the connection information from the second device, displaying status information indicating a status of the operator of the moving body on the second device; A communication method characterized in that, in the display control step, if the connection information is valid, the status information is displayed on the second device, and if the connection information is invalid, the status information is not displayed on the second device.

8. A communication program executed by a communication system for performing audio-video communication, the communication system including a computer, the first device moving together with a mobile object, and transmitting audio and video captured by the first device at the mobile object to a second device, the program comprising: a connection information providing step of providing connection information used for connecting the audiovisual communication; a display control step of displaying status information indicating a status of an operator of the moving body on the second device when the second device receives access to the connection information; A communication program for controlling the display control step so that, when the connection information is valid, the status information is displayed on the second device, and when the connection information is invalid, the status information is not displayed on the second device.

9. A communication system for performing audio-video communication includes a computer, the system including a first device moving together with a mobile object, and the first device acquiring audio and video of the mobile object and transmitting the audio and video to a second device. a connection information providing step of providing connection information used for connecting the audiovisual communication; a display control step of displaying status information indicating a status of an operator of the moving body on the second device when the second device receives access to the connection information; A computer-readable storage medium that stores a communication program for controlling the display control step so that the status information is displayed on the second device when the connection information is valid, and so that the status information is not displayed on the second device when the connection information is invalid.

10. A communication device that moves together with a mobile object, acquires audio and video from the mobile object, and transmits the audio and video to another device, a connection information providing unit that provides connection information used for connecting the audiovisual communication; a display control unit that, when receiving access to the connection information from the other device, causes the other device to display status information indicating a status of the operator of the moving body; A communication device characterized in that the display control unit displays the status information on the other device when the connection information is valid, and does not display the status information on the other device when the connection information is invalid.

11. A server device for performing audio / video communication in which audio and video of a mobile object captured by a first device moving together with the mobile object is transmitted to a second device, a connection information providing unit that provides connection information used for connecting the audiovisual communication; a display control unit that, when receiving access to the connection information from the second device, causes the second device to display status information indicating a status of the operator of the moving body; The server device is characterized in that the display control unit displays the status information on the second device when the connection information is valid, and does not display the status information on the second device when the connection information is invalid.

Images