COMMUNICATION SYSTEM, COMMUNICATION DEVICE, SERVER DEVICE, COMMUNICATION METHOD, COMMUNICATION PROGRAM, AND STORAGE MEDIUM

Abstract

To provide a communication system, a communication device, a server device, a communication method, a communication program, and a storage medium, for making it possible, in communication between a first device in a mobile body and a second device outside the mobile body, to perform a connection trial from the second device at timing at which probability of possibility of a response from the first device is high and to smoothly start communication.SOLUTION: A communication system for performing voice video communication of transmitting voice and video at a mobile body acquired by a first device moving together with the mobile body to a second device comprises: a driver state information acquisition unit that from the first device, acquires driver state information showing a state of a driver of the mobile body; and a display control unit that on the basis of the driver state information, makes the second device display start related information related to a trial of start of the voice video communication from the second device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a communication system, a communication device, a server device, a communication method, a communication program, and a storage medium, and more particularly to a communication system, a communication device, a server device, a communication method, a communication program, and a storage medium that can provide a user with video and audio from a mobile object, for example. [Background technology]

[0002] 2. Description of the Related Art There is a communication system that performs communication between an in-vehicle terminal located inside a vehicle and an external terminal located outside the vehicle.

[0003] For example, Patent Document 1 discloses an in-vehicle information distribution system that controls the vehicle so that it can communicate with the outside world in real time when it is determined that the vehicle is in a relatively safe state, such as waiting at a traffic light, and automatically sends a message to the information sender stating that ``the vehicle is currently driving and cannot make a call'' when it is determined that the vehicle is not in a safe state. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2003-158485 Summary of the Invention [Problem to be solved by the invention]

[0005] In an in-vehicle information distribution system such as that disclosed in Patent Document 1, for example, a user who sends information from outside the vehicle to a driver inside the vehicle will not receive a message such as "Currently driving and unable to make a call" sent from the in-vehicle information distribution system unless the user actually sends the information. Therefore, one of the issues is that the user cannot know in advance whether the driver will be able to make a call, etc., and cannot determine a good timing before sending the information.

[0006] The present invention has been made in consideration of the above-mentioned points, and one of its objectives is to provide a communication system, communication device, server device, communication method, communication program, and storage medium that enable a connection attempt from a second device outside a mobile body to be made at a time when there is a high probability that a response will be possible from the first device, thereby enabling communication to be started smoothly. [Means for solving the problem]

[0007] The invention described in claim 1 is a communication system for audio-visual communication in which audio and video from a mobile body acquired by a first device moving with the mobile body is transmitted to a second device, and is characterized in having a driver status information acquisition unit that acquires driver status information indicating the status of the driver of the mobile body from the first device, and a display control unit that displays start-related information related to an attempt to start the audio-visual communication from the second device on the second device based on the driver status information.

[0008] The invention described in claim 11 is a communication method executed by a communication system that performs audio-visual communication, in which audio and video from a mobile body acquired by a first device moving with the mobile body is transmitted to a second device, and is characterized in that it includes a driver status information acquisition step of acquiring driver status information indicating the status of the driver of the mobile body from the first device, and a display control step of displaying start-related information related to an attempt to start the audio-visual communication on the second device based on the driver status information.

[0009] The invention described in claim 12 is a communication program executed by a communication system that includes a computer and performs audio-visual communication, in which audio and video from a mobile body acquired by a first device moving with the mobile body is transmitted to a second device, and the communication program causes the computer to execute a driver status information acquisition step of acquiring driver status information indicating the status of the driver of the mobile body from the first device, and a display control step of displaying start-related information related to an attempt to start the audio-visual communication on the second device based on the driver status information.

[0010] The invention described in claim 13 is a computer-readable storage medium that stores a communication program that causes a communication system that has a computer and performs audio-visual communication, in which audio and video of a mobile body acquired by a first device moving with the mobile body is transmitted to a second device, to execute a driver status information acquisition step of acquiring driver status information indicating the status of the driver of the mobile body from the first device, and a display control step of displaying start-related information related to an attempt to start the audio-visual communication on the second device based on the driver status information.

[0011] The invention described in claim 14 is a communication device that moves together with a mobile body and performs audio-visual communication by acquiring audio and video from the mobile body and transmitting the acquired audio and video to another device, characterized in that it has a driver status information acquisition unit that acquires driver status information indicating the status of the driver of the mobile body, and a display control unit that displays start-related information related to an attempt to start the audio-visual communication on the other device based on the driver status information.

[0012] The invention described in claim 15 is a server device that performs audio-visual communication by acquiring audio and video of a mobile body acquired by a first device moving with the mobile body and transmitting the audio and video to a second device, characterized in that it has a driver status information acquisition unit that acquires driver status information indicating the status of the driver of the mobile body, and a display control unit that displays start-related information related to an attempt to start the audio-visual communication on the second device based on the driver status information. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a schematic diagram illustrating an overview of a communication system according to an embodiment of the present invention. [Figure 2] 1 is a diagram showing a configuration of a front seat portion of a vehicle according to an embodiment; [Figure 3] 1 is a block diagram showing an example of a configuration of an in-vehicle device according to an embodiment; [Figure 4] FIG. 2 is a block diagram illustrating an example of a configuration of a server device according to an embodiment. [Figure 5] FIG. 4 is a diagram illustrating an example of driver status information according to an embodiment. [Figure 6] FIG. 2 is a front view of an external device according to an embodiment. [Figure 7] FIG. 2 is a block diagram illustrating an example of a configuration of an external device according to an embodiment. [Figure 8] FIG. 2 is a sequence diagram illustrating an example of communication performed in a communication system according to an embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of a display screen displayed on an external device according to an embodiment. [Figure 10] 10 is a flowchart illustrating an example of a routine executed by a server device according to an embodiment. [Figure 11] FIG. 10 is a diagram illustrating an example of a display screen displayed on an external device according to an embodiment. [Figure 12] FIG. 10 is a diagram illustrating an example of a display screen displayed on an external device according to an embodiment. [Figure 13] FIG. 10 is a block diagram showing an example of the configuration of an in-vehicle device in a modified example of the communication system DETAILED DESCRIPTION OF THE INVENTION

[0014] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS In the following description and accompanying drawings, substantially the same or equivalent parts are designated by the same reference numerals. [Example]

[0015] [1. System Configuration] The configuration of a communication system 100 including an in-vehicle device 10 according to an embodiment will be described with reference to the accompanying drawings.

[0016] Fig. 1 shows a communication system 100 according to an embodiment of the present invention. As shown in Fig. 1, the communication system 100 includes an in-vehicle device 10, a server 30, and an external device 40. Fig. 1 shows a case in which the in-vehicle device 10 is mounted on an automobile M, which is an example of a moving body. Fig. 1 also shows a smartphone as an example of the external device 40.

[0017] The in-vehicle device 10, the server 30, and the external device 40 can transmit and receive data to and from each other via a network NW using a communication protocol such as TCP / IP or UDP / IP. The network NW can be constructed using, for example, a mobile communication network, wireless communication such as Wi-Fi (registered trademark), and internet communication including wired communication.

[0018] In the communication system 100, audio and video communication is possible in which audio and video captured by an in-vehicle device 10 as a first device traveling together with the automobile M is transmitted to an external device 40 as a second device.

[0019] In the communication system 100 of this embodiment, after a voice call is established between the in-vehicle device 10 and the external device 40, the in-vehicle device 10 distributes a video image captured in the automobile M to the external device 40.

[0020] In the following description, the communication mode in which a voice call is established between the in-vehicle device 10 and the external device 40 and video captured in the automobile M is distributed from the in-vehicle device 10 to the external device 40 as described above will be referred to as video communication as one mode of audio / video communication. In this embodiment, the in-vehicle device 10 and the external device 40 perform video communication via the server 30.

[0021] By performing such video communication, the user of the external device 40 watching the video transmitted from the in-vehicle device 10 can feel as if he or she is riding in the vehicle M with the driver of the vehicle M. In other words, the video communication can realize a virtual ride-along of the user of the external device 40 in the vehicle M. A system such as the communication system 100 of the present embodiment that can realize such video communication is also referred to as a virtual ride-along system.

[0022] As an example of a sequence for starting video communication in the communication system 100, consider a case where video communication is started in response to a connection attempt from the external device 40 and acceptance from the in-vehicle device 10. At the time of the connection attempt from the external device 40, the driver of the automobile M, who is the user of the in-vehicle device 10, may find it difficult to respond to the connection attempt for video communication or to engage in conversation even if video communication is started, due to the presence of an actual passenger in the automobile M or the driver being too focused on driving.

[0023] In such a case, the user of the external device 40 cannot know in advance whether the driver is available to respond, and if a connection attempt is not accepted, the user must wait a while and try again, which is time-consuming. Even if a connection attempt is accepted, the user of the external device 40 may have little opportunity to talk to the driver, so the user of the external device 40 may interrupt the video communication and try again later, which also requires unnecessary time and effort.

[0024] Therefore, in this embodiment, information related to the initiation of a connection attempt based on the driver's situation, for example, information that helps the user of the external device 40 decide whether or not to make a connection attempt to start video communication, is displayed on the external device 40, making it easier for the user of the external device 40 to determine the timing of the connection attempt.

[0025] In the following, in this embodiment, the in-vehicle device 10 is described as a car navigation device. Also, in this embodiment, the in-vehicle device 10 is described as a terminal device of a so-called cloud-type car navigation device that receives from a user a destination to which the user wishes to be guided, transmits the destination to the server 30, and the server 30 generates a route to the destination.

[0026] Fig. 2 is a perspective view showing the vicinity of the front seat of an automobile M equipped with an in-vehicle device 10 as a video transmission device or a video generation device. Fig. 1 shows, as an example of installation, a case where the in-vehicle device 10 is installed inside a dashboard DB of the front seat of the automobile M.

[0027] The GPS receiver 11 is a device that receives signals (GPS signals) from GPS (Global Positioning System) satellites. The GPS receiver 11 is disposed, for example, on a dashboard DB. The GPS receiver 11 may be disposed anywhere as long as it can receive GPS signals. The GPS receiver 11 is capable of transmitting the received GPS signals to the in-vehicle device 10.

[0028] The exterior camera 13 is an imaging device that captures images in front of the automobile M. In this embodiment, the exterior camera 13 is disposed on the dashboard DB so that the imaging direction faces forward. For example, the exterior camera 13 is a wide-angle camera that can capture images of a wide area in front of the automobile M through the windshield. The exterior camera 13 may be provided on the rearview mirror RM or attached to the inside of the windshield FG.

[0029] The in-vehicle camera 15 is an imaging device that captures images of the interior of the automobile M. In this embodiment, the in-vehicle camera 15 is provided on the upper end of the windshield FG or on the ceiling near the upper end, and is capable of capturing images of the driver of the automobile M.

[0030] During video communication, the video captured by the vehicle exterior camera 13 or the vehicle interior camera 15 is distributed to the external device 40.

[0031] The touch panel 17 is, for example, a touch panel monitor that combines a display, such as a liquid crystal display, capable of displaying images, with a touchpad. The touch panel 17 is disposed, for example, on the center console CC of the dashboard DB. The touch panel 17 may be disposed in a location that is visible to the driver and within the driver's reach. For example, the touch panel 17 may be attached to the dashboard DB.

[0032] The touch panel 17 can display a screen based on the control of the in-vehicle device 10. The touch panel 17 can also transmit a signal representing an input operation to the touch panel 17 received from a user to the in-vehicle device 10. For example, the touch panel 17 may display car navigation guidance. Furthermore, operations related to the car navigation function, such as setting a destination, may be possible via the touch panel 17.

[0033] Furthermore, information about video communication may be displayed on the touch panel 17, and an operation reception screen for making a connection for video communication (hereinafter also referred to as video connection) may be displayed on the touch panel 17. The occupant of the automobile M may perform an operation for connecting to video communication by inputting an input operation on the touch panel 17.

[0034] The speaker 19 is provided, for example, on the interior side of the A-pillar AP. The speaker 19 is capable of emitting sounds such as music and voices under the control of the in-vehicle device 10. During video communication, the speaker 19 emits sound from the external device 40 during a voice call.

[0035] The microphone 21 is a microphone device that picks up sounds inside the vehicle, and is arranged, for example, on the dashboard DB. The microphone 21 may be arranged anywhere, such as on the rearview mirror RM or the steering wheel, as long as it can pick up sounds inside the vehicle. During video communication, the sound picked up by the microphone 21 is transmitted to the external device 40 as the sound of the voice call. Furthermore, an operation input related to the start of video communication may be made by voice via the microphone 21.

[0036] 3 is a block diagram showing the configuration of the in-vehicle device 10. For example, the in-vehicle device 10 is a device in which a storage unit 24, a control unit 25, and a communication unit 27 cooperate with each other via a system bus.

[0037] The seating sensor 22 is capable of detecting whether or not a passenger is present in the passenger seat or other seats of the automobile M, and is, for example, a sensor capable of outputting a signal indicating the presence or absence of the detected passenger. The seating sensor 22 is, for example, provided on the seat surface or back of the passenger seat or other seats of the automobile M, and is a sensor that detects load. The passenger may be detected by image recognition of the image from the in-vehicle camera 15, or by directly detecting the passenger himself or herself using a photoelectric sensor such as an infrared sensor.

[0038] The automobile M is also equipped with an acceleration sensor 23. The acceleration sensor 23 may be, for example, a MEMS acceleration sensor that is capable of measuring the acceleration of the automobile M and outputting a signal indicating the measured acceleration. The acceleration sensor 23 may also be an inertial measurement unit (IMU). The acceleration sensor 23 may also be built into the in-vehicle device 10.

[0039] The storage unit 24 is a storage device configured by, for example, a hard disk drive, a solid state drive (SSD), a flash memory, etc. The storage unit 24 stores various programs executed in the in-vehicle device 10, such as an operating system and terminal software.

[0040] The various programs may be acquired, for example, from another server device or the like via a network, or may be recorded on a recording medium and read via various drive devices. In other words, the various programs stored in the storage unit 24 can be transmitted via a network, or can be recorded on a computer-readable recording medium and transferred.

[0041] The storage unit 24 also stores map information including road maps, and information indicating contact information of the external device 40 as the other party in the video communication, such as a telephone number or an email address.

[0042] The control unit 25 is configured with a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and functions as a computer. The control unit 25 realizes various functions by the CPU reading and executing various programs stored in the ROM and the storage unit 24. In this embodiment, the control unit 25 performs functions such as a function of presenting information regarding the start and end of video communication to the driver, a function of distributing video during video communication, and a car navigation function.

[0043] The control unit 25 is communicably connected to various devices provided in the automobile M, namely, the GPS receiver 11, the exterior camera 13, the interior camera 15, the touch panel 17, the speaker 19, the microphone 21, the seating sensor 22, and the acceleration sensor 23. The control unit 25 acquires data from the various devices provided in the automobile M.

[0044] Specifically, the control unit 25 acquires signals from the GPS receiver 11 one by one.

[0045] The control unit 25 sequentially acquires images captured by the exterior camera 13 and the interior camera 15.

[0046] The control unit 25 acquires a signal representing an input operation on the touch panel 17 .

[0047] In addition, the control unit 25 sequentially acquires the sounds in the automobile M collected by the microphone 21.

[0048] The control unit 25 also sequentially acquires information indicating the presence or absence of a passenger detected by the seating sensor 22. The control unit 25 also sequentially acquires a signal indicating acceleration measured by the acceleration sensor .

[0049] The communication unit 27 is a communication device that transmits and receives data to and from external devices in accordance with instructions from the control unit 25. The communication unit 27 is, for example, a network interface card (NIC) for connecting to the network NW.

[0050] The communication unit 27 is connected to the above-mentioned network NW, and transmits and receives various data to and from the server 30. The communication unit 27 also transmits and receives various data to and from the external device 40 via the server 30.

[0051] The communication unit 27 transmits, for example, information such as an image of the exterior or interior of the automobile M, current location information, and a travel route to the server 30. The communication unit 27 also receives, for example, information to be displayed on the touch panel 17 regarding video communication from the server 30.

[0052] For example, before the start of video communication, the control unit 25 transmits to the server 30 data such as images captured by the exterior camera 13 or the interior camera 15, sensor information from the seating sensor 22, and the like.

[0053] During the video communication, the control unit 25 sequentially transmits the images captured by the exterior camera 13 or the interior camera 15 to the server 30 via the communication unit 27 .

[0054] The control unit 25 receives display information relating to the start and end of video communication from the server 30 via the communication unit 27 and displays it on the touch panel 17. The control unit 25 may also output information relating to the start and end of video communication by sound from the speaker 19.

[0055] For example, the control unit 25 of the in-vehicle device 10 can transmit, via the communication unit 27, location identification information as location data that can identify the current location of the in-vehicle device 10, that is, information on the current location of the automobile M in this embodiment, to the server 30. Furthermore, for example, the control unit 25 can transmit, via the communication unit 27, to the server 30, information including a destination input by a user, and receive, from the server 30, route information or navigation information to the destination.

[0056] The control unit 25 can transmit audio data of the audio picked up by the microphone 21 to the external device 40 via the communication unit 27 for an audio call in the video communication. The control unit 25 can also receive audio data of the audio input to the external device 40 via the communication unit 27 for an audio call in the video communication.

[0057] 4 is a block diagram showing the configuration of the server 30. For example, the server 30 is a device in which a storage unit 31, a communication unit 33, and a control unit 35 cooperate with each other via a system bus.

[0058] The server 30 has a function of acquiring driver status information indicating the status of the driver of the automobile M between the in-vehicle device 10 and the external device 40, and displaying start-related information related to an attempt to start video communication from the external device 40 as a second device on the external device 40 based on the driver status information. For example, the start-related information is information that helps the user of the external device 40 determine whether or not to make a connection attempt to start video communication.

[0059] The server 30 also has a function similar to that of a SIP server that establishes a voice call between the in-car device 10 and the external device 40 during video communication and transfers data of the voice call.

[0060] The server 30 also has the function of receiving from the in-vehicle device 10 the location identification information of the automobile M and information on the destination set by the user who is an occupant of the automobile M, and generating a route to the destination based on the location identification information and the destination information.

[0061] The storage unit 31 is configured with, for example, a hard disk drive and an SSD (solid state drive), and stores various programs such as an operating system and software for the server 30.

[0062] The various programs may be acquired, for example, from another server device or the like via a network, or may be recorded on a recording medium and read via various drive devices. In other words, the various programs stored in the storage unit 31 can be transmitted via a network, or can be recorded on a computer-readable recording medium and transferred.

[0063] The storage unit 31 stores, for example, a communication program for the server 30 to control the second device to display start-related information related to an attempt to start video communication from the external device 40 in the communication system 100.

[0064] The storage unit 31 also stores a map information database (not shown) that stores map information including road maps. The map information in the map information database is a database that contains information equivalent to the map information used in, for example, a navigation device.

[0065] The communication unit 33 is connected to the network NW, and transmits and receives various data between the in-vehicle device 10 and the external device 40.

[0066] The control unit 35 is configured with a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and functions as a computer. The CPU reads and executes various programs stored in the ROM and the storage unit 31, thereby realizing various functions.

[0067] By executing the above-mentioned communication program, the control unit 35 realizes display control that causes the external device 40 to display start-related information related to an attempt to start video communication from the external device 40 in the communication system 100 based on the status of the driver of the automobile M.

[0068] The control unit 35 includes a driver status information acquisition unit 37 and a display control unit 39 .

[0069] The driver status information acquisition unit 37 is a functional unit that executes a driver status information acquisition step of acquiring driver status information indicating the status of the driver of the automobile M from the in-vehicle device 10 via the communication unit 33.

[0070] The driver status information acquisition unit 37 acquires, as driver status information, information indicating whether other audio / video communication is being performed from the in-vehicle device 10. The driver status information acquisition unit 37 acquires, from the in-vehicle device 10, information indicating whether communication has been established between the in-vehicle device 10 and another device other than the external device 40 and video communication is being performed.

[0071] Furthermore, the driver status information acquisition unit 37 acquires, as driver status information, information indicating whether or not a passenger is present in the automobile M from the in-vehicle device 10. The driver status information acquisition unit 37 receives, for example, sensor information from a seating sensor provided in the automobile M, and generates information indicating the presence or absence of a passenger based on the received sensor information.

[0072] Further, for example, the driver status information acquisition unit 37 may receive an image from the in-vehicle camera 15 of the automobile M, and generate information indicating the presence or absence of a passenger based on the received image of the interior of the automobile M.

[0073] Furthermore, the driver status information acquisition unit 37 acquires driving load information indicating the magnitude of the driving load imposed on the driver of the automobile M from the in-vehicle device 10 as the driver status information.

[0074] The driver status information acquisition unit 37 acquires, for example, an image of the front of the automobile M captured by the exterior camera 13 from the in-vehicle device 10, and acquires driving load information by performing image recognition using the image of the front.

[0075] For example, the driver status information acquisition unit 37 performs workload estimation to estimate the amount of driving-related load placed on the driver of the vehicle M through image recognition using an image of the front of the vehicle M, and as a result of the workload estimation, calculates a numerical value indicating the amount of load and uses this as driving load information.

[0076] In the workload estimation, the load is estimated according to the driving situation, for example, estimating that the load is higher when turning right or left than when going straight. Furthermore, when a route is generated, the load may be estimated to be high when passing through a guidance point such as an intersection.

[0077] For the workload estimation, map information or an image previously linked to map information may be used instead of or in addition to the forward image captured by the exterior camera 13. For example, workload estimation may be performed based on the shape of an intersection corresponding to the current position of the automobile M. For example, a numerical value indicating a greater load may be calculated as driving load information when passing through a three-way or five-way intersection than when passing through a crossroads.

[0078] The workload estimation may be performed based on biological information of the driver of the automobile M. In this case, the driver status information acquisition unit 37 may acquire biological information such as the pulse rate and brain waves of the driver of the automobile M from the in-vehicle device 10, and calculate a numerical value indicating the load on the driver as driving load information based on the biological information.

[0079] Furthermore, the driver status information acquisition unit 37 acquires, as the driver status information, information indicating whether or not the driver of the automobile M is engaged in a conversation from the in-vehicle device 10. The driver status information acquisition unit 37 may acquire, for example, from the in-vehicle device 10, an image of the driver of the automobile M captured by the in-vehicle camera 15 or an audio captured inside the automobile M, and determine whether or not the driver is engaged in a conversation based on the acquired image or audio.

[0080] Fig. 5 is a diagram showing an example of driver status information acquired and held by the driver status information acquisition unit 37. For example, the driver status information acquisition unit 37 successively acquires and updates the driver status information as shown in Fig. 5.

[0081] For example, the driver status information includes an in-vehicle device ID, which is an identifier capable of identifying the in-vehicle device 10, such as a MAC address, and information indicating the status of the driver of the automobile M.

[0082] In the example shown in Fig. 5, situation types indicating the driver's situation in video communication are assigned situation type IDs S1 to S4, respectively. As shown in Fig. 5, situation type ID "S1" indicates a situation in which the driver is "engaged in another audio / video communication" with a user of an external device other than external device 40.

[0083] Similarly, the situation type ID "S2" indicates a situation in which "there is a real passenger" in the automobile M, the situation type ID "S3" indicates a situation in which "the driver is under heavy load," and the situation type ID "S4" indicates a situation in which "the driver is engaged in a conversation."

[0084] In FIG. 5, in the "correct or incorrect" column, a circle is marked if the situation of the driver of the automobile M corresponds to each of the situation types S1 to S4, and an X is marked if the situation does not correspond.

[0085] 5, the situations indicated by the situation types S1 to S4 are all considered to be situations in which it may be difficult for the driver to respond to a connection attempt for video communication. In this embodiment, when the driver's situation corresponds to a situation in which it may be difficult to respond, it will be described as a situation in which it is impossible to respond to a connection attempt for video communication.

[0086] In this embodiment, for example, when the driver's situation corresponds to one or more of the situation types S1 to S4 in FIG. 5, it is determined that the situation is such that a response to a connection attempt for video communication is not possible.

[0087] In the example shown in Fig. 5, the situation of the driver of automobile M does not correspond to situation type S1, which is "engaged in another audio / video communication" (S1: ×), does not correspond to situation type S3, which is "driver's workload is high" (S3: ×), does not correspond to situation type S4, which is "driver is engaged in a conversation" (S4: ×), but corresponds to situation type S2, which is "there is a real passenger" (S2: ○). In such a case, in the example shown in Fig. 5, the driver's situation is determined to be one in which he is unable to respond to a video communication connection attempt.

[0088] 5, the driver status information acquisition unit 37 may acquire information indicating whether or not at least one of the status types S1 to S4 applies. Status types other than the status types S1 to S4 may also be set as the driver status information.

[0089] Moreover, which situation type is to be used as the driver situation information may be set in advance, or may be selectable and set by the driver.

[0090] The display control unit 39 is a functional unit that executes a display control step of causing the external device 40 to display start-related information related to an attempt to start video communication from the external device 40 based on the driver status information.

[0091] The display control unit 39 controls the external device 40 to display, as the start-related information, information that helps the user of the external device 40 decide whether or not to attempt a connection to start video communication, based on the driver status information acquired by the driver status information acquisition unit 37.

[0092] For example, the display control unit 39 causes the external device 40 to display, as the start-related information, information indicating whether or not the driver is in a state where he or she can respond to a connection attempt for video communication.

[0093] For example, when video communication is started, the display control unit 39 causes the external device 40 to display, as the start-related information, information indicating whether the driver is in a situation where it is difficult to respond to conversations, etc. during video communication.

[0094] For example, the display control unit 39 causes the external device 40 to display, as the initiation-related information, information indicating that execution after an attempt to start video communication from the external device 40 is recommended.

[0095] For example, when the external device 40 accesses a URL (Uniform Resource Locator) as connection information used for connection when performing video communication, the display control unit 39 refers to the driver status information acquired by the driver status information acquisition unit 37 and causes the external device 40 to display start-related information based on the driver status information. For example, when referring to the driver status information, the display control unit 39 may refer to all of the multiple status types indicating the driver's status, such as status types S1 to S4 shown in Fig. 5, or may refer to only some of the multiple status types.

[0096] For example, the display control unit 39 determines whether the driver's situation is such that the driver can respond to a connection attempt for video communication, based on the driver situation information. For example, in the example shown in Fig. 5, the display control unit 39 may determine that the driver cannot respond to a connection attempt for video communication when the driver's situation corresponds to any of situation types S1 to S4 of the driver situation information.

[0097] For example, if the result of the determination is that the driver is in a state where he or she is able to respond, the display control unit 39 causes the external device 40 to display information indicating that the driver of the automobile M is in a state where he or she is able to respond to an attempt to connect via video communication.

[0098] For example, if the determination result indicates that the driver is unable to respond to a connection attempt, the display control unit 39 causes the external device 40 to display information indicating that the driver of the automobile M is unable to respond to a connection attempt for video communication. Alternatively, if the driver is unable to respond, the display control unit 39 may cause the external device 40 to display information indicating that it is recommended that the external device 40 subsequently attempt to start video communication.

[0099] For example, if an attempt is made to start video communication from the external device 40 and a notification is sent to the driver on the in-vehicle device 10 at a time when the driver of the automobile M is concentrating on driving and is under a heavy driving load, the driver's attention may be reduced or he may panic, which may cause problems with driving.

[0100] Furthermore, for example, if video communication with external device 40 is initiated at a time when the driver is engaged in video communication with another device other than external device 40, when there is a passenger in automobile M, or when the driver is having a conversation with another person inside automobile M, and images from exterior camera 13 or interior camera 15 of automobile M or audio from inside the automobile M are transmitted to external device 40, this may be inconvenient for the driver.

[0101] For example, even if the user of the external device 40 attempts to connect to video communication at such a timing, the driver is likely to not consent to the start of the video communication, and the video communication is likely to be unable to start. In such a case, the user of the external device 40 will have to make connection attempts multiple times, which may be time-consuming and stressful.

[0102] Furthermore, even if video communication were to be initiated at such a timing, it is thought that the driver would not be able to respond satisfactorily to the conversation.

[0103] In this embodiment, the above-mentioned timings when the driving load is high or timings that are inconvenient for the driver are assumed to be timings when the driver is unable to respond to a connection attempt for video communication.

[0104] In this embodiment, as described above, the display control unit 39 of the server 30 causes the external device 40 to display start-related information related to an attempt to start video communication from the external device 40 based on the driver status information.

[0105] This allows the user of the external device 40 to know in advance whether the driver is available to respond, for example. Furthermore, based on the information displayed on the external device 40, the user can attempt to connect at a time when the driver is likely to be available to respond to a call, etc., and can start video communication smoothly. Furthermore, for the driver, stress is reduced because an attempt to start video communication is not made at a time when the driver is unable to respond.

[0106] The control unit 35 acquires location information indicating the current location of the automobile M from the in-vehicle device 10 via the communication unit 33. The control unit 35 also acquires destination information input to the in-vehicle device 10 by an occupant of the automobile M from the in-vehicle device 10 via the communication unit 33. The control unit 35 generates a route to the destination based on the location information and the destination information, and transmits information indicating the route to the in-vehicle device 10.

[0107] The control unit 35 also transfers the video and audio received from the in-vehicle device 10 to the external device 40 via the communication unit 33. The control unit 35 transfers the audio received from the external device 40 to the in-vehicle device 10 via the communication unit 33.

[0108] 6 is a front view of the external device 40. As described above, in this embodiment, the external device 40 is a smartphone.

[0109] The touch panel 41 is, for example, a touch panel monitor that combines a display, such as a liquid crystal display, capable of displaying images, with a touch pad. The touch panel 41 is capable of generating signals that represent input operations to the touch panel 41 received from a user. In this embodiment, images distributed from the in-vehicle device 10 are displayed on the touch panel 41.

[0110] Furthermore, information about the video communication and a screen for accepting an operation to start a video communication connection are displayed on the touch panel 41. The user of the external device 40 can perform an input operation to start the video communication by inputting information to the touch panel 41.

[0111] The speaker 43 is capable of emitting sounds such as music and voices. During video communication, the speaker 43 emits the sound from the in-car device 10 during voice communication.

[0112] The microphone 45 is a microphone device that receives sounds emitted toward the external device 40. During video communication, the sounds collected by the microphone 45 are transmitted to the in-car device 10 via the server 30 as voice calls.

[0113] 7 is a block diagram showing an example of the configuration of the external device 40. For example, the external device 40 is a device in which a storage unit 47, a control unit 49, and a communication unit 51 cooperate with each other via a system bus (not shown).

[0114] The storage unit 47 is configured by, for example, a hard disk drive, a solid state drive (SSD), a flash memory, etc., and stores various programs such as an operating system and software for the external device 40.

[0115] The various programs may be acquired, for example, from another server device or the like via a network, or may be recorded on a recording medium and read via various drive devices. That is, the various programs stored in the storage unit 47 can be transmitted via a network, or can be recorded on a computer-readable recording medium and transferred.

[0116] The control unit 49 is configured with a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (Random Access Memory), etc., and functions as a computer. The CPU reads and executes various programs stored in the ROM and the storage unit 47, thereby realizing various functions.

[0117] The control unit 49 is communicably connected to the touch panel 41, the speaker 43, and the microphone 45.

[0118] The control unit 49 can receive a signal indicating an input operation to the touch panel 41 and an audio input signal from the microphone 45. For example, the control unit 49 can accept an input operation for starting video communication made by the user via the touch panel 41 and the microphone 45.

[0119] Furthermore, the control unit 49 can transmit a video or image signal to the touch panel 41 to display it, and can transmit an audio signal to the speaker 43 to output sound.

[0120] For example, the control unit 49 causes the touch panel 41 to display a start screen for accepting a request to start video communication and a screen indicating whether the driver of the automobile M is available to respond.

[0121] Furthermore, for example, the control unit 49 causes the touch panel 41 to display an image during the video communication, and causes the speaker 43 to output audio collected in the automobile M during the video communication.

[0122] The communication unit 51 is connected to the above-mentioned network NW, and transmits and receives various data to and from the server 30. The communication unit 51 also transmits and receives various data, including video transmitted from the in-vehicle device 10, to and from the in-vehicle device 10 via the server 30.

[0123] The control unit 49 of the external device 40 can receive information via the communication unit 51 to display a start screen for accepting a request to start video communication and a screen indicating whether the driver of the automobile M is available to respond.

[0124] Furthermore, the control unit 49 can transmit audio data of the audio picked up by the microphone 45 to the in-vehicle device 10 via the communication unit 51 for audio communication in video communication. The control unit 49 can also receive audio data transmitted from the in-vehicle device 10 via the communication unit 51 for audio communication in video communication.

[0125] For example, the control unit 49 of the external device 40 can receive, via the communication unit 51, location identification information capable of identifying the current location of the automobile M transmitted from the in-vehicle device 10 from the server 30. Furthermore, for example, the control unit 49 can receive, via the communication unit 51, route information or navigation information for the automobile M from the server 30.

[0126] [2. Start of video communication] 8 is a sequence diagram showing an example of communication performed between the in-vehicle device 10 and the external device 40 via the server 30 when video communication is started in the communication system 100. Hereinafter, a case will be described in which the driver, who is the user of the in-vehicle device 10, plans to perform video communication with a virtual passenger, who is the user of the external device 40, while driving the automobile M.

[0127] As shown in FIG. 8, the driver registers in advance in the server 30 the contact information of a virtual fellow passenger who will be the other party in video communication with the external device 40 (step S11).

[0128] When registering a virtual passenger, registration information including a pair of a name by which the virtual passenger can be identified, such as a nickname, and information indicating a destination, such as an email address or a telephone number, required to send information to the virtual passenger, is transmitted to the server 30 and held by the server 30. The registration information includes, for example, an identifier by which the in-vehicle device 10 can be identified.

[0129] The virtual passenger is registered via the touch panel 41 of the in-vehicle device 10, but this is not limited to this and may also be done by sending input information to a terminal device other than the in-vehicle device 10, such as a smartphone used by the driver of the automobile M, to the server 30.

[0130] The timing for the driver to register the virtual passenger's contact information in advance may be, for example, the day before or before the day on which video communication is planned while driving automobile M, or may be just before the video communication.

[0131] Thereafter, when the driver starts driving the automobile M, the driver performs an input operation on the in-vehicle device 10 to instruct the external device 40 to send an invitation email. The input operation may be performed via the touch panel 17 of the in-vehicle device 10, or may be performed by voice via the microphone 21.

[0132] An invitation email transmission request is sent to the server 30 through an input operation by the driver (step S12). The invitation email transmission request includes, for example, information indicating the destination, such as an email address or a telephone number of the external device 40, used by the virtual passenger that has been registered in advance, and a nickname that can identify the virtual passenger.

[0133] When the server 30 receives the invitation email transmission request, it transmits the invitation email to the address of the external device 40 included in the invitation email transmission request (step S13). The invitation email includes a URL for opening a start screen for the virtual passenger to perform video communication.

[0134] When the virtual passenger opens the invitation email using the external device 40 and taps the URL (step S14), the server 30 determines the driver's situation (step S15).

[0135] As shown in step S15 of Figure 8, if the driver's situation is such that he or she is able to respond to a video communication connection attempt, a browser is launched and a start screen is presented on the touch panel 41 of the external device 40 for the virtual passenger to operate if he or she wishes to start video communication (step S16).

[0136] 9 is a diagram showing an example of a display screen displayed on the touch panel 41 of the external device 40 in step S16. As shown in Fig. 9, for example, the start screen includes a start button which is a display area for accepting a request to start video communication.

[0137] The start screen may display, for example, a message indicating that the driver is available to respond to a video communication connection attempt. Furthermore, as shown in Fig. 9, the start screen may be provided with an input field for inputting identification information for identity verification. For example, identity verification may be performed by inputting the last four digits of a phone number into the input field.

[0138] In addition, the start screen may allow the user to check the terms of use and privacy policy for video communication, and for example, a message may be displayed indicating that pressing the start button constitutes agreement to the terms of use and privacy policy.

[0139] When the virtual passenger taps the start button, the operation of the start button by the user is accepted (step S17), and a request to start video communication is transmitted to the server 30 (step S18).

[0140] When the server 30 receives the request to start video communication, it notifies the in-car device 10 that there has been a request to start video communication from the external device 40, and thereby a connection attempt for video communication is made (step S19).

[0141] In step S19, for example, when the in-vehicle device 10 receives a notification from the server 30, a selection button for whether to accept or decline the start of video communication is displayed on the touch panel 17. For example, when the driver taps the accept button, information indicating that the driver has accepted the start of video communication is transmitted to the server 30 (step S20).

[0142] When the server 30 receives information indicating that the driver has agreed to the start of video communication, it establishes a voice call between the in-vehicle device 10 and the external device 40 and starts video communication in which the video captured in the automobile M is distributed from the in-vehicle device 10 to the external device 40 (step S21).

[0143] In this way, when the driver is in a state where he or she can respond to the video communication connection attempt, the in-vehicle device 10 and the external device 40 start video communication via the server 30.

[0144] [3. Server Operation] A control routine of the server 30 for realizing the operation of the communication system 100 of the embodiment will be described below. The following description will be made on the assumption that the driver status information acquisition unit 37 acquires driver status information as needed and stores the latest driver status information after, for example, the server 30 receives a request to send an invitation email (FIG. 8, step S12).

[0145] 10 is a flowchart showing a display control routine RT1 executed by the display control unit 39 of the server 30. The display control unit 39 starts the display control routine RT1 when, for example, the server 30 is powered on.

[0146] When the display control routine RT1 starts, the display control unit 39 determines whether or not an invitation email has been sent from the server 30 to the external device 40 within a predetermined time (step S101).

[0147] In step S101, if the display control unit 39 determines that an invitation email has not been sent from the server 30 to the external device 40 within a predetermined time (step S101: NO), it terminates the display control routine RT1, starts the next display control routine RT1, and executes step S101 again.

[0148] In step S101, if the display control unit 39 determines that an invitation email has been sent from the server 30 to the external device 40 within a predetermined time (step S101: YES), it determines whether or not a URL included in the invitation email has been accessed (step S102). In step S102, for example, when the virtual passenger taps the URL included in the invitation email using the external device 40 and the external device 40 accesses the URL, it is determined that the URL included in the invitation email has been accessed.

[0149] In step S102, when the display control unit 39 determines that the URL has not been accessed (step S102: NO), it ends the display control routine RT1 and starts the next display control routine RT1.

[0150] In step S102, when the display control unit 39 determines that the URL has been accessed (step S102: YES), it refers to the driver's condition information acquired by the driver's condition information acquisition unit 37 (step S103).

[0151] In step S103, the display control unit 39 refers to driver status information indicating whether or not there is other video communication, whether or not there is an actual passenger, etc., as shown in FIG.

[0152] After executing step S103, the display control unit 39 determines whether the driver is in a situation where he or she can respond to the video communication connection attempt based on the driver status information (step S104). In step S104, it determines whether the driver is in a situation where he or she can respond based on whether the driver corresponds to each of a plurality of status types that indicate the driver's status, for example.

[0153] In step S104, for example, as illustrated in Figure 5, if the driver falls under any of multiple situation types that indicate situations in which it may be difficult for the driver to respond to a video communication connection attempt, it is determined that the driver is unable to respond to a video communication connection attempt.

[0154] In step S104, when the display control unit 39 determines that the driver is available to respond (step S104: YES), it causes the touch panel 41 of the external device 40 to display a start screen for presenting information regarding the start of video communication and accepting operation input (step S105). In step S105, for example, the display control unit 39 causes the external device 40 to display a start screen including a start button as shown in FIG.

[0155] In step S104, if the display control unit 39 determines that the driver is unable to respond (step S104: NO), it causes the touch panel 41 of the external device 40 to display a message indicating that the driver is unable to respond to the video communication connection attempt (step S106).

[0156] 11 is a diagram showing an example of a no-response screen displayed on the touch panel 41 of the external device 40 in step S106. As shown in Fig. 11, together with the start screen, a message saying "Mr. / Ms. XX is currently unable to respond" is displayed as information indicating whether or not the driver is in a situation where he / she can respond to an attempt to connect to video communication.

[0157] 11, a message saying "We recommend starting later" is displayed as information indicating that it is recommended to delay the attempt to start video communication from external device 40. Such a message recommending delay may be displayed together with a message indicating that a response is unavailable, as shown in FIG. 11, or may be displayed instead of a message indicating that a response is unavailable.

[0158] 12 is a diagram showing another example of the no-response screen displayed on the touch panel 41 of the external device 40 in step S106. In the example shown in Fig. 12, the start screen is displayed together with a message indicating that the driver has a passenger, and that video communication cannot be started.

[0159] For example, if a no-response screen such as that shown in Figure 11 or 12 is displayed, the virtual passenger can press the start button, but in this case, there is a high possibility that the driver of automobile M will not approve the start of video communication. Therefore, the virtual passenger can initially decide not to press the start button, and can access the URL again after a while to check the start screen or the no-response screen displayed on touch panel 41 of external device 40. This saves the virtual passenger the trouble of repeatedly pressing the start button to attempt to connect.

[0160] After executing step S105 or step S106, the display control unit 39 ends the display control routine RT1 and starts the next display control routine anew.

[0161] In this embodiment, situations such as a high driving load on the driver or an inconvenient situation for the driver are considered to be situations in which it may be difficult to respond to an attempt to connect to video communication, and if the driver's situation falls under such a situation, it is determined that it is impossible to respond to an attempt to connect to video communication, but this is not limited to this.

[0162] A situation in which the driving load is high or a situation inconvenient for the driver may be treated as a situation in which it is difficult to respond to a conversation, etc., even if video communication is initiated. In this case, for example, the display control unit 39 may determine in step S104 of the display control routine RT1 whether it is difficult to respond to a conversation, etc., during video communication, and in step S106, cause the external device 40 to display information indicating that it is difficult for the driver to respond to a conversation, etc., even if video communication is initiated.

[0163] Furthermore, for example, each type of situation may be treated as a situation in which it is impossible to respond to a connection attempt for video communication or a situation in which it is difficult to respond to a conversation during video communication. For example, a situation in which another video communication is in progress or a situation in which the driving load is high may be treated as a situation in which it is impossible to respond to a connection attempt for video communication, and a situation in which it is difficult to respond to a conversation during video communication when there is a passenger or when the driver is talking.

[0164] As described above in detail, the communication system 100 of this embodiment is capable of establishing a voice call between the in-vehicle device 10 as the first device and the external device 40 as the second device via the server 30, while also performing video communication in which images of the automobile M in which the in-vehicle device 10 is installed are transmitted to the external device 40.

[0165] The communication system 100 of this embodiment has a driver status information acquisition unit that acquires driver status information indicating the status of the driver of the moving body from the in-vehicle device 10, and a display control unit that displays start-related information related to an attempt to start video communication from the external device 40 on the external device 40 based on the driver status information.

[0166] For example, the initiation-related information is information that contributes to the determination of whether or not to perform a connection attempt to start video communication by the virtual passenger who is the user of the external device 40.

[0167] For example, the display control unit can cause the external device 40 to display, as the initiation-related information, information indicating whether or not the driver of the automobile M is in a state where he or she can respond to an attempt to start video communication.

[0168] Furthermore, for example, the display control unit can cause the external device 40 to display, as the start-related information, information indicating that execution after an attempt to start video communication from the external device 40 is recommended.

[0169] With this configuration, the virtual passenger, who is the user of the external device 40, can check the start-related information displayed on the external device 40 and decide whether or not to attempt a connection for video communication based on the start-related information.

[0170] By checking the information displayed on the external device 40, the virtual passenger can know in advance whether the driver is available to make a call or the like, and can determine the best timing to attempt a connection. Therefore, the virtual passenger can attempt to start video communication at a time when there is a high probability that the driver of the automobile M will be able to respond to the video communication connection attempt, thereby saving the trouble of making multiple connection attempts. Furthermore, even if the virtual passenger is able to start video communication, he or she can avoid a situation in which the driver is unable to respond satisfactorily to the conversation or the like.

[0171] The driver of the automobile M, who is the user of the in-vehicle device 10, can avoid receiving multiple connection request notifications in a situation where it is difficult to respond to video communication, for example. Therefore, the driver does not need to perform any operation such as accepting while driving, and can avoid panicking due to receiving notifications related to video communication while driving.

[0172] That is, for both the users of the in-car device 10 and the external device 40, video communication can be started smoothly and without stress.

[0173] Therefore, according to this embodiment, in communication between a first device inside a mobile body and a second device outside the mobile body, it is possible to provide a communication system, communication device, server device, communication method, communication program, and storage medium that enable a connection attempt from the second device to be made at a time when there is a high probability that a response will be possible from the first device, thereby enabling communication to be started smoothly.

[0174] [Variations] 13 is a block diagram showing an example of the configuration of an in-vehicle device 60 in a modified example of the communication system 100 of the embodiment. In this modified example, the in-vehicle device 60 has the functions of the driver status information acquisition unit 37 and the display control unit 39 that the server 30 has in the above embodiment.

[0175] Specifically, the control unit 61 of the in-vehicle device 60 has a driver status information acquisition unit 63 and a display control unit 65. The storage unit 67 also stores a communication program for causing the external device 40 to display start-related information related to an attempt to start video communication from the external device 40, based on the driver status information.

[0176] The control unit 61, like the control unit 35 of the in-vehicle device 10, is connected to various devices provided in the automobile M so as to be able to communicate with them.

[0177] For example, when the in-vehicle device 60 accesses a URL used for video communication from the external device 40, the driver status information acquisition unit 63 acquires driver status information indicating the status of the driver of the automobile M in which the in-vehicle device 60 is installed.

[0178] For example, the driver status information acquisition unit 63 acquires, as the driver status information, information indicating whether other audio / video communication is being performed. The driver status information acquisition unit 63 acquires information indicating whether communication is established between the in-vehicle device 10 and another device other than the external device 40 and video communication is being performed.

[0179] Furthermore, the driver status information acquisition unit 63 acquires, as the driver status information, information indicating whether or not there is a passenger inside the automobile M. For example, the driver status information acquisition unit 63 receives sensor information from a seating sensor provided inside the automobile M, and generates information indicating whether or not there is a passenger based on the received sensor information. Furthermore, for example, the driver status information acquisition unit 63 may receive video from the in-vehicle camera 15 of the automobile M, and generate information indicating whether or not there is a passenger based on the received video of the interior of the automobile M.

[0180] Furthermore, the driver status information acquisition unit 63 acquires driving load information indicating the magnitude of the driving load imposed on the driver of the automobile M as the driver status information.

[0181] Furthermore, the driver status information acquisition unit 63 acquires, as the driver status information, information indicating whether or not the driver of the automobile M is engaged in a conversation. The driver status information acquisition unit 63 may determine whether or not the driver is engaged in a conversation based on, for example, an image of the driver of the automobile M captured by the in-vehicle camera 15 or an audio recorded inside the automobile M.

[0182] The display control unit 65 of the in-vehicle device 60 controls the external device 40 to display start-related information related to an attempt to start video communication from the external device 40, based on the driver status information acquired by the driver status information acquisition unit 63. The start-related information is, for example, information that helps the user of the external device 40 determine whether or not to make a connection attempt to start video communication.

[0183] For example, the display control unit 65 causes the external device 40 to display, as the start-related information, information indicating whether the driver is in a state where he or she can respond to a connection attempt for video communication.

[0184] Furthermore, for example, the display control unit 65 causes the external device 40 to display, as the start-related information, information indicating that execution of the video communication of the external device 40 after the start of the video communication is recommended.

[0185] According to this modified example, with regard to video communication between the external device 40 and the in-vehicle device 60, the in-vehicle device 60 can control the display of start-related information on the external device 40 based on driver status information without going through the server 30.

[0186] Furthermore, according to this modification, video communication can be performed between the external device 40 and the in-vehicle device 60 without going through the server 30. For example, the video communication in this modification may be performed directly between the in-vehicle device 10 and the external device 40 by P2P (Peer to Peer) communication or the like without going through the server 30.

[0187] The configurations, routines, etc. of the in-vehicle device 10, the in-vehicle device 60, the server 30, and the external device 40 in the above-described embodiment and modified examples are merely examples, and can be appropriately selected or changed depending on the application, etc.

[0188] For example, in the above embodiment and modified example, the example of tapping the start button displayed on the touch panel 41 has been described as a mode of accepting a start request from the external device 40, but this is not limiting. The start request may also be accepted by, for example, a voice input operation.

[0189] In the above embodiment and modified example, an example of performing video communication in the communication system 100 has been described, but the present invention is not limited to this. The present invention is applicable to a case where audio-video communication is performed in the communication system 100, in which audio and video in the automobile M acquired by an in-vehicle device 10 as a first device traveling together with the automobile M is transmitted to an external device 40 as a second device. In other words, the present invention is also applicable to communication in a mode in which audio is not transmitted from the external device 40.

[0190] Furthermore, voice communication is not necessarily possible between the in-vehicle device 10 and the external device 40. For example, the communication between the in-vehicle device 10 and the external device 40 may be a communication mode in which video and audio are simply distributed from the in-vehicle device 10 to the external device 40 without two-way voice communication, such as YouTube (registered trademark) or Niconico Live Broadcast (registered trademark).

[0191] In the above embodiment, the in-vehicle device 10 and the external device 40 perform video communication via the server 30. However, the video communication may be performed directly between the in-vehicle device 10 and the external device 40.

[0192] In addition, while the above-mentioned video distribution is being performed from the in-vehicle device 10 to the external device 40 via the communication system 100, a voice call may be established between the in-vehicle device 10 and the external device 40 in parallel via a path separate from the communication system 100.

[0193] In the above embodiment and modified example, the in-vehicle device 10 and the in-vehicle device 60 are described as in-vehicle navigation devices, but the in-vehicle device 10 or the in-vehicle device 60 only needs to be capable of delivering video from the exterior camera 13 or the interior camera 15.

[0194] For example, the in-vehicle device 10 may be configured such that a terminal device having a configuration similar to that of the in-vehicle device 10, the exterior camera 13, and the touch panel 17 are integrated together. Specifically, the in-vehicle device 10 may be a terminal device such as a smartphone, tablet, or PC with a camera that is equipped with an application that performs the same functions as the in-vehicle device 10. In this case, the in-vehicle device 10 may be attached to the dashboard DB by, for example, a cradle or the like so that the built-in camera can capture an image of the area ahead of the automobile M through the windshield of the automobile M.

[0195] Furthermore, the in-vehicle device 10 may be configured not to display a screen to be presented to the driver of the automobile M. For example, the in-vehicle device 10 may have a configuration similar to that of a drive recorder, and may be a device integrated with the exterior camera 13. Specifically, the in-vehicle device 10 may be a device in which hardware that performs the video communication function of the in-vehicle device 10 is built into the housing of the exterior camera 13. In this case, the in-vehicle device 10 may not perform the various display outputs described above.

[0196] In the above embodiment and modified example, the external device 40 is described as a smartphone, but is not limited thereto. The external device 40 is a terminal device that can be used by a virtual passenger for video communication, and is configured to be able to display or present messages related to video communication, accept operation inputs required for video communication, send and receive audio data, and receive and display video. For example, the external device 40 may be a terminal device such as a tablet, PC, or wearable device.

[0197] In the above embodiment and modified example, the in-vehicle device 10 and the in-vehicle device 60 are mounted on the automobile M, but the in-vehicle device 10 may be mounted on other moving objects such as a bicycle, a motorbike, a boat, etc. Also, the in-vehicle device 10 may be a device with a built-in camera like a smartphone, which may be held by a person, and may perform video communication while walking, for example, to distribute images. [Explanation of symbols]

[0198] 10, 60 On-vehicle equipment 13. Exterior camera 15 In-car camera 17 Touch Panel 19 Speaker 21. Mike 22 Seat sensor 23 Acceleration sensor 24 Memory section 25 Control Unit 27 Communications Department 30 servers 31 Storage section 33 Communications Department 35 Control Unit 37 Driver status information acquisition unit 39 Display control unit 40 External device 41 Touch Panel 43 Speaker 45 Mike 47 Memory section 49 Control Unit 51 Communications Department

Claims

1. A communication system for performing audio-video communication in which audio and video captured by a first device moving together with a mobile object are transmitted to a second device, a driver status information acquisition unit that acquires driver status information indicating a status of a driver of the moving object from the first device; A communication system characterized by having a display control unit that displays start-related information related to an attempt to start audio-visual communication from a second device identified by notification from the first device to the first device based on the driver status information.

2. The communication system according to claim 1 , wherein the initiation-related information is information that contributes to a decision by a user of the second device as to whether or not to perform a connection attempt to start the audiovisual communication.

3. the driver status information includes information indicating whether the audiovisual communication is being performed between the first device and another device other than the second device, The communication system described in any one of claims 1 or 2, characterized in that the display control unit displays information related to the start of the audio-visual communication on the second device based on whether the audio-visual communication is being carried out with the other device.

4. the driver status information includes information indicating whether a passenger is present in the vehicle; The communication system described in any one of claims 1 to 3, characterized in that the display control unit displays information related to the start of the audio-visual communication on the second device based on whether or not there is a passenger in the vehicle.

5. the driver status information includes driving load information indicating the magnitude of a driving load imposed on the driver of the mobile body, The communication system according to any one of claims 1 to 4, wherein the display control unit causes the second device to display information related to the start of the audiovisual communication based on the driving load information.

6. the driver status information includes information indicating whether the driver is engaged in a conversation; The communication system according to any one of claims 1 to 5, characterized in that the display control unit causes the second device to display information related to the start of the audiovisual communication based on whether the driver is in a conversation or not.

7. The communication system described in any one of claims 1 to 6, characterized in that the display control unit displays, as the start-related information, information indicating whether the driver is in a position to respond to an attempt to start the audio-visual communication on the second device.

8. The communication system described in any one of claims 1 to 7, characterized in that the display control unit displays, as the start-related information, information indicating whether the driver is in a situation where it is difficult for him to respond to the audio-visual communication on the second device.

9. The communication system described in any one of claims 1 to 8, characterized in that the display control unit displays, as the start-related information, information on the second device indicating that execution after an attempt to start the audio-visual communication from the second device is recommended.

10. The communication system described in any one of claims 1 to 9, characterized in that when the display control unit receives access from the second device to connection information used to connect the audio-visual communication, it displays the start-related information on the second device.

11. 1. A communication method executed by a communication system for performing audio-video communication in which audio and video captured by a first device moving together with a mobile object are transmitted to a second device, the method comprising: a driver status information acquisition step of acquiring driver status information indicating a status of a driver of the moving body from the first device; A communication method characterized by including a display control step of displaying, on the identified second device, start-related information related to an attempt to start the audio-visual communication from a second device identified by notification from the first device to the first device based on the driver status information.

12. A communication program executed by a communication system for performing audio-video communication, the communication system comprising a computer, the first device moving together with a mobile object, and transmitting audio and video captured by the first device at the mobile object to a second device, the communication program comprising: The computer, a driver status information acquisition step of acquiring driver status information indicating a status of a driver of the moving body from the first device; a display control step of displaying, on the specified second device based on the driver status information, initiation-related information related to an attempt to start the audiovisual communication from the specified second device to the first device by the notification from the first device; A communication program for executing the above.

13. A communication system for performing audio-video communication includes a computer, the system including a first device moving together with a mobile object, and the first device acquiring audio and video of the mobile object and transmitting the audio and video to a second device. a driver status information acquisition step of acquiring driver status information indicating a status of a driver of the moving body from the first device; a display control step of displaying, on the specified second device based on the driver status information, initiation-related information related to an attempt to start the audiovisual communication from the specified second device to the first device by the notification from the first device; A computer-readable storage medium storing a communication program for executing the above.

14. A communication device that moves together with a mobile object, acquires audio and video from the mobile object, and transmits the audio and video to another device, a driver status information acquisition unit that acquires driver status information indicating a status of a driver of the moving body; A communication device characterized by having a display control unit that, before the audio-visual communication is started, causes the other device to display start-related information related to an attempt to start the audio-visual communication from the other device based on the driver status information.

15. A server device for performing audio / video communication, which acquires audio and video of a mobile object acquired by a first device moving together with the mobile object and transmits the acquired audio and video to a second device, a driver status information acquisition unit that acquires driver status information indicating a status of a driver of the moving body; A server device characterized by having a display control unit that displays start-related information related to an attempt to start the audio-visual communication from a second device identified by notification from the first device to the first device based on the driver status information.

Images