Communication system, information processing apparatus, information processing method, program, and recording medium

Abstract

To provide a communication system capable of starting audio / video communication at an arbitrary timing while a vehicle is traveling.SOLUTION: A communication system for performing audio / video communication in which audio and video in a vehicle, which are acquired by a first device moving together with the vehicle, are sequentially transmitted to a second device is characterized in that the communication system comprises a transmission unit for transmitting connection information used for connection with the first device to a destination specified based on the audio in accordance with the audio from a passenger in the vehicle, and a communication connection unit for connecting the first device and the second device communicably in accordance with a connection attempt from the second device using the connection information.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a communication system, an information processing device, an information processing method, a program, and a recording medium, and more particularly to a communication system, an information processing device, an information processing method, a program, and a recording medium for providing video from a mobile body to the outside. [Background technology]

[0002] There is a communication system that performs communication between an onboard device mounted in a vehicle and an external terminal located outside the vehicle. For example, Patent Document 1 discloses a system in which a voice call is performed between a driver of the vehicle and an operator of the external terminal, and while the voice call is being performed, video data showing an image of the area in front of the vehicle is transmitted from the onboard device to the external terminal, and the video is displayed on the external terminal (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-213791 Summary of the Invention [Problem to be solved by the invention]

[0004] In a system such as that described in Patent Document 1, communication for a voice call is established when the driver of the vehicle operates an operation unit located on the dashboard or the like to request the start of a voice call. However, the driver can only operate the operation unit when the vehicle is stopped, and cannot operate it while driving, which has led to the problem that the voice call cannot be smoothly started once the vehicle has started moving.

[0005] The present invention has been made in view of the above-mentioned points, and one of its objects is to provide a communication system that can start audio / video communication at any timing while a vehicle is traveling. [Means for solving the problem]

[0006] The invention described in claim 1 is a communication system for audio-visual communication in which audio and video in a vehicle acquired by a first device moving with the vehicle are sequentially transmitted to a second device, and is characterized by having a transmitting unit that transmits connection information to be used for connecting with the first device to a destination identified based on audio from a driver of the vehicle, and a communication connection unit that connects the first device and the second device so that they can communicate with each other, in response to a connection attempt from the second device using the connection information.

[0007] The invention described in claim 5 is an information processing device that performs audio-visual communication by sequentially transmitting images captured in a vehicle and audio from occupants of the vehicle, and is characterized by having a voice recognition unit that performs voice recognition on audio from occupants of the vehicle, and a transmission unit that transmits connection information to be used for the audio-visual communication to a destination identified based on the results of the voice recognition by the voice recognition unit. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is an overall configuration diagram of a communication system according to a first embodiment. [Figure 2] 1 is a diagram showing a configuration of a front seat portion of a vehicle according to a first embodiment. [Figure 3] 1 is a diagram illustrating a configuration of a terminal device 20 according to a first embodiment and a display mode of an image on the terminal device 20. FIG. [Figure 4] 1 is a block diagram of an in-vehicle device according to a first embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of information stored in a registrant database. [Figure 6] FIG. 10 is a diagram illustrating an example of an invitation message transmitted from an in-vehicle device. [Figure 7] FIG. 2 is a block diagram of a server according to the first embodiment. [Figure 8] FIG. 2 is a block diagram of a terminal device according to the first embodiment. [Figure 9]4 is a flowchart showing a start processing routine executed by a control unit of the in-vehicle device. [Figure 10] 10 is a flowchart showing a connection processing routine executed by a control unit of the server. [Figure 11] 4 is a flowchart showing an audio / video transmission processing routine executed by the in-vehicle device. [Figure 12] 10 is a flowchart showing an audio / video transmission processing routine executed by the server. [Figure 13] FIG. 10 is a diagram showing a modified example of information stored in the registrant database. DETAILED DESCRIPTION OF THE INVENTION

[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will now be described in detail. In the following description of the embodiments and the accompanying drawings, the same reference numerals are used to designate substantially the same or equivalent parts.

[0010] Fig. 1 is an overall configuration diagram of a communication system 100 according to this embodiment. As shown in Fig. 1, the communication system 100 is configured by connecting an in-vehicle device 10, a terminal device 20, and a server 30, which are mounted on a vehicle M as a moving body, via a network NW.

[0011] In this embodiment, the in-vehicle device 10 and the server 30 are connected via a network NW, and the terminal device 20 and the server 30 are connected via the network NW. In other words, the in-vehicle device 10 and the terminal device 20 are configured to be able to transmit and receive information via the server 30.

[0012] The network NW is a network that communicates via a mobile communication network via a connection to a base station such as LTE (Long Term Evolution), 4G (4th Generation), or 5G (5th Generation: 5th generation mobile communication system).

[0013] 2 is a diagram illustrating a front seat area of ​​a vehicle M equipped with an in-vehicle device 10 according to the first embodiment. The in-vehicle device 10 is connected to an exterior camera 11, an interior camera 12, a GPS receiver 13, a microphone 14, a touch panel display 15, and a speaker 16, which are installed in the vehicle M, and includes a control unit that controls these. The in-vehicle device 10 is disposed, for example, in the center of a dashboard DB in the front seat of the vehicle M.

[0014] The exterior camera 11 serving as the first photographing unit is an imaging device that photographs the situation outside the vehicle M. In this embodiment, the exterior camera 11 is a wide-angle camera that photographs a wide area in front of the vehicle M through the windshield FG. In this embodiment, the exterior camera 11 is disposed on the dashboard DB.

[0015] The interior camera 12 as the first photographing unit is an imaging device that photographs the interior of the vehicle M. In this embodiment, the interior camera 12 is a camera that photographs the driver of the vehicle M. In this embodiment, the interior camera 12 is provided on the ceiling at the top end of the windshield FG or near the top end.

[0016] The vehicle exterior photographing camera 11 and the vehicle interior photographing camera 12 may be a multi-directional camera that can photograph both the exterior and interior of the vehicle M with one camera as the first photographing unit.

[0017] The GPS receiver 13 is a receiver that receives signals (GPS signals) from GPS (Global Positioning System) satellites. In this embodiment, the GPS receiver 13 is arranged on the dashboard DB.

[0018] The microphone 14 as the first sound collection unit is a voice input device that receives sounds inside the vehicle, such as voices emitted by the driver and passengers (hereinafter collectively referred to as passengers) of the vehicle M, and converts them into an electrical signal. In this embodiment, the microphone 14 is disposed on the dashboard DB.

[0019] The touch panel display 15 as the first display unit is a display device that combines a display that displays a screen based on the control of the in-vehicle device 10 and a touch panel that accepts input operations from a passenger in the vehicle M. In this embodiment, the touch panel display 15 is disposed in the center of the dashboard DB.

[0020] The touch panel display 15 displays, for example, a navigation image in which the current position of the vehicle M (hereinafter also referred to as the current position) and a planned travel route (hereinafter also referred to as the travel route) are superimposed on a map.

[0021] The speaker 16 as a first sound output unit is an audio output device that outputs audio based on an electrical signal transmitted from the outside. In this embodiment, the speaker 16 is provided on each of the two A-pillars AP.

[0022] In this embodiment, a passenger of the vehicle M can make a voice call with an operator of the terminal device 20 (hereinafter also referred to as a user) via a microphone 14 and a speaker 16 connected to the in-vehicle device 10.

[0023] The positions of the on-board device 10, exterior camera 11, interior camera 12, GPS receiver 13, microphone 14, touch panel display 15 and speaker 16 in the front seat area of ​​the vehicle M described above are merely examples, and these may be arranged in other positions.

[0024] For example, the exterior vehicle camera 11 may be installed anywhere as long as it can capture an image of the situation ahead of the vehicle M, and may be installed at the top edge of the windshield FG or on the ceiling near the top edge.

[0025] In this embodiment, the in-vehicle device 10 sequentially transmits video information indicating either "exterior video", which is video of the outside of the vehicle M captured by the exterior camera 11, or "interior video", which is video of the inside of the vehicle M captured by the interior camera 12, to the terminal device 20 via the server 30. Whether the interior video or the exterior video is to be transmitted can be selected in advance by operating the touch panel display 15. The in-vehicle device 10 also sequentially transmits information on the current location of the vehicle M and information on the route the vehicle M is traveling to the server 30.

[0026] 3 is a diagram showing the configuration of the terminal device 20 and the display mode of an image displayed on the terminal device 20. The terminal device 20 is a communication device including a frame F and a microphone 21, a speaker 22, and a touch panel display 23 housed in the frame F. In this embodiment, the terminal device 20 is a smartphone capable of communicating with others via communication.

[0027] The microphone 21 serving as a second sound collector is a voice input device that receives voice uttered by the user of the terminal device 20 and converts it into an electrical signal. The microphone 21 is provided on one end side of the terminal device 20.

[0028] The speaker 22, which serves as a sound output unit or a second sound output unit, is an audio output device that outputs sound based on an electrical signal transmitted from the outside. The speaker 22 is provided on the other end side of the terminal device 20.

[0029] The first sound information indicating the sound collected by the microphone 14 serving as the first sound collection unit of the above-described in-vehicle device 10 is output as sound from the speaker 22 serving as the sound output unit by the control unit of the terminal device 20. In other words, the control unit of the terminal device 20 is a sound output control unit that controls the speaker 22 to output the sound collected in the vehicle M as moving body sound.

[0030] The second sound information indicating the sound collected by the microphone 21 serving as the second sound collection unit is output as sound from the speaker 16 serving as the first sound output unit by the control unit of the in-vehicle device 10 described above.

[0031] The touch panel display 23 serving as the display unit or the second display unit is a display device that combines a display that displays a screen based on the control of the terminal device 20 and a touch panel that accepts input operations from the user of the terminal device 20. The touch panel display 23 is provided in the center of the terminal device 20.

[0032] In this embodiment, the operator of the terminal device 20 is configured to be able to communicate by voice with the driver of the vehicle M in which the in-vehicle device 10 is installed via the microphone 21 and speaker 22 described above.

[0033] In this embodiment, the terminal device 20 sequentially receives the video information transmitted from the in-vehicle device 10 via the server 30. Specifically, the terminal device 20 sequentially receives the video information of the inside video or the outside video captured in the vehicle M from the in-vehicle device 10 via the server 30.

[0034] Furthermore, in this embodiment, the terminal device 20 sequentially receives image data indicating a map image sequentially transmitted from the server 30. Specifically, the terminal device 20 sequentially receives image data indicating a map image generated by the server 30 based on the current position and travel route of the vehicle M.

[0035] As described above, in this embodiment, the communication system 100 is configured to enable voice communication between the driver of the vehicle M equipped with the on-board device 10 and the operator of the terminal device 20. In addition, in this embodiment, the communication system 100 is configured to enable real-time display on the touch panel display 23 of the terminal device 20 of an image of the outside of the vehicle M captured by the exterior camera 11 and a map image including the current position and travel route of the vehicle M.

[0036] In other words, the communication system 100 is configured so that the in-vehicle device 10 can perform live streaming to the terminal device 20.

[0037] In this embodiment, a communication mode in which audio communication is established between the in-vehicle device 10 and the terminal device 20, and video, such as video of the outside of the vehicle M, and the current position and travel route of the vehicle M are transmitted in real time from the in-vehicle device 10 to the terminal device 20, is referred to as audio-visual communication here. In this embodiment, the in-vehicle device 10 and the terminal device 20 perform audio-visual communication via the server 30. This audio-visual communication allows a user of the terminal device 20 to experience a virtual passenger (hereinafter referred to as "VP") in the vehicle M.

[0038] The video and map image received by the terminal device 20 are displayed on the touch panel display 23. For example, as shown in FIG. 3, the first display area A1 displays the video of the exterior (forward) of the vehicle M, and the second display area A2 displays a map image showing the current position PL of the vehicle M and the travel route R of the vehicle M. The first display area A1 is larger than the second display area A2. In other words, the video of the exterior of the vehicle M is displayed in a larger size than the map image.

[0039] Next, the configurations of the in-vehicle device 10, the server 30, and the terminal device 20 that constitute the communication system 100 will be described with reference to FIGS.

[0040] 4 is a block diagram showing the configuration of the in-vehicle device 10. The control unit 17 is a processing device including a CPU (Central Processing Unit) and a RAM (Random Access Memory).

[0041] The storage unit 18 is a storage device that stores and manages data necessary for processing by the control unit 17. The storage unit 18 is configured with storage devices such as a hard disk, a flash memory, and an SSD (Solid State Drive). The storage unit 18 stores map data. The storage unit 18 also has a registered party database 181 (hereinafter referred to as registered party DB181) that stores callers registered as audiovisual communication partners (hereinafter referred to as registered callers) and their contact information.

[0042] 5 is a diagram showing an example of information registered in the registrant DB 181. The registrant DB 181 stores names of registered callers and contact information to which invitation messages are to be sent. In this embodiment, for example, telephone numbers of smartphones or the like are stored in the registrant DB 181 as contact information.

[0043] 4 again, the communication unit 19 is a communication device that transmits and receives data to and from external devices in accordance with instructions from the control unit 17. The communication unit 19 includes, for example, a network interface card (NIC) for connecting to the network NW. The communication unit 19 functions as a transmission unit that transmits, for example, audio acquired in the vehicle M, images inside or outside the vehicle M, and current location information of the vehicle M to the server 30.

[0044] Here, the voice recognition unit 171 and the invitation message sending unit 172, which are functional blocks of the control unit 17, will be described.

[0045] The voice recognition unit 171 is a functional block that performs voice recognition on the voice input from the microphone 14 (hereinafter referred to as input voice). The voice recognition unit 171 performs voice recognition by converting the input voice into a character string. For example, when the driver of the vehicle M calls out by voice, "VP with Mr. A!", the voice recognition unit 171 converts this into a character string and recognizes it as a voice requesting the start of audio-visual communication with Mr. A.

[0046] The invitation message sending unit 172 generates an invitation message including connection information used for a communication connection for audiovisual communication. This connection information is composed of, for example, a URL (Uniform Resource Locator) and is connection information used for a connection attempt operation to establish a communication connection for audiovisual communication with the in-vehicle device 10 from outside the vehicle M. The invitation message sending unit 172 sends the invitation message to a destination specified by the result of voice recognition by the voice recognition unit 171 and the contact information stored in the registrant DB 181. The invitation message is sent using, for example, a short message service addressed to a telephone number.

[0047] 6 is a diagram showing an example of an invitation message transmitted from the in-vehicle device 10. Here, an example is shown in which the invitation email is received by the terminal device 20 and displayed on the touch panel display 23.

[0048] The invitation message contains a URL as connection information. The operator of the terminal device 20 accesses the URL contained in the invitation email and attempts to connect. When the driver confirms this attempt, a connection for audio and video communication is established.

[0049] 7 is a block diagram showing the configuration of the server 30. The control unit 31 is a processing device including a CPU and a RAM.

[0050] The communication unit 32 is a communication device that transmits and receives data to and from external devices in accordance with instructions from the control unit 31. The communication unit 32 includes, for example, a NIC for connecting to the network NW.

[0051] The communication unit 32 functions as a receiving unit that receives, for example, images of the outside and inside of the vehicle M, audio acquired inside the vehicle M, and information on the current location of the vehicle M from the in-vehicle device 10. The communication unit 32 also functions as a transmitting unit that transmits, for example, images of the outside and inside of the vehicle M and audio acquired inside the vehicle M to the terminal device 20.

[0052] The storage unit 33 is a storage device that stores and manages data necessary for processing by the control unit 31. The storage unit 33 is configured by a storage device such as a hard disk, a flash memory, or an SSD, for example.

[0053] Here, a description will be given of the functional blocks of the control unit 31. Each functional block is formed by the control unit 31 reading and executing a program stored in the storage unit 33, for example.

[0054] The video information acquisition unit 34 is a functional block that acquires video information sequentially transmitted from the in-vehicle device 10 via the communication unit 32. Specifically, the video information acquisition unit 34 acquires, as video information, an in-vehicle video or an outside-vehicle video of the vehicle M transmitted from the in-vehicle device 10. The video information acquisition unit 34 transfers the received video information to the terminal device 20 via the communication unit 32.

[0055] The vehicle information acquisition unit 35 is a functional block that acquires current position information successively transmitted from the in-vehicle device 10, that is, current position information of the vehicle M successively.

[0056] The audio communication control unit 36 ​​is a functional block that controls audio communication between the in-vehicle device 10 and the terminal device 20. Specifically, the audio communication control unit 36 ​​has a function of establishing an audio call between the in-vehicle device 10 and the terminal device 20 in audio / video communication and transferring data of the audio call.

[0057] The connection control unit 37 is a functional block that controls connection for audio and video communication between the in-vehicle device 10 and the terminal device 20. For example, when a connection attempt operation is performed on the terminal device 20 using connection information included in the invitation message transmitted from the in-vehicle device 10, the connection control unit 37 transmits a confirmation message to the in-vehicle device 10 via the communication unit 32, asking the driver of the vehicle M whether or not to establish a connection for audio and video communication.

[0058] 8 is a block diagram showing the configuration of the terminal device 20. The control unit 24 is a processing device including a CPU and a RAM.

[0059] The communication unit 25 is a communication device that transmits and receives data to and from external devices in accordance with instructions from the control unit 24. The communication unit 25 includes, for example, a NIC for connecting to the network NW.

[0060] The communication unit 25 functions as a receiving unit that receives, for example, an image of the outside of the vehicle M or an image of the inside of the vehicle M from the server 30. The communication unit 25 also functions as a transmitting unit that transmits information according to an operation input using the touch panel display 23 to the server 30.

[0061] Next, an example of the specific operations of the in-vehicle device 10, the terminal device 20, and the server 30 in this embodiment will be described with reference to a flowchart.

[0062] First, a connection process for establishing a connection for audiovisual communication between the in-vehicle device 10 and the terminal device 20 (hereinafter referred to as an audiovisual communication connection) will be described.

[0063] The audiovisual communication connection process is triggered by the driver of vehicle M uttering a voice into the microphone 14 requesting the start of audiovisual communication. For example, if the driver of vehicle M wants to start audiovisual communication with "Mr. A," he or she utters a wake-up voice (wake word) into the microphone 14 to activate the voice recognition function of the in-vehicle device 10, followed by a voice requesting the start of audiovisual communication with Mr. A, such as "VP with Mr. A!" The control unit 17 of the in-vehicle device 10 starts a start processing routine triggered by the voice being input via the microphone 14.

[0064] FIG. 9 is a flowchart showing a start processing routine RST1 executed by the control unit 17 of the in-vehicle device 10 in connection processing for audio / video communication connection between the in-vehicle device 10 and the terminal device 20.

[0065] The control unit 17 determines whether or not a voice generated inside the vehicle M, that is, a voice from a passenger of the vehicle M, has been input via the microphone 14 (STEP 101).

[0066] If it is determined that no voice has been input (STEP 101: NO), the start processing routine RST1 is terminated. On the other hand, if it is determined that voice has been input (STEP 101: YES), the voice recognition unit 171 of the control unit 17 performs voice recognition on the input voice (STEP 102).

[0067] Based on the result of the voice recognition by the voice recognition unit 171, the control unit 17 determines whether the input voice is a voice requesting the start of audiovisual communication with the registered caller (STEP 103).

[0068] If it is determined that the voice does not request the start of audiovisual communication (STEP 103: NO), the start processing routine RST1 is terminated. On the other hand, if it is determined that the voice requests the start of audiovisual communication (STEP 103: YES), the control unit 17 reads out the contact information of the registered caller identified by the voice recognition of the voice recognition unit 171 from the registrant DB 181 (STEP 104). For example, if the voice recognition of the voice recognition unit 171 identifies the registered caller requested as the other party in the audiovisual communication as "Mr. A," the control unit 17 reads out Mr. A's contact information "090-11XX-XXX" from the registrant DB 181.

[0069] The invitation message sending unit 172 of the control unit 17 generates an invitation message including connection information for audiovisual communication, and sends the generated invitation message to the destination corresponding to the contact information read out in STEP 104 (STEP 105).

[0070] A registered caller (e.g., person A) who receives the invitation message performs a connection attempt operation using the connection information. For example, when the invitation message is received by the terminal device 20, the registered caller performs a connection attempt operation by operating the touch panel display 23 of the terminal device 20 to click on a URL as connection information included in the invitation message. In response to this connection attempt operation, the server 30 performs processing for an audio / video communication connection, and a connection for audio / video communication between the in-vehicle device 10 and the terminal device 20 is established.

[0071] FIG. 10 is a flowchart showing the processing routine RST2 for audiovisual communication connection executed by the control unit 31 of the server 30.

[0072] The control unit 31 determines whether or not a connection attempt operation has been performed using the connection information included in the invitation message (STEP 201).

[0073] If it is determined that a connection attempt operation has not been performed (STEP 201: NO), the processing routine RST2 is terminated. On the other hand, if it is determined that a connection attempt operation has been performed (STEP 201: YES), the control unit 31 transmits a confirmation message to the in-vehicle device 10 via the communication unit 32, asking the driver of the vehicle M whether or not to perform a voice communication connection (STEP 202).

[0074] The transmitted confirmation message is received by the communication unit 19 of the in-vehicle device 10 and is notified to the passengers of the vehicle M. For example, a confirmation message with the wording "Do you want to start a voice communication connection with person A?" is displayed on the touch panel display 15 of the in-vehicle device 10.

[0075] The driver of vehicle M responds vocally to the confirmation message displayed on the touch panel display 15. For example, the driver of vehicle M responds vocally by saying, "Let's start." Here, the response indicating the start of a voice communication connection is referred to as a "confirmation response." The voice of the confirmation response is input to the in-vehicle device 10 via the microphone 14, and the voice recognition unit 171 recognizes the voice. The control unit 17 transmits information indicating the confirmation response to the server 30 based on the result of the voice recognition by the voice recognition unit 171.

[0076] The control unit 31 of the server 30 determines whether or not an acknowledgement has been received from the in-vehicle device 10 via the communication unit 32 (STEP 203). For example, if the control unit 31 receives a response from the in-vehicle device 10 indicating that the voice call connection will not be established, or if the control unit 31 does not receive an acknowledgement from the in-vehicle device 10 within a predetermined time period after transmitting the acknowledgement message in STEP 202, the control unit 31 determines that an acknowledgement has not been received. If the control unit 31 determines that an acknowledgement has not been received (STEP 203: NO), the control unit 31 ends the processing routine RST2.

[0077] On the other hand, if it is determined that an acknowledgement has been received (STEP 203: YES), the connection control unit 37 of the control unit 31 establishes an audiovisual communication connection between the in-vehicle device 10 and the terminal device 20 (STEP 204).

[0078] In response to the establishment of the audio / video communication connection by the server 30, the control unit 17 of the in-vehicle device 10 starts an audio / video transmission processing routine REX1 for transmitting audio and video captured in the vehicle M to the server 30.

[0079] FIG. 11 is a flowchart showing an audio / video transmission processing routine REX1 executed by the control unit 17 of the in-vehicle device 10. As shown in FIG.

[0080] The control unit 17 determines whether an audio / video communication connection has been established between the in-car device 10 and the terminal device 20 and whether audio / video communication has started (STEP 301). If the control unit 17 determines that audio / video communication has not started (STEP 301: NO), it ends the audio / video transmission processing routine REX1.

[0081] On the other hand, when it is determined that audio-visual communication has started (STEP 301: YES), the control unit 17 sequentially transmits to the server 30 the image of the exterior of the vehicle M captured by the exterior camera 11, the image of the interior of the vehicle M captured by the interior camera 12, and the audio input via the microphone 14 (STEP 302).

[0082] The control unit 17 determines whether the audiovisual communication has ended (STEP 303). If it determines that the audiovisual communication has not ended (STEP 303: NO), the control unit 17 returns to STEP 102, where the control unit 17 repeatedly transmits video and audio. On the other hand, if it determines that the audiovisual communication has ended (STEP 303: YES), the control unit 17 stops transmitting video and audio (STEP 304), and ends the process.

[0083] The control unit 17 of the in-vehicle device 10 continues to transmit video and audio to the server 30 by the audio / video transmission processing routine REX1 as long as audio / video communication between the in-vehicle device 10 and the terminal device 20 continues.

[0084] FIG. 12 is a flowchart showing the audio / video transmission processing routine REX2 executed by the control unit 31 of the server 30.

[0085] The control unit 31 determines whether an audio / video communication connection has been established between the in-car device 10 and the terminal device 20 and whether audio / video communication has started (STEP 401). If the control unit 31 determines that audio / video communication has not started (STEP 401: NO), it ends the audio / video transmission processing routine REX2.

[0086] On the other hand, if it is determined that audio / video communication has started (STEP 401: YES), the video information acquisition unit 34 and audio communication control unit 36 ​​of the control unit 31 start sequentially receiving the video and audio transmitted from the in-vehicle device 10 (STEP 402).

[0087] The control unit 31 transfers the video and audio received in STEP 402 to the terminal device 20 (STEP 403).

[0088] The control unit 31 determines whether the audio / video communication has ended (STEP 404). If it determines that the audio / video communication has not ended (STEP 404: NO), the control unit 31 returns to STEP 402, and continues receiving and transferring video and audio.

[0089] On the other hand, if it is determined that the audio / video communication has ended (STEP 404: YES), the control unit 31 stops receiving and transferring video and audio (STEP 405) and ends the process.

[0090] As described above, in the communication system 100 of this embodiment, the driver of vehicle M makes an audio call to indicate that he or she is starting audio-visual communication and who the other party will be, and in response to this audio call, contact information is read and connection processing is performed, thereby establishing a connection for audio-visual communication.

[0091] With this configuration, the driver does not need to operate the touch panel or the like of the in-vehicle device 10 when starting audiovisual communication, and therefore audiovisual communication can be started at any time while the vehicle is traveling.

[0092] The present invention is not limited to the above-described embodiments. For example, the above-described embodiments have been described with reference to an example in which the voice recognition unit 171 converts voice input via the microphone 14 into a character string, and the invitation message sending unit 172 reads out an address corresponding to the converted character string from the registrant DB 181 and sends the invitation message. However, the voice recognition unit 171 may perform voice recognition by determining whether the voice input via the microphone 14 matches a voice pattern stored in the registrant DB 181.

[0093] 13 is a diagram showing an example of registration information in a case where destinations of registered callers and voice patterns corresponding to those destinations are registered in the registrant database. The voice recognition unit 171 determines whether a voice pattern matching the input voice input via the microphone 14 is stored in the registrant database. The control unit 17 reads out contact information of the registered caller corresponding to the voice pattern matching the input voice. The invitation message sending unit 172 sends an invitation message to the destination specified by the read contact information.

[0094] In the above embodiment, the in-vehicle device 10 transmits either the outside-vehicle video captured by the exterior-vehicle camera 11 or the inside-vehicle video captured by the interior-vehicle camera 12 to the server 30. However, the in-vehicle device 10 may transmit both the outside-vehicle video and the inside-vehicle video to the server 30, and the server 30 may transmit either the outside-vehicle video or the inside-vehicle video to the terminal device 20 when transferring the video to the terminal device 20. Alternatively, the server 30 may transmit both the outside-vehicle video and the inside-vehicle video to the terminal device 20, and the terminal device 20 may be configured to select which video to display. Alternatively, the touch panel display 23 of the terminal device 20 may simultaneously display the inside-vehicle video and the outside-vehicle video, with one video displayed as a main screen and the other as a sub-screen at different sizes.

[0095] In the above embodiment, a case has been described in which a telephone number of a smartphone or the like is stored in the registrant DB 181 as contact information, and an invitation message is sent using a short message service addressed to the telephone number. However, this is not limiting, and an email address may be stored in the registrant DB 181 as contact information, and the invitation message may be sent by email. In the processing routine shown in FIG. 5 of the above embodiment, the invitation message sending unit 172 creates the invitation message after reading the contact information, but the parts other than the destination, i.e., the parts related to the content of the invitation message, may be created in advance before executing the processing routine.

[0096] Also, when sending an invitation message using a short message service, there is a character limit, so if the message body is long, you can select only the last name or the first name to include in the invitation message. For example, in the invitation message example shown in Figure 6, "Yamada Taro" will be displayed as either "Yamada-san" or "Taro-san."

[0097] Furthermore, the in-vehicle device 10 may detect whether or not there is a passenger in the passenger seat of the vehicle M, and transmit information regarding the presence or absence of the passenger in addition to the invitation message. The presence or absence of a passenger can be detected, for example, based on in-vehicle images acquired by the in-vehicle camera 12, seat belt detection information, etc. This allows the registered caller, who is the other party in the audiovisual communication, to know in advance whether or not there is a passenger in the vehicle M.

[0098] In addition to sending the invitation message, the in-vehicle device 10 may detect whether a guide route (planned travel route) has been set, and send a message containing information such as whether a guide route is available, the estimated driving time, destination information, and the URL of a map of the guide route. This allows the registered caller who is the other party in the audiovisual communication to check information such as the route and driving time before participating in the audiovisual communication.

[0099] Furthermore, as mentioned above, since there is a character limit for short message services, the information to be sent as a message may be prioritized. For example, the first name has the highest priority, followed by whether there are passengers, the last name, whether a route has been set, the destination, and the URL showing the route. If it is not possible to include all the information in the message due to the number of characters, information with a lower priority will be omitted from the information to be sent.

[0100] In the above embodiment, the audiovisual communication is performed between the in-vehicle device 10 and one terminal device (one registered caller). However, a plurality of terminal devices may participate in the audiovisual communication. When audio communication is performed simultaneously with a plurality of terminal devices, information about the participating users may be sent in addition to the invitation message.

[0101] In the above embodiment, an example has been described in which a connection attempt is made using the connection information included in the invitation message, and an audio-video communication connection is established when the driver provides a voice confirmation response to the connection attempt. However, the driver's confirmation response is not limited to a voice response, and the confirmation response may be provided by operating the touch panel display 15. Alternatively, the audio-video connection may be established in response to a connection attempt using the connection information, without a confirmation response from the driver.

[0102] Furthermore, the series of processes in the control unit 17 of the in-vehicle device 10 and the control unit 31 of the server 30 described in the above embodiments may be performed by a computer executing a predetermined program. The program may be recorded on a computer-readable recording medium. The type of recording medium is not particularly limited and may be, for example, an optical disk, a hard disk, or a semiconductor memory such as a flash memory or an SSD. Furthermore, the program may be downloaded and installed in the in-vehicle device 10, the server 30, and the terminal device 20 via communication.

[0103] Furthermore, the control routines shown in the above embodiments are merely examples, and can be appropriately selected and changed depending on the application or conditions of use. [Explanation of symbols]

[0104] 100 Communication Systems 10 Onboard equipment 11. Exterior camera 12 In-car camera 13 GPS receiver 14. Mike 15 Touch panel display 16 speakers 17 Control Unit 171 Voice Recognition Unit 172 Invitation message sending section 18 Memory section 181 Registered User DB 19 Communications Department 20 Terminal equipment 21. Mike 22 speakers 23 Touch panel display 24 Control Unit 25 Communications Department 30 servers 31 Control Unit 32 Communications Department 33 Storage section 34 Video information acquisition unit 35 Vehicle information acquisition unit 36 Voice communication control section 37 Connection control section

Claims

[Claim 1] A communication system for performing audio-video communication in which audio and video captured by a first device moving together with a vehicle are sequentially transmitted to a second device, a transmitter configured to, in response to a voice from a passenger of the vehicle, transmit connection information used for connecting to the first device to a destination identified based on the voice; a communication connection unit that connects the first device and the second device so that they can communicate with each other in response to a connection attempt from the second device using the connection information; A communication system comprising:

Images