Video transmission apparatus, server apparatus, video transmission method, video transmission program, storage medium, and data structure

Abstract

To provide a video transmission device capable of securing high visual reproduction from a driver of a moving body when an operator of a vehicle exterior terminal views a video transmitted from the moving body while suppressing traffic in data.SOLUTION: A video transmission apparatus according to the present invention is a video transmission apparatus that moves together with a moving body, the video transmission apparatus including a video acquisition unit configured to sequentially acquire a video around the moving body from an image capturing apparatus configured to capture the video, a position information acquisition unit configured to acquire position information of the moving body, a map information acquisition unit configured to acquire map information including a point or a section on a road associated with a frame rate, and a video transmission unit configured to transmit the video at a frame rate based on the frame rate associated with the point or the section on the road indicated by the position information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a video transmission device, a server device, a video transmission method, a video transmission program, a storage medium, and a data structure, and more particularly to a video transmission device, a server device, a video transmission method, a video transmission program, a storage medium, and a data structure for providing a user with video from a mobile object, for example. [Background technology]

[0002] There is a communication system that performs communication between an onboard device mounted in a vehicle and an external terminal located outside the vehicle. For example, Patent Document 1 discloses a system in which a voice call is performed between a driver of the vehicle and an operator of the external terminal, and while the voice call is being performed, video data showing an image of the area in front of the vehicle is transmitted from the onboard device to the external terminal, and the video is displayed on the external terminal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-213791 Summary of the Invention [Problem to be solved by the invention]

[0004] In a system such as that described in Patent Document 1, it is desirable to reduce the amount of data communication when continuously transmitting video, for example. On the other hand, when a user of an external terminal is viewing a video of the scene ahead of the vehicle and talking to the driver of the vehicle or giving directions, it is also important to allow the user of the external terminal to experience a sensation similar to that experienced by the driver when looking outside the vehicle.

[0005] The present invention has been made in consideration of the above-mentioned points, and one of its objects is to provide a video transmission device, a server device, a video transmission method, a video transmission program, a storage medium, and a data structure that can ensure high reproducibility of the scenery seen by the driver of a mobile body when the operator of an external terminal views the video sent from the mobile body while reducing the amount of data communication. [Means for solving the problem]

[0006] The invention described in claim 1 is a video transmission device that moves with a moving body, characterized in that it has a video acquisition unit that sequentially acquires video from an imaging device that captures video around the moving body, a location information acquisition unit that acquires location information of the moving body, a map information acquisition unit that acquires map information including a point or section on a road to which a frame rate is linked, and a video transmission unit that transmits the video at a frame rate based on the frame rate linked to the point or section on the road indicated by the location information.

[0007] The invention described in claim 8 is a video transmission device that moves with a moving body, characterized in that it has a video acquisition unit that sequentially acquires video from an imaging device that captures video around the moving body, a location information transmission unit that acquires and transmits location information of the moving body, a frame rate receiving unit that receives information indicating a frame rate linked to a point or section on a road indicated by the location information, and a video transmitting unit that transmits the video at a frame rate based on the frame rate indicated by the information received by the receiving unit.

[0008] The invention described in claim 9 is a server device characterized by having a location information receiving unit that receives location information of a moving body from a video transmission device that moves with the moving body, and a frame rate transmitting unit that identifies a frame rate associated with a point or section on a road indicated by the location information received by the receiving unit based on map information including a point or section on a road to which a frame rate is associated, and transmits information indicating the identified frame rate to the video transmission device.

[0009] The invention described in claim 10 is a video transmission method executed by a video transmission device that moves together with a moving body, comprising: a video acquisition step of sequentially acquiring the video from an imaging device that captures video around the moving body; a location information acquisition step of acquiring location information of the moving body; a map information acquisition step of acquiring map information including a point or section on a road to which a frame rate is associated; and a video transmission step of transmitting the video at a frame rate based on the frame rate linked to the point or section on the road indicated by the location information.

[0010] The invention described in claim 11 is a video transmission program that includes a computer and is executed by a video transmission device that moves together with a moving body, and that causes the computer to execute a video acquisition step of sequentially acquiring video from an imaging device that captures video around the moving body, a location information acquisition step of acquiring location information of the moving body, a map information acquisition step of acquiring map information including a point or section on a road to which a frame rate is associated, and a video transmission step of transmitting the video at a frame rate based on the frame rate linked to the point or section on the road indicated by the location information.

[0011] The invention described in claim 12 is a computer-readable storage medium that stores a video transmission program that includes a computer and causes a video transmission device that moves with a moving body to execute the following steps: a video acquisition step of sequentially acquiring video from an imaging device that captures video around the moving body; a location information acquisition step of acquiring location information of the moving body; a map information acquisition step of acquiring map information including points or sections on a road to which a frame rate is associated; and a video transmission step of transmitting the video at a frame rate based on the frame rate linked to the point or section on the road indicated by the location information.

[0012] The invention described in claim 13 is a data structure of map information used by a video transmission device that has a computer and moves with a mobile body, and includes position data indicating a point or section on a road, and frame rate data indicating a frame rate associated with the point or section on the road indicated by the position data, wherein the video transmission device acquires the frame rate associated with the point or section on the road corresponding to the position of the mobile body on the road, and is used in a process of transmitting video at a frame rate based on the acquired frame rate. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating a video distribution system according to a first embodiment of the present invention. [Figure 2] 1 is a perspective view of a front seat portion of a vehicle equipped with an in-vehicle device. [Figure 3] FIG. 2 is a block diagram showing an example of the configuration of an in-vehicle device. [Figure 4] FIG. 3 is a diagram illustrating an example of information included in map information according to the present embodiment. [Figure 5] FIG. 3 is a diagram illustrating an example of information included in map information according to the present embodiment. [Figure 6] FIG. 3 is a diagram illustrating an example of information included in map information according to the present embodiment. [Figure 7] FIG. 2 is a block diagram illustrating an example of a configuration of a server. [Figure 8] FIG. [Figure 9] FIG. 2 is a block diagram showing an example of the configuration of an external device. [Figure 10] FIG. 4 is a flow diagram of an operation routine of the in-vehicle device. [Figure 11] FIG. 4 is a flow diagram of an operation routine of the in-vehicle device. [Figure 12] FIG. 4 is a flow diagram of an operation routine of the in-vehicle device. [Figure 13] FIG. 4 is a flow diagram of the operation routine of the server. DETAILED DESCRIPTION OF THE INVENTION [Example]

[0014] [1. System configuration] A video distribution system 100 according to a first embodiment of the present invention will be described below with reference to the accompanying drawings.

[0015] Fig. 1 shows a video distribution system 100 according to a first embodiment of the present invention. As shown in Fig. 1, the video distribution system 100 includes an in-vehicle device 10, a server 40, and an external device 70. Fig. 1 shows a case in which the in-vehicle device 10 is mounted on an automobile M, which is an example of a moving body. Fig. 1 also shows a smartphone as an example of the external device 70.

[0016] The in-vehicle device 10, the server 40, and the external device 70 can transmit and receive data to and from each other via a network NW using a communication protocol such as TCP / IP or UDP / IP. The network NW can be constructed using, for example, a mobile communication network, wireless communication such as Wi-Fi (registered trademark), and internet communication including wired communication.

[0017] In the video distribution system 100 of this embodiment, a voice call is established between the in-vehicle device 10 and the external device 70, and then the video captured in the automobile M is distributed from the in-vehicle device 10 to the external device 70. In the following description, the communication mode in which the video captured in the automobile M is distributed from the in-vehicle device 10 to the external device 70 while the voice call is established between the in-vehicle device 10 and the external device 70 as described above is referred to as video communication. In this embodiment, the in-vehicle device 10 and the external device 70 perform video communication via the server 40.

[0018] By performing such video communication, the user of the external device 70 viewing the video transmitted from the in-vehicle device 10 can feel as if he or she were riding in the vehicle M with the driver of the vehicle M. In other words, the video communication can realize a virtual ride-along with the user of the external device 70 in the vehicle M. A system that realizes such video communication is referred to as a virtual ride-along system. In the following, in the first embodiment, a case where the in-vehicle device 10 is a car navigation device will be described as an example. In the first embodiment, a case where the in-vehicle device 10 is a terminal device of a so-called cloud-based car navigation device will be described as an example, in which the in-vehicle device 10 receives from the user a destination to which the user wishes to be guided, transmits the destination to the server 40, and the server 40 generates a route to the destination.

[0019] Fig. 2 is a perspective view showing the vicinity of the front seat of an automobile M equipped with an in-vehicle device 10 as a video transmission device. Fig. 1 shows, as an example of installation, a case where the in-vehicle device 10 is installed inside a dashboard DB of the front seat of the automobile M.

[0020] The GPS receiver 11 is a device that receives signals (GPS signals) from GPS (Global Positioning System) satellites. The GPS receiver 11 is disposed, for example, on a dashboard DB. The GPS receiver 11 may be disposed anywhere as long as it can receive GPS signals. The GPS receiver 11 is capable of transmitting the received GPS signals to the in-vehicle device 10.

[0021] The exterior camera 12 serving as the photographing unit is an imaging device that photographs the area ahead of the automobile M. In this embodiment, the exterior camera 12 is disposed on the dashboard DB so that the photographing direction faces forward. For example, the exterior camera 12 is a wide-angle camera that can photograph a wide area ahead of the automobile M through the windshield. The exterior camera 12 may be provided on the rearview mirror RM or attached to the inside of the windshield FG.

[0022] The in-vehicle camera 13 is an imaging device that captures images of the interior of the automobile M. In this embodiment, the in-vehicle camera 13 is provided on the upper end of the windshield FG or on the ceiling near the upper end, and is capable of capturing images of the driver of the automobile M.

[0023] During video communication, video captured by the vehicle exterior camera 12 or the vehicle interior camera 13 is distributed to the external device 70. The following mainly describes the case where video captured by the vehicle exterior camera 12 is distributed to the external device 70.

[0024] The touch panel 14 is, for example, a touch panel monitor that combines a display, such as a liquid crystal display, capable of displaying images, with a touchpad. The touch panel 14 is disposed, for example, on the center console CC of the dashboard DB. The touch panel 14 may be disposed in a location that is visible to the driver and within the driver's reach. For example, the touch panel 14 may be attached to the dashboard DB.

[0025] The touch panel 14 can display a screen based on the control of the in-vehicle device 10. The touch panel 14 can also transmit a signal representing an input operation to the touch panel 14 received from the user to the in-vehicle device 10. For example, the touch panel 14 may display car navigation guidance. Furthermore, operations related to the car navigation function, such as setting a destination, may be possible via the touch panel 14.

[0026] Furthermore, information regarding video communication may be displayed on the touch panel 14, and an operation reception screen for making a connection for video communication (hereinafter also referred to as video connection) may be displayed on the touch panel 14. The occupant of the automobile M may perform an operation for connecting to video communication by inputting an input operation on the touch panel 14.

[0027] The speaker 15 is provided, for example, on the interior side of the A-pillar AP. The speaker 15 is capable of emitting sounds such as music and voices under the control of the in-vehicle device 10. During video communication, the speaker 15 emits sound from the external device 70 during the voice call.

[0028] The microphone 17 is a microphone device that picks up sounds inside the vehicle and is arranged, for example, on the dashboard DB. The microphone 17 may be arranged anywhere, such as on the rearview mirror RM or the steering wheel, as long as it can pick up sounds inside the vehicle. During video communication, the sound picked up by the microphone 17 is transmitted to the external device 70 as the sound of the voice call.

[0029] 3 is a block diagram showing the configuration of the in-vehicle device 10. For example, the in-vehicle device 10 is a device in which a large-capacity storage device 23, a control unit 25, an input unit 27, an output unit 29, an encoder unit 30, and a data communication unit 31 cooperate with each other via a system bus 21.

[0030] The mass storage device 23 is configured by, for example, a hard disk drive, a solid state drive (SSD), a flash memory, etc., and stores various programs such as an operating system and software for the terminal.

[0031] The various programs may be acquired, for example, from another server device or the like via a network, or may be recorded on a recording medium and read via various drive devices. That is, the various programs stored in the mass storage device 23 (including a program for executing processing in the in-vehicle device 10, which will be described later) can be transmitted via a network, or can be recorded on a computer-readable recording medium and transferred.

[0032] The large-capacity storage device 23 also has a map information database (shown as map information DB in the figure) 23A. Map information including road maps is stored in the map information database 23A. The map information includes position data, which is information indicating points or sections on roads.

[0033] In the map information, for example, points and sections on roads are represented by nodes on the roads and links on the roads. Nodes indicate intersections and dead-end points in the road network. Links indicate sections between nodes on the roads. The map information database 23A can be used, for example, to generate routes for route guidance.

[0034] In this embodiment, the in-vehicle device 10 transmits video at a frame rate based on the frame rate associated with the link or node where the automobile M is located. In the following description, the frame rate associated with the position data indicating the link or node on the road in the map information is also referred to as the "video transmission frame rate."

[0035] For example, the frame rate associated with a link is predetermined and assigned so that the impression given to the user of the external device 70 when the user of the external device 70 views the image acquired from the automobile M located at that link and displayed on the external device 70 is closer to the impression given to the driver of the automobile M viewing the same scene.

[0036] Fig. 4 is a diagram showing an information table included in the map information stored in the map information database 23A. In the example shown in Fig. 4, a frame rate is associated with each link on a road. As shown in Fig. 4, in the map information of the map information database 23A, frame rate data is associated with position data indicating a link.

[0037] The frame rate data indicates a frame rate that is predetermined and associated with a link so that a video can be properly viewed when the video is captured from a vehicle located on the link on a road.

[0038] The frame rate associated with the location data in the map information is, for example, a recommended frame rate for each link. Also, in the map information, for example, a lower limit value of the allowable frame rate may be associated with each link, or a range of the allowable frame rate may be associated with each link.

[0039] Fig. 5 shows an example of map information in which a frame rate is associated with each node. As shown in Fig. 5, for node X and node Y, a frame rate recommended for traveling in the uphill direction and a frame rate recommended for traveling in the downhill direction are associated. In other words, when a frame rate is associated with each node, the node becomes a point where the frame rate should be switched.

[0040] 4 and 5. As shown in FIG. 6, node X is a node between link A and link B, and node Y is a node between link B and link C. As shown in FIG. 6, if the direction from link A to link B is the upstream direction, it is recommended that, for example, upstream node X switch from the frame rate of 15 fps associated with link A to the frame rate of 20 fps associated with link B.

[0041] As described above, the example shown in Fig. 4 describes a case where a frame rate is associated with each link, and the example shown in Fig. 5 describes a case where a frame rate is associated with each node. Note that this embodiment is not limited to the case where a frame rate is associated with a link or a node, and for example, a frame rate may be associated with each region (area) on a map that indicates an area including roads.

[0042] The control unit 25 is configured with a CPU (Central Processing Unit) 25A, a ROM (Read Only Memory) 25B, a RAM (Random Access Memory) 25C, etc., and functions as a computer. The CPU 25A reads and executes various programs stored in the ROM 25B and the mass storage device 23, thereby realizing various functions. In this embodiment, the control unit 25 performs functions such as video distribution and car navigation.

[0043] The input unit 27 is an interface unit that communicatively connects the in-vehicle device 10 with the exterior camera 12, the interior camera 13, the touch panel 14, and the microphone 17. The in-vehicle device 10 can sequentially acquire images captured by the exterior camera 12 and the interior camera 13 via the input unit 27. In other words, the control unit 25 functions as an image acquisition unit that sequentially acquires image data showing images of the surroundings of the vehicle captured by the exterior camera 12 via the input unit 27.

[0044] The in-vehicle device 10 can receive, via the input unit 27, a signal indicating an input operation to the touch panel 14. For example, the in-vehicle device 10 can accept, via the input unit 27, a connection request made by the user via the touch panel 14 and the microphone 17, or a destination setting input for car navigation.

[0045] The input unit 27 is an interface unit that communicatively connects the in-vehicle device 10 and the GPS receiver 11. The in-vehicle device 10 receives a GPS signal from the GPS receiver 11 via the input unit 27 and can acquire information on the current position of the in-vehicle device 10, that is, the current position of the automobile M in this embodiment, from the GPS signal. In other words, the control unit 25 functions as a position information acquisition unit that acquires position information of the automobile M from the GPS receiver 11.

[0046] The output unit 29 is communicatively connected to the touch panel 14 and the speaker 15, and is capable of transmitting video or image signals to the touch panel 14 to display the same, and transmitting audio signals to the speaker 15 to output sound.

[0047] The encoder unit 30 is a part that encodes (hereinafter also referred to as encoding processing) the video (also referred to as captured video) captured by the exterior camera 12 or the interior camera 13 based on commands from the control unit 25. The encoder unit has a CPU for video encoding, a so-called GPU, and encoding may be performed by the GPU.

[0048] The encoder unit 30 generates encoded data as video encoded data by encoding the captured video using, for example, an encoding method conforming to the MPEG-4 standard. For example, the encoder unit 30 generates encoded data from the captured video using a codec such as H.264, Xvid, DivX, VP8, or VP9.

[0049] Furthermore, based on a command from the control unit 25, the encoder unit 30 encodes the video captured by the outside camera 12 at a frame rate instructed by the control unit 25. The control unit 25 reads map information from the map information database 23A in the mass storage device 23. In other words, the control unit 25 functions as a map information acquisition unit that acquires map information including points or sections on roads to which frame rates are linked.

[0050] As described above, in the map information of the map information database 23A, a video transmission frame rate is associated with position data indicating a point or section on a road. Based on the read map information, the control unit 25 obtains a video transmission frame rate corresponding to the position of the automobile M, i.e., a frame rate associated with the point or section on a road indicated by the position information of the automobile M, and instructs the encoder unit 30 to encode the video at a frame rate based on the obtained frame rate.

[0051] The data communication unit 31 is connected to the above-mentioned network NW, and transmits and receives various data to and from the server 40. The data communication unit 31 also transmits and receives various data to and from the external device 70 via the server 40.

[0052] For example, the control unit 25 of the in-vehicle device 10 can transmit, via the data communication unit 31, to the server 40, location identification information as location data that can identify the current location of the in-vehicle device 10, that is, in this embodiment, information on the current location of the automobile M. Furthermore, for example, the control unit 25 can transmit, via the data communication unit 31, to the server 40, information including a destination input by a user, and receive, from the server 40, route information or navigation information to the destination.

[0053] The control unit 25 can transmit audio data of the audio picked up by the microphone 17 to the external device 70 via the data communication unit 31 for audio calls in video communication. The control unit 25 can also receive audio data of the audio input to the external device 70 via the data communication unit 31 for audio calls in video communication.

[0054] Furthermore, the control unit 25 transmits the encoded data encoded by the encoder unit 30 to the external device 70 via the data communication unit 31. For example, the control unit 25 transmits the encoded data to the external device 70 via the data communication unit 31 while buffering it.

[0055] The control unit 25, the encoder unit 30, and the data communication unit 31 cooperate in the transmission process of encoded data during video connection in video communication, and function as a video transmission unit 32 that transmits video at a frame rate based on a frame rate linked to a point or section on a road indicated by the location information of the automobile M.

[0056] In this way, the in-vehicle device 10 transmits video at a frame rate associated with the position on the road based on the map information and in accordance with the position on the road where the automobile M is located. In other words, the in-vehicle device 10 generates video data for transmission at a frame rate associated with the position on the road, that is, encodes and transmits the video at the frame rate associated with the position on the road.

[0057] This can enhance the sense of sharing the video between the passengers of the automobile M and the user of the external device 70, allowing for smooth information sharing.

[0058] When delivering the video captured by the exterior camera 12 to the external device 70 via the server 40, the in-vehicle device 10 transmits the video at a low frame rate to reduce the amount of communication. For example, if the video is captured by the exterior camera 12 at 27.5 fps, the amount of communication can be reduced by lowering the frame rate to, for example, about 15 fps.

[0059] In this way, if the frame rate of the video to be transmitted is set low, the user of the external device 70 (hereinafter simply referred to as the external user) may feel uncomfortable when viewing the video sent from the in-vehicle device 10.

[0060] Specifically, when the frame rate is low, the external user may feel that the sense of speed visually obtained from the video, that is, the perceived speed, is faster than the actual speed of the automobile M. The difference between the actual perceived speed of the passengers of the automobile M (hereinafter simply referred to as passengers) and the perceived speed of the external user may weaken the sense of sharing the video between the passengers and the external user. Furthermore, in such a case, for example, there is a possibility that the external user may feel frightened.

[0061] Such differences in perceived speed, etc. may also be caused by differences in the scenery around the point or section on the road where the automobile M is located.

[0062] For example, if an external user views video at a low frame rate that is captured in a location where there are many objects such as fences and buildings near the road on which automobile M is traveling, for example, to the right front or left front, the external user may feel that the speed is faster or that the shaking is greater than that perceived by the occupants of automobile M. On the other hand, for example, if an external user views video at a low frame rate that is captured in a location where there are not many objects such as buildings near the road on which automobile M is traveling and where the scenery does not change much, the external user's perceived speed may be the same as the perceived speed of the occupants.

[0063] Therefore, in this embodiment, as described with reference to Figures 4 to 6, the in-vehicle device 10 associates appropriate frame rates with points or sections on a road (hereinafter also simply referred to as positions on a road) in advance, and uses map information in which the frame rates associated with the positions on the road are associated with position data indicating the positions on the road. For example, in the map information of this embodiment, points or sections that have a large impact on the perceived speed are associated with frame rates that are larger (higher) than points or sections that have a small impact on the perceived speed. Also, for example, points or sections with beautiful scenery may be associated with frame rates that are larger (higher) than other points or sections.

[0064] By using such map information to transmit video at a frame rate corresponding to the link on which automobile M is located, it is possible to reduce the amount of data communication while ensuring high reproducibility of the scenery seen by the driver of the mobile body when the operator of the external terminal views the video sent from the mobile body.

[0065] 7 is a block diagram showing the configuration of the server 40. For example, the server 40 is a device in which a mass storage device 43, a control unit 45, and a data communication unit 47 cooperate with each other via a system bus 41. The server 40 has a function similar to that of a SIP server that establishes a voice call between the in-vehicle device 10 and the external device 70 during video communication and transfers data of the voice call.

[0066] The server 40 also has the function of receiving from the in-vehicle device 10 the location identification information of the automobile M and information on the destination set by the user who is an occupant of the automobile M, and generating a route to the destination based on the location identification information and the destination information.

[0067] The server 40 also has a function of transferring the encoded data sent from the in-vehicle device 10 to the external device 70 .

[0068] The mass storage device 43 is configured by, for example, a hard disk drive and an SSD (solid state drive), and stores various programs such as an operating system and software for the server 40.

[0069] The large-capacity storage device 43 also includes a map information database (indicated as "map information DB" in the figure) 43A that stores map information including road maps. The map information in the map information database 43A is a database that contains information equivalent to the map information used in a navigation device, for example. The map information in the map information database 43A may also contain information indicating a video transmission frame rate in addition to the information equivalent to the map information used in a navigation device. That is, in the map information database 43A, the video transmission frame rate as described in FIG. 4 may be associated with position data indicating a point or section on a road.

[0070] The control unit 45 is configured with a CPU (Central Processing Unit) 45A, a ROM (Read Only Memory) 45B, a RAM (Random Access Memory) 45C, etc., and functions as a computer. The CPU 45A reads and executes various programs stored in the ROM 45B and the large-capacity storage device 43, thereby realizing various functions.

[0071] The data communication unit 47 is connected to the network NW, and transmits and receives various data between the in-vehicle device 10 and the external device 70.

[0072] The control unit 45 acquires location information indicating the current location of the automobile M from the in-vehicle device 10 via the data communication unit 47. The control unit 45 also acquires destination information input to the in-vehicle device 10 by an occupant of the automobile M from the in-vehicle device 10 via the data communication unit 47. The control unit 45 generates a route to the destination based on the location information and the destination information, and transmits information indicating the route to the in-vehicle device 10.

[0073] The control unit 45 also transfers the encoded data and audio received from the in-vehicle device 10 to the external device 70 via the data communication unit 47. The control unit 45 transfers the audio received from the external device 70 to the in-vehicle device 10 via the data communication unit 47.

[0074] Furthermore, the control unit 45 may acquire location information indicating the current location of the automobile M from the in-vehicle device 10, and transmit to the in-vehicle device 10 the video transmission frame rate associated with the acquired location information.

[0075] 8 is a front view showing the appearance of the external device 70. As described above, in the first embodiment, the external device 70 is a smartphone.

[0076] The touch panel 71 is, for example, a touch panel monitor that combines a display, such as a liquid crystal display, capable of displaying images, with a touch pad. The touch panel 71 is capable of generating signals that represent input operations received from a user onto the touch panel 71. In this embodiment, images distributed from the in-vehicle device 10 are displayed on the touch panel 71.

[0077] Furthermore, information about video communication or a screen for accepting operations for establishing a video connection may be displayed on the touch panel 71. The user of the external device 70 may perform an operation for establishing a video communication connection by performing an input operation on the touch panel 71.

[0078] The speaker 73 is capable of emitting sounds such as music and voices. During video communication, the speaker 73 emits the sound from the in-car device 10 during voice communication.

[0079] The microphone 75 is a microphone device that receives sounds emitted toward the external device 70. During video communication, the sounds collected by the microphone 75 are transmitted to the external device 70 as the sounds of the voice call.

[0080] 9 is a block diagram showing the configuration of the external device 70. For example, the external device 70 is a device in which a large-capacity storage device 83, a control unit 84, an input unit 85, an output unit 86, a data communication unit 87, and a decoder unit 88 cooperate with each other via a system bus 81.

[0081] The mass storage device 83 is configured by, for example, a hard disk drive, a solid state drive (SSD), a flash memory, etc., and stores various programs such as an operating system and software for the terminal.

[0082] The various programs may be acquired, for example, from another server device or the like via a network, or may be recorded on a recording medium and read via a drive device. That is, the various programs stored in the mass storage device 83 can be transmitted via a network, or can be recorded on a computer-readable recording medium and transferred.

[0083] The control unit 84 is configured with a CPU (Central Processing Unit) 84A, a ROM (Read Only Memory) 84B, a RAM (Random Access Memory) 84C, etc., and functions as a computer. The CPU 84A reads and executes various programs stored in the ROM 84B and the large-capacity storage device 83, thereby realizing various functions.

[0084] The input unit 85 is an input interface unit for the touch panel 71 and the microphone 75. The control unit 84 can receive, via the input unit 85, signals indicating input operations to the touch panel 71 and audio input signals from the microphone 75. For example, the control unit 84 can accept, via the input unit 85, a connection request made by the user via the touch panel 71 and the microphone 75.

[0085] The output unit 86 is an output interface to the touch panel 71 and the speaker 73. The control unit 84 can transmit a video or image signal to the touch panel 14 via the output unit 86 to display the image, or transmit an audio signal to the speaker 15 to output sound.

[0086] The data communication unit 87 is connected to the above-mentioned network NW, and transmits and receives various data to and from the server 40. The data communication unit 87 also transmits and receives various data, including encoded data of video transmitted from the in-vehicle device 10, to and from the in-vehicle device 10 via the server 40. For example, the control unit 84 functions as an encoded data receiving unit that receives encoded data via the data communication unit 87.

[0087] For example, the control unit 84 of the external device 70 can receive, via the data communication unit 31, location information capable of identifying the current location of the automobile M transmitted from the in-vehicle device 10 from the server 40. Furthermore, for example, the control unit 84 can receive route information or navigation information for the automobile M from the server 40 via the data communication unit 87.

[0088] Furthermore, the control unit 84 can transmit audio data of the audio picked up by the microphone 75 to the in-vehicle device 10 via the data communication unit 87 for audio communication in video communication. Furthermore, the control unit 84 can receive audio data transmitted from the in-vehicle device 10 via the data communication unit 87 for audio communication in video communication.

[0089] The decoder unit 88 as a video output unit decodes, plays back, and outputs encoded data received from the in-vehicle device 10 based on commands from the control unit 84. The decoder unit 88 decodes the encoded data using a codec used when encoding the encoded data, for example, an encoding method conforming to the MPEG-4 standard, and plays back and outputs the video. The played-back video is displayed on the touch panel 71 by the control unit 84.

[0090] In this way, the external device 70 receives and displays the video transmitted by the in-vehicle device 10.

[0091] [2. Operation of the video distribution system] The operation of the video distribution system 100 including the in-vehicle device 10, the server 40, and the external device 70 will be described below.

[0092] A description will be given of the control routines of the in-vehicle device 10 and the external device 70 for realizing the operation of the video distribution system 100 of the first embodiment. In the following description, it is assumed that when video distribution is started via video communication, the video transmission unit 32 starts encoding and transmitting the video at a low frame rate set to reduce the amount of communication traffic.

[0093] 10 is a flowchart showing a video distribution routine RT1 executed by the control unit 25 of the in-vehicle device 10. The control unit 25 starts the video distribution routine RT1 when the in-vehicle device 10 is powered on, for example, and repeatedly executes this routine.

[0094] First, the control unit 25 determines whether the in-vehicle device 10 has started video communication with the external device 70 (step S101). This determination is made, for example, based on whether video communication has been initiated in the in-vehicle device 10 and communication has been established with the external device 70. If the control unit 25 determines that video communication has not started (step S101: NO), it ends routine RT1.

[0095] When the control unit 25 determines that the in-vehicle device 10 has started video communication (step S101: YES), it acquires location information indicating the location of the automobile M based on a signal from the GPS receiver 11, acquires video from the outside camera 12, encodes the acquired video, and distributes it together with the location information to the external device 70 via the server 40, starting a video distribution operation (step S102).

[0096] In other words, in step S102, the control unit 25 functions as an image acquisition unit that sequentially acquires images from the outside camera 12. Also, in step S102, the control unit 25 functions as a location information acquisition unit that acquires location information indicating the location of the automobile M. Also, in step S102, the control unit 25 functions as a data transmission unit that encodes the image data at a default frame rate and transmits the encoded data. Note that, in step S102, the location information does not necessarily have to be transmitted to the external device 70.

[0097] When step S102 is executed and the video distribution operation is started, the control unit 25 determines whether the video communication between the in-car device 10 and the external device 70 has ended (step S103). If the control unit 25 determines that the video communication has not ended, that is, that the video communication is continuing (step S103: NO), it repeatedly executes step S103. That is, the video distribution operation continues as long as the video communication does not end.

[0098] When the control unit 25 determines in step S103 that the video communication with the external device 70 has ended, the control unit 25 ends the video distribution operation (step S104), and the routine R1 ends. In other words, when the control unit 25 determines that the video communication with the external device 70 has ended, the control unit 25 ends the acquisition of the location information, the acquisition of the video, and the encoding process and transmission of the video.

[0099] 11 is a flowchart showing the frame rate control routine RT2 executed by the control unit 25 of the in-vehicle device 10. The control unit 25 starts the frame rate control routine RT2, for example, when the in-vehicle device 10 is powered on, and executes this routine repeatedly. Note that in this embodiment, it is assumed that when a video distribution operation is started, encoding processing is initially performed at a default frame rate and video is transmitted.

[0100] First, the control unit 25 determines whether the in-car device 10 is performing a video distribution operation through video communication with the external device 70 (step S201).

[0101] When the control unit 25 determines that the in-vehicle device 10 is performing a video communication video distribution operation (step S201: YES), it acquires location information indicating the current location of the automobile M based on a signal from the GPS receiver 11 (step S202). Note that, if the control unit 25 has acquired location information in step S102 of the video distribution routine RT1, the control unit 25 may use the acquired location information in step S202.

[0102] After executing step S202, the control unit 25 refers to the map information in the map information database 23A and acquires (step S203) a frame rate associated with a point or section on a road indicated by the position information of the automobile M. In step S203, the control unit 25 acquires, as a video transmission frame rate, a frame rate associated with position data in the map information corresponding to the position information of the automobile M.

[0103] After executing step S203, the control unit 25 sets the frame rate of the video being distributed to the frame rate acquired in step S203 (step S204). In step S204, the control unit 25 instructs the encoder unit 30 to encode the video at the acquired frame rate.

[0104] In step S102 of the video distribution routine RT1 and steps S203 and S204 of the frame rate control routine RT2, the control unit 25, the encoder unit 30, and the data communication unit 31 cooperate in the video transmission process to function as a video transmission unit 32 that transmits video at a frame rate based on the frame rate associated with the point or section on the road indicated by the location information of the automobile M.

[0105] Step S102 of the video distribution routine RT1 corresponds to the location information acquisition step and video acquisition step in the video transmission method, program, and storage medium of the present invention. Also, step S203 of the frame rate control routine RT2 corresponds to the map information acquisition step in the video transmission method, program, and storage medium of the present invention. Step S102 of the video distribution routine RT1 and step S204 of the frame rate control routine RT2 correspond to the video transmission step in the video transmission method, program, and storage medium of the present invention.

[0106] If it is determined in step S201 that a moving image distribution operation of video communication is not being performed (step S201: NO), or after executing step S204, the control unit 25 ends the frame rate control routine RT2.

[0107] In step S204 of the frame rate control routine RT2, the control unit 25 may set the frame rate of the video being distributed to a frame rate based on the frame rate acquired in step S203. For example, in step S204, the frame rate of the video being distributed may be set to a frame rate equal to or higher than the frame rate acquired in step S203. In step S204, for example, the frame rate may be set to a frame rate obtained by adding a predetermined value to the frame rate acquired in step S203 (for example, adding 5 fps).

[0108] For example, in step S204, the control unit 25 may not change the frame rate setting if the frame rate of the video being distributed is equal to or higher than the frame rate acquired in step S203. Furthermore, if the frame rate acquired in step S203 is smaller than the initial frame rate value, the control unit 25 may set the frame rate to the initial value in step S204.

[0109] According to the video distribution routine RT1 and the frame rate control routine RT2, the in-vehicle device 10 sequentially acquires video from the exterior camera 12 that captures video around the automobile M, acquires location information of the automobile M, acquires map information including points or sections on roads to which frame rates are linked, and transmits the video captured by the exterior camera 12 at a frame rate based on the frame rate linked to the points or sections on the roads indicated by the location information. The transmitted video is displayed on the external device 70 via the server 40.

[0110] In the map information, a frame rate is associated with position data indicating a point or section on a road. The frame rate is predetermined and associated with each point or section on a road so that the video is displayed appropriately. The in-vehicle device 10 can always transmit the video at an appropriate frame rate by setting the frame rate of the video to be transmitted to a frame rate based on a frame rate corresponding to the point or section on the road where the automobile M is located. Therefore, the in-vehicle device 10 can appropriately transmit the video without, for example, a frame rate that is too low, which would impair the sense of sharing the video between the occupants of the automobile M and the user of the external device 70.

[0111] Therefore, it is possible to provide a video transmission device, a video transmission method, a video transmission program, and a recording medium that can ensure high reproducibility of the scenery seen by the driver of a mobile body when the operator of an external terminal views the video sent from the mobile body while suppressing the amount of data communication.

[0112] 12 and 13, a modified example of the frame rate control routine RT2 will be described, in which the server 40 specifies a frame rate suitable for video transmission according to the position of the automobile M, and the in-vehicle device 10 controls the frame rate based on the frame rate received from the server. The explanation of Figures 12 and 13 will be given on the assumption that, in the map information database 43A, the frame rate for video transmission is associated with position data indicating links on roads, as shown in Figure 4, for example.

[0113] 12 is a flowchart showing the frame rate control routine RT3 executed by the control unit 25 of the in-vehicle device 10. The control unit 25 starts the frame rate control routine RT3, for example, when the in-vehicle device 10 is powered on, and executes this routine repeatedly. Note that, as with the frame rate control routine RT2, the following description will be given assuming that when a video distribution operation is started, encoding is initially performed at a default frame rate and video is transmitted.

[0114] First, the control unit 25 determines whether the in-car device 10 is performing a video distribution operation through video communication with the external device 70 (step S301).

[0115] When the control unit 25 determines that the in-vehicle device 10 is performing a video communication moving image distribution operation (step S301: YES), it acquires location information indicating the current location of the automobile M based on the signal from the GPS receiver 11 and transmits the information to the server 40 (step S302). In step S302, the control unit 25 functions as a location information transmission unit that acquires and transmits location information of the moving object.

[0116] In step S302, the control unit 25 transmits the location information intermittently and continuously, for example, about once per second. Note that if the control unit 25 has acquired the location information in step S102 of the video distribution routine RT1, the control unit 25 may use the acquired location information in step S302.

[0117] After executing step S302, the control unit 25 waits to receive a frame rate from the server 40 (step S303). In step S303, the control unit 25 waits to receive a frame rate associated with a point or section on a road indicated by the position information of the automobile M.

[0118] After executing step S303, the control unit 25 determines whether or not a frame rate has been received from the server 40 (step S304). In step S304, the control unit 25 functions as a frame rate receiving unit that receives information indicating a frame rate associated with a point or section on a road indicated by the location information.

[0119] If the control unit 25 determines in step S304 that the frame rate has not been received (step S304: NO), it repeats step S304 and determines again whether or not the frame rate has been received.

[0120] If the control unit 25 determines in step S304 that the frame rate has been received (step S304: YES), it sets the frame rate of the video being distributed to the frame rate received in step S304 (step S305). In step S305, the control unit 25 instructs the encoder unit 30 to encode the video at the received frame rate.

[0121] In step S305, the control unit 25 may set the frame rate of the video being distributed to a frame rate based on the frame rate received in step S304. For example, in step S305, the frame rate of the video being distributed may be set to a frame rate equal to or higher than the frame rate received in step S304.

[0122] If the control unit 25 determines in step S301 that a moving image distribution operation of video communication is not being performed (step S301: NO), or after executing step S305, the control unit 25 ends the frame rate control routine RT3.

[0123] 13 is a flowchart showing the frame rate transmission routine RT4 executed by the control unit 45 of the server 40. For example, when the server 40 is powered on, the control unit 45 starts the frame rate transmission routine RT4 and executes it repeatedly.

[0124] First, the control unit 45 determines whether the in-car device 10 is performing a video communication video distribution operation with the external device 70 (step S401). In step S401, the control unit 45 determines that the in-car device 10 is performing a video communication video distribution operation with the external device 70 when the control unit 45 is performing a process of transferring encoded data received from the in-car device 10 to the external device 70 via the data communication unit 47.

[0125] When the control unit 45 determines that the in-vehicle device 10 is performing a video communication moving image distribution operation (step S401: YES), the control unit 45 waits for location information indicating the current location of the automobile M from the in-vehicle device 10 (step S402).

[0126] After executing step S402, the control unit 45 determines whether or not the position information of the automobile M has been received from the in-vehicle device 10 (step S403). In step S403, for example, the control unit 45 determines that the position information has been received when new position information of the automobile M is received via the data communication unit 47. In step S403, the control unit 45 functions as a position information receiving unit that receives the position information of the automobile M from the in-vehicle device 10 that moves together with the automobile M.

[0127] In step S403, if the control unit 45 determines that the position information has not been received (step S403: NO), it repeats step S403 and determines again whether or not the position information has been received.

[0128] In step S403, when the control unit 45 determines that the position information has been received (step S403: YES), it refers to the map information in the map information database 43A and identifies (step S404) a frame rate associated with the point or section on the road indicated by the position information of the automobile M. In step S404, the control unit 45 identifies a frame rate associated as a video transmission frame rate with the position data in the map information corresponding to the position information of the automobile M received in step S403.

[0129] After executing step S404, the control unit 45 transmits the frame rate determined in step S404 to the in-vehicle device 10 (step S405). In step S405, the control unit 45 transmits the frame rate to the in-vehicle device 10 via the data communication unit. In step S405, the control unit 45 functions as a frame rate transmission unit.

[0130] If the control unit 45 determines in step S401 that the in-car device 10 is performing a moving image distribution operation of video communication (step S401: NO), or after executing step S405, the control unit 45 ends the frame rate transmission routine RT4.

[0131] In step S102 of the video distribution routine RT1, steps S404 and S405 of the frame rate transmission routine RT4, and step S305 of the frame rate control routine RT3, the control unit 25, encoder unit 30, and data communication unit 31 of the in-vehicle device 10 cooperate in the video transmission process to function as a video transmission unit 32 that transmits video at a frame rate based on a frame rate linked to a point or section on a road indicated by the location information of the automobile M.

[0132] According to the frame rate control routine RT3 and the frame rate transmission routine RT4, the in-vehicle device 10 can obtain a frame rate associated with a point or section on a road corresponding to the position of the automobile M on the road based on the map information held by the server 40, and transmit the image captured by the exterior camera 12 at a frame rate based on the obtained frame rate.

[0133] Therefore, even if the server 40 has map information in which frame rate data is associated with position data, it is possible to transmit video at an appropriate frame rate.

[0134] Therefore, it is possible to provide a video transmission device, a video transmission method, a video transmission program, and a recording medium that can ensure high reproducibility of the scenery seen by the driver of a mobile body when the operator of an external terminal views the video sent from the mobile body while suppressing the amount of data communication.

[0135] In the above embodiment, the in-vehicle device 10 is an in-vehicle navigation device, but the in-vehicle device 10 may be various terminals such as a smartphone or a tablet.

[0136] The configurations, routines, etc. of the in-vehicle device 10, the server 40, and the external device 70 in the above-described embodiment are merely examples, and can be appropriately selected or changed depending on the application, etc.

[0137] In addition, in the above embodiment, the in-vehicle device 10 and the external device 70 perform video communication via the server 40, but the video communication may also be performed directly between the in-vehicle device 10 and the external device 70 via P2P (Peer to Peer) communication or the like.

[0138] In the above embodiment, the in-vehicle device 10 is a car navigation device connected to the touch panel 14, but the present invention is not limited to this. For example, the in-vehicle device 10 may communicate with a smartphone carried by the driver of the automobile M and display a screen related to video communication on the display of the smartphone instead of the touch panel 14. Furthermore, the in-vehicle device 10 may be configured not to display a screen to be presented to the driver of the automobile M.

[0139] For example, the in-vehicle device 10 may have a configuration similar to that of a drive recorder. Therefore, the in-vehicle device 10 may be integrated with, for example, an exterior camera 12. Specifically, the in-vehicle device 10 may be a device in which hardware that performs the functions of the in-vehicle device 10 is built into the housing of the exterior camera 12. In this case, the in-vehicle device 10 may not provide the various display outputs described above.

[0140] Furthermore, the video transmission device of the present application may be configured by integrating a terminal device (for example, FIG. 3) having a configuration similar to that of the in-vehicle device 10 in the above embodiment, the outside camera 12, and a display or touch panel.

[0141] In the above embodiment, an example has been described in which the image captured by the exterior camera 12 is transmitted to the external device 70, but the image transmitted to the external device 70 may be switched to the image captured by the interior camera 13. When the image captured by the interior camera 13 is being transmitted, the user of the external device 70 can, for example, communicate with the driver of the automobile M while viewing the interior of the automobile M.

[0142] The switching operation for switching the video to be transmitted to the external device 70 between the video from the outside camera 12 and the video from the inside camera 13 may be performed by the in-vehicle device 10. Alternatively, the switching operation may be performed remotely by operating the external device 70.

[0143] In the above embodiment, the control unit 84 of the external device 70 receives the image of the outside of the automobile M transmitted from the in-vehicle device 10 via the server 40, but in addition to the image, it may also receive a map image showing the current location of the automobile M and the planned route of travel, the name of the driver of the automobile M, the traveling speed of the automobile M, etc.

[0144] For example, the control unit 84 of the external device 70 may display on the touch panel 71 a map image, the name of the driver of the automobile M, and the traveling speed of the automobile M received from the in-vehicle device 10, superimposed on an image of the exterior of the automobile M, or may display them in a display area different from the image of the exterior of the automobile M. In this case, the map image, the name of the driver of the automobile M, and the traveling speed of the automobile M may be freely switchable between display and non-display by the operator of the external device 70.

[0145] For example, the control unit 25 of the in-vehicle device 10 may obtain the traveling speed of the automobile M based on the acceleration measured by an acceleration sensor provided in the automobile M and transmit the obtained speed together with the video during video communication. In addition, the control unit 25 may obtain the traveling speed calculated from the travel distance and time of the automobile M, or may obtain the traveling speed of the automobile M by receiving a vehicle speed signal (vehicle speed pulse).

[0146] In the above embodiment, an example has been described in which the in-vehicle device 10 is mounted on an automobile M, but the in-vehicle device 10 may be mounted on other moving objects such as a bicycle or a motorcycle. Also, the in-vehicle device may be held by a person, and video communication may be performed while the person is walking, for example, to distribute video.

[0147] In the above embodiment, the in-vehicle device 10 starts video distribution operation after communication between the in-vehicle device 10 and the external device 70 is established. However, video distribution may be performed in a manner similar to the video distribution of YouTube (registered trademark) or Niconico Live Broadcast (registered trademark) described above. That is, the video distribution operation may be started even if communication with a viewer's terminal such as the external device 70 is not established. Specifically, the in-vehicle device 10 may start uploading video data to the server 40 even if communication between the in-vehicle device 10 and the viewer's terminal such as the external device 70 is not established.

[0148] For example, the video distribution operation by the in-vehicle device 10 may be started after communication between the in-vehicle device 10 and the server 40 is established, without establishing a communication connection with the external device 70. In this case, an unspecified or authorized specific external device 70 can connect to the server 40 to receive the video distributed from the in-vehicle device 10, and the user of the external device 70 can watch the video.

[0149] In the video distribution system 100 of this embodiment, a case has been described in which a voice call is established between the in-vehicle device 10 and the external device 70, and then the video captured in the automobile M is distributed from the in-vehicle device 10 to the external device 70, but it is also possible that only the video is transmitted without establishing a voice call. [Explanation of symbols]

[0150] 100 Video Distribution System 10 Onboard equipment 23A, 43A map information database 25, 45, 84 Control section 30 Encoder section 31, 87 Data Communications Department 32 Video transmission unit 40 servers 70 External device 88 Decoder section

Claims

[Claim 1] A video transmission device that moves with a moving object, an image acquisition unit that sequentially acquires images of the surroundings of the moving object from an imaging device that captures the images; a location information acquisition unit that acquires location information of the moving object; a map information acquisition unit that acquires map information including points or sections on roads to which frame rates are associated; a video transmitting unit that transmits the video at a frame rate based on a frame rate associated with the point or section on the road indicated by the location information; A video transmission device comprising:

Images