Distribution system, computer program product and distribution method

Abstract

The present application provides a distribution system, a computer program product, and a distribution method, in which a viewer (U) of an image can further obtain a sense of being on a vehicle (3). The distribution system (1) includes a distribution unit (356) that distributes an image obtained by imaging by an imaging device (200A) provided on a vehicle, a terminal device (200B) that is provided at a location other than the vehicle and displays the distributed image, and an addition unit (355) that adds sound to the image, the addition unit adding an external environment sound of the vehicle based on position information of the vehicle, the terminal device displaying the image and outputting the external environment sound added to the image.

Description

Technical Field

[0001] This invention relates to distribution systems, computer program products, and distribution methods. Background Technology

[0002] In recent years, initiatives to provide sustainable transportation systems that take into account vulnerable groups among transportation participants, such as the elderly, disabled, and children, have become increasingly active. To realize these initiatives, research and development efforts are focused on further improving the safety and convenience of transportation through the development of mobility means for vulnerable groups. The use of transportation systems by vulnerable groups includes not only riding in the vehicle themselves, but also viewing methods of dynamic images distributed by image distributors (for example, see Patent Document 1). Patent Document 1 discloses devices for transmitting images captured by a camera unit installed in a vehicle to locations other than the vehicle, and devices for transmitting voice messages from inside and outside the vehicle to locations other than the vehicle.

[0003] [Existing technical documents]

[0004] [Patent Literature]

[0005] Patent Document 1: Japanese Patent Application Publication No. 2024-039702 Summary of the Invention

[0006] [The problem the invention aims to solve]

[0007] While sounds other than speech are generally audible inside a vehicle, Patent Document 1 is a structure that transmits both images and speech, without considering sounds other than speech. Therefore, in the structure of Patent Document 1, viewers of images transmitted from the vehicle may find it difficult to perceive that they are actually riding in a vehicle.

[0008] The present invention was made in view of the above circumstances, and its purpose is to enable viewers of images to further experience the feeling of being in a vehicle. Moreover, by connecting remote locations and vehicles, it helps to support good connections between urban areas and local regions, represented by urban peripheries, and contributes to the development of sustainable transportation systems.

[0009] [Methods used to solve problems]

[0010] One aspect of the present invention is a distribution system comprising: a distribution unit that distributes an image captured by a camera installed on a vehicle; a terminal device installed at a location outside the vehicle that displays the distributed image; and an attachment unit that attaches sound to the image, the attachment unit attaching external ambient sound of the vehicle based on the vehicle's location information, the terminal device displaying the image and outputting the external ambient sound attached to the image.

[0011] [Invention Effects]

[0012] According to one aspect of the invention, the viewer of the image is further able to perceive the feeling of being in a vehicle. Attached Figure Description

[0013] Figure 1 This is a diagram showing the system architecture of the distribution system.

[0014] Figure 2 This is a diagram showing an example of a distributor terminal installed in a vehicle.

[0015] Figure 3 This is a diagram showing an example of a distributor terminal installed in a vehicle.

[0016] Figure 4 This is a diagram showing the structure of the vehicle and the distribution server.

[0017] Figure 5 This is a diagram showing the functional units of the first processor.

[0018] Figure 6 It is a diagram used to illustrate the area of ​​focus.

[0019] Figure 7 This is a diagram showing the structure of the user information database.

[0020] Figure 8 This is a diagram showing the structure of the distribution management database.

[0021] Figure 9 This is a block diagram showing the structure of the viewer terminal.

[0022] Figure 10 This is a flowchart illustrating the actions of the vehicle-mounted device, distribution server, and viewer terminal; Figure 11 This is a flowchart illustrating the operation of the additional part. Detailed Implementation

[0023] [1. Structure of the distribution system]

[0024] Figure 1 This is a diagram showing the system architecture of distribution system 1.

[0025] The distribution system 1 includes a vehicle 3, a distributor terminal 200A, a distribution server 100, and a viewer terminal 200B. It is a system for live streaming content containing images captured by the distributor terminal 200A.

[0026] Live streaming refers to the real-time distribution of images captured by the distributor terminal 200A to the viewer terminal 200B. The images can be still images or moving images.

[0027] Distributor terminal 200A is an example of a "shooting device". Viewer terminal 200B is an example of a "terminal device".

[0028] Vehicle 3 is equipped with an on-board unit 30. The on-board unit 30 is wirelessly connected to a distributor terminal 200A held by a distributor S, who is an occupant of vehicle 3, via near-field communication or the like. The on-board unit 30 receives images captured by a camera equipped on the distributor terminal 200A.

[0029] Additionally, the vehicle-mounted device 30 receives signals from the in-vehicle microphone 37 (see reference 37). Figure 4 The collected sound. In addition, the in-vehicle microphone 37 is a microphone installed inside the vehicle 3.

[0030] Additionally, the vehicle-mounted device 30 receives signals from the external microphone 38 (see reference 38). Figure 4 The collected sound. In addition, the external microphone 38 is a microphone installed on the exterior of the vehicle body 3.

[0031] The vehicle-mounted device 30 uploads the received image and sound data to the distribution server 100.

[0032] The transmitter terminal 200A can be a smartphone or tablet PC, or a digital camera or a 360-degree camera capable of capturing the entire surroundings. Alternatively, when the transmitter terminal 200A is a smartphone or tablet PC, the in-vehicle microphone 37 can also be the transmitter terminal 200A.

[0033] The distributor terminal 200A is installed in vehicle 3.

[0034] Figure 2 and Figure 3 This is a diagram showing an installation example of the distributor terminal 200A installed in vehicle 3.

[0035] Figure 2 and Figure 3 An example is shown where the distributor terminal 200A is installed in the passenger seat 10 of the vehicle 3.

[0036] The distributor terminal 200A is installed on the seat 10 via the mounting accessory 50. The mounting accessory 50 is installed on the headrest support 13 that secures the headrest 12 of the seat 10, which serves as the passenger seat, to the seat back 11.

[0037] Figure 2An example is shown where the distributor terminal 200A is mounted on the passenger seat 10, with the front of the vehicle 3 visible from the distributor terminal 200A's perspective. This mounting position of the distributor terminal 200A is referred to as the first position. Figure 3 This illustration shows an example where the distributor terminal 200A is installed on the passenger seat 10, which is the front passenger seat, in a manner that includes the distributor S sitting in the driver's seat from the perspective of the distributor terminal 200A. This installation position of the distributor terminal 200A is referred to as the second position.

[0038] A first fastener 51 is provided at one end of the mounting accessory 50 to secure the distributor terminal 200A to the mounting accessory 50, and a second fastener is provided at the other end of the mounting accessory 50 to secure the mounting accessory 50 to the headrest support 13. (The second fastener is not shown in the diagram.) After changing the mounting direction of the mounting accessory 50 to the headrest support 13, the mounting accessory 50 is secured to the headrest support 13 by the second fastener, thereby allowing the distributor terminal 200A to be mounted in either a first or second position on the passenger seat 10.

[0039] return Figure 1 As explained, the distribution server 100 distributes the content distributed from vehicle 3 to the viewer terminal 200B of viewer U. This distribution can be implemented using known technologies such as HLS (HTTP Live Streaming), CMAF (Common Media Application Format), WebRTC (Web Real-Time Communications), and RTMP (Real-Time Messaging Protocol).

[0040] Viewer terminal 200B is a terminal device held by viewer U who is watching content distributed from distribution server 100. Figure 1 In the diagram, N audience members U are represented by U1, U2, ..., UN. N is any natural number.

[0041] Viewer terminal 200B can be a portable or handheld terminal device such as a smartphone, tablet, or laptop, or a fixed terminal device such as a desktop computer. Alternatively, viewer terminal 200B can be a television receiver or a head-mounted display device in which an image display unit is worn on the viewer's head and a display screen of the image display unit is positioned in front of the viewer's eyes. The head-mounted display device can be either transmissive or non-transmissive, as long as it can be capable of viewing and listening to content distributed from the distribution server 100.

[0042] Viewer terminal 200B is equipped with display 240 (see reference) Figure 9) and speaker 260 (reference) Figure 9 The image distributed from the distribution server 100 is displayed on the monitor 240, and the voice distributed from the distribution server 100 is output through the speaker 260.

[0043] In addition, the audience terminal 200B has a microphone 250 (see reference). Figure 9 The microphone 250 collects the voice data of the audience member U and uploads it to the distribution server 100. The distribution server 100 then sends the uploaded voice data of the audience member U to the vehicle-mounted device 30. Furthermore, the vehicle-mounted device 30 transmits the data to the speaker 36 (see reference 1). Figure 4 The system outputs the received voice data of the audience U into the vehicle's interior. Thus, the audience U and the distributor S can converse during content distribution.

[0044] In addition, the audience terminal 200B has a camera 230 (see reference). Figure 9 The system detects the gaze direction of the viewer U relative to the image displayed on the monitor 240 from the image captured by the camera 230. When the viewer U's gaze direction is detected, the viewer terminal 200B uploads gaze direction data representing the detected gaze direction to the distribution server 100. The distribution server 100 then sends the uploaded gaze direction data to the vehicle-mounted device 30.

[0045] Communication network 5 is a wired or wireless transmission path for information sent from devices connected to communication network 5. For example, communication network 5 may also include public networks such as the Internet, mobile communication networks, and satellite communication networks, as well as various LANs (Local Area Networks) and WANs (Wide Area Networks) including Ethernet. Additionally, communication network 5 may also include private networks such as IP-VPN (Internet Protocol-Virtual Private Network).

[0046] [2. Vehicle Structure]

[0047] Figure 4 A diagram illustrating the structure of vehicle 3 and distribution server 100.

[0048] First, refer to Figure 4 The structure of vehicle 3 will be explained.

[0049] Vehicle 3 includes an onboard device 30, an external communication interface 31, a first short-range wireless communication interface 32, an internal communication interface 33, a touch panel 34, a voice processing unit 35, a speaker 36, an internal microphone 37, an external microphone 38, and a sensor group 39. Hereinafter, the interface will be abbreviated as "interface".

[0050] First, the following components will be described: external communication interface 31, first short-range wireless communication interface 32, internal communication interface 33, touch panel 34, voice processing unit 35, speaker 36, internal microphone 37, external microphone 38, and sensor group 39.

[0051] The external communication interface 31 is a communication module that performs wireless communication according to mobile communication standards such as LTE (Long Term Evolution), 4G (Generation), 5G, and Wi-Fi. The external communication interface 31 is connected to the communication network 5 via an antenna (not shown) and exchanges data with the distribution server 100. Furthermore, the external communication interface 31 may be, for example, a TCU (Telematics Control Unit).

[0052] The first near-field wireless communication interface 32 is, for example, a communication device that performs wireless communication according to near-field wireless communication standards such as Bluetooth (registered trademark) and Wi-Fi. The first near-field wireless communication interface 32 is, for example, a communication module that communicates according to NFC (Near Field Communication) communication standards such as Bluetooth and Wi-Fi. The first near-field wireless communication interface 32 performs near-field wireless communication with a terminal device held by the occupant, receiving images and voice recordings from the terminal device.

[0053] The in-vehicle communication interface 33 is an input / output interface connected to an in-vehicle network (not shown). The in-vehicle network may use a communication bus corresponding to communication standards such as CAN (Controller Area Network), CAN-FD (CAN with Flexible Data Rate), or Ethernet. The in-vehicle communication interface 33 communicates with vehicle ECU 40, other relay devices, and other in-vehicle equipment via the in-vehicle network.

[0054] The touch panel 34 includes a display panel such as a liquid crystal panel or an organic EL (Electro-Luminescence) panel, and a touch sensor. The touch sensor detects the touch operation of the viewer U on the touch panel 34 and outputs position information indicating the location of the detected touch operation to the first control unit 330.

[0055] The voice processing unit 35, for example, converts the digital voice data output by the first control unit 330 into an analog voice signal. The voice processing unit 35 amplifies the converted analog voice signal and outputs the amplified voice signal through the speaker 36.

[0056] An in-vehicle microphone 37 is positioned at a designated location within the interior of vehicle 3 to collect ambient sounds (hereinafter, appropriately abbreviated as "ambient sounds") inside vehicle 3. When the transmitter S speaks inside the vehicle, the ambient sounds include the transmitter S's voice. The in-vehicle microphone 37 outputs the collected ambient sound data to the vehicle-mounted device 30.

[0057] An external microphone 38 is installed on the exterior of the vehicle 3 to collect ambient sounds from the outside of the vehicle 3 (hereinafter, as appropriate, it is referred to as "ambient sounds"). The external microphone 38 outputs the collected ambient sound data to the vehicle-mounted device 30.

[0058] In this embodiment, the vehicle 3 is equipped with multiple external microphones 38. Furthermore, in this embodiment, each external microphone 38 is installed in different locations on the exterior of the vehicle 3. For example, external microphones 38 are installed on the exterior of the vehicle 3 at the front right, front center, front left, right, left, rear right, and rear left sides. More specifically, in this embodiment, the external microphones 38 are installed at locations corresponding to the gaze area that the determination unit 354 can determine. The determination unit 354 and the gaze area will be described later.

[0059] The sensor group 39 is equipped with various sensors. In this embodiment, the sensor group 39 includes an acceleration sensor, a vehicle speed sensor, a GNSS (Global Navigation Satellite System) unit, and a raindrop sensor.

[0060] The accelerometer is a sensor that detects the acceleration of vehicle 3. The accelerometer detects the acceleration of vehicle 3 at a predetermined period, and outputs the detected value to the on-board unit 30 each time the acceleration of vehicle 3 is detected.

[0061] The vehicle speed sensor is a sensor that detects the speed of vehicle 3. The vehicle speed sensor detects the speed of vehicle 3 at a predetermined period, and outputs the detected value to the on-board unit 30 each time the speed of vehicle 3 is detected.

[0062] The GNSS unit locates the current position of vehicle 3. The GNSS unit generates position data representing the current position of vehicle 3 and outputs the generated position data to the on-board device 30.

[0063] A rain sensor, for example, is a sensor installed on the windshield of vehicle 3 to detect raindrops. The rain sensor outputs the detection result to the on-board unit 30.

[0064] Raindrops are an example of "objects".

[0065] The vehicle-mounted device 30 includes an input / output interface 310 and a first control unit 330.

[0066] The input / output interface 310 functions as a connection point between the external communication interface 31, the first short-range wireless communication interface 32, the in-vehicle communication interface 33, the touch panel 34, the voice processing unit 35, the in-vehicle microphone 37, the external microphone 38, and the sensor group 39, and the first control unit 330. The input / output interface 310 includes connectors and interface circuitry, and outputs data input from the external communication interface 31, the first short-range wireless communication interface 32, the in-vehicle communication interface 33, the touch panel 34, the voice processing unit 35, the in-vehicle microphone 37, the external microphone 38, and the sensor group 39 to the first control unit 330. Furthermore, the input / output interface 310 outputs data input from the first control unit 330 to the external communication interface 31, the first short-range wireless communication interface 32, the in-vehicle communication interface 33, the touch panel 34, and the voice processing unit 35.

[0067] The first control unit 330 is a computer device having a first memory 340 and a first processor 350.

[0068] The first processor 350 is an example of a "processor".

[0069] The first memory 340 may be composed of, for example, volatile semiconductor memory and non-volatile semiconductor memory. The first memory 340 may also consist only of non-volatile semiconductor memory. Alternatively, the first memory 340 may have an auxiliary storage device such as an SSD (Solid State Drive). The first memory 340 stores control programs 345, map data 346, location data 347, and setting data specifying the operation of the vehicle-mounted device 30, etc., executed by the first processor 350.

[0070] Control program 345 is an example of a "program".

[0071] Map data 346 stores road map information, building information, and other data. Road map information consists of a road network represented by lines on the map, including information related to road segments obtained by dividing the map into multiple parts as nodes such as intersections and branch points, and defining the parts between each node as road segments. Building information shows the location of buildings, the name of buildings, and the shape of buildings when viewed from above.

[0072] Location data 347 is data recording the location where external ambient sound is added by the appendix 355 (described later). The location recorded in location data 347 is predetermined.

[0073] The first processor 350 is, for example, a CPU (Central Processing Unit) or an MPU (Micro-Processing Unit) or other computing device. The first processor 350 may consist of a single processor or multiple processors. Alternatively, the first processor 350 may also consist of a System-on-a-Chip (SoC) that integrates part or all of the first memory 340 with other circuitry.

[0074] Figure 5 This is a diagram showing the functional units of the first processor 350.

[0075] The first processor 350 functions as a first acquisition unit 351, a second acquisition unit 352, a third acquisition unit 353, a determination unit 354, an additional unit 355, and a distribution unit 356 by reading and executing the control program 345 stored in the first memory 340.

[0076] [2-1. First Acquisition Section]

[0077] The first acquisition unit 351 acquires external ambient sound through the external microphone 38. The first acquisition unit 351 outputs the acquired external ambient sound data to the auxiliary unit 355.

[0078] [2-2. Second Acquisition Section]

[0079] The second acquisition unit 352 acquires the interior ambient sound through the in-vehicle microphone 37. The second acquisition unit 352 outputs the acquired interior ambient sound data to the third acquisition unit 353 and the auxiliary unit 355.

[0080] [2-3. Third Acquisition Section]

[0081] The third acquisition unit 353 acquires the conversation content between the viewer U and the distributor S during the content distribution process. The third acquisition unit 353 performs speech recognition on the ambient sound acquired by the second acquisition unit 352 and the speech of the viewer U sent from the distribution server 100, thereby acquiring the conversation content between the viewer U and the distributor S through text data. Furthermore, the speech recognition is performed using existing techniques that reference acoustic models and language models. The third acquisition unit 353 outputs text data showing the acquired conversation content to the auxiliary unit 355.

[0082] [2-4. Determining the Part]

[0083] The determination unit 354 determines the gaze region based on the viewer U's gaze direction during the content distribution process. The gaze region refers to the area in the distributed image that the viewer U is currently looking at. Based on the gaze direction data sent from the distribution server 100, the determination unit 354 determines the area of ​​the image that the viewer U's gaze direction is continuously directed towards for a predetermined time as the gaze region.

[0084] Figure 6 It is a diagram used to illustrate the area of ​​focus.

[0085] exist Figure 6 The example shows a viewer U watching image P1. Figure 6 Image P1 shown is an image taken from the passenger seat of vehicle 3, through the windshield, showing the area in front of vehicle 3. Additionally, in Figure 6 In the figure, the object marked with the reference numeral "LE" is the left eye, and the object marked with the reference numeral "RE" is the right eye.

[0086] In addition, Figure 6 The example shown illustrates the case where the determination unit 354 determines a gaze region from the first region LA, the second region CA, and the third region RA.

[0087] exist Figure 6 The example illustrates a scenario where the viewer U's gaze is directed to the right relative to image P1, and the viewpoint IP is located within the third region RA on the right. Therefore, in Figure 6 In the example, the determination unit 354 determines that the gaze region is the third region RA on the right side of image P1. Furthermore, the determination unit 354 determines which of the first region LA, the second region CA, and the third region RA the viewpoint IP is located in when the gaze direction is determined. Therefore, in Figure 6 In the example described above, the determination unit 354 determines that the gaze region is the third region RA.

[0088] Furthermore, when the viewer U's gaze direction is to the left relative to image P1 and the viewpoint IP is within the first region LA on the left, the determination unit 354 determines that the gaze area in image P1 is the first region LA on the left. Additionally, when the viewer U's gaze direction is approximately at the center of the long side of image P1 and the viewpoint IP is within the second region CA, the determination unit 354 determines that the gaze area is the second region CA sandwiched between the first region LA and the third region RA.

[0089] in addition, Figure 6 This is merely one example used to illustrate the determination of the gaze area. Therefore, the determination unit 354 can determine the gaze area from four or more areas, or from two areas: the left-side area and the right-side area. However, it is preferable to provide an external microphone 38 on the exterior of the vehicle body, which is located within the gaze area that the determination unit 354 can determine.

[0090] After determining the gaze area, the determination unit 354 outputs data representing the determined gaze area to the auxiliary unit 355.

[0091] [2-5. Additional Section]

[0092] The adder unit 355 adds sound to the image captured by the distributor terminal 200A. In this embodiment, the adder unit 355 adds at least one of internal ambient sound, external ambient sound, and vehicle-related sound. Furthermore, vehicle-related sound refers to sounds associated with the vehicle 3, such as engine sound, turn signal sound, and tire noise. The turn signal sound is the sound emitted when the turn indicator is operated.

[0093] [2-5-1. Additional Interior Ambient Sound]

[0094] First, we will describe the additional ambient sounds.

[0095] When the second acquisition unit 352 acquires the ambient sound, the attachment unit 355 attaches the ambient sound acquired by the second acquisition unit 352. Furthermore, the second acquisition unit 352 acquiring the ambient sound means that the second acquisition unit 352 acquires an ambient sound at or above a predetermined volume.

[0096] [2-5-2. Additional external ambient sound based on vehicle location information]

[0097] Next, we will describe the addition of external ambient sound.

[0098] The adder unit 355 adds external ambient sound based on the location information of the vehicle 3. Based on the location information of the vehicle 3 detected by the GNSS unit of the sensor group 39 and map data 346, the adder unit 355 determines the location corresponding to the current location of the vehicle 3. Next, the adder unit 355 determines whether the determined location is recorded in the location data 347, and if the determined location is recorded in the location data 347, adds the external ambient sound acquired by the first acquisition unit 351.

[0099] In addition, the additional unit 355 may also be attached with external ambient sounds stored in the first memory 340.

[0100] In this structure, location data 347 includes appropriate external ambient sound data for each location. For example, location data 347 stores data on the sound of waves for a location like a "coastal road." Additionally, for a location like "in the mountains," location data 347 stores data on bird calls, the sound of leaves rubbing, the sound of cold wind, the sound of waterfalls, etc. Furthermore, for a location like "intersections," location data 347 stores data on the sound of traffic lights. Finally, for a location like "busy streets," location data 347 stores data on the noise of bustling streets.

[0101] Furthermore, in the case of this structure, for example, if the determined location is a coastal road, the appendix 355 adds the sound of waves as an external ambient sound. Additionally, for example, if the determined location is in the mountains, the appendix 355 adds the sounds of birdsong, leaf rustling, cold wind, waterfalls, etc., as external ambient sounds. Furthermore, for example, if the determined location is an intersection, the appendix 355 adds the sound of traffic lights as an external ambient sound. Additionally, for example, if the determined location is a busy street, the appendix 355 adds the noise of the busy street as an external ambient sound.

[0102] [2-5-2. Additional ambient sounds during rainy weather]

[0103] When it is raining outside vehicle 3, the add-on unit 355 adds rain sound as an external ambient sound. More specifically, when the rain sensor of sensor group 39 detects raindrops, the add-on unit 355 adds rain sound as an external ambient sound. Furthermore, the rain sound added by the add-on unit 355 may include the sound produced when rain hits the vehicle body, or it may include the sound produced when rain hits the ground.

[0104] The sound of rain is an example of "object-related sound".

[0105] The attachment unit 355 can attach rain sounds based on the rain sound data stored in the first memory 340, or it can attach external ambient sounds acquired by the first acquisition unit 351 as rain sounds.

[0106] It should be noted that when it is raining outside vehicle 3, the auxiliary unit 355 can add water splashing sound as an external ambient sound along with or instead of the rain sound. Water splashing sound refers to the sound made when vehicle 3 passes through puddles. In this case, the auxiliary unit 355 uses the water splashing sound data stored in the first memory 340 to add the water splashing sound.

[0107] [2-5-3. Additional Vehicle-Related Sounds]

[0108] Next, we will provide an additional explanation of the vehicle-related audio.

[0109] The additional part 355 adds engine sound based on vehicle information related to the vehicle 3 being driven.

[0110] For example, the auxiliary unit 355 obtains the detection value from the acceleration sensor of the sensor group 39 as vehicle information related to the vehicle 3 being driven, and adds engine sound when the obtained detection value indicates an increase in acceleration.

[0111] Additionally, for example, the auxiliary unit 355 obtains the detection value from the vehicle speed sensor of the sensor group 39 as vehicle information related to the vehicle 3 being driven, and adds engine sound when the obtained detection value indicates an increase in speed.

[0112] In addition, the engine sound is stored in the first memory 340, and the engine sound stored in the first memory 340 is attached to the additional part 355.

[0113] The auxiliary unit 355 adds a turn signal audible signal based on vehicle information related to the vehicle 3 being driven. The auxiliary unit 355 obtains information from designated sensors regarding whether the turn indicator has been activated as vehicle information related to the vehicle 3 being driven. If the auxiliary unit 355 obtains information indicating that the turn indicator has been activated, it adds a turn signal audible signal.

[0114] In addition, the first memory 340 stores data of the turn signal indicator sound, and the auxiliary unit 355 adds the turn signal indicator sound stored in the first memory 340.

[0115] The additional part 355 adds the sound of the tires driving based on vehicle information related to the vehicle 3 being driven.

[0116] For example, the auxiliary unit 355 obtains the detection value from the acceleration sensor of the sensor group 39 as vehicle information related to the vehicle 3 being driven, and adds the sound of the tires driving when the obtained detection value indicates an increase in acceleration.

[0117] Additionally, for example, the auxiliary unit 355 obtains detection values ​​from the vehicle speed sensor of the sensor group 39 as vehicle information related to the vehicle 3 being driven, and adds the sound of the tires driving when the obtained detection value indicates an increase in speed.

[0118] In addition, the first memory 340 stores data on tire driving sounds, and the additional part 355 adds the tire driving sounds stored in the first memory 340.

[0119] [2-5-3. Setting the volume of additional sounds]

[0120] The add-on unit 355 sets the volume of the sound attached to the image captured by the distributor terminal 200A.

[0121] When adding internal ambient sound and external ambient sound, the auxiliary unit 355 sets the volume of the added internal ambient sound to be greater than the volume of the added external ambient sound.

[0122] Furthermore, if the second acquisition unit 352 fails to acquire the internal ambient sound for a predetermined period of time or more when the external ambient sound is acquired instead of the internal ambient sound, the volume of the acquired external ambient sound is set to be greater than the volume of the external ambient sound when the internal ambient sound is acquired.

[0123] Furthermore, the attachment unit 355 sets the volume of the external ambient sound to be attached based on the gaze area determined by the determination unit 354. In detail, the attachment unit 355 sets the volume of the external ambient sound corresponding to the gaze area determined by the determination unit 354 to be greater than the volume of the external ambient sound corresponding to the area outside the gaze area determined by the determination unit 354.

[0124] For example, suppose the gaze area determined by the determining unit 354 is Figure 6 The example shows the third region RA. In Figure 6 In the example case, the attachment 355 sets the volume of the external ambient sound collected by the external microphone 38 located at the right front of the vehicle 3 to be greater than the volume of the external ambient sound collected by external microphones other than the external microphone 38, and attaches it to the image P1.

[0125] Additionally, the add-on unit 355 sets the volume of the added sound based on the conversation content acquired by the third acquisition unit 353. Furthermore, the add-on unit 355 identifies the conversation content through text parsing.

[0126] For example, if the conversation content acquired by the third acquisition unit 353 is related to "weather" and includes rain sounds, the attachment unit 355 sets the volume of the added rain sounds to be louder than the added rain sounds if the conversation content is not related to "weather".

[0127] Furthermore, for example, if the conversation content acquired by the third acquisition unit 353 is related to "scenery" and external ambient sound is added based on the location information of vehicle 3, the addition unit 355 sets the volume of the added external ambient sound to be greater than the external ambient sound added when the conversation content is not related to "scenery".

[0128] Additionally, for example, if the conversation content acquired by the third acquisition unit 353 is related to "car" and is accompanied by vehicle-related sounds, the attachment unit 355 sets the volume of the accompanying vehicle-related sounds to be louder than the accompanying vehicle-related sounds if the conversation content is not related to "car".

[0129] [2-6. Distribution Department]

[0130] The distribution unit 356 distributes the image with sound added by the attachment unit 355 to the viewer terminal 200B via the external communication interface 31. More specifically, the distribution unit 356 uploads the content data of the image containing the sound added by the attachment unit 355, along with the distribution ID (identification), to the distribution server 100 via the external communication interface 31.

[0131] [3. Structure of the distribution server]

[0132] Next, we will refer to Figure 4 Describe the structure of distribution server 100.

[0133] The distribution server 100 includes a network communication interface 110 and a second control unit 130.

[0134] The network communication interface 110 is equipped with a communication card such as a NIC (Network Interface Card), and communicates with the vehicle 3 and the audience terminal 200B via the communication network 5.

[0135] The second control unit 130 is a computer device equipped with a second memory 140 and a second processor 150.

[0136] The second memory 140 may be configured as either a volatile semiconductor memory or a non-volatile semiconductor memory. Alternatively, the second memory 140 may only have a non-volatile semiconductor memory. Furthermore, the second memory 140 may also have a structure that includes auxiliary storage devices such as HDDs (Hard Disk Drives) or SSDs.

[0137] The second memory 140 stores the control program 141, the user information database 143, and the distribution management database 145 executed by the second processor 150.

[0138] Figure 7 This is a diagram showing the structure of the user information database 143.

[0139] A record in the user information database 143 includes the user ID, APP_ID, device information, and user information. APP is an abbreviation for application.

[0140] The user ID is identification information used to identify users who use the distribution service provided by the distribution server 100. Users include distributors S and viewers U.

[0141] APP_ID is identification information that identifies the application downloaded to the distributor terminal 200A or the viewer terminal 200B.

[0142] Distributor S and audience U download the APP from the designated website and install it on distributor terminal 200A and audience terminal 200B respectively.

[0143] The device information refers to information about the viewer terminal 200B. This device information includes, for example, information indicating whether the viewer terminal 200B is a smartphone, PC, head-mounted display device, or camera, such as its model number. Furthermore, the device information includes communication information used to communicate with the viewer terminal 200B. Address information is listed as an example of this communication information.

[0144] User information refers to the personal information of the distributor S and the audience U, such as name, address, gender, age, etc.

[0145] Figure 8 This is a diagram showing the structure of the distribution management database 145.

[0146] A record in the distribution management database 145 contains the distribution ID that identifies the distribution, the user ID of the distributor S, the user ID of the audience U, and the date and time the distribution started.

[0147] return Figure 4 Next, we will explain the structure of the distribution server 100.

[0148] The second processor 150 is, for example, a CPU, MPU, or other computing device. The second processor 150 can be composed of a single processor or multiple processors. Alternatively, the second processor 150 can also be composed of a System-on-a-Chip (SoC) that integrates part or all of the second memory 140 with other circuitry.

[0149] The second control unit 130 reads and executes the control program 141 through the second processor 150 to perform various actions.

[0150] [4. Structure of the viewer terminal]

[0151] Figure 9 This is a structural block diagram of the 200B viewer terminal.

[0152] The audience terminal 200B includes a wireless communication interface 210, a second short-range wireless communication interface 220, a camera 230, a display 240, a microphone 250, a speaker 260, and a third control unit 270.

[0153] The wireless communication interface 210 is a communication module that performs wireless communication according to mobile communication standards such as LTE, 4G, 5G, and Wi-Fi. The wireless communication interface 210 is connected to the communication network 5 via an antenna (not shown) and exchanges data with the distribution server 100.

[0154] The second near-field wireless communication interface 220 is, for example, a communication module that communicates according to NFC communication standards such as Bluetooth and Wi-Fi. The second near-field wireless communication interface 220 transmits images captured by the camera 230 and voice input by the microphone 250 to the vehicle-mounted device 30 via near-field wireless communication.

[0155] Camera 230 is a shooting device equipped with an image sensor and data processing circuitry. The image sensor may be, for example, a CCD (Charge-Coupled Device) or CMOS (Complementary Metal-Oxide-Semiconductor). The data processing circuitry generates a captured image based on the output of the image sensor. The shooting range of camera 230 is set to include the field of vision of the viewer U. The captured image generated by camera 230 is output to the third control unit 270.

[0156] Microphone 250 is a voice input device for inputting the voice of the transmitter S or the audience U, and speaker 260 is a voice output device for outputting the voice of the transmitter S or the audience U.

[0157] The display 240 includes a display panel such as a liquid crystal panel or an organic EL panel, and a touch sensor. The display 240 functions as a display unit for displaying images and a receiving unit for receiving operations from the distributor S and the viewer U.

[0158] The third control unit 270 is a computer device equipped with a third memory 280 and a third processor 290.

[0159] The third memory 280 is a non-volatile semiconductor memory. Alternatively, the third memory 280 may also have an auxiliary storage device such as an SD card. The third memory 280 stores control programs such as the OS (operating system) 281 and APP 283. APP 283 is an application that communicates with the distribution server 100, displays images contained in the content distributed by the distribution server 100 on the display 240, and outputs voice contained in the content to the speaker 260.

[0160] The third processor 290 is, for example, a CPU, MPU, or other computing device. The third processor 290 can consist of a single processor or multiple processors.

[0161] When the third processor 290 reads and executes APP 283, the third control unit 270 communicates with the distribution server 100 via the wireless communication interface 210.

[0162] In addition, the third control unit 270 reads and executes APP 283 through the third processor 290, causing the display 240 to display images contained in the content distributed by the distribution server 100.

[0163] In addition, the third control unit 270 reads and executes APP 283 through the third processor 290, causing the speaker 260 to output the voice contained in the content distributed by the distribution server 100.

[0164] In addition, the third control unit 270 reads and executes APP 283 through the third processor 290, detects the gaze direction of the viewer U who is watching the content, and uploads the gaze direction data representing the detected gaze direction to the distribution server 100.

[0165] [5. Action]

[0166] Next, the operation of each part of the distribution system 1 according to this embodiment will be explained.

[0167] First, the actions of each part of the distribution system 1 involved in the content distribution will be explained.

[0168] Figure 10 This is a flowchart illustrating the operation of the vehicle-mounted device 30, the distribution server 100, and the viewer terminal 200B. Figure 10 In the flowchart, FA shows the operation of the vehicle-mounted device 30, FB shows the operation of the distribution server 100, and FC shows the operation of the viewer terminal 200B.

[0169] As shown in flowchart FA, the distribution unit 356 determines whether to start the distribution of content (step SA1).

[0170] For example, if the touch panel 34 receives a distribution start instruction, the distribution unit 356 makes an affirmative determination in step SA1.

[0171] Furthermore, for example, when the vehicle-mounted device 30 receives distribution start information from the distributor terminal 200A, the distribution unit 356 makes a positive determination in step SA1. Additionally, the distribution start information indicates that distribution has begun.

[0172] If the distribution unit 356 determines that the distribution of content has begun (step SA1: Yes), it begins the distribution of content (step SA2). When the distribution of content begins, the distribution unit 356 uploads the content data to the distribution server 100 at a predetermined period (step SA3).

[0173] Step SA3 is an example of a "distribution step".

[0174] As shown in flowchart FB, the second control unit 130 of the distribution server 100 receives content data from the vehicle-mounted device 30 (step SB1).

[0175] Next, the second control unit 130 of the distribution server 100 sends the received content data to the viewer terminal 200B (step SB2).

[0176] Step SB2 is described in detail. The second control unit 130 determines from the distribution management database 145 a record containing a distribution ID received along with the content data, and obtains the user ID of the viewer U from the determined record. Next, the second control unit 130 obtains device information for each obtained user ID of the viewer U. Then, the second control unit 130 transmits the received content data based on communication information included in the obtained device information.

[0177] As shown in flowchart FC, the third control unit 270 of the viewer terminal 200B receives content data from the distribution server 100 (step SC1).

[0178] Next, the third control unit 270 displays the image contained in the content represented by the received content data on the display 240, and outputs the voice contained in the content represented by the received content data through the speaker 260 (step SC2).

[0179] Step SC2 is an example of "Show Steps".

[0180] Returning to the description of flowchart FA, the distribution unit 356 determines whether to end the content distribution (step SA4).

[0181] For example, if the touch panel 34 receives a dispensing end instruction, the dispensing unit 356 makes a positive determination in step SA4.

[0182] Furthermore, for example, if the vehicle-mounted device 30 receives distribution completion information from the distributor terminal 200A, the distribution unit 356 makes a positive determination in step SA4. Additionally, the distribution completion information indicates that the distribution has ended.

[0183] If the distribution unit 356 determines that the content distribution should not be terminated (step SA4: No), it returns the process to step SA3 and performs the processing after step SA3 again. Alternatively, the distribution unit 356 continues distributing content until a positive result is determined in step SA4.

[0184] On the other hand, if the distribution unit 356 determines that the distribution of content has ended (step SA4: Yes), it ends the distribution of content (step SA5).

[0185] As described above, the additional unit 355 adds sound to the image distributed to the viewer terminal 200B, that is, the image captured by the distributor terminal 200A.

[0186] Next, the operation of the additional part 355 will be explained.

[0187] Figure 11 This is a flowchart illustrating the operation of the additional part 355. Figure 11 The action is repeatedly performed during content distribution.

[0188] The auxiliary unit 355 determines whether to add external ambient sound (step SD1).

[0189] Step SD1 is described in detail.

[0190] The add-on unit 355 determines the location corresponding to the current location of the vehicle 3 based on the location information of the vehicle 3 detected by the GNSS unit of the sensor group 39 and the map data 346. Next, the add-on unit 355 determines whether the determined location is recorded in the location data 347. When the determined location is recorded in the location data 347, the add-on unit 355 determines in step SD1 that external ambient sound is added.

[0191] Additionally, if the raindrop sensor of the sensor group 39 detects raindrops, the auxiliary unit 355 determines in step SD1 that it is an additional external ambient sound.

[0192] On the other hand, if the location is not recorded in the location data 347 and the raindrop sensor of the sensor group 39 does not detect raindrops, the auxiliary unit 355 determines in step SD1 that no external ambient sound will be added.

[0193] Next, the auxiliary unit 355 determines whether to add internal ambient sound (step SD2).

[0194] Step SD2 will be described in detail.

[0195] If the second acquisition unit 352 acquires the internal ambient sound, the additional unit 355 determines in step SD2 that the internal ambient sound is added; if the second acquisition unit 352 does not acquire the internal ambient sound, the additional unit 355 determines in step SD2 that the internal ambient sound is not added.

[0196] Next, the auxiliary unit 355 determines whether to add a vehicle-related sound (step SD3).

[0197] Step SD3 will be described in detail.

[0198] If the additional vehicle-related sound is not triggered, the additional unit 355 determines in step SD2 that no vehicle-related sound is added; if the trigger is triggered, the additional unit 355 determines in step SD2 that a vehicle-related sound is added.

[0199] In addition, the triggering of additional vehicle-related sounds is, for example, when the acceleration of vehicle 3 increases, the speed of vehicle 3 increases, or when the information that the direction indicator has been activated is received.

[0200] Next, the attachment unit 355 determines, based on the determination results of steps SD1 to SD3, whether to attach sound to the distributed image (step SD4).

[0201] If the attachment unit 355 determines that no sound should be added to the distributed image (step SD4: No), it will not add external ambient sound, internal ambient sound, or vehicle-related sound to the distributed image.

[0202] If the attachment unit 355 determines that it is necessary to attach sound to the distributed image (step SD4: Yes), it attaches sound to the distributed image (step SD5).

[0203] Step SD5 is an example of an "additional step".

[0204] The following is a detailed description of step SD5.

[0205] There are a total of 7 modes of sound added in step SD5.

[0206] <First Mode>

[0207] The first mode is a mode where the added sound is only external ambient sound.

[0208] In the first mode, the appender 355 appends external ambient sound to the distributed image.

[0209] In the first mode, when the determining unit 354 has determined the gaze area, the additional unit 355 sets the volume of the external ambient sound corresponding to the gaze area among the multiple external ambient sounds to be greater than the volume of the external ambient sound corresponding to the area outside the gaze area.

[0210] In addition, in the first mode, if the second acquisition unit 352 fails to acquire the internal ambient sound for a specified period of time or more, the additional external ambient sound volume is set to be greater than the volume of the external ambient sound when the internal ambient sound is acquired.

[0211] <Second Mode>

[0212] The second mode is a mode where the added sound is only the internal ambient sound.

[0213] In the second mode, the add-on unit 355 adds internal ambient sound to the image obtained by the transmitter terminal 200A.

[0214] <Third Mode>

[0215] The third mode is a mode where the added sound is only the vehicle-related sound.

[0216] In the third mode, the add-on unit 355 adds vehicle-related audio to the image obtained by the transmitter terminal 200A.

[0217] <Fourth Mode>

[0218] The fourth mode is a mode in which the added sounds are external ambient sounds and internal ambient sounds.

[0219] In the fourth mode, the auxiliary unit 355 sets the volume of the internal ambient sound to be louder than the volume of the external ambient sound, and adds both the internal and external ambient sounds.

[0220] Furthermore, in the fourth mode, when the determining unit 354 has determined the gaze area, the additional unit 355 sets the volume of the external ambient sound corresponding to the gaze area among the additional multiple external ambient sounds to be greater than the external ambient sound corresponding to the area outside the gaze area.

[0221] Furthermore, in the fourth mode, when the conversation content acquired by the third acquisition unit 353 is content related to external ambient sound such as "weather" or "scenery", the volume of the added external ambient sound is set to be louder than the volume of the added external ambient sound when the conversation content is not related to the external environment.

[0222] <Fifth Mode>

[0223] The fifth mode is a mode with additional sounds including external ambient sounds and vehicle-related sounds.

[0224] In the fifth mode, when the determining unit 354 has determined the gaze area, the additional unit 355 sets the volume of the external ambient sound corresponding to the gaze area among the multiple external ambient sounds to be greater than the volume of the external ambient sound corresponding to the area outside the gaze area.

[0225] In addition, in the fifth mode, the auxiliary unit 355 can set the volume of the external ambient sound to be greater than the volume of the vehicle-related sound and add both the external ambient sound and the vehicle ambient sound, or it can set the volume of the vehicle-related sound to be greater than the volume of the external ambient sound and add both the external ambient sound and the vehicle-related sound.

[0226] <Sixth Mode>

[0227] The sixth mode is a mode with additional sounds including interior ambient sounds and vehicle-related sounds.

[0228] In the sixth mode, the auxiliary unit 355 sets the volume of the interior ambient sound to be higher than the volume of the vehicle-related sound, and adds the interior ambient sound and the vehicle-related sound.

[0229] In the sixth mode, when the conversation content acquired by the third acquisition unit 353 is content associated with vehicle 3, the attachment unit 355 sets the volume of the vehicle-associated sound to be attached to be greater than the volume of the vehicle-associated sound attached when the conversation content is not associated with vehicle 3.

[0230] <Seventh Mode>

[0231] The seventh mode is a mode with additional sounds including external ambient sound, internal ambient sound, and vehicle-related sounds.

[0232] In the seventh mode, the auxiliary unit 355 sets the volume of the internal ambient sound to be higher than the volume of the vehicle-related sound and the external ambient sound, and adds the external ambient sound, internal ambient sound and vehicle-related sound.

[0233] Furthermore, in the seventh mode, when the gaze area is determined by the determination unit 354, the additional unit 355 sets the volume of the external ambient sound corresponding to the gaze area among the additional multiple external ambient sounds to be greater than the volume of the external ambient sound corresponding to the area outside the gaze area.

[0234] Furthermore, in the seventh mode, when the conversation content acquired by the third acquisition unit 353 is content related to external ambient sound such as "weather" or "scenery", the volume of the added external ambient sound is set to be louder than the volume of the added external ambient sound when the conversation content is not related to the external environment.

[0235] In the seventh mode, if the conversation content obtained by the third acquisition unit 353 is related to the vehicle 3, the attachment unit 355 sets the volume of the vehicle-related sound to be attached to be higher than the volume of the vehicle-related sound to be attached when the conversation content is not related to the vehicle 3.

[0236] [6. Other Implementation Methods]

[0237] The above-described implementation method is merely one approach and can be arbitrarily modified and applied.

[0238] In other embodiments, the audience terminal 200B may also output ambient sound in a manner that the sound is emitted from a location corresponding to the source of the ambient sound. In this case, for example, the audience terminal 200B has a stereo speaker. For example, if the vehicle 3 is traveling and the view of the sea is on the right side of the direction of travel, the audience terminal 200B outputs wave sound as ambient sound so that the viewer U can hear the wave sound from the right side. Additionally, for example, if the vehicle 3 is traveling through a forest and the viewer U can hear bird sounds from the left side of the direction of travel, the audience terminal 200B outputs bird sounds as ambient sound so that the viewer U can hear the bird sounds from the left side. Furthermore, for example, if the vehicle 3 is traveling through a forest, the audience terminal 200B outputs the sound of wind passing through the trees as ambient sound so that the viewer U can hear the sound of wind passing through the trees from all directions.

[0239] In other embodiments, the "object" can also be an object other than a raindrop, such as a forest, sea, lake, or factory. In this other embodiment, the sound associated with the "object," or "object-related sound," can be set as follows: if the "object" is the sea, it can be the sound of the sea; if the "object" is a forest, it can be the sound of wind blowing through the forest, birdsong, or sounds related to water flow; if the "object" is a factory, it can be the sound of factory equipment; and if the "object" is a lake, it can be the sound of birdsong or fish jumping. Furthermore, in this other embodiment, the attachment unit 355 determines whether an object exists in the image captured by the "shooting device," and if the object is present in the image, it adds an object-related sound. The attachment unit 355 can determine whether an object exists in the image captured by the "shooting device" by means of color difference, pattern matching, etc., or it can determine whether an object exists in the image captured by the "shooting device" based on the position information of the vehicle 3.

[0240] In the above-described embodiment, the vehicle-mounted device 30 is a structure that adds internal ambient sound, external ambient sound, and vehicle-related sound to the distributed image. In other embodiments, the distribution server 100 may also add internal ambient sound, external ambient sound, and vehicle-related sound. In this structure, the distribution server 100 receives from the vehicle-mounted device 30 the current position of the vehicle 3, vehicle information related to the vehicle 3 (vehicle 3 acceleration, vehicle 3 speed, whether the direction indicator is activated), rain sensor detection results, etc. Furthermore, in this structure, the distribution server 100 obtains the internal ambient sound and external ambient sound of the vehicle 3 from the vehicle-mounted device 30. Moreover, in this structure, the second processor 150 functions as a determination unit 354, an addition unit 355, and a distribution unit 356, and the distribution server 100 adds sound to the image distributed from the vehicle-mounted device 30 and distributes the image with added sound to the viewer terminal 200B.

[0241] Furthermore, in the above-described embodiment, the vehicle-mounted device 30 is a structure that adds internal ambient sound, external ambient sound, and vehicle-related sound to the distributed image. In other embodiments, the viewer terminal 200B may also be configured to add internal ambient sound, external ambient sound, and vehicle-related sound. In this configuration, the viewer terminal 200B receives the current position of the vehicle 3, vehicle information related to the vehicle 3 (vehicle 3 acceleration, vehicle 3 speed, whether the direction indicator is activated), and rain sensor detection results from the vehicle-mounted device 30 via the distribution server 100. In this configuration, the second processor 150 functions as a determination unit 354, an addition unit 355, and a distribution unit 356, and the viewer terminal 200B adds sound to the output image and outputs content. Additionally, in this configuration, the third control unit 270 adds the internal ambient sound, external ambient sound, and vehicle-related sound stored in the third memory 280.

[0242] In the above embodiment, the distribution unit 356 of the vehicle-mounted device 30 uploads content data to the distribution server 100 via the external communication interface 31. In other embodiments, the distributor terminal 200A may also upload content data to the distribution server 100. More specifically, in this case, the distributor terminal 200A uploads content data to the distribution server 100, including images captured by a camera equipped with the distributor terminal 200A or voice input through a microphone equipped with the distributor terminal 200A.

[0243] in addition, Figure 4 as well as Figure 9The structure of each part of the distribution system 1 shown is an example, and the specific installation method is not particularly limited. That is, it is not necessarily necessary to install hardware corresponding to each part separately; of course, it can also be configured so that the functions of each part are implemented by a processor executing a program. In addition, in the above embodiment, a part of the function implemented by software can be made into hardware, or a part of the function implemented by hardware can be implemented by software.

[0244] in addition, Figure 10 as well as Figure 11 The steps shown are divided according to the main processing content, and the present invention is not limited to the method or name of the division of processing units. Depending on the processing content, it may be divided into more step units. Alternatively, a single step unit may contain more processes. Furthermore, the order of these steps may be appropriately replaced without affecting the spirit of the invention.

[0245] [7. Structure supported by the above embodiments]

[0246] The above implementation supports the following structures.

[0247] (Structure 1)

[0248] A distribution system comprising: a distribution unit that distributes an image captured by a camera installed on a vehicle; a terminal device installed at a location outside the vehicle that displays the distributed image; and an attachment unit that attaches sound to the image, the attachment unit attaching external ambient sound of the vehicle based on the vehicle's location information, the terminal device displaying the image and outputting the external ambient sound attached to the image.

[0249] According to the distribution system of Structure 1, viewers of the images can listen to and watch external ambient sounds appropriate to the vehicle's location along with the images. Therefore, viewers of the images can further perceive the feeling of being in a vehicle.

[0250] (Structure 2)

[0251] A distribution system comprising: a distribution unit that distributes an image captured by a camera installed on a vehicle; a terminal device installed outside the vehicle that displays the distributed image; and an attachment unit that attaches sound to the image, wherein, if an object associated with the sound attached to the image exists outside the vehicle, the attachment unit attaches a sound associated with the object, namely an object-related sound, as an external ambient sound of the vehicle; the terminal device displays the image and outputs the object-related sound attached to the image.

[0252] According to the distribution system of Structure 2, when an object exists outside the vehicle, the sound associated with that object can be presented together with the image for audiovisual viewing. Therefore, viewers of the image can further perceive the feeling of being in a vehicle.

[0253] (Structure 3)

[0254] According to the distribution system of structure 2, the object is a raindrop, and the associated sound of the object is the sound of rain.

[0255] According to the distribution system of Structure 3, when it is raining outside the vehicle, viewers of the images can experience the sound of rain along with the images. Therefore, viewers of the images can further perceive the feeling of being in a vehicle.

[0256] (Structure 4)

[0257] According to the distribution system of any one of structures 1 to 3, the additional part further adds engine sound to the image based on vehicle information related to the vehicle being driven, and the terminal device further outputs the engine sound added to the image.

[0258] According to the distribution system of Structure 4, viewers of the images can experience the engine sound along with the images. Therefore, viewers of the images can further perceive the feeling of being in a vehicle.

[0259] (Structure 5)

[0260] According to any one of structures 1 to 4, the distribution system includes a first acquisition unit that acquires the external ambient sound by means of an external microphone disposed on the vehicle, and an additional unit that acquires the external ambient sound acquired by the first acquisition unit.

[0261] According to the distribution system of Structure 5, viewers of the images can experience the actual external ambient sound along with the images. Therefore, viewers of the images can further perceive the feeling of being in a vehicle.

[0262] (Structure 6)

[0263] According to any one of structures 1 to 5, the distribution system includes a second acquisition unit that acquires the interior ambient sound of the vehicle by means of an in-vehicle microphone disposed in the vehicle, the additional unit that sets the volume to be louder than the external ambient sound to add the interior ambient sound acquired by the second acquisition unit, and the terminal device that also outputs the interior ambient sound added to the image.

[0264] According to the distribution system of Structure 6, viewers of the images can hear the interior ambient sound more loudly than the external ambient sound. Typically, the interior ambient sound is louder than the external ambient sound inside a vehicle. Therefore, according to the distribution system of Structure 6, viewers of the images can further perceive the feeling of being in a vehicle.

[0265] (Structure 7)

[0266] According to the distribution system of structure 6, if the additional unit fails to acquire the internal ambient sound for a specified period of time or more, it does not attach the internal ambient sound to the image, and sets the volume of the attached external ambient sound to be greater than the volume of the external ambient sound when the internal ambient sound is attached to the image.

[0267] According to the distribution system of Structure 7, without the addition of internal ambient sound, viewers of the images can hear external ambient sounds quite loudly. Generally, inside a vehicle, when there is no internal ambient sound or the internal ambient sound is quiet, the external ambient sound is heard loudly. Therefore, viewers of the images can further perceive the feeling of being in a vehicle.

[0268] (Structure 8)

[0269] According to any one of structures 1 to 7, the distribution system includes a determining unit that determines the viewing area that the viewer is looking at in the image based on the viewing direction of the viewer using the terminal device, and an attaching unit that sets the volume to be louder than the ambient sound corresponding to the area outside the viewing area determined by the determining unit, thereby attaching the ambient sound corresponding to the viewing area determined by the determining unit.

[0270] According to the distribution system of Structure 8, the viewer of the image is able to hear the ambient sound of the gazed area more loudly than the ambient sound of the non-gazed area.

[0271] (Structure 9)

[0272] The distribution system as described in any one of structures 1 to 8, wherein the distribution system includes a third acquisition unit that acquires conversation content between a viewer viewing the image using the terminal device and an occupant of the vehicle, and the supplementary unit sets the volume of supplementary sound based on the conversation content acquired by the third acquisition unit.

[0273] According to the distribution system of Structure 9, the volume of the sound heard by the viewer of the image can be set according to the content of the conversation. Therefore, it is possible to suppress the conversation between the viewer of the image and the occupants of the vehicle that is hindered by the sound added by the adder. In addition, by setting the volume of the adder, it is possible to suppress the situation where the viewer cannot hear sounds other than those in the conversation, and to suppress the situation where the viewer cannot get the feeling of being in a vehicle due to the conversation.

[0274] (Structure 10)

[0275] A computer program product that enables a processor to function as an add-on and a distribution unit, the add-on adding sound to an image obtained by a camera device installed on a vehicle, the distribution unit distributing the image with sound added by the add-on to a terminal device, the add-on adding external ambient sound of the vehicle based on the vehicle's location information.

[0276] According to the computer program product of structure 10, the terminal device is able to output images and ambient sounds, so that viewers of the images can listen to and watch ambient sounds appropriate to the vehicle's location along with the images. Therefore, viewers of the images can further experience the feeling of being in a vehicle.

[0277] (Structure 11)

[0278] A computer program product that enables a processor to function as an add-on unit and a distribution unit, wherein the add-on unit adds sound to an image obtained by a camera device installed on a vehicle, and the distribution unit distributes the image with the sound added by the add-on unit to a terminal device, wherein if there is an object outside the vehicle that is associated with the sound added to the image, the add-on unit adds a sound associated with the object, namely an object-related sound, as an external ambient sound of the vehicle.

[0279] According to the computer program product of structure 11, when there are objects outside the vehicle, the terminal device can output images and related audio. Therefore, in this case, the viewer of the image can listen to and watch the related audio along with the image. Thus, the viewer of the image can further experience the feeling of being in a vehicle.

[0280] (Structure 12)

[0281] A distribution method includes: a distribution step of distributing an image captured by a camera installed on a vehicle; a display step of displaying the distributed image on a terminal device located outside the vehicle; and an additional step of adding sound to the image, wherein, based on the location information of the vehicle, an external ambient sound of the vehicle is added to the image, and, in the display step, the image is displayed and the external ambient sound added to the image is output.

[0282] The distribution method of Structure 12 achieves the same effect as the distribution system of Structure 1.

[0283] (Structure 13)

[0284] A distribution method includes: a distribution step of distributing an image captured by a camera installed on a vehicle; a display step of displaying the distributed image on a terminal device located outside the vehicle; and an additional step of adding sound to the image, wherein, in the additional step, if an object associated with the sound added to the image exists outside the vehicle, a sound associated with the object, namely an object-related sound, is added as an external ambient sound of the vehicle; and in the display step, the image is displayed and the object-related sound added to the image is output.

[0285] The distribution method of Structure 13 achieves the same effect as the distribution system of Structure 1.

[0286] Explanation of reference numerals in the attached figures

[0287] 1…distribution system, 3…vehicle, 5…communication network, 10…seat, 11…seat backrest, 12…headrest, 13…headrest support, 30…in-vehicle device, 31…external communication interface, 32…first short-range wireless communication interface, 33…in-vehicle communication interface, 34…touch panel, 35…voice processing unit, 36…speaker, 37…in-vehicle microphone, 38…external microphone, 39…sensor group, 40…vehicle ECU, 50…mounting accessories, 51…first fastener, 100…distribution server, 110…network communication interface, 130…second control unit, 14… 0…Second memory, 141…Control program, 143…User information database, 145…Distribution management database, 150…Second processor, 200A…Distributor terminal (shooting device), 200B…Viewer terminal (terminal device), 210…Wireless communication interface, 220…Second short-range wireless communication interface, 230…Camera, 240…Display, 250…Microphone, 260…Speaker, 270…Third control unit, 280…Third memory, 281…OS, 283…APP, 290…Third processor, 310…Input / output interface, 330…First control unit, 340…First memory, 345…Control program (program), 346…Map data, 347…Location data, 350…First processor (processor), 351…First acquisition unit, 352…Second acquisition unit, 353…Third acquisition unit, 354…Determination unit, 355…Additional unit, 356…Distribution unit, CA…Second area, IP…Viewpoint, LA…First area, P1…Image, RA…Third area, S…Distributor, U…Viewer.

Claims

1. A distribution system, the distribution system comprising: The distribution unit distributes images obtained by a camera device installed on the vehicle; A terminal device, located outside the vehicle, displays the distributed image; and An additional component that adds sound to the image. The additional unit adds external ambient sound of the vehicle based on the vehicle's location information. The terminal device displays the image and outputs the external ambient sound attached to the image.

2. A distribution system, the distribution system comprising: The distribution unit distributes images obtained by a camera device installed on the vehicle; A terminal device, located outside the vehicle, displays the distributed image; and An additional component that adds sound to the image. If an object associated with the sound attached to the image exists outside the vehicle, the attachment part attaches the sound associated with the object, i.e., the object-associated sound, as the external ambient sound of the vehicle. The terminal device displays the image and outputs the object-related audio attached to the image.

3. The distribution system according to claim 2, wherein, The object being described is a raindrop. The sound associated with the object is the sound of rain.

4. The distribution system according to any one of claims 1 to 3, wherein, The additional component also adds engine sound to the image based on vehicle information related to the vehicle being driven. The terminal device also outputs the engine sound attached to the image.

5. The distribution system according to any one of claims 1 to 3, wherein, The distribution system includes a first acquisition unit that acquires the external ambient sound via an external microphone located in the vehicle. The additional part adds the external ambient sound acquired by the first acquisition part.

6. The distribution system according to any one of claims 1 to 3, wherein, The distribution system includes a second acquisition unit that acquires the ambient sound inside the vehicle via a microphone installed in the vehicle. The additional unit sets the volume to be louder than the external ambient sound to add the internal ambient sound acquired by the second acquisition unit. The terminal device also outputs the internal ambient sound attached to the image.

7. The distribution system according to claim 6, wherein, If the second acquisition unit fails to acquire the internal ambient sound for a specified period of time or more, the additional unit does not attach the internal ambient sound to the image, and sets the volume of the attached external ambient sound to be greater than the volume of the external ambient sound when the internal ambient sound is attached to the image.

8. The distribution system according to any one of claims 1 to 3, wherein, The distribution system includes a determining unit that determines the area of ​​focus that the viewer is looking at in the image based on the direction of the viewer's gaze when viewing the image using the terminal device. The supplementary unit sets the volume to be louder than the external ambient sound corresponding to the area outside the gaze area determined by the determining unit, in order to supplement the external ambient sound corresponding to the gaze area determined by the determining unit.

9. The distribution system according to any one of claims 1 to 3, wherein, The distribution system includes a third acquisition unit that acquires the conversation content between a viewer viewing the image using the terminal device and the occupants of the vehicle. The additional unit sets the volume of the additional sound based on the conversation content obtained by the third acquisition unit.

10. A computer program product that enables a processor to function as an add-on and distribution unit. The additional unit adds sound to the images obtained by the camera device installed on the vehicle. The distribution unit distributes the image, to which the additional unit has added sound, to the terminal device. The additional unit adds external ambient sound to the vehicle based on the vehicle's location information.

11. A computer program product that enables a processor to function as an add-on and distribution unit. The additional unit adds sound to the images obtained by the camera device installed on the vehicle. The distribution unit distributes the image, to which the additional unit has added sound, to the terminal device. If there is an object outside the vehicle that is associated with the sound attached to the image, the attached part adds the sound associated with the object, i.e., the object-associated sound, as the external ambient sound of the vehicle.

12. A distribution method, the distribution method comprising: The distribution step involves distributing images captured by a camera device installed on the vehicle. The display step involves a terminal device located outside the vehicle displaying the distributed image. as well as An additional step is to add sound to the image. In the additional step, based on the vehicle's location information, the image is appended with the vehicle's external ambient sound. In the display step, the image is displayed, and the external ambient sound attached to the image is output.

13. A distribution method, the distribution method comprising: The distribution step involves distributing images captured by a camera device installed on the vehicle. The display step involves a terminal device located outside the vehicle displaying the distributed image. as well as An additional step is to add sound to the image. In the additional step, if there is an object outside the vehicle that is associated with the sound attached to the image, the sound associated with that object, i.e., the object-associated sound, is attached as the external ambient sound of the vehicle. In the display step, the image is displayed, and the object-related sound attached to the image is output.

Images