Stereoscopic audio and video reproduction system for vehicle

Abstract

The invention relates to a stereoscopic audio and video reproduction system for a vehicle. When the stereoscopic audio / video reproduction system is used in a vehicle, excessive behavior of a driver caused by a positioned audio / video is suppressed. A stereoscopic audio / video reproduction system for a vehicle is provided with: an audio reproduction device having an audio / video generation unit; and a vehicle sensor capable of detecting the face or the line of sight of a vehicle driver or the direction of travel of the vehicle. The sound reproduction device sets a positioning permission angle range using the direction of the driver or the vehicle as a reference direction before starting to generate the audio-video of the sound source by the audio-video generation unit, and determines whether or not the direction of the original positioning of the audio-video is within the positioning permission angle range. When within the positioning permissible angle range, the sound reproduction device begins to generate an audio / video at the original positioning. When the positioning permissible angle range is not within, the sound reproduction device does not start generating the audio / video at the original positioning.

Description

Technical Field

[0001] The invention relates to a stereoscopic audio and video reproduction system for a vehicle. Background Art

[0002] Patent Document 1 discloses generating an audio and video image localized at an obstacle outside a vehicle.

[0003] Patent Document 2 discloses generating an audio and video that is positioned at a predicted attention point of a vehicle ahead or the like.

[0004] Patent Document 3 discloses generating an audio and video positioned at a predicted gaze point for guiding the driver's line of sight.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Publication No. 2009-301123

[0008] Patent Document 2: Japanese Patent Application Publication No. 2014-127935

[0009] Patent Document 3: Japanese Patent Application Publication No. 2014-127936 Summary of the invention

[0010] Problems to be solved by the invention

[0011] As described above, it has been proposed to generate an audio image with localization of a sound source in a vehicle using a plurality of speakers installed in the vehicle.

[0012] However, once the localized audio and video is generated, the attention of the vehicle driver will be directed to the localized audio and video.

[0013] In addition, Patent Documents 1 to 3 specifically disclose only the case where the sound image is localized on the front side of the vehicle or on the front side of the driver, but the sound image generated in the vehicle may be localized on the rear side of the vehicle or on the rear side of the driver.

[0014] In this case, the driver may turn his sight to the rear side opposite to the direction of travel of the vehicle. In particular, the driver may continue to turn his face or sight to the rear side in order to clearly identify the audio and video positioned at the rear side.

[0015] Thus, when a stereoscopic audio and video reproduction system is used in a vehicle, it is desired to be able to suppress excessive behavior of the driver caused by localized audio and video.

[0016] Technical solutions for solving technical problems

[0017] One embodiment of the present invention relates to a stereoscopic audio and video reproduction system for a vehicle, comprising: a plurality of speakers arranged on the vehicle; a sound reproduction device having an audio and video generation unit, the audio and video generation unit being used to generate audio and video that can be positioned at least in the front-rear direction of the vehicle using the plurality of speakers; and a vehicle sensor that can detect at least one of the direction of the face or the direction of sight of the driver of the vehicle and the direction of travel of the vehicle, wherein the sound reproduction device performs the following: before the audio and video generation unit starts generating the audio and video at the original positioned position of the sound source, the direction of the face of the driver of the vehicle, the direction of sight, and the direction of travel of the vehicle detected by the vehicle sensor before the audio and video generation starts The direction of travel, or the direction of the face or the direction of the line of sight and the direction of travel of the vehicle are set as a reference direction, and the angle range extending from the reference direction to the left and right directions of the vehicle is set as a positioning permission angle range, and it is determined whether the original positioning position of the audio and video to be generated is within the positioning permission angle range. When the original positioning position of the audio and video to be generated is within the positioning permission angle range, the audio and video generation unit starts to generate the audio and video at the original positioning position. When the original positioning position of the audio and video to be generated is not within the positioning permission angle range, the audio and video generation unit does not start to generate the audio and video at the original positioning position.

[0018] Effects of the Invention

[0019] In the present invention, the sound reproduction device has a sound image generation unit for generating sound images that can be located at least in the front-rear direction of the vehicle using a plurality of speakers. Furthermore, before the sound reproduction device starts generating sound images at the original location of the sound source, the direction of the face of the vehicle driver, the direction of the sight line, the direction of the vehicle's travel, or the direction between the direction of the face or the direction of the sight line and the direction of the vehicle's travel as detected by the vehicle sensor is set as a reference direction, and the angle range extending from the reference direction to the left and right directions of the vehicle is set as a positioning permission angle range. The sound reproduction device determines whether the original location direction of the sound image to be generated is within the positioning permission angle range.

[0020] Then, when the direction of the original positioning of the sound and image to be generated is within the positioning allowable angle range, the sound reproducing device starts generating the sound and image at the original positioning by the sound and image generating unit.

[0021] On the other hand, when the direction of the original positioning of the sound and image to be generated is not within the positioning allowable angle range, the sound reproduction device does not start generating the sound and image at the original positioning by the sound and image generating unit.

[0022] Thus, the audio and video in the present invention no longer positions the direction of the driver's face, the direction of the line of sight, or the direction of travel of the vehicle outside the positioning allowable angle range.

[0023] As a result, even if the driver of the vehicle turns his face or sight toward the positioned audio and video, the driver's face or sight can be maintained within the positioning permitted angle range. Moreover, the positioning permitted angle range is an angle range that extends from the reference direction to the left and right directions of the vehicle based on the direction of the face, the direction of sight, or the direction of travel of the vehicle before the generation of the audio and video begins. As a result, even if the driver continues to turn his face or sight toward the positioned audio and video, the driver's face or sight can be maintained in a state of not leaving the front side of the vehicle, and it is easy to maintain a state of keeping the direction of travel of the vehicle within the field of view.

[0024] In this way, in the present invention, it is possible to suppress excessive behavior of the driver caused by the localized audio and video. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is an explanatory diagram of a state of generation of a stereoscopic audio and video in a car to which the stereoscopic audio and video reproduction system according to the first embodiment of the present invention can be applied.

[0026] Figure 2 is set at Figure 1 Structural diagram of a stereoscopic audio and video reproduction system in a car.

[0027] Figure 3 Is Figure 2 Flowchart of stereo audio and video reproduction control executed by the control unit of the sound reproduction device in FIG.

[0028] Figure 4 When the car is going straight, Figure 3 An explanatory diagram of a state in which a stereoscopic audio and video reproduction control generates a stereoscopic audio and video.

[0029] Figure 5 When the car turns to the right, Figure 3 An explanatory diagram of a state in which a stereoscopic audio and video reproduction control generates a stereoscopic audio and video.

[0030] Figure 6 This is a flowchart of stereoscopic audio and video reproduction control executed by the control unit of the stereoscopic audio and video reproduction system according to the second embodiment of the present invention.

[0031] Figure 7 This is a flowchart of stereoscopic audio and video reproduction control executed by the control unit of the stereoscopic audio and video reproduction system according to the third embodiment of the present invention. DETAILED DESCRIPTION

[0032] Embodiments of the present invention will be described below based on the drawings.

[0033] [First embodiment]

[0034] Figure 1 It is an explanatory diagram of a state of generation of a stereoscopic audio and video in a car 1 to which the stereoscopic audio and video reproduction system 10 according to the first embodiment of the present invention can be applied.

[0035] The automobile 1 is an example of a vehicle. Vehicles in which passengers such as a driver ride include buses, trucks, trains, personal mobility vehicles, and the like.

[0036] Figure 1 The automobile 1 has a vehicle body 2. A plurality of seats including a driver's seat 4 are provided in a vehicle cabin 3 where passengers such as a driver of the vehicle body 2 sit. The driver who manages the operation of the automobile 1 sits in the driver's seat 4. Passengers other than the driver sit in seats other than the driver's seat 4.

[0037] In addition, a plurality of wheels arranged front and back are provided on the left and right sides of the vehicle body 2. Specifically, a right front wheel 5, a left front wheel 6, a right rear wheel 7, and a left rear wheel 8 are provided. During the driving process of the driver manually driving or the automatic driving, the automobile 1 drives the driving wheels among the plurality of wheels by rotating the driving wheels by a power source not shown in the figure. In addition, during the driving process of the driver manually driving or the automatic driving, the automobile 1 can change the traveling direction from, for example, the straight line direction shown by the dotted arrow in the figure to the turning direction to the right side shown by the dotted arrow, etc. by steering the steering wheels among the plurality of wheels by using a power source not shown in the figure.

[0038] Furthermore, the driver who manages the operation of the automobile 1 is required to turn his face or eyes toward the direction of travel of the automobile 1, even if the automobile 1 is driving automatically, and to check objects outside the automobile, etc., to confirm the safety of the operation. For example, when the automobile 1 is traveling straight as shown by the dashed arrow 40 in the figure, the driver is required to turn his face or eyes toward the straight direction as shown by the dashed arrow 41 in the figure. In addition, when the automobile 1 turns right as shown by the dashed arrow 50 in the figure, the driver is required to turn his face or eyes toward the far right side when viewed from the automobile 1 as shown by the dashed arrow 51 in the figure.

[0039] In addition, it is proposed that in the automobile 1, a plurality of speakers 11 to 14 provided in the vehicle interior 3 of the automobile 1 are used to reproduce the sound source as a stereoscopic sound image with its original localization. The figure shows an example of a right front speaker 11, a left front speaker 12, a right rear speaker 13, and a left rear speaker 14. In addition, in the automobile 1, if at least two speakers are provided, a stereoscopic sound image with the original localization of the sound source can be generated. Here, the right front speaker 11 and the left front speaker 12 are speakers located in front of the driver's seat 4 of the automobile 1.

[0040] In addition, the sound reproduced by stereo audio and video is sometimes also aided by a stereo audio and video reproduction system, but as shown in the figure from the first positioning position P1 to the eighth positioning position P8, it can be positioned in the direction of 360 degrees around the car 1. In addition, the figure shows the orthogonal three-axis direction based on XYZ. The direction of the X-axis is the left-right direction or the width direction of the car 1. The direction of the Y-axis is the front-back direction of the car 1. The direction of the Z-axis is the up-down direction of the car 1.

[0041] In this way, the sound produced by the stereo audio and video is not only positioned at the first positioning position P1, the second positioning position P2, and the eighth positioning position P8 on the front side of the car 1, but also positioned at the fourth positioning position P4, the fifth positioning position P5, and the sixth positioning position P6 on the rear side of the car 1, as well as the third positioning position P3, the seventh positioning position P7, etc. on the left and right sides of the car 1.

[0042] In addition, Figure 1 In the figure, for the convenience of illustration, each positioning position from the first positioning position P1 to the eighth positioning position P8 is shown in a circular shape in a manner of surrounding the body 2 of the automobile 1 from the outside. However, each positioning position actually generated by the stereo audio and video reproduction system is basically mostly the inner space of the plurality of speakers 11 to 14 arranged in the cabin 3 of the automobile 1. In this case, it can be understood that Figure 1 The multiple positioning positions in are drawn by mapping the inner spaces of the multiple speakers 11-14.

[0043] In this case, the driver or other passengers riding in the car 1 are likely to direct their attention in the direction of the position where the stereoscopic sound and image are localized.

[0044] As a result, the passengers, especially the driver, may turn their sight to the rear side opposite to the traveling direction of the automobile 1. In order to clearly recognize the sound and image at the sixth positioning position P6 positioned behind the passenger, such as the driver, as shown by the solid arrow 70 in the figure, the passenger may continue to turn their face or sight to the sound and image.

[0045] In this way, when the stereoscopic audio and video reproduction system 10 is used in the automobile 1 , it is desirable to suppress excessive behavior of the driver caused by audio and video localized in the vehicle interior 3 of the automobile 1 .

[0046] Figure 2 is set at Figure 1 1 is a block diagram of a stereoscopic audio and video reproduction system 10 in a car 1.

[0047] Figure 2 The stereo audio and video reproduction system 10 includes a right front speaker 11, a left front speaker 12, a right rear speaker 13, a left rear speaker 14 and a sound reproduction device 15.

[0048] The sound reproduction device 15 includes a memory 21, a receiver 22, an audio / video generator 23, a monaural generator 24, an alarm generator 25, an amplifier 26, and a control unit 27. The control unit 27 is connected to a timer 28 and a plurality of vehicle sensors provided in the automobile 1. Figure 2 As the plurality of vehicle sensors, a steering angle sensor 16 , an exterior camera 17 , a DMS (Driver Monitoring System) camera 18 , and a GNSS (Global Navigation Satellite System) receiver 19 provided in the automobile 1 are exemplified.

[0049] The memory 21 of the sound reproduction device 15 records reproduced sound data of various sound sources, for example, conversion function data for generating a stereoscopic sound image from reproduced sound data of a monaural sound source, and the like.

[0050] The reproduced sound data of the sound source includes, for example, monaural data, stereo data including the original localization of the sound source itself, reproduced sound data converted to stereo by a method other than stereo, such as a multi-channel method, etc. The reproduced sound data in a monaural method or the like without stereo may include information related to the position of the original localization for localizing the stereoscopic sound image based on the sound source.

[0051] In addition, the types of reproduced sound data of the sound source may be, for example, sound data for reproducing content, sound data for guiding a route, and the like.

[0052] The receiver 22 receives reproduced sound data such as notification sound data from road signs such as beacons installed on the road on which the automobile 1 travels, which is a sound source.

[0053] Such notification sound data can be used to notify, for example, precise locations such as intersections and warning areas on the road.

[0054] In addition, the receiving unit can receive wide-area road guide audio data, content playback audio data, conversation data with others, etc. as playback audio data of the sound source through the base station. The playback audio data from which the sound source can be received depends on the corresponding function and service of the receiver 22.

[0055] In addition, the reproduced sound data of the sound sources received by the receiver 22 may include information on the original positioning positions of the respective sound sources.

[0056] The sound image generation unit 23 generates a reproduced sound signal for reproducing the reproduced sound data of the sound source obtained from the memory 21 or the receiver 22 through the original localized stereoscopic sound image related to the sound source of the reproduced sound data. The sound image generation unit 23 outputs the generated reproduced sound signal to the right front speaker 11, the left front speaker 12, the right rear speaker 13, and the left rear speaker 14 through the amplifier 26. Figure 1 As shown, a stereoscopic sound image with original positioning of the sound source of the reproduced sound data is generated in the vehicle cabin 3 of the automobile 1 .

[0057] Here, when obtaining reproduced sound data of a sound source that does not contain original positioning in the sound source itself, such as a monaural sound source, the audio-visual generation unit 23 can convert the sound source using the conversion function data obtained from the memory 21, and generate a reproduced sound signal output to each of the multiple speakers 11 to 14.

[0058] On the other hand, when obtaining reproduced sound data of a sound source including original localization in the sound source itself, for example, a stereo sound source, the sound and image generating unit 23 can generate a reproduced sound signal including localization in the sound source itself.

[0059] In this way, the sound and image generation unit 23 can generate sound and images that can be localized at least in the front-rear direction of the automobile 1 using the plurality of speakers 11 to 14 .

[0060] Furthermore, the audio and video generation unit 23 can select conversion function data for generating audio and video according to the localization position included in the reproduced sound data, etc., thereby generating a stereo audio and video localized at a desired position based on the sound source of the reproduced sound data. The audio and video generation unit 23 can generate stereo audio and video localized at different positions from each other based on the sound source of the same reproduced sound data by changing the conversion function data for generating audio and video. The audio and video generation unit 23 can generate, for example, a stereo audio and video localized at a first localization position P1, or a stereo audio and video localized at a fifth localization position P5 based on the sound source of the same reproduced sound data. Therefore, for example, a plurality of conversion function data that are different for each localization position can be recorded in the memory 21 to at least correspond to the data from Figure 1 It corresponds to each positioning position from the first positioning position P1 to the eighth positioning position P8.

[0061] In addition, the sound image generation unit 23 can obtain conversion function data for changing the sound image localized at the first positioning position P1 to the sound image localized at the eighth positioning position P8, based on the conversion function data for localizing the reproduced sound data of the sound source, such as a monaural sound source, which does not include the localization of the stereoscopic sound image itself, at the first positioning position P1 and the conversion function data for localizing the sound source at the eighth positioning position P8. By using such conversion function data for changing the localization position, the sound image generation unit 23 can obtain the stereoscopic sound image localized at the position changed from the original position, based on the reproduced sound data of the sound source itself including the stereophonic sound of the original localization.

[0062] In addition, for example, the audio and video generation unit 23 can convert the stereoscopic reproduction sound data of the sound source itself including the original positioning into the reproduction sound data of the sound source in a monaural mode, and then, by using the conversion function data corresponding to the desired positioning position, the reproduction sound data of the sound source in a monaural mode obtained by the conversion is converted to obtain a stereo audio and video positioned at the position changed from the original position.

[0063] In this way, regardless of whether the sound source included in the reproduced sound data is localized, the sound image generation unit 23 can change the conversion function data used to generate the sound image based on the information included in the reproduced sound data, thereby generating a stereoscopic sound image localized at an arbitrary position. The sound image generation unit 23 can use the plurality of speakers 11 to 14 to generate a sound image that can be localized at least in the front-rear direction of the automobile 1. For example, the memory 21 can store, for each localized position, a data item for at least Figure 1 A plurality of conversion function data for positioning the positions from the first positioning position P1 to the eighth positioning position P8.

[0064] In addition, the conversion function data used by the audio and video generation unit 23 to generate a stereo audio and video from the reproduced sound data, for example, in the case of reproduced sound data of a sound source in a monaural manner, can basically be conversion function data obtained by converting the sound transfer method in the case where the sound source and the artificial head imitating the driver's head are arranged in a virtual space into a transfer function. Figure 1In the corresponding virtual space, the sound source is arranged at the fifth positioning position P5, and the simulation head is arranged at the position of the driver's seat 4. According to the waveform of the sound source and the listening sound waveform of the right ear of the simulation head in this case, the transfer function for the right ear can be obtained. In addition, similarly, according to the waveform of the sound source and the listening sound waveform of the left ear of the simulation head, the transfer function for the left ear can be obtained. These transfer functions are functions that change the waveform of the sound source to the left and right listening sound waveforms. The transfer function for the right ear and the transfer function for the left ear can be used as conversion function data for converting the reproduced sound data in a monophonic manner into a stereo audio and video positioned at the fifth positioning position P5. In the conversion function data, in practical applications, the conversion function data after further adjusting the above-mentioned transfer function according to the position of the car speaker, etc. can also be used.

[0065] Then, the audio and video generation unit 23 generates the audio and video data from the corresponding Figure 1 The conversion function data corresponding to the indicated positioning position is selected from the plurality of conversion function data corresponding to the plurality of positioning positions. In addition, the sound image generation unit 23 may select the conversion function data preset in the memory 21 according to the type of the reproduced sound data and the like.

[0066] For example, for the reproduced sound data guiding the right turn path, the audio and video generation unit 23 may select the conversion function data corresponding to the second positioning position P2 or the conversion function data corresponding to the third positioning position P3 to locate on the right side of the automobile 1 .

[0067] In addition, the reproduced sound data may include information indicating the position at which the sound is reproduced as located. For example, the notification sound data from a road sign may include position information based on the GNSS coordinates of the road sign for locating the notification sound of the road sign. In this case, the audio and video generation unit 23 selects the conversion function data based on the relative direction and distance between the current position of the automobile 1 and the position of the road sign so as to locate the position of the automobile 1 relative to the road sign.

[0068] In addition, the audio and video generation unit 23 may or may not use the conversion function data acquired from the memory 21 for the reproduced sound data converted into stereo sound in advance. The reproduced sound data converted into stereo sound in advance includes, for example, reproduced sound data in a stereo system, reproduced sound data in a 22.2 multi-channel system, reproduced sound data in a binaural system, etc. The audio and video generation unit 23 may switch whether to use the conversion function data when the reproduced sound data acquired from the memory 21 or the receiver 22 is reproduced sound data converted into stereo sound or when it is not.

[0069] However, in the case of changing the original positioning of the reproduced sound data in advance in stereophonic form, the sound and image generation unit 23 can select the conversion function data and change the positioning of the reproduced sound data in advance in stereophonic form using the selected conversion function data. At this time, the sound and image generation unit 23 can temporarily convert the sound source of the reproduced sound data in advance in stereophonic form into a monaural form, for example, and then use the conversion function data for positioning at a desired position in the memory 21 to convert the monaural sound source. In this way, the positioning position of the sound and image in the vehicle interior 3 of the automobile 1 can be changed from the original positioning position of the reproduced sound data in advance in stereophonic form. The sound and image generation unit 23 can convert the reproduced sound data in advance in stereophonic form into a monaural form using the monaural generation unit 24.

[0070] The monaural generation unit 24 converts the reproduced sound data obtained from the memory 21 or the receiver 22 into a monaural sound, and generates a reproduced sound signal. The reproduced sound data before the conversion may be reproduced sound data in a monaural mode, or reproduced sound data that has been pre-stereoized, such as in a stereo mode. The monaural generation unit 24 may, for example, overlap all waveforms included in the reproduced sound data that has been pre-stereoized, and generate reproduced sound data of a sound source in a monaural mode. The monaural generation unit 24 outputs the generated reproduced sound signal to the right front speaker 11, the left front speaker 12, the right rear speaker 13, and the left rear speaker 14 through the amplifier 26. In addition, the monaural generation unit 24 may output a meaningful reproduced sound signal to a part of the plurality of speakers 11 to 14, such as the right front speaker 11 and the left front speaker 12, and output a silent reproduced sound signal to the remaining right rear speaker 13 and the left rear speaker 14.

[0071] The alarm generating unit 25 generates an alarm reproduction sound signal based on the alarm reproduction sound data obtained from the memory 21. The alarm reproduction sound data may be a sound source in a monaural manner. The alarm generating unit 25 outputs the generated reproduction sound signal to the right front speaker 11, the left front speaker 12, the right rear speaker 13, and the left rear speaker 14 through the amplifier 26. In addition, the alarm generating unit 25 may output an intentional alarm reproduction sound signal to a part of the plurality of speakers 11 to 14. For example, the alarm generating unit 25 may output an intentional alarm reproduction sound signal to the right front speaker 11 and the left front speaker 12, and output a silent reproduction sound signal to the remaining right rear speaker 13 and the left rear speaker 14.

[0072] The audio and video generation unit 23, the monaural generation unit 24, and the alarm generation unit 25, as well as the right front speaker 11, the left front speaker 12, the right rear speaker 13, and the left rear speaker 14 are connected to the amplifier 26. The amplifier 26 amplifies the reproduced sound signal generated by the audio and video generation unit 23, the monaural generation unit 24, or the alarm generation unit 25, and outputs it to the right front speaker 11, the left front speaker 12, the right rear speaker 13, and the left rear speaker 14.

[0073] The timer 28 measures the elapsed time and the time.

[0074] The control unit 27 controls the operation of the sound reproduction device 15. The control unit 27 may be constituted by, for example, a CPU (Central Processing Unit). In this case, the CPU functions as the control unit 27 of the sound reproduction device 15 by reading and executing a program stored in the memory 21. For example, the control unit 27 selects one from the sound image generation unit 23, the monaural generation unit 24, and the alarm generation unit 25 to perform an operation. Thus, the sound reproduction device 15 can generate a stereoscopic sound image of the original positioning of the sound source of the reproduced sound data generated by the sound image generation unit 23 in the cabin 3 of the automobile 1, or generate a reproduced sound generated by the monaural generation unit 24 or the alarm generation unit 25 in the cabin 3 of the automobile 1. In addition, the control unit 27 can perform switching designation of the conversion function data for the sound reproduction device 15 to change the positioning of the stereoscopic sound image generated by the sound reproduction device 15.

[0075] Furthermore, the CPU can realize the audio and video generation unit 23 , the monaural generation unit 24 , or the alarm generation unit 25 in the sound reproduction device 15 in addition to the control unit 27 by executing the program.

[0076] The steering angle sensor 16 detects the steering angle of the steering wheels of the automobile 1. The steering wheels are generally the right front wheel 5 and the left front wheel 6. When the steering wheels are turned from the straight-ahead direction, the automobile 1 turns right or left according to the steering direction and the steering amount. The steering angle sensor 16 detects the steering direction and the steering amount as information indicating the traveling direction of the automobile 1.

[0077] The vehicle exterior camera 17 captures the exterior of the vehicle 1. The vehicle exterior camera 17 may be a stereo camera, a monocular camera, a 360-degree camera, or the like. In addition, a plurality of stereo cameras and a plurality of monocular cameras may be provided in the vehicle 1. Thus, the vehicle exterior camera 17 captures the surroundings of the vehicle 1, particularly the front side of the vehicle 1. In the photographic image of the vehicle exterior camera 17, a road extending in the traveling direction of the vehicle 1 may be captured. The vehicle exterior camera 17 captures the exterior of the vehicle as information indicating the traveling direction of the vehicle 1.

[0078] The DMS camera 18 captures a photo of the driver riding in the car 1. The DMS camera 18 may also capture photos of passengers other than the driver riding in the car 1. The face and eyes of the driver may be captured in the image captured by the DMS camera 18. The DMS camera 18 captures the face or eyes of the driver as information indicating the face or sight direction of the driver of the car 1.

[0079] In addition, the vehicle sensor capable of detecting information indicating the direction of the face or sight of the driver of the automobile 1 includes, in addition to the DMS camera 18 , a rear camera of a drive recorder installed in the cabin 3 of the automobile 1 .

[0080] In addition, the vehicle sensor capable of detecting information indicating the traveling direction of the automobile 1 includes, in addition to the steering angle sensor 16 and the exterior camera 17 , an acceleration sensor in the three-axis directions of XYZ and a gyro sensor.

[0081] The vehicle GNSS receiver 19 receives radio waves from GNSS satellites and generates the current position and current time of the vehicle 1 .

[0082] Figure 3 Is Figure 2 Flowchart of stereoscopic audio and video reproduction control executed by the control unit 27 of the sound reproduction device in FIG.

[0083] The control unit 27 of the sound reproduction device repeatedly executes Figure 3 Stereo audio and video reproduction control.

[0084] In step ST1 , the control unit 27 determines whether to start generating a stereoscopic audio and video.

[0085] For example, when a request is made to reproduce audio data for guiding a route as the car 1 moves forward, the control unit 27 may determine to start generating a stereoscopic audio and video.

[0086] Furthermore, for example, when the passenger instructs the reproduction of reproduced sound data of stereoscopic content, the control unit 27 may determine to start generating a stereoscopic audio and video.

[0087] Furthermore, for example, when the receiver 22 receives notification sound data, the control unit 27 may determine to start generating a stereoscopic audio and video.

[0088] Furthermore, for example, when the receiver 22 receives call data with another person, the control unit 27 may determine to start generating a stereoscopic audio and video.

[0089] Then, when it is determined that the generation of stereoscopic sound and image is to start (step ST1: Yes), the control unit 27 performs the processing of step ST2 in order to make a determination based on the direction of the driver's face or line of sight before starting to generate the stereoscopic sound and image of the original localization of the sound source of the reproduced sound data.

[0090] On the other hand, when it is determined that the generation of the stereoscopic audio and video is not to be started (step ST1 : NO), the control unit 27 ends the present control.

[0091] In step ST2 , the control unit 27 obtains detection information from the vehicle sensor.

[0092] The control unit 27 acquires detection information of vehicle sensors from, for example, the steering angle sensor 16 , the exterior camera 17 , the DMS camera 18 , and the like.

[0093] In step ST3 , the control unit 27 sets a reference direction as a reference for determining the localization position of the sound and image.

[0094] For example, the control unit 27 may set the direction of the face or the direction of the line of sight of the driver of the car 1 detected by the DMS camera 18 before the start of audio and video generation as the reference direction.

[0095] Furthermore, for example, the control unit 27 may set the traveling direction of the automobile 1 detected by the steering angle sensor 16 before the start of the generation of the audio and video as the reference direction.

[0096] In addition, for example, the control unit 27 can set the extending direction of the road on which the car 1 travels before the start of generating the audio and video, which can be detected by the vehicle exterior camera 17, as the reference direction. Here, the extending direction of the road is the traveling direction of the car 1.

[0097] Furthermore, for example, when there is a difference of more than a threshold angle between the direction of the driver's face or line of sight and the traveling direction of the automobile 1 , the direction therebetween, for example, the center direction therebetween may be set as the reference direction.

[0098] In addition, for example, when there is a difference of more than a threshold angle between the direction of the driver's face or line of sight and the extending direction of the road on which the car 1 is traveling, the direction therebetween, for example, the center direction thereof, may be set as the reference direction.

[0099] In step ST4 , the control unit 27 sets a positioning allowable angle range for determining the localization position of the sound and image.

[0100] The control unit 27 may set a predetermined angle range extending from the reference direction to the left and right directions of the automobile 1, for example, an angle range extending to the left and right sides of the reference direction, for example, 80 degrees on one side and 160 degrees as a whole, as the positioning permission angle range.

[0101] The positioning permitted angle range can be set, for example, so that even if the driver faces or looks toward the position of the stereoscopic image reproduced by positioning, the direction of travel of the automobile 1 is included in the driver's field of vision. The positioning permitted angle range can be set within an angle range that does not exceed 180 degrees at most, so that even if the driver faces or looks toward the positioning position of the stereoscopic image, it will not face backward.

[0102] In addition, the positioning permission angle range can be increased or decreased according to the driving conditions of the automobile 1, such as the speed and steering amount of the automobile 1. For example, when the speed is above a threshold, the control unit 27 can reduce the positioning permission angle range compared to when the speed is below the threshold.

[0103] In step ST5 , the control unit 27 sets the origin 45 for determining the localization position of the sound and image.

[0104] For example, Figure 1 As shown, the control unit 27 may set the center position of the front end of the driver's seat 4 of the automobile 1 as the origin 45 of the positioning allowable angle range.

[0105] Furthermore, for example, the center position of the front end of the automobile 1 may be set as the origin 45 of the positioning allowable angle range.

[0106] In step ST6 , the control unit 27 determines whether the original positioning position of the audio and video to be generated is within the positioning allowable angle range.

[0107] The position of the original localization of the audio image to be generated may be defined in the reproduced sound data itself or in the information of the reproduced sound data.

[0108] The control unit 27 determines whether the original positioning position of the audio and video to be generated is as follows: Figure 4 As shown, the position of the front end center of the driver's seat 4 is included in the range of the positioning allowable angle range starting from the origin 45 with the origin 45 being the origin.

[0109] Then, when the position of the original localization of the reproduced sound data is included in the localization allowable angle range, the control unit 27 proceeds to the process of step ST7 to generate a sound image at the original localization of the sound source of the reproduced sound data.

[0110] On the other hand, when the original localization position of the reproduced sound data is not within the localization allowable angle range, the control unit 27 proceeds to the process of step ST8 and does not start generating an audio image at the original localization of the sound source of the reproduced sound data.

[0111] In step ST7 , the control unit 27 starts a process of reproducing and outputting the audio and video to be generated at the original location of the reproduced audio data.

[0112] The control unit 27 reproduces the audio and video to be generated through the audio and video generation unit 23. The audio and video generation unit 23 generates a reproduction signal for generating the audio and video at the original positioning of the sound source of the reproduced sound data based on the audio and video to be generated. The audio and video generation unit 23 outputs the generated reproduction signal to the right front speaker 11, the left front speaker 12, the right rear speaker 13, and the left rear speaker 14 through the amplifier 26. Thus, the audio and video generation unit 23 starts to generate the audio and video at the original positioning of the sound source of the reproduced sound data in the vehicle cabin 3. In addition, a stereo audio and video with the original positioning of the reproduced sound data is generated in the vehicle cabin 3 of the automobile 1.

[0113] Then, the control unit 27 ends the present control.

[0114] In step ST8 , the control unit 27 starts a process of changing the localization position of the sound image to be generated from the original position, and reproducing and outputting the reproduced sound data.

[0115] The control unit 27 changes the localization position of the audio and video to be generated to be within the positioning allowable angle range through the audio and video generation unit 23, and reproduces the reproduced sound data. The audio and video generation unit 23 generates a reproduced signal using, for example, the reproduced sound data and the conversion function data for making the localization position within the positioning allowable angle range, and outputs the reproduced signal to the right front speaker 11, the left front speaker 12, the right rear speaker 13, and the left rear speaker 14 through the amplifier 26. Thus, the audio and video generation unit 23 starts to locate and generate audio and video within the positioning allowable angle range. In addition, in the vehicle cabin 3 of the automobile 1, a stereo audio and video that is localized within the positioning allowable angle range is generated.

[0116] Then, the control unit 27 ends the present control.

[0117] In this way, when the position of the original positioning of the sound image to be generated is not within the positioning allowable angle range, the control unit 27 does not generate the sound image at the original positioning of the sound source of the reproduced sound data by the sound image generating unit 23.

[0118] Furthermore, when the sound and image generating unit 23 does not generate the sound and image at the original localization of the sound source of the reproduced sound data, the control unit 27 changes the localization position of the sound and image to be within the localization allowable angle range, and generates the sound and image at the changed localization by the sound and image generating unit 23. Thus, even if the sound source of the reproduced sound data is not generated as the sound and image at the original localization of the sound source, the driver and other passengers of the car 1 can listen to the sound source as the sound and image source at the changed localization.

[0119] In particular, when the localization position of the sound image is changed to be within the localization allowable angle range, and the sound image generation unit 23 starts generating the sound image at the changed localization, the localized sound image can be generated. Even when the sound image is not generated at the original localization of the sound source of the reproduced sound data, the driver and other passengers of the car 1 can expect that the localized sound image will not easily feel a sense of disharmony with the reproduced sound.

[0120] Furthermore, the localization position of the sound and image is set within the localization permissible angle range based on the direction of the driver's face or line of sight before reproduction or the reference direction of the traveling direction of the automobile 1. Even if the driver notices the localization of the sound and image during the driving of the automobile 1, it is difficult for the driver to greatly move his face or line of sight away from the traveling direction of the automobile 1. The driver can easily maintain a state where his face is facing the front side of the automobile 1. The driver and others are less likely to look around due to the sound and image originally localized at the sound source of the reproduced sound data.

[0121] Next, a specific example of a state in which a stereoscopic audio and video is generated by the above-mentioned stereoscopic audio and video reproduction control will be described.

[0122] Figure 4 When car 1 is going straight, it passes Figure 3 An explanatory diagram of a state in which a stereoscopic audio and video reproduction control generates a stereoscopic audio and video.

[0123] exist Figure 4 In the figure, the center position of the front end of the driver's seat 4 is set as the origin 45. In addition, the positioning allowable angle range 46 shown by the dotted line in the figure is represented by the dotted line extending from the origin 45.

[0124] in this case, Figure 1 The first positioning position P1, the second positioning position P2, and the eighth positioning position P8 shown in FIG. 4 are located within the localization allowable angle range 46. When the original localization position of the sound source of the reproduced sound data is these localization positions, in step ST6, the control unit 27 determines that the original localization position of the reproduced sound data is included in the localization allowable angle range 46. In step ST7, the control unit 27 generates a stereoscopic sound image at the original localization position of the reproduced sound data based on the reproduced sound data.

[0125] On the other hand, when the original positioning position of the reproduced sound data is the third positioning position P3, the fourth positioning position P4, the fifth positioning position P5, the sixth positioning position P6, or the seventh positioning position P7, in step ST6, the control unit 27 determines that the original positioning position of the reproduced sound data is not included in the positioning allowable angle range 46. In step ST8, the control unit 27 changes the positioning position of the reproduced sound data so that it is within the positioning allowable angle range 46, and generates a stereoscopic audio image based on the reproduced sound data.

[0126] For example, when the original positioning position of the reproduced sound data is the third positioning position P3, the control unit 27 changes the positioning position of the reproduced sound data to the third changed positioning position PM3, and generates a stereoscopic sound image based on the reproduced sound data. Here, the third changed positioning position PM3 is a position where the third positioning position P3 is moved forward so as to be within the positioning allowable angle range 46.

[0127] Similarly, when the original positioning position of the reproduced sound data is the fourth positioning position P4, the control unit 27 changes the positioning position of the reproduced sound data to the fourth changed positioning position PM4, and generates a stereoscopic sound image based on the reproduced sound data. Here, the fourth changed positioning position PM4 is a position where the fourth positioning position P4 is moved forward so as to be within the positioning allowable angle range 46.

[0128] Similarly, when the original positioning position of the reproduced sound data is the fifth positioning position P5, the control unit 27 changes the positioning position of the reproduced sound data to the fifth changed positioning position PM5, and generates a stereoscopic sound image based on the reproduced sound data. Here, the fifth changed positioning position PM5 is a position where the fifth positioning position P5 is moved forward so as to be within the positioning allowable angle range 46.

[0129] Similarly, when the original positioning position of the reproduced sound data is the sixth positioning position P6, the control unit 27 changes the positioning position of the reproduced sound data to the sixth changed positioning position PM6, and generates a stereoscopic sound image based on the reproduced sound data. Here, the sixth changed positioning position PM6 is a position where the sixth positioning position P6 is moved forward so as to be within the positioning allowable angle range 46.

[0130] Similarly, when the original positioning position of the reproduced sound data is the seventh positioning position P7, the control unit 27 changes the positioning position of the reproduced sound data to the seventh changed positioning position PM7, and generates a stereoscopic sound image based on the reproduced sound data. Here, the seventh changed positioning position PM7 is a position where the seventh positioning position P7 is moved forward so as to be within the positioning allowable angle range 46.

[0131] As a result, the stereoscopic sound image actually generated in the vehicle interior 3 of the automobile 1 based on the reproduced sound data that has started to be reproduced is included in the localization allowable angle range 46 .

[0132] Figure 5 When car 1 turns to the right, Figure 3 An explanatory diagram of a state in which a stereoscopic audio and video reproduction control generates a stereoscopic audio and video.

[0133] exist Figure 5 In FIG. 4 , the center position of the front end of the driver's seat 4 is set as the origin 45 . The positioning allowable angle range 52 is indicated by a dotted line extending from the origin 45 .

[0134] in this case, Figure 1 The first positioning position P1, the second positioning position P2, the third positioning position P3 and the eighth positioning position P8 shown in FIG. 5 are located within the localization allowable angle range 52. When the original localization position of the sound source of the reproduced sound data is these positioning positions, in step ST6, the control unit 27 determines that the original localization position of the reproduced sound data is included in the localization allowable angle range 52. In step ST7, the control unit 27 generates a stereoscopic sound image at the original localization position of the reproduced sound data based on the reproduced sound data.

[0135] On the other hand, when the original positioning position of the reproduced sound data is the fourth positioning position P4, the fifth positioning position P5, the sixth positioning position P6, or the seventh positioning position P7, in step ST6, the control unit 27 determines that the original positioning position of the reproduced sound data is not included in the positioning allowable angle range 52. In step ST8, the control unit 27 changes the positioning position of the reproduced sound data so that it is within the positioning allowable angle range 52, and generates a stereoscopic audio image based on the reproduced sound data.

[0136] For example, when the original positioning position of the reproduced sound data is the fourth positioning position P4, the control unit 27 changes the positioning position of the reproduced sound data to the fourth changed positioning position PM4, and generates a stereoscopic sound image based on the reproduced sound data. Here, the fourth changed positioning position PM4 is a position where the fourth positioning position P4 is moved forward so as to be within the positioning allowable angle range 52.

[0137] Similarly, when the original positioning position of the reproduced sound data is the fifth positioning position P5, the control unit 27 changes the positioning position of the reproduced sound data to the fifth modified positioning position PM5, and generates a stereoscopic sound image based on the reproduced sound data. Here, the fifth modified positioning position PM5 is a position where the fifth positioning position P5 is moved forward so as to be within the positioning allowable angle range 52.

[0138] Similarly, when the original positioning position of the reproduced sound data is the sixth positioning position P6, the control unit 27 changes the positioning position of the reproduced sound data to the sixth changed positioning position PM6, and generates a stereoscopic sound image based on the reproduced sound data. Here, the sixth changed positioning position PM6 is a position where the sixth positioning position P6 is moved forward so as to be within the positioning allowable angle range 52.

[0139] Similarly, when the original positioning position of the reproduced sound data is the seventh positioning position P7, the control unit 27 changes the positioning position of the reproduced sound data to the seventh changed positioning position PM7, and generates a stereoscopic sound image based on the reproduced sound data. Here, the seventh changed positioning position PM7 is a position where the seventh positioning position P7 is moved forward so as to be within the positioning allowable angle range 52.

[0140] As a result, the stereoscopic sound image actually generated in the vehicle interior 3 of the automobile 1 based on the reproduced sound data that has started to be reproduced is included in the localization allowable angle range 52 .

[0141] In addition, Figure 5 In FIG. 6 , the reference direction and other positioning allowable angle ranges 60 are indicated by double-dashed lines.

[0142] The other positioning allowable angle range 60 of the double-dotted line is toward the right rear of the positioning allowable angle range of the dotted line. Figure 5 When turning further to the right than in the case of Figure 5 The middle dashed line is further to the right.

[0143] When the positioning allowable angle range is tilted rearwardly in this manner, the control unit 27 of the sound reproduction device 15 may narrow the positioning allowable angle range in the determination of step ST6 .

[0144] When there is a portion in the positioning allowable angle range that is further rearward than the width direction of the automobile 1, such as Figure 5 As shown by the dotted arrow in , the control unit 27 can set the angle range located on the front side in the vehicle width direction of the automobile 1 except for the portion on the rear side as the positioning permission angle range.

[0145] As a result, the stereoscopic image is no longer positioned behind the driver's car 1 in the vehicle width direction. Even if the driver turns his face or sight toward the positioned position of the stereoscopic image, he will not look back to the rear of the car 1 in the vehicle width direction.

[0146] As described above, in the present embodiment, the sound reproduction device 15 includes the sound image generation unit 23 for generating sound images that can be localized at least in the front-rear direction of the automobile 1 using the plurality of speakers 11 to 14. Before the sound image generation unit 23 starts generating the sound images at the original localization of the sound source of the reproduced sound data, the sound reproduction device 15 sets the direction of the face of the driver of the automobile 1, the direction of the sight line, the traveling direction of the automobile 1, or the direction between the direction of the face or the direction of the sight line and the traveling direction of the automobile 1 detected by the vehicle sensor as a reference direction, and sets a prescribed angle range extending from the reference direction to the left-right direction of the automobile 1 as a positioning permitted angle range.

[0147] Furthermore, the sound reproduction device 15 uses the position of the driver's seat 4 of the automobile 1 as the origin 45 of the positioning allowable angle range. Thus, the positioning allowable angle range can be set and determined based on the driver in charge of the operation of the automobile 1.

[0148] The sound reproduction device 15 sets the positioning allowable angle range to the angle range located in the front side in the vehicle width direction of the automobile 1. This makes it difficult for the driver of the automobile 1 to turn his face or eyes to the rear side in the vehicle width direction of the automobile 1 due to the localized sound and image.

[0149] On this basis, the sound reproduction device 15 determines whether the localization position of the sound and image to be generated is within the localization allowable angle range.

[0150] Then, when the position of the original localization of the sound image to be generated is within the localization allowable angle range, the sound reproduction device 15 starts generating the sound image at the original localization of the sound source of the reproduced sound data through the sound image generation unit 23 .

[0151] On the other hand, when the original position of the sound image to be generated is not within the positioning allowable angle range, the sound reproduction device 15 does not generate the sound image at the original position of the sound source of the reproduced sound data by the sound image generation unit 23 .

[0152] Thus, in this embodiment, the audio and video generated in the cabin 3 of the automobile 1 is no longer positioned outside the positioning allowable angle range relative to the driver's face, line of sight, or the traveling direction of the automobile 1 .

[0153] As a result, even if the driver of the car 1 turns his face or sight toward the positioned audio and video, the driver's face or sight can be maintained within the positioning permitted angle range. Moreover, the positioning permitted angle range is an angle range that expands from the reference direction to the left and right directions of the car 1 based on the direction of the face, the direction of sight, or the direction of travel of the car 1 before the audio and video starts to be generated. As a result, even if the driver sometimes continues to turn his face or sight toward the positioned audio and video, the driver's face or sight can be maintained in a state where it does not deviate from the front side of the car 1, and it is easy to maintain a state where the direction of travel of the car 1 is contained within the field of view.

[0154] In this way, in the present embodiment, it is possible to suppress excessive behavior of the driver caused by localized sound and image.

[0155] [Second Embodiment]

[0156] Next, a vehicle 1 according to a second embodiment of the present invention will be described.

[0157] In this embodiment, the stereoscopic audio and video reproduction control performed by the control unit 27 of the sound reproduction device 15 is different from that in the above-mentioned embodiment.

[0158] Specifically, the processing in step ST6 when it is determined that the localization position of the reproduced sound data is not included in the localization allowable angle range is different from that in the above-described embodiment.

[0159] The following mainly describes the differences from the above-mentioned embodiment.

[0160] Figure 6 2 is a flowchart of the stereoscopic audio and video reproduction control executed by the control unit 27 of the stereoscopic audio and video reproduction system 10 according to the second embodiment of the present invention.

[0161] The control unit 27 of the sound reproduction device repeatedly executes Figure 6 Stereo audio and video reproduction controls shown.

[0162] When the control unit 27 of the sound reproduction device determines in step ST6 that the original localization position of the reproduced sound data is not included in the localization allowable angle range, the process proceeds to step ST10 .

[0163] In step ST10 , in order to notify the occupant that the generation of the stereoscopic audio and video has stopped, the control unit 27 outputs an alarm for notifying that the generation of the stereoscopic audio and video has stopped through the alarm generation unit 25 .

[0164] In this case, the alarm generating unit 25 can output the alarm sound from the right front speaker 11 and the left front speaker 12 located in front of the driver's seat 4 of the automobile 1 among the plurality of speakers 11 to 14 provided in the automobile 1. By outputting such an alarm sound, the driver's consciousness, face, and sight are easily attracted to the sound from the front and directed toward the front of the automobile 1.

[0165] In step ST11 , the control unit 27 uses the timer 28 to measure the elapsed time from the current time point.

[0166] In step ST12, the control unit 27 determines whether the elapsed time measured by the timer 28 has passed a predetermined time as a threshold. Here, the predetermined time may be, for example, several seconds.

[0167] Then, when the elapsed time has not exceeded the predetermined time, the control unit 27 repeats the present process.

[0168] When the elapsed time reaches a predetermined time, the control unit 27 ends the present control.

[0169] In this way, when the control unit 27 of the sound reproduction device does not start generating the sound image at the original location of the sound source of the reproduced sound data through the sound image generation unit 23, the alarm sound is output from the speaker located in front of the driver's seat 4 of the car 1 among the multiple speakers 11 to 14.

[0170] After that, the control unit 27 of the sound reproduction device waits for a predetermined time to elapse in step ST12, and then ends the operation. Figure 6 Stereo audio and video reproduction control in.

[0171] As a result, the sound reproduction device performs the next Figure 6The timing of the stereoscopic audio and video reproduction control in step ST11 is delayed by the prescribed time measured from step ST11. Figure 6 In the stereoscopic audio and video reproduction control, the timing of the sound reproduction device executing step ST7 is also delayed by the predetermined time measured from step ST11. The timing of the audio and video generation unit 23 generating the audio and video at the original localization of the sound source of the reproduced sound data in step ST7 is delayed.

[0172] Even if the sound reproduction device Figure 6 Although the stereoscopic sound and image is not generated at the original position of the sound source of the reproduced sound data in the initial stereoscopic sound and image reproduction control, the stereoscopic sound and image can be generated at the original position of the sound source of the reproduced sound data in the subsequent stereoscopic sound and image reproduction control. The driver and other passengers of the car 1 can listen to the stereoscopic sound and image at the original position of the sound source of the reproduced sound data at the delayed timing. The driver and other passengers of the car 1 can expect that the reproduced stereoscopic sound and image itself will not feel uncomfortable.

[0173] In addition, since the localization position of the stereoscopic sound and image is within the localization permissible angle range as in the above-mentioned embodiment, even if the driver or other passengers notice the localization of the sound and image and turn their face or sight toward the localization position, they will not deviate greatly from the traveling direction of the automobile 1. During the driving of the automobile 1, the driver or other passengers can easily keep their faces facing the front side of the automobile 1. The driver or other passengers are less likely to look around due to the stereoscopic sound and image originally localized at the sound source of the reproduced sound data.

[0174] In addition, in the present embodiment, when the control unit 27 determines in step ST6 that the original positioning position of the reproduced sound data is not included in the positioning allowable angle range, the process proceeds to step ST10. In step ST10, the control unit 27 outputs an alarm for notifying that the generation of the stereoscopic sound and image has stopped in order to notify the occupant that the generation of the stereoscopic sound and image has stopped, through the alarm generation unit 25.

[0175] Alternatively, for example, when the control unit 27 determines in step ST6 that the original positioning position of the reproduced sound data is not included in the positioning allowable angle range, the control unit 27 may skip the process of step ST10 and proceed to the process of step ST11 .

[0176] [Third Embodiment]

[0177] Next, a vehicle 1 according to a third embodiment of the present invention will be described.

[0178] In this embodiment, the stereoscopic audio and video reproduction control performed by the control unit 27 of the sound reproduction device 15 is different from that in the above-mentioned embodiment.

[0179] Specifically, the processing in step ST6 when it is determined that the original positioning position of the reproduced sound data is not included in the positioning allowable angle range is different from that in the above-described embodiment.

[0180] The following mainly describes the differences from the above-mentioned embodiment.

[0181] Figure 7 2 is a flowchart of the stereoscopic audio and video reproduction control executed by the control unit 27 of the stereoscopic audio and video reproduction system 10 according to the third embodiment of the present invention.

[0182] The control unit 27 of the sound reproduction device repeatedly executes Figure 7 Stereo audio and video reproduction controls shown in .

[0183] When the control unit 27 of the sound reproduction device determines in step ST6 that the localization position of the reproduced sound data is not included in the localization allowable angle range, the process proceeds to step ST20 .

[0184] In step ST20 , the control unit 27 causes the monaural generation unit 24 to generate a monaural reproduction sound signal in order to stop generating the stereoscopic audio and video.

[0185] At this time, the monaural generation unit 24 may change the audio and video into monaural sound, and generate a monaural reproduction sound signal.

[0186] In step ST21 , the control unit 27 outputs a monaural reproduction sound signal through the monaural generation unit 24 from the right front speaker 11 and the left front speaker 12 located in front of the driver's seat 4 of the automobile 1 among the plurality of speakers 11 to 14 provided in the automobile 1 .

[0187] By outputting the monaural sound only from the front side speaker, the driver's consciousness, face, and sight are easily directed toward the front side of the automobile 1 .

[0188] As described above, in this embodiment, the sound reproduction device 15 does not start to generate stereoscopic sound and image at the original positioning of the sound source of the reproduced sound data through the sound and image generation unit 23, but changes the sound and image to monaural sound and outputs it from the speaker located in front of the driver's seat 4 of the car 1 among the multiple speakers 11 to 14.

[0189] Thus, even if the sound source is not generated as an original localized sound image in the reproduced sound data, the driver and other passengers of the car 1 can listen to the sound source through monaural sound. The driver and other passengers of the car 1 can expect to listen to the sound source of the reproduced sound data and are less likely to feel excessive discomfort with the reproduced sound.

[0190] Furthermore, the sound and image are changed to monaural sound and outputted from the speaker in front of the driver's seat 4 of the car 1 among the plurality of speakers 11 to 14. In this case, the driver and other passengers not only lose the localization of the sound and image and are less likely to notice the sound and image, but also their attention is easily attracted by the sound coming from the front of the car 1 and they tend to turn toward the front of the car 1. While the car 1 is running, the driver and other passengers tend to keep their faces facing the front of the car 1. The driver and other passengers are less likely to look around due to the stereoscopic sound and image originally localized at the sound source of the reproduced sound data.

[0191] The above-described embodiments are examples of preferred embodiments of the present invention, but the present invention is not limited thereto, and various modifications and changes can be made.

[0192] In the above embodiment, in step ST5 , the control unit 27 of the sound reproduction device 15 sets the position of the front edge center of the driver's seat 4 of the automobile 1 as the origin 45 of the positioning allowable angle range.

[0193] In addition to this, for example, the origin 45 of the positioning allowable angle range may be located at the center of the front edge of the vehicle 1 , which is located on the front side of the center of the front edge of the driver's seat 4 .

[0194] In the above embodiments, Figure 5 As shown by the double dotted lines in FIG, when there is a portion on the rear side in the vehicle width direction of the vehicle 1 within the specified angle range extending from the reference direction to the left and right directions of the vehicle 1, the control unit 27 of the sound reproduction device 15 sets the angle range on the front side in the vehicle width direction of the vehicle 1 except for the rear side portion as the positioning allowed angle range.

[0195] In addition, for example, even when there is a portion further rearward than the vehicle width direction of the automobile 1 within the angle range specified from the reference direction, the control unit 27 of the sound reproduction device 15 may set the original angle range as the positioning permission angle range.

[0196] However, it is not desirable for the driver of the automobile 1 to turn his face or his sight toward the rear side of the automobile 1 while driving. Therefore, in order to reduce the possibility that the driver of the automobile 1 turns his face or his sight toward the rear side of the automobile 1 while driving, the control unit 27 of the sound reproduction device 15 preferably sets the angle range located at the front side in the vehicle width direction of the automobile 1 except for the portion located at the rear side in the vehicle width direction of the automobile 1 as the positioning permission angle range.

[0197] In the above-mentioned first embodiment, when the control unit 27 of the sound reproduction device 15 does not start to generate a stereoscopic audio and video at the original positioning of the sound source of the reproduced sound data, as a first processing, the positioning position of the audio and video is changed to be within the positioning allowable angle range, and the audio and video at the changed positioning is generated by the audio and video generation unit 23.

[0198] In the second embodiment, when the control unit 27 of the sound reproduction device 15 does not start generating a stereoscopic sound image at the original positioning of the sound source of the reproduced sound data, as a second processing, an alarm sound is output from a speaker located in front of the driver's seat 4 of the automobile 1 among the multiple speakers 11 to 14, and as a third processing, the start timing of generating a stereoscopic sound image at the original positioning of the sound source of the reproduced sound data is delayed by the sound and image generation unit 23.

[0199] In the third embodiment, when the control unit 27 of the sound reproduction device 15 does not start to generate a stereoscopic sound image at the original positioning of the sound source of the reproduced sound data, as a fourth processing, the sound image is changed to a monaural sound and output from a speaker located in front of the driver's seat 4 of the car 1 among the multiple speakers 11 to 14.

[0200] In addition, for example, when the control unit 27 of the sound reproduction device 15 does not start generating a stereoscopic sound image at the original localization of the sound source of the reproduced sound data, the first to fourth processes may be appropriately combined or switched for execution.

[0201] Explanation of symbols

[0202] 1…car (vehicle), 2…body, 3…cabin, 4…driver's seat, 5…right front wheel, 6…left front wheel, 7…right rear wheel, 8…left rear wheel, 10…stereo audio and video reproduction system, 11…right front speaker, 12…left front speaker, 13…right rear speaker, 14…left rear speaker, 15…sound reproduction device, 16…steering angle sensor, 17…exterior camera, 18…DMS camera, 19…GNSS receiver, 21…memory, 22…receiver, 23…audio and video generation unit, 24…monaural generation unit, 25…alarm generation unit, 26…amplifier, 27…control unit, 28…timer, 45…origin, 46, 52, 60…positioning permitted angle range.

Claims

1. A vehicle stereo audio and video reproduction system, comprising: a plurality of speakers disposed on the vehicle; a sound reproduction device having a sound and image generation unit for generating a sound and image localizable at least in the front-rear direction of the vehicle using the plurality of the speakers; as well as a vehicle sensor capable of detecting at least one of a direction of a face or a direction of sight of a driver of the vehicle and a direction of travel of the vehicle, The sound reproduction device performs the following: Before the audio and video generating unit starts generating the audio and video at the original positioned position of the sound source, the direction of the face of the driver of the vehicle, the direction of the sight line, the direction of travel of the vehicle, the direction between the direction of the face and the direction of travel of the vehicle, or the direction between the direction of the sight line and the direction of travel of the vehicle detected by the vehicle sensor before the audio and video starts generating is set as a reference direction, and an angle range extending from the reference direction to the left and right directions of the vehicle is set as a positioning permitted angle range, Determine whether the original positioning position of the audio and video to be generated is within the positioning allowable angle range, When the original positioning position of the audio and video to be generated is within the positioning allowable angle range, the audio and video generating unit starts generating the audio and video at the original positioning position, When the original positioning position of the audio and video to be generated is not within the positioning allowable angle range, the audio and video will not be generated at the original positioning position.

2. The vehicle stereoscopic audio and video reproduction system according to claim 1, wherein: The sound reproduction device uses the position of the driver's seat of the vehicle as the origin of the positioning allowable angle range, and determines whether the original positioning position of the audio and video to be generated is within the positioning allowable angle range.

3. The vehicle stereoscopic audio and video reproduction system according to claim 2, wherein: When there is a portion on the rear side relative to the vehicle width direction within the angle range extending from the reference direction to the left and right directions of the vehicle, the sound reproduction device sets the angle range on the front side relative to the vehicle width direction excluding the rear side portion as the positioning allowed angle range.

4. The vehicle stereoscopic audio and video reproduction system according to claim 3, wherein: When the audio and video generation unit does not start to generate the audio and video at the originally positioned position, The sound reproduction device delays the start timing of generating the sound and image at the original positioning position through the sound and image generating unit, then changes the original positioning position of the sound and image to within the positioning allowable angle range, and generates the sound and image at the changed positioning position through the sound and image generating unit.

5. The vehicle stereoscopic audio and video reproduction system according to claim 3, wherein: When the audio and video generation unit does not start to generate the audio and video at the originally positioned position, The sound reproducing device changes the sound and image into a monaural sound, and outputs the sound and image from a speaker located on a front side of a driver's seat of the vehicle among the plurality of speakers.

Images