Stereoscopic sound image reproduction system for vehicle

Abstract

To suppress a driver's excessive behavior due to an oriented sound image when a stereoscopic sound image reproduction system is used for a vehicle.SOLUTION: A stereoscopic sound image reproduction system for vehicles comprises: a sound reproduction device having a sound image generation unit; and a vehicle sensor capable of detecting the direction of a vehicle or a driver. The sound reproduction device sets an angle scope extending in the right and left directions of the vehicle from a reference direction based on the driver or the vehicle while or before the generation of a sound image at the original orientation relative to a sound source, as an orientation-permitted angle scope. If the direction of the driver during the generation of the sound image remains within the orientation-permitted angle scope, the sound reproduction device continues the generation of the sound image at the original orientation. If the direction of the driver does not remain in the orientation-permitted angle scope, the sound reproduction device stops the generation of the sound image at the original orientation.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a vehicle stereo sound image reproduction system. [Background technology]

[0002] Patent Document 1 discloses generating a sound image that is localized to an obstacle outside the vehicle. Patent Document 2 discloses generating a sound image that is localized at a predicted gaze point of a preceding vehicle or the like. Patent Document 3 discloses generating a sound image that is localized at a predicted gaze point to guide the driver's line of sight. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2009-301123 A [Patent Document 2] JP 2014-127935 A [Patent Document 3] JP 2014-127936 A Summary of the Invention [Problem to be solved by the invention]

[0004] In this manner, it has been proposed to generate a sound image having a localized position of a sound source using a plurality of speakers provided in the vehicle. However, when a sound image with a localization is generated, the attention of the vehicle driver is directed to the localized sound image. In addition, Patent Documents 1 to 3 specifically disclose only cases where sound images are localized at the front of the vehicle or in front of the driver, but sound images generated in the vehicle may be localized at the rear of the vehicle or behind the driver. In this case, the driver may turn his / her eyes to the rear side, which is opposite to the traveling direction of the vehicle. In particular, the driver may continue to turn his / her face or eyes to the sound image located behind the vehicle in order to clearly recognize the sound image.

[0005] When used in a vehicle, it is desirable for the stereoscopic sound image reproduction system to be able to suppress excessive driver behavior caused by localized sound images. [Means for solving the problem]

[0006] According to one embodiment of the present invention, a three-dimensional sound image reproducing system for a vehicle includes a plurality of speakers provided in the vehicle, a sound reproducing device having a sound image generating unit capable of generating a sound image that can be localized at least in the front-rear direction of the vehicle using the plurality of speakers, and a vehicle sensor capable of detecting at least one of a face or line of sight of a driver of the vehicle and a traveling direction of the vehicle, and the sound reproducing device detects a direction of the face or line of sight of the driver, a traveling direction of the vehicle, or a direction between the direction of the face or line of sight of the driver and the traveling direction of the vehicle during or before the sound image is generated by the sound image generating unit in an original localization of a sound source, as a reference direction, and detects a direction of the vehicle from the reference direction. a positioning permission angle range that is an angle range extending in the left and right direction of the sound image; and determining whether or not the direction of the face or line of sight of the driver of the vehicle detected by the vehicle sensor is maintained within the positioning permission angle range during the generation of the sound image, the sound image generation unit continues to generate the sound image at the original position, and if the direction of the face or line of sight of the driver of the vehicle is not within the positioning permission angle range during the generation of the sound image, the sound image generation unit discontinues generating the sound image at the original position. Effect of the Invention

[0007] In the present invention, a sound reproduction device having a sound image generation unit for generating a sound image that can be localized at least in the fore-and-aft direction of the vehicle using multiple speakers sets the direction of the driver's face or gaze, the direction of travel of the vehicle, or the direction between the direction of the driver's face or gaze and the direction of travel of the vehicle as a reference direction during or before the sound image generation unit generates a sound image in an original position for the sound source, and sets an angle range extending from the reference direction to the left and right of the vehicle as the localization permitted angle range, and determines whether the direction of the face or gaze of the driver of the vehicle detected by the vehicle sensor remains within the localization permitted angle range during the generation of the sound image. If the direction of the vehicle driver's face or the direction of his / her gaze remains within the permissible positioning angle range during generation of the sound image, the sound reproduction device continues to generate the sound image at the original position by the sound image generation unit. In contrast, if the direction of the vehicle driver's face or line of sight is no longer within the permissible positioning angle range during generation of a sound image, the sound reproduction device interrupts the generation of the sound image at the original position by the sound image generation unit.

[0008] As a result, the sound image being generated in the present invention is not positioned outside the permissible positioning angle range with respect to the direction of the driver's face, the direction of his / her line of sight, or the direction of travel of the vehicle during or before the sound image is generated. As a result, even if the driver of the vehicle turns his / her face or gaze toward the localized sound image, the driver's face or gaze can be maintained within the localization allowable angle range. The localization allowable angle range is an angle range that spreads from the reference direction to the left and right of the vehicle, based on the face direction, gaze direction, or vehicle traveling direction before or during the generation of the sound image. As a result, even if the driver continues to turn his / her face or gaze toward the localized sound image, the driver's face or gaze can be maintained in a state that does not move away from the front of the vehicle, and it becomes easier to maintain a state in which the vehicle's traveling direction is included within the range of the field of vision. In this way, the present invention can suppress excessive driver behavior caused by localized sound images. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram of a state in which a stereoscopic sound image is generated in an automobile to which a stereoscopic sound image reproducing system according to a first embodiment of the present invention can be applied. [Diagram 2] FIG. 2 is a configuration diagram of a stereophonic sound image reproducing system installed in the automobile of FIG. [Diagram 3] FIG. 3 is a flowchart of a stereoscopic sound image reproduction assistance control executed by the control unit of the sound generating device of FIG. 2 for stereoscopic sound image reproduction control. [Figure 4] FIG. 4 is a flowchart of a stereoscopic sound image reproduction control executed by the control unit of the sound generating device of FIG. [Diagram 5] FIG. 5 is an explanatory diagram of a state in which a stereoscopic sound image is generated by the stereoscopic sound image reproduction control of FIG. 4 when the car is moving straight ahead. [Figure 6] FIG. 6 is an explanatory diagram of a state in which a stereoscopic sound image is generated by the stereoscopic sound image reproduction control of FIG. 4 when the car is being steered to the right. [Figure 7] FIG. 7 is a flowchart of stereoscopic sound image reproduction control executed by the control unit of the stereoscopic sound image reproduction system according to the second embodiment of the present invention. [Figure 8] FIG. 8 is a flowchart of stereoscopic sound image reproduction control executed by the control unit of the stereoscopic sound image reproduction system according to the third embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] [First embodiment] FIG. 1 is an explanatory diagram of a state in which a stereoscopic sound image is generated in an automobile 1 to which a stereoscopic sound image reproducing system 10 according to a first embodiment of the present invention can be applied. The automobile 1 is an example of a vehicle. Other examples of vehicles in which passengers such as a driver ride include buses, trucks, trains, and personal mobility vehicles.

[0011] An automobile 1 in Fig. 1 has a vehicle body 2. A passenger compartment 3 of the vehicle body 2, in which passengers such as a driver ride, is provided with a plurality of seats including a driver seat 4. A driver who manages the operation of the automobile 1 sits in the driver seat 4. Passengers other than the driver sit in seats other than the driver seat 4.

[0012] In addition, a plurality of wheels are provided on both the left and right sides of the vehicle body 2, arranged front to back. Specifically, a right front wheel 5, a left front wheel 6, a right rear wheel 7, and a left rear wheel 8 are provided. The automobile 1 travels by rotating drive wheels among the plurality of wheels with a power source (not shown) while the automobile 1 is traveling manually or automatically. In addition, the automobile 1 can change the traveling direction, for example, from a straight-ahead direction indicated by a dashed arrow in the figure to a steering direction to the right indicated by a dashed arrow by steering wheels among the plurality of wheels with a power source (not shown) while the automobile 1 is traveling manually or automatically. The driver who manages the operation of the automobile 1 is required to face and look in the direction of travel of the automobile 1, pay attention to objects outside the vehicle, and ensure the safety of the operation, even in the case of an automatic driving. For example, when the automobile 1 is moving straight as shown by the dashed arrow 40 in the figure, the driver is required to face and look in the direction of the straight ahead as shown by the dashed arrow 41 in the figure. Also, when the automobile 1 is turning right as shown by the dashed arrow 50 in the figure, the driver is required to face and look in the direction to the right rear as viewed from the automobile 1 as shown by the dashed arrow 51 in the figure.

[0013] Meanwhile, it has been proposed that, in an automobile 1, a plurality of speakers are provided in the cabin 3 of the automobile 1 to reproduce a three-dimensional sound image having an original localization of a sound source. In the figure, a right front speaker 11, a left front speaker 12, a right rear speaker 13, and a left rear speaker 14 are illustrated. If at least two or more speakers are provided in the automobile 1, it is possible to generate a three-dimensional sound image having an original localization of the sound source. Here, the right front speaker 11 and the left front speaker 12 are speakers located in front of the driver's seat 4 of the automobile 1. Furthermore, the sound reproduced by the stereoscopic sound image depends in part on the stereoscopic sound image reproduction system, but it is possible to localize the sound in a 360-degree direction around the automobile 1, as shown in the figure from the first localization position P1 to the eighth localization position P8. Note that the figure shows three orthogonal axial directions, XYZ. The X-axis direction is the front-rear direction of the automobile 1. The Y-axis direction is the left-right direction or width direction of the automobile 1. The Z-axis direction is the up-down direction of the automobile 1. In this way, the sound produced by the three-dimensional sound image is localized not only at the first localization position P1, second localization position P2, and eighth localization position P8 at the front of the automobile 1, but also at the fourth localization position P4, fifth localization position P5, and sixth localization position P6 at the rear of the automobile 1, and the third localization position P3 and seventh localization position P7 on the left and right sides of the automobile 1. In Fig. 1, for convenience of illustration, each of the localization positions from the first localization position P1 to the eighth localization position P8 is illustrated as a ring shape surrounding the body 2 of the automobile 1 from the outside, but the localization positions actually generated by the stereophonic sound image reproduction system are often basically the space inside the multiple speakers 11-14 provided in the passenger compartment 3 of the automobile 1. In this case, it can be understood that the multiple localization positions in Fig. 1 are drawn by mapping those in the space inside the multiple speakers 11-14.

[0014] In this case, passengers in the automobile 1, such as the driver, will be more likely to direct their attention in the direction of the position where the stereoscopic sound image is localized. As a result, passengers, especially the driver, may turn their eyes toward the rear, which is opposite to the traveling direction of the automobile 1. The passengers, such as the driver, may continue to turn their faces or eyes toward the sound image at the sixth localization position P6, which is localized behind the passenger, as shown by the solid arrow 70 in the figure, in order to clearly recognize the sound image. When the three-dimensional sound image reproducing system 10 is used in the automobile 1, it is desirable to be able to suppress excessive driver behavior caused by sound images localized in the passenger compartment 3 of the automobile 1.

[0015] FIG. 2 is a configuration diagram of a stereophonic sound image reproducing system 10 provided in the automobile 1 of FIG. The stereoscopic sound image reproducing system 10 in FIG. 2 includes a right front speaker 11, a left front speaker 12, a right rear speaker 13, a left rear speaker 14, and a sound reproducing device 15. The sound reproducing device 15 has a memory 21, a receiver 22, a sound image generating unit 23, a monaural generating unit 24, an alarm generating unit 25, an amplifier 26, and a control unit 27. A timer 28 and a plurality of vehicle sensors provided in the automobile 1 are connected to the control unit 27. In Fig. 2, a steering angle sensor 16, an outside camera 17, a DMS (Driver Monitoring System) camera 18, and a GNSS (Global Navigation Satellite System) receiver 19 provided in the automobile 1 are exemplified as the plurality of vehicle sensors.

[0016] The memory 21 of the sound reproducing device 15 stores reproduced sound data of various sound sources, for example, conversion function data for generating a three-dimensional sound image from reproduced sound data of a monaural sound source, and the like. The reproduced sound data of a sound source may be, for example, monaural, stereo in which the sound source itself includes the original localization, reproduced sound data that has been made into a stereophonic format, for example, a multi-channel format, etc. The reproduced sound data of a monaural format or the like that has not been made into a stereophonic format may include information about the position of the original localization for localizing a stereophonic sound image based on the sound source. The types of sound data reproduced from the sound source may be, for example, sound data of reproducible content, voice data for route guidance, and so on.

[0017] The receiver 22 receives notification sound data from road markers such as beacons provided on the road along which the automobile 1 is traveling as playback sound data from a sound source. Such notification sound data may be, for example, data for pinpointing a road intersection, a restricted area, or the like. The receiving unit may also receive, via a base station, audio data of wide-area road guidance, audio data of content, data of a call with another person, etc., as audio data of a sound source. The type of audio data of a sound source that can be received depends on the functions and services supported by the receiver 22. Additionally, the playback sound data of these sound sources received by receiver 22 may include information about the original localization position of each sound source.

[0018] The sound image generator 23 generates a playback sound signal for playing back the playback sound data of the sound source acquired from the memory 21 or the receiver 22 in a three-dimensional sound image of the original position of the sound source of the playback sound data. The sound image generator 23 outputs the generated playback sound signal to the right front speaker 11, the left front speaker 12, the right rear speaker 13, and the left rear speaker 14 through the amplifier 26. As a result, a three-dimensional sound image having the original position of the sound source of the playback sound data is generated in the passenger compartment 3 of the automobile 1, as exemplified in FIG. Here, when the sound image generation unit 23 acquires playback sound data of, for example, a mono sound source in which the sound source itself does not contain the original positioning, the sound image generation unit 23 may convert the sound source using conversion function data acquired from the memory 21 to generate a playback sound signal to be output to each of the multiple speakers 11 to 14. On the other hand, when the reproduced sound data is acquired from a stereo sound source in which the sound source itself includes the original localization, the sound image generating unit 23 may generate a reproduced sound signal including the localization of the sound source as it is. In this way, the sound image generating unit 23 can generate a sound image that can be localized at least in the front-rear direction of the automobile 1, using the multiple speakers 11 to 14.

[0019] The sound image generating unit 23 can generate a stereoscopic sound image localized at a desired position based on the sound source of the reproduced sound data by selecting the conversion function data used for generating the sound image according to the localization position included in the reproduced sound data, for example. The sound image generating unit 23 can generate a stereoscopic sound image localized at a different position based on the sound source of the same reproduced sound data by changing the conversion function data used for generating the sound image. The sound image generating unit 23 can generate a stereoscopic sound image localized at, for example, the first localization position P1 or a stereoscopic sound image localized at the fifth localization position P5 based on the sound source of the same reproduced sound data. For this purpose, the memory 21 may record a plurality of conversion function data different for each localization position so as to correspond to at least each localization position from the first localization position P1 to the eighth localization position P8 in FIG. 1.

[0020] Furthermore, the sound image generating unit 23 can obtain conversion function data for changing a sound image localized at the first localization position P1 to a sound image localized at the eighth localization position P8 from conversion function data for localizing, for example, reproduction sound data of a monaural sound source in which the sound source itself does not include the localization of a three-dimensional sound image at the first localization position P1 and conversion function data for localizing at the eighth localization position P8. By using such conversion function data for changing the localization position, the sound image generating unit 23 can obtain a three-dimensional sound image localized at a position changed from the original from the reproduction sound data that has been made into a three-dimensional sound and in which the sound source itself includes the original localization. In addition, for example, the sound image generation unit 23 can convert stereophonic playback sound data, which contains the original positioning in the sound source itself, into monophonic sound source playback sound data, and further convert the monophonic sound source playback sound data obtained by the conversion using conversion function data corresponding to the desired positioning position, thereby obtaining a stereophonic sound image that is localized at a position changed from the original. In this way, the sound image generating unit 23 can generate a three-dimensional sound image localized at any position by changing the conversion function data used for generating the sound image according to information included in the reproduced sound data, regardless of the presence or absence of localization at the sound source included in the reproduced sound data. The sound image generating unit 23 can generate a sound image that can be localized at least in the front-rear direction of the automobile 1 by using the multiple speakers 11 to 14. The memory 21 may store, for example, multiple pieces of conversion function data for localization at least at positions from the first localization position P1 to the eighth localization position P8 in FIG. 1, for each localization position.

[0021] In addition, the conversion function data used by the sound image generating unit 23 to generate the reproduced sound data can be, for example, when used for reproduced sound data of a monaural sound source, basically, a transfer function representing how sound is transmitted when a sound source and a dummy head simulating the head of a driver are placed in a virtual space. For example, in a virtual space corresponding to FIG. 1, a sound source is placed at the fifth localization position P5, and a dummy head is placed at the position of the driver's seat 4. From the waveform of the sound source in this case and the waveform of the sound heard by the right ear of the dummy head, a transfer function for the right ear can be obtained. Similarly, from the waveform of the sound source and the waveform of the sound heard by the left ear of the dummy head, a transfer function for the left ear can be obtained. These transfer functions change the waveform of the sound source into the waveforms of the sounds heard on the left and right ears. The transfer function for the right ear and the transfer function for the left ear can be used as conversion function data for generating a three-dimensional sound image that localizes the reproduced sound data of a monaural system at the fifth localization position P5. For practical purposes, the conversion function data may be the above-mentioned transfer function that has been further adjusted according to the position of the speaker in the automobile 1, etc. Then, based on the instruction information included in the reproduced sound data, the sound image generating unit 23 may select conversion function data corresponding to the specified localization position from the multiple conversion function data corresponding to the multiple localization positions in Fig. 1. The sound image generating unit 23 may also select conversion function data that is predefined in the memory 21 according to the type of reproduced sound data, etc. For example, for playback sound data of route guidance for turning right, the sound image generation unit 23 can select conversion function data corresponding to the second positioning position P2 and conversion function data corresponding to the third positioning position P3 so that the sound is localized on the right side of the automobile 1. The playback sound data may include information instructing the position at which the sound is to be played. For example, the notification sound data from a road marker may include information on the position of the road marker in GNSS coordinates at which the notification sound of the road marker is to be localized. In this case, the sound image generating unit 23 may select the conversion function data so that the sound is localized at the relative position of the road marker in the automobile 1 based on the relative direction and distance between the current position of the automobile 1 and the position of the road marker.

[0022] For playback sound data that has already been stereophonically processed, the sound image generating unit 23 may or may not use the conversion function data acquired from the memory 21. Examples of playback sound data that has already been stereophonically processed include stereo playback sound data, 22.2 multichannel playback sound data, and binaural playback sound data. The sound image generating unit 23 may switch between using and not using the conversion function data depending on whether the playback sound data acquired from the memory 21 or the receiver 22 is stereophonically processed playback sound data or not. However, when changing the original localization of the reproduced sound data that has already been stereophonically processed, the sound image generating unit 23 may select conversion function data and change the localization of the reproduced sound data that has already been stereophonically processed using the selected conversion function data. In this case, the sound image generating unit 23 may convert the sound source of the reproduced sound data that has already been stereophonically processed, for example, to a monaural format, and then convert the monaural format sound source using conversion function data from the memory 21 for localization at a desired position. This allows the position where the sound image is localized in the passenger compartment 3 of the automobile 1 to be changed from the original localization of the reproduced sound data that has already been stereophonically processed. The sound image generating unit 23 may use the monaural generating unit 24 to convert the reproduced sound data that has already been stereophonically processed to a monaural format.

[0023] The monaural generation unit 24 converts the playback sound data acquired from the memory 21 or the receiver 22 into monaural sound to generate a playback sound signal. The playback sound data before conversion may be monaural playback sound data or may be stereophonically processed playback sound data. For example, the monaural generation unit 24 may generate playback sound data of a monaural sound source by superimposing all waveforms included in the playback sound data that has been stereophonically processed in advance. The monaural generation unit 24 outputs the generated playback sound signal to the right front speaker 11, the left front speaker 12, the right rear speaker 13, and the left rear speaker 14 through the amplifier 26. The monaural generation unit 24 may output a significant playback sound signal to some of the multiple speakers 11 to 14, for example, the right front speaker 11 and the left front speaker 12, and output a silent playback sound signal to the remaining right rear speaker 13 and left rear speaker 14.

[0024] The alarm generating unit 25 generates an alarm playback sound signal from the alarm playback sound data acquired from the memory 21. The alarm playback sound data may be a monaural sound source. The alarm generating unit 25 outputs the generated playback sound signal to the right front speaker 11, the left front speaker 12, the right rear speaker 13, and the left rear speaker 14 through the amplifier 26. The alarm generating unit 25 may output a significant alarm playback sound signal to some of the multiple speakers 11 to 14. For example, the alarm generating unit 25 may output a significant alarm playback sound signal to the right front speaker 11 and the left front speaker 12, and output a silent playback sound signal to the remaining right rear speaker 13 and left rear speaker 14.

[0025] The amplifier 26 is connected to the sound image generating unit 23, the monaural generating unit 24, the warning generating unit 25, the right front speaker 11, the left front speaker 12, the right rear speaker 13, and the left rear speaker 14. The amplifier 26 amplifies the playback sound signal generated by the sound image generating unit 23, the monaural generating unit 24, or the warning generating unit 25, and outputs it to the right front speaker 11, the left front speaker 12, the right rear speaker 13, and the left rear speaker 14.

[0026] The timer 28 measures the elapsed time and the time.

[0027] The control unit 27 controls the operation of the sound reproduction device 15. The control unit 27 may be configured, for example, with a CPU (Central Processing Unit). In this case, the CPU functions as the control unit 27 of the sound reproduction device 15 by reading and executing a program recorded in the memory 21. The control unit 27 selects and operates one of the sound image generation unit 23, the monaural generation unit 24, and the alarm generation unit 25, for example. This allows the sound reproduction device 15 to generate a stereoscopic sound image of the original position of the sound source of the reproduced sound data generated by the sound image generation unit 23 in the cabin 3 of the automobile 1, or generate a reproduced sound by the monaural generation unit 24 or the alarm generation unit 25 in the cabin 3 of the automobile 1. The control unit 27 may also execute a switching instruction of conversion function data to be used for the sound reproduction device 15 so as to change the position of the stereoscopic sound image generated by the sound reproduction device 15. By executing a program, the CPU may realize, in addition to the control unit 27, a sound image generating unit 23, a monaural generating unit 24, or an alarm generating unit 25 in the sound reproducing device 15.

[0028] The steering angle sensor 16 detects the steering angle of the steering wheels of the automobile 1. The steering wheels are generally the right front wheel 5 and the left front wheel 6. When the steering wheels are steered away from a straight ahead direction, the automobile 1 turns right or left depending on the steering direction and steering amount. The steering angle sensor 16 detects the steering direction and steering amount as information indicating the traveling direction of the automobile 1.

[0029] Exterior camera 17 captures images of the outside of automobile 1. Exterior camera 17 may be a stereo camera, a monocular camera, a 360-degree camera, or the like. Automobile 1 may also be provided with multiple sets of stereo cameras or multiple monocular cameras. In this way, exterior camera 17 captures images of the surroundings of automobile 1, particularly the range in front of automobile 1. The images captured by exterior camera 17 may include images of roads extending in the traveling direction of automobile 1. Exterior camera 17 captures images of the outside of the vehicle as information indicating the traveling direction of automobile 1.

[0030] The DMS camera 18 captures an image of the driver in the automobile 1. The DMS camera 18 may capture an image of a passenger other than the driver in the automobile 1. The image captured by the DMS camera 18 may include the face and eyes of the driver. The DMS camera 18 captures an image of the driver's face and eyes as information indicating the face or line of sight of the driver of the automobile 1.

[0031] In addition to the DMS camera 18, other vehicle sensors capable of detecting information indicating the face or line of sight of the driver of the automobile 1 include, for example, a rear camera of a drive recorder provided in the passenger compartment 3 of the automobile 1. Vehicle sensors capable of detecting information indicating the traveling direction of automobile 1 include steering angle sensor 16, exterior camera 17, and also acceleration sensors in three axes (XYZ), gyro sensors, and the like.

[0032] The GNSS receiver 19 receives radio waves from GNSS satellites and generates the current position of the automobile 1 and the current time.

[0033] FIG. 3 is a flowchart of a stereoscopic sound image reproduction assistance control executed by the control unit 27 of the sound generating device of FIG. 2 for stereoscopic sound image reproduction control. The control unit 27 of the sound generating device repeatedly executes the stereoscopic sound image reproduction assistance control of FIG.

[0034] In step ST11, the control section 27 judges whether or not to start generating a stereoscopic sound image, and whether or not a predetermined time has elapsed since the start of the generation of a stereoscopic sound image. For example, when reproduction of audio data for route guidance is requested as the automobile 1 travels, the control unit 27 starts generating a stereoscopic sound image. In addition, for example, when a passenger instructs the reproduction of sound data of content that has been converted into stereophonic sound, the control unit 27 starts generating a stereophonic sound image. In addition, for example, when the receiver 22 receives notification sound data, the control unit 27 starts generating a stereoscopic sound image. In addition, for example, when the receiver 22 is receiving call data with another person, the control unit 27 starts generating a stereoscopic sound image. When starting the generation of these stereoscopic sound images, the control section 27 advances the process to step ST12. Furthermore, when the elapsed time from the start of generation of these stereoscopic sound images, as measured by the timer 28, has exceeded a predetermined time, the control section 27 advances the process to step ST12. Furthermore, the control unit 27 may advance the process to step ST12 if a predetermined time has elapsed since the elapsed time determined to have elapsed the previous predetermined time. This allows the control unit 27 to advance the process to step ST12 at predetermined time intervals after starting the generation of a stereoscopic sound image. In cases other than those described above, the control unit 27 ends this control.

[0035] In step ST12, the control unit 27 acquires detection information from the vehicle sensors. The control unit 27 acquires detection information from vehicle sensors, for example, the steering angle sensor 16, the outside camera 17, the DMS camera 18, and the like.

[0036] In step ST13, the control section 27 sets a reference direction that serves as a reference for determining the position of the localization of a sound image, based on the detection information currently acquired in step ST12. The control unit 27 may set, for example, the direction of the face or the direction of the line of sight of the driver of the automobile 1 before the generation of the sound image detected by the DMS camera 18 starts as the reference direction. Alternatively, for example, the control unit 27 may set the traveling direction of the automobile 1 before the generation of the sound image is started, which is detected by the steering angle sensor 16, as the reference direction. Alternatively, for example, the control unit 27 may set the extension direction of the road on which the automobile 1 is traveling before starting to generate a sound image that can be detected by the exterior camera 17 as the reference direction. Here, the extension direction of the road is the traveling direction of the automobile 1. In addition, for example, if there is a difference of more than a threshold angle between the direction of the driver's face or gaze and the direction in which the vehicle 1 is traveling, the direction between them, for example the direction that is midway between them, may be set as the reference direction. In addition, for example, if there is a difference of more than a threshold angle between the direction of the driver's face or line of sight and the direction in which the road on which the automobile 1 is traveling extends, the direction between them, for example the midpoint between them, may be set as the reference direction. This enables the control unit 27 to set the direction of the driver's face or line of sight, the direction of travel of the automobile 1, or the direction between the direction of the driver's face or line of sight and the direction of travel of the automobile 1 as the reference direction.

[0037] In step ST14, the control unit 27 sets a localization permission angle range for determining the position of the localization of the sound image. The control unit 27 may set the positioning permission angle range to a predetermined angle range extending from the reference direction to the left and right of the automobile 1, for example, an angle range extending 160 degrees overall on both the left and right sides of the reference direction, for example, 80 degrees on each side. The localization permission angle range should be set so that even if the driver turns his / her face or eyes to the position of the stereoscopic sound image reproduced with localization, the driver's field of vision includes the traveling direction of the automobile 1. The localization permission angle range should be set to an angle range no greater than 180 degrees so that the driver will not turn backwards even if he / she turns his / her face or eyes to the localization position of the stereoscopic sound image. The localization permission angle range may be increased or decreased depending on the driving conditions of the automobile 1, such as the vehicle speed or steering amount of the automobile 1. For example, when the vehicle speed is equal to or higher than a threshold, the control unit 27 may narrow the localization permission angle range compared to when the vehicle speed is less than the threshold.

[0038] In step ST15, the control section 27 sets an origin 45 for determining the position of the localization of the sound image. For example, as shown in FIG. 1, the control unit 27 may set the position of the center of the front end of the driver's seat 4 of the automobile 1 as an origin 45 of the positioning permission angle range. Alternatively, for example, the center position of the front end of the automobile 1 may be set as the origin 45 of the positioning permission angle range.

[0039] In step ST16, the control unit 27 stores the information generated in steps ST13 to ST15 based on the information currently acquired in step ST12 as reference information in the memory 21. The control unit 27 stores the latest reference direction information, positioning permission angle range information, and origin 45 information corresponding to the latest states of the automobile 1 and the driver in the memory 21 as reference information. Thereafter, the control unit 27 ends this control.

[0040] FIG. 4 is a flowchart of a stereoscopic sound image reproduction control executed by the control unit 27 of the sound generating device of FIG. The control unit 27 of the sound generating device repeatedly executes the stereoscopic sound image reproduction control of FIG.

[0041] In step ST1, the control unit 27 determines whether or not to start generating a stereo sound image. The control unit 27 causes the sound image generation unit 23 to reproduce the reproduction sound data to be generated. The sound image generation unit 23 generates a reproduction signal for generating a sound image in the original localization of the sound source of the reproduction sound data based on the reproduction sound data to be generated. The sound image generation unit 23 outputs the generated reproduction signal to the right front speaker 11, the left front speaker 12, the right rear speaker 13, and the left rear speaker 14 via the amplifier 26. As a result, the sound image generation unit 23 starts generating a sound image in the original localization of the sound source of the reproduction sound data in the passenger compartment 3. Also, a three-dimensional sound image having the original localization of the reproduction sound data is generated in the passenger compartment 3 of the automobile 1.

[0042] In step ST2, the control unit 27 judges whether or not the sound image generation unit 23 is reproducing a three-dimensional sound image of the reproduced sound data. Immediately after executing step ST1, the sound image generation unit 23 is reproducing a three-dimensional sound image of the reproduced sound data. If a stereoscopic sound image is being reproduced, the control unit 27 advances the process to step ST3 in order to monitor the driver's inattentiveness, etc., caused by the localization of the sound image being reproduced. On the other hand, if the reproduction of the stereoscopic sound image has ended and is not being continued, the control unit 27 ends this control.

[0043] In step ST3, the control unit 27 acquires the latest detection information of the vehicle sensors and the latest reference information stored in the memory 21. The control unit 27 may acquire detection information from vehicle sensors, for example, the steering angle sensor 16, the outside camera 17, the DMS camera 18, and the like.

[0044] In step ST4, the control unit 27 judges whether or not the latest face direction or line of sight direction of the driver during the generation of the sound image is maintained within the latest localization permission angle range included in the reference information. Here, the control unit 27 may set a positioning permission angle range (46) based on an origin 45 at the center of the front edge of the driver's seat 4, as shown in Figure 5 or Figure 6 described later, and determine whether the direction of the driver's face or the direction of his or her line of sight is within the positioning permission angle range. Then, if the latest face direction or gaze direction of the driver during generation of the sound image is within the latest positioning permission angle range, the control unit 27 determines that it is maintained within the positioning permission angle range, and proceeds to step ST6. On the other hand, if the latest face direction or gaze direction of the driver during generation of the sound image is not within the latest positioning permission angle range, the control unit 27 determines that the positioning permission angle range is not being maintained, and proceeds to step ST5.

[0045] In step ST5, the control unit 27 further judges whether or not the direction of the localization of the sound image being generated is the same as the direction of the driver's face or line of sight. At this time, the control unit 27 may determine whether or not the direction of the localization of the sound image being generated is the same as the direction of the driver's face or line of sight, for the latest reference direction. Then, if the direction of the positioning of the sound image being generated is the same as the direction of the driver's face or line of sight, the control unit 27 assumes that the driver is distracted by the positioned sound image and is looking away, and proceeds to step ST7. On the other hand, if the direction of the positioning of the sound image being generated is not the same as the direction of the driver's face or line of sight, the control unit 27 proceeds to step ST6 since it is highly likely that the driver is distracted by the sound image being positioned and not looking away.

[0046] In step ST6, the control unit 27 causes the sound image generating unit 23 to continue generating and outputting a stereoscopic sound image in the original localization, which has been started in step ST1. As a result, if the direction of the face or the direction of the gaze of the driver of the automobile 1 remains within the permissible positioning angle range during generation of the sound image, the control unit 27 can continue generating the sound image at the original positioning by the sound image generation unit 23. In addition, when the direction of the positioning of the sound image being generated is not the same as the direction of the face or line of sight of the driver of the automobile 1 detected by the vehicle sensor, the control unit 27 can continue generating the sound image at the original positioning by the sound image generation unit 23. Thereafter, the control unit 27 ends this control.

[0047] In step ST7, the control unit 27 causes the sound image generating unit 23 to suspend the generation and output of the stereoscopic sound image at the original localization, which has been started in step ST1. This allows the control unit 27 to interrupt the generation of a sound image at the original position by the sound image generation unit 23 if the direction of the face or line of sight of the driver of the automobile 1 is no longer within the permissible positioning angle range during generation of the sound image. In addition, the control unit 27 can interrupt the generation of a sound image at the original position by the sound image generation unit 23 when the direction of the positioning of the sound image being generated is the same as the direction of the face or line of sight of the driver of the automobile 1 detected by the vehicle sensor.

[0048] In step ST8, the control unit 27 changes the position of the sound image localization so that it is within the localization permissible angle range, and instructs the sound image generation unit 23 to continue generating the sound image at the changed localization. As a result, when the control unit 27 interrupts the generation of a sound image at the original position of the sound source by the sound image generation unit 23, it can change the position of the sound image so that it is within the permissible positioning angle range, and continue generating the sound image at the changed position. Furthermore, the sound image generating section 23 can continue to generate a three-dimensional sound image for the sound source, even though the position of the localization has changed. Thereafter, the control unit 27 ends this control.

[0049] In this way, by repeatedly executing the stereoscopic sound image reproduction assistance control of FIG. 3 and the stereoscopic sound image reproduction control of FIG. 4 during playback of a stereoscopic sound image, the control unit 27 can interrupt the generation of a sound image at the original position of the sound source by the sound image generation unit 23 during playback. Furthermore, in step ST4 of FIG. 4, the control unit 27 can determine whether or not the direction of the driver's face or line of sight after the interruption has returned to within the localization permission angle range. Then, when the direction of the driver's face or line of sight returns to within the localization permissible angle range, the control unit 27 can restart the generation of a sound image in the original localization by the sound image generation unit 23. On the other hand, if the direction of the driver's face or line of sight has not returned to within the localization permissible angle range, the control unit 27 can continue to suspend the generation of a sound image in the original localization by the sound image generation unit 23. The control unit 27 repeats these controls during the reproduction of a stereoscopic sound image, and can effectively prevent the driver from being distracted by the positioning of the sound image and continuously looking away.

[0050] Next, a specific example of a state in which a stereoscopic sound image is generated by the stereoscopic sound image reproduction control described above will be described.

[0051] FIG. 5 is an explanatory diagram of a state in which a stereoscopic sound image is generated by the stereoscopic sound image reproduction control of FIG. 4 when the automobile 1 is moving straight ahead. 5, the position of the center of the front end of the driver's seat 4 is set as an origin 45. A positioning permission angle range 46 shown by a dashed line in the figure is indicated by a dashed line extending from the origin 45. 1 are within the localization permitted angle range 46. If the original localization position for the sound source of the reproduced sound data is one of these localization positions, the control unit 27 determines in step ST4 that the original localization position for the reproduced sound data is included within the range of the localization permitted angle range 46. In step ST6, the control unit 27 continues to generate a stereoscopic sound image at the original localization of the reproduced sound data based on the reproduced sound data.

[0052] On the other hand, if the original localization position for the reproduced sound data is the third localization position P3, the fourth localization position P4, the fifth localization position P5, the sixth localization position P6, or the seventh localization position P7, the control unit 27 determines in step ST4 that the original localization position for the reproduced sound data is not included within the localization permitted angle range 46. In step ST8, the control unit 27 changes the localization position of the reproduced sound data so that it is within the localization permitted angle range 46, and continues generating a stereoscopic sound image based on the reproduced sound data. For example, when the original position of the reproduced sound data is the third position P3, the control unit 27 changes the position of the reproduced sound data to the third changed position PM3 and continues to generate a stereoscopic sound image based on the reproduced sound data. Here, the third changed position PM3 is a position obtained by moving the third position P3 forward so as to be within the permitted positioning angle range 46. Similarly, when the original position of the reproduced sound data is the fourth position P4, the control unit 27 changes the position of the reproduced sound data to the fourth changed position PM4 and continues to generate a stereoscopic sound image based on the reproduced sound data. Here, the fourth changed position PM4 is a position obtained by moving the fourth position P4 forward so that it is within the permitted positioning angle range 46. Similarly, when the original position of the reproduced sound data is the fifth position P5, the control unit 27 changes the position of the reproduced sound data to the fifth changed position PM5 and continues to generate a stereoscopic sound image based on the reproduced sound data. Here, the fifth changed position PM5 is a position obtained by moving the fifth position P5 forward so as to be within the permitted positioning angle range 46. Similarly, when the original position of the reproduced sound data is the sixth position P6, the control unit 27 changes the position of the reproduced sound data to the sixth changed position PM6 and continues to generate a stereoscopic sound image based on the reproduced sound data. Here, the sixth changed position PM6 is a position obtained by moving the sixth position P6 forward so as to be within the permitted positioning angle range 46. Similarly, when the original position of the reproduced sound data is the seventh position P7, the control unit 27 changes the position of the reproduced sound data to the seventh changed position PM7 and continues to generate a stereoscopic sound image based on the reproduced sound data. Here, the seventh changed position PM7 is a position obtained by moving the seventh position P7 forward so as to be within the permitted positioning angle range 46. As a result, the stereoscopic sound image that is actually generated in the passenger compartment 3 of the automobile 1 by the sound data that has started to be reproduced is included within the localization permission angle range 46 .

[0053] FIG. 6 is an explanatory diagram of a state in which a stereoscopic sound image is generated by the stereoscopic sound image reproduction control of FIG. 4 when the automobile 1 is being steered to the right. 6, the position of the center of the front end of the driver's seat 4 is set as an origin 45. The positioning permission angle range 52 is indicated by a dashed line extending from the origin 45. 1 are within the localization permitted angle range 52. If the original localization position for the sound source of the reproduced sound data is one of these localization positions, the control unit 27 determines in step ST4 that the original localization position for the reproduced sound data is included within the range of the localization permitted angle range 52. In step ST6, the control unit 27 continues to generate a stereoscopic sound image in the original localization of the reproduced sound data based on the reproduced sound data.

[0054] On the other hand, if the original localization position for the reproduced sound data is the fourth localization position P4, the fifth localization position P5, the sixth localization position P6, or the seventh localization position P7, the control unit 27 determines in step ST4 that the original localization position for the reproduced sound data is not included within the localization permitted angle range 52. In step ST8, the control unit 27 changes the localization position of the reproduced sound data so that it is within the localization permitted angle range 52, and continues generating a stereoscopic sound image based on the reproduced sound data. For example, when the original position of the reproduced sound data is the fourth position P4, the control unit 27 changes the position of the reproduced sound data to the fourth changed position PM4 and continues to generate a stereoscopic sound image based on the reproduced sound data. Here, the fourth changed position PM4 is a position obtained by moving the fourth position P4 forward so as to be within the permitted positioning angle range 52. Similarly, when the original position of the reproduced sound data is the fifth position P5, the control unit 27 changes the position of the reproduced sound data to the fifth changed position PM5 and continues to generate a stereoscopic sound image based on the reproduced sound data. Here, the fifth changed position PM5 is a position obtained by moving the fifth position P5 forward so as to be within the permitted positioning angle range 52. Similarly, when the original position of the reproduced sound data is the sixth position P6, the control unit 27 changes the position of the reproduced sound data to the sixth changed position PM6 and continues to generate a stereoscopic sound image based on the reproduced sound data. Here, the sixth changed position PM6 is a position obtained by moving the sixth position P6 forward so as to be within the permitted positioning angle range 52. Similarly, when the original position of the reproduced sound data is the seventh position P7, the control unit 27 changes the position of the reproduced sound data to the seventh changed position PM7 and continues to generate a stereoscopic sound image based on the reproduced sound data. Here, the seventh changed position PM7 is a position obtained by moving the seventh position P7 forward so as to be within the permitted positioning angle range 52. As a result, the stereoscopic sound image that is actually generated in the passenger compartment 3 of the automobile 1 by the sound data that has started to be reproduced is included within the range of the localization permission angle range 52 .

[0055] In addition, in FIG. 6, a reference direction and another localization permission angle range 60 are indicated by two-dot chain lines. Another localization permission angle range 60 indicated by the two-dot chain line is directed further to the right rear than the localization permission angle range indicated by the dashed line. When the vehicle is steered further to the right than in the case of Fig. 6, the reference direction and the localization permission angle range are directed further to the right as shown by the two-dot chain line in the figure compared to those indicated by the dashed line in Fig. 6. In this way, when the localization permission angle range is tilted significantly backward, the control unit 27 of the sound reproduction device 15 may narrow the localization permission angle range in the determination of step ST6. If the positioning permission angle range includes a portion that is rearward of the vehicle width direction of the automobile 1, the control unit 27 may set the positioning permission angle range to be an angle range that is forward of the vehicle width direction of the automobile 1, excluding the rear portion, as shown by the dotted line arrow 61 in Figure 6. As a result, the stereoscopic sound image is not localized behind the driver in the vehicle width direction of the automobile 1. Even if the driver turns his / her face or eyes toward the position where the stereoscopic sound image is localized, he / she will not look back behind the vehicle width direction of the automobile 1.

[0056] As described above, in this embodiment, the sound reproducing device 15 has a sound image generating unit 23 for generating a sound image that can be localized at least in the front-rear direction of the automobile 1 using a plurality of speakers. The sound reproducing device 15 sets the direction in front of the automobile 1, the traveling direction of the automobile 1, or the direction between the front of the automobile 1 and the traveling direction of the automobile 1 as the reference direction during or before the sound image generating unit 23 generates a sound image in the original localization of the sound source. The sound reproducing device 15 also sets an angle range extending from the reference direction to the left and right directions of the automobile 1 as the localization permission angle range. The sound reproducing device 15 then determines whether or not the direction of the face or the line of sight of the driver of the automobile 1 detected by the vehicle sensor during the generation of the sound image is maintained within the localization permission angle range. Then, if the direction of the face or the direction of the gaze of the driver of the automobile 1 remains within the permissible positioning angle range during the generation of the sound image, the sound reproduction device 15 continues to generate the sound image at the original position by the sound image generation unit 23. In contrast, if the direction of the face or the direction of the gaze of the driver of the automobile 1 is no longer within the permissible positioning angle range during generation of the sound image, the sound reproduction device 15 interrupts the generation of the sound image at the original position by the sound image generation unit 23.

[0057] As a result, the sound image being generated in this embodiment is not positioned outside the permissible positioning angle range with respect to the direction of the driver's face, the direction of line of sight, or the direction of travel of the automobile 1 during or before the sound image is generated. As a result, even if the driver of the automobile 1 turns his / her face or gaze toward the localized sound image, the driver's face or gaze can be maintained within the localization permission angle range. The localization permission angle range is an angle range that spreads from the reference direction to the left and right of the automobile 1, based on the face direction, gaze direction, or traveling direction of the automobile 1 before or during the generation of the sound image. As a result, even if the driver continues to turn his / her face or gaze toward the localized sound image, the driver's face and gaze can be maintained without moving away from the front of the automobile 1, and it becomes easier to maintain a state in which the traveling direction of the automobile 1 is included within the range of the field of vision. In this manner, in this embodiment, excessive driver behavior caused by a localized sound image can be suppressed.

[0058] In this embodiment, the sound playback device 15 further determines, with respect to the reference direction, whether the direction of the positioning of the sound image being generated is the same as the direction of the face or line of sight of the driver of the automobile 1 detected by the vehicle sensor. Then, when the direction of the positioning of the sound image being generated is the same as the direction of the face or gaze of the driver of the automobile 1 detected by the vehicle sensor, the sound reproduction device 15 interrupts the generation of the sound image at the original position by the sound image generation unit 23. In contrast, if the direction of the positioning of the sound image being generated is not the same as the direction of the face or gaze of the driver of the automobile 1 detected by the vehicle sensor, the sound reproduction device 15 continues to generate the sound image at the original position by the sound image generation unit 23. In this embodiment, the position of the driver's seat 4 of the automobile 1 is used as the origin of the localization permission angle range. This makes it possible to set and determine the localization permission angle range based on the driver who is in charge of driving the automobile 1. In this embodiment, the localization permission angle range is set to an angle range that falls in front of the vehicle width direction of the automobile 1. This makes it difficult for the driver of the automobile 1 to turn his / her face or gaze rearward of the vehicle width direction of the automobile 1 due to the localized sound image. As a result, in this embodiment, when the driver is looking away regardless of the localized sound image, it is possible to continue generating a sound image having the original localization.

[0059] In this embodiment, when the sound reproduction device 15 interrupts the generation of a sound image at the original position of the sound source, it changes the position of the sound image so that it is within the permissible positioning angle range, and continues the generation of the sound image at the changed position by the sound image generation unit 23. As a result, even if a sound image is not generated in the original localization, passengers such as the driver of the automobile 1 can hear the sound source that is the source of the sound image with a localization. Even if the sound image generating unit 23 does not generate a sound image in the original localization, passengers such as the driver of the automobile 1 are expected to be less likely to feel uncomfortable with the reproduced sound because the localized sound image remains.

[0060] In this embodiment, after the sound reproduction device 15 interrupts the generation of a sound image at the original position of the sound source by the sound image generation unit 23, it can determine whether the direction of the face or the direction of the gaze of the driver of the automobile 1 detected by the vehicle sensor has returned to within the permissible positioning angle range. Then, when the direction of the face or the direction of gaze of the driver of the automobile 1 detected by the vehicle sensor returns to within the positioning permission angle range, the sound reproduction device 15 resumes the generation of a sound image at the original position by the sound image generation unit 23. On the other hand, if the direction of the face or the direction of gaze of the driver of the automobile 1 detected by the vehicle sensor has not returned to within the permissible positioning angle range, or if it is again no longer within the permissible positioning angle range, the sound reproduction device 15 continues to suspend the generation of the sound image at the original position by the sound image generation unit 23. This allows passengers such as the driver of the automobile 1 to understand that a localized sound image is generated only when the direction of the driver's face or line of sight is within the localization permitted angle range. It is expected that the driver of the automobile 1 will understand that it is undesirable to turn his / her face or line of sight toward a localized sound image so that it is outside the localization permitted angle range, and will not turn his / her face or line of sight so that it is outside the localization permitted angle range even if a localized sound image is generated. This may lead to the driver not paying excessive attention to the localized sound image.

[0061] [Second embodiment] Next, a vehicle 1 according to a second embodiment of the present invention will be described. In this embodiment, the stereoscopic sound image reproduction control by the control unit 27 of the sound reproduction device 15 is different from that of the above-described embodiment. Specifically, the process when it is determined in step ST4 that the orientation of the driver is not within the localization allowable angle range is different from that in the above-described embodiment. In the following, differences from the above-described embodiment will be mainly described.

[0062] FIG. 7 is a flowchart of stereoscopic sound image reproduction control executed by the control unit 27 of the stereoscopic sound image reproduction system 10 according to the second embodiment of the present invention. The control unit 27 of the sound generating device repeatedly executes the stereoscopic sound image reproduction control of FIG. 7 at regular intervals. In step ST5, if the control unit 27 of the sound generating device determines that the direction of the positioning of the sound image being generated is the same as the direction of the driver's face or line of sight, it assumes that the driver is distracted by the positioned sound image and is looking away, and proceeds to step ST21.

[0063] In step ST21, the control unit 27 causes the sound image generating unit 23 to suspend the generation and output of a stereoscopic sound image at the original localization, which has been started in step ST1. This allows the control unit 27 to interrupt the generation of a sound image at the original position by the sound image generation unit 23 if the direction of the face or line of sight of the driver of the automobile 1 is no longer within the permissible positioning angle range during generation of the sound image. In addition, the control unit 27 can interrupt the generation of a sound image at the original position by the sound image generation unit 23 when the direction of the positioning of the sound image being generated is the same as the direction of the face or line of sight of the driver of the automobile 1 detected by the vehicle sensor.

[0064] In step ST22, the control unit 27 causes the warning generation unit 25 to output a warning to inform the occupants that the generation of the stereoscopic sound image has been stopped. In this case, the warning generating unit 25 may output a warning sound from the right front speaker 11 and the left front speaker 12 located in front of the driver's seat 4 of the automobile 1, among the multiple speakers 11 to 14 provided in the automobile 1. By outputting such a warning sound, the driver's consciousness, face, and line of sight are likely to be directed toward the front of the automobile 1 due to the sound from the front.

[0065] In step ST23, the control unit 27 causes the timer 28 to measure the elapsed time from the current time.

[0066] In step ST24, the control unit 27 determines whether or not the elapsed time measured by the timer 28 has exceeded a predetermined time serving as a threshold value. Here, the predetermined time may be, for example, about several seconds. If the elapsed time has not exceeded the predetermined time, the control unit 27 repeats this process. When the elapsed time reaches a predetermined time, the control unit 27 ends this control.

[0067] In this way, when the control unit 27 of the sound generating device does not start generating a sound image at the original position of the sound source of the playback sound data using the sound image generating unit 23, an alarm sound is output from a speaker in front of the driver's seat 4 of the automobile 1 among the multiple speakers 11 to 14. Thereafter, the control unit 27 of the sound generating device waits for a predetermined time to elapse in step ST12, and then ends the stereoscopic sound image reproduction control of FIG. As a result, the timing when the sound generation device next executes the stereoscopic sound image reproduction control of Fig. 7 is delayed by the predetermined time from which measurement starts from step ST23. Therefore, the timing when the sound generation device executes step ST6 in the stereoscopic sound image reproduction control of Fig. 7 at the next timing is also delayed by the predetermined time from which measurement starts from step ST23. The timing when the sound image generation unit 23 generates a sound image at the original position for the sound source of the reproduced sound data in step ST6 is delayed.

[0068] Even if the sound generating device does not generate a stereoscopic sound image in the original localization for the sound source of the reproduced sound data in the first stereoscopic sound image reproduction control of Fig. 7, the sound generating device can generate a stereoscopic sound image in the original localization for the sound source of the reproduced sound data in the subsequent stereoscopic sound image reproduction control. The driver and other passengers of the automobile 1 can hear the stereoscopic sound image in the original localization for the sound source of the reproduced sound data at a delayed timing. It is expected that the driver and other passengers of the automobile 1 will not feel uncomfortable with the reproduced stereoscopic sound image itself. Also, since the position of the stereoscopic sound image falls within the localization permission angle range as in the above-described embodiment, even if a driver or other occupant is distracted by the localization of the sound image and turns his / her face or gaze toward the localization position, he / she will not deviate significantly from the traveling direction of the automobile 1. It becomes easier for the driver or other occupant to keep his / her face facing the front of the automobile 1 while the automobile 1 is traveling. The driver or other occupant is less likely to be distracted by the stereoscopic sound image in the original localization of the sound source of the reproduced sound data.

[0069] [Third embodiment] Next, a vehicle 1 according to a third embodiment of the present invention will be described. In this embodiment, the stereoscopic sound image reproduction control by the control unit 27 of the sound reproduction device 15 is different from that of the above-described embodiment. Specifically, the process when it is determined in step ST4 that the orientation of the driver is not within the localization allowable angle range is different from that in the above-described embodiment. In the following, differences from the above-described embodiment will be mainly described.

[0070] FIG. 8 is a flowchart of stereoscopic sound image reproduction control executed by the control unit 27 of the stereoscopic sound image reproduction system 10 according to the third embodiment of the present invention. The control unit 27 of the sound generating device repeatedly executes the stereoscopic sound image reproduction control of FIG. In step ST5, if the control unit 27 of the sound generating device determines that the direction of the sound image being generated is the same as the direction of the driver's face or line of sight, it assumes that the driver is distracted by the sound image being generated and is looking away, and proceeds to step ST31.

[0071] In step ST31, the control unit 27 causes the monaural generation unit 24 to generate a monaural reproduction sound signal in order to stop the generation of a stereoscopic sound image. At this time, the monaural generation section 24 may change the sound image to a monaural sound and generate a monaural reproduction sound signal.

[0072] In step ST32, the control unit 27 controls the mono generation unit 24 to output a mono playback sound signal from the right front speaker 11 and the left front speaker 12, which are located in front of the driver's seat 4 of the automobile 1, among the multiple speakers 11 to 14 provided in the automobile 1. By outputting monaural sound only from the front speakers, the driver's attention, face and gaze are more likely to be directed toward the front of the automobile 1.

[0073] As described above, in this embodiment, when the sound reproduction device 15 does not start generating a stereo sound image at the original position of the sound source of the reproduced sound data by the sound image generation unit 23, the sound image is changed to a mono sound and output from the speaker in front of the driver's seat 4 of the automobile 1 among the multiple speakers 11 to 14. This allows passengers, such as the driver of the automobile 1, to hear the sound source as monaural sound even if the sound image is not generated in the original position of the playback sound data. It is expected that passengers, such as the driver of the automobile 1, will listen to the sound source of the playback sound data and will not feel excessive discomfort from the reproduced sound. Also, the sound image is changed to a monaural sound and output from the speaker in front of the driver's seat 4 of the automobile 1 among the multiple speakers 11-14. In this case, not only is the sound image no longer localized and passengers such as the driver less likely to pay attention to it, but they are also more likely to be drawn to the sound from the front of the automobile 1 and direct their attention to the front of the automobile 1. Passengers such as the driver are more likely to keep their faces facing the front of the automobile 1 while the automobile 1 is traveling. The driver is less likely to be distracted by the stereoscopic sound image in the original localization of the sound source of the reproduced sound data.

[0074] The above-described embodiment is an example of a preferred embodiment of the present invention, but the present invention is not limited to this and various modifications and changes are possible.

[0075] In each of the above-described embodiments, in step ST15, the control unit 27 of the sound reproducing device 15 sets the position of the center of the front edge of the driver's seat 4 of the automobile 1 as the origin 45 of the positioning permission angle range. Alternatively, for example, origin 45 of the positioning permission angle range may be the center of the front edge of automobile 1 , which is forward of the center of the front edge of driver seat 4 .

[0076] In each of the above-described embodiments, the control unit 27 of the sound playback device 15 sets the positioning permission angle range to an angle range that is in front of the vehicle width direction of the automobile 1, excluding the rear part, when there is a part that is rear of the vehicle width direction of the automobile 1 within a specified angle range extending from the reference direction to the left and right of the automobile 1, as shown by the dotted line 60 in Figure 6. In addition, for example, the control unit 27 of the sound playback device 15 may set the original angle range as the positioning permission angle range even if there is a portion that is rearward of the vehicle width direction of the automobile 1 within a specified angle range from the reference direction. However, it is not desirable for the driver of the automobile 1 to turn his / her face or gaze to the rear of the automobile 1 while driving. Therefore, in order to reduce the possibility that the driver of the automobile 1 will turn his / her face or gaze to the rear of the automobile 1 while driving, it is preferable that the control unit 27 of the sound reproducing device 15 sets the positioning permission angle range to an angle range that falls in front of the automobile 1 in the vehicle width direction, excluding the part that is rear of the automobile 1 in the vehicle width direction.

[0077] In the first embodiment described above, when the control unit 27 of the sound reproduction device 15 does not start generating a three-dimensional sound image at the original position for the sound source of the reproduced sound data, as a first process, it changes the position of the sound image so that it is within the permissible positioning angle range, and continues generating the sound image at the changed position by the sound image generation unit 23. In the second embodiment, when the control unit 27 of the sound reproduction device 15 does not start generating a stereoscopic sound image at the original position of the sound source of the reproduced sound data, as a second process, it outputs an alarm sound from a speaker among the multiple speakers 11 to 14 that is in front of the driver's seat 4 of the automobile 1, and further, as a third process, it delays the start timing of generating a stereoscopic sound image at the original position of the sound source of the reproduced sound data by the sound image generation unit 23. In the third embodiment, when the control unit 27 of the sound playback device 15 does not start generating a stereo sound image at the original position of the sound source of the playback sound data, as a fourth process, the control unit 27 changes the sound image to a mono sound and outputs it from a speaker in front of the driver's seat 4 of the automobile 1 among the multiple speakers 11 to 14. In addition, for example, when the control unit 27 of the sound playback device 15 does not start generating a stereoscopic sound image at the original position of the sound source of the playback sound data, the control unit 27 may execute the first to fourth processes in an appropriate combination or switch between them. [Explanation of symbols]

[0078] 1...Automobile (vehicle), 2...Vehicle body, 3...Vehicle interior, 4...Driver's seat, 5...Right front wheel, 6...Left front wheel, 7...Right rear wheel, 8...Left rear wheel, 10...Three-dimensional sound image reproduction system, 11...Right front speaker, 12...Left front speaker, 13...Right rear speaker, 14...Left rear speaker, 15...Sound reproduction device, 16...Steering angle sensor, 17...Exterior camera, 18...DMS camera, 19...GNSS receiver, 21...Memory, 22...Receiver, 23...Sound image generation unit, 24...Monaural generation unit, 25...Alarm generation unit, 26...Amplifier, 27...Control unit, 28...Timer, 45...Origin, 46...Allowed positioning angle range, 52...Allowed positioning angle range, 60...Allowed positioning angle range

Claims

1. A plurality of speakers provided in a vehicle; a sound reproducing device having a sound image generating unit capable of generating a sound image that can be localized at least in a front-rear direction of the vehicle using a plurality of the speakers; A vehicle sensor capable of detecting at least one of a face or a line of sight of a driver of the vehicle and a traveling direction of the vehicle; having The sound reproducing device comprises: The direction of the driver's face or line of sight, the traveling direction of the vehicle, or the direction between the direction of the driver's face or line of sight and the traveling direction of the vehicle during or before the generation of the sound image in the original localization of the sound source by the sound image generation unit is set as a reference direction, and an angle range extending from the reference direction to the left and right of the vehicle is set as a localization permission angle range; determining whether or not the face direction or the line of sight direction of the driver of the vehicle detected by the vehicle sensor is maintained within the localization allowable angle range during generation of the sound image; If the face direction or the line of sight direction of the driver of the vehicle is maintained within the localization allowable angle range during the generation of the sound image, the sound image generation unit continues to generate the sound image in the original localization, When the direction of the face or the direction of the line of sight of the driver of the vehicle is not within the localization allowable angle range during the generation of the sound image, the generation of the sound image at the original localization by the sound image generation unit is interrupted. Vehicle three-dimensional sound reproduction system.

2. The sound reproducing device further comprises: Regarding the reference direction, it is determined whether or not a direction of the localization of the sound image being generated and a face direction or a line of sight direction of the driver of the vehicle detected by the vehicle sensor are the same direction; When the direction of the localization of the sound image being generated is the same as the direction of the face or the direction of the line of sight of the driver of the vehicle detected by the vehicle sensor, the generation of the sound image in the original localization by the sound image generation unit is interrupted; When the direction of the localization of the sound image being generated is not the same as the direction of the face or the direction of the line of sight of the driver of the vehicle detected by the vehicle sensor, the sound image generation unit continues to generate the sound image in the original localization.

2. A vehicle stereophonic sound image reproducing system according to claim 1.

3. The sound reproducing device comprises: using a position of a driver's seat of the vehicle as an origin of the localization allowable angle range, and determining whether or not the direction of the face or the direction of the line of sight of the driver of the vehicle detected by the vehicle sensor during the generation of the sound image is within the localization allowable angle range; 3. A vehicle stereophonic sound image reproducing system according to claim 2.

4. The sound reproducing device comprises: When the angle range extending from the reference direction to the left and right of the vehicle includes a portion that is rearward of the vehicle width direction of the vehicle, an angle range that is forward of the vehicle width direction of the vehicle is set, excluding the rear portion.

4. A vehicle stereophonic sound image reproducing system according to claim 3.

5. The sound reproducing device comprises: When the generation of the sound image at the original localization of the sound source by the sound image generation unit is interrupted, outputting an alarm sound from a speaker located in front of a driver's seat of the vehicle among the plurality of speakers; Delaying the timing of generating the sound image at the original localization; Change the position of the sound image so that it is within the range of allowable angles for localization, and continue generating the sound image at the changed position, or The sound image is changed to a monaural sound and output from a speaker located in front of a driver's seat of the vehicle among the plurality of speakers.

5. A vehicle stereophonic sound image reproducing system according to claim 4.

6. The sound reproducing device comprises: After interrupting the generation of the sound image by the sound image generation unit in the original localization of the sound source, it is determined whether or not the face direction or the line of sight direction of the driver of the vehicle detected by the vehicle sensor has returned to within the localization allowable angle range; When the face direction or the line of sight direction of the driver of the vehicle detected by the vehicle sensor returns to within the localization allowable angle range, the sound image generation unit resumes generation of the sound image at the original localization.

6. A vehicle stereophonic sound image reproducing system according to claim 5.

Images