In-vehicle device and method for adjusting sound image localization position
The in-vehicle device uses gaze detection to adjust sound localization, addressing the mismatch between line of sight and sound perception, enhancing the audio-visual experience by aligning sound sources with the occupant's gaze direction.
Patent Information
- Application Number
- JP2022108764
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2022-07-06
- Publication Date
- 2026-05-13
- Estimated Expiration
- 2042-07-06
AI Technical Summary
There is a discrepancy between the line of sight of a vehicle occupant and the sound image localization position when viewing video content on a display while the vehicle is stopped, leading to a mismatch between the direction of gaze and the perceived sound generation.
An in-vehicle device equipped with a gaze detection sensor and a control unit adjusts the sound image localization position based on the occupant's gaze direction, using sound field tuning to align the perceived sound source with the direction of gaze.
The device reduces the discrepancy between the line of sight and sound image localization, ensuring a more immersive audio-visual experience by aligning the perceived sound source with the direction of gaze, particularly when viewing video content.
Smart Images
Figure 0007857815000001 
Figure 0007857815000002 
Figure 0007857815000003
Abstract
Description
Technical Field
[0001] The present invention relates to an in-vehicle device and an audio-visual positioning position adjustment method.
Background Art
[0002] In vehicles such as passenger cars, speakers for outputting sounds such as music and voices are installed. Conventionally, in order to improve the sound quality in a vehicle, sound field tuning suitable for the vehicle has been performed. Sound field tuning is to adjust the output of the speakers installed in the vehicle so that music and voices are felt to be generated from a specific position. The specific position is, for example, an arbitrary position such as in front of the line of sight of a passenger sitting in the driver's seat or the passenger seat, or in the center of the back of the instrument panel (IP). When sounds such as music and voices are generated, the position felt by the person listening to the sound is called the audio-visual positioning position. That is, sound field tuning is to adjust the audio-visual positioning position. Sound field tuning is performed, for example, by adjusting the balance of the output magnitudes of a plurality of speakers, the delay time, and the like.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Patent Document 2
Summary of the Invention
Problems to be Solved by the Invention
[0004] On the other hand, when a passenger is viewing video content displayed on a CID (Center Information Display) while the vehicle is stopped, etc., there is a difference between the direction in which the passenger is looking at the video (line-of-sight direction) and the direction in which the sound is felt to be generated.
[0005] The embodiments of the disclosure aim to reduce the discrepancy between the line of sight of an occupant riding in a vehicle and the sound image localization position of the sound within the vehicle. [Means for solving the problem]
[0006] To solve the above problems, the following measures will be taken. In other words, the first embodiment is an in-vehicle device mounted on a vehicle and comprising a gaze detection sensor and a control unit. The gaze detection sensor detects the direction of gaze of an occupant riding in the vehicle. Based on the direction of gaze of the occupant detected by the gaze detection sensor, the control unit adjusts the sound image localization position of the sound output inside the vehicle so that it corresponds to the direction of gaze.
[0007] The mode of disclosure may be realized by the execution of a program by an information processing device. That is, the configuration of the disclosure can be identified as a program that causes an information processing device to execute the processing performed by each of the means in the above mode, or as a recording medium readable by the information processing device that stores the program. Alternatively, the configuration of the disclosure may be identified as a method by which the information processing device executes the processing performed by each of the means described above. Alternatively, the configuration of the disclosure may be identified as a system including an information processing device that performs the processing performed by each of the means described above. The information processing device is, for example, a computer. A computer is sometimes called a personal computer or a server. [Effects of the Invention]
[0008] The disclosed embodiment can reduce the discrepancy between the line of sight of an occupant in a vehicle and the sound image localization position of the sound within the vehicle. [Brief explanation of the drawing]
[0009] [Figure 1] Figure 1 shows an example of the configuration of an in-vehicle device according to an embodiment. [Figure 2] Figure 2 shows an example of a vehicle 1 on which the in-vehicle device 100 is installed. [Figure 3] Figure 3 shows an example of the operation flow of the in-vehicle device 100. [Figure 4] Figure 4 shows an example of the operation flow of a modified example 2 of the in-vehicle device 100. [Modes for carrying out the invention]
[0010] Embodiments will be described below with reference to the drawings. The configurations of the embodiments are illustrative, and the configuration of the invention is not limited to the specific configurations of the disclosed embodiments. In carrying out the invention, specific configurations may be adopted as appropriate depending on the embodiment.
[0011] [Embodiment] <Example Configuration> Figure 1 shows an example configuration of an in-vehicle device according to this embodiment. The in-vehicle device 100 in Figure 1 includes a control unit 102, a storage unit 104, an input / output interface (IF) 106, a communication interface (IF) 108, a display 122, a speaker 124, and a gaze detection sensor 126. The in-vehicle device 100 in Figure 1 is mounted in a vehicle such as a passenger car having a driver's seat and a passenger seat. The storage unit 104, the input / output interface 106, and the communication interface 108 are connected to each other by a bus. The display 122, the speaker 124, and the gaze detection sensor 126 are connected to the control unit 102, etc., via the input / output interface 106. The in-vehicle device 100 can be realized, for example, by a car navigation system equipped with a computer, a personal computer, etc.
[0012] Figure 2 shows an example of a vehicle 1 on which the in-vehicle device 100 is installed. Figure 2 is a view of the interior of vehicle 1 from above. In the example of Figure 2, vehicle 1 includes a left door mirror 242, a right door mirror 244, a rearview mirror 246, a right front seat (driver's seat) 310, a left front seat (passenger seat) 320, a right rear seat 330, and a left rear seat 340. In addition, multiple speakers 124 are installed in vehicle 1. In the example of Figure 2, speakers 124 are installed on the left front and right front of vehicle 1. Speakers 124 may be installed not only in front of vehicle 1, but also on the sides and rear of vehicle 1. Speakers 124 may include panel speakers that output sound by vibrating the screen of the display 122. In addition, vehicle 1 is installed with a gaze detection sensor 126 that detects the gaze of the occupants. A gaze detection sensor may be installed for each occupant. The left door mirror 242 is a mirror installed on the left front door of vehicle 1. The right door mirror 244 is a mirror installed on the right front door of vehicle 1. The rearview mirror 246 is a mirror installed in the front center of the interior of vehicle 1. The left door mirror 242, the right door mirror 244, and the rearview mirror 246 may be electronic mirrors that display images captured by a camera installed in vehicle 1. In this case, the left door mirror 242, the right door mirror 244, and the rearview mirror 246 are displays. The left door mirror 242, the right door mirror 244, and the rearview mirror 246 may also include panel speakers. The right front seat 310 is the driver's seat where the driver sits. The left front seat 320, the right rear seat 330, and the left rear seat 340 are seats where passengers other than the driver sit. The driver is one of the passengers. The display 122 is installed in a position where the occupant (driver) seated in the right front seat (driver's seat) 310 and the occupant (passenger seat occupant) seated in the left front seat 320 of the vehicle can see the display 122. The vehicle to which this embodiment is applied is not limited to a vehicle like vehicle 1 shown in Figure 2, and may be a vehicle with a different number of seats or seat arrangement. Also, the number and arrangement of the display 122, speaker 124, and gaze detection sensor 126 are not limited to those shown in Figure 2, and may be different in number and arrangement.
[0013] The in-vehicle device 100 detects the gaze of the occupants of vehicle 1 and adjusts the sound image localization position by sound field tuning.
[0014] The control unit 102 is a central processing unit that controls the entire in-vehicle device 100. The control unit 102 is also called a processor. However, the control unit 102 is The configuration is not limited to a single processor, but may be a multi-processor configuration. Furthermore, a single control unit 102 connected via a single socket may have a multi-core configuration. The control unit 102 deploys programs stored in the memory unit 104 into an executable workspace and provides functions that meet predetermined purposes by controlling peripheral devices through program execution. The control unit 102 may include dedicated LSIs (Large Scale Integration) such as DSPs (Digital Signal Processors), ASICs (Application Specific Integrated Circuits), FPGAs (Field-Programmable Gate Arrays), and other digital circuits.
[0015] The memory unit 104 is a recording medium that stores programs and data used by the control unit 102, expands the work area, and stores operation setting information, etc. The memory unit 104 can be, for example, flash memory, RAM (Random Access Memory), ROM (Read Only Memory), HDD (Hard-disk Drive), or SSD (Solid State Drive). Alternatively, the memory unit 104 can be, for example, EPROM (Erasable Programmable ROM), flash memory, USB memory, or S This includes a D (Secure Digital) memory card, etc. Furthermore, the storage unit 104 may include a portable recording medium readable by a computer, and a drive device capable of reading and writing data to the recording medium.
[0016] The input / output IF 106 is an interface for inputting and outputting data to / from devices such as the display 122. The input / output IF 106 is connected to input devices such as a keyboard, a remote controller (remote control), a pointing device, and a touch panel. Also, the input / output IF 106 is connected to output devices such as the display 122 and the speaker 124. Further, the input / output IF 106 is connected to a gaze detection sensor 126 and the like. The control unit 102 communicates with the display 122, the speaker 124, and the gaze detection sensor 126 via the input / output IF 106. The input / output IF 106 and the display 122, the speaker 124, and the gaze detection sensor 126 may be connected so as to be communicable either by wire or wirelessly (such as radio waves or infrared rays).
[0017] The communication IF 108 is a communication interface for communicating with other information processing devices and the like via a communication network or the like. The communication IF 108 can be wirelessly connected to other information processing devices and the like according to well-known wireless communication standards such as a wireless LAN and Bluetooth (registered trademark). Also, the communication IF 108 is connected to an in-vehicle network such as CAN and can communicate with other devices provided in the vehicle 1.
[0018] The display 122 is a display device that displays video and character information. The display 122 is, for example, an LCD (Liquid Crystal Display), an EL (Electroluminescence) panel , a CRT (Cathode Ray Tube) display, a PDP (Plasma Display Panel), or the like. The display 122 is, for example, a CID (Central Information Display) installed in the front central portion of the vehicle 1. The speaker 124 is an output device that outputs music, voice, and the like It is configured as follows. The speaker 124 may include a panel speaker that outputs sound by vibrating the display panel of the display 122. By using the panel speaker, it becomes easy to adjust the sound image localization position to the position of the display 122. The line-of-sight detection sensor 126 is a sensor that detects the line of sight of the occupant. The line-of-sight detection sensor 126 is, for example, a camera having a semiconductor imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). The line-of-sight detection sensor 126 is directed, for example, in the direction of the occupant's head in order to detect the line of sight of the occupant. The in-vehicle device 100 may have a plurality of line-of-sight detection sensors 126.
[0019] Note that each of the above components may be provided in plural, or some of the components may not be provided. Also, the above components may be included in the components of a device (for example, a car navigation system, etc.) mounted on the vehicle 1. [[ID=*5]]
[0020] (Operation example) FIG. 3 is a diagram showing an example of the operation flow of the in-vehicle device 100. Here, the control unit 102 of the in-vehicle device 100 detects the line of sight of the occupant boarding the vehicle 1 and performs sound field tuning of the vehicle 1 according to the detected line of sight. Sound field tuning is to adjust the output of the speaker 124 installed in the vehicle 1 so that music or voice is felt to be generated from a specific position (sound image localization position). Here, the control unit 102 of the in-vehicle device 100 adjusts the output of the speaker 124 so that the front of the vehicle 1 on which the in-vehicle device 100 is mounted or the position of the display 122 becomes the sound image localization position. Here, it is assumed that at least a driver sitting in the right front seat 310 and a passenger in the front passenger seat 320 sitting in the left front seat 320 are boarding the vehicle 1.
[0021] In S101, the control unit 102 of the in-vehicle device 100 adjusts the output of the speakers 124 installed in the vehicle 1 so that the sound image localization position is in front of the vehicle 1 on which the in-vehicle device 100 is mounted. The front of the vehicle 1 is, for example, the center of the windshield or the center of the back of the instrument panel. Occupants riding in the vehicle 1 will perceive the sound as being emitted from the sound image localization position (in front of the vehicle 1). The sound image localization position is adjusted by adjusting the speaker output. The speaker output is adjusted by, for example, adjusting the magnitude of the speaker output, the balance of speaker outputs between speakers, the delay time of the speaker output, etc. For example, the control unit 102 increases the sound output (volume) from the speakers 124 installed on the left and right sides of the dashboard of the vehicle 1. At this time, the control unit 102 may also decrease the sound output (volume) from speakers installed in other locations on the vehicle 1. As a result, the relative loudness of the sound coming from the front increases, making the driver and passenger feel as if the sound is coming from the front of vehicle 1. In other words, the sound image localization position becomes the front of vehicle 1. In addition to controlling the sound output (volume), the control unit 102 may also control the timing (delay time) of the sound emitted from each speaker of vehicle 1, either independently of the output (volume) control. For example, the control unit 102 makes the timing of the sound emitted from the speakers 124 installed on the left and right sides of the vehicle's dashboard earlier than the timing of the sound emitted from the other speakers of the vehicle. In other words, the control unit 102 delays the sound emitted from the other speakers of vehicle 1 relative to the sound emitted from the speakers 124 installed on the left and right sides of the vehicle's dashboard. As a result, the sound emitted from the speakers 124 installed on the left and right sides of the vehicle's dashboard reaches the driver and passenger first, and due to the lead sound effect, the driver and passenger feel as if the sound is coming from the front of vehicle 1. In other words, the sound image localization position will be in front of vehicle 1.
[0022] In S102, the control unit 102 determines whether the image displayed on the display 122 is video content. Video content is content in which video and sound are output together, such as television broadcasts, video content on recording media such as DVDs (Digital Versatile Disks), and streaming videos from the internet. Examples of video content include movies and music videos. Examples of content other than video content include the operation screen of the in-vehicle device 100 and map screens. When video content is displayed on the display 122 and the driver or other person is looking at the display 122, it is preferable that the sound associated with the video content is played back from the display 122. The control unit 102 determines that the image displayed on the display 122 is video content if it contains video content. The control unit 102 determines that the image is video content if it is displaying an image on the display 122 based on video content such as television broadcasts. The control unit 102 determines, for example, based on information associated with the image displayed on the display 122. The control unit 102 may perform the determination by obtaining information from the provider (source) of the image data whether or not the image displayed on the display 122 is video content. If the image displayed on the display 122 is video content (S102; YES), the process proceeds to S103. If the image displayed on the display 122 is not video content (S102; NO), the process proceeds to S101.
[0023] In S103, the control unit 102 detects the gaze direction of the driver and the passenger (occupant). The driver is the occupant who drives the vehicle 1 equipped with the on-board device 100. The driver is the occupant who sits in the driver's seat of vehicle 1. The passenger is the occupant who sits in the passenger seat of vehicle 1. The control unit 102 detects the gaze direction of the driver and the passenger using the gaze detection sensor 126. The control unit 102 detects the gaze direction of the occupants from the position of the occupants' heads, the position of their pupils, etc., using well-known gaze detection technology, image recognition technology, etc. The gaze direction of the occupants is assumed to be the direction of their gaze from the position of the occupants' heads.
[0024] In S104, the control unit 102 determines whether the driver's gaze direction detected in S103 is the direction of the display 122. The control unit 102 determines that the driver's gaze direction is the direction of the display 122 if a straight line extended from the driver's head position in the direction of the gaze passes through the position of the display screen of the display 122. If a straight line extended from the driver's head position in the direction of the gaze passes through the position of the display screen of the display 122, it is considered that the driver is watching video content. However, even if the display screen of the display 122 is located within a certain angular range (for example, within 30 degrees) with respect to a straight line extended in the direction of the gaze centered on the driver's head position, it may still be determined that the driver's gaze direction is the direction of the display 122. In other words, even if the display screen of the display 122 is located within a predetermined range of directions including the driver's gaze direction (for example, within 30 degrees), it may still be determined that the driver's gaze direction is the direction of the display 122. If the driver's gaze direction is the direction of the display 122 (S104; YES), the process proceeds to S105. If the driver's gaze direction is not towards the display 122 (S104; NO), the process proceeds to S101. If the driver is not looking at the display 122, it can be determined that there is a high probability that the driver is looking ahead to the front of the vehicle. Therefore, in S101, the sound image localization position is set to the front of the vehicle 1. In this case, the process returns to S101, so the processes from S105 onwards, i.e., the determination of the passenger's gaze direction, are not performed. Normally, drivers often look ahead while driving, and if the sound image localization position changes to match the passenger's gaze direction at this time, the driver may find it bothersome. Therefore, when the driver is looking ahead, it is preferable to set the sound image localization position to the front of the vehicle 1 regardless of the passenger's gaze direction. The position information of the display screen of the display 122 is stored in the memory unit 104 in advance.
[0025] In S105, the control unit 102 determines whether the direction of the passenger's gaze detected in S103 is towards the display 122. The control unit 102 determines that the passenger's gaze is towards the display 122 if a straight line extended from the position of the passenger's head in the direction of their gaze passes through the position of the display screen of the display 122. If a straight line extended from the position of the passenger's head in the direction of their gaze passes through the position of the display screen of the display 122, it is considered that the passenger is watching video content. However, even if the display screen of the display 122 is located within a certain angular range (for example, within 30 degrees) with respect to a straight line extended in the direction of the passenger's gaze centered on the position of the passenger's head, it may still be determined that the passenger's gaze is towards the display 122. In other words, even if the display screen of the display 122 is located within a predetermined range of directions including the passenger's gaze (for example, within 30 degrees), it may still be determined that the passenger's gaze is towards the display 122. If the passenger's gaze is directed towards the display 122 (S105; YES), the process proceeds to S107. If the passenger's gaze is not directed towards the display 122 (S105; NO), The process proceeds to S106.
[0026] In S106, the control unit 102 determines whether the direction of the passenger's gaze detected in S103 is forward. The control unit 102 determines that the passenger's gaze is forward if the direction of the passenger's gaze is parallel or nearly parallel to the direction of travel of the vehicle 1. Nearly parallel means, for example, that the angle between the passenger's gaze and the direction of travel of the vehicle 1 is within a predetermined angle (for example, within 30 degrees). In other words, if the passenger's gaze is within a predetermined range relative to the direction of travel of the vehicle 1 (for example, within 30 degrees), it is determined that the passenger's gaze is forward. Alternatively, it can also be determined that the passenger's gaze is forward if a straight line extended from the position of the passenger's head in the direction of their gaze passes through the position of the vehicle 1's front windshield or a predetermined area within the front windshield. Furthermore, if the control unit 102 cannot detect the direction of the passenger's gaze (for example, if the passenger has their eyes closed (is sleeping)), it determines that the passenger's gaze is not directed forward. Also, for example, if the passenger is operating a mobile device such as a smartphone, their gaze will be directed downward, and therefore, the passenger's gaze is not directed forward. If the passenger's gaze is directed forward (S106; YES), the process proceeds to S101. In this state, the passenger is looking forward and the driver is looking at the display 122, but in this case, the passenger's gaze direction is prioritized, and in S101, the sound image localization position is set to the front of the vehicle 1. Generally, the driver of the vehicle 1 is often looking forward and is unlikely to be looking at the display 122 for a long time, so it is preferable to prioritize the passenger's gaze direction (forward). If the passenger's gaze is not directed forward (S106; NO), the process proceeds to S107. In this state, the passenger is either sleeping or operating a mobile device such as a smartphone, and is not looking forward or at the display 122, with only the driver looking at the display 122. Therefore, in S107, the control unit 102 adjusts the output of the speaker 124 installed in the vehicle 1 so that the sound image localization position is at the position of the display 122, in accordance with the driver's gaze direction.
[0027] In S107, the control unit 102 of the in-vehicle device 100 adjusts the output of the speakers 124 installed in the vehicle 1 so that the sound image localization position becomes the position of the display 122. The occupants of the vehicle 1 perceive that the sound is being emitted from the sound image localization position (display 122). For example, the control unit 102 increases the output (volume) of the sound coming from the speakers 124 installed on the left and right sides of the dashboard of the vehicle 1, and the output (volume) of the sound coming from the speakers 124 installed in the left and right front doors of the vehicle 1. At this time, the control unit 102 may also decrease the output (volume) of the sound coming from speakers installed in other locations in the vehicle 1. As a result, the sound coming from the position of the display 122, which is at an intermediate height between the speakers 124 on the dashboard and the speakers 124 installed in the front doors, becomes relatively louder, and the driver and passenger in the front seat perceive that the sound is coming from the position of the display 122. In other words, the sound image localization position becomes the position of the display 122. The control unit 102 may also make the output (volume) of the sound emitted from the panel speaker that directly vibrates the display 122 and emits sound from the surface of the display 122 relatively louder than the output (volume) of the sound emitted from other speakers in the vehicle 1. This makes the sound coming from the location of the display 122 relatively louder, so that the driver and passengers perceive the sound as coming from the location of the display 122. In other words, the sound image localization position becomes the location of the display 122. In addition to controlling the sound output (volume), or independently of controlling the output (volume), the control unit 102 may also make the timing of the sound emitted from the panel speaker that directly vibrates the display 122 and emits sound from the surface of the display 122 earlier than the timing of the sound emitted from other speakers in the vehicle 1. In other words, the control unit 102 delays the sound emitted from other speakers in the vehicle 1 relative to the sound emitted from the panel speaker that directly vibrates the display 122 and emits sound from the surface of the display 122. This makes the sound from the panel speaker The sound is emitted and reaches the driver and front passenger first, and due to the sound-forward effect, the driver and front passenger perceive the sound as coming from the panel speaker (i.e., display 122). In other words, the sound image localization position becomes the position of display 122. The process then proceeds to S102.
[0028] This allows the sound localization position to be changed to the position of the display 122 when video content is displayed on the vehicle's display 122 and the driver and front passenger are looking at the display 122. At this time, the sound localization position is located in the direction of the driver's and front passenger's line of sight. In other words, the in-vehicle device 100 can reduce the discrepancy between the sound localization position and the direction of line of sight when the driver and front passenger are looking at the display 122. The driver and front passenger can comfortably view the video content displayed on the display 122 because the sound localization position is located at the position of the display 122 when they are interested in the video content.
[0029] S106 may be omitted. If S106 is omitted, and in S105 the passenger's line of sight is not towards the display 122 (S105; NO), the process proceeds to S101. If S106 is omitted and the driver's and passenger's line of sight is towards the display 122, the sound image localization position is adjusted to the position of the display 122. As described above, in this embodiment, when both the driver and the passenger are looking in the direction of the display 122 (or when the driver is looking at the display 122 and the passenger is not looking forward or at the display 122), the sound image localization position is adjusted to the position of the display 122. Also, when either the driver or the passenger is looking forward, the sound image localization position is adjusted forward, so that the sound image localization position can be appropriately set to match the line of sight of both the driver and the passenger. Furthermore, the sound image localization position may be adjusted forward when both the driver and the front passenger are looking forward, and adjusted to the position of the display 122 when either the driver or the front passenger is looking in the direction of the display 122. For example, in autonomous driving level 4, the driver does not need to be constantly looking forward. In such cases, both the driver and the front passenger can enjoy the content displayed on the display 122 even while driving, so it is preferable that the sound image localization position be adjusted to the position of the display 122 when either of them is looking in the direction of the display 122. In this embodiment, an example has been described in which the sound image localization position is set to the front of the vehicle 1 or to the position of the display 122, but the sound image localization position is not limited to these. For example, when the driver or passenger of the vehicle 1 is looking to the side of the vehicle 1, the control unit 102 may set the side of the vehicle 1 as the sound image localization position, or when the driver or passenger of the vehicle 1 is looking to the rear of the vehicle 1, the control unit 102 may set the rear of the vehicle 1 as the sound image localization position. In other words, when the driver or passenger of the vehicle 1 is looking in a predetermined direction, the control unit 102 can set that predetermined direction as the sound image localization position. For example, the control unit 102 adjusts the output (volume) of sound emitted from multiple speakers mounted on the vehicle 1 so that the loudness of sound coming from a predetermined direction of the vehicle 1 is relatively louder than the loudness of sound coming from other directions. Specifically, the sound output (volume) from the speaker installed at a predetermined location in vehicle 1 is made relatively louder than the sound output (volume) from the speaker installed at other locations in vehicle 1. As a result, the driver and passengers in the front seat of vehicle 1 perceive the sound as coming from a predetermined direction in vehicle 1. In other words, the sound image localization position becomes a predetermined direction in vehicle 1. In addition to controlling the sound output (volume), or independently of controlling the output (volume), the control unit 102 may also control the timing (delay time) of the sound emitted from each speaker in vehicle 1. For example, the control unit 102 makes the timing of the sound emitted from the speaker installed at a predetermined location in vehicle 1 earlier than the timing of the sound emitted from the speaker installed at other locations in vehicle 1. In other words, the control unit 102 This system delays the sound emitted from speakers located at other positions on Vehicle 1 relative to the sound emitted from speakers located at predetermined positions on Vehicle 1. As a result, the sound emitted from the speakers at the predetermined positions on Vehicle 1 reaches the driver and passengers first, and due to the lead sound effect, the driver and passengers perceive the sound as coming from a predetermined direction on the vehicle. In other words, the sound image localization position becomes the predetermined direction on Vehicle 1.
[0030] (Variation 1) In the above example of operation, when the display 122 is located in the direction of the occupant's line of sight, the sound image localization position is set to the position of the display 122, which is installed in the center front of the vehicle 1. In the modified example 1, the sound image localization position is adjusted when the occupant's line of sight is in a predetermined direction.
[0031] The operation flow of Modification 1 is the same as the operation flow of Figure 3, except for the following point. In S104, if the driver's line of sight is within a predetermined range (direction of the predetermined range, predetermined direction) (S104; YES), the process proceeds to S105; otherwise, the process proceeds to S101. Also, in S105, if the passenger's line of sight is within a predetermined range (S105; YES), the process proceeds to S107; otherwise, the process proceeds to S101. The predetermined range is, for example, that the angle between the direction of travel of vehicle 1 and the direction of the occupant's line of sight is within a predetermined angle (for example, 60 degrees). That is, if the occupant's line of sight is within a predetermined range relative to the direction of travel of vehicle 1 (for example, within 60 degrees), it is determined that the occupant's line of sight is forward. Note that the predetermined range is not limited to the range including the direction of travel of the vehicle, but may also include the rear and sides of the vehicle. The process in S106 is omitted. In S107, the control unit 102 of the in-vehicle device 100 adjusts the output of the speaker 124 installed in the vehicle 1 so that the sound image localization position becomes a predetermined position within a predetermined range of directions. Here, the predetermined position is a position within the region where a predetermined range of directions from the driver's head position and a predetermined range of directions from the passenger's head position overlap. The occupants of the vehicle 1 perceive that the sound is being emitted from the sound image localization position (predetermined position). That is, the discrepancy between the direction of the occupants' line of sight in the vehicle 1 and the position from which they perceive the sound being emitted becomes smaller.
[0032] (Modification 2) In Modification 2, the in-vehicle device 100 detects the driver's gaze direction and adjusts the sound image localization position based on the driver's gaze direction. Although the driver is used as an example in the following explanation, the device may also detect the gaze direction of other occupants, such as the passenger in the front seat, and adjust the sound image localization position based on the passenger's gaze direction.
[0033] Figure 4 shows an example of the operation flow of a modified example 2 of the in-vehicle device 100. The operation flow in Figure 4 has similarities with the operation flow in Figure 3. Here, we will mainly explain the differences.
[0034] S201 and S202 are the same as S101 and S102 in the operation flow of Figure 3, respectively.
[0035] In S203, the control unit 102 detects the driver's gaze direction. The control unit 102 detects the driver's gaze direction using the gaze detection sensor 126. Here, the control unit 102 does not need to detect the gaze directions of other occupants.
[0036] In S204, the control unit 102 determines whether the driver's gaze direction detected in S203 is towards the display 122. The control unit 102 determines that the driver's gaze direction is towards the display 122 if a straight line extended from the driver's head position in the direction of the gaze passes through the position of the display screen of the display 122. If the driver's gaze direction is towards the display 122 (S204; YES), the process proceeds to S205. If the direction is not that of display 122 (S204; NO), the process proceeds to S201.
[0037] In S205, the control unit 102 of the in-vehicle device 100 adjusts the output of the speaker 124 installed in the vehicle 1 so that the sound image localization position is at the location of the display 122. The driver perceives that the sound is being emitted from the sound image localization position (display 122). The process then proceeds to S202.
[0038] As a result, when video content is displayed on the vehicle's display 122 and the driver is looking at the display 122, the sound image localization position can be changed to the position of the display 122. At this time, the sound image localization position is located in the driver's line of sight. In other words, the in-vehicle device 100 can reduce the discrepancy between the sound image localization position and the direction of the driver's line of sight when the driver is looking at the display 122. When the driver is looking at the display 122, the sound image localization position becomes the position of the display 122, allowing the driver to comfortably view the video content displayed on the display 122.
[0039] (Variation 3) Modification 3 describes a case where multiple displays are installed in the vehicle 1. In the following description, the driver is used as an example, but as in the above embodiment, the direction of gaze of other occupants, such as the passenger in the front seat, may be detected, and the sound image localization position may be adjusted based on the passenger's gaze direction. Here, the in-vehicle device 100 detects the driver's gaze direction and adjusts the sound image localization position based on the driver's gaze direction. Here, it is assumed that the vehicle 1 is equipped with a first display and a second display as displays 122. The number of displays installed in the vehicle 1 is not limited to two, and the same applies when three or more displays 122 are installed. When the left door mirror 242, right door mirror 244, and rearview mirror 246 are electronic mirrors, each mirror may be one of the displays 122. That is, the displays 122 may include electronic mirrors. Furthermore, each of the multiple displays installed in the vehicle may be a panel speaker that directly vibrates the display and emits sound from the surface of the display 122.
[0040] Modification 3 shares similarities with Modification 2. Here, we will mainly explain the differences from the operation flow of Modification 2 (Figure 4).
[0041] In S204, the control unit 102 determines whether the driver's gaze direction detected in S203 is the direction of the display 122. Furthermore, the control unit 102 determines whether the driver's gaze direction is the direction of the first display or the direction of the second display. The control unit 102 determines that the driver's gaze direction is the direction of the first display if a straight line extended from the position of the driver's head in the direction of the gaze passes through the position of the display screen of the first display. Similarly, the control unit 102 determines that the driver's gaze direction is the direction of the second display if a straight line extended from the position of the driver's head in the direction of the gaze passes through the position of the display screen of the second display. If the driver's gaze direction is the direction of the display 122 (S204; YES), the process proceeds to S205. If the driver's gaze direction is not the direction of the display 122 (S204; NO), the process proceeds to S201.
[0042] In S205, the control unit 102 of the in-vehicle device 100 adjusts the output of the speaker 124 installed in the vehicle 1 so that the sound image localization position becomes the position of the first display if the gaze direction detected in S204 is the direction of the first display. Also, the control unit 102 of the in-vehicle device 100 adjusts the output of the speaker 124 installed in the vehicle 1 so that the sound image localization position becomes the position of the second display if the gaze direction detected in S204 is the direction of the second display. For example, when the control unit 102 sets the position of the first display as the sound image localization position, the first display The output (volume) of sound emitted from a panel speaker that directly vibrates the display and emits sound from the surface of the first display is made relatively louder than the output (volume) of sound emitted from other speakers in vehicle 1. Alternatively, independently of volume control, the timing of the sound emitted from the panel speaker that directly vibrates the first display and emits sound from the surface of the first display is made earlier than the timing of the sound emitted from other speakers in vehicle 1. As a result, the driver and others perceive the sound as coming from the panel speaker (i.e., the first display). In other words, the sound image localization position becomes the position of the first display. The control unit 102 also, when the position of the second display is set as the sound image localization position, makes the output (volume) of the sound emitted from the panel speaker that directly vibrates the second display and emits sound from the surface of the second display relatively louder than the output (volume) of the sound emitted from other speakers in the vehicle. Alternatively, independently of volume control, it makes the timing of the sound emitted from the panel speaker that directly vibrates the second display and emits sound from the surface of the second display earlier than the timing of the sound emitted from other speakers in the vehicle. As a result, the driver and others perceive the sound as coming from the panel speaker (i.e., the second display). In other words, the sound image localization position becomes the position of the second display.
[0043] This allows the sound localization position to be changed to the position of the display 122 that the driver is looking at when video content is displayed on the display 122 of vehicle 1. At this time, the sound localization position is located in the direction of the driver's line of sight. The driver can comfortably view the video content displayed on the display 122 because the sound localization position is at the position of the display 122 that they are looking at. Even if the driver changes the display 122 they are looking at, they can still perceive that the sound is being emitted from the display 122 they are looking at.
[0044] (others) In the example above, vehicle 1 had two occupants: the driver and the passenger in the front seat. However, vehicle 1 may have three or more occupants. Furthermore, while the in-vehicle device 100 detected the gaze directions of the driver and the passenger in the front seat, it may also detect the gaze directions of other occupants and adjust the sound image localization position accordingly. In this case, for example, the sound image localization position is set to the location of the region where the gaze directions of each occupant overlap, or the region where the gaze directions of many occupants (a predetermined percentage or more of occupants, or a predetermined number of occupants (e.g., two or more)) overlap. This allows the in-vehicle device 100 to make it appear as if the sound is being emitted from a position that many occupants are looking at. Alternatively, the in-vehicle device 100 may prioritize the gaze direction of a specific occupant (e.g., the driver) and adjust the sound image localization position based on that specific occupant's gaze direction. Here, the gaze direction includes directions shifted within a predetermined angle (e.g., 30 degrees) from the gaze direction.
[0045] (Effects and mechanisms of the embodiment) The in-vehicle device 100 of this embodiment detects the gaze direction of the occupants in the vehicle 1 and adjusts the sound image localization position based on the detected gaze direction. The occupants of the vehicle 1 perceive that the sound is being emitted from the sound image localization position. Furthermore, if the occupants' gaze direction is towards the display 122, the in-vehicle device 100 adjusts the position of the display 122 to the sound image localization position. This allows the in-vehicle device 100 to make it appear as if the sound is being emitted from the display 122 when the occupants of the vehicle 1 are looking at the display 122. At this time, the sound image localization position is located in the gaze direction of the driver and the front passenger. In other words, when the driver and the front passenger are looking at the display 122, the in-vehicle device 100 can reduce the discrepancy between the sound image localization position and the gaze direction. Furthermore, if the front passenger is not looking at the display 122 or the front of the vehicle 1, the in-vehicle device 100 sets the sound image localization position to the front of the vehicle 1 (the normal position). As a result, the in-vehicle device 100 can position the sound image localization point in front of the vehicle 1 when the passenger in the front seat is sleeping or operating a mobile device. This is the case when the user is not interested in the video content displayed on display 122.
[0046] The configuration of the disclosure is not limited to the embodiments described above and various modifications are possible. The embodiments described above can be implemented in combination as appropriate and as much as possible.
[0047] <Computer-readable recording medium> A program that enables a computer or other machine or device (hereinafter referred to as "computer, etc.") to perform any of the above functions can be recorded on a recording medium that the computer, etc. can read. By having the computer, etc. read and execute the program on this recording medium, it can be made to provide that function.
[0048] Here, a recording medium that can be read by a computer refers to a recording medium that stores information such as data and programs through electrical, magnetic, optical, mechanical, or chemical means and can be read by a computer. Examples of such recording media that can be removed from a computer include flexible disks, magneto-optical disks, CD-ROMs, CD-R / Ws, DVDs, Blu-ray discs, DATs, 8mm tapes, and memory cards such as flash memory. In addition, recording media that are fixed to a computer include hard disks and ROMs (read-only memory). Furthermore, SSDs (Solid State Drives) can also be used as a recording medium that can be removed from a computer. It can also be used as a recording medium fixed to a surface, etc. [Explanation of Symbols]
[0049] 1 vehicle 100 In-vehicle equipment 102 Control Unit 104 Storage section 106 Input / Output Interfaces 108 Communication IF 122 displays 124 speakers 126 Eye-tracking sensor
Claims
1. An in-vehicle device installed in a vehicle, A gaze detection sensor for detecting the direction of the gaze of an occupant riding in the vehicle, A display installed in the aforementioned vehicle, The vehicle includes a control unit that adjusts the sound image localization position of the sound output inside the vehicle, When video content is displayed on the display, the control unit adjusts the sound image localization position to the position of the display in accordance with the direction of the occupant's gaze if the direction of the occupant's gaze detected by the gaze detection sensor is the direction of the display, and adjusts the sound image localization position to a position in front of the vehicle if the direction of the occupant's gaze detected by the gaze detection sensor is not the direction of the display. In-vehicle device.
2. When an occupant is seated in the driver's seat and the passenger seat of the vehicle, the gaze detection sensor detects the gaze direction of both the driver and the passenger, The control unit adjusts the sound image localization position to the position of the display if both the driver's gaze direction and the passenger's gaze direction detected by the gaze detection sensor are in the direction of the display, and adjusts the sound image localization position to a position in front of the vehicle if neither the driver's gaze direction nor the passenger's gaze direction is in the direction of the display. The in-vehicle device according to claim 1.
3. The control unit adjusts a position within a predetermined range of directions, including the occupant's line of sight, as the sound image localization position. The in-vehicle device according to claim 1.
4. The display is a panel speaker that directly vibrates the surface of the display and outputs sound from that surface. The control unit adjusts the sound image localization position by adjusting the sound output from the panel speaker. The in-vehicle device according to claim 1.
5. The aforementioned vehicle is equipped with multiple displays, When the gaze detection sensor detects that the direction of the occupant's gaze is the direction of the display, the control unit adjusts the position of the display among the plurality of displays that is in the direction of the occupant's gaze to become the sound image localization position. The in-vehicle device according to claim 1.
6. The in-vehicle equipment installed in the vehicle, The direction of the gaze of the occupants riding in the vehicle is detected, When video content is displayed on a display installed in the vehicle, if the detected direction of the occupant's gaze is towards the display, the sound image localization position of the sound output inside the vehicle is adjusted to the position of the display in accordance with the direction of the occupant's gaze; if the detected direction of the occupant's gaze is not towards the display, the sound image localization position is adjusted to a position in front of the vehicle. Sound image localization position adjustment method.