Control device, control system, and control method

Abstract

To comprehensively control (i) the position of a sound image of a sound output from a certain sound output device worn by a user, and (ii) the position of a sound image of a sound output from another sound output device.SOLUTION: A control device (10) includes an information acquisition unit (11) that acquires, from a detection device (20), position information indicating the head position and orientation information indicating the head orientation for each of a plurality of users staying within a specified space, a sound image position determination unit (13) that determines the position of a sound image of an output sound from headphones (31) worn by at least one of the plurality of users and the position of a sound image of an output sound from a speaker (32) arranged within the specified space, and an output control unit (14) that controls the output of each of the headphones (31) and the speaker (32) on the basis of the head position and head orientation of each of the plurality of users such that a sound image of the output sound is formed at the position determined by the sound image position determination unit (13).SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a control device, a control system, and a control method. [Background technology]

[0002] In recent years, various techniques for controlling the sound image of the output sound from an audio output device have been developed, such as the techniques described in Patent Documents 1 and 2. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 5-183998 [Patent Document 2] Japanese Patent Application Publication No. 5-336599 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the techniques described in Patent Documents 1 and 2 are techniques for controlling the position of a sound image of sound output from a single headphone. In other words, each of the techniques described in Patent Documents 1 and 2 merely discloses a method for controlling the position of a sound image in a single audio output device.

[0005] One aspect of the present disclosure aims to comprehensively control (i) the position of the sound image of output sound from a certain audio output device worn by a user, and (ii) the position of the sound image of output sound from another audio output device. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, a control device according to a first aspect of the present invention includes an information acquisition unit that acquires, from a detection device, position information indicating the head position and orientation information indicating the head orientation for each of a plurality of users staying within a specified space; a sound image position determination unit that determines the position of a sound image of an output sound of a first output device worn by at least one of the plurality of users and the position of a sound image of an output sound of a second output device placed within the specified space; and an output control unit that controls the output of each of the first output device and the second output device based on the head position and head orientation of each of the plurality of users so that the sound image of the output sound is formed at the position determined by the sound image position determination unit.

[0007] According to the above configuration, it is possible to comprehensively control the position of the sound image of the output sound of the first output device worn by the user and the position of the sound image of the output sound of the second output device placed in a predetermined space.

[0008] A control device according to aspect 2 of the present invention may be configured such that, in aspect 1, the sound image position determination unit determines the positions of the sound images of the output sounds of the first output device and the second output device so that the positions of the sound images of the output sounds of the first output device and the second output device are the same.

[0009] According to the above configuration, a user wearing the first output device and a user listening to the output sound from the second output device can feel as if the sound is coming from the same position.

[0010] The control device according to aspect 3 of the present invention is, in accordance with aspect 2, further comprising an audio acquisition unit that acquires the audio of an utterance made by one of the plurality of users from a microphone disposed within the specified space or worn by the user, and when the audio acquisition unit acquires the audio of the utterance, the sound image position determination unit determines the position of the sound image based on the head position and head orientation of the user who made the utterance so that the position of the sound image of the output sound of each of the first output device and the second output device is near the user who made the utterance, and the output control unit may output the audio of the utterance acquired by the audio acquisition unit from each of the first output device and the second output device.

[0011] According to the above configuration, the sound image position determination unit determines the sound image positions of the output sounds of the first output device and the second output device to be near the speaker. Then, the output control unit can cause the first output device and the second output device to output the sound of the utterance acquired by the sound acquisition unit. This makes it easier to hear the speech of each user within a predetermined space.

[0012] A fourth aspect of the present invention provides the control device of the first aspect, wherein the sound image position determination unit determines the positions of the sound images of the output sounds of the first output device and the second output device individually.

[0013] According to the above configuration, the sound image positions of the output sounds from the first output device and the second output device can be determined individually, thereby making each user feel as if they are hearing the sound from a position appropriate for them.

[0014] A control device according to aspect 5 of the present invention may be any one of aspects 1 to 4, wherein when causing the first output device and the second output device to output a specific sound, the sound image position determination unit determines the position of a sound image of each of the first output device and the second output device, and the output control unit controls the output of each of the first output device and the second output device so that a sound image of the specific sound is formed at the position determined by the sound image position determination unit.

[0015] According to the above configuration, when a sound image of a specific sound is output from the first output device and the second output device, the position of the sound image of the output sound can be controlled.

[0016] A control device according to aspect 6 of the present invention is any one of aspects 1 to 5, wherein the detection device includes a status sensor for measuring or acquiring status information for identifying the status of each of the multiple users, the information acquisition unit acquires the status information of each user, and the control device may include an audio determination unit that determines the output sounds of the first output device and the second output device based on the status information of each user.

[0017] According to the above configuration, it is possible to output an appropriate sound according to the state of the user.

[0018] In order to solve the above-mentioned problems, a control system according to a seventh aspect of the present invention includes the control device according to any one of the first to sixth aspects, the detection device, the first output device, and the second output device. According to the above-mentioned configuration, the same effects as those of the control device according to any one of the first to sixth aspects can be achieved.

[0019] In order to solve the above-mentioned problems, a control method according to an eighth aspect of the present invention includes: an information acquisition step of acquiring, from a detection device, position information indicating a head position and orientation information indicating a head orientation for each of a plurality of users staying in a predetermined space; a sound image position determination step of determining a position of a sound image of an output sound of a first output device worn by at least one of the plurality of users and a position of a sound image of an output sound of a second output device arranged in the predetermined space; and an output control step of controlling the output of each of the first output device and the second output device based on the head position and head orientation of each of the plurality of users so that the sound image of the output sound is formed at the position determined in the sound image position determination step. This configuration achieves the same effects as the control device according to the first aspect. [Effects of the Invention]

[0020] According to one aspect of the present disclosure, it is possible to comprehensively control (i) the position of the sound image of the output sound from a certain audio output device worn by a user, and (ii) the position of the sound image of the output sound from another audio output device. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 2 is a diagram illustrating an outline of the operation of the control system according to the first embodiment. [Figure 2] 1 is a block diagram showing an example of a configuration of a main part of a control system according to a first embodiment. [Figure 3] 1 is a diagram showing an example of an outline of an interior space of a vehicle according to a first embodiment, and an example of the positional relationship between a detection device, headphones 31, and a speaker. [Figure 4] 1 is a diagram showing the relationship between the position and direction of the head of an occupant and the position of a sound image on the spatial coordinate system inside the vehicle in the first embodiment. FIG. [Figure 5] 4 is a flowchart showing an example of a flow of processing executed by the control device according to the first embodiment. [Figure 6] FIG. 10 is a diagram illustrating an outline of the operation of the control system according to the second embodiment. [Figure 7] 10 is a diagram showing the relationship between the position and direction of the head of an occupant and the position of a sound image on the spatial coordinate system inside the vehicle in the second embodiment. FIG. [Figure 8] 11 is a flowchart showing an example of a flow of processing executed by a control device according to a third embodiment. [Figure 9] FIG. 10 is a diagram showing an outline of the operation of a control system according to a third embodiment. [Figure 10] FIG. 10 is a block diagram showing an example of a configuration of a main part of a control system according to a third embodiment. [Figure 11] FIG. 11 is a diagram showing the relationship between the position and direction of the head of an occupant and the position of a sound image on the spatial coordinate system inside the vehicle in the third embodiment. [Figure 12] 11 is a flowchart showing an example of a flow of processing executed by a control device according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0022] A control system according to the present disclosure is a system for comprehensively controlling a plurality of audio output devices present in a predetermined space. The control system controls the output of each audio output device, thereby controlling the position of a sound image perceived by a person in the predetermined space when listening to the output sound.

[0023] In this disclosure, the term "predetermined space" refers to a space large enough for humans to stay in, such as the interior space of a moving object or a building. Examples of moving objects include automobiles, airplanes, and trains. Examples of buildings include movie theaters, commercial facilities, and homes.

[0024] Furthermore, in this disclosure, "sound image" refers to the sound source perceived by humans. Therefore, in this disclosure, the "position of the sound image" of a certain sound refers to the position where a human perceives "the source of this sound." Note that the position of the sound image does not necessarily coincide with the position of the actual sound source. Therefore, the position of the sound image of a certain sound may be the position of the virtual sound source of the certain sound.

[0025] In addition, in this disclosure, the "audio output device" to be controlled by the control system is classified into a first output device and a second output device. The first output device is, for example, an audio output device worn by a user. The first output device can be realized, for example, by headphones or earphones. In contrast, the second output device is an audio output device placed in a predetermined space. The second output device can be realized, for example, by in-car speakers or speakers integrated with a television.

[0026] A control system according to the present disclosure controls a plurality of first output devices. Alternatively, a control system according to the present disclosure controls one or more first output devices and one or more second output devices. Various embodiments of the control system according to the present disclosure will be specifically described below.

[0027] In the following embodiments, the "predetermined space" refers to the interior space of a vehicle. That is, the users staying in the predetermined space are the vehicle occupants. In the following embodiments, the first output device is a headphone worn by the occupant, and the second output device is a speaker arranged in the interior space of the vehicle. In the following embodiments, the vehicle is assumed to have a total of four occupants, including a driver. The three occupants other than the driver are each wearing headphones. However, the configuration of the present disclosure is not limited to these specific examples.

[0028] In addition, in the following embodiments, it is assumed that some audio data is output from the headphones and the speaker, respectively. However, the configuration of the present disclosure is not limited to this. For example, even when the same audio data is output from the headphones and the speaker, the configuration of the embodiments described below is applicable. Also, for example, the control system may be configured to control the position of the sound image as described above at the timing when audio is output from the headphones and the speaker.

[0029] [Embodiment 1] Overview The control system of this embodiment is characterized in that it controls multiple audio output devices (i.e., multiple headphones, or one or more headphones and one or more speakers) respectively, thereby forming sound images of the output sounds from the multiple audio output devices at the same position within the vehicle.

[0030] Fig. 1 is a diagram showing an outline of the operation of a control system 100 according to this embodiment. In the example of Fig. 1, four occupants A to D, who are users of the control system 100, are sitting in seats provided in the interior space of a vehicle 1. Occupant B is wearing headphones 31b, occupant C is wearing headphones 31c, and occupant D is wearing headphones 31d. Speakers 32a and 32e are also arranged near occupant A.

[0031] The control system 100 controls the positions of the headphones 31b to 31d and the sound images of the output sounds from the speakers 32a and 32e, which are positioned as shown in Fig. 1. The control system 100 controls the headphones worn by each passenger and / or the speakers placed near each passenger, based on the position and orientation of the head of each passenger.

[0032] 1, the control system 100 controls the headphones 31b to 31d and the speakers 32a and 32e so that the sound images of the output sounds from the headphones 31b to 31d and the speakers 32a and 32e are all formed within the range R1 shown in the figure. This allows the occupants A to D, who are listening to the output sounds from different audio output devices, to feel as if they are all hearing the sounds from the same position.

[0033] ≪Main part configuration≫ Next, a specific configuration of the control system 100 will be described. Fig. 2 is a block diagram showing an example of the configuration of the main parts of the control system 100 according to this embodiment. In the example of Fig. 2, the control system 100 includes a control device 10, a detection device 20, a storage device 30, headphones 31, and a speaker 32.

[0034] Note that headphones 31 is a general term for one or more headphones, and speakers 32 is a general term for one or more speakers. More specifically, in this embodiment, headphones 31 refer to headphones 31b, 31c, and 31d shown in FIG. 1. Furthermore, the storage device 30 is not an essential component of the control system 100. The control device 10 is communicatively connected to the detection device 20, the storage device 30, the headphones 31, and the speakers 32, respectively, via wire or wireless.

[0035] 2, the control device 10, the detection device 20, the storage device 30, and the speaker 32 are arranged in the interior space of the vehicle 1. A user staying in the interior space of the vehicle 1 is wearing headphones 31. The structure of the vehicle 1 and the positional relationship between the detection device 20, the headphones 31, and the speaker 32 will be described in detail later.

[0036] (Detection device 20) The detection device 20 is a group of one or more devices arranged in the interior space of the vehicle 1. Each device included in the detection device 20 detects at least one of information indicating the position of the user's head (position information) and information indicating the orientation of the user's head (orientation information), and transmits it to the control device 10. The detection device 20 includes one or more devices that can detect position information and one or more devices that can detect orientation information. This allows the detection device 20 as a group of devices to detect position information and orientation information and transmit it to the control device 10.

[0037] The format of the "position information" is not particularly limited as long as it is information that can identify the position of the user's head. Similarly, the format of the "orientation information" is not particularly limited as long as it is information that can identify the orientation of the user's head. Furthermore, the specific configurations of the various devices included in the detection device 20 are not limited to the example in FIG. 2.

[0038] In the example of FIG. 2, the detection device 20 includes a camera 21 and a gyro sensor 22. The camera 21 captures, for example, an image of the upper body of the user and transmits the captured image to the control device 10. The captured image may be a still image or a video. As will be described in detail later, the control device 10 that receives the captured image can identify the position and orientation of the user's head in the interior space of the vehicle 1 based on the captured image and the structure of the interior space. Therefore, the image captured by the camera 21 is position information and orientation information.

[0039] The gyro sensor 22 is a sensor built into the headphones 31. When worn by a user together with the headphones 31, the gyro sensor 22 can detect a value indicating the tilt of the user's head (i.e., the orientation of the head). Therefore, the measurement information of the gyro sensor 22 is orientation information.

[0040] (Vehicle 1 structure and device positioning) 3 is a diagram showing an example of an outline of the interior space of the vehicle 1 according to this embodiment, and an example of the positional relationship between the detection device 20, headphones 31, and speaker 32. In FIG. 3, the traveling direction of the vehicle 1 is defined as "forward." The front, rear, left, and right directions are as shown in the figure. Furthermore, the direction toward the ceiling of the vehicle 1 is defined as the upward direction, and the direction toward the floor of the vehicle 1 is defined as the downward direction.

[0041] In the example of Fig. 3, four seats ST1 to ST4 are provided in the interior space R of the vehicle 1. Four passengers, namely, passenger A, passenger B, passenger C, and passenger D, are seated in the seats as shown in Fig. 3.

[0042] Camera 21 is provided in a position where it can capture the upper body (or from the neck up) of a passenger sitting in seats ST1 to ST4. For example, as shown in FIG. 3, camera 21 is provided at an angle from the center of the front of vehicle 1 so as to capture the rear. Note that multiple cameras 21 may be provided on the ceiling or wall of vehicle 1. Furthermore, camera 21 may be a 360° camera and may be provided in a location in the center of the front-rear and left-right of vehicle 1.

[0043] In the example of FIG. 3, similar to the example of FIG. 2, occupant B wears headphones 31b, occupant C wears headphones 31c, and occupant D wears headphones 31d. Each of the headphones 31b to 31d is an example of the headphones 31 in FIG. 2. As described above, each of the headphones may be equipped with a gyro sensor 22 that is part of the detection device 20. In the example of FIG. 3, the gyro sensor 22 is located at the position of the headphones 31b to 31d.

[0044] In the example of FIG. 3, speakers 32a to 32h are arranged in the interior space of the vehicle 1. Speakers 32a to 32h are each an example of the speaker 32 in FIG. 2. Speakers 32a to 32h are arranged, for example, as follows: Speaker 32a is arranged in the front right door of the vehicle 1. Speaker 32b is arranged in the front left door of the vehicle 1. Speaker 32c is arranged in the rear right door of the vehicle 1. Speaker 32d is arranged in the rear left door of the vehicle 1. Speaker 32e is arranged on an instrument panel or the like in the front of the vehicle 1. Speaker 32f is arranged in the center of the ceiling inside the vehicle 1. Speaker 32g is arranged behind seat ST3. Speaker 32h is arranged behind seat ST4. The number and arrangement positions of the speakers 32 can be changed as appropriate.

[0045] (Headphones 31 and Speakers 32) The following description will be given again with reference to FIG. 2. The headphones 31 are configured to be capable of adjusting the position of the sound image of the output sound. The headphones 31 output sound under the control of the control device 10. For example, the headphones 31 may be a device equipped with multiple (e.g., nine) speakers inside. In this case, the position of the sound image of the output sound from the headphones 31 can be adjusted by adjusting the volume and output timing of each speaker included in the headphones 31.

[0046] The speaker 32 is a speaker having a configuration that allows adjustment of the sound image position of the output sound. The speaker 32 outputs sound under the control of the control device 10. The speaker 32 forms a sound image either alone or by a combination of multiple speakers 32.

[0047] (Storage device 30) The storage device 30 stores various data necessary for the operation of the control device 10. The storage device 30 may also store sound source data to be output to the headphones 31 and / or the speaker 32. More specifically, the sound source data may be data such as music, a navigation voice for providing guidance or information, or a warning sound such as a buzzer. The storage device 30 may be an external device to the control device 10 as shown in FIG. 2, or may be a storage unit built into the control device 10.

[0048] (Control device 10) The control device 10 controls the headphones 31 and the speaker 32 based on the information acquired from the detection device 20. The control device 10 includes an information acquisition unit 11, a coordinate processing unit 12, a sound image position determination unit 13, and an output control unit 14.

[0049] The information acquisition unit 11 receives position information and orientation information from various devices of the detection device 20. The information acquisition unit 11 outputs the received position information and orientation information to the coordinate processing unit 12.

[0050] The coordinate processing unit 12 identifies the position and orientation of each occupant's head as a numerical value or vector on three-dimensional coordinates representing the interior space of the vehicle 1. The coordinate processing unit 12 includes a position identification unit 121 and a direction identification unit 122. Hereinafter, the "three-dimensional coordinates representing the interior space of the vehicle 1" will be simply referred to as "in-vehicle space coordinates." Note that, while a "vector" generally has a magnitude and a direction, in the present disclosure, the magnitude of the vector indicating the head orientation may be any as long as the directional component can be identified.

[0051] The position identifying unit 121 identifies the head position of each occupant in the vehicle interior space coordinates from the position information. The position identifying unit 121 plots the identified head position in the vehicle interior space coordinates. Note that the head position may be indicated by the coordinate of a single point or by a range of coordinate values.

[0052] The direction identifying unit 122 identifies the head direction of each occupant in the vehicle interior space coordinate system from the orientation information. For example, the direction identifying unit 122 plots the head direction of each occupant on the vehicle interior space coordinate system as a vector in the vehicle interior space coordinate system. The method by which the position identifying unit 121 identifies the head position and the method by the direction identifying unit 122 identify the head direction may be determined appropriately depending on the type and content of information input to the coordinate processing unit 12.

[0053] For example, suppose an image captured by the camera 21 is input to the coordinate processing unit 12. In this case, the position identification unit 121 identifies the position of the occupant's head in the interior space of the vehicle 1 from the positions and shapes of the windows, doors, and seats captured in the captured image and the position and shape of the user's head captured in the same image. Similarly, the direction identification unit 122 identifies the orientation of the occupant's head in the interior space of the vehicle 1 from the positions and shapes of the windows, doors, and seats captured in the captured image and the position and shape of the user's head captured in the same image.

[0054] For example, assume that measurement information from the gyro sensor 22 is input to the coordinate processing unit 12 in addition to the captured image. In this case, the direction identification unit 122 may identify the head orientation of each occupant from the measurement information from the gyro sensor 22. Alternatively, the direction identification unit 122 may identify the head orientation of each occupant by taking into account both the measurement information from the gyro sensor 22 and the captured image from the camera 21.

[0055] The head position and head direction of each occupant may be identified in random order or in parallel. Furthermore, the position identifying unit 121 and the direction identifying unit 122 may refer to each other's processing results or information in the middle of processing in order to identify which occupant the information identified by each unit relates to.

[0056] The coordinate processing unit 12 outputs the identified head position coordinates and head direction vectors of each occupant to the output control unit 14. The coordinate processing unit 12 may also output the identified head position coordinates and head direction vectors of each occupant to the sound image position determination unit 13. The coordinate processing unit 12 may plot the head position coordinates of each occupant identified by the position identification unit 121 on the vehicle interior space coordinates. The coordinate processing unit 12 may also plot the head direction vector of each occupant identified by the direction identification unit 122 on the vehicle interior space coordinates. The coordinate processing unit 12 may then output information on the vehicle interior space coordinates on which the head positions and head directions of each occupant are plotted to the output control unit 14 (and the sound image position determination unit 13).

[0057] The coordinate processing unit 12 may also identify the number of occupants from the number of coordinates (i.e., the position coordinates of the occupants' heads) identified by the position identification unit 121. The coordinate processing unit 12 may also output information indicating the identified number of people to the sound image position determination unit 13 and the output control unit 14.

[0058] The sound image position determination unit 13 determines the position coordinates of the sound image to be perceived by each occupant on the vehicle interior spatial coordinate system. The "sound image to be perceived by each occupant" refers to the sound image that each occupant of the vehicle 1 (four occupants A to D in this embodiment) feels when hearing sound from their corresponding audio output device (headphones 31 or speakers 32). The sound image position determination unit 13 may determine the position of the sound image according to the position and orientation of each occupant's head input from the coordinate processing unit 12. The sound image position determination unit 13 outputs the determined position coordinates of the sound image to the output control unit 14. The position coordinates of each sound image may be defined as a single point or may be indicated by a range of coordinate values.

[0059] The sound image position determination unit 13 according to this embodiment determines one position coordinate (or one range) of the sound image of the headphones 31 and the speaker 32. That is, the sound image position determination unit 13 determines the position of the sound image so that the sound image positions of the output sounds of the headphones 31 and the speaker 32 are the same.

[0060] The output control unit 14 determines the headphones 31 and / or speakers 32 for which the position of a sound image is to be controlled, and controls the output of the determined headphones 31 and / or speakers 32. When the position of a sound image is input from the sound image position determination unit 13, the output control unit 14 first determines the headphones 31 and / or speakers 32 that are to form a sound image at that position. In other words, the output control unit 14 determines the headphones 31 and / or speakers 32 to be controlled. Note that the output control unit 14 may determine only some of the headphones 31 and / or speakers 32 to be controlled, or may determine all of the headphones 31 and / or speakers 32 to be controlled.

[0061] Next, the output control unit 14 controls each of the headphones 31 and / or speakers 32 to be controlled so that a sound image of the output sound from the headphones 31 and / or speakers 32 is formed at a specified position. More specifically, the output control unit 14 controls each of the headphones 31 and / or speakers 32 based on the relative positional relationship between the position and orientation of each occupant's head and the position of the sound image. In this way, sound images of the output sound from the headphones 31 and / or speakers 32 that are present at different positions and in different orientations on the in-vehicle spatial coordinate system can be formed at the respective target positions.

[0062] Note that which headphones 31 each occupant is wearing and which speaker 32 each occupant can hear the sound from may be identified by the coordinate processing unit 12 from the position information and orientation information of the detection device 20. For example, the coordinate processing unit 12 may compare the image captured by the camera 21 with the measurement information of the gyro sensor 22 to identify which headphone 31 or speaker 32 each occupant can hear the sound output from.

[0063] In this embodiment, occupants B to D wear headphones 31b to 31d, respectively. Therefore, the sound images heard by occupants B to D are the sounds output from headphones 31b to 31d, respectively. On the other hand, occupant A does not wear headphones 31. Therefore, the sound images perceived by occupant A are the sounds output from one or more of speakers 32a to 32h.

[0064] Note that the output control unit 14 only needs to execute at least the above-described output control on the headphones 31 and / or the speaker 32, and does not have to be involved in determining the audio data itself to be output from the headphones 31 and / or the speaker 32. Specifically, the audio to be output from the headphones 31 and / or the speaker 32 may be determined by the output control unit 14, or may be determined by an external device such as a music player (not shown) connected to the headphones 31 and / or the speaker 32.

[0065] <Examples of in-car spatial coordinates> FIG. 4 is a diagram showing the relationship between the position and orientation of the head of an occupant and the position of a sound image in the present embodiment on the vehicle interior space coordinate system. The definitions of front, rear, left, right, top, and bottom of the vehicle 1 in FIG. 4 are the same as those in FIG. 3. In addition, in the vehicle interior space coordinate system in FIG. 4, as shown, the front left point of the vehicle 1 is set as the origin, the axis extending from the left to the right is the x-axis, and the axis extending from the front to the rear, which is perpendicular to the x-axis, is the y-axis. In addition, in FIG. 4, the axis extending from the bottom (floor side) to the top (ceiling side) of the vehicle 1, which is perpendicular to the x-axis and y-axis, is the z-axis.

[0066] In the example of FIG. 4, the head positions of occupants A to D, the directions of the heads of occupants A to D, and the positions of sound images are schematically shown in the vehicle interior space coordinate system.

[0067] The output control unit 14 controls the output of headphones 31b in accordance with the relative positional relationship between the position of occupant B's head (i.e., the position of headphones 31b) and range R1, and the orientation of occupant B's head (i.e., the orientation of headphones 31b), so that a sound image of the output sound of headphones 31b (i.e., the sound heard by occupant B) is formed in range R1. Similarly, the output control unit 14 controls the output of headphones 31c in accordance with the relative positional relationship between the position of occupant C's head and range R1, and the orientation of occupant C's head, so that a sound image of the output sound of headphones 31c is formed in range R1. The same applies to headphones 31d.

[0068] The output control unit 14 also controls the output of each speaker 32 according to the relative positional relationship between the position of each speaker 32 and the position of range R1, and the orientation of each speaker 32. Note that the output control unit 14 may cause multiple speakers 32 to cooperate with each other to form a sound image at the position of range R1.

[0069] <Processing flow> Fig. 5 is a flowchart showing an example of the flow of processing executed by the control device 10 according to this embodiment. The series of processing shown in Fig. 5 may be started at any timing. For example, the series of processing shown in Fig. 5 may be started when the detection device 20 detects the position information and orientation information of any occupant.

[0070] The information acquisition unit 11 of the control device 10 acquires position information and orientation information from the detection device 20 (S1, information acquisition step). In this embodiment, the information acquisition unit 11 acquires an image (position information and orientation information) captured by the camera 21 and measurement information (orientation information) from the gyro sensor 22. The information acquisition unit 11 outputs the acquired position information and orientation information to the coordinate processing unit 12.

[0071] The coordinate processing unit 12 identifies the head position and head direction of each occupant from the position information and direction information (S2). The coordinate processing unit 12 outputs the identified head position and head direction to the output control unit 14 (and the sound image position determination unit 13). The coordinate processing unit 12 may plot the identified coordinates of the head position and the vector of the head direction on the vehicle interior space coordinate system in a format such as that shown in Fig. 4 (S3).

[0072] The sound image position determination unit 13 determines the position of the sound image of the output sound from the headphones 31 and the speaker 32 at a certain coordinate or a certain coordinate range on the vehicle interior spatial coordinate system. That is, the sound image position determination unit 13 determines the position of the sound image so that the sound images of the output sounds from the headphones 31 and the speaker 32 are at the same position (S4, sound image position determination step). The sound image position determination unit 13 outputs the determined position coordinate or coordinate range of the sound image to the output control unit 14.

[0073] The output control unit 14 determines the audio output devices to be controlled (S5). In the example of the present embodiment, the output control unit 14 determines, for example, three headphones 31, 31b to 31d, and two speakers 32, 32a and 32e, as the audio output devices to be controlled.

[0074] The output control unit 14 controls the outputs of the headphones 31 and the speakers 32 determined to be controlled based on the position and orientation of the head of each occupant. As a result, the output control unit 14 causes the headphones 31 and the speakers 32 to form a sound image at the position determined by the sound image position determination unit 13 (S6, output control step).

[0075] [Embodiment 2] Other embodiments of the present disclosure will be described below. For ease of explanation, the same reference numerals will be used to designate components having the same functions as those described in the previous embodiment, and the description thereof will not be repeated. This also applies to the following embodiments.

[0076] Overview The control system 200 according to this embodiment is characterized in that it controls a plurality of audio output devices (i.e., a plurality of headphones, or one or more headphones and one or more speakers) to individually form sound images of the output sounds from the plurality of audio output devices.

[0077] Fig. 6 is a diagram showing an outline of the operation of the control system 200 according to this embodiment. Fig. 6 corresponds to Fig. 1 in the first embodiment. Therefore, the same points as Fig. 1 will not be described repeatedly.

[0078] 6, the control system 200 controls each of these audio output devices so that the sound image of the output sound from headphone 31b is formed in range A, the sound image of the output sound from headphone 31c is formed in range B, the sound image of the output sound from headphone 31d is formed in range C, and the sound images of the output sounds from speakers 32a and 32e are formed in range D. In other words, the control system 200 controls each audio output device so that the sound image of the output sound from each headphone 31 and the sound image formed by one or more speakers 32 are formed individually. This makes it possible for occupants A to D to feel as if they are hearing sounds from positions appropriate to each individual.

[0079] <Examples of in-car spatial coordinates> 7 is a diagram showing the relationship between the position and orientation of the head of an occupant and the position of a sound image in the present embodiment on the spatial coordinate system inside the vehicle. FIG. 7 corresponds to FIG. 4 in the first embodiment. Therefore, the same points as in FIG. 4 will not be described again. In the control system 200 according to this embodiment, the sound image position determination unit 13 determines the position of the sound image formed by the output sound of each headphone 31 and speaker 32 individually for each occupant (i.e., for each headphone 31 and for each group of speakers 32), as shown in ranges A to D in FIG. 7.

[0080] <Processing flow> Fig. 8 is a flowchart showing an example of the flow of processing executed by the control device 10 according to this embodiment. In the flowchart of Fig. 8, the processing of S11 to S13 is the same as the processing of S1 to S3 in Fig. 5, respectively. However, the coordinate processing unit 12 according to this embodiment also outputs the head position and head orientation of each occupant to the sound image position determination unit 13.

[0081] The sound image position determination unit 13 determines the position of the sound image to be perceived by each occupant individually according to the head position and head orientation of each occupant (S14, sound image position determination step). The sound image position determination unit 13 outputs information indicating the determined position of the sound image (range A to range D in the example of FIG. 7) to the output control unit 14.

[0082] 5, the output control unit 14 determines the audio output devices to be controlled (S15). Then, the output control unit 14 controls the outputs of the headphones 31 and the speakers 32 determined to be the control targets based on the position and orientation of the head of each occupant. As a result, the output control unit 14 causes the headphones 31 and the speakers 32 to form sound images at the positions determined by the sound image position determination unit 13 (S16, output control step).

[0083] [Embodiment 3] Overview The control system 300 according to this embodiment outputs the speech of the occupant from a plurality of audio output devices (i.e., a plurality of headphones, or one or more headphones 31 and one or more speakers 32). The control system 300 is characterized in that it forms a sound image of the output sound (i.e., the speech) from the plurality of audio output devices near the speaker.

[0084] Fig. 9 is a diagram showing an outline of the operation of the control system 300 according to this embodiment. Fig. 9 corresponds to Fig. 1 in the first embodiment. Therefore, the same points as Fig. 1 will not be described repeatedly.

[0085] 9, the control system 300 includes microphones 23a to 23d as the detection device 20. These microphones are arranged on the ceiling of each seat of the vehicle 1, for example, as shown in FIG. 9, and capture the speech of the occupant sitting in each seat. In the example of FIG. 9, the speaker is occupant B, as shown in the figure. Microphone 23b captures the speech of occupant B.

[0086] The control system 300 reproduces the speech captured by the microphone 23b through the headphones 31b to 31d and the speakers 32a and 32e. At this time, the control system 300 controls the headphones 31b to 31d and the speakers 32a and 32e so that sound images of the output sounds from the headphones 31b to 31d and the speakers 32a and 32e are all formed in a range R2 near the speaker, occupant B. This makes it possible for occupants A to D, including the speaker, occupant B, to easily hear the speech of occupant B. In particular, the speech of occupant B can be heard by occupants B to D who are wearing headphones 31 as output sounds from the headphones 31, making it easier for occupants B to D to hear the speech.

[0087] ≪Main part configuration≫ 10 is a diagram showing the configuration of the main parts of a control system 300 according to this embodiment. The control system 300 differs from the control systems 100 and 200 in that it includes a microphone 23.

[0088] The microphone 23 acquires the speech of each passenger in real time. The microphone 23 transmits the acquired speech to the information acquisition unit 11 as needed. When acquiring speech from the microphone 23, the information acquisition unit (speech acquisition unit) 11 outputs identification information of the microphone from which the speech was acquired to the sound image position determination unit 13. In addition, the information acquisition unit 11 outputs the speech acquired from the microphone 23 to the output control unit 14.

[0089] The sound image position determination unit 13 determines the position of the sound image based on the identification information of the microphone 23 and the position of the occupant's head input from the coordinate processing unit 12. Specifically, the sound image position determination unit 13 determines the position of the sound image to be a position near the occupant who uttered the speech voice acquired by the microphone 23 (i.e., the speaker).

[0090] The output control unit 14 causes the audio output device determined as the control target to output the speech sound. At this time, the output control unit 14 causes the audio output device to form a sound image at the position determined by the sound image position determination unit 13 (i.e., a position near the speaker).

[0091] <Examples of in-car spatial coordinates> Fig. 11 is a diagram showing the relationship between the position of the occupant's head and the position of a sound image in the present embodiment on the vehicle interior spatial coordinate system. Fig. 11 corresponds to Fig. 4 in the first embodiment. Therefore, the same points as Fig. 4 will not be described repeatedly. The output control unit 14 controls each headphone 31 so that the position of the sound image of the speech sound output from the headphones 31b to 31d is within a position in range R2. The output control unit 14 also controls each speaker 32 so that the position of the sound image of the speech sound output from each speaker 32 is within a position in range R2.

[0092] <Processing flow> 12 is a flowchart showing an example of the flow of processing executed by the control device 10 according to this embodiment. In the flowchart of FIG. 12, the processing of S21 to S23 is the same as the processing of S1 to S3 in FIG.

[0093] The sound image position determination unit 13 determines whether the occupant has spoken (S24). For example, when the sound image position determination unit 13 acquires identification information of the microphone 23 from the information acquisition unit 11, the sound image position determination unit 13 may determine that the occupant has spoken.

[0094] If the sound image position determination unit 13 determines that the occupant is not speaking (NO in S24), the sound image position determination unit 13 does not perform the subsequent processing, and the output control unit 14 does not need to execute any particular output control. Alternatively, the sound image position determination unit 13 may determine the sound image positions of the headphones 31 and the speakers 32 as described in the first or second embodiment. In this case, the output control unit 14 controls the headphones 31 and the speakers 32 so that sound images are formed at the determined positions.

[0095] On the other hand, if the sound image position determination unit 13 determines that the occupant is speaking (YES in S24), the sound image position determination unit 13 determines a position near the speaker as the position of the sound image (S25, sound image position determination step). The sound image position determination unit 13 outputs the determined position of the sound image to the output control unit 14.

[0096] The output control unit 14 determines the audio output device to be controlled (S26). The output control unit 14 controls the output settings of the headphones 31 and the speaker 32 so that the sound image of the speech voice is formed at the position determined by the sound image position determination unit 13 (S27, output control step). Then, the output control unit 14 transmits the speech voice input from the information acquisition unit 11 to the headphones 31 and the speaker 32, thereby causing the headphones 31 and the speaker 32 to output the speech voice (S28). In this way, the output control unit 14 causes the headphones 31 and the speaker 32 to form the sound image of the speech voice at the position determined by the sound image position determination unit 13.

[0097] [Modification] (Variation 1: Controlling the sound image when playing specific audio) In the control device 10 according to each embodiment, the sound image position determination unit 13 may determine the position of the sound image of each headphone 31 (and speaker 32) when a specific sound is output to multiple headphones 31, or to one or more headphones 31 and one or more speakers 32. Then, the output control unit 14 may control the output of the headphones 31 (and speaker 32) so that the sound image of the specific sound is formed at the position determined by the sound image position determination unit 13.

[0098] The specific content of the "specific sound" is not particularly limited as long as it is sound that is played by the output control unit 17 of the control device 10 instructing the headphones 31 and the speaker 32. For example, the specific sound may be background music played by the headphones 31 and the speaker 32, or a warning sound notifying an abnormality in the vehicle 1, a voice providing road information, or other sound that is output together with the background music or by interrupting the output of the background music. In this modification, the storage device 30 may store audio data of the specific sound to be played by the headphones 31 (and the speaker 32). The output control unit 17 may then read the audio data from the storage device 30 and transmit it to the headphones 31 and the speaker 32. According to this modification, when a sound image of the specific sound is output from the headphones 31 (and the speaker 32), the position of the sound image of the output sound can be controlled. For example, assume that background music is being output from the headphones 31 and the speaker 32. In this state, when a warning sound, which is a specific sound, is output from the headphones 31 and the speaker 32, the position of the sound image of this warning sound can be controlled.

[0099] (Variation 2: Outputting voice data according to the state of the occupant) In the control systems 100 to 300 according to each embodiment, the detection device 20 may include a status sensor that measures or acquires status information for identifying the status of each of the occupants A to D. The status sensor is, for example, a biosensor, such as a heart rate monitor, for measuring vital data of the occupants. Note that the status of the occupants A to D can also be determined from images captured by the camera 21. Therefore, the camera 21 may also function as the status sensor.

[0100] The information acquisition unit 11 of this modified example acquires state information of each occupant from a state sensor. Then, the output control unit (audio determination unit) 14 of the control device 10 controls the sound image of the headphones 31 (and the speaker 32) and determines the sound to be output from each of the headphones 31 and the speaker 32 based on the state information. Then, the output control unit 14 outputs the determined sound from the headphones 31 and the speaker 32. At this time, the output control unit 14 controls the position of the sound image of the output sound as described in each embodiment. This makes it possible to output content according to the state of each occupant to the headphones 31 (and the speaker 32). Note that the sound data that the output control unit 14 outputs to the headphones 31 (and the speaker 32) may be stored in the storage device 30.

[0101] The control device 10 may identify the level of drowsiness, the level of discomfort, etc. of the occupant by analyzing the state information acquired by the information acquisition unit 11. Then, the output control unit 14 may output content corresponding to the identified state from the headphones 31 (and the speakers 32). For example, if it is analyzed that a certain occupant is very drowsy, a warning sound may be output from the headphones 31 corresponding to that occupant.

[0102] (Variation 3: Method for Acquiring Audio Data) The control device 10 according to each embodiment may include a communication unit for performing Internet communication and / or a receiving antenna for receiving broadcast waves of television, radio, etc. The output control unit 14 may output, to the headphones 31 and / or the speakers 32, audio data downloaded via the Internet and / or audio data included in the broadcast waves received by the receiving antenna.

[0103] (Variation 4: When a timer is provided) The control systems 100 to 300 according to each embodiment may include a boarding / exiting sensor that detects whether an occupant has gotten on or off the vehicle 1, and a timing unit that measures time. The control device 10 of the control systems 100 to 300 may determine the audio data to be output from the headphones 31 and / or the speaker 32 according to the riding time of each occupant measured by the timing unit.

[0104] In this modified example, the boarding / exiting sensor and the timing unit are connected to the control device 10 by wire or wirelessly. The timing unit may be built into the control device 10. When the boarding / exiting sensor detects that an occupant has boarded the vehicle 1, it notifies the control device 10 that the occupant has boarded. When the control device 10 receives this notification, it causes the timing unit to start measuring time.

[0105] The storage device 30 may store audio data to be output from the headphones 31 and / or the speaker 32. The output control unit 14 of the control device 10 causes the headphones 31 and / or the speaker 32 to output audio corresponding to the time measured by the timer unit (i.e., the passenger's riding time) at any timing.

[0106] For example, when the riding time exceeds a predetermined time (for example, one hour), the output control unit 14 reads out data of a guidance voice urging the passenger to take a break from the storage device 30 and outputs this data from the headphones 31 and the speaker 32. Note that if the sound image position determination unit 13 has determined the position of the sound image, the output control unit 14 may also control the position of the sound image when outputting the guidance voice.

[0107] When the getting-in / out sensor detects that the occupant has dismounted, it notifies the control device 10. When the control device 10 receives this notification, it stops the measurement of the timing unit.

[0108] [Software implementation example] The functions of the control device 10 (hereinafter referred to as "device") can be realized by a program that causes a computer to function as each control block of the device.

[0109] In this case, the device is a computer having, as hardware for executing the program, at least one memory for expanding the program and at least one processor for executing the arithmetic processing described in the program. By executing the program by the memory and the processor in this way, each function described in each of the above embodiments is realized.

[0110] The processor may be, for example, a CPU (Central Processing Unit). The program may be recorded non-transitoryly on one or more computer-readable recording media. The recording media may or may not be included in the device. In the former case, the recording media may be, for example, a ROM (Read Only Memory), a tape, a disk, a card, or a semiconductor memory. In the latter case, the program may be supplied to the device via any wired or wireless transmission medium.

[0111] In addition, some or all of the functions of each control block can be realized by a logic circuit. For example, an integrated circuit in which a logic circuit that functions as each control block is formed is also included in the scope of the present disclosure. In addition, the functions of each control block can be realized by, for example, a quantum computer.

[0112] The present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present disclosure. [Explanation of symbols]

[0113] 1 vehicle 100, 200, 300 control systems 10 Control device 11 Information acquisition department 12 Coordinate processing section 121 Location identification part 122 Direction identification part 13 Sound image position determination section 14 Output control section 20 Detection device 21 Camera 22 Gyro sensor 23. Mike 30 Storage device 31 Headphones 32 speakers

Claims

1. an information acquisition unit that acquires, from a detection device, position information indicating a head position and orientation information indicating a head orientation for each of a plurality of users staying in a predetermined space; a sound image position determination unit that determines the position of a sound image of an output sound of a first output device worn by at least one of the plurality of users and the position of a sound image of an output sound of a second output device disposed within the predetermined space; an output control unit that controls the output of each of the first output device and the second output device based on the head position and head orientation of each of the plurality of users so that a sound image of the output sound is formed at the position determined by the sound image position determination unit; a voice acquisition unit that acquires a voice of one of the plurality of users from a microphone that is placed in the predetermined space or that is worn by the user, the sound image position determination unit determines positions of sound images of the output sounds of the first output device and the second output device such that positions of the sound images of the output sounds of the first output device and the second output device are the same; When the voice acquisition unit acquires the voice of the utterance, the sound image position determination unit determines the positions of the sound images of the output sounds of the first output device and the second output device based on the head position and head orientation of the user who has spoken, so that the vicinity of the user who has spoken is the position of the sound image of each of the output sounds of the first output device and the second output device; The control device, wherein the output control unit causes the first output device and the second output device to output the speech acquired by the speech acquisition unit.

2. the sound image position determination unit determines positions of sound images of the first output device and the second output device when causing the first output device and the second output device to output specific sounds; 2. The control device according to claim 1, wherein the output control unit controls the outputs of the first output device and the second output device so that a sound image of the specific sound is formed at a position determined by the sound image position determination unit.

3. the detection device includes a status sensor for measuring or acquiring status information for identifying a status of each of the plurality of users; the information acquisition unit acquires the status information of each user; 3. The control device according to claim 1, further comprising: a sound determination unit that determines output sounds of the first output device and the second output device based on the state information of each of the users.

4. The control device according to any one of claims 1 to 3, the detection device; the first output device; the second output device.

5. an information acquisition step of acquiring, from a detection device, position information indicating a head position and orientation information indicating a head orientation for each of a plurality of users staying in a predetermined space; a sound image position determination step of determining a position of a sound image of an output sound of a first output device worn by at least one of the plurality of users and a position of a sound image of an output sound of a second output device disposed within the predetermined space; an output control step of controlling outputs of the first output device and the second output device based on a head position and a head orientation of each of the plurality of users so that a sound image of the output sound is formed at the position determined in the sound image position determination step; a voice acquisition step of acquiring a voice of one of the plurality of users from a microphone disposed in the predetermined space or worn by the user, the sound image position determining step determines positions of sound images of the output sounds of the first output device and the second output device so that the positions of the sound images of the output sounds of the first output device and the second output device are the same; When the speech of the utterance is acquired in the speech acquisition step, in the sound image position determination step, based on a head position and a head orientation of the user who has spoken, positions of the sound images of the output sounds of the first output device and the second output device are determined so that the vicinity of the user who has spoken is the position of the sound image of each of the output sounds of the first output device and the second output device; The control method, wherein the output control step causes the first output device and the second output device to output the speech acquired in the speech acquisition step.

Images