Audio devices

Abstract

The present invention provides an acoustic device that localizes a sound image formed by sounds emitted from multiple speakers within a predetermined space to a specific location within that predetermined space. [Solution] In the sound system, the sound device 10 includes a control unit 4 (sound image control unit 42) that localizes the sound image inside the vehicle, which is formed by the sound emitted from speakers 3L and 3R, to a specific position. The control unit 4 generates and acquires a three-dimensional spatial image of the inside of the vehicle based on point cloud information acquired from the lidar 7, and outputs the three-dimensional spatial image to the display unit 6 in order to accept the localization position specification from a touch panel 5 that can accept the localization position specification.

Description

Technical Field

[0001] The present invention relates to an acoustic device that localizes a sound image in a predetermined space formed by sounds emitted from a plurality of speakers at a specific position in the predetermined space.

Background Art

[0002] For example, in an in-vehicle audio system, it is already known to control the localization position of a sound image by providing a delay time difference to the output of each of a plurality of speakers.

[0003] Patent Document 1 describes that reception of a designation of a localization position is accepted in accordance with a position input on a dashboard image 350 of a localization position reception window 300, and a delay time of an audio signal output to each speaker 3 is set so that a sound image is localized at the accepted localization position. It is also described that the localization position can be designated by displaying an in-vehicle layout image 850, which is a layout image in the horizontal direction of the vehicle, and accepting the designation of the localization position.

Prior Art Literature

Patent Literature

[0004]

Patent Literature 1

Summary of the Invention

Problem to be Solved by the Invention

[0005] In the case of the invention described in Patent Document 1, since the localization position is designated by a two-dimensional position, for example, one of the width direction, height direction, and depth direction of the vehicle cannot be designated, making it difficult to designate an accurate localization position.

[0006] An example of the problem to be solved by the present invention is to facilitate accurate designation of a localization position.

Means for Solving the Problem

[0007] To solve the above problems, the invention described in claim 1 comprises: an acquisition unit that acquires a three-dimensional spatial image of a predetermined space generated based on three-dimensional information acquired from a sensor; an output unit that outputs the three-dimensional spatial image acquired by the acquisition unit to a display unit; and a reception unit that receives a specification of a specific position in the three-dimensional spatial image output to the display unit, wherein the output unit outputs at least two images to the display unit in which two directions among the width direction, height direction and depth direction of the predetermined space can be identified based on the three-dimensional spatial image; the reception unit receives a specification of the specific position for each of the at least two images output to the display unit; the first of the at least two images output to the display unit is displayed on the display unit, and after the reception unit receives a specification of the specific position for the first image, the second image is displayed on the display unit in place of the first image, and the reception unit receives a specification of the specific position for the second image. Furthermore, the invention described in claim 2 comprises: an acquisition unit that acquires a three-dimensional spatial image of a predetermined space generated based on three-dimensional information acquired from a sensor; an output unit that outputs the three-dimensional spatial image acquired by the acquisition unit to a display unit; and a reception unit that receives the designation of a specific position in the three-dimensional spatial image output to the display unit, whether or not readjustment of the designated specific position is necessary, or the redesignation of the specific position. The reception unit is characterized in that, when it receives that readjustment is necessary, it can receive whether or not regeneration of the three-dimensional spatial image is necessary; when the reception unit receives that regeneration is necessary, the acquisition unit acquires the regenerated three-dimensional spatial image, and the reception unit can redesignate based on the regenerated three-dimensional spatial image; and when the reception unit receives that regeneration is unnecessary, the regenerated three-dimensional spatial image is not acquired, and the reception unit can redesignate.

[0008] The invention described in claim 8 comprises: an acquisition unit that acquires a three-dimensional spatial image of a predetermined space generated based on three-dimensional information acquired from a sensor; an output unit that outputs the three-dimensional spatial image acquired by the acquisition unit to a display unit; and a reception unit that receives a specification of a specific position in the three-dimensional spatial image output to the display unit, wherein the output unit, after outputting the three-dimensional spatial image to the display unit, sequentially outputs two images to the display unit based on the three-dimensional spatial image, in which two directions among the width direction, height direction, and depth direction of the predetermined space can be identified, wherein the first image and the second image differ from each other in one of the two directions, the first image is displayed on the display unit, and after the reception unit receives a specification of the specific position for the first image, the second image is displayed on the display unit in place of the first image, and the reception unit receives a specification of the specific position for the second image. [Brief explanation of the drawing]

[0009] [Figure 1] This is a functional configuration diagram of a system having an acoustic device according to a first embodiment of the present invention. [Figure 2] Figure 1 is a flowchart illustrating the operation of the sound device. [Figure 3] This is an example of a three-dimensional spatial image. [Figure 4] Figure 3 shows the three-dimensional spatial image from a side view of the vehicle's interior. [Figure 5] Figure 3 shows the three-dimensional spatial image from a viewpoint looking at the vehicle interior from the front. [Figure 6] This is a functional configuration diagram of a system having an acoustic device according to a second embodiment of the present invention. [Figure 7] Figure 6 is a flowchart illustrating the operation of the sound device. [Figure 8] This is a flowchart illustrating the operation of an acoustic device according to a third embodiment of the present invention. [Figure 9] This is an example diagram showing how to identify and display people. [Figure 10] This is an explanatory diagram of motion that follows a person. [Figure 11] This is a flowchart illustrating the operation of an acoustic device according to a fourth embodiment of the present invention. [Modes for carrying out the invention]

[0010] The following describes an acoustic device according to one embodiment of the present invention. In the acoustic device according to one embodiment of the present invention, an acquisition unit acquires a three-dimensional spatial image of a predetermined space generated based on three-dimensional information acquired from a sensor capable of acquiring three-dimensional information of a predetermined space, and an output unit outputs the three-dimensional spatial image acquired by the acquisition unit to a display unit in order to receive a specification of a specific position from a reception unit capable of receiving the specification of a specific position. By doing so, it becomes possible to specify a specific position for localizing the sound image based on the three-dimensional spatial image displayed on the display unit, and it becomes possible to easily specify an accurate localization position using a three-dimensional spatial image that reproduces the predetermined space.

[0011] Furthermore, the three-dimensional spatial image may be updated sequentially, and the output unit may output the updated three-dimensional spatial image. In this way, a specific location can be specified based on the three-dimensional spatial image that corresponds to the entry and exit of people and changes in their positions within a predetermined space.

[0012] Furthermore, the system includes an identification unit that identifies a person in the three-dimensional spatial image acquired by the acquisition unit, an output unit that outputs the three-dimensional spatial image in which the person has been identified, a reception unit that allows the person to be specified, and a localization control unit that, if a person is specified by the reception unit, may localize the sound image to the position corresponding to the specified person. In this way, for example, if a person is seated in a seat inside a vehicle, specifying that person makes it possible to localize the sound image to the position corresponding to that person.

[0013] Furthermore, the identifying unit may detect a change in the position of a person's head from a plurality of three-dimensional spatial images, and the localization control unit may cause the localization position of a sound image to follow the head based on the change in the position of the head detected by the identifying unit when a person is designated by the receiving unit. With this configuration, even when, for example, the orientation of the face or the angle of the seat changes, the localization position of the sound image can follow such changes.

[0014] Furthermore, the sensor may be configured to measure a distance to an object by emitting an electromagnetic wave into a predetermined space and receiving the electromagnetic wave reflected by the object in the predetermined space. With this configuration, a three-dimensional spatial image can be generated based on point cloud information acquired by so-called LiDAR (Light Detection And Ranging).

[0015] Furthermore, the acoustic device may be installed in a vehicle, and the predetermined space may be a vehicle cabin space. With this configuration, a sound image in the vehicle cabin space can be localized at a specific position.

[0016] Furthermore, in the terminal device according to an embodiment of the present invention, an obtaining unit obtains a three-dimensional spatial image of a predetermined space generated based on three-dimensional information obtained from a sensor capable of obtaining three-dimensional information of the predetermined space, and a display unit displays the three-dimensional spatial image obtained by the obtaining unit to receive designation of a specific position from a receiving unit capable of receiving the designation of the specific position in the predetermined space. Then, an output unit outputs the specific position received by the receiving unit as information on a position where a sound image formed by sounds emitted from a plurality of speakers is to be localized. With this configuration, a specific position for localizing a sound image can be designated based on the three-dimensional spatial image displayed on the display unit, and accurate designation of a localization position can be easily performed from the terminal device by means of the three-dimensional spatial image reproducing the predetermined space.

[0017] Further, an acoustic control method according to an embodiment of the present invention acquires, in an obtaining step, a three-dimensional spatial image of a predetermined space generated based on three-dimensional information obtained from a sensor capable of obtaining three-dimensional information of the predetermined space, and outputs, in an output step, the three-dimensional spatial image obtained by an obtaining unit to a display unit to receive designation of a specific position from a receiving unit capable of receiving designation of the specific position. With this configuration, it is possible to designate a specific position where a sound image is to be localized based on the three-dimensional spatial image displayed on the display unit, and accurate designation of the localization position can be facilitated by the three-dimensional spatial image that reproduces the predetermined space.

[0018] Further, the above-described acoustic control method may be executed by a computer. With this configuration, using a computer, it is possible to designate a specific position where a sound image is to be localized based on the three-dimensional spatial image displayed on the display unit, and accurate designation of the localization position can be facilitated.

Examples

[0019] An acoustic device according to a first embodiment of the present invention will be described with reference to FIGS. 1 to 5. The acoustic device 10 is mounted, for example, in a vehicle such as an automobile. FIG. 1 shows a functional configuration of an acoustic system including the acoustic device 10. The acoustic system includes an audio device 1, delay adjustment units 2 (2R, 2L), speakers 3 (3R, 3L), a control unit 4, a touch panel 5, a display unit 6, and a lidar 7. In the present embodiment, the acoustic system is mounted on a vehicle and controls acoustics in a vehicle cabin as a predetermined space.

[0020] The audio device 1 is a device that outputs an audio signal intended for listening by a user, such as a CD player, memory audio, a radio tuner, or a streaming distribution receiver.

[0021] In this embodiment, the delay adjustment unit 2 consists of a delay adjustment unit 2L for the left channel and a delay adjustment unit 2R for the right channel. The delay adjustment unit 2L delays the left channel audio signal output from the audio device 1 to the left channel speaker 3L by a set delay time. The delay adjustment unit 2R delays the right channel audio signal output from the audio device 1 to the right channel speaker 3R by a set delay time.

[0022] The control unit 4 comprises a generation unit 41 and a sound image control unit 42. The control unit 41 is composed of, for example, a microcomputer having a CPU. The generation unit 41 and the sound image control unit 42 can be implemented as functions of a computer program executed by the CPU.

[0023] The generation unit 41 generates a three-dimensional spatial image by modeling the point cloud information output by the lidar 7 with polygons or the like using a well-known method. In other words, the control unit 4 functions as an acquisition unit that generates and acquires a three-dimensional spatial image based on the point cloud information. The generation unit 41 also outputs the generated three-dimensional spatial image to the display unit 6 for display on the display unit 6. In other words, the control unit 4 (generation unit 41) functions as an output unit that outputs the three-dimensional spatial image to the display unit 6.

[0024] The sound image control unit 42 acquires the position coordinates touched on the touch panel 5 and identifies the three-dimensional coordinates (x, y, z) on the three-dimensional spatial image corresponding to those position coordinates. Then, it adjusts and sets the delay amounts of the delay adjustment units 2L and 2R so that the sound image formed by the sound emitted from speakers 3L and 3R is localized to the position in real space corresponding to the identified three-dimensional coordinates. In other words, the control unit 4 (sound image control unit 42) functions as a localization control unit that localizes the sound image inside the vehicle, formed by the sound emitted from speakers 3L and 3R, to a specific position.

[0025] The touch panel 5 is positioned so as to overlap the display surface of the display unit 6. The touch panel 5 is a well-known input device that outputs the two-dimensional coordinates of the position touched, for example, by a finger. The display unit 6 is composed of, for example, a liquid crystal display. The display unit 6 displays a three-dimensional spatial image, etc., generated by the control unit 4. In this embodiment, the touch panel 5 is positioned so as to overlap the display surface of the display unit 6, but the display unit 6 may be positioned on the dashboard, and the touch panel 5 may be provided as a touchpad near the shift lever or on the steering wheel, etc., with each component being positioned separately.

[0026] The LiDAR 7 is installed in a designated space, either inside the vehicle or in a position that allows it to scan the vehicle interior. The LiDAR 7 is a sensor that recognizes objects in its surroundings, and is also referred to as LiDAR (Light Detection And Ranging). The LiDAR 7 is a known sensor that emits electromagnetic waves such as laser light and discretely measures the direction and distance to objects within the scanning range using the reflected waves (reflected light), recognizing the position and shape of those objects as a three-dimensional point cloud. Therefore, the point cloud recognized by the LiDAR 7 is output as point cloud information as three-dimensional information.

[0027] There may be more than one Rider 7 installed. The number and placement of Rider 7s can be determined appropriately according to the size of the passenger compartment, for example, in the case of a vehicle.

[0028] Here, the aforementioned delay adjustment unit 2 and control unit 4 function as the sound device 10 according to this embodiment.

[0029] Next, the operation (sound control method) of the sound device 10 with the above configuration will be explained with reference to Figures 2 to 5. The flowchart shown in Figure 2 is executed by the control unit 4. Furthermore, the flowchart shown in Figure 2 can be configured as a sound control program by being executed by the CPU of the control unit 4.

[0030] First, in step S11, the generation unit 41 acquires point cloud information from the lidar 7. Next, in step S12, as described above, the generation unit 41 models the point cloud information acquired from the lidar 7 using polygons or the like in a well-known method to generate a three-dimensional spatial image. If there are multiple lidars 7, the three-dimensional spatial image may be generated by combining the point cloud information of multiple lidars. An example of a three-dimensional spatial image is shown in Figure 3. Figure 3 is a three-dimensional spatial image of the interior of a vehicle viewed from a viewpoint diagonally above and to the left front. Although Figure 3 shows a viewpoint from diagonally above and to the left front, the viewpoint can be changed to any viewpoint by rotating, zooming in, zooming out, etc., by operating the touch panel 5.

[0031] Next, in step S13, the generation unit 41 displays the three-dimensional spatial image generated in step S12 on the display unit 6.

[0032] Next, in step S14, the sound image control unit 42 obtains (accepts) a designation of a specific position for localizing the sound image (hereinafter simply referred to as the localization position) from the touch panel 5. Here, the method for identifying the coordinate position indicating the localization position from the three-dimensional spatial image displayed on the display unit 6 will be explained. In a three-dimensional spatial image with a viewpoint as shown in Figure 3, even if the position of the driver's seat is specified by touching, for example, the display on the display unit 6 is two-dimensional, so it is difficult to determine the accurate three-dimensional coordinates (x, y, z) as is. Therefore, in this embodiment, first, the vehicle interior is displayed from a side viewpoint as shown in Figure 4, and the user is asked to touch the touch panel 5 to specify the position of the localization position P in the depth and height directions of the vehicle. Next, the vehicle interior is displayed from a front viewpoint as shown in Figure 5, and the user is asked to touch the touch panel 5 to specify the position of the localization position P in the width direction of the vehicle. In this way, the three-dimensional coordinates of the specified localization position in the three-dimensional spatial image are identified.

[0033] Note that the display order of Figure 4 and Figure 5 may be reversed. If the display order of Figure 4 and Figure 5 is reversed, the user will be asked to specify the width and height of the vehicle at position P in Figure 5, and then to specify the depth of the vehicle at position P in the subsequent display of Figure 4. Alternatively, when displaying Figure 5 after Figure 4, the position in the height direction specified in Figure 4 may be displayed for reference when specifying in Figure 5, or the height direction of position P may be re-specified when displaying Figure 5.

[0034] Furthermore, the viewpoint doesn't have to be a side view of the passenger compartment; a top view is also acceptable. In short, it's sufficient to identify a position that spans at least two of the vehicle's width, height, and depth directions.

[0035] In other words, in step S13, the control unit 4 outputs a three-dimensional spatial image to the display unit 6 in order to receive a specification of a localization position from the touch panel 5 (reception unit), which is capable of receiving a specification of a specific position for localizing the sound image, and in step S14, it receives the specification of the localization position from the touch panel 5.

[0036] Next, in step S15, the sound image control unit 42 adjusts the delay adjustment units 2L and 2R respectively so that the sound image is localized to the localization position specified in step S14. Specifically, since the placement positions of speakers 3L and 3R inside the vehicle are known in advance, the position coordinates of speakers 3L and 3R in the three-dimensional spatial image are also known. Then, the distance from the localization position specified in step S14 to speaker 3L and the distance from the localization position specified in step S14 to speaker 3R are determined based on each other's coordinates. Then, the output from the speaker 3 with the shorter distance is delayed according to the difference in distance between it and the speaker 3 with the longer distance.

[0037] Next, in step S16, the sound image control unit 42 displays a message on the display unit 6, for example, regarding whether readjustment is necessary. If the touch panel 5 indicates that readjustment is necessary (YES), the process proceeds to step S17. On the other hand, if the touch panel 5 indicates that readjustment is not necessary (NO), the flowchart ends.

[0038] Next, in step S17, it is determined whether or not regeneration of the three-dimensional spatial image is necessary. If it is necessary (YES), the process returns to step S11; otherwise, the process returns to step S14. In step S17, for example, the display unit 6 displays whether or not regeneration of the three-dimensional spatial image is necessary, allowing the user to select. If the user selects that it is necessary, the process returns to step S11 and the three-dimensional spatial image is generated again. If the user selects that it is not necessary, the process returns to step S14 and the localization position is acquired. In this way, the three-dimensional spatial image is updated sequentially, and the updated three-dimensional spatial image is output to the display unit 6 as needed.

[0039] As is clear from the above explanation, step S12 functions as an acquisition step, and step S13 functions as an output step.

[0040] In the flowchart of Figure 2, the three-dimensional spatial image was updated sequentially, but it may be generated once and then not updated. The generated three-dimensional spatial image may also be saved in the memory of the control unit 4. Furthermore, the storage location is not limited to the control unit 4; for example, it may be a removable memory card, or it may be sent to an external server for storage using a communication unit. If it is saved to an external server, when specifying the localization position the next time, steps S11 and S12 can be omitted, and the step of acquiring the three-dimensional spatial image from the external server should be executed before executing steps S13 and onward.

[0041] In this embodiment, the sound device 1 generates and acquires a three-dimensional spatial image of the vehicle interior based on point cloud information acquired by the control unit 4 from the lidar 7, and outputs the three-dimensional spatial image to the display unit 6 in order to accept the specification of the localization position from the touch panel 5 which is capable of accepting the specification of the localization position. In this way, it becomes possible to specify a specific position for localizing the sound image based on the three-dimensional spatial image displayed on the display unit 6, and it becomes easy to specify an accurate localization position using a three-dimensional spatial image that reproduces the vehicle interior space.

[0042] Furthermore, the three-dimensional spatial image is updated sequentially, and the control unit 4 outputs the updated three-dimensional spatial image. In this way, the localized position can be specified based on the three-dimensional spatial image corresponding to passengers getting on and off inside the vehicle. [Examples]

[0043] Next, an acoustic device 10A according to a second embodiment of the present invention will be described with reference to Figures 6 and 7. Note that parts identical to those in the first embodiment described above are denoted by the same reference numerals and their descriptions are omitted.

[0044] Figure 6 shows the functional configuration of an acoustic system having an acoustic device 10A according to this embodiment. The acoustic system includes an in-vehicle unit 20, speakers 3 (3L, 3R), a lid 7, and a terminal device 30. Here, the speakers 3 (3L, 3R) and the lid 7 are the same as in the first embodiment.

[0045] The in-vehicle unit 20 is installed, for example, in a vehicle. As shown in Figure 6, the in-vehicle unit 20 includes an audio device 1, a delay adjustment unit 2 (2L, 2R), a control unit 4A, and a communication unit 8.

[0046] The audio equipment 1 and delay adjustment unit 2 are the same as in the first embodiment. The control unit 4A comprises a generation unit 41A and a sound image control unit 42A.

[0047] The generation unit 41A generates a three-dimensional spatial image, similar to the first embodiment, but differs from the first embodiment in that the generated three-dimensional spatial image is transmitted to the terminal device 30 via the communication unit 8.

[0048] The sound image control unit 42A adjusts the delay amount of the delay adjustment units 2L and 2R to localize the sound image formed by the sound emitted from speakers 3L and 3R to a position in real space corresponding to the calculated three-dimensional coordinates, similar to the first embodiment. However, it differs from the first embodiment in that it acquires the position coordinates of the touch on the touch panel 33 from the terminal device 30 via the communication unit 8 and identifies the three-dimensional coordinates on the three-dimensional spatial image corresponding to those position coordinates.

[0049] The communication unit 8 transmits the three-dimensional spatial image generated by the generation unit 41A to the terminal device 30 and receives the position coordinates of the touch on the touch panel 33 from the terminal device 30. The communication unit 8 communicates with the terminal device 30 by wireless communication such as Bluetooth®, Wi-Fi®, or infrared communication.

[0050] The terminal device 30 is a portable electronic device such as a smartphone. The terminal device 30 comprises a communication unit 31, a control unit 32, a touch panel 33, and a display unit 34.

[0051] The communication unit 31 transmits the position coordinates of the touch on the touch panel 33 to the in-vehicle unit 20 and receives the three-dimensional spatial image generated by the generation unit 41A from the in-vehicle unit 20.

[0052] The control unit 32 is responsible for the overall control of the terminal device 30. The control unit 32 displays the three-dimensional spatial image received by the communication unit 31 on the display unit 34, obtains the position coordinates touched by the touch panel 33, and transmits them to the in-vehicle device 20 via the communication unit 31.

[0053] The touch panel 33 is positioned so as to overlap the display surface of the display unit 34. The touch panel 33 is a well-known input device that outputs the two-dimensional coordinates of the position touched, for example, by a finger. The display unit 34 is composed of, for example, a liquid crystal display. The display unit 34 displays a three-dimensional spatial image or the like output from the control unit 32.

[0054] Here, the aforementioned delay adjustment unit 2, control unit 4, and communication unit 8 function as the sound device 10A according to this embodiment.

[0055] Furthermore, the communication unit 31 of the terminal device 30 functions as an acquisition unit that acquires a three-dimensional spatial image of the vehicle interior generated based on point cloud information acquired from the lidar 7, and as an output unit that outputs a specific position received by the touch panel 33 as position information for localizing the sound image. The display unit 34 functions as a display unit that displays the three-dimensional spatial image received by the communication unit 31 in order to receive the specification of a specific position for localizing the sound image in the vehicle interior from the touch panel 33, which is capable of receiving the specification of a specific position.

[0056] Next, the operation (sound control method) of the sound device 10A with the above configuration will be explained with reference to Figure 7. The flowchart shown in Figure 7 is executed by the control unit 4A of the in-vehicle unit 20 and the control unit 32 of the terminal device 30.

[0057] First, let's explain the operation of the in-vehicle unit 20. In step S21, the control unit 4A acquires point cloud information from the lidar 7. Next, in step S22, the control unit 4A generates a three-dimensional spatial image from the point cloud information acquired from the lidar 7 using a well-known method.

[0058] Next, in step S23, the control unit 4A receives request information for a three-dimensional spatial image from the terminal device 30 and determines whether or not a request for a three-dimensional spatial image has been made. If there is no request (NO), it returns to step S21. If there is a request (YES), in step S24, it causes the communication unit 8 to transmit the three-dimensional spatial image generated by the generation unit 41A to the terminal device 30. If step S23 returns to step S21 because there is no request, the point cloud information is acquired again (step S21), and the three-dimensional spatial image is regenerated (step S22), thereby updating sequentially.

[0059] Next, in step S25, the control unit 4A receives localization position information from the terminal device 30 via the communication unit 8. Then, in step S26, the control unit 4A adjusts the delay adjustment units 2L and 2R respectively so that the sound image is localized to the localization position received in step S24, and returns to step S21. The adjustment method is the same as in the first embodiment.

[0060] Returning to step S21, the point cloud information is acquired again (step S21), and the three-dimensional spatial image is regenerated (step S22), resulting in sequential updates. If there is a request for the regenerated three-dimensional spatial image from the terminal device 30, steps S24 onwards are executed; otherwise, the process returns to step S21.

[0061] Next, the operation of the terminal device 30 will be explained. In step S31, the control unit 32 instructs the communication unit 31 to transmit the request information for the three-dimensional spatial image to the in-vehicle device 20. Next, in step S32, the control unit 32 receives the three-dimensional spatial image from the in-vehicle device 20 via the communication unit 31.

[0062] Next, in step S33, the control unit 32 displays the three-dimensional spatial image received in step S32 on the display unit 34.

[0063] Next, in step S34, the control unit 32 obtains (accepts) the designation of the sound image localization position from the touch panel 33. The method for obtaining the localization position is the same as in the first embodiment.

[0064] Next, in step S35, the control unit 32 causes the communication unit 31 to transmit the positional position acquired in step S34 as positional position information to the in-vehicle unit 20.

[0065] Next, in step S36, the control unit 32 displays a message on the display unit 34, for example, regarding whether readjustment is necessary. If the touch panel 33 indicates that readjustment is necessary (YES), the process proceeds to step S37. On the other hand, if the touch panel 33 indicates that readjustment is not necessary (NO), the flowchart ends.

[0066] Next, in step S37, it is determined whether or not it is necessary to regenerate the three-dimensional spatial image. If it is necessary (YES), the system returns to step S31; if it is not necessary (NO), the system returns to step S34. In step S37, for example, the display unit 6 displays whether it is necessary to regenerate the three-dimensional spatial image and allows the user to select. If the user selects that it is necessary, the system returns to step S31 and requests the regenerated three-dimensional spatial image. If the user selects that it is not necessary, the system returns to step S34 and acquires the localization position.

[0067] In this embodiment, the terminal device 30 acquires a three-dimensional spatial image of the vehicle interior generated by the control unit 32 based on three-dimensional information acquired from the lidar 7, and the display unit 34 displays the three-dimensional spatial image to accept the specification of a localized position from the touch panel 33, which is capable of accepting the specification of a localized position within the vehicle interior. Then, the communication unit 31 outputs the localized position information. In this way, it becomes possible to specify a localized position for localizing the sound image based on the three-dimensional spatial image displayed on the display unit 34, making it easy for the terminal device 30 to specify an accurate localized position.

[0068] Furthermore, since the fixed position can be specified from the portable terminal device 30, the fixed position can be specified regardless of the installation location of the in-vehicle unit 20. [Examples]

[0069] Next, an acoustic device 10 according to a third embodiment of the present invention will be described with reference to Figures 8 to 10. Note that parts identical to those in the first and second embodiments described above are denoted by the same reference numerals and their descriptions are omitted.

[0070] This embodiment has the same functional configuration as the first embodiment, but the operation (sound control method) of the sound device 10 is slightly different. The operation (sound control method) of the sound device 10 according to this embodiment will be explained with reference to Figure 8. The flowchart shown in Figure 8 is executed by the control unit 4.

[0071] First, in step S41, the generation unit 41 acquires point cloud information from the lidar 7. Next, in step S42, the generation unit 41 generates a three-dimensional spatial image from the point cloud information acquired from the lidar 7, as described above.

[0072] In step S42, when the generation unit 41 generates a three-dimensional spatial image, it recognizes people (passengers) inside the vehicle from point cloud information using object recognition or the like, and identifies and displays the people in the three-dimensional spatial image. That is, the control unit 4 (generation unit 41) functions as an identification unit that identifies people in the three-dimensional spatial image. An example of a three-dimensional spatial image in which people have been identified is shown in Figure 9. Figure 9 is Figure 6 with people added. As shown in Figure 9, each person is assigned a label such as "A" and "B" so that they can be identified. Alternatively, different colors or patterns may be added to each person instead of labels.

[0073] Next, in step S43, the generation unit 41 displays the three-dimensional spatial image generated in step S42 on the display unit 6.

[0074] Next, in step S44, the sound image control unit 42 obtains (accepts) the designation of a person for which the sound image should be localized from the touch panel 5. For example, in Figure 8, a person is designated by touching either "A" or "B". In this embodiment, regardless of whether the person is touched on the head or the torso, the localization position is set to the center of the face (head). The position coordinates of this localization position are automatically determined based on the position of the person in the three-dimensional spatial image. Furthermore, the localization position automatically determined when a person is touched is not limited to the center of the face; it may also be other positions such as the ears. Alternatively, for example, if the chest is touched, the chest may be set as the localization position.

[0075] Next, in step S45, the sound image control unit 42 adjusts the delay adjustment units 2L and 2R, respectively, so that the sound image is localized to the head of the person specified in step S44. The adjustment method is the same as in the first embodiment. That is, if a person is specified, the control unit 4 (sound image control unit 42) localizes the sound image to the position corresponding to the specified person.

[0076] Next, in step S46, the generation unit 41 acquires point cloud information from the lidar 7. This step S46 is performed in order to make a decision in step S47. Then, in step S47, it is determined whether or not there is a change in the tracking position. If there is a change, the process returns to step S45; if there is no change, the process returns to step S46.

[0077] The tracking position in step S47 is the head of the person whose sound image was localized in step S45. This is because if the position of the person's head in the three-dimensional spatial image generated from the point cloud information acquired in step S46 has changed from the position of the person's head acquired in step S41 or the previous step S46, the system assumes the head has moved and returns to step S45 to track the head's movement. Then, the sound image is localized to the position of the person's head in the three-dimensional spatial image generated from the point cloud information acquired in step S46.

[0078] An example of tracking is shown in Figure 10. Figure 10 shows an example where a person seated in a seat reclines the backrest, changing the angle of the backrest. In Figure 10, the local position P moves to local position P' in response to the change in the position of the head caused by the change in the seat angle.

[0079] On the other hand, if the position of the person's head in the three-dimensional spatial image generated from the point cloud information acquired in step S46 has not changed from the position of the person's head acquired in step S41 or the previous step S46, then it is assumed that the head has not moved, and step S45 is not executed, and the point cloud information is acquired again in step S46.

[0080] In this embodiment, a three-dimensional spatial image may be generated (updated) based on the point cloud information acquired in step S46 and displayed on the display unit 6.

[0081] Furthermore, while the flowchart shown in Figure 8 illustrates the operation when a person is specified, if, for example, a sound image is to be localized between "A" and "B" in a three-dimensional spatial image as shown in Figure 9, the procedure can be followed in the manner of the first embodiment. In other words, depending on the touch position (specified position), steps S44 onwards may be executed if a person is touched, and steps S14 onwards may be executed if something other than a person is touched.

[0082] In this embodiment, the generation unit 41 of the control unit 4 identifies a person in a three-dimensional spatial image and outputs the three-dimensional spatial image in which the person has been identified. The touch panel 5 allows for the selection of a person, and the delay adjustment unit 2, when a person is selected on the touch panel 5, localizes the sound image to the position corresponding to the selected person. In this way, for example, if a person is seated in a seat inside a vehicle, selecting that person makes it possible to localize the sound image to the position corresponding to that person.

[0083] Furthermore, the control unit 4 detects changes in the position of a person's head from multiple three-dimensional spatial images, and the delay adjustment unit 2, when a person is selected on the touch panel 5, adjusts the localization position of the sound image to follow the head based on the changes in the head position detected by the control unit 4. In this way, even if, for example, the direction of the face or the angle of the seat changes, the localization position of the sound image can be adjusted accordingly. [Examples]

[0084] Next, an acoustic device 10A according to a fourth embodiment of the present invention will be described with reference to Figure 11. Note that parts identical to those in the first to third embodiments described above are denoted by the same reference numerals and their descriptions are omitted.

[0085] This embodiment has the same functional configuration as the second embodiment, but the operation (sound control method) of the sound device 10A is slightly different. The operation (sound control method) of the sound device 10A according to this embodiment will be explained with reference to Figure 11. The flowchart shown in Figure 11 is executed by the control unit 4A of the in-vehicle unit 20 and the control unit 32 of the terminal device 30.

[0086] First, the operation of the in-vehicle device 20 will be explained. In step S51, the control unit 4A acquires point cloud information from the lidar 7. Next, in step S52, the control unit 4A generates a three-dimensional spatial image using a well-known method. The three-dimensional spatial image generated in step S52 can be displayed with the identification of a person, similar to the third embodiment.

[0087] Next, in step S53, the control unit 4A determines whether or not there is a request for a three-dimensional spatial image, based on the information received from the terminal device 30. If there is no request (NO), the control unit waits in this step. If there is a request (YES), in step S54, the control unit 4A causes the communication unit 8 to transmit the three-dimensional spatial image generated by the generation unit 41A to the terminal device 30.

[0088] Next, in step S55, the control unit 4A receives person designation information from the terminal device 30, which is information specifying the person for which the sound image should be localized. Then, the delay adjustment units 2L and 2R are adjusted respectively so that the sound image is localized to the head of the person indicated by the person designation information received in step S55. The adjustment method is the same as in the first and third embodiments.

[0089] Next, in step S57, the generation unit 41 acquires point cloud information from the lidar 7. This step S57 is performed in order to make a decision in step S58. Then, in step S58, it is determined whether or not there is a change in the tracking position. If there is a change, the process returns to step S56; if there is no change, the process returns to step S57. The tracking method in step S58 is the same as in the third embodiment.

[0090] In this embodiment as well, a three-dimensional spatial image may be generated (updated) based on the point cloud information acquired in step S57 and displayed on the display unit 6.

[0091] Next, the operation of the terminal device 30 will be explained. In step S61, the control unit 32 causes the in-vehicle device 20 to send request information for a three-dimensional spatial image to the communication unit 31. Next, in step S62, the three-dimensional spatial image is received from the in-vehicle device 20 via the communication unit 31.

[0092] Next, in step S63, the control unit 32 displays the three-dimensional spatial image received in step S62 on the display unit 34.

[0093] Next, in step S64, the control unit 32 obtains (accepts) the designation of the person for whom the sound image should be localized from the touch panel 5. The method for obtaining the designation of the person is the same as in the third embodiment.

[0094] Next, in step S65, the control unit 32 causes the communication unit 31 to transmit the person specified in step S34 as person designation information to the in-vehicle unit 20.

[0095] Next, in step S66, the control unit 32 displays a message on the display unit 34, for example, regarding whether readjustment is necessary. If the touch panel 33 indicates that readjustment is necessary (YES), the system returns to step S64. On the other hand, if the touch panel 33 indicates that readjustment is not necessary (NO), the flowchart ends.

[0096] In this embodiment as well, if an input indicating that readjustment is necessary is received in step S66, it may be possible to determine whether or not it is necessary to regenerate the three-dimensional spatial image.

[0097] In this embodiment, the terminal device 30 acquires a three-dimensional spatial image in which a person inside the vehicle is identified, which is generated by the control unit 32 based on three-dimensional information acquired from the lidar 7. The display unit 34 displays the three-dimensional spatial image in order to receive a designation of a person to whom the sound image should be localized from a touch panel 33 that can accept the designation of a localized position inside the vehicle. The communication unit 31 then outputs the person designation information. In this way, for example, if a person is seated in a seat inside the vehicle, the terminal device 30 can designate that person to localize the sound image to the position corresponding to that person.

[0098] Furthermore, the control unit 4A detects changes in the position of a person's head from multiple three-dimensional spatial images, and the delay adjustment unit 2, when a person is selected on the touch panel 5, adjusts the localization position of the sound image to follow the head based on the changes in head position detected by the control unit 4A. In this way, even if, for example, the direction of the face or the angle of the seat changes, the localization position of the sound image can be adjusted accordingly.

[0099] Although the above-described embodiment uses a lidar as the sensor, other sensors may be used. For example, any sensor capable of acquiring three-dimensional information for generating a three-dimensional spatial image, such as a stereo camera, would suffice.

[0100] In addition, although the above-described embodiment was explained using a vehicle's passenger compartment as the designated space, it is not limited to a passenger compartment; it may also be an indoor space within a building. Furthermore, although the above-described embodiment was explained using a two-channel sound system with a right channel and a left channel, it can also be applied to sound systems that localize sound images using multiple speakers, such as a four-channel sound system consisting of four speakers.

[0101] Furthermore, the present invention is not limited to the embodiments described above. That is, those skilled in the art can implement the invention in various ways, without departing from the core principles, in accordance with conventionally known knowledge. As long as such modifications still incorporate the acoustic device of the present invention, they are of course included within the scope of the present invention. [Explanation of Symbols]

[0102] 2. Delay adjustment unit (position control unit) 3 speakers 4. Control Unit (Acquisition Unit, Output Unit, Identification Unit) 4A Control Unit (Acquisition Unit, Identification Unit) 5. Touch panel (reception area) 6 Display section 7 Riders 8. Communication section (output section) 10, 10A sound equipment 20 Onboard equipment 30 Terminal devices 31 Communications Department (Acquisition Department) 32 Control Unit (Output Unit) 33. Touch panel (reception area) 34 Display section

Claims

[Claim 1] An acoustic device comprising a localization control unit that localizes a sound image formed by sounds emitted from multiple speakers in a predetermined space to a specific position in the predetermined space, An acquisition unit that acquires a three-dimensional spatial image of the predetermined space generated based on three-dimensional information acquired from a sensor capable of acquiring three-dimensional information of the predetermined space, An output unit that outputs the three-dimensional spatial image acquired by the acquisition unit in order to receive the specification of the specified location from a receiving unit that is capable of receiving the specification of the specified location to a display unit, An acoustic device characterized by being equipped with

Images