Sound apparatus

Abstract

To make it easy to specify an exact position.SOLUTION: An acoustic apparatus 1 generates and acquires a three-dimensional space image in a passenger compartment generated on the basis of point group information acquired from a lidar 7 by a control unit 4 and outputs the three-dimensional space image to a display unit 6 in order to accept a designation of a localization position from a touch panel 5 that can accept the designation of the localization position.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an acoustic device that localizes a sound image formed in a given space by sounds emitted from a plurality of speakers at a specific position in the given space. [Background technology]

[0002] For example, in an in-vehicle audio system, it is already known to control the localization position of a sound image by imparting a delay time difference to the output of each of a plurality of speakers.

[0003] Patent Document 1 describes that a designation of a localization position is accepted in response to a position input on a dashboard image 350 in a localization position acceptance window 300, and a delay time of an audio signal output to each speaker 3 is set so that a sound image is localized at the accepted localization position. It also describes that the designation of a localization position is accepted by displaying an interior layout image 850, which is a layout image in the horizontal direction of the vehicle. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2006-196941 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the case of the invention described in Patent Document 1, the localization position is specified in two dimensions, so for example, it is not possible to specify one of the vehicle's width, height, and depth directions, making it difficult to specify an accurate localization position.

[0006] One example of a problem to be solved by the present invention is to facilitate the specification of an accurate localization position. [Means for solving the problem]

[0007] In order to solve the above problem, the invention described in claim 1 comprises an acquisition unit that acquires a three-dimensional spatial image of a specified space generated based on three-dimensional information acquired from a sensor, an output unit that outputs the three-dimensional spatial image acquired by the acquisition unit to a display unit, and a reception unit that accepts designation of a specific position in the three-dimensional spatial image output to the display unit, wherein the output unit outputs to the display unit at least two images in which two of the width direction, height direction, and depth direction in the specified space can be identified based on the three-dimensional spatial image, and the reception unit accepts designation of the specific position for each of the at least two images output to the display unit, and a first image of the at least two images output to the display unit is displayed on the display unit, and after the reception unit accepts the designation of the specific position for the first image, a second image is displayed on the display unit in place of the first image, and the reception unit accepts the designation of the specific position for the second image.

[0008] The invention described in claim 7 comprises an acquisition unit that acquires a three-dimensional spatial image of a specified space generated based on three-dimensional information acquired from a sensor, an output unit that outputs the three-dimensional spatial image acquired by the acquisition unit to a display unit, and a reception unit that accepts designation of a specific position in the three-dimensional spatial image output to the display unit, wherein after outputting the three-dimensional spatial image to the display unit, the output unit sequentially outputs to the display unit two images in which two directions of the width direction, height direction, and depth direction in the specified space can be identified based on the three-dimensional spatial image, and a first image and a second image of the two images differ from each other in one of the two directions, and after the first image is displayed on the display unit and the reception unit accepts the designation of the specific position for the first image, the second image is displayed on the display unit in place of the first image, and the reception unit accepts the designation of the specific position for the second image. [Brief description of the drawings]

[0009] [Figure 1] 1 is a functional configuration diagram of a system having an audio device according to a first embodiment of the present invention. [Diagram 2] 2 is a flowchart of the operation of the audio device shown in FIG. 1. [Diagram 3] FIG. 2 is a diagram showing an example of a three-dimensional space image. [Figure 4] 4 is a diagram showing the three-dimensional space image shown in FIG. 3 as viewed from the side of the vehicle interior. [Diagram 5] 4 is a diagram showing the three-dimensional space image shown in FIG. 3 as viewed from the front of the vehicle interior. [Figure 6] FIG. 11 is a functional configuration diagram of a system having an audio device according to a second embodiment of the present invention. [Figure 7] 7 is a flowchart of the operation of the audio device shown in FIG. 6. [Figure 8] 10 is a flowchart showing the operation of an audio device according to a third embodiment of the present invention. [Figure 9] FIG. 11 is a diagram showing an example in which people are identified and displayed. [Figure 10] FIG. 13 is an explanatory diagram of an operation of following a person. [Figure 11] 10 is a flowchart showing the operation of an audio device according to a fourth embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] An acoustic device according to an embodiment of the present invention will be described below. In the acoustic device according to an embodiment of the present invention, an acquisition unit acquires a three-dimensional spatial image of a predetermined space generated based on three-dimensional information acquired from a sensor capable of acquiring three-dimensional information of the predetermined space, and an output unit outputs the three-dimensional spatial image acquired by the acquisition unit to a display unit in order to receive a designation of a specific position from a reception unit capable of receiving the designation of a specific position. In this way, it is possible to designate a specific position for localizing a sound image based on the three-dimensional spatial image displayed on the display unit, and it is possible to easily designate an accurate localization position by the three-dimensional spatial image that reproduces the predetermined space.

[0011] The three-dimensional space image may be updated sequentially, and the output unit may output the updated three-dimensional space image. In this way, it is possible to specify a specific position based on the three-dimensional space image corresponding to the entrance and exit of people and changes in their positions within a given space.

[0012] The apparatus may further include an identification unit that identifies a person in the three-dimensional space image acquired by the acquisition unit, the output unit outputs the three-dimensional space image in which the person is identified, the reception unit is capable of designating the person, and the localization control unit may localize the sound image at a position corresponding to the designated person when the person is designated by the reception unit. In this way, for example, when a person is seated in a seat in the passenger compartment of a vehicle, it is possible to localize the sound image at a position corresponding to the person by designating the person.

[0013] The identification unit may detect a change in the position of the person's head from the multiple three-dimensional space images, and the localization control unit may, when a person is specified by the reception unit, make the localization position of the sound image follow the head based on the change in the position of the head detected by the identification unit. By doing so, even if the direction of the face or the angle of the seat, for example, changes, the localization position of the sound image can be made to follow them accordingly.

[0014] The sensor may be capable of measuring the distance to an object by emitting electromagnetic waves into a predetermined space and receiving the electromagnetic waves reflected by the object in the predetermined space. In this way, a three-dimensional space image can be generated based on point cloud information acquired by a so-called LiDAR (Light Detection And Ranging).

[0015] In addition, the acoustic device may be installed in a vehicle, and the predetermined space may be the vehicle interior space. In this way, the sound image in the vehicle interior space can be localized at a specific position.

[0016] In addition, in a terminal device according to an embodiment of the present invention, an acquisition unit acquires a three-dimensional spatial image of a predetermined space generated based on three-dimensional information acquired from a sensor capable of acquiring three-dimensional information of the predetermined space, and a display unit displays the three-dimensional spatial image acquired by the acquisition unit in order to receive a designation of a specific position from a reception unit capable of receiving a designation of a specific position in the predetermined space. Then, an output unit outputs the specific position received by the reception unit as information on a position for localizing a sound image formed by sounds emitted from multiple speakers. In this way, it is possible to designate a specific position for localizing a sound image based on the three-dimensional spatial image displayed on the display unit, and the three-dimensional spatial image reproducing the predetermined space makes it easy to designate an accurate localization position from the terminal device.

[0017] In addition, in an acoustic control method according to an embodiment of the present invention, an acquisition step acquires a three-dimensional spatial image of a predetermined space generated based on three-dimensional information acquired from a sensor capable of acquiring three-dimensional information of the predetermined space, and an output step outputs the three-dimensional spatial image acquired by the acquisition unit to a display unit in order to receive a designation of a specific position from a reception unit capable of receiving a designation of a specific position. In this way, it becomes possible to designate a specific position for localizing a sound image based on the three-dimensional spatial image displayed on the display unit, and it becomes possible to easily designate an accurate localization position by the three-dimensional spatial image that reproduces the predetermined space.

[0018] The above-mentioned acoustic control method may be executed by a computer, which makes it possible to specify a specific position for localizing a sound image based on a three-dimensional spatial image displayed on a display unit using the computer, and thus makes it easy to specify an accurate localization position. EXAMPLES

[0019] An audio device according to a first embodiment of the present invention will be described with reference to Figs. 1 to 5. The audio device 10 is mounted on a vehicle such as an automobile. Fig. 1 shows the functional configuration of an audio system having the audio device 10. The audio system includes an audio device 1, a delay adjustment unit 2 (2R, 2L), speakers 3 (3R, 3L), a control unit 4, a touch panel 5, a display unit 6, and a rider 7. In this embodiment, the audio system is mounted on the vehicle and controls the sound in the vehicle interior as a predetermined space.

[0020] The audio device 1 is a device that outputs an audio signal to be listened to by a user, such as a CD player, a memory audio device, a radio tuner, or a receiving device for streaming distribution.

[0021] In this embodiment, the delay adjustment unit 2 is composed of a delay adjustment unit 2L for the left channel and a delay adjustment unit 2R for the right channel. The delay adjustment unit 2L delays the left channel audio signal output from the audio device 1 to the left channel speaker 3L by a set delay time. The delay adjustment unit 2R delays the right channel audio signal output from the audio device 1 to the right channel speaker 3R by a set delay time.

[0022] The control unit 4 includes a generation unit 41 and a sound image control unit 42. The control unit 41 is configured by, for example, a microcomputer having a CPU. The generation unit 41 and the sound image control unit 42 may be realized as functions of a computer program executed by the CPU.

[0023] The generation unit 41 generates a three-dimensional space image by modeling the point cloud information output by the rider 7 using polygons or the like using a well-known method. That is, the control unit 4 functions as an acquisition unit that generates and acquires a three-dimensional space image based on the point cloud information. In addition, the generation unit 41 outputs the generated three-dimensional space image to the display unit 6 to display it on the display unit 6. That is, the control unit 4 (generation unit 41) functions as an output unit that outputs the three-dimensional space image to the display unit 6.

[0024] The sound image control unit 42 acquires the position coordinates touched on the touch panel 5, and identifies the three-dimensional coordinates (x, y, z) on the three-dimensional space image corresponding to the position coordinates. Then, the delay adjustment units 2L, 2R are adjusted and set so that the sound image formed by the sound emitted from the speakers 3L, 3R is localized at a position in real space corresponding to the identified three-dimensional coordinates. That is, the control unit 4 (sound image control unit 42) functions as a localization control unit that localizes the sound image in the vehicle cabin formed by the sound emitted from the speakers 3L, 3R at a specific position.

[0025] The touch panel 5 is disposed so as to overlap the display surface of the display unit 6. The touch panel 5 is a well-known input device that outputs two-dimensional coordinates of a position touched by, for example, a finger. The display unit 6 is configured, for example, with a liquid crystal display. The display unit 6 displays a three-dimensional spatial image generated by the control unit 4. In this embodiment, the touch panel 5 is disposed so as to overlap the display surface of the display unit 6, but the display unit 6 may be disposed on the dashboard and provided as a touchpad near the shift lever or on the steering wheel, etc., in such a manner that the touch panel 5 and the display unit 6 are disposed separately from each other.

[0026] The lidar 7 is installed in the vehicle interior or in a position where the vehicle interior can be scanned as a predetermined space. The lidar 7 is a sensor that recognizes objects present in the surroundings scanned by the lidar 7, and is also referred to as LiDAR (Light Detection And Ranging). The lidar 7 is a known sensor that irradiates electromagnetic waves such as laser light, and uses the reflected waves (reflected light) of the electromagnetic waves to discretely measure the direction and distance to an object present in the scanning range, and recognizes the position, shape, etc. of the object as a three-dimensional point cloud. Therefore, the point cloud recognized by the lidar 7 is output as point cloud information as three-dimensional information.

[0027] The number of the riders 7 is not limited to one, and may be multiple. The number and installation positions of the riders 7 may be appropriately determined according to the size of the target vehicle interior space, for example, in the case of a vehicle.

[0028] Here, the delay adjustment unit 2 and the control unit 4 described above function as the audio device 10 according to this embodiment.

[0029] Next, the operation of the acoustic device 10 having the above-mentioned configuration (acoustic control method) will be described with reference to Fig. 2 to Fig. 5. The flowchart shown in Fig. 2 is executed by the control unit 4. Moreover, the flowchart shown in Fig. 2 can be configured as a program executed by a CPU possessed by the control unit 4 to become an acoustic control program.

[0030] First, in step S11, the generation unit 41 acquires point cloud information from the LIDAR 7. Next, in step S12, the generation unit 41 generates a three-dimensional space image by modeling the point cloud information acquired from the LIDAR 7 using polygons or the like by a well-known method, as described above. When there are multiple LIDARs 7, the three-dimensional space image may be generated by combining multiple pieces of point cloud information. An example of a three-dimensional space image is shown in FIG. 3. FIG. 3 is a three-dimensional space image of the interior of the vehicle viewed from a viewpoint diagonally above and to the left front. The viewpoint shown in FIG. 3 is from a viewpoint diagonally above and to the left front, but the viewpoint can be changed to any viewpoint by operating the touch panel 5, such as by rotating, zooming in, or zooming out.

[0031] Next, in step S13, the generation unit 41 causes the display unit 6 to display the three-dimensional space image generated in step S12.

[0032] Next, in step S14, the sound image control unit 42 acquires (accepts) the designation of a specific position (hereinafter, simply referred to as a localization position) where the sound image is to be localized from the touch panel 5. Here, a method of specifying a coordinate position indicating the localization position from the three-dimensional space image displayed on the display unit 6 will be described. Even if the position of the driver's seat is touched and specified in the three-dimensional space image of the viewpoint shown in FIG. 3, since the display on the display unit 6 is two-dimensional, it is difficult to specify the accurate three-dimensional coordinates (x, y, z) as it is. Therefore, in this embodiment, first, the vehicle interior as shown in FIG. 4 is displayed from a viewpoint seen from the side, and the user is made to touch the touch panel 5 to specify the position of the localization position P in the depth direction and height direction of the vehicle, and then the vehicle interior as shown in FIG. 5 is displayed from a viewpoint seen from the front, and the user is made to touch the touch panel 5 to specify the position of the localization position P in the width direction of the vehicle. In this way, the three-dimensional coordinates of the specified localization position in the three-dimensional space image are specified.

[0033] The display order of Fig. 4 and Fig. 5 may be reversed. When the display order of Fig. 4 and Fig. 5 is reversed, the user is asked to specify the position of the localization position P in the width direction and height direction of the vehicle in the display of Fig. 5, and then the user is asked to specify the position of the localization position P in the depth direction of the vehicle in the display of Fig. 4. Furthermore, when Fig. 5 is displayed after the display of Fig. 4, the position in the height direction specified in Fig. 4 may be displayed so that it can be referred to when specifying in Fig. 5, or the position in the height direction of the localization position P may be re-specified when displaying Fig. 5.

[0034] Furthermore, the viewpoint may be a top view rather than a side view of the vehicle interior. In short, it is sufficient to be able to identify the position in at least two of the vehicle width, height, and depth directions.

[0035] That is, in step S13, the control unit 4 outputs a three-dimensional spatial image to the display unit 6 in order to receive a specification of a position from the touch panel 5 (reception unit) capable of receiving a specification of a specific position to position the sound image, and in step S14, receives the specification of the position from the touch panel 5.

[0036] Next, in step S15, the sound image control unit 42 adjusts the delay adjustment units 2L and 2R so that the sound image is localized at the localization position specified in step S14. Specifically, since the positions of the speakers 3L and 3R in the vehicle are known in advance, the position coordinates of the speakers 3L and 3R in the three-dimensional space image are also known. Then, the distance from the localization position specified in step S14 to the speaker 3L and the distance from the localization position specified in step S14 to the speaker 3R are calculated based on the respective coordinates. Then, the output from the speaker 3 with the shorter distance is delayed according to the difference in distance from the speaker 3 with the longer distance.

[0037] Next, in step S16, the sound image control unit 42 displays, for example, a message on the display unit 6 as to whether or not readjustment is necessary, and proceeds to step S17 if an input indicating that readjustment is necessary has been received from the touch panel 5 (if YES). On the other hand, if an input indicating that readjustment is not necessary has been received from the touch panel 5 (if NO), the flow chart ends.

[0038] Next, in step S17, it is determined whether or not regeneration of the three-dimensional space image is necessary, and if it is necessary (YES), the process returns to step S11, and if it is not necessary (NO), the process returns to step S14. In step S17, for example, the display unit 6 displays whether or not regeneration of the three-dimensional space image is necessary, and the user is prompted to select. If the user selects that it is necessary, the process returns to step S11, and the three-dimensional space image is generated again. If the user selects that it is not necessary, the process returns to step S14, and the localization position is acquired. In this way, the three-dimensional space image is updated successively, and the updated three-dimensional space image is output to the display unit 6 as necessary.

[0039] As is clear from the above description, step S12 functions as an acquisition step, and step S13 functions as an output step.

[0040] In the flowchart of Fig. 2, the three-dimensional space image is updated successively, but it may be generated once and not updated thereafter. The generated three-dimensional space image may be stored in a memory or the like in the control unit 4. The storage destination is not limited to the control unit 4, and may be, for example, a removable memory card, or a communication unit or the like may be provided to transmit the image to an external server or the like for storage. If the image is stored in the external server, when the next localization position is specified, steps S11 and S12 may be omitted, and a step of acquiring the three-dimensional space image from the external server may be executed, and then step S13 and subsequent steps may be executed.

[0041] According to this embodiment, the acoustic device 1 generates and acquires a three-dimensional spatial image of the vehicle interior generated by the control unit 4 based on point cloud information acquired from the rider 7, and outputs the three-dimensional spatial image to the display unit 6 to receive a designation of the localization position from the touch panel 5 capable of receiving the designation of the localization position. In this manner, it becomes possible to designate a specific position for localizing a sound image based on the three-dimensional spatial image displayed on the display unit 6, and the three-dimensional spatial image reproducing the vehicle interior space makes it easy to designate an accurate localization position.

[0042] The three-dimensional space image is updated successively, and the control unit 4 outputs the updated three-dimensional space image. In this way, it is possible to specify a fixed position based on the three-dimensional space image corresponding to the passenger getting on and off in the vehicle cabin. EXAMPLES

[0043] Next, an audio device 10A according to a second embodiment of the present invention will be described with reference to Figures 6 to 7. Note that the same parts as those in the first embodiment described above are given the same reference numerals and the description thereof will be omitted.

[0044] 6 shows the functional configuration of an audio system having an audio device 10A according to this embodiment. The audio system has an in-vehicle device 20, speakers 3 (3L, 3R), a rider 7, and a terminal device 30. Here, the speakers 3 (3L, 3R) and the rider 7 are the same as those in the first embodiment.

[0045] The vehicle-mounted device 20 is mounted on, for example, a vehicle. The vehicle-mounted device 20 includes an audio device 1, a delay adjustment unit 2 (2L, 2R), a control unit 4A, and a communication unit 8, as shown in FIG.

[0046] The audio device 1 and the delay adjustment section 2 are similar to those in the first embodiment. The control section 4A includes a generation section 41A and a sound image control section 42A.

[0047] The generation unit 41A generates a three-dimensional space image in the same manner as in the first embodiment, but differs from the first embodiment in that the generated three-dimensional space image is transmitted to the terminal device 30 via the communication unit 8.

[0048] The sound image control unit 42A is similar to the first embodiment in that it adjusts the delay amount of the delay adjustment units 2L, 2R so as to position the sound image formed by the sound emitted from the speakers 3L, 3R at a position in real space corresponding to the calculated three-dimensional coordinates. However, it differs from the first embodiment in that it obtains the position coordinates of the touch on the touch panel 33 from the terminal device 30 via the communication unit 8, and identifies the three-dimensional coordinates on the three-dimensional space image corresponding to those position coordinates.

[0049] The communication unit 8 transmits the three-dimensional space image generated by the generation unit 41A to the terminal device 30, and receives position coordinates of a touch on the touch panel 33 from the terminal device 30. The communication unit 8 communicates with the terminal device 30 by wireless communication such as Bluetooth (registered trademark), Wi-Fi (registered trademark), or infrared communication.

[0050] The terminal device 30 is a portable electronic device such as a smartphone, etc. The terminal device 30 includes a communication unit 31, a control unit 32, a touch panel 33, and a display unit .

[0051] The communication unit 31 transmits the position coordinates of the touched position on the touch panel 33 to the vehicle-mounted device 20, and receives from the vehicle-mounted device 20 the three-dimensional space image generated by the generation unit 41A.

[0052] The control unit 32 is responsible for the overall control of the terminal device 30. The control unit 32 causes the display unit 34 to display the three-dimensional space image received by the communication unit 31, obtains the position coordinates of the touch on the touch panel 33, and transmits the obtained coordinates to the vehicle-mounted device 20 via the communication unit 31.

[0053] The touch panel 33 is disposed so as to overlap the display surface of the display unit 34. The touch panel 33 is a well-known input device that outputs two-dimensional coordinates of a position touched by, for example, a finger or the like. The display unit 34 is composed of, for example, a liquid crystal display. On the display unit 34, a three-dimensional spatial image output from the control unit 32 and the like are displayed.

[0054] Here, the delay adjustment unit 2, the control unit 4, and the communication unit 8 described above function as the audio device 10A according to this embodiment.

[0055] The communication unit 31 of the terminal device 30 functions as an acquisition unit that acquires a three-dimensional spatial image of the vehicle interior generated based on the point cloud information acquired from the rider 7, and as an output unit that outputs the specific position accepted by the touch panel 33 as information on the position at which a sound image is to be localized. The display unit 34 functions as a display unit that displays the three-dimensional spatial image received by the communication unit 31 in order to accept the designation of the specific position from the touch panel 33 that can accept the designation of the specific position at which a sound image is to be localized within the vehicle interior.

[0056] Next, the operation (acoustic control method) of the audio device 10A configured as described above will be described with reference to Fig. 7. The flow chart shown in Fig. 7 is executed by the control unit 4A of the vehicle-mounted device 20 and the control unit 32 of the terminal device 30.

[0057] First, the operation of the vehicle-mounted device 20 will be described. In step S21, the control unit 4A acquires point cloud information from the LIDAR 7. Next, in step S22, the control unit 4A generates a three-dimensional space image from the point cloud information acquired from the LIDAR 7 by a well-known method.

[0058] Next, in step S23, the control unit 4A determines whether or not a three-dimensional space image has been requested by receiving request information for a three-dimensional space image from the terminal device 30, and if there has been no request (if NO), the process returns to step S21, and if there has been a request (if YES), in step S24, the control unit 4A causes the communication unit 8 to transmit the three-dimensional space image generated by the generation unit 41A to the terminal device 30. If there has been no request and the process returns to step S21 in step S23, the control unit 4A acquires point cloud information again (step S21) and regenerates the three-dimensional space image (step S22), thereby performing sequential updating.

[0059] Next, in step S25, the control unit 4A receives localization position information from the terminal device 30 via the communication unit 8. Then, in step S26, the control unit 4A adjusts the delay adjustment units 2L and 2R so that the sound image is localized at the localization position received in step S24, and returns to step S21. The adjustment method is the same as in the first embodiment.

[0060] Returning to step S21, the point cloud information is acquired again (step S21), and the three-dimensional space image is regenerated (step S22), thereby sequentially updating. If there is request information for the regenerated three-dimensional space image from the terminal device 30, steps S24 and after are executed, and if there is no request information for the three-dimensional space image, the process returns to step S21.

[0061] Next, the operation of the terminal device 30 will be described. In step S31, the control unit 32 causes the communication unit 31 to transmit request information for a three-dimensional space image to the vehicle-mounted device 20. Next, in step S32, the control unit 32 receives the three-dimensional space image from the vehicle-mounted device 20 via the communication unit 31.

[0062] Next, in step S33, the control unit 32 causes the display unit 34 to display the three-dimensional space image received in step S32.

[0063] Next, in step S34, the control unit 32 acquires (accepts) a designation of the localization position of the sound image from the touch panel 33. The method of acquiring the localization position is the same as in the first embodiment.

[0064] Next, in step S35, the control unit 32 causes the communication unit 31 to transmit the localization position acquired in step S34 to the vehicle-mounted device 20 as localization position information.

[0065] Next, in step S36, the control unit 32 causes the display unit 34 to display, for example, a message regarding whether or not readjustment is necessary, and if an input indicating that readjustment is necessary has been received from the touch panel 33 (if YES), the process proceeds to step S37. On the other hand, if an input indicating that readjustment is not necessary has been received from the touch panel 33 (if NO), the flow chart ends.

[0066] Next, in step S37, it is determined whether or not regeneration of the three-dimensional space image is necessary, and if it is necessary (YES), the process returns to step S31, and if it is not necessary (NO), the process returns to step S34. In step S37, for example, the display unit 6 displays whether or not regeneration of the three-dimensional space image is necessary, and the user is prompted to select it. If the user selects that it is necessary, the process returns to step S31 and a regenerated three-dimensional space image is requested. If the user selects that it is not necessary, the process returns to step S34 and the localization position is acquired.

[0067] According to this embodiment, the terminal device 30 acquires a three-dimensional spatial image of the vehicle interior generated by the control unit 32 based on the three-dimensional information acquired from the rider 7, and the display unit 34 displays the three-dimensional spatial image in order to receive a designation of a localization position from the touch panel 33 capable of receiving a designation of a localization position in the vehicle interior. Then, the communication unit 31 outputs the localization position information. In this manner, it becomes possible to designate a localization position for localizing a sound image based on the three-dimensional spatial image displayed on the display unit 34, and it becomes easy to designate an accurate localization position from the terminal device 30.

[0068] Furthermore, since the fixed position can be specified from the portable terminal device 30, the fixed position can be specified regardless of the installation location of the vehicle-mounted device 20. EXAMPLES

[0069] Next, an audio device 10 according to a third embodiment of the present invention will be described with reference to Figures 8 to 10. Note that the same parts as those in the first and second embodiments described above are given the same reference numerals and the description thereof will be omitted.

[0070] In this embodiment, the functional configuration is similar to that of the first embodiment, but the operation (acoustic control method) of the acoustic device 10 is partially different. The operation (acoustic control method) of the acoustic device 10 according to this embodiment will be described with reference to Fig. 8. The flowchart shown in Fig. 8 is executed by the control unit 4.

[0071] First, in step S41, the generation unit 41 acquires point cloud information from the LIDAR 7. Next, in step S42, the generation unit 41 generates a three-dimensional space image for the point cloud information acquired from the LIDAR 7, as described above.

[0072] In step S42, when generating the three-dimensional space image, the generation unit 41 recognizes people (passengers) in the vehicle cabin from the point cloud information by object recognition or the like, and is capable of identifying and displaying the people in the three-dimensional space image. That is, the control unit 4 (generation unit 41) functions as an identification unit that identifies people in the three-dimensional space image. An example of a three-dimensional space image in which people have been identified is shown in FIG. 9. FIG. 9 is an image in which people have been added to FIG. 6. As shown in FIG. 9, labels such as "A" and "B" are given to the people so that they can be identified. Note that, instead of labels, different colors, patterns, etc. may be added to each person.

[0073] Next, in step S43, the generation unit 41 causes the display unit 6 to display the three-dimensional space image generated in step S42.

[0074] Next, in step S44, the sound image control unit 42 acquires (accepts) the designation of the person for localizing the sound image from the touch panel 5. For example, in FIG. 8, the person is designated by touching either "A" or "B." In this embodiment, the position of the person touched is the center of the front of the face (head) regardless of whether the touched position is the head or the torso. The position coordinates of this localization position are automatically specified based on the position of the person in the three-dimensional space image. Also, the localization position automatically specified when a person is touched is not limited to the center of the front of the face, but may be another position such as the ear. Alternatively, for example, when the chest is touched, the chest may be the localization position.

[0075] Next, in step S45, the sound image control unit 42 adjusts the delay adjustment units 2L and 2R so that the sound image is localized at the head of the person specified in step S44. The adjustment method is the same as in the first embodiment. That is, when a person is specified, the control unit 4 (sound image control unit 42) localizes the sound image at a position corresponding to the specified person.

[0076] Next, in step S46, the generation unit 41 acquires point cloud information from the LIDAR 7. This step S46 is performed for the judgment of step S47. Then, in step S47, it is judged whether or not there is a change in the tracking position, and if there is a change, the process returns to step S45, and if there is no change, the process returns to step S46.

[0077] The tracking position in step S47 is the head of the person for which the sound image is localized in step S45. If the position of the person's head in the three-dimensional space image generated from the point cloud information acquired in step S46 has changed from the position of the person's head acquired in step S41 or the previously executed step S46, it is determined that the head has moved, and the process returns to step S45 to track the movement of the head. Then, the sound image is localized at the position of the person's head in the three-dimensional space image generated from the point cloud information acquired in step S46.

[0078] An example of tracking is shown in Figure 10. Figure 10 shows an example where a person sitting in a seat reclins the backrest to change the angle of the backrest. In the case of Figure 10, the position of the head changes due to the change in the seat angle, and the localization position P moves to localization position P'.

[0079] On the other hand, if the position of the person's head in the three-dimensional space image generated from the point cloud information acquired in step S46 has not changed from the position of the person's head acquired in step S41 or the previously executed step S46, it is determined that the head has not moved, step S45 is not executed, and point cloud information is acquired again in step S46.

[0080] In this embodiment, a three-dimensional space image may be generated (updated) and displayed on the display unit 6 based on the point cloud information acquired in step S46.

[0081] Also, the flow chart shown in Fig. 8 shows the operation when a person is specified, but when localizing a sound image between "A" and "B" in a three-dimensional space image as shown in Fig. 9, for example, the operation can be performed in the same manner as in the first embodiment. That is, depending on the touch position (specified position), if a person is touched, step S44 and subsequent steps are executed, and if something other than a person is touched, step S14 and subsequent steps are executed.

[0082] According to this embodiment, the generating unit 41 of the control unit 4 identifies a person in the three-dimensional space image and outputs the three-dimensional space image in which the person is identified. In addition, the touch panel 5 can specify a person, and when a person is specified on the touch panel 5, the delay adjustment unit 2 localizes a sound image at a position corresponding to the specified person. In this way, for example, when a person is seated in a seat in the passenger compartment of a vehicle, it is possible to localize a sound image at a position corresponding to the person by specifying the person.

[0083] Furthermore, the control unit 4 detects changes in the position of the person's head from the multiple three-dimensional space images, and when a person is specified on the touch panel 5, the delay adjustment unit 2 causes the localization position of the sound image to follow the head based on the change in the head position detected by the control unit 4. By doing so, even if the direction of the face or the angle of the seat, for example, changes, the localization position of the sound image can be made to follow them accordingly. EXAMPLES

[0084] Next, an audio device 10A according to a fourth embodiment of the present invention will be described with reference to Fig. 11. Note that the same parts as those in the first to third embodiments described above are given the same reference numerals and the description thereof will be omitted.

[0085] In this embodiment, the functional configuration is similar to that of the second embodiment, but the operation (acoustic control method) of the audio device 10A is partially different. The operation (acoustic control method) of the audio device 10A according to this embodiment will be described with reference to Fig. 11. The flow chart shown in Fig. 11 is executed by the control unit 4A of the vehicle-mounted device 20 and the control unit 32 of the terminal device 30.

[0086] First, the operation of the vehicle-mounted device 20 will be described. In step S51, the control unit 4A acquires point cloud information from the LIDAR 7. Next, in step S52, the control unit 4A generates a three-dimensional space image by a well-known method. Note that the three-dimensional space image generated in step S52 can identify and display people, similar to the third embodiment.

[0087] Next, in step S53, the control unit 4A determines whether or not a request for a three-dimensional space image has been made by receiving request information for a three-dimensional space image from the terminal device 30, and if there is no request (NO), it waits in this step, and if there is a request (YES), in step S54, it causes the communication unit 8 to transmit the three-dimensional space image generated by the generation unit 41A to the terminal device 30.

[0088] Next, in step S55, the control unit 4A receives person designation information, which is information designating a person to localize a sound image, from the terminal device 30. Next, the delay adjustment units 2L and 2R are adjusted so that the sound image is localized at the head of the person indicated by the person designation information received in step S55. The adjustment method is the same as in the first and third embodiments.

[0089] Next, in step S57, the generation unit 41 acquires point cloud information from the LIDAR 7. This step S57 is performed for the judgment of step S58. Then, in step S58, it is judged whether or not there is a change in the tracking position, and if there is a change, the process returns to step S56, and if there is no change, the process returns to step S57. The tracking method in step S58 is the same as in the third embodiment.

[0090] In this embodiment as well, a three-dimensional space image may be generated (updated) and displayed on the display unit 6 based on the point cloud information acquired in step S57.

[0091] Next, the operation of the terminal device 30 will be described. In step S61, the control unit 32 causes the vehicle-mounted device 20 to transmit request information for a three-dimensional space image to the communication unit 31. Next, in step S62, the three-dimensional space image is received from the vehicle-mounted device 20 via the communication unit 31.

[0092] Next, in step S63, the control unit 32 causes the display unit 34 to display the three-dimensional space image received in step S62.

[0093] Next, in step S64, the control unit 32 acquires (accepts) the designation of the person for which the sound image is to be localized from the touch panel 5. The method of acquiring the designation of the person is the same as in the third embodiment.

[0094] Next, in step S65, the control unit 32 causes the communication unit 31 to transmit the person designated in step S34 to the vehicle-mounted device 20 as person designation information.

[0095] Next, in step S66, the control unit 32 causes the display unit 34 to display, for example, a message regarding whether or not readjustment is necessary, and if an input indicating that readjustment is necessary has been received from the touch panel 33 (if YES), the process returns to step S64. On the other hand, if an input indicating that readjustment is not necessary has been received from the touch panel 33 (if NO), the flow chart ends.

[0096] In this embodiment as well, if there is an input indicating that readjustment is necessary in step S66, it may be determined whether or not the three-dimensional space image needs to be regenerated.

[0097] According to this embodiment, the terminal device 30 acquires a three-dimensional space image in which a person in the vehicle cabin is identified, which is generated by the control unit 32 based on the three-dimensional information acquired from the rider 7, and the display unit 34 displays the three-dimensional space image in order to receive designation of the person to be localized for the sound image from the touch panel 33 capable of receiving designation of the localization position in the vehicle cabin. Then, the communication unit 31 outputs person designation information. In this manner, for example, if a person is seated in a seat in the vehicle cabin, it is possible to localize a sound image at a position corresponding to the person by designating the person from the terminal device 30.

[0098] Furthermore, the control unit 4A detects changes in the position of the person's head from multiple three-dimensional space images, and when a person is specified on the touch panel 5, the delay adjustment unit 2 causes the localization position of the sound image to follow the head based on the change in the head position detected by the control unit 4A. By doing so, even if the direction of the face or the angle of the seat, for example, changes, the localization position of the sound image can be made to follow them accordingly.

[0099] In the above embodiment, the sensor is a lidar, but other sensors may be used, such as a stereo camera, as long as the sensor can obtain three-dimensional information for generating a three-dimensional spatial image.

[0100] In the above embodiment, the predetermined space is described as a vehicle cabin, but it may be an indoor space in a building. In addition, the above embodiment is described as a two-channel sound system having a right channel and a left channel, but the present invention can be applied to a sound system that localizes a sound image using multiple speakers, such as a four-channel sound system consisting of four speakers.

[0101] Furthermore, the present invention is not limited to the above-mentioned embodiment. In other words, a person skilled in the art can implement the present invention by modifying it in various ways in accordance with the conventionally known knowledge without departing from the gist of the present invention. As long as the acoustic device of the present invention is still provided even after such modification, it is of course included in the scope of the present invention. [Explanation of symbols]

[0102] 2 Delay adjustment section (localization control section) 3 Speakers 4. Control section (acquisition section, output section, identification section) 4A Control section (acquisition section, identification section) 5. Touch panel (reception area) 6 Display section 7. Rider 8 Communication section (output section) 10, 10A sound equipment 20 Onboard equipment 30 Terminal Equipment 31 Communication Department (Acquisition Department) 32 Control section (output section) 33 Touch panel (reception area) 34 Display section

Claims

1. an acquisition unit that acquires a three-dimensional space image of a predetermined space generated based on three-dimensional information acquired from a sensor; an output unit that outputs the three-dimensional space image acquired by the acquisition unit to a display unit; a reception unit that receives a designation of a specific position in the three-dimensional space image output to the display unit, the output unit outputs, to the display unit, at least two images in which two of a width direction, a height direction, and a depth direction in the predetermined space can be identified based on the three-dimensional spatial image; the receiving unit receives designation of the specific position for each of at least two images output to the display unit; A first image of the at least two images output to the display unit is displayed on the display unit; after the reception unit receives the designation of the specific position for the first image, a second image is displayed on the display unit in place of the first image; accepting, by the accepting unit, a designation of the specific position on the second image; An input device comprising:

2. At least two of the images output to the display unit are The image is an image in which one of the two directions constituting the image is different from the other.

2. The input device according to claim 1 .

3. the output unit outputs a selection regarding regeneration of the three-dimensional spatial image to the display unit; regenerating the three-dimensional spatial image in response to the selection made by the user.

3. The input device according to claim 1 or 2.

4. An input device according to any one of claims 1 to 3, characterized in that of the at least two images output to the display unit, one image specifies a position in two of the width, height, and depth directions in the specified space, and the other image specifies a position in the remaining direction different from the two directions.

5. 5. The input device according to claim 1, further comprising a localization control unit that localizes a sound image formed by sounds emitted from a plurality of speakers at the specific position in the predetermined space.

6. The input device according to claim 1 , wherein the accepting unit is capable of directly accepting designation of the specific positions for at least two of the images outputted to the display unit.

7. an acquisition unit that acquires a three-dimensional space image of a predetermined space generated based on three-dimensional information acquired from a sensor; an output unit that outputs the three-dimensional space image acquired by the acquisition unit to a display unit; a reception unit that receives a designation of a specific position in the three-dimensional space image output to the display unit, the output unit outputs the three-dimensional spatial image to the display unit, and then sequentially outputs to the display unit two images in which two directions among a width direction, a height direction, and a depth direction in the predetermined space can be identified based on the three-dimensional spatial image; Of the two images, a first image and a second image are different from each other in one of the two directions; the first image is displayed on the display unit, and the reception unit receives a designation of the specific position for the first image, and then the second image is displayed on the display unit in place of the first image, and the reception unit receives the designation of the specific position for the second image.

1. An input device comprising:

8. the output unit outputs a selection regarding regeneration of the three-dimensional spatial image to the display unit; regenerating the three-dimensional spatial image in response to the selection made by the user.

8. The input device according to claim 7.

9. The input device according to claim 7 or 8, characterized in that, of the at least two images output to the display unit, one image specifies a position in two of the width, height, and depth directions in the specified space, and the other image specifies a position in the remaining direction different from the two directions.

10. 10. The input device according to claim 7, further comprising a localization control unit that localizes a sound image formed by sounds emitted from a plurality of speakers at the specific position in the predetermined space.

11. 11. The input device according to claim 7, wherein the accepting unit is capable of directly accepting designation of the specific positions for at least two of the images outputted to the display unit.

Images