Processing apparatus, processing method, and processing program

Abstract

To provide a processing device capable of generating sound data that allows a listener to easily perceive the direction of a sound.SOLUTION: A processing device including a head-related transfer function processing unit configured to apply a head-related transfer function corresponding to an angle of a virtual sound source with respect to a listening position to monaural sound source data to generate stereo first sound data, a volume panning processing unit configured to perform volume panning processing on the first sound data based on the angle of the virtual sound source to generate stereo second sound data, and a correction processing unit configured to correct a delay amount and a volume of the second sound data based on arrangement information of a plurality of sound output devices to generate stereo third sound data to be output by two of the plurality of sound output devices.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a processing device, a processing method, and a processing program. [Background technology]

[0002] Patent document 1 describes a signal processing device that includes a rendering method selection unit that selects one or more rendering processing methods for localizing the sound image of an audio signal within a listening space from a plurality of methods including panning processing such as VBAP, and a rendering processing unit that performs rendering processing of the audio signal using the method selected by the rendering method selection unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 116890 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with VBAP, it can be difficult to instantly perceive the correct direction of a sound, for example, as if the sound is coming from the front even though it is actually coming from a speaker located at the rear. Also, because VBAP is based on a concentric speaker arrangement, in environments where multiple speakers are not arranged concentrically around the listener, such as the interior of a car, it can be difficult for listeners to perceive the direction of the sound. [Means for solving the problem]

[0005] One aspect of the processing apparatus according to the present invention is a head-related transfer function processing unit that applies a head-related transfer function according to an angle of a virtual sound source with respect to a listening position to the monaural sound source data to generate first stereo sound data; a volume panning processing unit that performs volume panning processing on the first sound data based on the angle of the virtual sound source to generate second stereo sound data; and a correction processing unit that corrects the delay amount and volume of the second sound data based on arrangement information of the plurality of sound output devices and generates stereo third sound data to be output from two of the plurality of sound output devices.

[0006] One aspect of the processing method according to the present invention is to a head-related transfer function processing step of applying a head-related transfer function according to an angle of a virtual sound source relative to a listening position to the monaural sound source data to generate stereo first sound data; a volume panning process for performing a volume panning process on the first sound data based on the angle of the virtual sound source to generate second stereo sound data; and a correction processing step of correcting the delay amount and volume of the second sound data based on arrangement information of the plurality of sound output devices, and generating stereo third sound data to be output from two of the plurality of sound output devices.

[0007] One aspect of the processing program according to the present invention is a head-related transfer function processing step of applying a head-related transfer function according to an angle of a virtual sound source relative to a listening position to the monaural sound source data to generate stereo first sound data; a volume panning process for performing a volume panning process on the first sound data based on the angle of the virtual sound source to generate second stereo sound data; correcting the delay amount and volume of the second sound data based on arrangement information of a plurality of sound output devices; and a correction processing step of generating stereo third sound data to be output from two of the plurality of sound output devices. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of a processing apparatus according to a first embodiment. [Figure 2] FIG. 4 is a diagram for explaining specific processing by a head-related transfer function processing unit. [Figure 3] FIG. 4 is a diagram for explaining specific processing by a volume panning processing unit. [Figure 4] FIG. 4 is a diagram for explaining specific processing by a correction processing unit. [Figure 5] FIG. 3 is a flowchart showing the procedure of the processing method according to the first embodiment. [Figure 6] FIG. 6 is a flowchart showing an example of the procedure of the head-related transfer function processing step in FIG. 5. [Figure 7] FIG. 6 is a flowchart showing an example of the procedure of the volume panning process in FIG. 5. [Figure 8] FIG. 6 is a flowchart showing an example of the procedure of the correction processing step in FIG. 5. [Figure 9] FIG. 10 is a diagram showing an example of the arrangement of a processing apparatus according to a second embodiment. [Figure 10] FIG. 2 is a diagram showing an example of the arrangement of a virtual sound source and a virtual noise source. [Figure 11] FIG. 10 is a diagram showing another example of the arrangement of the virtual sound source and the virtual noise source. [Figure 12] FIG. 10 is a flowchart showing the procedure of a processing method according to a second embodiment. [Figure 13] FIG. 13 is a flowchart showing an example of the procedure of the second head-related transfer function processing step of FIG. 12. [Figure 14] FIG. 13 is a flowchart showing an example of the procedure of the second volume panning processing step in FIG. 12. [Figure 15] FIG. 13 is a flowchart showing an example of the procedure of the correction processing step in FIG. 12. [Figure 16] FIG. 10 is a diagram showing an example of the arrangement of a processing apparatus according to a third embodiment. [Figure 17] FIG. 10 is a flowchart showing the procedure of a processing method according to a third embodiment. [Figure 18] FIG. 18 is a flowchart showing an example of the procedure of the head-related transfer function processing step in FIG. 17. [Figure 19] FIG. 10 is a diagram showing an example of the configuration of a processing apparatus according to a fourth embodiment. [Figure 20] FIG. 2 is a functional block diagram of the electronic device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Note that the embodiments described below do not unduly limit the content of the present invention as defined in the claims. Furthermore, not all of the configurations described below are necessarily essential components of the present invention.

[0010] 1. Processing equipment 1-1. First embodiment 1-1-1. Processing equipment configuration Fig. 1 is a diagram showing an example of the configuration of a processing device according to a first embodiment. As shown in Fig. 1, the processing device 1 according to the first embodiment is a device that outputs sound signals DOX1 to DOXn to n sound output devices 3-1 to 3-n, respectively. n is an integer equal to or greater than 2. The sound output devices 3-1 to 3-n may be speakers, for example.

[0011] The processing device 1 of the first embodiment includes a control unit 10, a sound source memory 20, a head-related transfer function processing unit 30, a volume panning processing unit 40, a correction processing unit 50, n amplifiers 60-1 to 60-n, and a storage unit 70. The processing device 1 may be a single-chip semiconductor integrated circuit device, or may be configured with multiple semiconductor integrated circuit devices, or at least a portion of the processing device 1 may be configured with electronic components other than a semiconductor integrated circuit device. For example, the control unit 10, the head-related transfer function processing unit 30, the volume panning processing unit 40, the correction processing unit 50, and the storage unit 70 may be realized as a single-chip semiconductor integrated circuit device, and the sound source memory 20 and the amplifiers 60-1 to 60-n may be provided outside the semiconductor integrated circuit device. Note that the processing device 1 may be configured by omitting or modifying some of the components shown in FIG. 1, or by adding other components. For example, the amplifiers 60-1 to 60-n do not need to be components of the processing device 1.

[0012] The sound source memory 20 stores m pieces of monaural sound source data 21-1 to 21-m, where m is an integer equal to or greater than 1. The sound source memory 20 may be, for example, a flash memory. Each of the sound source data 21-1 to 21-m may be, for example, pulse code modulated (PCM) audio data or adaptive differential pulse code modulated (ADPCM) audio data. PCM stands for Pulse Code Modulation, and ADPCM stands for Adaptive Differential Pulse Code Modulation. The sound source data 21-1 to 21-m is data that serves as the basis for various sounds, such as warning sounds, sound effects, and sounds that imitate the sound of a person speaking.

[0013] The storage unit 70 is composed of RAM, ROM, registers, etc. ROM stands for Read Only Memory. The storage unit 70 may include a non-volatile memory.

[0014] The control unit 10 is a circuit that performs data communication with the microcontroller unit 2 and controls the operation of each part of the processing device 1. The control unit 10 may include, for example, an SPI interface circuit or an I2C interface circuit. SPI stands for Serial Peripheral Interface, and I2C stands for Inter-Integrated Circuit.

[0015] The control unit 10 receives various commands transmitted from the micro control unit 2 and generates various control signals in response to the received commands. For example, when the control unit 10 receives a command to write data to the storage unit 70, the control unit 10 generates a control signal for writing the specified data to an address specified by the command. Also, for example, when the control unit 10 receives an audio playback command for the sound source data 21-i among the sound source data 21-1 to 21-m stored in the sound source memory 20, the control unit 10 generates a control signal instructing audio playback for the sound source data 21-i. As a result, the sound source data 21-i is output from the sound source memory 20 as sound source data SDI and input to the head-related transfer function processing unit 30. Note that, if necessary, the sound source data SDI obtained by decoding the sound source data 21-i by a decoder (not shown) may be input to the head-related transfer function processing unit 30. Also, the processing device 1 may include a sound source data SDI that generates the sound source data SDI in real time.

[0016] The head-related transfer function processing unit 30 adds an angle θ of the virtual sound source VS to the listening position to the monaural sound source data SDI. S The head-related transfer function according to the angle θ is applied to generate the first stereo sound data. S is set based on an instruction from the micro control unit 2, for example, and is output from the control unit 10. The first sound data is made up of right sound data SDR and left sound data SDL.

[0017] 2 is a diagram for explaining specific processing by the head-related transfer function processing unit 30. As shown in FIG. 2, at a listening position P U With respect to angle θ S In this embodiment, it is assumed that a virtual sound source VS that emits sound according to the sound source data SDI exists in the direction of the angle θ S is the counterclockwise angle seen from above the listener U, with the front of the listener U being 0°. Therefore, as shown in Figure 2, the angle θ S If the angle is 45°, the virtual sound source VS is located in front of the listener U on the left, and the angle θ SIf the angle is 240°, the virtual sound source VS is located to the right rear of the listener U. The head-related transfer function h R (θ S ) is the angle θ S It represents the frequency characteristics of the sound emitted from the virtual sound source VS in the direction of the sound until it reaches the right ear of the listener U. Similarly, the head-related transfer function h L (θ S ) is the angle θ S It represents the frequency characteristics of the sound emitted from the virtual sound source VS in the direction of the listener U until it reaches the left ear of the listener U. For example, sounds from behind the listener U are blocked by the ear and the high frequency components are greatly attenuated, while sounds from in front of the listener U have only certain frequency bands emphasized or weakened depending on the shape of the listener U's ear. The head-related transfer function h R (θ S ),h L (θ S ) is a transfer function that represents such characteristics. The head-related transfer function processing unit 30 calculates the angle θ S Assuming that a virtual sound source VS in the direction of emits sound according to the sound source data SDI, the head-related transfer function h R (θ S ) to generate right sound data SDR, and the head-related transfer function h L (θ S ) to generate the left sound data SDL.

[0018] Returning to the explanation of FIG. 1, the volume panning processing unit 40 adjusts the angle θ of the virtual sound source VS. S Based on this, the head-related transfer function processing unit 30 performs volume panning processing on the generated first sound data to generate n-channel sound data SD1 to SDn including stereo second sound data. The second sound data is any two of the sound data SD1 to SDn, and the angle θ S and the location information 71 of the sound output devices 3-1 to 3-n. The location information 71 is, for example, U The information is stored in advance in the storage unit 70. However, the information is stored in advance in the storage unit 70. UWhen the difference between the listening position P and the listening position P is detected by a sensor (not shown), the microcontroller unit 2 U The position information 71 may be generated based on the position information 71 and transmitted to the control unit 10, or the microcontrol unit 2 may generate the position information 71 based on the position information 71 and transmit it to the control unit 10. U The information is transmitted to the control unit 10, and the control unit 10 determines the listening position P U The placement information 71 may be updated accordingly.

[0019] FIG. 3 is a diagram for explaining the specific processing of the volume panning processing unit 40. There are several types of volume panning, but in the following, the volume panning will be described as being VBAP. VBAP is an abbreviation for Vector-Based Amplitude Panning. As shown in FIG. 3, at a listening position P U angle θ S If a virtual sound source VS exists in the direction of the vector a, the sound emitted by the virtual sound source VS is VS It is expressed as a vector a VS The direction of is the listening position P U corresponds to the direction from the virtual sound source VS, and the vector a VS The length of the virtual sound source VS corresponds to the volume of the sound emitted by the virtual sound source VS. U When we consider reproducing the sound emitted by the virtual sound source VS using the right virtual sound output device VSR and the left virtual sound output device VSL that are concentrically arranged around the virtual sound source VS, the vector a VS is a vector a VSR and vector a VSL The vector a VSR The direction of is the listening position P U corresponds to the direction from the right virtual sound output device VSR, and the vector a VSR The length of the vector a corresponds to the volume emitted by the right virtual sound output device VSR. VSL The direction of is the listening position P U corresponds to the direction from the left virtual sound output device VSL, and the vector a VSL The length of the vector a corresponds to the volume emitted by the left virtual sound output device VSL. VSR ,a VSLThe right virtual sound output device VSR and the left virtual sound output device VSL in each direction are assigned vectors a VSR ,a VSL By generating sounds corresponding to the lengths of the virtual sound source VS, the sound emitted by the virtual sound source VS can be reproduced.

[0020] The volume panning processor 40 first calculates the angle θ S and the sound output devices 3-1 to 3-n, based on the arrangement information 71 of the sound output devices 3-r and 3-l that generate stereo sound. r and l are each an integer between 1 and n. For example, the volume panning processing unit 40 selects two sound output devices 3-r and 3-l that generate stereo sound based on the arrangement information 71. U The direction of the sound output devices 3-1 to 3-n relative to the S With respect to the direction of the virtual sound source VS determined by (1), the sound output device 3-r closest to the right and the sound output device 3-l closest to the left are selected.

[0021] Next, the volume panning processing unit 40 calculates a vector a representing the sound emitted by the virtual sound source VS in accordance with the sound source data SDI, assuming that the right virtual sound output device VSR and the left virtual sound output device VSL exist in two directions of the sound output devices 3-r and 3-l. VS Two vectors a VSR ,a VSL Then, the volume panning processing unit 40 calculates the vector a VSR Set the length of the vector a to the volume of the right sound data SDR. VSL The length of the second sound data SDR and the left sound data SDL is set to the volume of the left sound data SDL. The right sound data SDR is sound data SDr of the rth channel corresponding to the sound output device 3-r, and the left sound data SDL is sound data SDl of the lth channel corresponding to the sound output device 3-l. The volume panning processing unit 40 sets the volume of all the sound data SD1 to SDn of the n channels other than the sound data SDr and SDl to zero.

[0022] Returning to the explanation of FIG. 1, the correction processing unit 50 corrects the delay amount and volume of the sound data SDr, SDl, which are the second sound data generated by the volume panning processing unit 40, based on the arrangement information 71 of the sound output devices 3-1 to 3-n, and generates n-channel sound data DO1 to DOn including stereo third sound data to be output to two sound output devices 3-r, 3-l of the sound output devices 3-1 to 3-n. The third sound data is two of the sound data DO1 to DOn: sound data DOr of the r-th channel corresponding to the sound output device 3-r and sound data DOl of the l-th channel corresponding to the sound output device 3-l. As described above, the VBAP performed by the volume panning processing unit 40 is performed by adjusting the sound output positions of the right virtual sound output device VSR and the left virtual sound output device VSL at the listening position P U This processing is based on the premise that the sound output devices 3-1 to 3-n are arranged concentrically around the listening position P U The virtual sound sources VS1 to VSn are not necessarily arranged concentrically around the center. Therefore, the volume panning processing unit 40 selects any two of the concentric virtual sound sources VS1 to VSn corresponding to the sound output devices 3-1 to 3-n, respectively, as the right virtual sound output device VSR and the left virtual sound output device VSL, and applies VBAP. Therefore, the correction processing unit 50 generates sound data DOr and DOl by correcting the delay amount and volume of the sound data SDr and SDl.

[0023] 4 is a diagram for explaining the specific processing of the correction processing unit 50. In FIG. U The positions of the virtual sound source VS, the right virtual sound output device VSR, and the left virtual sound output device VSL relative to the listener U are the same as in Fig. 3. In Fig. 4, four sound output devices 3-1 to 3-4 are arranged around the listener U, but the sound output devices 3-1 to 3-4 are located at the listening position P U Therefore, the volume panning processing unit 40 is not arranged concentrically with respect to the listening position P U Concentrically with the sound output device 3-3 farthest from the SAssume that a virtual sound source VS is placed in the direction of the sound output device 3-1, a right virtual sound output device VSR is placed in the direction of the sound output device 3-1, and a left virtual sound output device VSL is placed in the direction of the sound output device 3-2, and sound data SD1 and SD2 are generated as sound data SDr and SDl. U The sound output devices 3-1 and 3-2 are not arranged concentrically with the right virtual sound output device VSR and the left virtual sound output device VSL, respectively, at a listening position P U Therefore, if sounds corresponding to the sound data SD1 and SD2 are output from the sound output devices 3-1 and 3-2, respectively, the listener U will not perceive the sounds as if they are coming from the virtual sound source VS.

[0024] First, the correction processing unit 50 calculates the position of the listening position P based on the arrangement information 71. U Next, the correction processing unit 50 obtains the distances d1 to d4 from the sound source 10 to the sound output devices 3-1 to 3-4, respectively. max The correction processing unit 50 corrects the delay amount of the sound data SD1 to SD4 in a direction to increase the delay amount in accordance with the ratio of the maximum distance d max and each of the distances d1 to d4, the correction processing unit 50 corrects the volume of the sound data SD1 to SD4 in a decreasing direction. Then, the correction processing unit 50 outputs the sound data SD1 to SD4 with the delay amount and volume corrected as sound data DO1 to DO4. When sounds corresponding to this sound data DO1 and DO2 are output from the sound output devices 3-1 and 3-2, respectively, the listener U can perceive as if the sounds are being emitted from the virtual sound source VS. On the other hand, since the volume of the sound data SD3 and SD4 before correction is set to zero, the volume of the sound data DO3 and DO4 also becomes zero, and the sounds corresponding to the sound data DO3 and DO4 are not output from the sound output devices 3-3 and 3-4.

[0025] It is to be noted that the correction processing unit 50 can easily perform correction in a direction to reduce the delay amount of the second sound data. Since there is no listening position P U Distance d from the farthest sound output device 3-j maxOn the other hand, since the correction processing unit 50 can easily correct the volume of the second sound data in either a decreasing or increasing direction, it is preferable to correct the volume of the second sound data in either a decreasing or increasing direction based on the listening position P U Distance d from the farthest sound output device 3-j max The volume of the second sound data may be corrected to a lower value based on the listening position P U distance d from the nearest sound output device 3-i min In the latter case, for example, the volume panning processing unit 40 may adjust the volume of the second sound data in a direction to increase the volume of the second sound data based on the listening position P U distance d from the nearest sound output device 3-i min The volume of the second sound data can be set based on this.

[0026] Returning to the explanation of Fig. 1, the amplifiers 60-1 to 60-n convert the n-channel sound data DO1 to DOn output from the correction processing unit 50 into sound signals DOX1 to DOXn and output them to the sound output devices 3-1 to 3-n, respectively. As a result, sounds corresponding to the sound signals DOX1 to DOXn are output from the sound output devices 3-1 to 3-n, respectively. The sounds output from the sound output devices 3-1 to 3-n may be various sounds such as warning sounds or sound effects, or may be sounds that imitate the sound of a person speaking.

[0027] 1, the sound source data 21-1 to 21-m are stored in a sound source memory 20 inside the processing device 1, but they may be stored in a memory external to the processing device 1. Alternatively, the sound source data 21-1 to 21-m may be stored in a memory built into the micro control unit 2, and the micro control unit 2 may transmit the sound source data SDI to the processing device 1 together with an audio playback command.

[0028] 1-1-2. Processing procedure 5 is a flow chart showing the procedure of the processing method of the first embodiment. The processing method of the first embodiment is executed by the processing apparatus 1 of the first embodiment shown in FIG.

[0029] As shown in FIG. 5, first, in the head-related transfer function processing step S1, the processing device 1 adds a head-related transfer function (HRTF) to the monaural sound source data SDI. U Angle θ of virtual sound source VS with respect to S This head-related transfer function processing step S1 is executed by the head-related transfer function processing unit 30 of the processing device 1.

[0030] Next, in the volume panning process S2, the processing device 1 calculates the angle θ of the virtual sound source VS. S Based on this, volume panning processing is performed on the first sound data generated in the head-related transfer function processing step S1 to generate second stereo sound data. This volume panning processing step S2 is executed by the volume panning processing unit 40 of the processing device 1.

[0031] Finally, in the correction process step S3, the processing device 1 corrects the delay amount and volume of the second sound data generated in the volume panning process step S2 based on the arrangement information 71 of the sound output devices 3-1 to 3-n, and generates stereo third sound data to be output from two of the sound output devices 3-1 to 3-n. U to the farthest sound output device 3-j among the sound output devices 3-1 to 3-n, d max The processing device 1 may correct the delay amount of the second sound data based on the listening position P U to the farthest sound output device 3-j among the sound output devices 3-1 to 3-n, d max or the distance d to the nearest sound output device 3-i min The volume of the second sound data may be corrected based on the above-mentioned correction processing step S3. This correction processing step S3 is executed by the correction processing unit 50 of the processing device 1.

[0032] Fig. 6 is a flowchart showing an example of the procedure of the head-related transfer function processing step S1 in Fig. 5. As shown in Fig. 6, first, in step S11, the processing device 1 calculates the angle θ of the virtual sound source VS. S Next, in step S12, the processing device 1 obtains the right Head-related transfer function to the ears h R (θ S ) to generate right sound data SDR. In step S13, the processing device 1 applies a head-related transfer function h L (θ S ) to generate left sound data SDL. Finally, in step S14, the processing device 1 generates first sound data consisting of the right sound data SDR and the left sound data SDL.

[0033] Fig. 7 is a flowchart showing an example of the procedure of the volume panning process step S2 in Fig. 5. As shown in Fig. 7, first, in step S21, the processing device 1 calculates the angle θ of the virtual sound source VS. S and acquires the arrangement information 71 of the sound output devices 3-1 to 3-n. Next, in step S22, the processing device 1 calculates the angle θ of the virtual sound source VS. S and the arrangement information 71, the processing device 1 selects two sound output devices 3-r, 3-l that will output stereo sound. Next, in step S23, the processing device 1 calculates two volume levels by decomposing the volume of the virtual sound source VS into the directions of the two sound output devices 3-r, 3-l selected in step S22 using VBAP, and sets these volume levels as the volume of the right sound data SDR and the left sound data SDL. Next, in step S24, the processing device 1 generates second sound data consisting of the right sound data SDR and left sound data SDL whose volumes were set in step S23. Finally, in step S25, the processing device 1 sets the second sound data generated in step S24 as two-channel sound data SDr, SDl corresponding to the two sound output devices 3-r, 3-l selected in step S22, and sets the volume of the other channels to zero, thereby generating n-channel sound data SD1 to SDn.

[0034] Fig. 8 is a flowchart showing an example of the procedure of the correction processing step S3 in Fig. 5. As shown in Fig. 8, first, in step S31, the processing device 1 calculates the listening position P U to the sound output devices 3-1 to 3-n, respectively. n Next, in step S32, the processing device 1 acquires the distances d1 to d n The maximum distance dmax and distance d1~d n In step S33, the processing device 1 corrects the delay amount of the sound data SD1 to SDn of n channels in accordance with the ratio of the maximum distance d max and distance d1~d n or the ratio of each of the distances d1 to d n The minimum distance d min and distance d1~d n Finally, in step S34, the processing device 1 sets the n-channel sound data SD1 to SDn, whose delay amounts and sound volumes have been corrected in steps S32 and S33, as n-channel sound data DO1 to DOn.

[0035] 1-1-3.Effects As described above, according to the processing device 1 of the first embodiment, the head-related transfer function processing unit 30 can generate right sound data SDR and left sound data SDL by adding frequency characteristics to the sound source data SDI that allow the listener U to easily sense the direction. Furthermore, according to the processing device 1 of the first embodiment, the correction processing unit 50 can correct the delay amount and volume of the sound data SD1 to SDn, thereby creating a state in which the sound output devices 3-1 to 3-n are virtually arranged concentrically. Therefore, even if the sound output devices 3-1 to 3-n are not arranged concentrically, the volume panning processing unit 40 can apply VBAP, and sound data DO1 to DOn can be generated that allows the listener U to easily perceive the direction of the sound. Therefore, for example, if the sound corresponding to the sound source data SDI is a warning sound, the listener U can correctly perceive the direction of the warning sound output from the sound output devices 3-1 to 3-n and quickly and accurately recognize the situation. Furthermore, for example, if the sound corresponding to the sound source data SDI is a sound effect or voice, the listener U can correctly perceive the direction of the sound effect or voice output from the sound output devices 3-1 to 3-n, and can obtain a high sense of realism and immersion.

[0036] 1-2. Second embodiment Hereinafter, for the processing device 1 of the second embodiment, the same symbols will be used for configurations similar to those of the first embodiment, and explanations similar to those of the first embodiment will be omitted or simplified, with the main focus being on the differences from the first embodiment.

[0037] Fig. 9 is a diagram showing an example of the configuration of a processing device 1 according to a second embodiment. As shown in Fig. 9, the processing device 1 according to the second embodiment, like the processing device 1 according to the first embodiment, includes a control unit 10, a sound source memory 20, a head-related transfer function processing unit 30, a volume panning processing unit 40, a correction processing unit 50, n amplifiers 60-1 to 60-n, and a storage unit 70. The processing device 1 according to the second embodiment further includes a noise sound generation unit 80, a second head-related transfer function processing unit 90, a second volume panning processing unit 100, and a mixing processing unit 110. The processing device 1 may be a one-chip semiconductor integrated circuit device, or may be composed of semiconductor integrated circuit devices on multiple chips, or may be at least partially composed of electronic components other than semiconductor integrated circuit devices. For example, the control unit 10, the head-related transfer function processing unit 30, the volume panning processing unit 40, the correction processing unit 50, the storage unit 70, the noise sound generation unit 80, the second head-related transfer function processing unit 90, the second volume panning processing unit 100, and the mixing processing unit 110 may be realized as a single-chip semiconductor integrated circuit device, and the sound source memory 20 and the amplifiers 60-1 to 60-n may be provided outside the semiconductor integrated circuit device. Note that the processing device 1 may be configured such that some of the components in FIG. 9 are omitted or modified, or other components are added. For example, the amplifiers 60-1 to 60-n do not have to be components of the processing device 1.

[0038] The noise sound generation unit 80 generates monaural noise source data NDI in real time. The noise source data NDI may be, for example, white noise with a uniform energy distribution, or pink noise with an energy distribution inversely proportional to the frequency. The noise source data NDI generated by the noise sound generation unit 80 is input to the second head-related transfer function processing unit 90. The noise source data NDI may be stored in advance in the sound source memory 20.

[0039] The second head-related transfer function processing unit 90 adds a noise source signal corresponding to the listening position P U Angle θ of virtual noise source VN with respect to N The head-related transfer function according to the angle θ is applied to generate the first noise data of the stereo. N is set based on an instruction from the microcontrol unit 2, for example, and is output from the control unit 10. The first noise data is composed of right noise data NDR and left noise data NDL. Specifically, the second head-related transfer function processing unit 90 calculates the angle θ N Assuming that a virtual noise source VN in the direction of emits sound according to the noise source data NDI, the head-related transfer function h R (θ N ) to generate right noise data NDR, and the head-related transfer function h L (θ N ) to generate left noise data NDL. As shown in FIG. 10, the angle θ S and the angle θ of the virtual noise source VN N As shown in FIG. 11, the angle θ of the virtual sound source VS may be the same. S and the angle θ of the virtual noise source VN N The difference between these angles may be 90° or more.

[0040] The second volume panning processor 100 adjusts the angle θ of the virtual noise source VN. N The second head-related transfer function processing unit 90 performs volume panning on the generated first noise data to generate n-channel noise data ND1 to NDn including stereo second noise data. The second noise data is any two of the noise data ND1 to NDn, and the angle θ N and the location information 71 of the sound output devices 3-1 to 3-n.

[0041] Specifically, the second volume panning processor 100 first calculates the angle θ Nand the arrangement information 71 of the sound output devices 3-1 to 3-n, the second volume panning processing unit 100 selects two sound output devices 3-rn and 3-ln that generate stereo noise sounds. rn and ln are each an integer between 1 and n. For example, the second volume panning processing unit 100 selects two sound output devices 3-rn and 3-ln that generate stereo noise sounds based on the arrangement information 71. U The direction of the sound output devices 3-1 to 3-n relative to the N The second volume panning processing unit 1 selects the sound output device 3-rn closest to the right and the sound output device 3-ln closest to the left with respect to the direction of the virtual noise source VN determined by Next, the right virtual noise source VNR and the left virtual noise source VNL are assumed to exist in two directions of the sound output devices 3-rn and 3-ln, and the vector a VN Two vectors a VNR ,a VNL Then, the second volume panning processing unit 100 calculates the vector a VNR Set the length of the vector a to the volume of the right noise data NDR. VNL The second volume panning processing unit 100 sets the length of the right noise data NDR to the volume of the left noise data NDL, and generates second noise data consisting of the right noise data NDR and the left noise data NDL. The right noise data NDR is noise data NDrn of the rnth channel corresponding to the sound output device 3-rn, and the left noise data NDL is noise data NDln of the lnth channel corresponding to the sound output device 3-ln. The second volume panning processing unit 100 sets the volume of all the noise data ND1 to NDn of the n channels other than the noise data NDrn and NDln to zero.

[0042] The mixing processing unit 110 generates mixed data by mixing the second sound data generated by the volume panning processing unit 40 and the second noise data generated by the second volume panning processing unit 100. Specifically, the mixing processing unit 110 sets the second sound data to a volume specified by the sound source volume information 72, sets the second noise data to a volume specified by the noise volume information 73, and mixes n-channel sound data SD1 to SDn including the second sound data with n-channel noise data ND1 to NDn including the second noise data to generate n-channel mixed data MD1 to MDn. The sound source volume information 72 and the noise volume information 73 are stored in advance in the storage unit 70. However, the micro control unit 2 may generate the sound source volume information 72 and the noise volume information 73 and transmit them to the control unit 10.

[0043] The correction processing unit 50 corrects the delay amount and volume of the sound data SDr, SDl, which is second sound data included in the mixed data generated by the mixing processing unit 110, based on the arrangement information 71 of the sound output devices 3-1 to 3-n, to generate stereo third sound data to be output by two of the sound output devices 3-r, 3-l among the sound output devices 3-1 to 3-n. The correction processing unit 50 also corrects the delay amount and volume of the noise data NDrn, NDln, which is second noise data included in the mixed data generated by the mixing processing unit 110, based on the arrangement information 71, to generate stereo third noise data to be output by two of the sound output devices 3-1 to 3-n, 3-rn, 3-ln among the sound output devices 3-1 to 3-n. The correction processing unit 50 then outputs n-channel sound data DO1 to DOn, which include the sound data SDr, SDl, which are third sound data with the delay amount and volume corrected, and the noise data NDrn, NDln, which are third noise data with the delay amount and volume corrected. The third noise data is included in the sound data DOrn of the rnth channel corresponding to the sound output device 3-rn and the sound data DOln of the lnth channel corresponding to the sound output device 3-ln, among the sound data DO1 to DOn.

[0044] Specifically, the correction processing unit 50 first calculates the listening position P Uto the sound output devices 3-1 to 3-n, respectively. n Next, the correction processing unit 50 obtains the distances d1 to d n The maximum distance d max and distance d1~d n The correction processing unit 50 corrects the delay amount of the mixed data MD1 to MDn in a direction to increase the delay amount in accordance with the ratio of the distances d1 to d n The maximum distance d max and distance d1~d n The correction processing unit 50 corrects the volume of the mixed data MD1 to MDn in a decreasing direction according to the ratio of the delay amount and the volume to each of the mixed data MD1 to MDn. Then, the correction processing unit 50 outputs the mixed data MD1 to MDn with the corrected delay amount and volume as sound data DO1 to DOn.

[0045] It should be noted that the correction processing unit 50 does not easily correct the delay amount of the second noise data in a direction to reduce the delay amount. U Distance d from the farthest sound output device 3-j max On the other hand, since the correction processing unit 50 can easily correct the volume of the second noise data in either a decreasing or increasing direction, it is preferable to correct the volume of the second noise data in either a decreasing or increasing direction based on the listening position P U Distance d from the farthest sound output device 3-j max The volume of the second noise data may be corrected to decrease based on the listening position P U distance d from the nearest sound output device 3-i min In the latter case, for example, the second volume panning processing unit 100 may correct the volume of the second noise data in a direction to increase the volume of the second noise data based on the listening position P U distance d from the nearest sound output device 3-i min The volume of the second noise data may be set based on this.

[0046] Other configurations and functions of the processing device 1 of the second embodiment are similar to those of the processing device 1 of the first embodiment, and therefore, description thereof will be omitted.

[0047] FIG. 12 is a flowchart showing the steps of the processing method of the second embodiment. The processing method of the second embodiment is executed by the processing device 1 of the second embodiment shown in FIG. 9. As shown in FIG. 12, first, as in the first embodiment, the processing device 1 performs a head-related transfer function processing step S100 and a volume panning processing step S110 to generate second stereo sound data. The processing of the head-related transfer function processing step S100 is similar to the processing of the head-related transfer function processing step S1 of FIG. 5, and its specific procedure is similar to the procedure of FIG. 6, so illustration and description thereof will be omitted. This head-related transfer function processing step S100 is executed by the head-related transfer function processing unit 30 of the processing device 1. Similarly, the processing of the volume panning processing step S110 is similar to the processing of the volume panning processing step S2 of FIG. 5, and its specific procedure is similar to the procedure of FIG. 7, so illustration and description thereof will be omitted. This volume panning processing step S110 is executed by the volume panning processing unit 40 of the processing device 1.

[0048] Next, in the second head-related transfer function processing step S120, the processing device 1 adds a second head-related transfer function (HRTF) to the monaural noise source data NDI. U Angle θ of virtual noise source VN with respect to N This second head-related transfer function processing step S120 is executed by the second head-related transfer function processing unit 90 of the processing device 1.

[0049] Next, in the second volume panning process step S130, the processing device 1 adjusts the angle θ of the virtual noise source VN. N Based on this, the processing device 1 performs volume panning processing on the first noise data generated in the second head-related transfer function processing step S120 to generate stereo second noise data. Specifically, the processing device 1 mixes n-channel sound data SD1 to SDn including the second sound data and n-channel noise data ND1 to NDn including the second noise data to generate n-channel mixed data MD1 to MDn. This second volume panning processing step S130 is executed by the second volume panning processing unit 100 of the processing device 1.

[0050] Next, in a mixing process S140, the processing device 1 mixes the second sound data generated in the volume panning process S110 and the second noise data generated in the second volume panning process S130 to generate mixed data. This mixing process S140 is executed by the mixing processing unit 110 of the processing device 1.

[0051] Finally, in a correction process step S150, the processing device 1 corrects the delay amount and volume of the second sound data included in the mixed data generated in the mixing process step S140 based on the arrangement information 71 of the sound output devices 3-1 to 3-n, and generates stereo third sound data to be output from two of the sound output devices 3-1 to 3-n. Also, the processing device 1 corrects the delay amount and volume of the second noise data included in the mixed data generated in the mixing process step S140 based on the arrangement information 71, and generates stereo third noise data to be output from two of the sound output devices 3-1 to 3-n. In the correction process step S150, the processing device 1 corrects the delay amount and volume of the second noise data included in the mixed data generated in the mixing process step S140 based on the arrangement information 71, and generates stereo third noise data to be output from two of the sound output devices 3-1 to 3-n. U to the farthest sound output device 3-j among the sound output devices 3-1 to 3-n, d max The processing device 1 may correct the delay amounts of the second sound data and the second noise data based on the listening position P U to the farthest sound output device 3-j among the sound output devices 3-1 to 3-n, d max or the distance d to the nearest sound output device 3-i min of The volume of the second sound data and the second noise data may be corrected using the second sound data and the second noise data as a reference. This correction processing step S150 is executed by the correction processing unit 50 of the processing device 1.

[0052] Fig. 13 is a flowchart showing an example of the procedure of the second head-related transfer function processing step S120 in Fig. 12. As shown in Fig. 13, first, in step S121, the processing device 1 calculates the angle θ of the virtual noise source VN. N Next, in step S122, the processing device 1 obtains a head-related transfer function h R (θ N) to generate right noise data NDR. In step S123, the processing device 1 applies a head-related transfer function h L (θ N ) to generate left noise data NDL. Finally, in step S124, the processing device 1 generates first noise data consisting of the right noise data NDR and the left noise data NDL.

[0053] Fig. 14 is a flowchart showing an example of the procedure of the second volume panning process step S130 in Fig. 12. As shown in Fig. 14, first, in step S131, the processing device 1 adjusts the angle θ of the virtual noise source VN. N and acquires the arrangement information 71 of the sound output devices 3-1 to 3-n. Next, in step S132, the processing device 1 calculates the angle θ of the virtual noise source VN. N and the arrangement information 71, the processing device 1 selects two sound output devices 3-rn, 3-ln that will output stereo noise sounds. Next, in step S133, the processing device 1 calculates two volume levels by decomposing the volume of the virtual noise source VN into the directions of the two sound output devices 3-rn, 3-ln selected in step S132 using VBAP, and sets these volume levels as the volume levels of the right noise data NDR and the left noise data NDL. Next, in step S134, the processing device 1 generates second noise data consisting of the right noise data NDR and the left noise data NDL whose volume levels were set in step S133. Finally, in step S135, the processing device 1 sets the second noise data generated in step S134 as two-channel noise data NDrn, NDln corresponding to the two sound output devices 3-rn, 3-ln selected in step S132, and sets the volume levels of the other channels to zero, thereby generating n-channel noise data ND1 to NDn.

[0054] Fig. 15 is a flowchart showing an example of the procedure of the correction processing step S150 in Fig. 12. As shown in Fig. 15, first, in step S151, the processing device 1 calculates the listening position P U to the sound output devices 3-1 to 3-n, respectively. nNext, in step S152, the processing device 1 acquires the distances d1 to d n The maximum distance d max and distance d1~d n In step S153, the processing device 1 corrects the delay amount of the mixed data MD1 to MDn of n channels in accordance with the ratio of the maximum distance d max and distance d1~d n or the ratio of each of the distances d1 to d n The minimum distance d min and distance d1~d n In accordance with the ratio of each of the delay amounts and the volume of the n-channel mixed data MD1 to MDn generated in the mixing process step S140, the processing device 1 corrects the volume of the n-channel mixed data MD1 to MDn generated in the mixing process step S140. Finally, in step S154, the processing device 1 sets the n-channel mixed data MD1 to MDn, the delay amounts and volume of which have been corrected in steps S152 and S153, as n-channel sound data DO1 to DOn.

[0055] According to the processing device 1 of the second embodiment described above, even if the sound source data SDI has only a narrow frequency spectrum, a wideband noise sound that is easy to perceive direction is added, so it is possible to generate sound data that makes it easier for the listener U to perceive the direction of the sound. In addition, according to the processing device 1 of the second embodiment, the same effects as those of the processing device 1 of the first embodiment can be obtained.

[0056] 1-3. Third embodiment Hereinafter, for the processing device 1 of the third embodiment, the same symbols will be used for configurations similar to those of the first or second embodiment, and explanations similar to those of the first or second embodiment will be omitted or simplified, with the focus being mainly on the differences from the first and second embodiments.

[0057] FIG. 16 is a diagram illustrating an example of the configuration of a processing device 1 according to a third embodiment. As illustrated in FIG. 16, the processing device 1 according to the third embodiment includes a control unit 10, a sound source memory 20, a head-related transfer function processing unit 30, a volume panning processing unit 40, a correction processing unit 50, n amplifiers 60-1 to 60-n, a storage unit 70, a noise sound generation unit 80, and a mixing processing unit 110. The processing device 1 may be a single-chip semiconductor integrated circuit device, or may be configured with multiple semiconductor integrated circuit devices, or at least a portion of the processing device 1 may be configured with electronic components other than a semiconductor integrated circuit device. For example, the control unit 10, the head-related transfer function processing unit 30, the volume panning processing unit 40, the correction processing unit 50, the noise sound generation unit 80, and the mixing processing unit 110 may be implemented as a single-chip semiconductor integrated circuit device, and the sound source memory 20 and the amplifiers 60-1 to 60-n may be provided outside the semiconductor integrated circuit device. The processing device 1 may have a configuration in which some of the components illustrated in FIG. 16 are omitted or modified, or other components are added. For example, the amplifiers 60-1 to 60-n do not have to be components of the processing device 1.

[0058] The mixing processing unit 110 generates mixed data MDI by mixing the monaural sound source data SDI with the monaural noise source data NDI generated by the noise sound generation unit 80. Specifically, the mixing processing unit 110 sets the sound source data SDI to a volume specified by the sound source volume information 72, sets the noise source data NDI to a volume specified by the noise volume information 73, and mixes the sound source data SDI and the noise source data NDI to generate mixed data MDI.

[0059] In this embodiment, the listening position P U Angle θ of virtual sound source VS with respect to S and the angle θ of the virtual noise source VN N The head-related transfer function processing unit 30 adds the angle θ of the virtual sound source VS and the virtual noise source VN to the monaural mixing data MDI. S The head-related transfer function processing unit 30 applies a head-related transfer function according to the angle θ to generate first stereo sound data. The first sound data is composed of right sound data SDR and left sound data SDL. Specifically, the head-related transfer function processing unit 30 applies a head-related transfer function according to the angle θS The virtual sound source VS in the direction of emits sound according to the sound source data SDI, and the angle θ S Assuming that a virtual noise source VN in the direction of emits sound according to the noise source data NDI, the head-related transfer function h R (θ S ) to generate right sound data SDR, and the head-related transfer function h L (θ S ) to generate the left sound data SDL.

[0060] The processing by the volume panning processing unit 40, the correction processing unit 50, and the amplifiers 60-1 to 60-n is the same as in the first embodiment, and therefore the description thereof will be omitted. Also, the processing by the noise sound generation unit 80 is the same as in the second embodiment, and therefore the description thereof will be omitted.

[0061] 17 is a flow chart showing the procedure of the processing method of the third embodiment. The processing method of the third embodiment is executed by the processing apparatus 1 of the third embodiment shown in FIG.

[0062] 17, first, in a mixing process step S200, the processing device 1 mixes the monaural sound source data SDI and the monaural noise source data NDI to generate mixed data MDI. This mixing process step S200 is executed by the mixing processing unit 110 of the processing device 1.

[0063] Next, in the head-related transfer function processing step S210, the processing device 1 adds a head-related transfer function (HRTF) to the monaural mixing data MDI generated in the mixing processing step S200. U The angle θ of the virtual sound source VS and the virtual noise source VN with respect to S This head-related transfer function processing step S210 is executed by the head-related transfer function processing unit 30 of the processing device 1.

[0064] Next, in the volume panning process S220, the processing device 1 calculates the angle θ between the virtual sound source VS and the virtual noise source VN. S Based on this, volume panning processing is performed on the first sound data generated in the head-related transfer function processing step S210 to generate stereo second sound data. This volume panning processing step S220 is executed by the volume panning processing unit 40 of the processing device 1.

[0065] Finally, in a correction process step S230, the processing device 1 corrects the delay amount and volume of the second sound data generated in the volume panning process step S220 based on the arrangement information 71 of the sound output devices 3-1 to 3-n, and generates stereo third sound data to be output from two of the sound output devices 3-1 to 3-n. U to the farthest sound output device 3-j among the sound output devices 3-1 to 3-n, d max The processing device 1 may correct the delay amount of the second sound data based on the listening position P U to the farthest sound output device 3-j among the sound output devices 3-1 to 3-n, d max or the distance d to the nearest sound output device 3-i min The volume of the second sound data may be corrected based on the above-mentioned correction processing step S230. This correction processing step S230 is executed by the correction processing unit 50 of the processing device 1.

[0066] Fig. 18 is a flowchart showing an example of the procedure of the head-related transfer function processing step S210 in Fig. 17. As shown in Fig. 18, first, in step S211, the processing device 1 calculates the angle θ between the virtual sound source VS and the virtual noise source VN. S Next, in step S212, the processing device 1 obtains a head-related transfer function h R (θ S ) to generate right sound data SDR. In step S213, the processing device 1 applies a head-related transfer function h L (θ S) to generate left sound data SDL. Finally, in step S214, the processing device 1 generates first sound data consisting of the right sound data SDR and the left sound data SDL.

[0067] Note that the specific procedure of the volume panning processing step S220 in Fig. 17 is the same as the procedure in Fig. 7, and therefore illustration and description thereof will be omitted. Similarly, the specific procedure of the correction processing step S230 in Fig. 17 is the same as the procedure in Fig. 8, and therefore illustration and description thereof will be omitted.

[0068] According to the processing device 1 of the third embodiment described above, even if the sound source data SDI is sound having only a narrow frequency spectrum, a wideband noise sound that is easy to perceive direction is added, so it is possible to generate sound data that makes it easier for the listener U to perceive the direction of the sound. In addition, according to the processing device 1 of the third embodiment, the same effects as those of the processing device 1 of any of the first to third embodiments can be obtained.

[0069] 1-4. Fourth embodiment Hereinafter, for the processing device 1 of the fourth embodiment, the same symbols will be used for configurations that are similar to any of the first to third embodiments, and explanations that are similar to any of the first to third embodiments will be omitted or simplified, and the following will mainly describe the contents that are different from any of the first to third embodiments.

[0070] FIG. 19 is a diagram showing an example of the configuration of a processing device 1 of the fourth embodiment. As shown in FIG. 19, the processing device 1 of the fourth embodiment includes a control unit 10, a sound source memory 20, n amplifiers 60-1 to 60-n, a storage unit 70, and a processing unit 200. The processing device 1 may be a one-chip semiconductor integrated circuit device, or may be configured with semiconductor integrated circuit devices on multiple chips, or at least a part of the processing device may be configured with electronic components other than semiconductor integrated circuit devices. For example, the control unit 10, the storage unit 70, and the processing unit 200 may be realized by a one-chip semiconductor integrated circuit device, and the sound source memory 20 and the amplifiers 60-1 to 60-n may be provided outside the semiconductor integrated circuit device. Note that the processing device 1 may be configured by omitting or changing some of the components shown in FIG. 19, or by adding other components. For example, the amplifiers 60-1 to 60-n may be configured by the processing It does not have to be a component of the device 1.

[0071] The processing unit 200 acquires the sound source data SDI, performs predetermined calculation processing, and outputs n-channel sound data DO1 to DOn. Specifically, the processing unit 200 executes a processing program 301 stored in an information storage medium 300, and performs various calculation processing on the sound source data SDI. The processing unit 200 is realized by, for example, a CPU or a DSP. CPU is an abbreviation for Central Processing Unit, and DSP is an abbreviation for Digital Signal Processor. The information storage medium 300 is realized by, for example, a hard disk, a flexible disk, an MO, an MT, various types of memory, a CD-ROM, a DVD-ROM, or the like.

[0072] By executing the processing program 301, the processing unit 200 functions as a head-related transfer function processing unit 30, a volume panning processing unit 40, a correction processing unit 50, a noise sound generation unit 80, a second head-related transfer function processing unit 90, a second volume panning processing unit 100, and a mixing processing unit 110, and executes each procedure of the flowchart shown in Fig. 12. In other words, the processing program 301 is a program that causes the processing device 1, which is a computer, to execute each procedure of the flowchart shown in Fig. 12. In this way, the processing device 1 of the fourth embodiment realizes the same functions as the processing device 1 of the second embodiment by the processing unit 200 executing the processing program 301.

[0073] The processing unit 200 may execute the processing program 301 to function as the head-related transfer function processing unit 30, the volume panning processing unit 40, and the correction processing unit 50, and may execute each procedure of the flowchart shown in Fig. 5. In other words, the processing program 301 may be a program that causes the processing device 1, which is a computer, to execute each procedure of the flowchart shown in Fig. 5. That is, the processing device 1 of the fourth embodiment may realize the same functions as the processing device 1 of the first embodiment by the processing unit 200 executing the processing program 301.

[0074] Furthermore, the processing unit 200 may execute the processing program 301 to function as the head-related transfer function processing unit 30, the volume panning processing unit 40, the correction processing unit 50, and the mixing processing unit 110, and may execute each procedure of the flowchart shown in Fig. 17. In other words, the processing program 301 may be a program that causes the processing device 1, which is a computer, to execute each procedure of the flowchart shown in Fig. 17. That is, the processing device 1 of the fourth embodiment may realize the same functions as the processing device 1 of the third embodiment by the processing unit 200 executing the processing program 301.

[0075] According to the processing apparatus 1 of the fourth embodiment described above, the same effects as those of any of the first to third embodiments can be obtained.

[0076] 2.Electronic equipment Fig. 20 is a functional block diagram showing an example of the configuration of an electronic device using the processing device 1 of any of the above-described embodiments. In Fig. 20, the same components as those in Fig. 1, Fig. 9, Fig. 16, or Fig. 19 are denoted by the same reference numerals.

[0077] 20, electronic device 400 of this embodiment includes a processing device 1, a micro control unit 2, n sound output devices 3-1 to 3-n, a sensor 410, an operation unit 420, a storage unit 430, and a display unit 440. Note that electronic device 400 of this embodiment may be configured such that some of the components in FIG. 20 are omitted or changed, or other components are added.

[0078] The MCU 2 performs control processing of each part of the electronic device 400 and various data processing. The CU2 transmits various commands to the processing device 1 and controls the operation of the processing device 1. The MCU2 also performs various processes in response to detection signals from the sensor 410, various processes in response to operation signals from the operation unit 420, and a process of transmitting a display signal for displaying various information on the display unit 440.

[0079] The sensor 410 is any sensor such as an acceleration sensor, an angular velocity sensor, a speed sensor, a pressure sensor, or a temperature sensor, and outputs a detection signal to the MCU 2.

[0080] The operation unit 420 is an input device configured with operation keys, button switches, etc., and outputs an operation signal to the MCU 2 in response to an operation by a user.

[0081] The storage unit 430 stores programs, data, etc. for performing various calculation processes and control processes by the MCU 2. The storage unit 430 is realized by, for example, a hard disk, a flexible disk, an MO, an MT, various types of memory, a CD-ROM, or a DVD-ROM.

[0082] The display unit 440 is a display device configured with an LCD or the like, and displays various information based on input display signals. LCD is an abbreviation for Liquid Crystal Display. The display unit 440 may be provided with a touch panel that functions as the operation unit 420.

[0083] The processing device 1 generates sound signals DOX1 to DOXn based on various commands transmitted from the MCU 2 and outputs them to the sound output devices 3-1 to 3-n, respectively. The sound output devices 3-1 to 3-n output sounds corresponding to the sound signals DOX1 to DOXn, respectively.

[0084] For example, the MCU2 may transmit a command to the processing device 1 to instruct the reproduction of an attention-calling warning sound based on a detection signal from the sensor 410. Also, for example, the MCU2 may transmit a command to the processing device 1 to instruct the reproduction of a guidance sound based on an operation signal from the operation unit 420. Also, for example, the MCU3 may transmit a command to the processing device 1 to instruct the reproduction of various sounds such as sound effects, voices, melodies, etc. at a predetermined timing. The processing device 1 reads out sound source data stored in the sound source memory 20, generates sound signals DOX1 to DOXn, and outputs them to the sound output devices 3-1 to 3-n, respectively. As a result, sounds output from any two of the sound output devices 3-1 to 3-n are transmitted to the listening position P U By mixing these sounds together, various warning sounds, sound effects, voices, melodies, etc. with an emphasis on direction are reproduced.

[0085] Various electronic devices are possible as such electronic device 400, such as warning devices, televisions, car navigation devices, electronic game devices, fish finders, various measuring devices, instruments for vehicles, aircraft, ships, etc., flight simulators, head-mounted displays, etc.

[0086] The present invention is not limited to the present embodiment, and various modifications are possible within the scope of the present invention.

[0087] The above-described embodiment and modifications are merely examples, and the present invention is not limited to these. For example, the embodiments and modifications can be combined as appropriate.

[0088] The present invention includes configurations that are substantially the same as the configurations described in the embodiments, for example, configurations with the same functions, methods, and results, or configurations with the same purpose and effects. The present invention also includes configurations that replace non-essential parts of the configurations described in the embodiments. The present invention also includes configurations that achieve the same effects as the configurations described in the embodiments, or configurations that can achieve the same purpose. The present invention also includes configurations that add publicly known technology to the configurations described in the embodiments. Includes:

[0089] The following can be derived from the above-described embodiment and modifications.

[0090] One aspect of the processing device is a head-related transfer function processing unit that applies a head-related transfer function according to an angle of a virtual sound source with respect to a listening position to the monaural sound source data to generate first stereo sound data; a volume panning processing unit that performs volume panning processing on the first sound data based on the angle of the virtual sound source to generate second stereo sound data; and a correction processing unit that corrects the delay amount and volume of the second sound data based on arrangement information of the plurality of sound output devices and generates stereo third sound data to be output from two of the plurality of sound output devices.

[0091] According to this processing device, it is possible to generate sound data to which frequency characteristics that allow a listener to easily perceive the direction of a sound are added by head-related transfer function processing. Furthermore, according to this processing device, it is possible to create a state in which multiple sound output devices are virtually arranged concentrically by processing to correct the delay amount and volume of the sound data, so that even if the multiple sound output devices are not arranged concentrically, it is possible to apply volume panning processing and generate sound data that allows a listener to easily perceive the direction of a sound.

[0092] In one aspect of the processing device, The correction processing unit correcting a delay amount of the second sound data based on a distance from the listening position to the farthest sound output device among the plurality of sound output devices; The volume of the second sound data may be corrected based on the distance from the listening position to the farthest sound output device of the plurality of sound output devices or the distance to the closest sound output device.

[0093] One aspect of the processing device is a second head-related transfer function processing unit that applies a head-related transfer function according to an angle of the virtual noise source with respect to the listening position to the monaural noise source data to generate first stereo noise data; a second volume panning processing unit that performs volume panning processing on the first noise data based on the angle of the virtual noise source to generate second stereo noise data; a mixing processing unit that mixes the second sound data and the second noise data to generate mixed data, The correction processing unit may generate the third sound data by correcting the delay amount and volume of the second sound data included in the mixing data based on the placement information, and may generate the third noise data by correcting the delay amount and volume of the second noise data included in the mixing data.

[0094] According to this processing device, even if the sound source data has only a narrow frequency spectrum, a wideband noise sound that is easy to perceive direction is added, so that sound data can be generated that makes it easier for the listener to perceive the direction of the sound.

[0095] In one aspect of the processing device, The angle of the virtual sound source and the angle of the virtual noise source may be the same.

[0096] According to this processing device, a wideband noise sound emitted by a virtual noise source in the same direction as the virtual sound source is added to the sound source data, thereby generating sound data that allows a listener to more easily perceive the direction of the sound.

[0097] In one aspect of the processing device, The difference between the angle of the virtual sound source and the angle of the virtual noise source may be 90° or more.

[0098] One aspect of the processing device is a mixing processing unit that mixes the sound source data and monaural noise source data to generate mixed data; The head-related transfer function processing unit is The first sound data may be generated by applying a head-related transfer function to the mixing data according to the angle of the virtual sound source relative to the listening position.

[0099] According to this processing device, even if the sound source data has only a narrow frequency spectrum, a wideband noise sound that is easy to perceive direction is added, so that sound data can be generated that makes it easier for the listener to perceive the direction of the sound.

[0100] One aspect of the processing method is a head-related transfer function processing step of applying a head-related transfer function according to an angle of a virtual sound source relative to a listening position to the monaural sound source data to generate stereo first sound data; a volume panning process for performing a volume panning process on the first sound data based on the angle of the virtual sound source to generate second stereo sound data; and a correction processing step of correcting the delay amount and volume of the second sound data based on arrangement information of the plurality of sound output devices, and generating stereo third sound data to be output from two of the plurality of sound output devices.

[0101] According to this processing method, it is possible to generate sound data to which frequency characteristics that allow a listener to easily perceive the direction of a sound are added by head-related transfer function processing. Furthermore, according to this processing method, it is possible to create a state in which multiple sound output devices are virtually arranged concentrically by correcting the delay amount and volume of the sound data, so that even if multiple sound output devices are not arranged concentrically, it is possible to apply volume panning processing and generate sound data that allows a listener to easily perceive the direction of a sound.

[0102] One aspect of the processing program is a head-related transfer function processing step of applying a head-related transfer function according to an angle of a virtual sound source relative to a listening position to the monaural sound source data to generate stereo first sound data; a volume panning process for performing a volume panning process on the first sound data based on the angle of the virtual sound source to generate second stereo sound data; and a correction processing step of correcting the delay amount and volume of the second sound data based on arrangement information of the plurality of sound output devices, and generating stereo third sound data to be output from two of the plurality of sound output devices.

[0103] According to this processing program, it is possible to generate sound data to which frequency characteristics that allow a listener to easily perceive the direction of a sound are added by head-related transfer function processing. Furthermore, according to this processing program, it is possible to create a state in which multiple sound output devices are virtually arranged concentrically by processing to correct the delay amount and volume of the sound data, so that even if the multiple sound output devices are not arranged concentrically, it is possible to apply volume panning processing and generate sound data that allows a listener to easily perceive the direction of a sound. [Explanation of symbols]

[0104] 1... processing device, 2... microcontrol unit, 3-1 to 3-n... sound output device, 10... control unit, 20... sound source memory, 21-1 to 21-m... sound source data, 30... head-related transfer function Processing unit, 40...volume panning processing unit, 50...correction processing unit, 60-1 to 60-n...amplifiers, 70...storage unit, 71...placement information, 72...sound source volume information, 73...noise volume information, 80...noise sound generation unit, 90...second head-related transfer function processing unit, 100...second volume panning processing unit, 110...mixing processing unit, 200...processing unit, 300...information storage medium, 301...processing program, 400...electronic device, 410...sensor, 420...operation unit, 430...storage unit, 440...display unit

Claims

1. a head-related transfer function processing unit that applies a head-related transfer function according to an angle of the virtual sound source with respect to the listening position to the monaural sound source data to generate first sound data in stereo; a volume panning processing unit that performs volume panning processing on the first sound data based on the angle of the virtual sound source to generate second stereo sound data; a correction processing unit that corrects a delay amount and a volume of the second sound data based on arrangement information of a plurality of sound output devices, and generates stereo third sound data to be output from two of the plurality of sound output devices.

2. In claim 1, The correction processing unit correcting a delay amount of the second sound data based on a distance from the listening position to the farthest sound output device among the plurality of sound output devices; a processing device that corrects the volume of the second sound data based on a distance from the listening position to the farthest sound output device of the plurality of sound output devices or a distance from the listening position to the closest sound output device.

3. In claim 1, a second head-related transfer function processing unit that applies a head-related transfer function according to an angle of the virtual noise source with respect to the listening position to the monaural noise source data to generate first stereo noise data; a second volume panning processing unit that performs volume panning processing on the first noise data based on the angle of the virtual noise source to generate second stereo noise data; a mixing processing unit that mixes the second sound data and the second noise data to generate mixed data, The correction processing unit corrects the delay amount and volume of the second sound data included in the mixing data based on the placement information to generate the third sound data, and corrects the delay amount and volume of the second noise data included in the mixing data to generate the third noise data.

4. In claim 3, The processing device wherein the angle of the virtual sound source and the angle of the virtual noise source are the same.

5. In claim 3, A processing device, wherein the difference between the angle of the virtual sound source and the angle of the virtual noise source is 90° or greater.

6. In claim 1, a mixing processing unit that mixes the sound source data and monaural noise source data to generate mixed data; The head-related transfer function processing unit is a processing device that applies a head-related transfer function according to the angle of the virtual sound source relative to the listening position to the mixing data to generate the first sound data.

7. a head-related transfer function processing step of applying a head-related transfer function according to an angle of a virtual sound source relative to a listening position to the monaural sound source data to generate stereo first sound data; a volume panning process for performing a volume panning process on the first sound data based on the angle of the virtual sound source to generate second stereo sound data; a delay amount and a volume of the second sound data are corrected based on arrangement information of the plurality of sound output devices, and third sound data of stereo to be output from two of the plurality of sound output devices is generated; and a forward processing step.

8. a head-related transfer function processing step of applying a head-related transfer function according to an angle of a virtual sound source relative to a listening position to the monaural sound source data to generate stereo first sound data; a volume panning process for performing a volume panning process on the first sound data based on the angle of the virtual sound source to generate second stereo sound data; and a correction processing step of correcting the delay amount and volume of the second sound data based on placement information of the plurality of sound output devices, and generating stereo third sound data to be output from two of the plurality of sound output devices.

Images