Information processing device, information processing method, and program
By convolving the harmonic distortion transmission characteristics in the acoustic space into the input signal in the information processing device and combining them with other sound transmission characteristics, the problem of sound reproduction distortion in the prior art is solved, and accurate reproduction in the acoustic space is achieved The effect of sound.
Patent Information
- Application Number
- CN202380034793.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-05-11
- Filing Date
- 2023-04-24
- Publication Date
- 2024-11-29
AI Technical Summary
The prior art cannot accurately reproduce harmonic distortion in cinemas and studios in sound reproduction, resulting in distortion of sound reproduction.
By convolving the harmonic distortion transmission characteristics in the acoustic space into the input signal in the information processing device and combining them with other sound transmission characteristics, a signal can accurately reproduce the dynamic behavior of the speakers in the acoustic space.
Accurate reproduction of sound in the acoustic space is achieved, eliminating the need to go to the cinema, allowing producers to edit movie sounds at home and able to reproduce sounds in the acoustic space including harmonic distortion.
Smart Images

Figure CN119054304A_ABST
Abstract
Description
Technical Field
[0001] The present technology relates to an information processing device, an information processing method, and a program, and more particularly, to an information processing device, an information processing method, and a program capable of accurately reproducing sound in an acoustic space. Background Art
[0002] By convolving a head-related transfer function (HRTF) indicating sound transfer characteristics in an acoustic space (e.g., a movie theater or a studio) into an audio signal, localization of a sound image at a predetermined position and stereoscopic reproduction of sound heard from headphones can be achieved. For example, Patent Document 1 describes actually reproducing the sound pressure of an audio source from a specific position by forming and using HRTF for each individual.
[0003] Thus, the sound reproduced from the audio source (eg, speakers in a movie theater or studio) can be heard through the headphones.
[0004] Reference List
[0005] Patent Literature
[0006] Patent Document 1: Japanese Patent Application Publication No. 2015-19360 Summary of the invention
[0007] Problems to be solved by the present invention
[0008] Harmonic distortion occurs in actual movie theaters and studios due to reverberation caused by the characteristics of walls and speakers. However, the harmonic distortion in movie theaters and studios cannot be reproduced in the reproduced sound using HRTF.
[0009] The present technology has been made in view of such circumstances, and is capable of accurately reproducing sound in an acoustic space.
[0010] Solution to the problem
[0011] In one aspect of the present technology, an information processing device includes: a harmonic signal generating unit configured to generate a first signal by convolving a transmission characteristic of harmonic distortion in an acoustic space onto an input signal; and a combining unit configured to combine the first signal and a second signal together, wherein the sound transmission characteristic other than the harmonic distortion in the acoustic space is convolved onto the input signal.
[0012] In one aspect of the present technology, an information processing method includes causing an information processing device to perform processing, the processing including: generating a first signal by convolving a transmission characteristic of harmonic distortion in an acoustic space onto an input signal; and combining the first signal and a second signal together, wherein the sound transmission characteristic other than the harmonic distortion in the acoustic space is convolved onto the input signal.
[0013] In one aspect of the present technology, a program causes a computer to perform processing, including: generating a first signal by convolving a transfer characteristic of harmonic distortion in an acoustic space onto an input signal; and combining the first signal and a second signal together, wherein the sound transfer characteristic other than the harmonic distortion in the acoustic space is convolved onto the input signal.
[0014] In one aspect of the present technology, a first signal is generated by convolving a transfer characteristic of harmonic distortion in an acoustic space onto an input signal, and the first signal and a second signal are combined together, wherein the sound transfer characteristic other than the harmonic distortion in the acoustic space is convolved onto the input signal. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] Figure 1 is a diagram showing an example of HRIR.
[0016] Figure 2 is a diagram showing an example of sound transfer characteristics that can be measured in an HRTF measurement environment.
[0017] Figure 3 : is a diagram showing a configuration example of a sound production system according to an embodiment of the present technology.
[0018] Figure 4 is a diagram illustrating a method of measuring HRTF using a TSP signal.
[0019] Figure 5 is a diagram illustrating a method of measuring HRTF using a Log-TSP signal.
[0020] Figure 6 is a diagram showing an example of an impulse response measured by a measuring device.
[0021] Figure 7 is a block diagram showing a configuration example of a conventional information processing apparatus.
[0022] Figure 8 is a flowchart illustrating a reproduction process performed by a conventional information processing apparatus.
[0023] Fig. 9 is a block diagram showing a configuration example of an information processing device of the present technology.
[0024] Fig.10 This is a flowchart for explaining the reproduction process performed by the information processing device of the present technology.
[0025] Fig.11 is a block diagram showing another configuration example of the information processing device.
[0026] Fig.12 is a diagram showing a display example of a setting screen.
[0027] Fig.13 This is a flowchart illustrating the reproduction process performed by the information processing device.
[0028] Fig.14 is a block diagram showing a configuration example of hardware of a computer. DETAILED DESCRIPTION
[0029] Hereinafter, a mode for carrying out the present technology will be described. The description will be given in the following order.
[0030] 1. Configuration of the sound generation system
[0031] 2. Configuration and operation of information processing devices
[0032] 3. Modifications
[0033] <1. Configuration of sound generation system>
[0034] Overview
[0035] A sound image can be stereoscopically reproduced in headphones by using a head-related transfer function (HRTF) indicating sound transfer characteristics from an audio source to both ears in a specific acoustic space.
[0036] HRTF (i.e., frequency domain information) is measured, for example, in the form of a head-related impulse response (HRIR) (i.e., time domain information indicating the impulse response from an audio source to the user's ears in an acoustic space), such as Figure 1 shown.
[0037] Figure 2 is a diagram showing an example of sound transfer characteristics that can be measured in an HRTF measurement environment.
[0038] A speaker 1 serving as an audio source is placed in a studio RM1 serving as an HRTF measurement environment. Reproduced sound based on a predetermined measurement signal is output from the speaker 1 and collected by a microphone 2 placed at a predetermined position in the studio RM1, thereby measuring the characteristics of the sound field of the studio RM1. The characteristics of the sound field include the characteristics of the speaker 1 and the resonance of the studio RM1, such as Figure 2 shown in Balloon #1.
[0039] The microphone is worn on both ears of the user U1 at the HRTF measurement position. In this state, the reproduced sound based on the predetermined measurement signal is output from the speaker 1 and collected by the microphone worn on both ears of the user U1, so that the HRTF from the speaker 1 to the user's two ears is measured in the studio RM1, as shown in balloon #2.
[0040] By having the user U1 actually go to the studio RM1 and measure the HRTF, the measured HRTF is personalized for the user U1. Note that the method for acquiring the HRTF personalized for the user U1 is not limited to the method in which the user U1 actually goes to the measurement environment and measures the HRTF. For example, the HRTF personalized for the user U1 may be acquired based on capturing an image of the ear of the user U1.
[0041] In studio RM1, the sound output from speaker 1 is acoustically affected in the order of the characteristics of speaker 1, the resonance of studio RM1, the torso, head, outer ear, and eardrum of user U1. The sound output then reaches the eardrum of user U1. Therefore, the HRTF from speaker 1 to both ears includes the characteristics of speaker 1, the resonance of studio RM1, and the influence on the torso, head, outer ear, and eardrum of user U1.
[0042] For example, in a case where the studio RM1 is wide or the speaker 1 is large, it is considered that harmonic distortion occurs due to reverberation caused by the wall of the studio RM1 or the characteristics of the speaker 1 .
[0043] The conventional sound production system causes the earphones used by the user U1 to output the reproduced sound obtained by convolving the HRTF from the speaker 1 to the audio signal to the two ears of the user U1, so that the sound from the speaker 1 in the studio RM1 is reproduced. Specifically, a reproduction filter generated by convolving the inverse function of the HRTF from the earphone to both ears with the HRTF (SP HRTF) from the speaker 1 to both ears is convolved into the audio signal. In the conventional sound production system, the algorithm for convolving the reproduction filter to the audio signal is a linear system.
[0044] On the other hand, the actual system before the reproduced sound reaches both ears of the user U1 in the acoustic space is a nonlinear system, so that the algorithm of the conventional sound generation system (simple convolution processing) which is a linear system cannot reproduce harmonic distortion.
[0045] The embodiment of the present technology has been conceived focusing on the above points. The embodiment proposes a technology that can reproduce the dynamic behavior of a speaker in an acoustic space by acquiring highly accurate sound transmission characteristics including harmonic distortion in the acoustic space and convolving the transmission characteristics into an audio signal. The embodiment is described in detail below.
[0046] Configuration of the sound generation system
[0047] Figure 3 : is a diagram showing a configuration example of a sound production system according to an embodiment of the present technology. Figure 3The sound generation system of the present invention is a system for mixing audio of content such as a movie. A user uses the sound generation system to generate the sound of, for example, a movie.
[0048] The sound of the movie includes various sounds such as sound effects, environmental sounds, background music, etc. in addition to the lines, the actors' narration and other human sounds. In the following, when there is no need to distinguish the types of sounds, the sounds are collectively described as sounds. However, the sound of the movie actually includes sounds of types other than speech.
[0049] like Figure 3 As shown on the left side of , a movie theater called a soundtrack station and the like for sound generation is a measurement environment. In addition to the screen, a plurality of speakers are provided in the movie theater. The movie theater is also provided with a measurement device 11 that acquires the measurement results of the sound transmission characteristics in the measurement environment and generates an HRTF file. The measurement device 11 is composed of, for example, a PC.
[0050] exist Figure 3 The personalized HRTF, which is an HRTF personalized to the producer of the movie sound, is measured in the measurement environment of the sound production system of the movie. The HRTF of the sound other than the harmonic distortion in the movie theater and the HRTF of the harmonic distortion of each order in the movie theater are measured as the personalized HRTF.
[0051] like Figure 3 As shown by the arrow in , a personalized HRTF file recording data indicating the measurement result of the personalized HRTF is provided to the information processing device 21 provided in the reproduction environment. The personalized HRTF file can be provided to the information processing device 21 via a network such as the Internet or by using a recording medium such as a flash memory.
[0052] The reproduction environment is an environment at a place different from a movie theater, such as a producer's studio or home. The reproduction environment can be prepared at the same place as the measurement environment.
[0053] The information processing device 21 is a device for editing the sound of a movie and is set in a reproduction environment. The information processing device 21 is composed of a PC, for example. The producer edits the sound of the movie in a reproduction environment (for example, home) using headphones 22. The headphones 22 are output devices prepared in the reproduction environment.
[0054] The audio signal is reproduced using the personalized HRTF in the information processing device 21. Reproduction using the personalized HRTF is the reproduced sound output from the speaker of the movie theater used for the measurement of the personalized HRTF.
[0055] Therefore, the producer can perform editing in the same audio environment as that of a movie theater using the headphones 22. That is, the same acoustic environment as that of a movie theater is virtually reproduced in the reproduction environment. In the environment in which the sound of a movie is produced, the reproduced sound output from the speaker of the movie theater is generally used as a reference. The sound production system of the present technology eliminates the need to go to a movie theater, so that the producer can also perform editing, etc. at home.
[0056] HRTF measurement method
[0057] Next, we will refer to Figure 4 and Figure 5 A method is described for measuring HRTF by the measuring device 11. In a conventional impulse response measuring system, one HRTF includes an HRTF of harmonic distortion and an HRTF of a sound other than the harmonic distortion, so that the HRTF of the harmonic distortion cannot be separated.
[0058] In order to measure the HRTF of harmonic distortion alone, a method is known to use a swept sine (SS) signal to extract the harmonic distortion of each order and the impulse response of the sound other than the harmonic distortion. The SS signal is a sine signal whose frequency increases or decreases over time. A time-stretched pulse (TSP) signal and a logarithmic time-stretched pulse (Log-TSP) signal are referred to as types of SS signals.
[0059] The TSP signal is a signal whose frequency increases or decreases in proportion to time. An example of the time-frequency characteristics of the response of the TSP signal in the acoustic space is shown in Figure 4 As shown on the left side of . Regarding the time-frequency characteristics, the horizontal axis represents time and the vertical axis represents frequency. Figure 4 In the example of , SP1 represents the response of the main signal (signal other than harmonic distortion). Figure 4 In the example of FIG. 1 , SP2 represents the response of the second-order harmonic distortion, and SP3 represents the response of the third-order harmonic distortion.
[0060] Similar to the TSP signal, the response of the TSP signal increases or decreases in frequency in proportion to time, such as Figure 4 as shown on the left.
[0061] The transformation of multiplying SP1 to SP3 by the inverse characteristic of the main signal simultaneously aggregates all frequency components of SP1, as in Figure 4 As shown in the center of , the impulse response of the main signal can be obtained. However, the frequency components of SP2 and SP3 are not aggregated at the same time. Therefore, as a result of measuring the impulse response using the TSP signal, the response of the harmonic distortion mixed with the second-order harmonic distortion and the third-order harmonic distortion can be obtained at a time before the main signal response, as shown in Figure 4 as shown on the right.
[0062] On the other hand, a Log-TSP signal is a signal whose frequency increases as an exponential function of time. An example of the time-frequency characteristic of the response of a Log-TSP signal is shown in Figure 5 SP1 is also shown on the left side. Figure 5 The main signal response in the example. In addition, Figure 5 In the example of FIG. 1 , SP2 represents the response of the second-order harmonic distortion, and SP3 represents the response of the third-order harmonic distortion.
[0063] The response frequency of the Log-TSP signal, similar to the Log-TSP signal, increases as an exponential function of time, such as Figure 5 Here, the fundamental wave (main signal) in the Log-TSP signal is expressed by the following formula (1), and the first-order harmonic is expressed by the following formula (2):
[0064] [Mathematical formula 1]
[0065] fo(t)=eαt·(1)
[0066]
[0067] As shown in formula (2), the time intervals between the first-order harmonics and the fundamental wave of all frequencies in the Log-TSP signal are equal. In addition, in the Log-TSP signal, the time intervals between each harmonic except the first-order harmonic and the fundamental wave are also equal at all frequencies of each harmonic order.
[0068] Therefore, if Figure 5 As shown in the center of , all frequency components of SP1 to SP3 are individually aggregated at once by multiplying SP1 to SP3 by the transformation of the inverse characteristic of the main signal. Therefore, as a result of measuring the impulse response of the Log-TSP signal, the impulse response of the main signal, the impulse response of the second-order harmonic distortion, and the impulse response of the third-order harmonic distortion are obtained, respectively, as shown in Figure 5 as shown on the right.
[0069] The measuring device 11 measures the HRTF of the harmonic distortion of each order and the HRTF of the sound other than the harmonic distortion using the above-mentioned Log-TSP signal.
[0070] Figure 6 is a diagram showing an example of an impulse response measured by the measurement device 11 .
[0071] For example, the measuring device 11 measures the impulse response during the period P1 as the impulse response of the main signal (HRTF of the sound other than the harmonic distortion). In addition, the measuring device 11 measures the impulse response in the period P2 before the period P1 as the impulse response of the first-order harmonic distortion (HRTF of the first-order harmonic distortion), and measures the impulse response in the period P3 before the period P2 as the impulse response of the second-order harmonic distortion (HRTF of the second-order harmonic distortion).
[0072] Therefore, the measuring device 11 can use the Log-TSP signal to measure the harmonic distortion of each order. Figure 6 The order of harmonic distortion measured by the measuring device 11 in is an example, and the measuring device 11 can measure the HRTF of harmonic distortion up to any order.
[0073] <2. Configuration and Operation of Information Processing Device>
[0074] ·Configuration and operation of conventional information processing devices
[0075] Figure 7 21A is a block diagram showing a configuration example of a conventional information processing apparatus 21A.
[0076] like Figure 7 As shown, the conventional information processing apparatus 21A includes an input signal acquisition unit 31A, an HRTF acquisition unit 32A, a convolution unit 33A, and a reproduction control unit 34A.
[0077] For example, the input signal acquisition unit 31A acquires an audio signal of sound of a movie to be edited as an input signal x and supplies the input signal x to the convolution unit 33A.
[0078] The HRTF acquisition unit 32A acquires a personalized HRTF file provided from a device that measures HRTF, reads a personalized HRTF with reference to the personalized HRTF file, and provides the personalized HRTF to the convolution unit 33A.
[0079] The convolution unit 33A loads the personalized HRTF provided from the HRTF acquisition unit 32A into the memory as the FIR coefficient (Finite Impulse Response) of the FIR filter. The convolution unit 33A generates a reproduction signal by convolving the FIR filter with the input signal x provided from the input signal acquisition unit 31A, and provides the reproduction signal to the reproduction control unit 34A.
[0080] The reproduction control unit 34A causes the headphone to output the reproduced sound based on the reproduction signal supplied from the convolution unit 33A.
[0081] Reference Figure 8 The flowchart of describes the reproduction process performed by the conventional information processing device 21A. Figure 8 When the reproduction process starts, the personalized HRTF is loaded into the memory of the convolution unit 33A as FIR coefficients.
[0082] In step S1 , the input signal acquisition unit 31A acquires an input signal.
[0083] In step S2, the convolution unit 33A performs FIR filter convolution processing on the input signal, so that a reproduction signal is generated.
[0084] In step S3, the reproduction control unit 34A causes the headphone to output the reproduced sound based on the reproduction signal.
[0085] Configuration and operation of the information processing device of the present technology
[0086] Fig. 9 : is a block diagram showing a configuration example of the information processing device 21 of the present technology.
[0087] exist Figure 7 In the conventional information processing apparatus 21A shown in FIG. , a personalized HRTF is convolved with the input signal. Fig. 9 In the information processing device 21 of the present technology shown in , the personalized HRTF of the harmonic distortion is loaded into the memory as FIR coefficients for each order of the harmonic distortion, and each personalized HRTF is convolved to the input signal.
[0088] like Fig. 9 As shown, the information processing device 21 of the present technology includes an input signal acquisition unit 41 , an HRTF acquisition unit 42 , a harmonic signal generation unit 43 , a convolution unit 44 , an addition unit 45 , and a reproduction control unit 46 .
[0089] For example, the input signal acquisition unit 41 acquires an audio signal of sound of a movie to be edited as an input signal x, and supplies the input signal x to the harmonic signal generation unit 43 and the convolution unit 44 .
[0090] The HRTF acquisition unit 42 acquires the personalized HRTF file provided by the measurement device 11, and reads the HRTF file with reference to the personalized HRTF file. SP1 , and HRTF SP1 Provided to the convolution unit 44, HRTF SP1 is a personalized HRTF for the sound excluding harmonic distortion. The HRTF acquisition unit 42 also reads the HRTF with reference to the personalized HRTF file. SP2 To HRTF SP5 , and HRTF SP2 To HRTF SP5 Provided to the harmonic signal generating unit 43, HRTF SP2 To HRTFSP5 It is a personalized HRTF for each order of harmonic distortion.
[0091] The harmonic signal generating unit 43 includes multiplication units 51 - 1 to 51 - 4 and convolution units 52 - 1 to 52 - 4 .
[0092] The multiplication unit 51-1 multiplies the input signal x by the input signal x to generate a harmonic signal x corresponding to the second-order harmonic distortion. 2 , and the harmonic signal x 2 Supplied to the multiplication unit 51-2 and the convolution unit 52-1.
[0093] The multiplication unit 51-2 multiplies the input signal x by the harmonic signal x 2 Multiply to generate the harmonic signal x corresponding to the third harmonic distortion 3 , and the harmonic signal x 3 Supplied to the multiplication unit 51-3 and the convolution unit 52-2.
[0094] The multiplication unit 51-3 multiplies the input signal x by the harmonic signal x 3 Multiply to generate the harmonic signal x corresponding to the fourth-order harmonic distortion 4 , and the harmonic signal x 4 Supplied to the multiplication unit 51-4 and the convolution unit 52-3.
[0095] The multiplication unit 51-4 multiplies the input signal x by the harmonic signal x 4 To generate a harmonic signal x corresponding to the fifth-order harmonic distortion 5 , and the harmonic signal x 5 Provided to convolution unit 52-4.
[0096] The convolution units 52 - 1 to 52 - 4 convolute the HRTF of the harmonic distortion to a harmonic signal, which is generated by multiplying the input signal by the same number as the order of the harmonic distortion.
[0097] Specifically, the convolution unit 52-1 converts the HRTF as the HRTF of the second harmonic distortion into SP2 The convolution unit 52-1 convolves the FIR filter onto the harmonic signal x 2 To generate a convolution signal and provide the convolution signal to the adding unit 45.
[0098] The convolution unit 52-2 converts the HRTF as the HRTF of the third harmonic distortion SP3 The convolution unit 52-2 convolves the FIR filter onto the harmonic signal x 3 To generate a convolution signal and provide the convolution signal to the adding unit 45.
[0099] The convolution unit 52-3 converts the HRTF as the HRTF of the fourth-order harmonic distortion SP4 The convolution unit 52-3 convolves the FIR filter with the harmonic signal x 4 To generate a convolution signal and provide the convolution signal to the adding unit 45.
[0100] The convolution unit 52-4 converts the HRTF as the HRTF of the fifth-order harmonic distortion SP5 The convolution unit 52-4 convolves the FIR filter with the harmonic signal x 5 To generate a convolution signal and provide the convolution signal to the adding unit 45.
[0101] The convolution unit 44 converts the HRTF supplied from the HRTF acquisition unit 32A into SP1 The convolution unit 44 convolves the FIR filter with the convolution signal x supplied from the input signal acquisition unit 41 to generate an input signal and supplies the convolution signal to the addition unit 45 .
[0102] It should be noted that in the convolution unit 44 and the convolution units 52-1 to 52-4, in addition to the personalized HRTF, the inverse function of the HRTF from the earphone 22 to both ears of the user is also convolved to the input signal and the harmonic signal.
[0103] The adding unit 45 functions as a combining unit that adds and combines the convolution signals respectively supplied from the convolution unit 44 and the convolution units 52 - 1 to 52 - 4 . The adding unit 45 combines the convolution signals to generate a reproduction signal, and supplies the reproduction signal to the reproduction control unit 46 .
[0104] The reproduction control unit 46 causes the headphone 22 to output the reproduced sound based on the reproduction signal supplied from the adding unit 45 .
[0105] Next, refer to Fig.10 , describing the reproduction process performed by the information processing device 21 of the present technology having the above configuration. For example, the personalized HRTF is used as the FIR coefficient in Fig.10 When the reproduction processing of is started, it is loaded into the memory of each of the convolution unit 44 and the convolution units 52-1 to 52-4.
[0106] In step S21, the input signal acquisition unit 41 acquires an input signal. The multiplication units 51-1 to 51-4 generate harmonic signals respectively corresponding to the orders of harmonic distortion.
[0107] In step S22, the convolution unit 44 performs FIR filter convolution processing on the input signal to generate a convolved signal.
[0108] In step S23, the convolution units 52-1 to 52-4 perform FIR filter convolution processing on the harmonic signals respectively corresponding to the orders of the harmonic distortion to generate convolution signals.
[0109] In step S24, the adding unit 45 combines the convolution signals generated in steps S22 and S23 to generate a reproduction signal.
[0110] In step S25 , the reproduction control unit 46 causes the headphone to output the reproduced sound according to the reproduction signal.
[0111] As described above, the information processing device 21 combines the convolution signal of the HRTF of harmonic distortion as the input signal with the convolution signal of the HRTF other than the harmonic distortion as the input signal, so as to generate a reproduction signal. The HRTF of each order of harmonic distortion is convolved with the input signal processed separately corresponding to the order of harmonic distortion, so that the HRTF of the second order harmonic distortion is convolved as the harmonic signal x 2 , and the HRTF of the third-order harmonic distortion is convolved into the harmonic signal x 3 .
[0112] Causing the reproduced sound to be output from an output device such as the headphone 22 based on the reproduction signal generated in this manner enables the sound production system to accurately reproduce the sound in the acoustic space including harmonic distortion.
[0113] <3. Modifications>
[0114] Example of adjusting FIR coefficients according to sound pressure
[0115] The behavior of harmonic distortion changes according to the sound pressure of the reproduced sound output from the speaker in the measurement environment. The FIR coefficient can be adjusted according to the sound pressure of the reproduced sound output from the headphone 22 in the reproduction environment.
[0116] Fig.11 21 is a block diagram showing another configuration example of the information processing device 21. Fig.11 In, with reference Fig. 9 The same components as those described above are denoted by the same reference numerals, and repeated descriptions are appropriately omitted.
[0117] Fig.11 The configuration of the information processing device 21 shown in Fig. 9 The configuration of the information processing apparatus 21 in FIG. 1 is different in that a display control unit 101 , an operation input unit 102 , and an amplifier 103 are provided.
[0118] The display control unit 101 causes a display device (eg, a display connected to the information processing device 21 in a reproduction environment) to display a setting screen serving as a graphical user interface (GUI) that receives operation input for setting target sound pressure, volume, and the like.
[0119] Fig.12 is a diagram showing a display example of a setting screen.
[0120] like Fig.12 As shown, a form F1 for inputting a target sound pressure is displayed on the upper left side of the setting screen. The user sets the sound pressure (sound pressure level) of the reproduced sound by, for example, selecting a desired target sound pressure from a plurality of options. Fig.12 In the example above, 85dBSPL is entered as the target sound pressure.
[0121] A form F2 for inputting a file name of a personalized HRTF file (FIR file) for reproducing the input signal is displayed below the form F1. For example, the user selects a desired file from the personalized HRTF files readable by the information processing device 21.
[0122] A bar B1 for adjusting the volume is displayed on the lower left side of the setting screen. In addition, a button B11 for outputting the reproduced sound and a button B12 for stopping outputting the reproduced sound are displayed on the lower right side of the setting screen. The user controls the reproduction of the input signal by the information processing device 21 by operating these displays, etc.
[0123] return Fig.11 , the operation input unit 102 receives the operation input of the user on the setting screen. The operation input unit 102 provides information indicating the content of the operation performed by the user to the HRTF acquisition unit 42, the convolution unit 44, the convolution units 52-1 to 52-4, and the amplifier 103. Note that the target sound pressure and volume can be set by a method other than the operation on the setting screen.
[0124] The HRTF acquisition unit 42 reads the personalized HRTF with reference to the personalized HRTF file set by the user. SP1 To HRTF SP5 Record personalized HRTF files.
[0125] The convolution unit 44 and the convolution units 52-1 to 52-4 load the personalized HRTF as FIR coefficients into the memory according to the sound pressure set by the user. Specifically, the convolution unit 44 and the convolution units 52-1 to 52-4 use the personalized HRTF read from the personalized HRTF file as is or adjust the personalized HRTF read from the personalized HRTF file with gain, etc. according to the sound pressure set by the user.
[0126] It should be noted that the information processing device 21 can also SP2 To HRTF SP5 Adjust to 0 to reproduce the reproduced sound in an acoustic space with no harmonic distortion.
[0127] The amplifier 103 adjusts the reproduction signal supplied from the adding unit 45 according to the volume set by the user, and supplies the adjusted reproduction signal to the reproduction control unit 46 .
[0128] The reproduction control unit 46 causes the headphone 22 to output the reproduced sound based on the adjusted reproduction signal supplied from the reproduction control unit 46 .
[0129] Next, we will refer to Fig.13 The flowchart of FIG. 1 describes the reproduction process performed by the information processing apparatus 21 having the aforementioned configuration. Fig.13 When the reproduction process of the HRTF file is started, a user input of a file name of a personalized HRTF file is received and the personalized HRTF is loaded into the memory as FIR coefficients from the personalized HRTF file set by the user.
[0130] In step S41, the input signal acquisition unit 41 acquires an input signal. The multiplication units 51-1 to 51-4 generate harmonic signals respectively corresponding to the orders of harmonic distortion.
[0131] In step S42, the operation input unit 102 receives input of a user operation to set the sound pressure of the reproduced sound and acquires the sound pressure of the reproduced sound. The convolution unit 44 and the convolution units 52-1 to 52-4 load the personalized HRTF as FIR coefficients into the memory according to the sound pressure set by the user.
[0132] In step S43 , the convolution unit 44 performs FIR filter convolution processing on the input signal, so that a convolution signal is generated.
[0133] In step S44 , the convolution units 52 - 1 to 52 - 4 perform FIR filter convolution processing on the harmonic signals respectively corresponding to the orders of the harmonic distortion, so that convolution signals are generated.
[0134] In step S45, the adding unit 45 combines the convolution signals generated in step S43 and step S44 to generate a reproduction signal. The amplifier 103 adjusts the reproduction signal according to the volume set by the user.
[0135] In step S46 , the reproduction control unit 46 causes the headphone to output the reproduced sound according to the reproduction signal.
[0136] As described above, the sound generation system is capable of reproducing the difference in the behavior of harmonic distortion according to the sound pressure of the sound reproduced in the measurement environment and accurately reproducing the sound in the acoustic space including the harmonic distortion.
[0137] ·other
[0138] An example has been described above in which the HRTF of harmonic distortion and the HRTF of sound other than harmonic distortion are measured in the acoustic space serving as the measurement environment. However, these HRTFs may be acquired using a method other than measurement, such as acoustic simulation.
[0139] ·computer
[0140] The above-mentioned series of processing can be executed by hardware or software. In the case where the series of processing is executed by a program, the software is installed on a computer built in dedicated hardware or a general-purpose personal computer from a program recording medium.
[0141] Fig.14 1 is a block diagram showing a configuration example of the hardware of a computer that executes the above-described series of processes according to a program. The measuring device 11 and the information processing device 21 are composed of, for example, a computer having Fig.14 A PC configured similar to the configuration shown in FIG.
[0142] A central processing unit (CPU) 501 , a read only memory (ROM) 502 , and a random access memory (RAM) 503 are connected to one another via a bus 504 .
[0143] An input / output interface 505 is also connected to the bus 504. An input unit 506 including a keyboard, a mouse, etc. and an output unit 507 including a display, a speaker, etc. are connected to the input / output interface 505. In addition, a storage unit 508 including a hard disk, a nonvolatile memory, etc., a communication unit 509 including a network interface, etc., and a drive 510 that drives a removable medium 511 are connected to the input / output interface 505.
[0144] In the computer configured as described above, for example, the CPU 501 loads a program stored in the storage unit 508 into the RAM 503 via the input / output interface 505 and the bus 504 , and executes the program to perform the above-described series of processing.
[0145] For example, the program executed by the CPU 501 is recorded in the removable medium 511 , or provided via a wired or wireless transmission medium such as a local area network, the Internet, or digital broadcasting, and then installed in the storage unit 508 .
[0146] The program executed by the computer may be a program in which processing is performed in time series in the order described in this specification, or may be a program in which processing is performed in parallel or at necessary timing such as when a call is made.
[0147] Note that in this specification, a system means an assembly of multiple components (devices, modules (components), etc.), and it does not matter whether all the components are located in the same housing. Therefore, multiple devices housed in separate housings and connected to each other via a network and a device in which multiple modules are housed in one housing are both systems.
[0148] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may be provided.
[0149] The embodiments of the present technology are not limited to the above-described embodiments, and various modifications may be made without departing from the scope of the present technology.
[0150] For example, the present technology may be embodied in cloud computing in which functions are shared and performed by a plurality of devices via a network.
[0151] Furthermore, each step described in the above-mentioned flowchart may be performed by one device or may be shared and performed by a plurality of devices.
[0152] Furthermore, in the case where a plurality of processes are included in one step, the plurality of processes included in one step may be executed by one device or may be executed by a plurality of devices in a shared manner.
[0153] <Configuration combination example>
[0154] The present technology can also be implemented as follows. (1)
[0156] An information processing device, comprising:
[0157] a harmonic signal generating unit configured to generate a first signal by convolving a transfer characteristic of harmonic distortion in an acoustic space to an input signal; and
[0158] A combining unit is configured to combine the first signal and the second signal together, wherein the sound transmission characteristics except for the harmonic distortion in the acoustic space are convolved to the input signal. (2)
[0160] The information processing device according to (1), wherein
[0161] The harmonic signal generating unit convolves the transfer characteristic of the harmonic distortion of each order into the input signals respectively processed corresponding to the orders of the harmonic distortion. (3)
[0163] The information processing device according to (2), wherein
[0164] The harmonic signal generating unit convolves the transfer characteristic of the harmonic distortion onto the input signal multiplied by the same number as the order of the harmonic distortion. (4)
[0166] The information processing device according to any one of (1) to (3), wherein
[0167] Harmonic distortion is caused by the characteristics of the loudspeakers in the acoustic space. (5)
[0169] The information processing device according to any one of (1) to (4), wherein
[0170] Harmonic distortion is caused by reverberation in an acoustic space. (6)
[0172] The information processing device according to any one of (1) to (5), wherein
[0173] The harmonic signal generating unit convolves the transfer characteristic of the harmonic distortion measured in the acoustic space to the input signal. (7)
[0175] The information processing device according to (6), wherein
[0176] The transfer characteristics of harmonic distortion are measured using a logarithmic TSP signal. (8)
[0178] The information processing device according to any one of (1) to (7), wherein
[0179] The harmonic signal generating unit convolves the transfer characteristic of the harmonic distortion according to the sound pressure of a reproduced sound based on a reproduced signal in which the first signal and the second signal are combined, onto the input signal. (9)
[0181] The information processing device according to (8), wherein
[0182] The harmonic signal generating unit refers to a file recording the transfer characteristic of the harmonic distortion for each sound pressure, and acquires the transfer characteristic of the harmonic distortion according to the sound pressure. (10)
[0184] The information processing device according to (8) or (9), further comprising:
[0185] A display control unit is configured to perform display for receiving an input of a user operation for setting the sound pressure. (11)
[0187] The information processing device according to any one of (1) to (10), wherein
[0188] The transfer characteristics of the sound, excluding the user-personalized harmonic distortion, are convolved onto the input signal. (12)
[0190] The information processing device according to any one of (1) to (11), wherein
[0191] The harmonic signal generating unit convolves the user-personalized harmonic distortion transfer characteristic onto the input signal. (13)
[0193] An information processing method, comprising:
[0194] causing an information processing device to perform a process, the process comprising:
[0195] generating a first signal by convolving a transfer characteristic of harmonic distortion in an acoustic space to an input signal; and
[0196] The first signal and the second signal are combined together, wherein the sound transmission characteristics except for harmonic distortion in the acoustic space are convolved to the input signal. (14)
[0198] A program for causing a computer to execute a process, the process comprising:
[0199] generating a first signal by convolving a transfer characteristic of harmonic distortion in an acoustic space to an input signal; and
[0200] The first signal and the second signal are combined together, wherein the sound transmission characteristics except for harmonic distortion in the acoustic space are convolved to the input signal.
[0201] Reference Symbols List
[0202] 1 Speaker
[0203] 11. Measuring device
[0204] 21 Information processing device
[0205] 22 Headphones
[0206] 41 Input signal acquisition unit
[0207] 42HRTF acquisition unit
[0208] 43. Harmonic signal generation unit
[0209] 44 Convolutional Units
[0210] 45 Addition unit
[0211] 46 Reproduction control unit
[0212] 51-1 to 51-4 multiplication units
[0213] Convolutional units 52-1 to 52-4
[0214] 101 Display control unit
[0215] 102 Operation input unit
[0216] 103 amplifier.
Claims
1. An information processing device, comprising: a harmonic signal generating unit configured to generate a first signal by convolving a transfer characteristic of harmonic distortion in an acoustic space to an input signal; as well as A combining unit is configured to combine the first signal and a second signal in which a sound transmission characteristic other than the harmonic distortion in the acoustic space is convolved to the input signal.
2. The information processing device according to claim 1, wherein: The harmonic signal generating unit convolves the transfer characteristic of the harmonic distortion of each order into the input signals processed respectively corresponding to the orders of the harmonic distortion.
3. The information processing device according to claim 2, wherein: The harmonic signal generating unit convolves the transfer characteristic of the harmonic distortion onto the input signal multiplied by the same number as the order of the harmonic distortion.
4. The information processing device according to claim 1, wherein: The harmonic distortion is caused by loudspeaker characteristics in the acoustic space.
5. The information processing device according to claim 1, wherein: The harmonic distortion is caused by reverberation in the acoustic space.
6. The information processing device according to claim 1, wherein: The harmonic signal generating unit convolves the transfer characteristic of the harmonic distortion measured in the acoustic space with the input signal.
7. The information processing device according to claim 6, wherein: The transfer characteristic of the harmonic distortion is measured using a logarithmic TSP signal.
8. The information processing device according to claim 1, wherein: The harmonic signal generating unit convolves the transfer characteristic of the harmonic distortion according to the sound pressure of a reproduced sound based on a reproduced signal in which the first signal and the second signal are combined, onto the input signal.
9. The information processing device according to claim 8, wherein: The harmonic signal generating unit refers to a file of the transfer characteristic of the harmonic distortion recorded for each sound pressure, and acquires the transfer characteristic of the harmonic distortion according to the sound pressure.
10. The information processing device according to claim 8, further comprising: A display control unit is configured to perform a display for receiving an input of a user operation for setting the sound pressure.
11. The information processing device according to claim 1, wherein: The transfer characteristics of the sound, excluding the user personalized harmonic distortion, are convolved onto the input signal.
12. The information processing device according to claim 1, wherein: The harmonic signal generating unit convolves a user-personalized harmonic distortion transfer characteristic onto the input signal.
13. An information processing method, comprising: causing an information processing device to execute a process, the process comprising: generating a first signal by convolving a transfer characteristic of harmonic distortion in an acoustic space to an input signal; and The first signal and a second signal are combined together, in which the sound transmission characteristics other than the harmonic distortion in the acoustic space are convolved to an input signal.
14. A program for causing a computer to execute a process, the process comprising: generating a first signal by convolving a transfer characteristic of harmonic distortion in an acoustic space to an input signal; as well as The first signal and a second signal are combined together, in which the sound transmission characteristics other than the harmonic distortion in the acoustic space are convolved to an input signal.
Citation Information
Patent Citations
Determination of individual hrtfs
JP2015019360A
