Spatial sound processing device, spatial sound processing method, and program

Abstract

The present invention provides a spatial sound processing device, a spatial sound processing method, and a program that allow the user to become aware of changes in localization in order to suppress the decrease in localization effect due to acclimatization. [Solution] The spatial sound processing device 100 is a spatial sound processing device that processes an input signal with spatial sound, and includes a forward localization processing unit 115 that processes the input signal with localization processing so that the sound image is localized to a first localization position in front of the listener, an output processing unit 116 that outputs the signal processed by the forward localization processing unit 115 as a playback signal to a playback device worn by the listener, and a switching unit 131 that switches the localization processing so that the input signal is localized to a second localization position different from the first localization position when playback of the input signal starts.

Description

Technical Field

[0001] The present disclosure relates to a spatial acoustic processing device, a spatial acoustic processing method, and a program.

Background Art

[0002] Patent Document 1 discloses an external head localization processing device that performs external head localization processing on an input signal. In the external head localization processing device of Patent Document 1, the reproduced signal subjected to external head localization processing is output to headphones.

[0003] Also, Patent Document 2 discloses a headphone reproduction device having a virtual sound image localization processing device. This headphone reproduction device has first and second virtual sound image localization processing devices that localize sound images in different directions. And the output signals of the first and second virtual sound image processing devices are switched at a predetermined cycle.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] Thus, when localizing a sound image, if the sound image is localized at the same localization position for a long time, the localization may become difficult to feel due to habituation. Especially in the case of a vocal where the localization does not move, the influence of habituation tends to increase.

[0006] The present disclosure has been made in view of the above points, and provides a spatial acoustic processing device, a spatial acoustic processing method, and a program that can make a user aware of a change in localization so as to suppress a decrease in the localization effect due to habituation.

Means for Solving the Problems

[0007] The spatial sound processing device according to this embodiment is a spatial sound processing device that performs spatial sound processing on an input signal, and comprises: a forward localization processing unit that performs localization processing on the input signal so as to localize the sound image to a first localization position in front of the listener; an output processing unit that outputs the signal localized by the forward localization processing unit as a playback signal to a playback device worn by the listener; and a switching unit that switches the localization processing to localize the input signal to a second localization position different from the first localization position when playback of the input signal begins.

[0008] The spatial sound processing method according to this embodiment is a spatial sound processing method that performs spatial sound processing on an input signal, comprising the steps of: performing localization processing on the input signal so as to localize the sound image to a first localization position in front of the listener; outputting the localized signal as a playback signal to a playback device worn by the listener; and switching the localization processing to localize the input signal to a second localization position different from the first localization position when playback of the input signal begins.

[0009] The program according to this embodiment is a program that causes a computer to execute a spatial sound processing method for an input signal, wherein the spatial sound processing method includes the steps of: performing localization processing on the input signal so as to localize the sound image to a first localization position in front of the listener; outputting the localized signal as a playback signal to a playback device worn by the listener; and switching the localization processing to localize the input signal to a second localization position different from the first localization position when playback of the input signal begins. [Effects of the Invention]

[0010] According to this disclosure, it is possible to provide a spatial sound processing device, a spatial sound processing method, and a program that make the user aware of changes in localization in order to suppress the decrease in localization effect due to acclimatization. [Brief explanation of the drawing]

[0011] [Figure 1] This is a block diagram showing a spatial acoustic processing device according to this first embodiment. [Figure 2] This is a schematic diagram to explain the change in localization position. [Figure 3] This is a block diagram showing a spatial acoustic processing device according to a modified example 1 of this embodiment. [Figure 4] This is a block diagram showing a spatial acoustic processing device according to a modified example 2 of this embodiment 1. [Figure 5] This is a schematic diagram to explain the change in localization position. [Figure 6] This is a flowchart showing the spatial sound processing method according to this embodiment 1. [Figure 7] This is a flowchart showing the spatial acoustic processing method for variation 3. [Figure 8] This is a block diagram showing a spatial acoustic processing device according to this second embodiment. [Figure 9] This is a flowchart showing the spatial sound processing method according to this second embodiment. [Modes for carrying out the invention]

[0012] Embodiment 1 The spatial sound processing device according to this embodiment performs spatial sound processing on an input signal. The processed signal is then output to headphones as a playback signal. Here, the playback device that reproduces the playback signal is not limited to headphones; it may also be earphones. In other words, the headphones or earphones worn by the user become the playback device that reproduces the playback signal towards the user.

[0013] Spatial acoustic processing, as shown in Patent Document 1, is an out-of-head localization process that localizes the sound image outside the user's head. Therefore, the spatial acoustic processing device uses a spatial acoustic filter that shows the spatial acoustic characteristics from the speaker to the user's ears, and an inverse filter that cancels out the headphone characteristics.

[0014] The arithmetic processing unit of the spatial audio processing device is a personal computer (PC), a tablet terminal, a smartphone, etc., and includes a memory and a processor. The memory stores processing programs, various parameters, measurement data, etc. The processor executes the processing program stored in the memory. By the processor executing the processing program, each process is executed. The processor may be, for example, a CPU (Central Processing Unit), an FPGA (Field-Programmable Gate Array), a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or a GPU (Graphics Processing Unit), etc.

[0015] The spatial audio processing device according to this embodiment will be described with reference to FIG. 1. FIG. 1 is a block diagram showing the configuration of the spatial audio processing device 100. The spatial audio processing device 100 includes an audio input unit 111, a rhythm detection unit 112, a sound source movement processing unit 113, a front localization processing unit 115, an output processing unit 116, and a switching unit 131.

[0016] Furthermore, the spatial audio processing device 100 has headphones 43 that serve as a playback device. The user U is wearing the headphones 43. The headphones 43 have output units 43L and 43R. Note that the connection between the spatial audio processing device 100 and the headphones 43 may be a wired connection or a wireless connection such as Bluetooth (registered trademark).

[0017] The spatial audio processing device 100 is not limited to a physically single device, and some processes may be performed by different devices. For example, some processes may be performed by a smartphone or the like, and the remaining processes may be performed by a DSP (Digital Signal Processor) or the like built into the headphones.

[0018] An input signal, which is an audio signal, is input to the audio input unit 111. The input signal is a stereo input signal of Lch and Rch. That is, the input signal has stereo input signals of Lch and Rch. The stereo input signals of Lch and Rch are analog audio signals output from a CD (Compact Disc) player or the like, or digital audio data such as mp3 (MPEG Audio Layer-3). Also, the input signal may be audio data transferred by streaming or the like. Note that the audio signal or digital audio data is collectively referred to as the input signal. The audio input unit 111 may acquire the input signal by downloading audio data from a server or the like.

[0019] For example, the input signal is a music signal. Alternatively, it may be an audio signal included in various contents such as movies, dramas, games, and videos. When the user U starts playing a music piece or content, an input signal is input to the audio input unit 111. The audio input unit 111 outputs the input signal to the rhythm detection unit 112 and the sound source movement processing unit 113.

[0020] The rhythm detection unit 112 detects a rhythm based on the input signal. The rhythm detection unit 112 may use AI (Artificial Intelligence) to detect the rhythm of the input signal. Alternatively, the rhythm detection unit 112 may detect the rhythm by detecting the repetitive waveform in the low frequency range of the input signal. The rhythm detection unit 112 outputs a timing signal corresponding to the detected rhythm. The timing signal is a signal indicating a period based on the rhythm.

[0021] When the input signal is a music signal, the rhythm detection unit 112 can detect a rhythm corresponding to the music. When the input signal is not a music signal but an audio signal such as conversation, the rhythm detection unit 112 may not be able to detect a rhythm. In this case, the rhythm detection unit 112 may output a timing signal that determines the timing of fluctuations at a preset period or a random period. Also, if the user U desires, the rhythm detection unit 112 may output a timing signal that determines the timing of fluctuations at a preset period or a random period. Furthermore, even if the rhythm detection unit 112 detects a constant rhythm, if the period is too fast, the effect will be more like an effector than a shift in localization. In this case, instead of detecting the rhythm for periods of 1.2 seconds or less, the unit detects periods that are integer multiples of 1.2 seconds or longer as rhythms.

[0022] The sound source movement processing unit 113 performs sound source movement processing based on the timing signal from the rhythm detection unit 112. The sound source movement processing unit 113 applies a volume difference to the left and right stereo input signals. In this way, the sound source movement processing unit 113 can move the sound source position left and right. In other words, the position of the sound image localized by the forward localization processing unit 115, which will be described later, changes left and right.

[0023] The localization position of the sound image will be explained using Figure 2. Figure 2 is a top view illustrating the localization position of the sound image. If the sound source movement processing unit 113 has not moved the sound source position, the forward localization processing unit 115, which will be described later, will localize the sound image to a first localization position P1 located in front of the user U. The first localization position P1 is an external position directly in front of the user U. Note that the directions shown in Figure 2 are directions based on the orientation of the user U. When the sound source movement processing unit 113 performs processing, the localization position of the sound image moves from the first localization position P1 to the second localization position P21 or the second localization position P22. The second localization position P21 or the second localization position P22 is a different position from the first localization position P1. Specifically, the second localization position P21 is shifted to the left from the first localization position P1, and the second localization position P22 is shifted to the right from the first localization position P1. The process of localizing the sound image to the first localization position P1 is called the first localization process. The process of localizing the sound image to the second localization positions P21 and P22 is called the second localization process.

[0024] For example, the sound source movement processing unit 113 increases the volume of the Lch input signal. This moves the localization position of the sound image to the second localization position P21 on the left. The sound source movement processing unit 113 also increases the volume of the Rch input signal. This moves the localization position of the sound image to the second localization position P22 on the right. The sound source movement processing unit 113 can adjust the localization position in the left-right direction by changing the volume difference. In addition to the volume difference, the sound source movement processing unit 113 may also provide a delay between the two stereo input signals. For example, to move the localization position to the left, a delay is applied to the Rch input signal, and to move the localization position to the right, a delay is applied to the Lch input signal.

[0025] Furthermore, the sound source movement processing unit 113 may move the position of the sound source in accordance with the rhythm detected by the rhythm detection unit 112. The rhythm detection unit 112 generates a timing signal corresponding to the rhythm of the input signal. The sound source movement processing unit 113 moves the sound image at the period indicated by the timing signal. For example, the sound source movement processing unit 113 moves the localization position of the sound image back and forth once from left to right at a predetermined period. Alternatively, the sound source movement processing unit 113 moves the localization position of the sound image from the first localization position P1 to the second localization position P21 once at a predetermined period. The movement of the localization position by the sound source movement processing unit 113 may be continuous or discrete.

[0026] The forward localization processing unit 115 localizes the sound image to a first localization position located in front of the user U. As shown in Figure 2, the first localization position P1 is outside the user U's head. Specifically, the first localization position P1 is directly in front of the user U. The forward localization processing unit 115 performs out-of-head localization processing using a filter, as shown in Patent Document 1. That is, it generates a localized playback signal by convolving a filter onto the input signal. The forward localization processing unit 115 outputs the playback signal to the output processing unit 116.

[0027] The output processing unit 116 outputs the playback signal, which has been processed for external localization, to the headphones 43. The output processing unit 116 outputs the output signal to the headphones 43 according to the connection interface of the headphones 43. For example, if the headphones 43 are connected wirelessly, such as via Bluetooth, the output processing unit 116 modulates the output signal and transmits it wirelessly. If the headphones 43 are connected via a wired connection, the output processing unit 116 transmits the output signal to the headphones 43 via an audio cable.

[0028] The playback signal is a stereo signal consisting of Lch and Rch. The output processing unit 116 outputs the stereo playback signal to the headphones 43. The signal is output to the left and right output units 43L and 43R of the headphones 43, respectively. Therefore, the left output unit 43L of the headphones 43 plays the playback sound corresponding to the Lch playback signal towards the left ear. The right output unit 43R of the headphones 43 plays the playback sound corresponding to the Rch playback signal towards the left ear. As a result, user U can hear the playback sound corresponding to the playback signal that has been processed for external localization. User U can hear the sound image that is localized outside of their head.

[0029] In this embodiment, the sound source movement processing unit 113 changes the position of the sound image. Here, as shown in Figure 2, the sound source movement processing unit 113 changes the localization position of the sound image from the first localization position P1 to the second localization positions P21 and P22.

[0030] The switching unit 131 outputs a switching signal to switch the operation of the sound source movement processing unit 113. When playback of the input signal begins, the switching unit 131 switches the localization processing so that the sound is localized to a second localization position that is different from the first localization position. For example, the switching unit 131 controls the operation of the sound source movement processing unit 113 on or off. When the switching unit 131 turns off the operation of the sound source movement processing unit 113, the sound source movement processing unit 113 outputs the input signal as is to the forward localization processing unit 115. Therefore, the localization position of the sound image becomes the first localization position P1. When the switching unit 131 turns on the operation of the sound source movement processing unit 113, the sound source movement processing unit 113 provides the volume difference as described above. Therefore, the localization positions of the sound image become the second localization positions P21 and P22. In this way, the switching unit 131 can switch the localization position of the sound image between the first localization position and the second localization position.

[0031] The switching unit 131 switches the operation of the sound source movement processing unit 113 so that the localization position of the sound image changes between normal operation and the start of playback of the input signal. Here, the start of playback is the period from the time when playback of the input signal begins until a predetermined time has elapsed. Normal operation is the period after the predetermined time has elapsed.

[0032] For example, when playback of an input signal begins, the switching unit 131 turns on the operation of the sound source movement processing unit 113. As a result, when playback of the input signal begins, the localization position of the sound image becomes the second localization position P21, P22. After a predetermined time has elapsed since the start of playback of the input signal, the switching unit 131 turns off the operation of the sound source movement processing unit 113. As a result, the localization position of the sound image switches from the second localization position P21, P22 to the first localization position P1. In this way, at the start of playback, the switching unit 131 can set the localization position of the sound image to a second localization position different from the first localization position. Then, after a predetermined time has elapsed since playback began, the switching unit 131 turns off the operation of the sound source movement processing unit. As a result, the localization position of the sound image is fixed at the first localization position.

[0033] This approach makes user U aware of changes in the sound image's localization. It mitigates the effects of becoming accustomed to the localization. It prevents a decrease in concentration and adaptation, thus preventing a reduction in the forward localization effect. As a result, it becomes possible to maintain the localization effect for a long period of time.

[0034] Furthermore, in the case of music signals, randomly moving the localization position or moving it independently of the rhythm may result in an unnatural sound. Therefore, in this embodiment, it is preferable to move the localization position in accordance with the rhythm detected by the rhythm detection unit 112. By doing so, it is possible to reduce the unnaturalness of the music played back by the playback signal. In particular, in the case of music signals, unnaturalness can be reduced by changing the localization position in accordance with the rhythm.

[0035] Furthermore, although the above explanation states that the switching unit 131 sets the sound image localization position to the second localization position when playback of the input signal begins, the localization position may be changed to the second localization position at other times. The switching unit 131 may also switch the operation of the sound source movement processing unit 113 so that the localization position moves periodically or randomly. The amount of movement of the localization position does not need to be large enough to be perceived as a slight movement. There is no need to move the localization position significantly from right to left, for example, and spatial sound processing can be performed effectively.

[0036] Furthermore, if the input signal is a music signal, the switching unit 131 may be controlled to switch the localization position when the music or track changes. When a new music starts, the sound source movement processing unit 113 operates to set the localization position of the sound image to the second localization position. Then, after the new music has been played for a predetermined period, the operation of the sound source movement processing unit 113 is turned off, and the localization position of the sound image changes to the first localization position. This makes it possible to change the localization position of the sound image. By attracting attention and reducing habituation, a decrease in the localization effect can be prevented.

[0037] Variation 1 Next, a modification 1 of this embodiment will be described using Figure 3. Figure 3 is a block diagram showing the configuration of the spatial sound processing device 100 according to modification 1. In modification 1, the spatial sound processing device 100 has a coefficient processing unit 114. A switching unit 131 controls the coefficient processing unit 114 in order to change the localization position of the sound image. Also, in Figure 3, the rhythm detection unit 112 shown in Figure 1 is not provided. The basic configuration of the spatial sound processing device 100 is the same as in Figure 1, so the explanation will be omitted as appropriate. Also, although a sound source movement processing unit 113 is shown in Figure 3, the processing of the sound source movement processing unit 113 is optional. In this case, in modification 1, the spatial sound processing device 100 does not need to have a sound source movement processing unit 113.

[0038] The coefficient processing unit 114 can change the coefficients of the filter in the forward localization processing unit 115. The coefficient processing unit 114 sets the coefficients so that the localization position of the sound image is different during normal operation and at the start of playback. For example, as shown in Figure 5, the filter coefficients are changed so that the localization position of the sound image is the second localization position P23. Here, the second localization position P23 is a position inside the user U's head. Alternatively, the coefficient processing unit 114 can change the filter coefficients so that the sound image is localized at the second localization position P24, which is outside the user U's head. The second localization position may be inside or outside the head. The process of localizing the sound image to the first localization position P1 is called the first localization process. The process of localizing the sound image to the second localization positions P23 and P24 is called the second localization process.

[0039] The coefficient processing unit 114 changes the coefficients of the spatial acoustic filter, which indicates the spatial acoustic transmission characteristics from the speaker to the ear. Alternatively, it changes the coefficients of the inverse filter that cancels out the headphone characteristics. In this way, it is possible to change the localization position of the sound image to any desired position. The coefficient processing unit 114 can change the coefficients according to the direction and distance of the shift in the localization position. The coefficient processing unit 114 may also prepare two or more sets of filters in advance and switch the filter used when changing the localization position.

[0040] Furthermore, the method for changing the localization position of the sound image to the second localization position P23 inside the head is not particularly limited. For example, the switching unit 131 may control the switching of the processing in the front localization processing unit 115 on or off. In other words, at the start of playback, the switching unit 131 turns off the localization processing of the front localization processing unit 115. As a result, the output processing unit 116 outputs the normal stereo input signal as a playback signal to the headphones 43. In other words, the output processing unit 116 outputs the stereo input signal, which has not been convolved with the above-mentioned filter, as a playback signal to the headphones 43. Therefore, the headphones 43 play back the playback signal that has not undergone front localization processing. As a result, user U can hear a sound image localized to the second localization position P23 inside the head.

[0041] During normal playback of the input signal, the switching unit 131 turns on the localization processing of the forward localization processing unit 115. As a result, the output processing unit 116 outputs a signal that has been processed for forward localization of the normal stereo input signal to the headphones 43 as a playback signal. Therefore, the headphones 43 play back the playback signal that has been processed for forward localization. This allows the user U to hear a sound image localized at the first localization position P1 in front of them.

[0042] Variation 2 The spatial sound processing device according to Modification 2 will be explained using Figure 4. Figure 4 is a block diagram showing the configuration of the spatial sound processing device 100 according to Modification 2. In Figure 4, a rhythm detection unit 112 is added to the configuration in Figure 3. The rhythm detection unit 112 performs the same processing as the rhythm detection unit 112 shown in Figure 1.

[0043] For example, the coefficient processing unit 114 can change its localization position according to the rhythm detection result of the rhythm detection unit 112. In other words, the coefficient processing unit 114 changes its localization position in synchronization with the rhythm period. By doing so, it is possible to reduce the unnaturalness of the music played back by the playback signal. In particular, in the case of music signals, unnaturalness can be reduced by changing the localization position in accordance with the rhythm. Furthermore, the direction in which the localization position is changed is not limited to the front-to-back direction, but may also be the up-and-down direction or the left-to-right direction. Of course, the direction in which the localization position is changed may also be diagonal.

[0044] It is also possible to use a combination of the configurations shown in Figure 1 and Figure 3. In other words, the spatial sound processing device 100 can change the localization position of the sound image to a second localization position through the processing of the sound source movement processing device 113 and the coefficient processing device 114.

[0045] The spatial sound processing method according to this embodiment will be explained with reference to Figure 6. Figure 6 is a flowchart of the spatial sound processing method. For example, Figure 6 shows the processing method in the spatial sound processing device 100 shown in Figure 1. First, when user U starts playback, an input signal is input to the audio input unit 111 (S101). The rhythm detection unit 112 detects the rhythm of the music based on the input signal (S102). This sets a period for moving the sound image. If the input signal is not a music signal or if an appropriate rhythm is not detected, a predetermined period set in advance may be set.

[0046] Next, the sound source movement processing unit 113 and the forward localization processing unit 115 perform a second localization process (S103). Here, the second localization process is the process in which the sound source movement processing unit 113 moves the sound source, and then the forward localization processing unit 115 performs sound image localization processing. In other words, the switching unit 131 turns on the processing of the sound source movement processing unit 113. After the sound source movement processing unit 113 applies a volume difference to the left and right stereo signals, the forward localization processing unit 115 convolves a filter onto the stereo signal. As a result, as shown in Figure 2, the sound image is localized to a second localization position different from the first localization position P1. At this time, the sound source movement processing unit 113 moves the sound image in accordance with the rhythm. Alternatively, as explained in the modified example 2 in Figure 4, the coefficient processing unit 114 may move the sound image instead of the sound source movement processing unit 113. For example, the coefficient processing unit 114 may move the sound image by changing the coefficient of the filter in accordance with the rhythm.

[0047] Next, the switching unit 131 determines whether a predetermined time has elapsed during the playback time of the second localization process (S104). If the predetermined time has not elapsed during the playback time of the second localization process (NO in S104), the spatial sound processing device 100 repeats the second localization process until the predetermined time has elapsed.

[0048] If a predetermined playback time has elapsed in the second localization processing (YES in S104), the forward localization processing unit 115 performs the first localization processing (S105). In other words, the switching unit 131 turns off the processing of the sound source movement processing unit 113. As a result, the forward localization processing unit 115 localizes the sound image at the first localization position P1 in front of the user U by convolving a filter into the input signal. The forward localization processing unit 115 then performs the first localization processing until the user stops playback.

[0049] This approach makes user U aware of the change in the sound image's localization. This is expected to mitigate the effects of becoming accustomed to the localization. It is also expected to prevent a decrease in concentration and adaptation, thus preventing a reduction in the forward localization effect.

[0050] The sound source movement processing unit 113 moves its localization position in accordance with the rhythm detected by the rhythm detection unit 112. This makes it possible to reduce the unnaturalness of the music played back by the playback signal. In particular, with music signals, changing the localization position in accordance with the rhythm can reduce unnaturalness.

[0051] Variation 3 Figure 7 is a flowchart showing the spatial sound processing method in the spatial sound processing device 100 according to Modification 3. In Figure 7, the rhythm detection process is omitted from the flowchart in Figure 6.

[0052] When user U starts playback, an input signal is input to the audio input unit 111 (S301). Next, the sound source movement processing unit 113 and the front localization processing unit 115 perform a second localization process (S303). Here, the second localization process is the same as in step S103, in which the sound source movement processing unit 113 moves the sound source and then the front localization processing unit 115 performs sound image localization processing. Alternatively, as shown in Figure 3, the coefficient processing unit 114 may move the sound image and then the front localization processing unit 115 may perform sound image localization processing. Furthermore, the second localization process may also produce an output without performing front localization processing on the input signal input from the audio input unit 111. As a result, the output processing unit 116 outputs the normal stereo input signal as is to the headphones 43 as a playback signal. In this case, the sound image is localized at the second localization position P23 inside user U's head as shown in Figure 5. Here, the sound source's position does not move in accordance with the rhythm. For example, the second localization position remains constant.

[0053] Next, the switching unit 131 determines whether a predetermined time has elapsed during the playback time of the second localization process (S304). If the predetermined time has not elapsed during the playback time of the second localization process (NO in S304), the spatial sound processing device 100 repeats the second localization process until the predetermined time has elapsed.

[0054] If a predetermined playback time has elapsed in the second localization processing (YES in S304), the forward localization processing unit 115 performs the first localization processing (S305). In other words, the switching unit 131 turns off the processing of the sound source movement processing unit 113. As a result, the forward localization processing unit 115 localizes the sound image at the first localization position P1 in front of the user U by convolving a filter into the input signal. The forward localization processing unit 115 then performs the first localization processing until the user stops playback. According to the above processing, the rhythm detection unit 112 can be omitted in the configuration shown in Figure 1.

[0055] Embodiment 2 The spatial sound processing device 100 according to this embodiment will be described with reference to Figure 8. Figure 8 is a block diagram showing the configuration of the spatial sound processing device 100. In this embodiment, the spatial sound processing device 100 has a sound source separation processing unit 121 and a synthesis unit 122. The spatial sound processing device 100 also has two forward localization processing units 115a and 115b. The basic configuration of the spatial sound processing device 100 is the same as the configuration shown in Embodiment 1, so the explanation will be omitted as appropriate.

[0056] The audio input unit 111 outputs the input signal to the rhythm detection unit 112 and the sound source separation processing unit 121. The processing of the rhythm detection unit 112 is the same as in Embodiment 1, so its description is omitted. The sound source separation processing unit 121 separates the sound sources included in the input signal. For example, the sound source separation processing unit 121 separates the vocal sound and background sound included in the music signal. Since the vocal sound is approximately center-localized, the sound source separation processing unit 121 separates the components that are in phase and have the same volume on the left and right as the vocal sound. The component remaining after removing the vocal sound from the input signal is then used as the background sound.

[0057] Alternatively, the sound source separation processing unit 121 can also separate vocal sounds using AI or the like. This allows for more accurate separation of sound sources in music signals where the vocals are not centered, even in music signals sung by two or more people.

[0058] The sound source separation processing unit 121 outputs the vocal music signal to the sound source shifting processing unit 113. The sound source shifting processing unit 113 moves the sound source position of the vocal music signal left or right. In other words, it applies a volume difference to the Lch and Rch music signals, thereby shifting the position of the vocal sound source left or right.

[0059] The sound source movement processing unit 113 outputs the music signal with the sound source's position moved to the forward localization processing unit 115a. The forward localization processing unit 115a performs forward localization processing on the vocal music signal using a filter.

[0060] The sound source separation processing unit 121 outputs the background music signal to the front localization processing unit 115b. The front localization processing unit 115b performs front localization processing on the background music signal using a filter. The processing of the front localization processing unit 115a and the front localization processing unit 115b is the same as in Embodiment 1, so a description is omitted.

[0061] The forward localization processing unit 115a and the forward localization processing unit 115b output the music signal, which has been processed for forward localization, to the synthesis unit 122. The synthesis unit 122 synchronizes and synthesizes the music signal of the vocal sound and the music signal of the background sound. The synthesis unit 122 outputs the synthesized signal obtained by the synthesis to the output processing unit 116. The output processing unit 116 outputs the synthesized signal as a playback signal to the headphones 43.

[0062] The switching unit 131 controls the on / off operation of the sound source movement processing unit 113. In other words, when the switching unit 131 turns on the operation of the sound source movement processing unit 113, the sound source position of the vocal sound, which has been processed for forward localization by the forward localization processing unit 115a, moves to the left or right. When the switching unit 131 turns off the operation of the sound source movement processing unit 113, the sound source position of the vocal sound remains at the center position. In this way, only the sound image localization position of the vocal sound can be moved. Therefore, a more natural localization effect can be obtained.

[0063] Furthermore, although the synthesis unit 122 is located after the forward localization processing units 115a and 115b, it may also be located before the forward localization processing units 115a and 115b. In this case, the synthesis unit 122 synchronizes and synthesizes the music signals before the forward localization processing is performed. That is, the synthesis unit 122 synthesizes the music signal of the vocal sound, which has undergone sound source movement processing, with the music signal of the background sound to generate a synthesized signal. Note that when the synthesis unit 122 synthesizes the music signals before the forward localization processing is performed, one forward localization processing unit 115 is sufficient, as in Figure 1. The synthesis unit 122 outputs the synthesized signal to the forward localization processing unit 115 shown in Figure 1. This also allows only the localization position of the vocal sound source to be moved.

[0064] In the example above, the sound source separation processing unit 121 separates the vocal sound from the music signal, but the sound source to be separated is not limited to the vocal sound. The sound source separation processing unit 121 may also separate a center voice, such as a conversation, from other background voices. In this case as well, the sound source separation processing unit 121 can separate the sound source at the center position from other sound sources by comparing the phase and volume of the left and right channels.

[0065] Figure 9 is a flowchart of the spatial sound processing method according to this embodiment. Note that the sound source separation process in S203 is added to the flowchart shown in Figure 6. The processes other than the sound source separation process are the same as in Figure 6, so their explanations will be omitted as appropriate. Specifically, steps S201, S202, S204, S205, and S206 in Figure 9 correspond to steps S101, S102, S103, S104, and S105 in Figure 6, respectively.

[0066] The sound source separation processing unit 121 separates the center sound source from other sound sources based on the input signal (S203). This separates the center sound source, such as the vocal sound, from the background sound sources. Then, the sound source movement processing unit 113 moves the position of only the center sound source left or right. Finally, the synthesis unit 122 synthesizes the signal from the center sound source with the signals from the other sound sources and outputs it to the output processing unit 116. The output processing unit 116 outputs the synthesized signal to the headphones 43.

[0067] As a result, only the sound source position of the center sound source moves left to right. User U can then hear a sound image localized outside their head. Therefore, a more natural localization effect can be obtained. The sound source separation process shown in Embodiment 2 can also be applied to the processes of Modifications 1 and 3. Therefore, step S202 in Figure 9 can be omitted. In other words, the sound source position of the center sound source may or may not move in accordance with the rhythm.

[0068] Some or all of the above processes may be executed by a computer program. The above-described program can be stored and supplied to a computer using various types of non-transitory computer-readable medium. Non-transitory computer-readable mediums include various types of tangible storage mediums. Examples of non-transitory computer-readable mediums include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, and semiconductor memory (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory)). Programs may also be supplied to a computer using various types of transient computer-readable mediums. Examples of transient computer-readable mediums include electrical signals, optical signals, and electromagnetic waves. Transitory computer-readable mediums can be supplied to a computer via wired communication channels such as electric wires and optical fibers, or via wireless communication channels.

[0069] Although the present invention has been specifically described above based on embodiments, it goes without saying that the present invention is not limited to the above embodiments and can be modified in various ways without departing from its essence. [Explanation of symbols]

[0070] U User 43 headphones 100 Spatial Acoustic Processing Device 111 Voice Input Section 112 Rhythm detection unit 113 Sound source transfer processing unit 114 Coefficient Processing Unit 115 Forward positioning processing unit 116 Output Processing Unit 121 Sound Source Separation Processing Unit 122 Synthesis section 131 Switching section P1 1st stereotaxic position P21~P24 2nd stereotaxic position

Claims

1. A spatial sound processing device that performs spatial sound processing on an input signal, A forward localization processing unit performs localization processing on the input signal so that the sound image is localized to a first localization position in front of the listener, An output processing unit outputs the signal processed for localization by the forward localization processing unit as a playback signal to the playback device worn by the listener, The system includes a switching unit that switches the positioning process to position the input signal at a second positioning position different from the first positioning position when playback of the input signal begins, Spatial acoustic processing device.

2. The spatial sound processing apparatus according to claim 1, wherein the second localization position is located inside the listener's head.

3. The spatial sound processing apparatus according to claim 1, further comprising a sound source movement process that controls the volume difference between the left and right sides so that the position obtained by moving the first localization position to the left or right becomes the second localization position.

4. The system further includes a rhythm detection unit that detects the rhythm of the input signal, A spatial sound processing apparatus according to any one of claims 1 to 3, which moves a sound image in accordance with the aforementioned rhythm.

5. A spatial sound processing method that applies spatial sound processing to an input signal, The steps include: performing localization processing on the input signal so that the sound image is localized to a first localization position in front of the listener; The steps include outputting the localized signal as a playback signal to the playback device worn by the listener, The system includes a step of switching the localization process so that the input signal is localized to a second localization position different from the first localization position when playback of the input signal begins. Spatial acoustic processing method.

6. A program that causes a computer to execute a spatial sound processing method that performs spatial sound processing on an input signal, The aforementioned spatial acoustic processing method is, The steps include: performing localization processing on the input signal so that the sound image is localized to a first localization position in front of the listener; The steps include outputting the localized signal as a playback signal to the playback device worn by the listener, The system includes a step of switching the localization process so that the input signal is localized to a second localization position different from the first localization position when playback of the input signal begins. program.

Images