Information processing method, program, and sound reproducing device

Abstract

To provide an information processing method that makes a user perceive three-dimensional sound more appropriately.SOLUTION: An information processing method includes the steps of: calculating an angle volume of a variation in a predetermined direction on a time axis from sound information including information related to predetermined sound and information related to the predetermined direction at each point on the time axis; selecting a three-dimensional sound filter for making input sound perceived as a sound from an arrival direction from among a plurality of three-dimensional sound filter as candidates which are prepared for each arrival direction on the basis of the information related to the predetermined direction; inputting the information related to the predetermined sound to the selected three-dimensional sound filter, and generating an output signal. In selecting the three-dimensional sound filter, the three-dimensional sound filter is selected so as to make a user perceive the predetermined sound to be more emphasized when the angle volume of the variation in the predetermined direction calculated is smaller than a threshold, than when the angle volume of the variation in the predetermined direction is equal to the threshold value or larger.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present disclosure relates to an acoustic playback device, and an information processing method and a program related to the acoustic playback device.

Background Art

[0002] Conventionally, there has been known a technique related to acoustic playback for making a user perceive three-dimensional sound by controlling the position of a sound image, which is a virtual sound source object, in a virtual three-dimensional space (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] On the other hand, when making a user perceive sound as three-dimensional sound in a three-dimensional sound field, there may be a case where sound that is difficult for the user to perceive is generated. In the information processing method in a conventional acoustic playback device or the like, appropriate processing may not be performed on such sound that is difficult to perceive.

[0005] In view of the above, an object of the present disclosure is to provide an information processing method or the like that more appropriately makes a user perceive three-dimensional sound.

Means for Solving the Problems

[0006] An information processing method according to one aspect of the present disclosure is an information processing method for generating an output sound signal for causing a user to perceive a predetermined sound in time series as a sound arriving from an arrival direction on a three-dimensional sound field corresponding to the predetermined direction from sound information including information on the predetermined sound and information on a predetermined direction at each time point on a time axis, the method calculating an angular amount of variation of the predetermined direction on the time axis, selecting, based on the information on the predetermined direction, a stereophonic filter for causing an input input sound to be perceived as a sound from the arrival direction from among a plurality of candidate stereophonic filters prepared for each arrival direction, inputting the information on the predetermined sound to the selected stereophonic filter to generate the output signal, and in the selection of the stereophonic filter, when the calculated angular amount of variation of the predetermined direction is smaller than a threshold value, selecting the stereophonic filter so as to cause the user to perceive the predetermined sound more prominently than when the angular amount of variation of the predetermined direction is greater than or equal to the threshold value.

[0007] Also, an acoustic playback device according to one aspect of the present disclosure is an acoustic playback device that generates and plays back an output sound signal for causing a user to perceive a predetermined sound as a sound arriving from an arrival direction on a three-dimensional sound field corresponding to the predetermined direction from sound information including information on the predetermined sound and information on a predetermined direction at each time point on a time axis, the device including an acquisition unit that acquires the sound information, a filter selection unit that calculates an angular amount of variation of the predetermined direction on the time axis and selects, based on the information on the predetermined direction, a stereophonic filter for causing an input input sound to be perceived as a sound from the arrival direction from among a plurality of candidate stereophonic filters prepared for each arrival direction, an output sound generation unit that inputs the information on the predetermined sound as the input sound to the selected stereophonic filter to generate the output sound signal, and an output unit that outputs sound by using the generated output sound signal, wherein the filter selection unit selects the stereophonic filter so as to cause the user to perceive the predetermined sound more prominently than when the calculated angular amount of variation of the predetermined direction is greater than or equal to a threshold value when the calculated angular amount of variation of the predetermined direction is smaller than the threshold value.

[0008] In addition, one aspect of the present disclosure can also be realized as a program for causing a computer to execute the acoustic reproduction method described above.

[0009] These general or specific aspects may be realized by a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be realized by any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

Advantages of the Invention

[0010] According to the present disclosure, it is possible to more appropriately cause a user to perceive three-dimensional sound.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0012] (Knowledge Underlying the Disclosure) Conventionally, there has been known a technique related to acoustic reproduction for causing a user to perceive three-dimensional sound by controlling the position of a sound image, which is a source object in the user's sense, in a virtual three-dimensional space (hereinafter sometimes referred to as a three-dimensional sound field) (see, for example, Patent Document 1). By localizing a sound image at a predetermined position in a virtual three-dimensional space, the user can perceive this sound as if it is a sound arriving from a direction parallel to the straight line connecting the predetermined position and the user (i.e., a predetermined direction). In order to localize a sound image at a predetermined position in such a virtual three-dimensional space, for example, calculation processing for generating an arrival time difference of sound between both ears and a level difference (or sound pressure difference) of sound between both ears, which are perceived as three-dimensional sound, for the recorded sound is required.

[0013] As an example of such calculation processing, a process of convolving a head-related transfer function for causing a sound to be perceived as arriving from a predetermined direction with a signal of the target sound is known. By performing the convolution process of this head-related transfer function with higher resolution, the sense of presence felt by the user is improved. On the other hand, it is known that in the convolution of the head-related transfer function, it is difficult to perceive fluctuations in the time domain of the sound arrival direction. For this reason, for a sound with slight fluctuations in the time domain, the user may erroneously perceive this sound as not fluctuating.

[0014] In recent years, the development of technologies related to virtual reality (VR) has been actively carried out. In virtual reality, the main focus is on the ability to make the position in a virtual three-dimensional space not follow the user's movement, but rather to give the user the feeling as if they are moving within the virtual space. In particular, in this virtual reality technology, attempts have been made to enhance the sense of presence by incorporating visual elements and auditory elements. For example, when a sound image is localized in front of the user, if the user turns to the right, the sound image moves to the left direction of the user, and if the user turns to the left, the sound image moves to the right direction of the user. Thus, it becomes necessary to move the localization position of the sound image in the virtual space in the direction opposite to the user's movement with respect to the user's movement. Such processing is performed by applying a stereophonic filter to the original sound information.

[0015] In view of the above, in the present disclosure, while using a stereophonic filter for causing a user to perceive sound from a predetermined direction within a three-dimensional sound field, more appropriate computational processing is performed to improve the ease of perception of sound with little variation in the time domain. The present disclosure aims to provide an information processing method or the like for causing a user to perceive three-dimensional sound by means of such appropriate computational processing.

[0016] More specifically, an information processing method according to an aspect of the present disclosure is an information processing method for generating an output sound signal for causing a user to perceive a predetermined sound along a time series as a sound arriving from an arrival direction on a three-dimensional sound field corresponding to a predetermined direction from sound information including information about a predetermined sound and information about a predetermined direction at each time point on a time axis. The method calculates an angular amount of variation in a predetermined direction on the time axis, and based on the information about the predetermined direction, selects a stereophonic filter for causing an input input sound to be perceived as a sound from the arrival direction from among a plurality of candidate stereophonic filters, which are a plurality of stereophonic filters prepared for each arrival direction. The method inputs information about the predetermined sound into the selected stereophonic filter to generate an output signal. In the selection of the stereophonic filter, when the calculated angular amount of variation in the predetermined direction is smaller than a threshold value, the stereophonic filter is selected so that the predetermined sound is more emphasized and perceived by the user compared to the case where the angular amount of variation in the predetermined direction is equal to or greater than the threshold value.

[0017] According to such an information processing method, when the calculated angular amount of variation in the predetermined direction is smaller than the threshold value, that is, when the predetermined sound includes minute variations that are difficult for the user to perceive the variation in its arrival direction, the predetermined sound can be more emphasized and perceived by the user. Since the user's attention is directed to the predetermined sound, the minute variation in the arrival direction of the predetermined sound can be more appropriately perceived by the user.

[0018] Also, for example, in the selection of the stereophonic filter, when the calculated angular amount of variation in the predetermined direction is smaller than the threshold value, the angular amount of variation in the arrival direction perceived by the output sound signal when using the selected stereophonic filter is made larger than the angular amount of variation in the arrival direction perceived by the output sound signal when using the stereophonic filter selected when the calculated angular amount of variation in the predetermined direction is equal to or greater than the threshold value, and the stereophonic filter may be selected.

[0019] According to this, compared with the case of applying a stereophonic filter that is selected when having an angular amount of variation equal to or greater than a threshold value with respect to a predetermined sound, that is, a stereophonic filter that generates an output sound signal so as to have the angular amount of variation preset in the content, in order to emphasize the predetermined sound more, the stereophonic filter can be selected so that the angular amount of variation becomes larger. As a result, with the output sound signal, the angular amount of variation is expanded, so that the predetermined sound is emphasized and perceived.

[0020] Also, for example, in the selection of the stereophonic filter, the stereophonic filter may be selected so that the smaller the calculated angular amount of variation in a predetermined direction, the larger the angular amount of variation in the arrival direction perceived by the output sound signal when the selected stereophonic filter is used.

[0021] According to this, compared with the case of applying a stereophonic filter that is selected when having an angular amount of variation equal to or greater than a threshold value with respect to a predetermined sound, that is, a stereophonic filter that generates an output sound signal so as to have the angular amount of variation preset in the content, in order to emphasize the predetermined sound more, the stereophonic filter can be selected so that the angular amount of variation becomes larger. As a result, with the output sound signal, the angular amount of variation is expanded, so that the predetermined sound is emphasized and perceived. At this time, since the smaller the angular amount of variation on the sound information, the larger the angular amount of variation of the sound in the output sound signal, a predetermined sound that is less likely to be perceived because the variation is small on the original content is emphasized so as to be more easily perceived and presented to the user.

[0022] Also, for example, when the calculated angular amount of variation in a predetermined direction is smaller than a threshold value, when the selected stereophonic filter is used, the smaller the angular amount of variation in the predetermined direction on the sound information, the larger the expansion coefficient α (α > 1) whose value becomes larger is multiplied, so that the angular amount of variation in the arrival direction on the time axis of the output sound signal is expanded, and the relationship between the angular amount of variation in the predetermined direction and the expansion coefficient α may be non-linear.

[0023] According to this, the smaller the angular amount of variation in the sound information, the larger the angular amount of variation in the sound of the output sound signal. Therefore, for a predetermined sound with a small variation on the original content and thus difficult to perceive the variation, it is emphasized so as to be more easily perceived and presented to the user. Since the expansion coefficient α is multiplied, the relationship between the angular amount of variation in a predetermined direction and the arrival direction of the predetermined sound in the output sound information becomes non-linear, and it becomes possible to make the emphasis effect more prominent for a predetermined sound with a small variation.

[0024] Also, for example, in the selection of a stereophonic filter, when the predetermined direction is on the front side of the virtual boundary surface that divides the user's head into front and back, and the calculated angular amount of variation in the predetermined direction is smaller than the threshold value, compared to the angular amount of variation in the arrival direction on the time axis when the selected stereophonic filter is used, when the predetermined direction is on the back side of the boundary surface and the calculated angular amount of variation in the predetermined direction is smaller than the threshold value, the stereophonic filter may be selected so that the angular amount of variation in the arrival direction on the time axis when the selected stereophonic filter is used is larger.

[0025] According to this, on the back side of the boundary surface where it is difficult to sense the variation in the arrival direction, the emphasis effect can be made larger than on the front side of the boundary surface.

[0026] Also, for example, in the selection of a stereophonic filter, the stereophonic filter may be selected so that the arrival direction perceived by the output sound signal when the selected stereophonic filter is used vibrates on the time axis compared to the predetermined direction in the sound information.

[0027] According to this, in the output sound information, a predetermined sound with a vibrating arrival direction can be presented to the user. Since the arrival direction vibrates on the time axis and the user can more easily perceive the predetermined sound compared to other sounds, there is an effect of making the variation of this predetermined sound more easily perceivable.

[0028] Also, for example, when using a selected stereophonic filter, the arrival direction at the Nth (N is an integer of 2 or more) time point on the time axis of the output sound signal is the difference value between the predetermined direction on the sound information corresponding to the (N - 1)th time point on the time axis of the output sound signal and the predetermined direction on the sound information corresponding to the Nth time point. Multiply the difference value by the numerical value at the corresponding time point in the vibration function whose numerical value vibrates on the time axis, and add the multiplied difference value to the predetermined direction on the sound information corresponding to the (N - 1)th time point to calculate it.

[0029] According to this, in the output sound information, a predetermined sound whose arrival direction vibrates can be presented to the user. Since the arrival direction vibrates on the time axis and the user can more easily perceive the predetermined sound compared to other sounds, there is an effect that it is easier to perceive the variation of this predetermined sound.

[0030] Also, for example, in the selection of the stereophonic filter, when the amount of angular variation of the calculated predetermined direction is smaller than the threshold value, the amount of change in the sound pressure of the predetermined sound on the time axis perceived by the output sound signal when using the selected stereophonic filter is greater than that when using the stereophonic filter selected when the amount of angular variation of the calculated predetermined direction is equal to or greater than the threshold value. The stereophonic filter may be selected.

[0031] According to this, compared to the case of applying a stereophonic filter selected when having an amount of angular variation equal to or greater than the threshold value for a predetermined sound, that is, a stereophonic filter that generates an output sound signal so as to have the amount of angular variation preset in the content, in order to emphasize the predetermined sound more, the stereophonic filter can be selected so that the amount of change in the sound pressure becomes larger. As a result, the predetermined sound is emphasized and perceived by the output sound signal due to the increase in the amount of change in the sound pressure.

[0032] Also, for example, an information processing method for generating an output sound signal for causing a user to perceive a predetermined sound as a sound arriving from an arrival direction on a three-dimensional sound field corresponding to a predetermined direction in time series from sound information including information about the predetermined sound and information about the predetermined direction at each time point on the time axis, calculating an angular amount of variation in the predetermined direction on the time axis, and when the calculated angular amount of variation in the predetermined direction is smaller than a threshold value, correcting the information about the predetermined direction so that the predetermined sound is perceived by the user more emphasized than when the angular amount of variation in the predetermined direction is equal to or greater than the threshold value, and inputting the information about the predetermined sound into a stereophonic filter selected based on the corrected information about the predetermined direction from among a plurality of candidate stereophonic filters, which are prepared for each arrival direction, to generate an output signal may be performed.

[0033] According to this, when the calculated angular amount of variation in the predetermined direction is smaller than the threshold value, that is, when the predetermined sound includes minute variations that are difficult for the user to perceive the variation in the arrival direction, this predetermined sound can be made to be perceived by the user more emphasized. For this purpose, by correcting the information about the predetermined direction included in the sound information, the stereophonic filter to be selected later can be made into a stereophonic filter for making the user perceive the predetermined sound more emphasized. As a result, since the user's attention is directed to the predetermined sound, minute variations in the arrival direction of the predetermined sound can be made to be perceived by the user more appropriately.

[0034] Also, a program according to one aspect of the present disclosure is a program for causing a computer to execute the information processing method described above.

[0035] According to this, the same effect as the information processing method described above can be achieved using a computer.

[0036] Also, an acoustic playback device according to one aspect of the present disclosure is an acoustic playback device that generates and plays an output sound signal for causing a user to perceive a predetermined sound as a sound arriving from an arrival direction on a three-dimensional sound field corresponding to a predetermined direction from sound information including information regarding the predetermined sound and information regarding the predetermined direction at each time point on the time axis, the acoustic playback device including: an acquisition unit that acquires the sound information; a filter selection unit that calculates an angular amount of variation in the predetermined direction on the time axis and selects a stereophonic filter, from a plurality of candidate stereophonic filters, for making an input sound input thereto be perceived as a sound from the arrival direction based on the information regarding the predetermined direction, the plurality of candidate stereophonic filters being prepared for each arrival direction; an output sound generation unit that inputs the information regarding the predetermined sound as an input sound to the selected stereophonic filter to generate the output sound signal; and an output unit that outputs sound by using the generated output sound signal, wherein the filter selection unit selects the stereophonic filter so as to make the user perceive the predetermined sound more emphasized when the calculated angular amount of variation in the predetermined direction is smaller than a threshold value than when the angular amount of variation in the predetermined direction is greater than or equal to the threshold value.

[0037] According to this, the same effects as those of the information processing method described above can be obtained.

[0038] Furthermore, these general or specific aspects may be implemented by a system, an apparatus, a method, an integrated circuit, a computer program, or a non-transitory recording medium such as a computer-readable CD-ROM, or may be implemented by any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0039] Hereinafter, embodiments will be specifically described with reference to the drawings. Note that all of the embodiments described below show comprehensive or specific examples. Numerical values, shapes, materials, components, the arrangement positions and connection forms of components, steps, the order of steps, etc. shown in the following embodiments are merely examples and are not intended to limit the present disclosure. In addition, among the components in the following embodiments, components not described in the independent claims are described as optional components. Note that each drawing is a schematic diagram and is not necessarily drawn precisely. Also, in each drawing, substantially the same configuration is denoted by the same reference numeral, and redundant descriptions may be omitted or simplified.

[0040] In addition, in the following description, ordinal numbers such as first, second, and third may be attached to elements. These ordinal numbers are attached to the elements to identify them and do not necessarily correspond to a meaningful order. These ordinal numbers may be appropriately interchanged, newly assigned, or removed.

[0041] (Embodiment) [Overview] First, an overview of the audio playback device according to the embodiment will be described. FIG. 1 is a schematic diagram showing a usage example of the audio playback device according to the embodiment. In FIG. 1, a user 99 who uses the audio playback device 100 is shown.

[0042] The audio playback device 100 shown in FIG. 1 is used simultaneously with the stereoscopic video playback device 200. By simultaneously viewing a stereoscopic image and stereoscopic sound, the image enhances the auditory sense of presence, and the sound enhances the visual sense of presence, allowing the user to feel as if they are at the scene where the image and sound were captured. For example, when an image (moving image) of a person talking is displayed, it is known that even if the sound image localization of the conversation sound is deviated from the mouth of the person, the user 99 can perceive it as the conversation sound emitted from the mouth of the person. In this way, the sense of presence may be enhanced by combining the image and the sound, such as correcting the position of the sound image by visual information.

[0043] The stereoscopic video playback device 200 is an image display device worn on the head of the user 99. Therefore, the stereoscopic video playback device 200 moves integrally with the head of the user 99. For example, as shown in the figure, the stereoscopic video playback device 200 is a glasses-type device supported by the ears and nose of the user 99.

[0044] The stereoscopic video playback device 200 changes the image to be displayed according to the movement of the head of the user 99, so that the user 99 is perceived as moving the head in the three-dimensional image space. That is, when an object in the three-dimensional image space is located in front of the user 99, when the user 99 turns to the right, the object moves to the left direction of the user 99, and when the user 99 turns to the left, the object moves to the right direction of the user. In this way, the stereoscopic video playback device 200 moves the three-dimensional image space in the direction opposite to the movement of the user 99 with respect to the movement of the user 99.

[0045] The stereoscopic video playback device 200 displays two images with a disparity shift in the left and right eyes of the user 99 respectively. The user 99 can perceive the three-dimensional position of the object on the image based on the disparity shift of the displayed images. Note that when the user 99 uses it with eyes closed, such as using the audio playback device 100 for playing soothing sounds for sleep induction, etc., the stereoscopic video playback device 200 does not need to be used at the same time. That is, the stereoscopic video playback device 200 is not an essential component of the present disclosure.

[0046] The audio playback device 100 is a sound presentation device worn on the head of the user 99. Therefore, the audio playback device 100 moves integrally with the head of the user 99. For example, the audio playback device 100 in the present embodiment is a so-called over-ear headphone type device. Note that the form of the audio playback device 100 is not particularly limited. For example, it may be two earplug type devices independently worn on the left and right ears of the user 99 respectively. These two devices communicate with each other to present the sound for the right ear and the sound for the left ear synchronously.

[0047] The audio reproduction device 100 causes the user 99 to perceive as if the user 99 is moving their head within the three-dimensional sound field by changing the sound presented according to the movement of the user 99's head. For this reason, as described above, the audio reproduction device 100 moves the three-dimensional sound field in a direction opposite to the movement of the user with respect to the movement of the user 99.

[0048] Here, it is known that when the change in the time domain of the sound image presented to the user (also referred to as the fluctuation on the time axis) becomes small, the user 99 becomes ambiguous in identifying the movement of the sound image within the three-dimensional sound field. The audio reproduction device 100 according to the present embodiment can cause the user 99 to perceive that there is movement in the sound image by correcting the sound presented by information processing so as to complement this phenomenon. That is, the audio reproduction device 100 acquires the amount of movement of the sound image, and when the acquired amount of movement is smaller than the threshold value, it makes the user 99 perceive a predetermined sound on the three-dimensional sound field more strongly.

[0049] Since this threshold value is a numerical value related to the amount of movement such that the user 99 cannot fully grasp the amount of movement, it becomes a unique threshold value for the user 99. Therefore, it is preferable that a value obtained experimentally or empirically be set as this threshold value. Also, a generalized threshold value based on statistics of a plurality of users 99 may be applied. Note that the amount of movement here is the amount of fluctuation in the arrival direction of a predetermined sound in a minute time, and is the amount of change angle per minute time in a predetermined direction as seen from the user 99. That is, the amount of movement is expressed by the maximum value of the angle formed between the arrival directions of two predetermined sounds corresponding to each of two time points from the first time point to the second time point.

[0050] [Configuration] Next, with reference to FIG. 2, the configuration of the audio reproduction device 100 according to the present embodiment will be described. FIG. 2 is a block diagram showing the functional configuration of the audio reproduction device according to the embodiment.

[0051] As shown in FIG. 2, the audio reproduction device 100 according to the present embodiment includes a processing module 101, a communication module 102, a detector 103, and a driver 104.

[0052] The processing module 101 is an arithmetic unit for performing various signal processes in the acoustic playback device 100. The processing module 101 includes, for example, a processor and a memory, and various functions are exhibited when the program stored in the memory is executed by the processor.

[0053] The processing module 101 includes an acquisition unit 111, a filter selection unit 121, an output sound generation unit 131, and a signal output unit 141. Details of each functional unit included in the processing module 101 will be described below in conjunction with details of configurations other than the processing module 101.

[0054] The communication module 102 is an interface device for receiving input of sound information to the acoustic playback device 100. The communication module 102 includes, for example, an antenna and a signal converter, and receives sound information from an external device by wireless communication. More specifically, the communication module 102 receives a radio signal indicating sound information converted into a format for wireless communication using an antenna, and performs reconversion from the radio signal to sound information by the signal converter. Thereby, the acoustic playback device 100 acquires sound information from an external device by wireless communication. The sound information acquired by the communication module 102 is acquired by the acquisition unit 111. In this way, the sound information is input to the processing module 101. Note that communication between the acoustic playback device 100 and an external device may be performed by wired communication.

[0055] The audio information acquired by the audio playback device 100 is encoded in a predetermined format such as, for example, MPEG-H 3D Audio (ISO / IEC 23008-3). As an example, the encoded audio information includes information about a predetermined sound reproduced by the audio playback device 100 and information regarding the localization position when localizing the sound image of the sound at a predetermined position within the three-dimensional sound field (that is, making it perceived as a sound arriving from a predetermined direction), that is, information regarding a predetermined direction. For example, the audio information includes information about a plurality of sounds including a first predetermined sound and a second predetermined sound, and the sound images are localized so that when each sound is reproduced, the sound images are perceived as sounds arriving from different directions within the three-dimensional sound field.

[0056] With this three-dimensional sound, for example, the sense of presence of content to be viewed, such as in combination with an image viewed using the stereoscopic video playback device 200, can be improved. Note that the audio information may include only information about a predetermined sound. In this case, information regarding a predetermined direction may be acquired separately. Also, as described above, the audio information includes first audio information regarding a first predetermined sound and second audio information regarding a second predetermined sound, but a plurality of pieces of audio information including these separately may be acquired and reproduced simultaneously to localize the sound images at different positions within the three-dimensional sound field. Thus, there is no particular limitation on the form of the input audio information, and the audio playback device 100 may be provided with an acquisition unit 111 corresponding to various forms of audio information.

[0057] Here, an example of the acquisition unit 111 will be described with reference to FIG. 3. FIG. 3 is a block diagram showing the functional configuration of the acquisition unit according to the embodiment. As shown in FIG. 3, the acquisition unit 111 in the present embodiment includes, for example, an encoded audio information input unit 112, a decoding processing unit 113, and a sensing information input unit 114.

[0058] The encoded sound information input unit 112 is a processing unit into which the encoded (i.e., encoded) sound information acquired by the acquisition unit 111 is input. The encoded sound information input unit 112 outputs the input sound information to the decoding processing unit 113. The decoding processing unit 113 generates, in a form used for subsequent processing, information regarding a predetermined sound included in the sound information and information regarding a predetermined direction by decoding (i.e., decoding) the sound information output from the encoded sound information input unit 112. The sensing information input unit 114 will be described below together with the function of the detector 103.

[0059] The detector 103 is a device for detecting the movement speed of the head of the user 99. The detector 103 is configured by combining various sensors used for detecting movement, such as a gyro sensor and an acceleration sensor. In the present embodiment, the detector 103 is built into the acoustic playback device 100. However, for example, it may be built into an external device such as a stereoscopic video playback device 200 that operates in response to the movement of the head of the user 99 in the same manner as the acoustic playback device 100. In this case, the detector 103 may not be included in the acoustic playback device 100. Further, as the detector 103, an external imaging device or the like may be used to image the movement of the head of the user 99, and the movement of the user 99 may be detected by processing the captured image.

[0060] The detector 103 is, for example, integrally fixed to the housing of the acoustic playback device 100 and detects the speed of movement of the housing. Since the acoustic playback device 100 including the above housing moves integrally with the head of the user 99 after the user 99 wears it, the detector 103 can, as a result, detect the speed of movement of the head of the user 99.

[0061] The detector 103 may detect, for example, as the amount of movement of the head of the user 99, a rotation amount having at least one of three axes orthogonal to each other in a three-dimensional space as a rotation axis, or a displacement amount having at least one of the above three axes as a displacement direction. Further, the detector 103 may detect both the rotation amount and the displacement amount as the amount of movement of the head of the user 99.

[0062] The sensing information input unit 114 acquires the movement speed of the head of the user 99 from the detector 103. More specifically, the sensing information input unit 114 acquires, as the movement speed, the amount of movement of the head of the user 99 detected by the detector 103 per unit time. In this way, the sensing information input unit 114 acquires at least one of the rotational speed and the displacement speed from the detector 103. The amount of movement of the head of the user 99 acquired here is used to determine the coordinates and orientation of the user 99 in the three-dimensional sound field. In the acoustic playback device 100, based on the determined coordinates and orientation of the user 99, the relative position of the sound image is determined and the sound is played back. Specifically, the above functions are realized by the filter selection unit 121 and the output sound generation unit 131.

[0063] The filter selection unit 121 is a processing unit that determines, based on the determined coordinates and orientation of the user 99, for a predetermined sound, from which direction in the three-dimensional sound field the sound arrives and is perceived by the user 99, and selects a stereophonic filter to be applied to the predetermined sound. The stereophonic filter is a function filter that convolves the input predetermined sound with a specific head-related transfer function, and causes the user 99 to perceive the predetermined sound as a sound arriving from a predetermined direction based on the specific head-related transfer function. In other words, when a predetermined sound (or information related to the predetermined sound) is input to the stereophonic filter, a sound pressure difference, a time difference, a phase difference, etc. are generated in the left and right sound signals of the predetermined sound, and a sound signal that can reproduce the predetermined sound with the arrival direction controlled can be output.

[0064] A plurality of stereophonic filters that are candidates for selection are prepared in advance, adjusted for each user 99, for example.

[0065] Here, an example of the filter selection unit 121 will be described with reference to FIG. 4. FIG. 4 is a block diagram showing the functional configuration of the filter selection unit according to the embodiment. As shown in FIG. 4, the filter selection unit 121 in the present embodiment includes, for example, a filter storage unit 122, a variable angle calculation unit 123, and a filter determination unit 124.

[0066] The filter memory unit 122 is a storage device for storing a plurality of stereophonic filters that are calculated and generated in advance for each sound arrival direction as described above. The fluctuation angle calculation unit 123 is a processing unit that calculates the amount (angle amount) of fluctuation in a predetermined direction in a minute time based on sound information. For example, the fluctuation angle calculation unit 123 calculates the amount of fluctuation in a predetermined direction within a period fixed within a range of several milliseconds to several seconds from information regarding a predetermined direction. Here, the fluctuation angle calculation unit 123 calculates, as the above angle amount, the angle difference when the angle difference in a predetermined direction becomes maximum within the above period. The fluctuation angle calculation unit 123 compares the calculated angle amount with a threshold value. The comparison result such as the calculated angle amount being smaller than the threshold value is used for determining the stereophonic filter to be selected by the filter determination unit 124.

[0067] The filter determination unit 124 is a processing unit that determines the stereophonic filter to be selected so as to emphasize a predetermined sound more and make the user 99 perceive it when the angle amount calculated by the above fluctuation angle calculation unit 123 is smaller than the threshold value. The stereophonic filter determined by the filter determination unit 124 is output by being read from the filter memory unit 122, that is, output as the stereophonic filter selected by the filter selection unit 121. The details of the determination of the stereophonic filter by the filter determination unit 124 (that is, the selection of the stereophonic filter by the filter selection unit 121) will be described later.

[0068] The output sound generation unit 131 is a processing unit that generates an output sound signal by inputting information regarding a predetermined sound included in the sound information into the selected stereophonic filter using the stereophonic filter selected by the filter selection unit 121.

[0069] Here, an example of the output sound generation unit 131 will be described with reference to FIG. 5. FIG. 5 is a block diagram showing the functional configuration of the output sound generation unit according to the embodiment. As shown in FIG. 5, the output sound generation unit 131 in the present embodiment includes, for example, a filter processing unit 132. The filter processing unit 132 sequentially reads the filters continuously selected by the filter selection unit 121 and inputs information regarding a corresponding predetermined sound on the time axis, thereby continuously outputting a sound signal in which the arrival direction of the predetermined sound arriving on the three-dimensional sound field is controlled. In this way, the sound information segmented at each processing unit time on the time axis is output as a continuous sound signal (output sound signal) on the time axis.

[0070] The signal output unit 141 is a functional unit that outputs the generated output sound signal to the driver 104. The signal output unit 141 generates a waveform signal by performing signal conversion such as converting a digital signal to an analog signal based on the output sound signal, generates a sound wave in the driver 104 based on the waveform signal, and presents sound to the user 99. The driver 104 has, for example, a diaphragm, a driving mechanism such as a magnet and a voice coil. The driver 104 operates the driving mechanism according to the waveform signal, and vibrates the diaphragm by the driving mechanism. In this way, the driver 104 generates a sound wave due to the vibration of the diaphragm according to the output sound signal, the sound wave propagates through the air and is transmitted to the ear of the user 99, and the user 99 perceives the sound.

[0071] [Operation] Next, with reference to FIG. 6, the operation of the acoustic reproduction device 100 described above will be described. FIG. 6 is a flowchart showing the operation of the acoustic reproduction device according to the embodiment. First, when the operation of the acoustic reproduction device 100 is started, the acquisition unit 111 acquires sound information via the communication module 102. The sound information is decoded by the decoding processing unit 113 into information regarding a predetermined sound and information regarding a predetermined direction, and filter selection is started.

[0072] In the filter selection unit 121, as an initial value, a stereo filter that plays a predetermined sound so as to have an arrival direction (an arrival direction that coincides with a predetermined direction) preset in the content is read from the filter storage unit 122. On the other hand, in the fluctuation angle calculation unit 123, an angular amount of fluctuation in a predetermined direction is calculated (S101). Then, the fluctuation angle calculation unit 123 determines whether the angular amount of fluctuation is smaller than a threshold value (S102). When the angular amount of fluctuation is equal to or greater than the threshold value (No in S102), the filter selection unit 121 ends the process and outputs a stereo filter in which the predetermined direction and the arrival direction coincide to the output sound generation unit 131.

[0073] On the other hand, when the angular amount of fluctuation is smaller than the threshold value (Yes in S102), determination of a stereo filter by the filter determination unit 124 (S103) is performed. The determination of the stereo filter can also be read as making a selection for changing the stereo filter selected as the initial value. The arrival direction of the sound in the output sound signal at this time is a direction different from the predetermined direction on the sound information.

[0074] Note that the stereo filter directly determined by the filter determination unit 124 may be read from the filter storage unit 122 without setting the initial value of the stereo filter as described above. That is, the change of the stereo filter is an expression used for convenience of explanation, and directly selecting and outputting the stereo filter without using the initial value is also included in the present disclosure.

[0075] Hereinafter, the determination of the stereo filter will be described with reference to FIGS. 7 to 9. FIG. 7 is a first diagram for explaining the arrival direction of a predetermined sound by the selected stereo filter according to the embodiment. FIG. 8 is a second diagram for explaining the arrival direction of a predetermined sound by the selected stereo filter according to the embodiment. FIG. 9 is a third diagram for explaining the arrival direction of a predetermined sound by the selected stereo filter according to the embodiment.

[0076] In FIG. 7, the arrival direction of a predetermined sound when the stereophonic filter is not changed is shown on the left side of the white right arrow. The arrival direction of the predetermined sound at the first time point is indicated by a solid line, and the arrival direction of the predetermined sound at the second time point following the first time point is indicated by a dashed line. Further, in FIG. 7, the arrival direction of the predetermined sound when the stereophonic filter is changed is shown on the right side of the white right arrow. The arrival direction of the predetermined sound at the first time point is indicated by a solid line, and the arrival direction of the predetermined sound at the second time point following the first time point is indicated by a dashed line. Also, in FIG. 7, the user 99 in the posture with the direction on the paper surface as the front is schematically shown as a circle with "U" attached. This user 99 is in an upright posture in a direction perpendicular to the paper surface.

[0077] Furthermore, in FIG. 7, the position where the predetermined sound is localized is shown as a black dot, and virtual speakers are also shown.

[0078] As shown in FIG. 7, the position where the first predetermined sound at the first time point is localized is the first position S1. This first predetermined sound moves to the second position S1a at the second time point when there is no change in the stereophonic filter. The predetermined direction rotates from the first direction connecting the first position S1 and the user 99 to the second direction connecting the second position S1a and the user 99. It is assumed that the first predetermined sound moves linearly both temporally and spatially between the first time point and the second time point. When the amount of rotation from this first direction to the second direction (the amount of variation in the arrival direction of the predetermined sound) is smaller than the threshold value, it is difficult for the user 99 to recognize that the first predetermined sound has moved.

[0079] On the other hand, by changing the stereophonic filter, the predetermined sound moves to the third position S1b at the second time point. The predetermined direction rotates from the first direction connecting the first position S1 and the user 99 to the third direction connecting the third position S1b and the user 99. The angular difference between the second direction and the third direction (the angle enlarged by the change) may be a fixed angle such as 5 degrees, 10 degrees, 15 degrees, 20 degrees, etc., or based on the angular difference between the first direction and the second direction, the angular difference between the first direction and the third direction may be an angle that sufficiently exceeds the minimum discrimination angle of a human (about 10 degrees).

[0080] Also, the smaller the angular difference between the first direction and the second direction (i.e., the angular amount of variation in the predetermined direction on the original sound information), the larger the angular difference between the second direction and the third direction may be. Specifically, the filter determination unit 124 may determine the stereophonic filter such that the smaller the angular amount of variation in the predetermined direction on the original sound information, the larger the angular amount of variation in the arrival direction of the output sound signal when the changed stereophonic filter is used. For example, as shown in FIG. 8, an expansion coefficient α (α>1), and the smaller the angular amount of variation in the predetermined direction on the sound information, the larger the numerical value of the expansion coefficient α is multiplied by the angular difference between the first direction and the second direction to determine the third direction. From this third direction, the stereophonic filter may be changed such that a predetermined sound at the second time point arrives. Also, this expansion coefficient α may have a non-linear relationship in order to expand the angular amount of variation caused by the change as the angular amount of variation is smaller.

[0081] Also, in FIG. 7, the one-dot chain line extending to the left and right of the user 99 indicates a virtual boundary surface that divides the head of the user 99 into front and back. This boundary surface may be a surface along the external auditory canal of the user 99, a surface passing through the rearmost point of the auricle of the user 99, or simply a surface passing through the center of gravity of the head of the user 99. It is known that there is a difference in the ease of hearing sound in front of and behind such a boundary surface, that is, in front of and behind the user 99. Therefore, it is effective to make the characteristics of the change in the stereophonic filter different between the front surface side and the rear surface side with the boundary surface as the boundary.

[0082] In FIG. 7, the positioned position of the second predetermined sound at the first time point is the fourth position S2. This second predetermined sound moves to the fifth position S2a at the second time point when there is no change in the stereo filter. The predetermined direction rotates from the fourth direction connecting the fourth position S2 and the user 99 to the fifth direction connecting the fifth position S2a and the user 99. The first direction and the fourth direction are parallel, and the second direction and the fifth direction are parallel. Therefore, during the period from the first time point to the second time point, the angular amount of variation of the second predetermined sound is equal to the angular amount of variation of the first predetermined sound. However, since the second predetermined sound is on the rear side (behind the user 99) of the boundary surface, it can be said that the variation is less recognizable by the user 99 compared to the front side.

[0083] For this reason, the filter determination unit 124 determines the stereo filter such that the angular amount of variation of the arrival direction on the time axis when using the changed stereo filter when the predetermined direction is on the front side of the boundary surface is smaller than the angular amount of variation of the arrival direction on the time axis when using the changed stereo filter when the predetermined direction is on the rear side of the boundary surface. For example, in FIG. 7, the positioned position of the predetermined sound changes from the fourth position S2 in the fourth direction to the sixth position S2b so as to have a variation angle larger than the angle formed by the first direction and the third direction. The angular difference between the fifth direction and the sixth direction may be a fixed angle such as 10 degrees, 15 degrees, 20 degrees, 25 degrees, etc., or may be set to be an angular difference that is a multiple of the angular difference on the front side, such as twice, three times, four times, five times, etc.

[0084] Another example of the determination of the stereo filter by the filter determination unit 124 is shown in FIG. 9. FIG. 9 shows a diagram having the same configuration as that in FIG. 7, and the configuration after the change of the stereo filter (the right side of the white right arrow) is different. Note that in FIG. 9, only the sound image on the front side of the boundary surface is shown, and on the right side of the white right arrow, for readability, the illustration regarding the first position S1 where the first predetermined sound is positioned at the first time point is omitted.

[0085] As shown in FIG. 9, the first predetermined sound at the second time point is localized at the seventh position S1c, and a stereophonic filter may be selected so as to vibrate separately from the fluctuation in the predetermined direction on the sound information. Here, around the seventh position S1c, within the angular region sandwiched by two dashed-dotted lines, the state in which the first predetermined sound is vibrating is shown. In this way, the first predetermined sound may be emphasized by a method that does not expand the magnitude of the fluctuation in the arrival direction. In the example of FIG. 9, by causing periodic and regular fluctuations in the first predetermined sound, the user 99 is made to direct their attention, and even if the change is somewhat small, it can be made easier for the user 99 to recognize by paying attention.

[0086] Note that such periodic and regular fluctuations can be caused by multiplying or adding a vibration function whose numerical values vibrate on the time axis, such as a sine function or a cosine function, to the angular amount of the fluctuation in the arrival direction of the predetermined sound. For example, when using the changed stereophonic filter, the arrival direction (corresponding to the changed angular amount) at the Nth (N is an integer of 2 or more) time point on the time axis of the output sound signal is the difference value (original angular amount) between the predetermined direction on the sound information corresponding to the (N - 1)th time point on the time axis of the output sound signal and the predetermined direction on the sound information corresponding to the Nth time point, and the numerical value at the corresponding time point in the vibration function is multiplied, and the multiplied difference value is added to the predetermined direction on the sound information corresponding to the (N - 1)th time point to calculate.

[0087] In addition, in order to emphasize the arrival direction of the predetermined sound at the second time point, the stereophonic filter may be changed so that the amount of change in the sound pressure of the predetermined sound on the time axis perceived by the output sound signal when using the changed stereophonic filter is larger than when using the stereophonic filter before the change. Also, these examples of changing the stereophonic filter are not mutually exclusive and may be used as any combination.

[0088] In this way, in the present embodiment, when the variation in the arrival direction of a predetermined sound is smaller than a threshold value, the phenomenon that this variation is difficult for the user 99 to recognize can be used to generate an output sound signal by changing the stereo filter so as to emphasize the variation. Therefore, since the small variation in the arrival direction of the predetermined sound that is difficult for the user 99 to recognize becomes clearer, it is possible to more appropriately cause the user 99 to perceive three-dimensional sound.

[0089] (Other embodiments) As described above, the embodiments have been explained, but the present disclosure is not limited to the above-described embodiments.

[0090] For example, in the above embodiment, an example where the sound does not follow the movement of the user's head has been described, but the content of the present disclosure is also effective when the sound follows the movement of the user's head. That is, in the operation of causing the user to perceive a predetermined sound as the sound arriving from the first position that relatively moves with the movement of the user's head, when the amount of variation in the arrival direction of the predetermined sound is smaller than the threshold value, a stereo filter may be selected so that the variation is emphasized.

[0091] Also, for example, the sound playback device described in the above embodiment may be realized as a single device having all the components, or may be realized by allocating each function to a plurality of devices and having these plurality of devices cooperate with each other. In the latter case, an information processing device such as a smartphone, a tablet terminal, or a PC may be used as the device corresponding to the processing module.

[0092] As a configuration different from the description of the above embodiment, for example, the decoding processing unit can also select a modified stereo filter by correcting the original sound information. Specifically, the decoding processing unit in this example is a processing unit that generates information regarding a predetermined direction included in the sound information and corrects the original sound information. The decoding processing unit calculates the angular amount of variation in a predetermined direction on the time axis, and when the calculated angular amount of variation in the predetermined direction is smaller than a threshold value, it corrects the information regarding the predetermined direction so that a predetermined sound is more emphasized and perceived by the user compared to the case where the angular amount of variation in the predetermined direction is equal to or greater than the threshold value. Thereby, based on the corrected information regarding the predetermined direction output from the decoding processing unit, only a stereo filter that defines the arrival direction from which the predetermined sound arrives is selected, and the modified stereo filter in the above embodiment is applied.

[0093] In this way, the information processing method and the like disclosed in the present application may also be realized by correcting the information regarding the predetermined direction in the original sound information. The above-described decoding processing unit can realize an acoustic playback device that can achieve the same effect as the present disclosure by simply replacing and inserting it with a processing unit that performs the decoding processing of a conventional stereo playback device.

[0094] Further, the acoustic playback device of the present disclosure can also be realized as an acoustic processing device that is connected to a playback device including only a driver and outputs an output sound signal using the stereo filter selected based on the acquired sound information to the playback device. In this case, the acoustic processing device may be realized as hardware including a dedicated circuit, or may be realized as software for causing a general-purpose processor to execute a specific process.

[0095] Also, in the above embodiment, the process executed by a specific processing unit may be executed by another processing unit. Also, the order of a plurality of processes may be changed, or a plurality of processes may be executed in parallel.

[0096] In addition, in the above-described embodiments, each component may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or a processor reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

[0097] Also, each component may be realized by hardware. For example, each component may be a circuit (or an integrated circuit). These circuits may form one circuit as a whole, or may be separate circuits respectively. Further, these circuits may be general-purpose circuits or dedicated circuits respectively.

[0098] Also, the general or specific aspects of the present disclosure may be realized by an apparatus, an apparatus, a method, an integrated circuit, a computer program, or a recording medium such as a computer-readable CD-ROM. Further, the general or specific aspects of the present disclosure may be realized by any combination of an apparatus, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.

[0099] For example, the present disclosure may be realized as a method for reproducing an audio signal executed by a computer, or may be realized as a program for causing a computer to execute the method for reproducing an audio signal. The present disclosure may be realized as a computer-readable non-transitory recording medium on which such a program is recorded.

[0100] In addition, forms obtained by applying various modifications that can be conceived by those skilled in the art to each embodiment, or forms realized by arbitrarily combining the components and functions in each embodiment without departing from the gist of the present disclosure are also included in the present disclosure.

Industrial Applicability

[0101] The present disclosure is useful in acoustic reproduction such as enabling a user to perceive three-dimensional sound.

Explanation of Signs

[0102] 99 users 100 audio playback devices 101 processing module 102 communication module 103 detector 104 driver 111 acquisition unit 112 encoded audio information input unit 113 decoding processing unit 114 sensing information input unit 121 filter selection unit 122 filter storage unit 123 variable angle calculation unit 124 filter determination unit 131 output sound generation unit 132 filter processing unit 141 signal output unit 200 stereoscopic video playback device S1 First position S1a Second position S1b Third position S1c Seventh position S2 Fourth position S2a Fifth position S2b Sixth position

Claims

1. obtaining sound information including a sound signal related to a predetermined sound and information related to a predetermined position at each time point on a time axis, and sensor information indicating the coordinates and orientation of a user in a three-dimensional sound field; calculating a first predetermined position at a first time point and a first arrival direction based on the coordinates and orientation of a listener at the first time point using the sound information and the sensor information; calculating a second predetermined position at a second time point and a second arrival direction based on the coordinates and orientation of a listener at the second time point using the sound information and the sensor information; determining whether a first angular difference indicating an angular difference between the first arrival direction and the second arrival direction is smaller than a threshold value; when the first angular difference is smaller than the threshold value; determining a third arrival direction by correcting the second arrival direction; generating an output sound signal by convolving the sound signal with a head-related transfer function corresponding to the corrected third arrival direction An information processing method.

2. In the determination of the third arrival direction, the second arrival direction is corrected so that a second angular difference indicating an angular difference between the first arrival direction and the third arrival direction is larger than the first angular difference. The information processing method according to Claim 1.

3. In the determination of the third arrival direction, the third arrival direction is determined such that the second angular difference is expanded by multiplying an expansion coefficient α (α > 1) whose numerical value increases as the first angular difference becomes smaller. The relationship between the first angular difference and the expansion coefficient α is non-linear. The information processing method according to Claim 2.

4. In the determination of the third arrival direction, when the first predetermined position is on the rear side of the virtual boundary surface that divides the user's head into front and rear, the third arrival direction is determined so that the second angular difference is larger than when the first predetermined position is on the front side of the boundary surface with respect to the virtual boundary surface. The information processing method according to Claim 2.

5. In the determination of the third arrival direction, the third arrival direction is determined so that the third arrival direction perceived by the output sound signal vibrates on the time axis compared to the first arrival direction. The information processing method according to Claim 1.

6. When convolving with the head-related transfer function corresponding to the third arrival direction, the arrival direction at the Nth (N is an integer of 2 or more) time point on the time axis of the output sound signal is Multiply the difference value between the arrival direction corresponding to the (N - 1)-th time point on the time axis of the output sound signal and the arrival direction corresponding to the N-th time point by the value at the corresponding time point in an oscillation function whose value oscillates on the time axis, The calculated value after multiplication is added to the arrival direction corresponding to the (N - 1)-th time point The information processing method according to claim 5.

7. In the generation of the output sound signal, when the first angular difference is smaller than the threshold value, the amount of change in the sound pressure of the predetermined sound on the time axis perceived by the output sound signal generated by convolving the head transfer function corresponding to the third arrival direction is greater than the amount of change in the sound pressure of the predetermined sound perceived by the output sound signal generated when the first angular difference is greater than or equal to the threshold value, and the output sound signal is generated accordingly. The information processing method according to claim 1.

8. For causing a computer to execute the information processing method according to any one of claims 1 to 7 Program.

9. An acquisition unit that acquires sound information including a sound signal related to a predetermined sound and information related to a predetermined position at each time point on the time axis, and sensor information indicating the coordinates and orientation of the user in the three-dimensional sound field; A filter selection unit, Using the sound information and the sensor information, calculate a first predetermined position at a first time point and a first arrival direction based on the coordinates and orientation of the listener at the first time point; Using the sound information and the sensor information, calculate a second predetermined position at a second time point and a second arrival direction based on the coordinates and orientation of the listener at the second time point; Determine whether a first angular difference indicating the angular difference between the first arrival direction and the second arrival direction is smaller than a threshold value; A filter selection unit that determines a third arrival direction by correcting the second arrival direction when the first angular difference is smaller than the threshold value; An output sound generation unit that generates an output sound signal by convolving the sound signal with a head transfer function corresponding to the corrected third arrival direction; An output unit that outputs sound by using the generated output sound signal, comprising An acoustic playback device.

Images