Acoustic signal processing device, acoustic signal processing method, and program

By selectively distinguishing objects based on metadata in the audio signal processing device, and only the required objects are given a swing effect, the problem of waste and unnaturally swinging resources of stereo audio processing in virtual reality in the prior art is solved, and more appropriateness is achieved Stereo sound perception and processing efficiency.

CN118947143APending Publication Date: 2024-11-12PANASONIC INTELLECTUAL PROPERTY CORP OF AMERICA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202380030389.1
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-07-29
Filing Date
2023-04-05
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The prior art devices used in stereo processing in virtual reality (VR) or augmented reality (AR) fail to effectively and appropriately enable users to perceive stereo sound, especially when processing multiple objects, resulting in wasteful and unnatural processing resources. swing effect.

Method used

By selectively distinguishing objects based on position, orientation and directional information in the metadata in the audio signal processing device, only the required objects are given a swing effect, avoiding processing of moving or rotating objects, thereby reducing processing resources Expand and improve the actual presence and positioning.

Benefits of technology

It realizes that users can perceive stereo sound more appropriately, reduces waste of processing resources, improves processing efficiency, avoids unnatural swing effects, and enhances the presence of objects in the virtual sound space.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118947143A_ABST
    Figure CN118947143A_ABST
Patent Text Reader

Abstract

The audio objects respectively contain sound data of sound emitted by the objects and metadata containing position information of the objects in the sound space; an acoustic signal processing device (101) is provided with: a selection unit (141) that selects an audio object to be converted from among one or more audio objects; and a swing imparting unit (151) which, by converting the selected audio object to be converted, imparts a swing effect to the audio object, the swing effect being a sound swing emitted by an object corresponding to the converted audio object when the acoustic signal is reproduced. The selection unit (141) does not select the audio object corresponding to the object whose position is moving in the sound space on the basis of the temporal transition of the position information included in the metadata.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to an audio signal processing device, an audio signal processing method, and a program. Background Art

[0002] In recent years, the development of technologies related to virtual experiences from the user's perspective, such as virtual reality (VR) or augmented reality (AR), has been promoted. In VR or AR, users can feel (i.e., immerse) themselves in a virtual space. In particular, the sense of immersion is enhanced by providing a stereoscopic auditory experience together with a stereoscopic visual experience, so in VR or AR, technologies related to stereoscopic auditory experience are also valued. Because of this background, the preparation of a standard called MPEG-I is being promoted as a standard for reproducing stereo sound in virtual spaces such as VR or AR.

[0003] In MPEG-I, the standardization is being advanced in a manner that expands the standard that previously envisioned the rotational freedom (3DoF) around each of the three orthogonal axes at a certain listening point, and further adds the freedom of movement of the listening point along the directions of the above three axes (in a manner that can achieve 6DoF). That is, the user can move in the virtual sound space and hear the sound of the object arranged in the sound space from various listening points.

[0004] On the other hand, an acoustic signal processing device has been developed that increases the real presence of an object by giving a swing effect to the sound emitted by the object in a virtual sound space (Patent Document 1).

[0005] More specifically, in the stereo sound processing device described as an example of an audio signal processing device in Patent Document 1, in order to give a stereoscopic sense of positioning to an object in a virtual sound space (the user perceives the position of the object in the sound space), in order to make the user notice the positioning of the object itself (the sound stimulus acts on the human consciousness), the following stereo sound processing is performed: in the sound space where the object is positioned, a slight change in the sense of positioning is generated by swaying the positioning of the sound image, thereby making it easy for the user to notice the stereoscopic positioning of the object itself. This processing is based on the following understanding: human hearing filters the changes in sound stimulation through the action of FM neurons, etc., and only the corresponding part of the change reaches the cerebral cortex, so in order to make people notice the sound stimulation, it is effective to give non-steady-state stimulation that always changes continuously.

[0006] Prior art literature

[0007] Patent Literature

[0008] Patent Document 1: Japanese Patent Application Publication No. 2005-295416 Summary of the invention

[0009] Problems to be solved by the invention

[0010] However, in the stereo processing described in Patent Document 1 mentioned above, appropriate processing may not be performed.

[0011] In view of the above, an object of the present disclosure is to provide an acoustic signal processing device or the like that allows a user to perceive stereoscopic sound more appropriately.

[0012] Means used to solve problems

[0013] A sound signal processing device according to a technical solution of the present disclosure is a sound signal processing device that converts input data into a sound signal, wherein the input data includes audio objects corresponding to one or more objects configured in a virtual sound space, and the audio objects respectively include: sound data of the sound emitted by the object corresponding to the audio object; and metadata including position information of the object corresponding to the audio object in the sound space; the sound signal processing device comprises: a selection unit that selects the audio object as a transformation object from among the one or more audio objects; and a swing imparting unit that imparts a swing effect to the audio object by transforming the audio object selected as the transformation object, wherein the swing effect refers to the swing of the sound emitted by the object corresponding to the transformed audio object when the sound signal is reproduced; the selection unit does not select the audio object corresponding to the object whose position in the sound space is moving as the transformation object based on the temporal transition of the position information included in the metadata.

[0014] In addition, a sound signal processing method related to a technical solution of the present disclosure is a sound signal processing method executed by a computer, which converts input data into a sound signal, the input data includes audio objects corresponding to one or more objects configured in a virtual sound space, the audio objects respectively include: sound data of the sound emitted by the object corresponding to the audio object; and metadata including position information of the object corresponding to the audio object in the sound space; in the sound signal processing method, the audio object as a transformation object is selected from the one or more audio objects; by transforming the audio object selected as the transformation object, a swing effect is given to the audio object, the swing effect means that when the sound signal is reproduced, the sound emitted by the object corresponding to the transformed audio object swings; in the selection of the audio object as the transformation object, based on the temporal transition of the position information included in the metadata, the audio object corresponding to the object whose position in the sound space is moving is not selected as the transformation object.

[0015] Furthermore, one technical aspect of the present disclosure may be implemented as a program for causing a computer to execute the above-described sound processing method.

[0016] In addition, these inclusive or specific technical solutions can also be implemented by systems, devices, methods, integrated circuits, computer programs, or non-temporary recording media such as computer-readable CD-ROMs, or by any combination of systems, devices, methods, integrated circuits, computer programs, and recording media.

[0017] Effects of the Invention

[0018] According to the present disclosure, it is possible to allow a user to perceive three-dimensional sound more appropriately. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a schematic diagram showing an example of use of the sound signal reproducing device according to the first embodiment.

[0020] Figure 2 This is a block diagram showing the functional structure of the sound signal reproducing device according to the first embodiment.

[0021] Figure 3 This is a flowchart showing the operation of the sound signal processing device according to the first embodiment.

[0022] Figure 4 This is a flowchart showing the operation of the sound signal processing device according to the second embodiment.

[0023] Figure 5 This is a flowchart showing the operation of the sound signal processing device according to the second embodiment. DETAILED DESCRIPTION

[0024] (Knowledge as the basis for disclosure)

[0025] As described in the background art section, there has been development of an acoustic signal processing device that increases the real presence of an object by giving a swing effect to the sound emitted by the object in a virtual sound space.

[0026] On the other hand, in a virtual sound space, the process of giving a swing effect to an object arranged therein requires processing the sound emitted by the object differently over time. Since a variety of objects are arranged in a virtual sound space imagined in VR or AR, giving a swing effect to each of them requires a huge amount of processing.

[0027] In particular, in the sound space that can achieve 6DoF as envisioned by MPEG-I, it is necessary to perform convolution processing of appropriate head-related transfer functions that vary over time based on the relationship between the relative positions of each object and the user's listening point that changes over time. Therefore, if in addition to this, computational processing is performed to impart the swaying effect described above, it is necessary to further expand the processing resources (also called computing resources) to match it, which becomes an obstacle to the popularization of VR or AR.

[0028] Here, not all objects in the sound space have their actual presence increased by applying the swing effect. For example, this is the case when applying the swing effect to an object that moves itself (mobile objects such as cars and airplanes, flying objects such as balls and bullets), an object that rotates itself (rotating objects such as gyroscopes and fans), or an object that emits non-directional sounds (wind chimes, etc.).

[0029] Furthermore, if a swaying effect is given to an object that cannot sway spontaneously (non-living things, i.e., inanimate objects, etc.), it will feel unnatural and the user is expected to feel uncomfortable. The same can be said when a swaying effect is given to an object that generally cannot move (non-moving objects, i.e., immovable objects, such as wall clocks and volcanoes).

[0030] In view of the above, in the sound signal processing device etc. disclosed in the present invention, the purpose is to enhance the actual sense of presence and positioning of the required objects by selectively distinguishing between objects to be given a swing effect (transformation objects) and objects not to be given a swing effect (non-transformation objects) from among a variety of objects arranged in the sound space, while on the other hand, objects that have less advantage in giving a swing effect are not processed to suppress the expansion of processing resources, thereby providing a sound signal processing device etc. that allows users to perceive stereo sound more appropriately from the perspective of processing efficiency.

[0031] (Public Summary)

[0032] A summary of the present disclosure is as follows.

[0033] The sound signal processing device of the first technical solution related to the present disclosure is a sound signal processing device that converts input data into a sound signal, the input data includes audio objects corresponding to one or more objects configured in a virtual sound space, the audio objects respectively include: sound data of the sound emitted by the object corresponding to the audio object; and metadata including position information of the object corresponding to the audio object in the sound space; the sound signal processing device comprises: a selection unit that selects an audio object as a transformation object from among the one or more audio objects; and a swing imparting unit that imparts a swing effect to the audio object by transforming the audio object selected as the transformation object, the swing effect being that the sound emitted by the object corresponding to the transformed audio object swings when the sound signal is reproduced; the selection unit does not select the audio object corresponding to the object whose position in the sound space is moving as the transformation object based on the temporal transition of the position information included in the metadata.

[0034] Such an acoustic signal processing device can prevent an audio object corresponding to an object whose position is moving in the sound space from being selected as a transformation object based on the temporal transition of position information included in metadata. For an object whose position is moving in the sound space, even if the sound is swayed, the effect of improving the sense of presence and the sense of localization brought about by the swaying may not be sufficiently obtained, so it may not match the allocation of processing resources used to give the swaying effect. Therefore, by preventing an audio object corresponding to an object whose position is moving in the sound space from being selected as a transformation object, the possibility of wasting processing resources can be reduced. In this way, by a selection unit that selects an audio object as a transformation object from one or more audio objects based on at least one of the displacement, rotation, and directivity of the sound emitted by the object, objects to be given the swaying effect (transformation objects) and objects to which the swaying effect is not given (non-transformation objects) are selectively distinguished from a variety of objects arranged in the sound space, thereby improving the sense of presence and the sense of localization for the necessary objects, and on the other hand, the objects to which the advantage of giving the swaying effect is less are not processed to suppress the expansion of processing resources, and the user can more appropriately perceive three-dimensional sound from the viewpoint of processing efficiency.

[0035] In addition, the sound signal processing device related to the second technical solution of the present disclosure is a sound signal processing device that converts input data into a sound signal, the input data includes audio objects corresponding to one or more objects configured in a virtual sound space, the audio objects respectively include: sound data of the sound emitted by the object corresponding to the audio object; and metadata, including orientation information of the object corresponding to the audio object in the sound space; the sound signal processing device comprises: a selection unit that selects an audio object as a transformation object from among the one or more audio objects; and a swing imparting unit that imparts a swing effect to the audio object by transforming the audio object selected as the transformation object, the swing effect being that the sound emitted by the object corresponding to the transformed audio object swings when the sound signal is reproduced; the selection unit does not select the audio object corresponding to the object rotating in the sound space as the transformation object based on the temporal transition of the orientation information included in the metadata.

[0036] Such an audio signal processing device can prevent an audio object corresponding to an object rotating in a sound space from being selected as a transformation object based on the temporal transition of the orientation information included in the metadata. For an object rotating in a sound space, even if the sound is swayed, the effect of improving the sense of presence and the sense of localization brought about by the swaying may not be sufficiently obtained, so it may not match the allocation of processing resources used to give the swaying effect. Therefore, by preventing an audio object corresponding to an object rotating in a sound space from being selected as a transformation object, the possibility of wasting processing resources can be reduced. In this way, by a selection unit that selects an audio object as a transformation object from one or more audio objects based on at least one of the displacement, rotation, and directivity of the sound emitted by the object, objects to be given a swaying effect (transformation objects) and objects to which a swaying effect is not given (non-transformation objects) are selectively distinguished from a variety of objects arranged in the sound space, thereby improving the sense of presence and the sense of localization for the required objects, and on the other hand, the objects to which the advantage of giving a swaying effect is less are not processed to suppress the expansion of processing resources, and the user can more appropriately perceive a three-dimensional sound from the viewpoint of processing efficiency.

[0037] In addition, the sound signal processing device related to the third technical solution of the present disclosure is a sound signal processing device that converts input data into a sound signal, the input data includes audio objects corresponding to one or more objects configured in a virtual sound space, the audio objects respectively include: sound data of the sound emitted by the object corresponding to the audio object; and metadata, including directionality information of the sound emitted by the object corresponding to the audio object; the sound signal processing device comprises: a selection unit, which selects an audio object as a transformation object from among the one or more audio objects; and a swing imparting unit, which imparts a swing effect to the audio object by transforming the audio object selected as the transformation object, the swing effect being that the sound emitted by the object corresponding to the transformed audio object swings when the sound signal is reproduced; the selection unit does not select the audio object corresponding to the object whose sound emitted is non-directional as the transformation object based on the directionality information included in the metadata.

[0038] Such an acoustic signal processing device can prevent an audio object corresponding to an object whose sound emitted by the object is non-directional from being selected as a transformation object based on the directivity information included in the metadata. For an object whose sound emitted by the object is non-directional, even if the sound is swung, there is a case where the effect of improving the sense of presence and the sense of positioning brought about by the swaying cannot be fully obtained, so it is sometimes not suitable for the allocation of processing resources used to give the swaying effect. Therefore, by preventing an audio object corresponding to an object whose sound emitted by the object is non-directional from being selected as a transformation object, the possibility of wasting processing resources can be reduced. In this way, by a selection unit that selects an audio object as a transformation object from one or more audio objects based on at least one of the displacement, rotation and directivity of the sound emitted by the object, objects to be given a swaying effect (transformation object) and objects to which a swaying effect is not given (non-transformation object) are selectively distinguished from a variety of objects arranged in a sound space, thereby improving the sense of presence and the sense of positioning for the required objects, and on the other hand, the objects to which the advantage of giving a swaying effect is less are not processed to suppress the expansion of processing resources, and the user can more appropriately perceive three-dimensional sound from the viewpoint of processing efficiency.

[0039] In addition, in the sound signal processing device of the fourth technical solution of the present disclosure, in the sound signal processing device described in any one of the first to third technical solutions, the metadata also includes a biological marker indicating whether the object belongs to a living thing; and the selection unit does not select the audio object whose biological marker indicates that the object does not belong to a living thing.

[0040] Such an audio signal processing device can select whether to give a swing effect to the audio object by how to set the biological marker included in the metadata (for example, what value is set). The biological marker can be automatically assigned a value based on whether the object is a living thing, according to the attributes of the object, etc., or can be set individually for each audio object when creating input data. And, based on the value represented by the biological marker in the metadata, it is appropriately determined whether to give a swing effect to the audio object depending on whether the object is a living thing.

[0041] In addition, in the sound signal processing device of the 5th technical solution of the present disclosure, in the sound signal processing device described in any one of the 1st to 4th technical solutions, the metadata also includes a moving body flag indicating whether the object is a moving body; and the selection unit does not select the audio object whose object is indicated by the moving body flag as not being a moving body.

[0042] Such an audio signal processing device can select whether to give a swing effect to the audio object by how to set the moving body flag included in the metadata (for example, what value is set). The moving body flag can be automatically assigned a value based on whether the object is a moving body according to the attributes of the object, or can be set individually for each audio object when creating input data. And, based on the value represented by the moving body flag of the metadata, it is appropriately determined whether to give a swing effect to the audio object according to whether the object is a moving body.

[0043] In addition, the sound signal processing method related to the sixth technical solution of the present disclosure is a sound signal processing method executed by a computer, which transforms input data into a sound signal, the input data includes audio objects corresponding to one or more objects configured in a virtual sound space, the audio objects respectively include: sound data of the sound emitted by the object corresponding to the audio object; and metadata, including position information of the object corresponding to the audio object in the sound space; in the sound signal processing method, an audio object as a transformation object is selected from one or more audio objects; by transforming the audio object selected as the transformation object, a swing effect is given to the audio object, and the swing effect means that when the sound signal is reproduced, the sound emitted by the object corresponding to the transformed audio object swings; in the selection of the audio object as the transformation object, based on the temporal migration of the position information included in the metadata, the audio object corresponding to the object whose position in the sound space is moving is not selected as the transformation object.

[0044] Such an acoustic signal processing method can produce the same effect as the acoustic signal processing device described in the first technical solution.

[0045] In addition, the sound signal processing method of the seventh technical solution of the present disclosure is a sound signal processing method executed by a computer, which transforms input data into a sound signal, the input data including audio objects corresponding to one or more objects configured in a virtual sound space, the audio objects respectively including: sound data of the sound emitted by the object corresponding to the audio object; and metadata including orientation information of the object corresponding to the audio object in the sound space; in the sound signal processing method, an audio object as a transformation object is selected from one or more audio objects; by transforming the audio object selected as the transformation object, a swing effect is given to the audio object, the swing effect means that when the sound signal is reproduced, the sound emitted by the object corresponding to the transformed audio object swings; in the selection of the audio object as the transformation object, based on the temporal transition of the orientation information included in the metadata, the audio object corresponding to the object rotating in the sound space is not selected as the transformation object.

[0046] Such an acoustic signal processing method can produce the same effect as the acoustic signal processing device described in the second technical solution.

[0047] In addition, the sound signal processing method of the 8th technical solution of the present disclosure is a sound signal processing method executed by a computer, which converts input data into a sound signal, and the input data includes audio objects corresponding to one or more objects configured in a virtual sound space, and the audio objects respectively include: sound data of the sound emitted by the object corresponding to the audio object; and metadata, including directionality information of the sound emitted by the object corresponding to the audio object; in the sound signal processing method, an audio object as a transformation object is selected from one or more audio objects; by transforming the audio object selected as the transformation object, a swing effect is given to the audio object, and the swing effect means that when the sound signal is reproduced, the sound emitted by the object corresponding to the transformed audio object swings; in the selection of the audio object as the transformation object, based on the directionality information included in the metadata, the audio object corresponding to the object whose sound emitted is non-directional is not selected as the transformation object.

[0048] Such an acoustic signal processing method can produce the same effect as the acoustic signal processing device described in the third technical solution.

[0049] Furthermore, a program according to a ninth aspect of the present disclosure is a program for causing a computer to execute the sound signal processing method according to any one of the sixth to seventh aspects.

[0050] Such a program can achieve the same effects as the sound signal processing method according to any one of the sixth to seventh technical aspects using a computer.

[0051] Hereinafter, the embodiments will be described in detail with reference to the drawings.

[0052] In addition, the embodiments described below all represent inclusive or specific examples. The numerical values, shapes, materials, constituent elements, configuration positions of constituent elements and connection forms, steps, the order of steps, etc. shown in the following embodiments are examples and are not intended to limit the present disclosure. In addition, regarding the constituent elements of the following embodiments that are not recorded in the independent claims representing the highest concept, they are described as arbitrary constituent elements. In addition, the figures are not necessarily strictly illustrated. In each figure, the same reference numerals are given to substantially the same structure, and repeated descriptions are sometimes omitted or simplified.

[0053] (Implementation Method 1)

[0054] [summary]

[0055] First, an overview of the sound signal reproducing device according to Embodiment 1 will be described. Figure 1 1 is a schematic diagram showing an example of using the sound signal reproducing device according to Embodiment 1. Figure 1 , (a) shows a user 99 using one of the sound signal reproducing apparatuses 100 in the two examples, and (b) shows a user 99 using the other of the sound signal reproducing apparatuses 100 in the two examples.

[0056] Figure 1 The sound signal reproducing device 100 shown is used together with a display device that displays an image for visual VR or AR experience or a device for stereoscopic image reproduction (neither of which is shown).

[0057] The sound signal reproducing device 100 is a sound prompting device worn on the head of the user 99. Therefore, the sound signal reproducing device 100 moves integrally with the head of the user 99. For example, the sound signal reproducing device 100 of this embodiment can be as follows: Figure 1 (a) is a so-called headphone type device, which can also be Figure 1 (b) shows two earbud-type devices that are independently worn on the left and right ears of the user 99. These two devices communicate with each other to synchronously present sounds for the right ear and the left ear.

[0058] The sound signal reproducing device 100 changes the prompt sound according to the movement of the head of the user 99, so that the user 99 perceives that the user 99 moves his head in the three-dimensional sound field. Therefore, the sound signal reproducing device 100 moves the three-dimensional sound field in the opposite direction to the movement of the user 99.

[0059] [structure]

[0060] Next, refer to Figure 2 The configuration of the sound signal reproducing device 100 according to the present embodiment will be described. Figure 2 This is a block diagram showing the functional structure of the sound signal reproducing device according to the first embodiment.

[0061] like Figure 2 As shown, the sound signal reproducing device 100 according to the present embodiment includes a sound signal processing device 101 , a communication module 102 , a sensor 103 , and a driver 104 .

[0062] The sound signal processing device 101 is a computing device for performing various signal processing in the sound signal reproducing device 100 . The sound signal processing device 101 includes, for example, a processor and a memory, and various functions are achieved by the processor executing a program stored in the memory.

[0063] The sound signal processing device 101 includes an acquisition unit 111 , a sound data extraction unit 121 , a metadata extraction unit 131 , a selection unit 141 , a swing imparting unit 151 , and a sound signal conversion unit 161 .

[0064] The acquisition unit 111 is configured to be able to communicate with the communication module 102. The communication module 102 is an interface device for accepting input data to the sound signal reproduction device 100. The communication module 102 includes, for example, an antenna and a signal converter, and receives input data from an external device through wireless communication. In more detail, the communication module 102 uses the antenna to receive a wireless signal representing the input data converted into a format for wireless communication, and the signal converter performs reconversion from the wireless signal to the input data. In this way, the sound signal reproduction device 100 acquires input data from the external device through wireless communication. The input data acquired by the communication module 102 is acquired by the acquisition unit 111. In this way, the input data is input to the sound signal processing device 101. In addition, the communication between the sound signal reproduction device 100 and the external device can also be performed through wired communication.

[0065] The input data obtained by the sound signal reproduction device 100 is encoded in a format specified by, for example, MPEG-I. As an example, the encoded input data includes information about the sound reproduced by the sound signal reproduction device 100 (sound data) and other information (metadata), and the other data (metadata) includes information related to the localization position when the sound image of the sound is localized at a predetermined position in the three-dimensional sound field (that is, perceived as a sound arriving from a predetermined direction). The sound data and the metadata correspond to one or more audio objects included in the input data. The audio object is information corresponding to virtual objects arranged in the sound space when the input data is converted into a sound signal and the sound signal is reproduced. Therefore, the input data includes the same number of audio objects as the number of objects arranged in the sound space. Similarly, the input data includes the same number of sound data as the number of objects arranged in the sound space. Similarly, the input data includes the same number of metadata as the number of objects arranged in the sound space.

[0066] For example, in the sound signal processing device 101, the conversion from input data to the sound signal is performed so that the sounds emitted by the plurality of objects including the first object and the second object in the sound space are perceived as sounds reaching the listening point (the position of the user 99 in the sound space) from the objects respectively arranged at their generation positions when the sound signal is reproduced. In addition, as described above, the position and direction of the listening point change in accordance with the movement of the head of the user 99.

[0067] By the stereoscopic sound when the sound signal is reproduced, for example, the sense of presence of the audiovisual content can be improved together with the image recognized by the display device. In addition, the input data may only include sound data. In this case, metadata may also be obtained separately. In addition, as described above, the input data includes the first sound data related to the sound emitted by the first object and the second sound data related to the sound emitted by the second object, but they may also be included separately. In other words, a plurality of input data separately including each audio object may be obtained, and each object may be positioned at a different position in the sound space by simultaneous reproduction. In this way, the form of the input data input is not particularly limited, as long as the sound signal reproduction device 100 (especially, the sound signal processing device 101) has an acquisition unit 111 corresponding to various forms of input data.

[0068] The acquisition unit 111 of the present embodiment includes, for example, a coded sound information input unit, a decoding processing unit, and a sensing information input unit.

[0069] The coded sound information input unit is a processing unit for the coded (in other words, encoded) input data acquired by the input acquisition unit 111. The coded sound information input unit outputs the input data to the decoding processing unit. The decoding processing unit converts one or more audio objects included in the sound information into a form that can be processed separately by decoding (in other words, decoding) the input data output from the coded sound information input unit. The sensing information input unit is described below together with the function of the sensor 103.

[0070] The sensor 103 is a device for detecting the movement speed of the user 99's head. The sensor 103 is composed of a combination of various sensors used for detecting movement, such as a gyro sensor and an acceleration sensor. In the present embodiment, the sensor 103 is built into the sound signal reproduction device 100, but for example, it can also be built into an external device such as a stereoscopic image reproduction device that operates in accordance with the movement of the user 99's head, similarly to the sound signal reproduction device 100. In this case, the sensor 103 may not be included in the sound signal reproduction device 100. In addition, as the sensor 103, an external camera device or the like can be used to capture the movement of the user 99's head, and the movement of the user 99's head can be detected by processing the captured image.

[0071] The sensor 103 is, for example, integrally fixed to the housing of the sound signal reproduction device 100, and detects the movement speed and amount of the housing. After the sound signal reproduction device 100 including the housing moves integrally with the head of the user 99 after being worn by the user 99, the sensor 103 can detect the movement speed and amount of the head of the user 99 as a result.

[0072] For example, as the movement speed of the user's 99 head, the sensor 103 can detect the angular velocity of rotation with at least one of the three axes orthogonal to each other in the sound space as the rotation axis, and can also detect the acceleration of displacement with at least one of the above three axes as the displacement direction.

[0073] For example, as the movement amount of the user's 99 head, the sensor 103 can detect the rotation amount with at least one of the three axes orthogonal to each other in the sound space as the rotation axis, and can also detect the displacement amount with at least one of the above three axes as the displacement direction.

[0074] The sensing information input unit of the acquisition unit 111 acquires the movement speed and movement amount of the user 99's head from the sensor 103. More specifically, the sensing information input unit acquires the movement amount of the user 99's head detected by the sensor 103 as the movement speed per unit time. In this way, the sensing information input unit acquires at least one of the rotation speed, the rotation amount, the displacement speed, and the displacement amount from the sensor 103 as the sensing result. The movement amount of the user 99's head acquired here is used to determine the coordinates and orientation (the position and direction of the listening point) of the user 99 in the sound space. In the sound signal reproduction device 100, based on the determined coordinates and orientation of the user 99, the relative position of each object is determined and the sound is reproduced. In other words, in the sound signal reproduction device 100, for each object position respectively arranged at a specified position in the sound space, the relative position and direction of the listening point are determined and the sound is reproduced.

[0075] The sound data extraction unit 121 is a processing unit that extracts sound data of the sound emitted by the corresponding object from the audio object as a data set for one or more audio objects obtained by decoding the input data. The sound data contains information related to the frequency and strength of the sound emitted by the object. The sound data extraction unit 121 outputs the extracted sound data to the sound signal conversion unit 161. The function of the sound signal conversion unit 161 will be described later.

[0076] The metadata extraction unit 131 is a processing unit that extracts metadata including position information of the corresponding object from the audio objects as a data set for one or more audio objects obtained by decoding the input data. The metadata includes, in addition to the position information of the object, orientation information related to the orientation of the object, preference information related to the directionality of the sound emitted by the object, a biological marker indicating whether the object is a living thing, and a moving body marker indicating whether the object is a moving body. The metadata is used to process the sound emitted by the object corresponding to the audio object containing the metadata, from which position in the sound space, in which direction, and with what directionality (characteristics regarding the intensity of the sound generated in each direction from the sound generation position). Therefore, the metadata extraction unit 131 outputs the extracted metadata to the sound signal conversion unit 161.

[0077] In this embodiment, information included in the metadata is used to select whether or not the audio object is to be given a swing effect. This selection process is performed by the selection unit 141. Therefore, the metadata extraction unit 131 also outputs the extracted metadata to the selection unit 141.

[0078] The selection unit 141 is a processing unit that selects an audio object as a transformation object (i.e., a transformation object to be given a swing effect) from one or more audio objects. The selection unit 141 uses the metadata output from the metadata extraction unit to perform the above-mentioned determination. Specifically, the selection unit 141 uses the position information of the object included in the metadata to determine whether the object is moving. If the object is stopped, the audio object is selected as a transformation object to give a swing effect to the audio object. Conversely, if the object is moving, the swing effect is not given to the audio object.

[0079] Here, the swing effect is explained. As mentioned above, the swing effect is a signal processing for making the sound emitted by the object swing in the time domain when the sound signal is reproduced. Sound swing refers to the position of the object emitting the sound changing in the time domain (position swing), the orientation of the object emitting the sound changing in the time domain (angle swing), and the directivity of the object emitting the sound changing in the time domain (directivity swing). In the present embodiment, any swing selected from one or more of the above can be applied, and there is no particular limitation on the type and combination of swings. Regarding such a swing of the sound, in order to prevent the swing itself from causing a sense of awkwardness to the user 99, signal processing is performed in a manner that swings within a small range. Therefore, in the case where the object moves or rotates beyond the range of the swing, or in the case where there is no directivity of the object (non-directional), even if the swing effect is applied, the effect will be offset.

[0080] In the selection unit 141, the object is determined to be moving when a position change beyond the range of the swing occurs, such as to offset the swing effect. For example, when the object moves at a speed exceeding the average walking speed of a person, the selection unit 141 determines that the object is moving so as not to give the swing effect. In this case, even if the object moves at a speed lower than the average walking speed of a person, the selection unit 141 determines that the object is not moving (i.e., stopped). Therefore, the stop here refers to a substantial stop, such as movement below a certain speed.

[0081] The selection unit 141 determines whether the object has moved based on the temporal migration of the position information of the object. The selection unit 141 only selects the audio object corresponding to the object determined to be stationary. However, in the case where the object corresponding to the audio object is an inanimate object or an immovable object, giving a swaying effect will instead create a sense of awkwardness, so the selection unit 141 is configured not to select such inanimate objects or immovable objects. Specifically, the metadata includes a biological marker and a moving body marker, and the selection unit 141 selects an audio object whose biological marker is true, indicating that it belongs to a biological object, and whose moving body marker is true, indicating that it is a moving body. In addition, the determination using the biological marker and the moving body marker is not necessary. The selection unit 141 may also determine the selection of the audio object only according to whether the object has moved.

[0082] The information that an audio object is selected is associated with information such as an ID that specifies the audio object and is output to the swing imparting unit 151. The information that an audio object is not selected may not be output, or may be associated with information such as an ID that specifies the audio object and output to the swing imparting unit 151 in the same manner as described above.

[0083] The swing imparting unit 151 is a processing unit that performs transformation to impart a swing effect to the audio object selected by the selection unit 141. The imparting of the swing effect is performed, for example, by transforming the data included in the metadata so that it changes over time. In this example, the transformed metadata is output to the sound signal conversion unit 161, and the processing of the sound signal conversion unit 161 is performed according to the metadata transformed by updating the metadata output from the metadata extraction unit 131.

[0084] The sound signal conversion unit 161 performs processing based on the sound data and metadata so that the sound emitted by the object corresponding to each audio object is generated in the sound space at the position, direction and directivity of the information contained in the metadata. The sound signal conversion unit 161 performs processing to convert the sound signal as an analog signal based on the sound data and metadata, and outputs the converted sound signal. If the audio object is selected by the selection unit 141, it becomes an audio object given a swing effect by the swing giving unit 151. For example, in the example of converting the metadata, the metadata is updated to change the position, direction and directivity with time. Therefore, if the processing is performed so that the position, direction and directivity of the information contained in the updated metadata are generated, a sound signal that sounds like it arrives at the listening point in a swinging manner is generated.

[0085] In addition, in the generation of the sound signal, the sound signal conversion unit 161 uses the coordinates and orientation (position and direction of the listening point) of the user 99 in the sound space obtained by information processing via the sensor 103, etc., and convolves various head-related transfer functions so that the sound arrives from the position of each object, thereby generating a sound signal that can reproduce the sound that arrives at the listening point from the positions of each object respectively configured at a specified position in the sound space.

[0086] Then, the sound signal conversion unit 161 outputs the generated sound signal to the driver 104. The sound signal conversion unit 161 causes the driver 104 to generate sound waves based on the waveform signal represented by the sound signal, and prompts the user 99 with sound. The driver 104 has, for example, a vibration plate, and a driving mechanism such as a magnet and a voice coil. The driver 104 operates the driving mechanism according to the waveform signal, and vibrates the vibration plate through the driving mechanism. In this way, the driver 104 generates sound waves through the vibration of the vibration plate corresponding to the sound signal, and the sound waves propagate in the air and are transmitted to the ears of the user 99, and the user 99 perceives the sound.

[0087] [action]

[0088] Next, refer to Figure 3 The operations of the sound signal reproducing device 100 and the sound signal processing device 101 described above will be described. Figure 3 1 is a flowchart showing the operation of the sound signal processing device according to Embodiment 1. First, when the operation of the sound signal reproducing device 100 is started, the acquisition unit 111 acquires input data via the communication module 102 (S101). The input data is decoded to extract one or more audio objects. Next, the sound data extraction unit 121 extracts sound data from the audio object (S102). The extracted sound data is output to the sound signal conversion unit 161. On the other hand, the metadata extraction unit 131 extracts metadata from the audio object (S103). The extracted metadata is output to the selection unit 141 and the sound signal conversion unit 161.

[0089] The selection unit 141 first determines whether the object is moving based on the temporal migration of the position information included in the metadata (S104). If it is determined that the object is not moving, that is, it is stopped ("No" in S104), the selection unit 141 further determines whether it indicates that the object does not belong to a living thing based on the biological marker (S105). If it is determined that it does not indicate that the object does not belong to a living thing, that is, it belongs to a living thing ("No" in S105), the selection unit 141 further determines whether it indicates that the object is not a moving body based on the moving body marker (S106). If it is determined that it does not indicate that the object is not a moving body, that is, it is a moving body ("No" in S106), the selection unit 141 selects the audio object (S107). On the other hand, when it is determined that the object is moving ("Yes" in S104), when it is determined that the object is not a living thing (inanimate object) ("Yes" in S105), and when it is determined that the object is not a moving object (inanimate object) ("Yes" in S106), the selection unit 141 does not select the audio object (S108).

[0090] Then, it is determined whether the selection / non-selection determination is completed for all audio objects (S109). If the selection / non-selection determination is not completed for all audio objects (No in S109), the same process is repeated from step S102 for the next audio object.

[0091] If the selection / non-selection determination is completed for all audio objects (Yes in S109), the swing imparting unit 151 imparts a swing effect to the selected audio objects (S110). In addition, the swing imparting unit 151 does not impart a swing effect to the unselected audio objects.

[0092] Next, the sound signal conversion unit 161 converts the sound into a sound signal and outputs the sound to the driver 104, thereby reproducing the sound in which the swing effect is applied only to the required audio object.

[0093] In this way, by selectively distinguishing objects to be given a swing effect (transformation objects) and objects not to be given a swing effect (non-transformation objects) from among a variety of objects arranged in the sound space, the actual presence and positioning sense of the required objects are improved. On the other hand, objects with less advantage of giving a swing effect are not processed, thereby suppressing the expansion of processing resources. From the perspective of processing efficiency, an audio signal processing device can be implemented that allows users to perceive three-dimensional sound more appropriately.

[0094] In addition, in the first embodiment, it is assumed that no swing component is given when the object moves or rotates, but it is effective to give a swing effect when the object moves or rotates in the following cases. That is, by giving a swing in a direction other than the direction of movement or a rotational swing to an object moving in a straight line, or by making the position of an object rotating at a fixed position swing back and forth, left and right, or up and down, the effect of the present application can be exerted. Therefore, when the relationship with the type of swing given satisfies the specific situation as described above, the audio object corresponding to the object is selected as the transformation object. For example, in Figure 3 After the judgment is "yes" in step S104, a judgment is made as to whether the audio object meets the above situation. If the audio object meets the above situation in this judgment, the process proceeds to step S105. On the other hand, if the audio object does not meet the above situation in the judgment as to whether the audio object meets the above situation, the process proceeds to step S108.

[0095] (Implementation Method 2)

[0096] Next, the sound signal reproducing device according to the second embodiment will be described. In the second embodiment, the differences from the first embodiment will be described, and the description of the corresponding structures or substantially the same structures will be omitted. In the second embodiment described below, the operation of the selection unit 141 is different, and the other parts are the same. Therefore, the second embodiment can also be described as follows. Figure 2 The illustrated sound signal reproducing device 100 is regarded as a sound signal reproducing device according to the second embodiment.

[0097] Figure 4 1 is a flowchart showing the operation of the sound signal processing device according to Embodiment 2. Figure 4 As shown, in the sound signal processing device according to the second embodiment, step S104a is executed instead of step S104 of the sound signal processing device 101 according to the first embodiment.

[0098] In the second embodiment, the selection unit 141 uses the orientation information of the object included in the metadata to determine whether the object is rotating, and if the object is not rotating, the audio object is selected as the transformation object to give the sway effect to the audio object. Conversely, if the object is rotating, the sway effect is not given to the audio object.

[0099] The object is determined to be rotating in the selection unit 141 when a change in orientation (rotating) occurs that offsets the swaying effect, i.e., exceeds the range of the swaying. For example, when the orientation of the object changes by more than a threshold value such as 5 degrees, 10 degrees, 15 degrees, 45 degrees, 60 degrees, 90 degrees, or 180 degrees within a specified period, the selection unit 141 determines that the object is rotating so as not to impart the swaying effect. At this time, the selection unit 141 determines that the object is not rotating (i.e., not rotating) even if the orientation of the object changes by an angle below the threshold. Therefore, not rotating here refers to substantial non-rotation including changes in orientation below the threshold.

[0100] In step S104a, the selection unit 141 first determines whether the object is rotating based on the temporal transition of the orientation information included in the metadata. If it is determined that the object is not rotating, that is, there is no rotation ("No" in S104a), the process proceeds to step S105. In addition, if it is determined that the object is rotating ("Yes" in S104a), the process proceeds to step S108. The subsequent processing is performed in the same manner as in the first embodiment.

[0101] (Implementation 3)

[0102] Next, the sound signal reproducing device according to the third embodiment will be described. In the third embodiment, the differences from the first embodiment will be described, and the description of the corresponding structures or substantially the same structures will be omitted. In the third embodiment described below, the operation of the selection unit 141 is different, and the other parts are the same. Figure 2 The illustrated sound signal reproducing device 100 is regarded as a sound signal reproducing device according to the third embodiment.

[0103] Figure 5 : is a flowchart showing the operation of the sound signal processing device according to Embodiment 3. Figure 5 As shown, in the sound signal processing device according to the third embodiment, step S104b is executed instead of step S104 of the sound signal processing device 101 according to the first embodiment.

[0104] In the third embodiment, the selection unit 141 uses the directional information of the object included in the metadata to determine whether the object is non-directional. If the object is not non-directional, the audio object is selected as the transformation object to give the swing effect to the audio object. Conversely, if the object is non-directional, the swing effect is not given to the audio object.

[0105] The object is determined to be non-directional in the selection unit 141, which is a case where the object does not have a directivity that is not sharp, such as a directivity that cancels out the swaying effect, that is, a change in the range of the swaying. For example, in the case of a directivity whose sharpness cannot be perceived based on the human auditory resolution, the selection unit 141 determines that the object is non-directional so as not to impart the swaying effect. At this time, even if the object has a directivity with a sharpness less than the sharpness that cannot be perceived based on the human auditory resolution, the selection unit 141 determines that the object does not have directivity (i.e., non-directional). Therefore, the non-directional here refers to substantial non-directionality, including the case of a directivity with a sharpness less than the sharpness that cannot be perceived based on the human auditory resolution.

[0106] In step S104b, the selection unit 141 first determines whether the object is non-directional based on the directional information included in the metadata. If it is determined that the object is not non-directional, that is, it has directionality ("No" in S104b), the process proceeds to step S105. In addition, if it is determined that the object is non-directional ("Yes" in S104b), the process proceeds to step S108. The subsequent processing is performed in the same manner as in Implementation 1.

[0107] Furthermore, two or more of the first to third embodiments may be combined to determine whether to provide a swing effect, that is, whether to select or not select an audio object, under a plurality of conditions.

[0108] (Other embodiments)

[0109] As mentioned above, although embodiment was described, this disclosure is not limited to the said embodiment.

[0110] For example, in the above-mentioned embodiment, an example in which the sound does not follow the user's head movement is described, but the content of the present disclosure is also effective in the case where the sound follows the user's head movement. That is, in the action of making the user perceive the prescribed sound as a sound arriving from the first position that moves relatively with the movement of the user's head, it is possible to determine whether to swing the prescribed sound based on metadata to enhance the actual presence and positioning of the object emitting the prescribed sound.

[0111] In addition, for example, the sound signal reproducing device described in the above-mentioned embodiment can be implemented as a single device having all the components, or can be implemented by assigning each function to multiple devices and cooperating with the multiple devices. In the latter case, as a device equivalent to the processing module, an information processing device such as a smart phone, a tablet terminal or a PC can also be used.

[0112] In addition, the sound signal reproducing device of the present disclosure can also be implemented as a sound processing device that is connected to a reproducing device having only a driver and outputs only a sound signal to the reproducing device based on the acquired input data. In this case, the sound signal processing device can be implemented as hardware having a dedicated circuit or as software for causing a general-purpose processor to perform specific processing.

[0113] Furthermore, in the above-described embodiment, the processing executed by a specific processing unit may be executed by another processing unit. Furthermore, the order of a plurality of processing may be changed, or a plurality of processing may be executed in parallel.

[0114] In addition, in the above-mentioned embodiments, each component can also be implemented by executing a software program suitable for each component. Each component can also be implemented by a program execution unit such as a CPU or a processor reading and executing a software program recorded in a recording medium such as a hard disk or a semiconductor memory.

[0115] In addition, each component may also be implemented by hardware. For example, each component may also be a circuit (or integrated circuit). These circuits may constitute one circuit as a whole, or they may be different circuits. In addition, these circuits may be general circuits or dedicated circuits.

[0116] In addition, the entirety or specific form of the present disclosure may also be implemented by an apparatus, device, method, integrated circuit, computer program, or a computer-readable recording medium such as a CD-ROM. In addition, the entirety or specific form of the present disclosure may also be implemented by any combination of an apparatus, device, method, integrated circuit, computer program, and recording medium.

[0117] For example, the present disclosure may be implemented as a sound signal reproducing method executed by a computer, or as a program for causing a computer to execute the sound signal reproducing method. The present disclosure may also be implemented as a computer-readable non-transitory recording medium having such a program recorded thereon.

[0118] In addition, various modifications that can be conceived by those skilled in the art to the embodiments or embodiments achieved by arbitrarily combining the components and functions of the embodiments without departing from the gist of the present disclosure are also included in the present disclosure.

[0119] Industrial Applicability

[0120] The sound signal processing device according to the present invention is practical for sound processing of AR or VR devices in that it improves the sense of actual presence and positioning of objects in the sound space while suppressing the increase of processing resources beyond the required amount.

[0121] Description of symbols

[0122] 99 users

[0123] 100 Sound signal reproduction device

[0124] 101 Audio signal processing device

[0125] 102 Communication module

[0126] 103 Sensors

[0127] 104 Driver

[0128] 111 Acquisition Department

[0129] 121 Sound Data Extraction Department

[0130] 131 Metadata Extraction Department

[0131] 141 Selection Department

[0132] 151 Swing Granting Department

[0133] 161 Sound signal conversion unit

Claims

1. A sound signal processing device for converting input data into a sound signal, wherein the input data includes audio objects corresponding to one or more objects arranged in a virtual sound space, wherein: The above audio objects respectively contain: Sound data of the sound emitted by the object corresponding to the audio object; and metadata, including position information of the object corresponding to the audio object in the sound space, The above-mentioned sound signal processing device comprises: A selection unit that selects the audio object to be transformed from one or more of the audio objects; and a swing imparting unit that imparts a swing effect to the audio object selected as a transformation object by transforming the audio object, wherein the swing effect refers to the swinging of the sound emitted by the object corresponding to the transformed audio object when the sound signal is reproduced; The selection unit does not select, as a transformation target, an audio object corresponding to the object whose position in the sound space is moving, based on the temporal transition of the position information included in the metadata.

2. A sound signal processing device that converts input data into a sound signal, wherein the input data includes audio objects corresponding to one or more objects arranged in a virtual sound space, wherein: The above audio objects respectively contain: Sound data of the sound emitted by the object corresponding to the audio object; and metadata, including orientation information of the object corresponding to the audio object in the sound space, The above-mentioned sound signal processing device comprises: A selection unit that selects the audio object to be transformed from one or more of the audio objects; and a swing imparting unit that imparts a swing effect to the audio object selected as a transformation object by transforming the audio object, wherein the swing effect refers to the swinging of the sound emitted by the object corresponding to the transformed audio object when the sound signal is reproduced; The selection unit does not select, as a transformation target, an audio object corresponding to the object rotating in the sound space based on a temporal transition of the orientation information included in the metadata.

3. A sound signal processing device that converts input data into a sound signal, wherein the input data includes audio objects corresponding to one or more objects arranged in a virtual sound space, wherein: The above audio objects respectively contain: Sound data of the sound emitted by the object corresponding to the audio object; and Metadata, including the directionality information of the sound emitted by the above-mentioned object corresponding to the audio object, The above-mentioned sound signal processing device comprises: A selection unit that selects the audio object to be transformed from one or more of the audio objects; and a swing imparting unit that imparts a swing effect to the audio object selected as a transformation object by transforming the audio object, wherein the swing effect refers to the swinging of the sound emitted by the object corresponding to the transformed audio object when the sound signal is reproduced; The selection unit does not select, as a conversion target, an audio object corresponding to the object whose sound emitted by the object is non-directional, based on the directionality information included in the metadata.

4. The sound signal processing device according to any one of claims 1 to 3, wherein: The metadata also includes a biological flag indicating whether the object is a living thing. The selection unit does not select the audio object for which the biological flag indicates that the object does not belong to the biological being.

5. The sound signal processing device according to any one of claims 1 to 3, wherein: The metadata further includes a moving body flag indicating whether the object is a moving body. The selection unit does not select the audio object for which the moving object flag indicates that the object is not the moving object.

6. A sound signal processing method, executed by a computer, for converting input data into a sound signal, wherein the input data includes audio objects corresponding to one or more objects arranged in a virtual sound space, wherein: The above audio objects respectively contain: Sound data of the sound emitted by the object corresponding to the audio object; and metadata, including position information of the object corresponding to the audio object in the sound space, In the above-mentioned sound signal processing method, selecting the audio object as a transformation object from among one or more of the audio objects; By transforming the audio object selected as the transformation object, a swing effect is given to the audio object, wherein the swing effect refers to the swing of the sound emitted by the object corresponding to the transformed audio object when the sound signal is reproduced, In selecting the audio object as a transformation target, based on the temporal transition of the position information included in the metadata, an audio object corresponding to the object whose position in the sound space is moving is not selected as a transformation target.

7. A sound signal processing method, executed by a computer, for converting input data into a sound signal, wherein the input data includes audio objects corresponding to one or more objects arranged in a virtual sound space, wherein: The above audio objects respectively contain: Sound data of the sound emitted by the object corresponding to the audio object; and metadata, including orientation information of the object corresponding to the audio object in the sound space, In the above-mentioned sound signal processing method, selecting the audio object as a transformation object from among one or more of the audio objects; By transforming the audio object selected as the transformation object, a swing effect is given to the audio object, wherein the swing effect refers to the swing of the sound emitted by the object corresponding to the transformed audio object when the sound signal is reproduced, In selecting the audio object as a transformation target, based on the temporal transition of the orientation information included in the metadata, an audio object corresponding to the object rotating in the sound space is not selected as a transformation target.

8. A sound signal processing method, executed by a computer, for converting input data into a sound signal, wherein the input data includes audio objects corresponding to one or more objects arranged in a virtual sound space, wherein: The above audio objects respectively contain: Sound data of the sound emitted by the object corresponding to the audio object; and Metadata, including the directionality information of the sound emitted by the above-mentioned object corresponding to the audio object, In the above-mentioned sound signal processing method, selecting the audio object as a transformation object from among one or more of the audio objects; By transforming the audio object selected as the transformation object, a swing effect is given to the audio object, wherein the swing effect refers to the swing of the sound emitted by the object corresponding to the transformed audio object when the sound signal is reproduced, In the selection of the audio object as a target for transformation, based on the directionality information included in the metadata, an audio object corresponding to the object whose sound emitted by the object is non-directional is not selected as a target for transformation. 9 . A program for causing a computer to execute the sound signal processing method according to claim 6 .