Acoustic signal processing method and audio signal processing program

Abstract

To provide an audio signal processing method and an audio signal processing program that allow humans to experience sound that is closer to reality while reducing the processing load on a computer relating to acoustic processing in virtual reality.SOLUTION: In an audio signal processing method for outputting sound perceived by a listener when a sound source and a listener exist in a three-dimensional virtual space with a plurality of subspaces (a plurality of rooms), an identifier is assigned to each of the plurality of subspaces. A first identifier corresponding to the sound source and a second identifier corresponding to the listener are identified using the position of the sound source and the position of the listener, a first parameter corresponding to the first identifier and a second parameter corresponding to the second identifier are searched for by using the first identifier and the second identifier, and acoustic processing information is output on the basis of the first parameter and the second parameter.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention provides sound The present invention relates to an acoustic signal processing method and an acoustic signal processing program. [Background technology]

[0002] When simulating the real world in a game or virtual space, images, sounds, vibrations, etc. are perceived by the five human senses, allowing humans to experience virtual reality.

[0003] To make virtual reality more realistic, computers are often required to perform more calculations, which has led to a demand for technology that makes it easier for computers to process virtual reality.

[0004] In particular, the following techniques exist as examples of conventional techniques relating to acoustics. The positions in the virtual space of a sound source and a listener for producing acoustic expression in the virtual space are identified. A playback means is provided that plays back the sound emitted by a sound source object set as a virtual sound source in the virtual space. The position of the sound source object is then identified by searching a quadtree that contains position information of small areas included in a predetermined area in the virtual space. There is a technology that causes a program to generate sound in the virtual space by having the playback means process the sound data according to the result of the position identification (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2020-18620 Summary of the Invention [Problem to be solved by the invention]

[0006] The disclosed technology aims to reduce the processing burden on computers involved in sound processing in virtual reality while allowing humans to experience sounds that are closer to reality. [Means for solving the problem]

[0007] According to the disclosed technology, there is provided an acoustic signal processing method for outputting acoustic processing information used to obtain an acoustic signal including a sound signal perceived by a listener when a sound source and a listener exist in a virtual three-dimensional space including a plurality of subspaces and a sound emitted from the sound source propagates through the three-dimensional space and reaches the listener, the acoustic signal processing method comprising: preparing, in a memory, first relational data defining a correspondence relationship between a plurality of subspaces included in the three-dimensional space and identifiers assigned to each of the plurality of subspaces; and second relational data defining a correspondence relationship between the identifiers and signal processing parameters for obtaining the acoustic signal, the acoustic signal processing method comprising: An acoustic signal processing method is provided, comprising: receiving a position of a sound source and a position of the listener; referring to the first relationship data using the position of the sound source and the position of the listener to identify a first identifier corresponding to the sound source and a second identifier corresponding to the listener; referring to the second relationship data using the first identifier and the second identifier to search for a first parameter corresponding to the first identifier and a second parameter corresponding to the second identifier; and outputting acoustic processing information based on at least the first parameter and the second parameter to obtain the acoustic signal. [Effects of the Invention]

[0008] According to the disclosed technology, it is possible to reduce the processing load on a computer related to sound processing in virtual reality, while allowing people to experience sounds that are closer to reality. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 shows a subspace, a listener, and a sound source that exist in a virtual space. [Figure 2]FIG. 2 shows a process for outputting sound processing information used to obtain a sound signal perceived by a listener from the position of a sound source and the position of the listener. [Figure 3] FIG. 3 shows an example of dividing a three-dimensional space into three-dimensional meshes. [Figure 4] FIG. 4 shows the positional relationship between multiple subspaces and a three-dimensional mesh. [Figure 5] FIG. 5 shows a three-dimensional space filled with a number of rectangular parallelepipeds each made up of a three-dimensional mesh. [Figure 6] Figure 6 shows a rectangular parallelepiped with subspaces and overlaps. [Figure 7] FIG. 7 shows the positional relationship between the sound source, the listener, and the rectangular parallelepiped when the three-dimensional space is viewed from the Z-axis direction. [Figure 8] FIG. 8 is a process flow showing in more detail the process of identifying the first identifier and the second identifier. [Figure 9] Fig. 9A shows a process flow for selecting parameters related to sound attenuation, etc. based on identifier information, and Fig. 9B is a table defining groups of multiple layers. [Figure 10] FIG. 10 shows a detailed processing flow relating to the process of including parameters relating to sound attenuation and the like in the sound processing information. [Figure 11] FIG. 11 shows the processing flow for parameters relating to sound attenuation and the like. [Figure 12] FIG. 12 shows a processing flow for parameters relating to the generation of environmental sounds or sound reverberation effects. [Figure 13] 13A and 13B show how identifiers are extracted for each sound source and listener using multiple rectangular parallelepipeds in virtual space when a bridge is present outdoors. [Figure 14] FIG. 14 shows a process of changing the first parameter or the second parameter according to the first rule when the first identifier and the second identifier are different. [Figure 15] FIG. 15 shows another example of processing when the first identifier and the second identifier are different. [Figure 16] 16A and 16B show an example in which a partition exists in one partial space. [Figure 17] 17A and 17B show how identifiers are assigned to each rectangular parallelepiped in the virtual space near the partition. [Figure 18] Fig. 18A shows detailed processing for determining whether a first identifier and a second identifier have a predetermined relationship, and Fig. 18B shows a table used for determining whether a combination of a first identifier and a second identifier has a predetermined relationship. [Figure 19] Fig. 19A shows a table defining the identifiers and types of subspaces, and Fig. 19B shows a table defining the sound attenuation amounts and cutoff frequencies, which are examples of low-pass filter parameters, assigned to the types of spaces. [Figure 20] Fig. 20A shows a table indicating the relationship between identifier values ​​and subspace sizes, and Fig. 20B shows a table defining environmental sound definitions and names representing reverb parameters corresponding to subspace sizes. [Figure 21] FIG. 21 shows the hardware configuration of the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described with reference to the drawings. Note that the following embodiments are not mutually exclusive. Unless contradictory, part of one embodiment can be replaced with or combined with part of another embodiment. Furthermore, unless contradictory, each step of the disclosed flow can be replaced. Furthermore, each flow can be omitted or executed simultaneously. Part of the embodiment can be realized by either a program or hardware. Furthermore, part of the program can utilize the functions of an operating system. The program can be stored in a non-transitory recording medium. The following embodiments relate to the generation and processing of sound. The generation and processing of sound may be performed using a pure computer program or using hardware. Furthermore, the sound may be processed digitally or by an analog circuit.

[0011] <Embodiment 1> FIG. 1 shows a subspace, a listener Li1, and a sound source S1 that exist in a virtual space V. In FIG. 1, a bird is depicted as an example of the sound source S1.

[0012] There are multiple subspaces in virtual space V. Virtual space V has five subspaces: Room A, Room B, Room C, and Outdoor E. Each of the five subspaces is assigned the following identifier (ID): Room A ID=10 Room B ID=20 Room C ID=30 Outdoor E ID=1

[0013] A sound (for example, a bird's song) emitted from a sound source S1 present in room B passes through the wall of room B and the wall of room C to reach listener Li1. In addition to passing through walls, sound can reach listener Li1 from sound source S1 via various paths, such as passing through the wall of room B, passing through outdoor E, passing through the wall of room C, and then reaching listener Li1. In this embodiment, the sound heard by listener Li1 may include components of the sound that pass through the walls of room B and room C. In this way, by considering only the attenuation of sound passing through adjacent walls, calculations related to sound propagation can be simplified.

[0014] Furthermore, the calculation of sound attenuation can be further simplified by using the amount of sound attenuation at the walls of room B or room C, whichever has the greater attenuation effect, as the amount of sound attenuation. Furthermore, even if there is another room (not shown) between room B and room C, for example, the calculation of sound attenuation can be further simplified by omitting the attenuation of sound passing through the other room and its walls and selecting only the amount of sound attenuation at the walls of room B where sound source S1 is located and the amount of sound attenuation at the walls of room C where listener Li1 is located, as described above. The amount of sound attenuation may be determined by adding together the amount of sound attenuation at the walls of room B and room C where listener Li1 is located.

[0015] Because the human sense of sound is less sensitive than other senses such as vision, even if acoustic signals such as the amount of sound attenuation are calculated using the above method, it is possible to obtain with high accuracy the sensation that the sound gives to listener Li1 when it reaches listener Li1. In this way, it is possible to avoid performing complex calculations through detailed simulations of sound propagation.

[0016] For example, if the attenuation of the wall of room B is -3 dB and the attenuation of the wall of room C is -6 dB, then the -6 dB should be adopted since the sound attenuation in room C is greater. By adopting this adopted -6 dB as the attenuation of the sound transmitted from sound source S1 to listener Li1, the volume of the sound perceived by listener Li1 can be obtained. In addition to attenuating the volume, the effects on sound propagation (filtering, etc.) may be processed in the same way as the above-mentioned approach regarding attenuation of the volume.

[0017] For example, when sound passes through a wall, the higher the frequency, the greater the attenuation. Therefore, it is desirable to apply a low-pass filter to the sound. For this low-pass filter processing, parameters can be adopted that, for example, increase the attenuation of high frequencies as the wall becomes thicker.

[0018] For example, if the cutoff frequency of the low-pass filter corresponding to the wall of room B is 1200 Hz and the cutoff frequency of the low-pass filter corresponding to the wall of room C is 800 Hz, the wall of room C will pass sounds that are more limited to the low frequency range than the wall of room B and will be less likely to pass sounds in the high frequency range, so the processing parameters of the low-pass filter of room C can be applied to the sound. The above is one aspect of an embodiment related to parameter processing related to sound attenuation, etc. Parameters related to sound attenuation, etc. (i.e., sound attenuation parameters and low-pass filter processing parameters, etc.) are an example of a first type of parameter.

[0019] On the other hand, parameters relating to sound reverberation and the like can be processed in the following manner. The effect of sound reverberation tends to be strongly influenced by the reverberation characteristics of the room that the listener Li1 is in. Therefore, for sound effects such as reverb or echo, it is desirable to employ parameters related to the room of the listener Li1.

[0020] Furthermore, with regard to environmental sounds, the environmental sounds in the room of the listener Li1 tend to have a stronger influence on the listener Li1 than the environmental sounds in other rooms. Therefore, it is desirable to adopt parameters relating to the environmental sounds in the room where the listener Li1 is present. Examples of environmental sounds include the sound of cars passing by on a nearby road if outdoors E, and the sound of wind blowing through a window if in room B. These environmental sounds may be generated directly by adding the reverberation effect of the room in advance, or the environmental sounds may be added with a reverberation effect (e.g., echo or reverb). In addition, the reverberation effect may be added to the sound that reaches the listener Li1 from the sound source S1 after being subjected to attenuation processing and low-pass filter processing. In outdoor E, for example, a delay effect simulating the effect of an "echo" may be added.

[0021] To determine in which subspace of the virtual space the sound source S1 and the listener Li1 exist, it is sufficient to determine in which subspace the three-dimensional coordinates of the sound source S1 and the listener Li1 (or the ears of the listener Li1) in the virtual space exist. For this determination, if a three-dimensional model in which a solid object such as a room is configured with polygons is used, it is possible to determine whether the sound source S1 and the listener Li1 exist inside or outside the polygons that configure the boundary of the subspace based on polygon information. Then, an identifier is assigned to each subspace of the three-dimensional model. Depending on the model that defines the three-dimensional space, it is possible to identify the subspace in which the sound source S1 and the listener Li1 exist and the identifier assigned to the subspace. The three-dimensional model in which identifiers are assigned to each of the subspaces is an example of first relational data. Furthermore, a table that associates various parameters of acoustic processing in each room and outdoors with identifiers (IDs) assigned to each room and outdoors can be stored in memory. If the identifiers of the subspaces in which the sound source S1 and the listener Li1 exist can be specified, the acoustic processing parameters can be specified by searching this table using the identifiers. This table is an example of second relational data.

[0022] For example, if the identifier of the subspace is set in advance to increase in response to an increase in the effect of a parameter related to sound attenuation, etc., the identifier to be used to determine the parameter related to sound attenuation, etc. to be adopted can be determined by determining the size of the identifier. The same applies to low-pass filter parameters. In this way, by determining the size of the identifier of the subspace in which the sound source S1 and the listener Li1 exist, it is possible to select and determine the acoustic processing parameters to be adopted (for example, sound attenuation and low-pass filter).

[0023] Note that if the sound source S1 and the listener Li1 exist in the same subspace, sound attenuation processing may not be performed. Also, low-pass filter processing may not be added. Alternatively, the attenuation parameter value may be changed according to the distance between the sound source S1 and the listener Li1, so that the amount of attenuation increases as the distance increases.

[0024] Although one identifier is sufficient for outdoor E, multiple identifiers may be provided. A detailed description will be given later, but one example will be briefly described here. For example, assume that an identifier 7 is assigned to the space under a bridge in outdoor E, and an identifier 1 is assigned to the other space in outdoor E. Suppose that listener Li1 is located under the bridge, i.e., in the area identified by identifier 7, and sound source S1 is located in the area identified by identifier 1. If listener Li1 and sound source S1 are located at approximately the same height (i.e., both listener Li1 and sound source S1 are located under the bridge), the sound reaching listener Li1 from sound source S1 is not blocked by the bridge, and sound attenuation may not be performed. If listener Li1 and sound source S1 are located at different heights such that a bridge exists between them, parameter selection may be changed so that sound attenuation is performed. This parameter change may be performed by storing a table in memory containing multiple parameters related to sound attenuation and other factors that are predetermined for each case. This process is similar to that for low-pass filter parameters. The above bridge embodiment can be applied not only outdoors but also indoors as well. The bridge example above can also be applied to cases where a partition is present. For example, this can be achieved by applying multiple identifiers to the same subspace. For a bridge, the acoustic parameters can be changed based on the height relationship (layer relationship) between the sound source S1 and the listener Li1. In the case of a partition, the acoustic parameters can be changed based on whether the relationship between the identifier corresponding to the sound source S1 and the identifier corresponding to the listener Li1 is a predetermined relationship. These details will be described later.

[0025] It should be noted that sounds uttered by the listener Li1 himself (for example, words uttered by the listener Li1) may be processed using parameters relating to reverberation and the like associated with the subspace in which the listener Li1 exists.

[0026] 2 is a flowchart for outputting acoustic processing information used to obtain an acoustic signal perceived by the listener Li1 from the position of the sound source S1 and the position of the listener Li1. The acoustic processing information is used as control information for a DSP (Digital Signal Processor) or the like to obtain a sound signal that can be perceived by humans.

[0027] Each step in FIG. 2 will be described below. It is assumed that a first table defining the correspondence between a plurality of subspaces included in a three-dimensional space and the identifiers assigned to each of the plurality of subspaces is stored in advance in memory. Similarly, it is assumed that a second table defining the correspondence between the identifiers and the signal processing parameters used to generate an acoustic signal is also stored in advance in memory. The first table and the second table are examples of first relational data and second relational data, respectively. It goes without saying that the first table and the second table may be combined and exist in memory as a single table. Alternatively, the information in these tables does not necessarily need to be in the form of a table in memory.

[0028] Each step in FIG. 2 is explained below. [Step S110] The position of the sound source S1 and the position of the listener Li1 in the virtual space are received. The three-dimensional positions may be expressed in three-dimensional coordinates using mutually orthogonal X-, Y-, and Z-axes.

[0029] [Step S112] The first table is searched to identify the first identifier and the second identifier corresponding to the position of the sound source S1 and the position of the listener Li1, respectively.

[0030] [Step S116] The first parameter and the second parameter are obtained by searching the second table using the identifiers corresponding to the position of the sound source S1 and the position of the listener Li1.

[0031] [Step S118] Appropriate acoustic processing information is output based on at least the first parameter and the second parameter. Then, for example, the output acoustic processing information is input to a DSP (Digital Signal Processor) or the like to obtain a sound signal that can be perceived by humans.

[0032] The outline of how the first parameter and the second parameter are used to output the acoustic processing information will be explained below, and the details will be described later.

[0033] For example, if the first identifier and the second identifier are the same, it is determined that there is no obstruction. That is, the sound source S1 and the listener Li1 are in the same subspace. In this case, the volume from the sound source may not be attenuated or low-pass filter processing may not be performed. That is, since the first identifier and the second identifier are the same, the first parameter and the second parameter are also equal, and this parameter may be used. Alternatively, the volume may not be attenuated or low-pass filter processing may not be performed regardless of the parameter value. Alternatively, parameters (not shown) related to sound attenuation and the like when the first identifier and the second identifier are the same may be stored in memory. Note that, for parameters related to environmental sound or sound reverberation, it is desirable to apply the first parameter (or a second parameter having the same value) when the first identifier and the second identifier are the same.

[0034] On the other hand, for example, if the first identifier and the second identifier are different, the sound source S1 and the listener Li1 will in principle be in different subspaces. In this case, either the first parameter or the second parameter may be used. Note that when the first identifier and the second identifier are different, the process of changing parameters related to sound attenuation and the like in specific cases has already been outlined using the examples of the bridge and the partition, and will be further explained in the embodiments described below. <Embodiment 2>

[0035] In the first embodiment, in order to identify the space in which the sound source S1 and the listener Li1 exist, the positional relationship between the sound source S1 and the listener Li1 and the walls separating the subspaces is determined in accordance with the three-dimensional model that configures the three-dimensional space in the calculation of the subspace. In the first embodiment, it was necessary to calculate this positional relationship (i.e., to determine whether the sound source S1 and the listener Li1 exist inside or outside the subspace). Below, a second embodiment that simplifies this calculation will be described.

[0036] FIG. 3 shows an example of dividing a three-dimensional space into a three-dimensional mesh. The three-dimensional mesh is created using planes that are perpendicular to the X-axis, Y-axis, and Z-axis and spaced apart at a predetermined interval. The virtual space is divided into small spaces by a plurality of rectangular parallelepipeds 300 separated by this three-dimensional mesh. The rectangular parallelepipeds 300 may also be cubes. For convenience, the following description will be given assuming that the XY plane is the horizontal plane in three-dimensional space and the Z axis direction is an axis perpendicular to the horizontal plane.

[0037] Figure 4 shows the positional relationship between multiple subspaces and a three-dimensional mesh. The relationship between Room A, Room B, Room C, and Outdoor E shown in Figure 1 and the three-dimensional mesh is shown.

[0038] FIG. 5 shows a three-dimensional space filled with a number of rectangular parallelepipeds each made up of a three-dimensional mesh. For convenience, a plate-like space consisting of multiple rectangular parallelepipeds arranged parallel to a horizontal plane is sometimes called a layer. Figure 5 shows Layer 1 and Layer 2 as examples. It is desirable that each layer be assigned a number in order. The position of each rectangular parallelepiped on the layer plane can be identified by this number. Then, by assigning a number to each layer, the position of the layer in the Z-axis direction can be identified. The position of the rectangular parallelepiped on the layer plane and the layer number can be used to identify the position of the rectangular parallelepiped in three-dimensional space. Note that the method for identifying the position of the rectangular parallelepiped in three-dimensional space is not limited to this method.

[0039] Fig. 6 shows a rectangular parallelepiped that overlaps with the subspace. For clarity, the rectangular parallelepiped corresponding to outdoor E is omitted from Fig. 6. The same identifier as the identifier corresponding to each subspace is assigned to the corresponding rectangular parallelepiped. Note that, for example, as shown in rectangular parallelepiped 600, there are cases where rectangular parallelepiped 600 and room C do not share a space with each other at all. In such cases, for example, each rectangular parallelepiped 600 may be associated with the subspace with which it shares the most space. Then, the identifier of the associated subspace may be assigned to rectangular parallelepiped 600.

[0040] By doing this, it is possible to search for rectangular parallelepipeds that exist at the positions of the sound source and the listener, and extract the identifiers assigned to the searched rectangular parallelepipeds, thereby making it easier to extract the identifiers corresponding to each of the sound source and the listener.

[0041] 7 shows the positional relationship between a sound source S2, a listener Li2, and a rectangular parallelepiped in a certain layer when three-dimensional space is viewed from the Z-axis direction. The identifiers assigned to the rectangular parallelepipeds located at the positions of the sound source S2 and the listener Li2 can be extracted by referring to a first table showing the relationship between the rectangular parallelepipeds and the identifiers. The first table can be stored in memory in advance.

[0042] By doing the above, it is possible to search for the rectangular parallelepipeds that exist at the respective positions of the sound source S2 and the listener Li2, and then refer to the first table to check the correspondence between the rectangular parallelepipeds and the identifiers, thereby more easily obtaining the identifiers associated with each of the sound source S2 and the listener Li2.

[0043] Then, by referring to the second table that defines the relationship between the identifiers and the acoustic parameters from the identifiers associated with the sound source S2 and the listener Li2, various acoustic parameters associated with the sound source S2 and the listener Li2 can be easily acquired. Based on the acquired acoustic parameters, acoustic processing information for generating an acoustic signal perceived by the listener can be easily output.

[0044] 8 is a process flow showing in more detail step S112 of identifying the first identifier and the second identifier. Each process step will be described below. It is assumed that a first table in which an identifier is associated with each of a plurality of rectangular parallelepipeds is stored in advance in memory. The first table is an example of first relational data. [Step S210] Based on the position of the sound source S2 and the position of the listener Li2, a first rectangular parallelepiped and a second rectangular parallelepiped corresponding to the respective positions are searched for.

[0045] [Step S212] Using the found first and second rectangular parallelepipeds, the first table is referenced to extract a first identifier associated with the first rectangular parallelepiped and a second identifier associated with the second rectangular parallelepiped. The first identifier is an identifier of a subspace in which the sound source S2 exists, and the second identifier is an identifier of a subspace in which the listener Li2 exists. In this way, it is possible to more easily identify the identifiers corresponding to the sound source S2 and the listener Li2.

[0046] Fig. 9A shows a process flow for selecting parameters related to sound attenuation, etc. based on identifier information, and Fig. 9B is a table defining groups of multiple layers. The processing flow of FIG. 9A will be described below. [Step S310] If the first identifier and the second identifier are different, the first parameter or the second parameter is changed according to the first rule. An example of this case is when an obstruction such as a bridge or a partition is present. The outline of this case has already been described, but below we will explain an example of the case when a bridge is present.

[0047] When there is an obstacle separating the sound source from the listener, such as a bridge, and the sound source is separated from the listener by the bridge, the sound from the sound source is attenuated by the bridge before reaching the listener. However, when both the sound source and the listener are below the bridge, there is no sound obstruction by the bridge, so sound attenuation is negligible or eliminated. The same is true when both the sound source and the listener are above the bridge. As shown in Fig. 13B, consider a case where there is an obstacle, such as a bridge, that attenuates sound propagation in the vertical direction. In this case, it is desirable to be able to select parameters related to sound attenuation that correspond to the identifier assigned to the subspace, depending on whether the sound source and listener are both in the same layer or the same layer group. When multiple layers exist under a bridge, such as when layers L0 to L3 exist under the bridge in Figure 13B, layers L0 to L3 can be defined as forming a layer group. FIG. 9B is a table that defines layer groups assuming the above example. Fig. 19A is a table that defines the identifiers and types of subspaces, and Fig. 19B is a table that defines the amount of sound attenuation assigned to each space type and the cutoff frequency, which is an example of a low-pass filter parameter. Fig. 9B is a table showing that layers L0 to L3 form layer group 1, and layers 11 and 12 form layer group 2. By referring to Fig. 9B, multiple layers can be treated as a group, and if a sound source and a listener both exist in layers of the same group, the parameters corresponding to the same layer group from the tables of Figs. 19A and 19B can be adopted. 9B, it is possible to eliminate the need to define layer groups. In this case, if the sound source and the listener are on the same layer, the parameters of the same layer group may be adopted. Alternatively, the parameters of the same layer group in FIG. 19B may be adopted by limiting it to the layer group defined in FIG. 9B. In this case, effects such as sound attenuation may be provided according to the distance between the sound source and the listener. For example, when FIGS. 9A and 9B and FIGS. 19A and 19B are applied to FIG. 13B, the following results are obtained. Referring to Fig. 9B, sound source S41 and listener Li4 in Fig. 13B exist in different layer groups. Identifier 1 is assigned to sound source S41, and identifier 7 is assigned to listener Li4. If, for example, parameters with a larger identifier are adopted as described below, in Figs. 19A and 19B, the parameters of the different layer group for listener Li4 with identifier 7 are a volume attenuation parameter of -6 dB and a low-pass filter cutoff frequency parameter of 16,000 Hz. Therefore, these parameters should be applied to the sound from sound source S41. Referring to Fig. 9B, sound source S42 and listener Li4 in Fig. 13B exist in the same layer group. Sound source S42 is assigned identifier 1, and listener Li4 is assigned identifier 7. If, for example, parameters with the larger identifier are adopted, then in Figs. 19A and 19B, the parameters of listener Li4 in the same layer group with identifier 7 are a volume attenuation parameter of 0 dB and a low-pass filter cutoff frequency parameter of "none" (i.e., the same as not applying a low-pass filter). Therefore, it is sufficient to have no sound attenuation (0 dB) and no low-pass filter applied to the sound from sound source S42. Details of FIGS. 13A and 13B will be described later.

[0048] When an obstruction such as a bridge is present, rather than determining whether or not an obstruction is present by applying the above-mentioned processing to the sound from the sound source, it is possible to apply processing according to the propagation of the sound based on the presence or absence of an obstruction by changing the parameters applied depending on the relative positions of the sound source and the listener. By referring to Figures 19A and 19B, including when the sound source and listener are located outdoors, in a room, in a castle, a cave, etc., it is possible to change the volume attenuation parameters and low-pass filter parameters depending on whether the sound source and listener are in the same layer group or different layer groups. Note that while Figures 19A and 19B show an example in which the same volume attenuation parameters and low-pass filter parameters are assigned to multiple identifiers, it goes without saying that volume attenuation parameters and low-pass filter parameters may also be assigned to each identifier.

[0049] As already mentioned, for sound reflections and environmental sounds in a subspace, the acoustic parameters in the subspace where the listener is present can be used, and the influence of obstructions does not need to be taken into consideration.

[0050] FIG. 10 shows a detailed processing flow relating to the process of including parameters relating to sound attenuation and the like in the sound processing information. [Step S410] It is checked whether the first identifier and the second identifier are different. If the first identifier and the second identifier are different (Yes), it is determined that the sound source and the listener exist in different subspaces. If the first identifier and the second identifier are the same (No), it is determined that the sound source and the listener exist in the same subspace. If this check is Yes, the process proceeds to step S412. If this check is No, the process proceeds to step S414.

[0051] [Step S412] The first type of parameters assigned to one of the first and second identifiers selected according to the second rule are included in the acoustic processing information. Here, for example, parameters related to sound attenuation and low-pass filter parameters are examples of the first type of parameters. Then, the process returns. The second rule will be explained later.

[0052] [Step S414] The acoustic processing information includes the first type of parameters when the sound source and the listener are in the same subspace. In this case, since the sound source and the listener are in the same subspace, sound attenuation and low-pass filter processing may not be performed. Alternatively, the distance between the sound source and the listener may be detected, and sound attenuation and low-pass filter processing may be changed depending on the distance. Then, the process returns. The handling of parameters relating to environmental sounds and sound reverberation will be explained with reference to FIG.

[0053] The second rule will be explained below with reference to FIG. FIG. 11 shows a processing flow to which the second rule is applied when the parameters are, for example, sound attenuation and low-pass filter processing. [Step S510] Parameters corresponding to identifiers having parameters with greater sound shielding effects are included in the acoustic processing information. As already mentioned, if the acoustic processing is sound attenuation or low-pass filtering, it is desirable to adopt the parameter associated with the identifier with the parameter that has the greatest amount of attenuation and apply that parameter to the sound. Adopting the parameter associated with the identifier with the parameter that has the greatest amount of attenuation is an example of the second rule. By adopting parameters associated with an identifier having parameters with a larger amount of attenuation, the amount of calculation of the amount of sound attenuation can be reduced. If a parameter with a larger attenuation amount is associated with a larger identifier, a rule may be used in which the parameter related to sound attenuation corresponding to the larger identifier between the first identifier and the second identifier is adopted. This rule is an example of the second rule. Note that parameters relating to sound attenuation and the like associated with the first and second identifiers may be added (or the effects of the parameters may be superimposed) and included in the sound processing information.

[0054] FIG. 12 shows a processing flow for outputting sound processing information when the sound processing is processing related to the generation of environmental sounds or the reverberation effect of sounds. As already mentioned, when the acoustic processing is processing related to the generation of environmental sounds or the reverberation effect of sounds, it is desirable to generate the environmental sounds or add the reverberation effect by including in the acoustic processing information parameters related to an identifier corresponding to the subspace in which the listener is located. It is also desirable to apply parameters related to the reverberation effect associated with an identifier corresponding to the subspace in which the listener is located to the environmental sounds, sounds from the sound source that have been attenuated, and sounds emitted by the listener himself. It should be noted that parameters relating to the reverberation effect of sound associated with the subspace in which the sound source exists (i.e., the identifier corresponding to the sound source) may be applied to the sound from the sound source.

[0055] 13A and 13B show how identifiers are extracted for each sound source and listener using multiple rectangular parallelepipeds in virtual space when a bridge is present outdoors.

[0056] 13A shows a bridge 1720 outdoors. The bridge 1720 is built on a riverbank 1750. A listener Li4 is present in a space 1730 directly below the bridge. River water 1740 exists below the bridge. A sound source S41 exists directly above the bridge 1720. A sound source S42 exists at a lower position in the Z-axis direction than the bridge 1720, but is not present directly below the bridge 1720.

[0057] Figure 13B shows an E-E cross section of Figure 13A. Figure 13B shows identifiers assigned to multiple rectangular parallelepipeds filling the three-dimensional space. The rectangular parallelepiped under the bridge 1720 is assigned identifier 7. The rectangular parallelepipeds in the outdoor space other than those directly under the bridge 1720 are assigned identifier 1. The identifier of the subspace in which listener Li4 exists is 7, and the identifiers of the subspaces in which sound source S41 and sound source S42 exist are both 1. Note that listener Li4 or sound source S42, etc., spans multiple rectangular parallelepipeds. When the identifiers assigned to the multiple rectangular parallelepipeds are different, for example, the identifier assigned to the rectangular parallelepiped that overlaps largely with listener Li4 or sound source S42, etc., may be associated with listener Li4 or sound source S42. In this case, the first rule for changing the first parameter or the second parameter is as follows:

[0058] (First rule: Example of a bridge) Even if the combination of the identifiers of listener Li4 and sound source S42 is 1 and 7, if the layers to which listener Li4 and sound source S42 belong are both layers L0 to L3, they are determined to belong to the same layer group. In this case, listener Li4 and sound source S42, which belong to the same layer group, are both located in approximately horizontal positions, and there is no bridge between them that blocks the propagation of sound.

[0059] Furthermore, even if the combination of the identifiers of listener Li4 and sound source S41 is 1 and 7, if the layer to which one of listener Li4 and sound source S41 belongs is one of layers L0 to L3 and the layer to which the other belongs is other than layers L0 to L3, then they are determined to belong to different layer groups.

[0060] By applying the first rule described above, even in a space where a bridge or the like is present outdoors 1710 (or inside a room), the sound attenuation or low-pass filter parameters can be appropriately changed by taking into account the presence of the bridge and referring to the tables shown in Figures 9B and 19B. As already mentioned, in both cases, the acoustic processing related to the environmental sound and the sound reflection can be performed by applying the parameters related to the identifier 7 associated with the listener Li4. The environmental sound can be the sound of a flowing river, etc. The parameter related to the sound reflection can be a parameter that simulates the sound reflection under a bridge. <Embodiment 3>

[0061] 14 shows a process for changing the first parameter or the second parameter in accordance with the first rule when the first identifier and the second identifier are different. Each flow in FIG. 14 will be described below.

[0062] [Step S710] It is determined whether the sound source and listener are in the same layer group. If the determination is Yes, the process proceeds to step S712. If the determination is No, the process proceeds to step S714.

[0063] [Step S712] If the first identifier and the second identifier are different, and for parameters related to sound attenuation, etc., if the sound source and listener are in the same layer group, corresponding first and second parameters are searched for. This process makes it possible to use the corresponding first and second parameters when the sound source and the listener are in the same layer group. [Step S714] If the first identifier and the second identifier are different, and the sound source and the listener are in different layer groups, the corresponding first and second parameters are searched for, with respect to parameters related to sound attenuation, etc. This process allows the first parameter and the second parameter to be changed depending on whether the sound source and the listener are in different layer groups. The process in FIG. 14 is an example of the first rule.

[0064] <Embodiment 4> (First rule: Example of a partition) 15 shows another example of processing for changing the first parameter or the second parameter according to the first rule when the first identifier and the second identifier are different. This processing flow is for the case where a partition is present, as shown in FIGS. 16A, 16B, 17A, 17B, 18A, and 18B.

[0065] [Step S812] It is determined whether the first identifier and the second identifier have a predetermined relationship. If this check is affirmative (Yes), the process proceeds to step S814. If this check is negative (No), the process returns.

[0066] [Step S814] With regard to parameters relating to sound attenuation and the like, if the first identifier and the second identifier match, the parameters are determined as the first parameter and the second parameter. In this case, the first parameter and the second parameter are not obtained from the table shown in Fig. 19B, but the parameters when the first identifier and the second identifier match (i.e., when the sound source and the listener exist in the same subspace) are used as the first parameter and the second parameter. By doing so, even if the first identifier and the second identifier are different, the sound source and the listener belong to the same subspace, and processing can be performed in the same way as when there is no obstruction between them. The process in FIG. 15 is an example in which the first rule is applied to a partition. A specific example of the processing in FIG. 15 will be described below using a partition as an example.

[0067] 16A and 16B show an example in which a partition 180 exists in one of the partial spaces. 16A, a listener Li5 is located in front of a partition 180. A sound source S51 is located on the opposite side of the partition 180. A sound source S52 is located to the side of the partition 180.

[0068] Fig. 16B is an XY plane view that crosses the listener Li5 in the virtual space of Fig. 16A. The mesh shown in Fig. 16B is a three-dimensional mesh in which the boundaries formed by each rectangular parallelepiped are indicated by dashed lines when viewed from the Z-axis direction.

[0069] As shown in FIG. 16B, a rectangular parallelepiped with identifier 5 exists on the listener Li5 side of the partition 180. Furthermore, a rectangular parallelepiped with identifier 6 exists on the sound source S51 side of the partition 180. The surrounding rectangular parallelepipeds other than these rectangular parallelepipeds are assigned identifier 1. In this case, identifiers 1, 5, and 6 are assigned to the rectangular parallelepipeds that share the outdoor space. A shield (partition 180) exists between the space where the rectangular parallelepiped with identifier 5 exists and the space where the rectangular parallelepiped with identifier 6 exists. There are no obstructions between the space where the rectangular parallelepiped with identifier 5 exists and the space where the rectangular parallelepiped with identifier 1 exists, and between the space where the rectangular parallelepiped with identifier 6 exists and the space where the rectangular parallelepiped with identifier 1 exists. (Note that, among the rectangular parallelepipeds with identifier 1, there are also combinations of rectangular parallelepipeds with obstructions (partitions 180) present, but because the partitions 180 are far apart between these rectangular parallelepipeds, taking into account sound deflection, it is possible to assume that no obstructions exist, thereby enabling a more realistic simulation of sound propagation.

[0070] 17A and 17B show how identifiers are assigned to each rectangular parallelepiped in the virtual space near a partition 180. As shown in FIG. 17A, a subspace 1902 exists on one side near a partition 1900, and a subspace 1904 exists on the other side near the partition 1900.

[0071] 17B, ​​an identifier 5 is assigned to each of a set of rectangular parallelepipeds 1952 that overlap with partial spaces 1902 located on both sides of partition 1900. An identifier 6 is assigned to each of a set of rectangular parallelepipeds 1954 that overlap with partial spaces 1904. By doing so, the identifier 1, the identifier 5 or the identifier 6 can be associated with the sound source S51 or the sound source S52 and the listener Li5.

[0072] Returning to Fig. 16B, when the position of the listener Li5 is associated with identifier 5 and the sound source S51 is associated with identifier 6, processing may be performed assuming that a partition 180 (that is, an obstruction) is present.

[0073] On the other hand, when the identifier 5 is associated with the position of the listener Li5 and the identifier 1 is associated with the sound source S52, processing may be performed in the same manner as if the partition 180 (that is, the obstruction) were not present.

[0074] Alternatively, if identifier 1 is associated with the position of listener Li5 and identifier 1 is also associated with sound source S52 (not shown), processing may be performed in the same manner as if partition 180 (i.e., an obstruction) were not present.

[0075] By doing so, even when sound source S51 or sound source S52 and listener Li5 exist in the same subspace, it is possible to perform acoustic processing that takes into account the presence of objects that attenuate sound propagation, such as partitions.

[0076] Figure 18A shows detailed processing for determining whether a first identifier and a second identifier have a predetermined relationship (step S710 in Figure 15). Figure 18B shows a table used for determining whether a combination of a first identifier and a second identifier has a predetermined relationship. Each step of the processing flow in Figure 18A will be described below.

[0077] [Step S912] It is determined whether the combination of the first identifier and the second identifier exists in a predetermined table. If the determination result is positive, for example, as shown in table 1800 in Figure 18B, the combination of identifiers is 1 and 5 or 1 and 6 (Yes), the process proceeds to step S914. If the determination result is negative, for example, the combination of identifiers is 5 and 6 (No), the process returns. [Step S914] It is determined that the first identifier and the second identifier have a predetermined relationship. According to the processing flow of Figures 15 and 18A, if the combination of identifiers is 1 and 5, or 1 and 6, it is determined that the first identifier and the second identifier have a predetermined relationship, and in step S814 of Figure 15, the parameters when the first identifier and the second identifier match are set as the first parameter and the second parameter. By the above process, if sound source S51 is associated with identifier 6 and listener Li5 is associated with identifier 5, parameters related to acoustic attenuation and the like associated with identifier 6, which has the larger identifier value, are searched for in tables similar to those in Figures 19A and 19B. In this case, it is not necessary to determine whether the layer groups are different. As a variant, if there is a space with a structure that combines a partition and a bridge, parameters related to sound attenuation, etc. may be searched for taking into account the differences between layer groups (not shown). The process flow in FIG. 18A using table 1800 in FIG. 18B is an example of the first rule. In this way, even when a partition or the like is present in the partial space, processing related to sound propagation can be performed appropriately. If the identifier 1 and the identifier 2 corresponding to the sound source and the listener, respectively, are both "1", the result in step S410 in FIG. 10 is No, and the process in step S414 is performed.

[0078] <Embodiment 5> 19A shows a table 1900 that defines the identifiers of subspaces and the types of subspaces. A plurality of identifiers may be assigned to each of outdoors, rooms, castles, caves, and the like.

[0079] FIG. 19B shows a table 1901 that defines sound attenuation parameters and cutoff frequencies that are examples of low-pass filter (LPF) parameters assigned to the types of spaces.

[0080] Table 1901 shows two parameters: one for the same layer group and one for different layer groups. By preparing a table in which the sound attenuation and low-pass filter parameters differ depending on whether the sound source and listener are in different layer groups, it is possible to process sound propagation when there is a bridge or similar object, as mentioned above. It is desirable to store a table (for example, FIG. 9B) that defines layer groups in advance in memory.

[0081] 20A shows a table 2000 showing the relationship between the identifier value and the size of the subspace. The size of the subspace may be expressed as the length of one side of a rectangular parallelepiped having the same volume as the subspace. Alternatively, the size of the subspace may be expressed, for example, by the volume of the subspace. FIG. 20B shows a table 2001 in which definitions of environmental sounds corresponding to the sizes of subspaces and names representing reverb parameters are defined.

[0082] The environmental sounds shown in Table 2001 are merely examples. Reverb is an example of an effect parameter that indicates the reverberation of sound. The reverberation of sound is not limited to reverb, and may be echo, delay, or the like.

[0083] It goes without saying that the sound effects are not limited to the examples given above, and various sound effects such as a high-pass filter, a phase shifter, distortion, and chorus can be used. Table 1900, table 1901, table 2000, and table 2001 are examples of the second relational data.

[0084] Furthermore, in the above embodiment, sound is used as an example, but vibration may also be used.

[0085] 21 shows the hardware configuration of the embodiment. The hardware configuration includes a CPU 1001, a ROM 1002 in which the program and data of the embodiment can be stored, a RAM 1003, a network interface 1005, an input interface 1006, a display interface 1007, and an external memory interface 1008. These pieces of hardware are connected to each other by a bus 1004.

[0086] The network interface 1005 is connected to a network 1015. The network 1015 may be a wired LAN, a wireless LAN, the Internet, a telephone network, or the like. The input interface 1006 is connected to an input unit 1016. The display interface 1007 is connected to a display unit 1017. The external memory interface 1008 is connected to a storage medium 1018. The storage medium 1018 may be a RAM, a ROM, a CD-ROM, a DVD-ROM, a hard disk, a memory card, a USB memory, or the like. The program and method for realizing the above-described embodiment can be executed by a computer having the hardware configuration shown in FIG. It should be noted that there may be a plurality of listeners and a plurality of sound sources. Furthermore, the first relationship data may be changed when the partial spaces are connected, for example, when a door separating adjacent partial spaces is opened. Alternatively, first relationship data may be prepared separately assuming a case where a door is opened, and when the door is opened, different first relationship data may be appropriately switched and used by changing a pointer indicating the location of the first relationship data in memory. The above embodiments are illustrative of the invention defined in the claims and should not be construed as limiting. [Explanation of symbols]

[0087] 1004 Bus 1005 Network Interface 1006 Input Interface 1007 Display Interface 1008 External Memory Interface 1015 Network 1016 Input section 1017 Display section

Claims

1. 1. An acoustic signal processing method for outputting acoustic processing information used to obtain an acoustic signal including a sound signal perceived by a listener when a sound source and a listener exist in a virtual three-dimensional space including a plurality of subspaces and a sound emitted from the sound source propagates through the three-dimensional space and reaches the listener, the method comprising: a first relational data defining a correspondence relationship between a plurality of subspaces included in the three-dimensional space and an identifier assigned to each of the plurality of subspaces, and a second relational data defining a correspondence relationship between the identifier and a parameter of a signal processing for obtaining the acoustic signal are prepared in a memory; The acoustic signal processing method includes: receiving a position of the sound source and a position of the listener; referring to the first relationship data using the position of the sound source and the position of the listener to identify a first identifier corresponding to the sound source and a second identifier corresponding to the listener; referring to the second relationship data using the first identifier and the second identifier to search for a first parameter corresponding to the first identifier and a second parameter corresponding to the second identifier; outputting acoustic processing information based on at least the first parameter and the second parameter so as to obtain the acoustic signal; An acoustic signal processing method comprising:

2. the three-dimensional space is divided into a plurality of rectangular parallelepipeds having vertices at four mutually adjacent intersections of a three-dimensional grid formed of a plurality of grid lines parallel to each of X, Y and Z axes which are orthogonal coordinate axes placed in the three-dimensional space, an identifier assigned to the partial space is associated with each of a set of the rectangular parallelepipeds which overlaps with the partial space, and the first relationship data is constituted by defining a correspondence relationship between each of the plurality of rectangular parallelepipeds and an identifier; The identifying step includes: Searching for a first rectangular parallelepiped located at the position of the sound source and a second rectangular parallelepiped located at the position of the listener; extracting the first identifier associated with the first rectangular parallelepiped and the second identifier associated with the second rectangular parallelepiped by referring to the first relationship data; The method of claim 1 , further comprising:

3. The outputting step includes: if the first identifier and the second identifier are different, modifying the first parameter or the second parameter according to a first rule; Including, 3. The method for processing an acoustic signal according to claim 1.

4. 4. The acoustic signal processing method according to claim 1, wherein, when the first identifier and the second identifier are different, the outputting includes, in the acoustic processing information, a first type of parameter assigned to one identifier selected from the first identifier and the second identifier according to a second rule.

5. The acoustic signal processing method according to claim 4 , wherein the second rule is a rule for selecting an identifier having a parameter that has a greater effect of shielding the acoustic signal.

6. The acoustic signal processing method according to claim 1 , wherein the outputting step includes including, among the parameters assigned to the second identifier, a parameter related to environmental sound or sound reverberation in the acoustic processing information.

7. The acoustic signal processing method according to claim 3 , wherein the first rule is a rule for determining whether or not the positions of the sound source and the listener are in a predetermined positional relationship.

8. The acoustic signal processing method according to claim 3 , wherein the first rule is a rule for determining whether or not the first identifier and the second identifier have a predetermined relationship.

9. 9. An audio signal processing program for causing a computer to execute the audio signal processing method according to claim 1.

10. A storage medium storing the acoustic signal processing program according to claim 9.

Images