Techniques for adding distance dependent reverberation to audio signals of virtual sound sources

Abstract

Various embodiments disclose a computer-implemented method for generating a perceived location of a sound source in an acoustic environment. The method comprises: determining a current position of a first virtual audio source relative to a listening area of an acoustic environment; determining a wet signal distribution based on the current location of the first virtual audio source; generating a first virtual audio signal for a first physical sound source included in the acoustic environment, wherein the first virtual audio signal is associated with the first virtual audio source and is distributed based on the wet signal; and transmitting the first virtual audio signal to the first physical sound source to be output by the first physical sound source.

Description

Technical Field

[0001] Various embodiments relate generally to audio systems and, more particularly, to techniques for adding distance-dependent reverberation to audio signals of virtual sound sources. Background Art

[0002] Thanks to advances in real-time audio processing and distribution, augmented reality can be implemented in acoustic environments to produce sound localization of virtual sound sources. For example, differences in the timing and intensity of sounds produced by different speakers in an acoustic environment can simulate the auditory cues that the human auditory system relies on to perceive location. Such augmented reality technologies enable users to have a more realistic experience when encountering virtual sound sources in an acoustic environment.

[0003] One drawback of conventional augmented reality technology is its limited ability to accurately simulate large distances. For example, in a small acoustic environment, the speakers that produce virtual audio sources are relatively close to each other, and the sound localization produced by the timing and intensity differences of the sounds output by such speakers cannot realistically simulate large rooms or large distances for the human auditory system. Therefore, conventional augmented reality technology cannot simulate virtual sound sources located in large acoustic spaces or at relatively long distances, and listeners cannot get an immersive experience.

[0004] In view of this, more efficient techniques are needed to generate audio signals for virtual sound sources. Summary of the invention

[0005] Various embodiments disclose a computer-implemented method for generating a perceived position of a sound source in an acoustic environment. The method includes: determining a current position of a first virtual audio source relative to a listening area of ​​the acoustic environment; determining a wet signal distribution based on the current position of the first virtual audio source; generating a first virtual audio signal for a first physical sound source included in the acoustic environment, wherein the first virtual audio signal is associated with the first virtual audio source and is based on the wet signal distribution; and transmitting the first virtual audio signal to the first physical sound source for output by the first physical sound source.

[0006] Further embodiments provide, among other things, non-transitory computer-readable storage media storing instructions for implementing the above methods, and devices or systems configured to implement the above methods.

[0007] At least one technical advantage of the disclosed technology relative to the prior art is that the disclosed technology enhances the distance perception associated with the virtual sound source and the surrounding acoustic environment. Another advantage of the disclosed technology is that more accurate spatial information is conveyed to the listener, thereby facilitating the perception of the location of the virtual sound source. In some cases, the listener will perceive the virtual sound source as being farther away than it actually is, while in other cases, the listener will perceive the virtual sound source as being in a larger acoustic environment than it actually is. In summary, the above advantages provide the listener with a more immersive acoustic environment experience. These technical advantages provide one or more technical advances over prior art methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to be able to understand in detail the features of the various embodiments described above, the inventive concepts briefly outlined above may be more particularly described with reference to various embodiments, some of which are illustrated in the accompanying drawings. However, it should be noted that the drawings illustrate only typical embodiments of the inventive concepts and are therefore not to be construed as limiting the scope in any way, and that there are other equally effective embodiments.

[0009] Figure 1 shows a block diagram of an audio system configured to implement one or more aspects of the present disclosure;

[0010] Figure 2 is a conceptual diagram of an acoustic environment in which a listening area is disposed, according to various embodiments;

[0011] Figure 3 is a graph showing a function for determining a wet signal distribution 186 according to an embodiment;

[0012] Figure 4 A flowchart showing method steps for simulating a distant virtual sound source in an acoustic environment according to various embodiments of the present disclosure;

[0013] Figure 5 is a conceptual diagram of an acoustic environment in which a listening area is disposed, according to various embodiments;

[0014] Figure 6 is a graph showing a function for determining a wet signal distribution 186 according to an embodiment; and

[0015] Figure 7 A flowchart showing method steps for distant virtual sound sources in a large acoustic environment according to various embodiments of the present disclosure. DETAILED DESCRIPTION

[0016] The following description sets forth numerous specific details to provide a more thorough understanding of various embodiments. However, it will be apparent to one skilled in the art that the concepts of the present invention may be practiced without one or more of these specific details.

[0017] System Overview

[0018] Figure 1 A block diagram of an audio system 100 configured to implement one or more aspects of the present invention is shown. According to an embodiment, the audio system 100 makes the virtual sound sources in the acoustic environment 102 sound more realistic to the listener 104 or other users present in the acoustic environment 102. Therefore, the listener 104 perceives the sounds generated by the physical sound sources 112, 114, and 116 (collectively referred to as the physical sound sources 110 herein) as virtual sound sources to more realistically and accurately simulate the sounds of the real world. The acoustic environment 102 may be, but is not limited to, a space or room in which the physical sound sources 110, the virtual sound sources 130, and the listener 104 are disposed. Figure 1 In the illustrated embodiment, audio system 100 includes, but is not limited to, a plurality of physical sound sources 110, a controller 150, a virtual sound source 130, and in some embodiments, one or more position sensors 140. Controller 150 includes, but is not limited to, a processor 152, a memory 154, and a reverberation application 156.

[0019] The physical sound source 110 is an audio output device that generates a sound output 106 within the acoustic environment 102, such as a sound output associated with the virtual sound source 130. For example, the physical sound source 110 may include a device capable of providing a sound output, such as a loudspeaker. The physical sound source 110 may be a wired or wireless speaker system ( For example , one or more loudspeakers, amplifiers, etc.), or any other device that generates sound output 106. In some embodiments, each physical sound source 110 may include a position sensor 140 that indicates the location of that particular physical sound source 110, such as an inertial measurement unit (IMU), wireless transmitter, or other electronic device, in the acoustic environment 102.

[0020] The virtual sound source 130 can be any positionally trackable object that the audio system 100 can use to generate a perceived spatial location within the acoustic environment 102 for the listener 104, such as through the sound output 106. For example, in some embodiments, the virtual sound source 130 can correspond to a toy or other electronically or visually marked device that is configured such that the position sensor 140 can generate position information indicating the current location of the virtual sound source 130. In some embodiments, the virtual sound source 130 includes an electronic device or tag that transmits position information indicating the current location of the virtual sound source 130 within the acoustic environment 102 to one or more position sensors 140. Alternatively or additionally, in some embodiments, the virtual sound source 130 includes one or more visual tags or markers that enable an optical sensor included in the position sensor 140 to optically determine the position information indicating the current location of the virtual sound source 130 within the acoustic environment 102. Therefore, when the virtual sound source 130 moves and / or is repositioned within the acoustic environment 102, the audio system 100 can determine the current position of the virtual sound source 130 through the position sensor 140 and the controller 150. The audio system 100 can then generate appropriate sound output 106 that causes the listener 104 to perceive the sound emanating from the current position of the virtual sound source 130.

[0021] The position sensor 140 may include, but is not limited to, an array of one or more different sensors configured to receive signals, perform measurements, or otherwise generate position information 170 indicating the current position of the virtual sound source 130 within the acoustic environment 102. In some embodiments, the position sensor 140 may detect or measure one or more properties associated with the virtual sound source 130 and / or the acoustic environment 102. The position sensor 140 may include, but is not limited to, an optical sensor (visible light or infrared), an acoustic sensor (e.g., an ultrasonic sensor, an active sonar, etc.), a radar (RADAR) sensor, a laser radar (LIDAR) sensor, a depth sensor, a stereo imaging sensor, a terrain mapping sensor, a telematic sensor, a receiver, etc. In such embodiments, the position sensor 140 is configured to receive sensor data and transmit the sensor data as position information 170 to the controller 150 for processing. Alternatively, in some embodiments, the position sensor 140 receives the position information 170 transmitted from the virtual sound source 130, such as the position information 170 transmitted from an IMU or other electronic device included in the virtual sound source 130.

[0022] The controller 150 generates one or more virtual audio signals 180 and drives the physical sound source 110 to generate the sound output 106, which can produce the perceived position of the virtual sound source 130 to the listener 104. Figure 1In the illustrated embodiment, the controller 150 includes a processor 152, a memory 154, and a reverb application 156. The processor 152 is configured to read data from and write data to the memory 154. In various embodiments, an interconnect bus (not shown) connects the processor 152, the memory 154, and any other components of the processor 152.

[0023] The reverberation application 156 generates a reverberation signal, such as a reverberation signal 184, based on a specific reference audio signal 182. The reverberation application 156 may be any technically feasible software application for generating a reverberation signal from an input signal. The reverberation application 156 may include any suitable reverberation algorithm.

[0024] The processor 152 may be any suitable processor, such as a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), and / or any other type of processing unit, or a combination of different processing units, such as a CPU configured to operate in conjunction with a GPU. In general, the processor 152 may be any technically feasible hardware unit capable of processing data and / or executing software applications. In some embodiments, the reverberation application 156 and / or other instructions resident in the memory 154 may be executed by the processor 152 to implement the overall functionality of the controller 150.

[0025] The memory 154 may include a random access memory (RAM) module, a flash memory unit, an EEPROM, or any other type of memory unit or combination thereof. In various embodiments, the memory 154 includes a non-volatile memory, such as an optical drive, a magnetic disk drive, a flash drive, or other memory. In certain embodiments, a separate data storage, such as an external data storage included in a network ("cloud storage"), may supplement or constitute the memory 154.

[0026] In operation, the controller 150 receives one or more audio signals 160 and position information 170. Based on the position information 170, the controller 150 modifies the one or more audio signals 160 to generate a virtual audio signal 180 for the physical sound source 110. In some embodiments, the controller 150 receives a different audio signal 160 for each physical sound source 110 and generates a different virtual audio signal 180 for each physical sound source 110. For example, in some embodiments, the audio signal 160 for each physical sound source is different to produce a sound localization of the virtual sound source 130 within the acoustic environment 102.

[0027] In some embodiments, each different audio signal 160 is associated with the same virtual sound source 130, but is customized for a different physical sound source 110. For example, in one such embodiment, the amplitude of each audio signal 160 is different. In such embodiments, the amplitude of each audio signal 160 is different to accurately distribute audio from the virtual sound source 130 to the physical sound source 110 in a perceptually accurate manner in terms of timbre and localization accuracy, so that the listener 104 perceives the source of the sound output 106 to be consistent with the current position of the virtual sound source 130. However, as previously described, the sound localization produced in this manner cannot realistically simulate the large room or large distance of the human auditory system. Therefore, according to various embodiments, the controller 150 generates each virtual audio signal 180 by modifying the corresponding audio signal 160, as described below.

[0028] In some embodiments, the controller 150 generates a signal based on a specific reference audio signal 182 ( For example , the reverberation signal 184 of the audio signal 160 of the physical sound source 112 to generate a specific virtual audio signal 180 (For example , a virtual audio signal 180 of the physical sound source 112). The reference audio signal 182 and the corresponding reverberation signal 184 are then mixed according to the distance-dependent wet signal distribution 186. In some embodiments, the controller 150 determines the distance-dependent wet signal distribution 186 based on the position of the virtual sound source 130 in the acoustic environment 102, and in other embodiments, the controller 150 determines the distance-dependent wet signal distribution 186 based on the size of the listening area within the acoustic environment 102. Figures 2 to 7 Examples of such implementations are described.

[0029] Reverberation for simulating distant virtual sound sources

[0030] In some embodiments, the controller 150 generates an acoustic signal component ( For example , reverberation signal 184) to simulate a distant virtual sound source 130, such as when the virtual sound source 130 is located outside the listening area of ​​the acoustic environment 102. In such embodiments, the controller 150 determines a distance-dependent wet signal distribution 186 based on the location of the virtual sound source 130 in the acoustic environment 102 relative to the listening area. Figure 2 One embodiment of a listening zone in the acoustic environment 102 is described.

[0031] Figure 21 is a conceptual diagram of an acoustic environment 102 in which a listening area 200 is arranged, according to various embodiments. Also shown in the acoustic environment 102 are virtual sound sources 130, a listener 104, a boundary 202 (dashed line) of the listening area 200, and a maximum reverberation boundary 208 (dashed line). When the virtual sound source 130 changes position within or near the listening area 200, the audio system 100 modifies the virtual audio signal provided to the physical sound source 110 to produce sound localization within the listening area 200, which makes the listener 104 feel that the sound output from the physical sound source 110 is consistent with the current position of the virtual sound source 130. According to various embodiments, the audio system 100 also generates these virtual audio signals, wherein the reverberation signal component ( For example , Figure 1 The reverberation signal in 184) and the distance-dependent wet signal distribution ( For example, The reference audio signal ( example like , reference audio signal 182) mixed.

[0032] As shown, the listening area 200 includes a boundary 202 indicating the extent of the listening area 200. In some embodiments, the controller 150 determines the boundary 202 based on the location of the physical sound source 110. For example, in Figure 2 In the illustrated embodiment, the boundary 202 is implemented as a perimeter ( For example , surrounds all physical sound sources 110 in the acoustic environment 102. In some embodiments, the size and position of the circle are selected to be a circle with a minimum radius that encloses all physical sources 110. In other embodiments, the boundary 202 can be implemented as any other technically feasible shape, such as a polygon, where each physical sound source 110 corresponds to a vertex of the polygon, or a geometric shape ( For example , square, rectangle, triangle, hexagon, etc.).

[0033] According to various embodiments, when the virtual sound source 130 is disposed within the range of the boundary 202 of the listening area 200, a first reverberation method is adopted to generate the virtual audio signal 180. In contrast, when the virtual sound source 130 is positioned outside the range of the boundary 202 of the listening area 200, a second reverberation method is adopted to generate the virtual audio signal 180. In such embodiments, in the first reverberation method, a virtual audio signal 180 without a reverberation signal component is generated, while in the second reverberation method, a virtual audio signal 180 with a reverberation signal component is generated, the reverberation signal component being mixed with a reference audio signal based on a distance-dependent wet signal distribution. Therefore, in such embodiments, when the virtual sound source 130 is disposed outside the boundary 202, by generating the virtual audio signal 180 for the physical sound source 110, the listener 104 feels that the position of the virtual sound source 130 is farther, thereby producing a more realistic and natural listening experience for the listener 104.

[0034] In some embodiments, when the virtual sound source 130 is located outside the boundary 202, the controller 150 (e.g., through the reverberation application 156) generates each virtual audio signal 180 based on the specific reference audio signal 182, the reverberation signal 184, and the distance-dependent wet signal distribution 186. In such embodiments, the controller 150 determines the value of the distance-dependent wet signal distribution 186 based on the distance 206 between the virtual sound source 130 and the boundary 202. In some embodiments, the controller 150 also determines the value of the distance-dependent wet signal distribution 186 based on whether the virtual sound source 130 is set at or outside the maximum reverberation boundary 208. Figure 3 Various embodiments for determining the wetness signal distribution 186 are described.

[0035] Figure 3 3 is a graph 300 showing a function 310 for determining the value of the wet signal distribution 186 according to one embodiment. The function 310 enables the virtual sound source 130 to be determined based on the distance between the virtual sound source 130 and the boundary 202 of the listening area 200 (e.g. Figure 2 206 in the boundary 202). As shown, the function 310 increases from a minimum wet signal value 304 value of 0% (corresponding to when the virtual sound source 130 is set at or within the boundary 202) to a maximum wet signal value 306 (corresponding to when the virtual sound source 130 is set at a position greater than or equal to the maximum reverberation distance 308). Therefore, as the virtual sound source 130 moves away from the boundary 202, the value of the wet signal distribution 186 increases, thereby causing the virtual audio signal 180 to include more reverberation signal 184 mixed in, and causing the listener to perceive that the virtual sound source 130 is located at a farther distance.

[0036] It is noteworthy that as the value of the wet signal distribution 186 (as shown in the function 310) increases, the virtual audio signal 180 includes a smaller proportion of the reference audio signal 182 mixed. Therefore, the function 310 indirectly indicates the mixing of the dry signal of the virtual audio signal 180. For example, when the function 310 indicates that the wet signal distribution value is 0%, the virtual audio signal 180 is mixed to include 100% of the reference audio signal 182. Similarly, when the function 310 indicates that the wet signal distribution value is 50%, the virtual audio signal 180 is mixed to include 50% of the reference audio signal 182.

[0037] exist Figure 3 In the illustrated embodiment, function 310 varies nonlinearly with distance from boundary 202 up to a maximum wet signal value 306. For example, Figure 3 In the illustrated embodiment, the maximum wet signal value 306 corresponds to a wet signal distribution value of 80%. In other embodiments, the function 10 varies linearly with the distance from the boundary 202 up to the wet signal value 306. For example, in one such embodiment, the function 310 starts at 0% when the virtual sound source 130 is set at or within the boundary 202, and increases to the maximum wet signal value 306 when the virtual sound source 130 is set at or outside the maximum reverberation distance 308 (where the maximum reverberation distance 308 corresponds to being set at Figure 2 In yet other embodiments, the function 310 may include linear and nonlinear parts.

[0038] Figure 4 A flowchart showing method steps for simulating a distant virtual sound source in an acoustic environment according to various embodiments of the present disclosure is shown. Figure 1 and 2 Although the method steps are described with reference to a system, those skilled in the art will understand that any system configured to implement the method steps in any order falls within the scope of the present disclosure.

[0039] Prior to the method steps, the controller 150 determines the boundaries 202 of the listening area 200, for example based on the locations of the physical sound sources 110 included in the audio system 100. In some embodiments, the listening area 200 may be a two-dimensional area in the acoustic environment 102. In other embodiments, the listening area 200 may be a three-dimensional area in the acoustic environment 102.

[0040] As shown, the computer-implemented method 400 begins at step 402, where the controller 150 determines the current position of the virtual sound source 130, for example, based on the position information 170 received from the position sensor 140. In some embodiments, the position sensor 140 measures or otherwise determines the position information 170, while in other embodiments, the position sensor 140 receives the position information 170 from the virtual sound source 130.

[0041] In step 404, the controller 150 determines the distance between the virtual sound source 130 and the boundary 202 of the listening area 200 (such as Figure 2 The distance 206 in the wet signal distribution 186 is determined. For example, in some embodiments, the wet signal distribution 186 is determined using Figure 3 The value of the wet signal distribution 186 is determined by a function consistent with the function 310 in FIG. In general, as the distance 206 increases above 0, the value of the wet signal distribution 186 also increases.

[0042] In step 406, the controller 150 selects a physical sound source 110 from a plurality of physical sound sources 110 included in the audio system 100. In step 408, the controller 150 generates a virtual audio signal 180 for the selected physical sound source 110. In some embodiments, the controller 150 generates a reverberation signal 184 based on the reference audio signal 182 of the selected physical sound source 110. In such embodiments, the reverberation function generates the reverberation signal 184. Then, the controller 150 generates the virtual audio signal 180 for the selected physical sound source 110 by mixing the reference audio signal 182 with the reverberation signal 184 according to the wet signal distribution 186 determined in step 404. It is worth noting that in the case where the value of the wet signal distribution 186 is 0%, the controller 150 may not generate the reverberation signal 184 in step 408 because no wet reverberation signal 184 is mixed with the reference audio signal 182 to generate the virtual audio signal 180. As described above, when reverberation is used to simulate a distant virtual sound source, when the virtual sound source 130 is disposed at or within the boundary 202 , the virtual audio signal 180 does not contain reverberation.

[0043] In step 410, the controller 150 determines whether any remaining physical audio sources require a virtual audio signal 180. If so, the computer-implemented method 400 returns to step 406; if not, the computer-implemented method 400 continues to step 412. In step 412, the controller 150 transmits each virtual audio signal 180 to a corresponding physical sound source 110 in the audio system 100. Typically, the virtual audio signals 180 are transmitted to the physical sound sources 110 simultaneously.

[0044] Reverberation simulation of virtual sound sources in large acoustic environments

[0045] In some embodiments, the controller 150 generates a signal having a reverberation signal component ( For example , reverberation signal 184) to make the listener feel that the virtual sound source 130 is set in a large acoustic environment (e.g., a very large room). In such an embodiment, the controller 150 determines the distance-dependent wet signal distribution 186 based on the size of the listening area where the virtual sound source 130 is located. Figure 5 One embodiment of a listening zone in the acoustic environment 102 is described.

[0046] Figure 5 1 is a conceptual diagram of an acoustic environment 102 in which a listening area 500 is arranged according to various embodiments. Also shown in the acoustic environment 102 are a virtual sound source 130, a listener 104, a boundary 502 of the listening area 500, a center point 504 of the listening area 500, and a characteristic length 506 of the boundary 502. When the virtual sound source 130 changes position within the listening area 500, the audio system 100 modifies the virtual audio signal provided to the physical sound source 110 to produce sound localization within the listening area 500, which causes the listener 104 to perceive the sound output from the physical sound source 110 as being consistent with the current position of the virtual sound source 130. According to various embodiments, the audio system 100 also utilizes a reverberation signal component ( For example , Figure 1 The virtual audio signals are modified by the reverberation signal 184 in the image, which is related to the wet signal distribution based on the distance correlation ( For example , the reference audio signal ( For example , the reference audio signal 182). For example, as the size of the listening area increases, the value of the distance-dependent wet signal distribution increases.

[0047] As shown, the listening area 500 (dashed line) includes a boundary 502 indicating the extension of the listening area 500. In some embodiments, the controller 150 determines the boundary 502 based on the location of the physical sound source 110. For example, in Figure 5 In the illustrated embodiment, the boundary 502 is implemented as a circle that encloses or surrounds all physical sound sources 110 in the acoustic environment 102. In some embodiments, the size and position of the circle are selected to be a circle with a minimum radius that encloses all physical sources 110. In other embodiments, the boundary 202 can be implemented as any other technically feasible shape, such as a polygon, in which each physical sound source 110 corresponds to a vertex, or a geometric shape ( For example , square, rectangle, triangle, hexagon, etc.).

[0048] According to various embodiments, when the size of the listening area 500 is less than a threshold, a first reverberation method is used when generating the virtual audio signal 180. In contrast, when the size of the listening area 500 is greater than the threshold, a second reverberation method is used when generating the virtual audio signal 180. In such embodiments, in the first reverberation method, a virtual audio signal 180 without a reverberation signal component is generated, while in the second reverberation method, a virtual audio signal 180 with a reverberation signal component is generated, which is mixed with a reference audio signal based on a distance-dependent wet signal distribution. Therefore, in such embodiments, when the listening area 500 extends beyond a certain size, by generating a virtual audio signal 180 for the physical sound source 110, the listener 104 feels that the virtual sound source 130 is in a larger acoustic environment, thereby producing a more realistic and natural listening experience for the listener 104.

[0049] In some embodiments, when the size of the listening area 500 exceeds a threshold, the controller 150 (e.g., via the reverberation application 156) generates each virtual audio signal 180 based on the specific reference audio signal 182, the reverberation signal 184, and the distance-dependent wet signal distribution 186. In such embodiments, the controller 150 determines the value of the distance-dependent wet signal distribution 186 based on a value indicating the size of the listening area 500. For example, in some embodiments, the value indicating the size of the listening area 500 is the size of the listening area 500. Alternatively, in some embodiments, the value indicating the size of the listening area 500 is a characteristic length 506 of the listening area 500. Figure 5 In the illustrated embodiment, the boundary 502 of the listening area 500 is a circle that encompasses or surrounds all physical sound sources 110, and the characteristic length 506 is the radius of the circle, extending from the center point 504 to the boundary 502. Therefore, in such embodiments, when the radius of the listening area 500 exceeds a certain threshold, the controller 150 determines a non-zero value of the distance-dependent wet signal distribution 186 and generates each virtual audio signal 180 based in part on the reverberant signal 184 and the distance-dependent wet signal distribution 186. Figure 6 Various embodiments for determining the wetness signal distribution 186 are described.

[0050] In embodiments where the listening area 500 is not circular, the characteristic length 506 is typically a feature of the border 502 other than the radius. For example, when the listening area 500 is square or rectangular, the characteristic length 506 may be the width of the border 502, the length of the border 502, or the diagonal of the border 502.

[0051] Figure 6 6 is a graph 600 illustrating a function 610 for determining a value of a wet signal distribution 186 according to one embodiment. The function 610 can be based on a characteristic length 506 of the listening area 500 (such as Figure 5 ) selects the value of the wet signal distribution 186 based on the radius of the boundary 502 in the sound output. As shown, the function 610 increases from a minimum wet signal value 604 value of 0% (corresponding to when the feature length 506 is less than the threshold 612) to a maximum wet signal value 606 (corresponding to the maximum acceptable reverberation level in the sound output, such as when the feature length 506 is equal to or exceeds the maximum reverberation length 608). In some embodiments, the maximum wet signal value 606 is selected based on a reverberation level that is known to be distracting or unrealistic. Therefore, as the size of the listening area 500 increases, the value of the wet signal distribution 186 also increases, resulting in the virtual audio signal 180 including a larger mix of the reverberant signal 184 and causing the listener to perceive that the virtual sound source 130 is located in a larger acoustic environment. It is noteworthy that as the value of the wet signal distribution 186 (as shown in the function 610) increases, the virtual audio signal 180 includes a smaller proportion of the reference audio signal 182. Therefore, the function 610 indirectly indicates the dry signal mix of the virtual audio signal 180.

[0052] exist Figure 6 In the illustrated embodiment, function 610 varies nonlinearly with characteristic length 506 up to a maximum wet signal value 606. For example, Figure 6 In the illustrated embodiment, the maximum wet signal value 606 corresponds to a wet signal distribution value of 70%. In other embodiments, the function 610 varies linearly with distance from the boundary 502 up to the wet signal value 606. For example, in one such embodiment, when the listening area 500 is less than a threshold size, the function 610 starts at 0% and increases linearly to the maximum wet signal value 606 as the size of the listening area 500 increases. In still other embodiments, the function 610 may include linear and non-linear portions.

[0053] Figure 7 A flowchart showing method steps for a distant virtual sound source in a large acoustic environment according to various embodiments of the present disclosure is shown. Figure 1 and 5 Although the method steps are described with reference to a system, those skilled in the art will understand that any system configured to perform the method steps in any order falls within the scope of the present disclosure.

[0054] As shown, the computer-implemented method 700 begins at step 702, where the controller 150 determines the current size of the listening area 500, for example, based on the boundary 502. In some embodiments, the controller 150 first determines the boundary 502 based on the current position of the physical sound source 110 included in the audio system 100. For example, in some embodiments, the controller 150 receives the current position of the physical source 110 via the position information 170 received from the position sensor 140. In some embodiments, the listening area 500 can be a two-dimensional area in the acoustic environment 102. In other embodiments, the listening area 500 can be a three-dimensional area in the acoustic environment 102. In some embodiments, the controller 150 quantifies the current size of the listening area 500 based on the characteristic length 506 of the boundary 502.

[0055] In step 704 , the controller 150 determines whether the size of the listening area exceeds a threshold. If not, the computer-implemented method 700 returns to step 702 ; if yes, the computer-implemented method 700 continues to step 706 .

[0056] In step 706, the controller 150 determines the size of the listening area 500 based on the size of the listening area 500 (such as Figure 5 The characteristic length 506 in the wet signal distribution 186 is determined. For example, in some embodiments, the wet signal distribution 186 is determined using Figure 6 The value of the wet signal distribution 186 is determined by a function consistent with the function 610 in . In general, as the characteristic length 506 increases above 0, the value of the wet signal distribution 186 also increases.

[0057] In step 708, the controller 150 selects a physical sound source 110 from a plurality of physical sound sources 110 included in the audio system 100. In step 710, the controller 150 generates a virtual audio signal 180 for the selected physical sound source 110. In some embodiments, the controller 150 generates a reverberation signal 184 based on the reference audio signal 182 of the selected physical sound source 110. In such embodiments, the reverberation application generates the reverberation signal 184. Then, the controller 150 generates the virtual audio signal 180 for the selected physical sound source 110 by mixing the reference audio signal 182 with the reverberation signal 184 according to the wet signal distribution 186 determined in step 706.

[0058] In step 712, the controller 150 determines whether any remaining physical audio sources require a virtual audio signal 180. If so, the computer-implemented method 700 returns to step 708; if not, the computer-implemented method 700 continues to step 714. In step 712, the controller 150 transmits each virtual audio signal 180 to a corresponding physical sound source 110 in the audio system 100. Typically, the virtual audio signals 180 are transmitted to the physical sound sources 110 simultaneously. The computer-implemented method 700 returns to step 702.

[0059] In summary, the audio system determines the value of the wet signal distribution based on the distance between the virtual sound source and the boundary of the listening area or alternatively based on the size of the listening area. Then, a virtual audio signal is generated based on the value of the wet signal distribution, and the virtual audio signal is transmitted to the physical sound source of the audio system for output. The output audio signal generates a reverberant acoustic environment.

[0060] At least one technical advantage of the disclosed technology over the prior art is that the disclosed technology enhances the distance perception associated with the virtual sound source and the surrounding acoustic environment. Another advantage of the disclosed technology is that more accurate spatial information is conveyed to the listener, thereby facilitating the perception of the location of the virtual sound source. In some cases, the listener will feel that the virtual sound source is farther away than it actually is, while in other cases, the listener will feel that the virtual sound source is in a larger acoustic environment than it actually is. In summary, the above advantages provide the listener with a more immersive acoustic environment experience. These technical advantages provide one or more technical advances over prior art methods.

[0061] 1. In some embodiments, a computer-implemented method for generating a perceived position of a sound source in an acoustic environment includes: determining a current position of a first virtual audio source relative to a listening area of ​​the acoustic environment; determining a wet signal distribution based on the current position of the first virtual audio source; generating a first virtual audio signal for a first physical sound source included in the acoustic environment, wherein the first virtual audio signal is associated with the first virtual audio source and is based on the wet signal distribution; and transmitting the first virtual audio signal to the first physical sound source for output by the first physical sound source.

[0062] 2. A computer-implemented method according to clause 1, further comprising: generating a second virtual audio signal for a second physical sound source included in the acoustic environment, wherein the second virtual audio signal is associated with the first virtual audio source and is distributed based on the wet signal; and transmitting the second virtual audio signal to the second physical sound source for output by the second physical sound source.

[0063] 3. A computer-implemented method according to clause 1 or 2, wherein at least a portion of the first virtual audio signal is transmitted to a first physical sound source and at least a portion of the second virtual audio signal is transmitted to a second physical sound source.

[0064] 4. A computer-implemented method according to any of clauses 1-3, wherein generating the second virtual audio signal based on the wet signal distribution comprises: generating a reverberation signal based on the audio signal of the second physical sound source; and combining the reverberation signal with the audio signal based on the wet signal distribution.

[0065] 5. A computer-implemented method according to any of clauses 1-4, wherein generating the first virtual audio signal based on the wet signal distribution comprises: generating a reverberation signal based on the audio signal of the first physical sound source; and combining the reverberation signal with the audio signal based on the wet signal distribution.

[0066] 6. The computer-implemented method of any of clauses 1-5, further comprising determining a boundary of the listening area based on a first position of the first physical sound source in the acoustic environment and a second position of the second physical sound source in the acoustic environment.

[0067] 7. The computer-implemented method of any of clauses 1-6, wherein determining the wet signal profile comprises setting the wet signal profile to zero when the current position of the first virtual audio source is disposed within the boundary.

[0068] 8. A computer-implemented method according to any of clauses 1-7, wherein determining the wet signal distribution comprises setting the wet signal distribution to zero when: the current position of the first virtual audio source is set within the boundary; and the size of the listening area is less than a threshold.

[0069] 9. The computer-implemented method of any of clauses 1-8, wherein determining the wet signal profile comprises setting the wet signal profile to a value greater than zero when the current position of the first virtual audio source is set outside of the boundary.

[0070] 10. The computer-implemented method of any of clauses 1-9, further comprising: determining a distance between a current position of the first virtual audio source and the border; and determining a value greater than zero based on the distance.

[0071] 11. A computer-implemented method according to any of clauses 1-10, further comprising: determining a first position of a first physical sound source based on a first signal received from the first physical sound source or one of the optically determined positions of the first physical sound source; and determining a second position of a second physical sound source based on a second signal received from the second physical sound source or one of the optically determined positions of the second physical sound source.

[0072] 12. A computer-implemented method as recited in any of clauses 1-11, wherein a boundary of the listening area corresponds to a circular area surrounding the first physical sound source and the second physical sound source.

[0073] 13. A computer-implemented method according to any of clauses 1-12, wherein the current position of the first virtual audio source corresponds to the current position of the toy or marking device.

[0074] 14. In some embodiments, a non-transitory computer-readable medium includes a set of instructions, which in response to being executed by a processor of a computing system causes the processor to simulate a large acoustic space in an acoustic environment by performing the following steps: determining a size of a listening area based on a first position of a first physical sound source in the acoustic environment and a second position of a second physical sound source in the acoustic environment; in response to determining that the size of the listening area exceeds a threshold, determining a wet signal distribution based on the size; generating a first virtual audio signal for the first physical sound source, wherein the first virtual audio signal is associated with the first virtual audio source and is based on the wet signal distribution; and transmitting the first virtual audio signal to the first physical sound source to be output by the first physical sound source.

[0075] 15. A non-transitory computer-readable medium according to clause 14, further comprising instructions which, when executed by a processor, cause the processor to further perform the following steps: generating a second virtual audio signal for a second physical sound source, wherein the second virtual audio signal is associated with the first virtual audio source and is based on a wet signal distribution; and transmitting the second virtual audio signal to the second physical sound source for output by the second physical sound source.

[0076] 16. The non-transitory computer-readable medium of clause 14 or 15, wherein at least a portion of the second virtual audio signal is transmitted to the second physical sound source while at least a portion of the first virtual audio signal is transmitted to the first physical sound source.

[0077] 17. The non-transitory computer-readable medium of any of clauses 14-16, wherein the size comprises a characteristic length of a boundary of a listening area.

[0078] 18. The non-transitory computer-readable medium of any of clauses 14-17, wherein the characteristic length of the border comprises one of a radius of the border, a width of the border, a length of the border, or a diagonal of the border.

[0079] 19. The non-transitory computer-readable medium of any of clauses 14-18, wherein the first virtual audio source corresponds to a toy or a marking device.

[0080] 20. In some embodiments, an audio system includes: a first physical sound source and a second physical sound source set in an acoustic environment; a memory storing instructions; and a processor, the processor being communicatively coupled to the memory and configured to perform the following steps when executing the instructions: determining a size of a listening area based on a first position of the first physical sound source in the acoustic environment and a second position of the second physical sound source in the acoustic environment; in response to determining that the size of the listening area exceeds a threshold, determining a wet signal distribution based on the size; generating a first virtual audio signal for the first physical sound source, wherein the first virtual audio signal is associated with the first virtual audio source and is based on the wet signal distribution; and transmitting the first virtual audio signal to the first physical sound source to be output by the first physical sound source.

[0081] Any and all combinations of any claim elements recited in any claim and / or any elements described in this application, in any manner, are within the intended scope of the invention and protection.

[0082] The description of the various embodiments is for illustrative purposes only and is not intended to be exhaustive or limited to the embodiments disclosed. It will be apparent to those skilled in the art that many modifications and variations can be made without departing from the scope and spirit of the described embodiments.

[0083] Aspects of the present embodiment may be embodied as a system, method, or computer program product. Therefore, aspects of the present disclosure may take the form of a complete hardware implementation, a complete software implementation (including firmware, resident software, microcode, etc.), or an implementation combining software and hardware aspects, which may generally be referred to herein as a "module," "system," or "computer." In addition, any hardware and / or software technology, process, function, component, engine, module, or system described in the present invention may be implemented as a circuit or circuit group. In addition, aspects of the present invention may take the form of a computer program product embodied in one or more computer-readable media, wherein a computer-readable program code is embodied.

[0084] Any combination of one or more computer-readable media may be used. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or apparatus, or any suitable combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media would include the following: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the context of this document, a computer-readable storage medium may be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, device or device.

[0085] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiments of the present disclosure. It should be understood that each frame in the flowchart and / or block diagram and the combination of frames in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine. The instruction enables the function / action specified in the flowchart and / or block diagram frame to be implemented when the processor of the computer or other programmable data processing device is executed. Such processors may be, but are not limited to, general-purpose processors, special-purpose processors, application-specific processors or field programmable gate arrays.

[0086] The flow chart and block diagram in the figure show the architecture, function and operation of the possible implementation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each frame in the flow chart or block diagram can represent a part of a module, segment or code, which includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the function indicated in the frame may not occur in the order indicated in the figure. For example, two frames displayed continuously can actually be executed substantially at the same time, or frames can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each frame in the block diagram and / or flow chart and the combination of frames in the block diagram and / or flow chart can be implemented by a special hardware-based system or a combination of special hardware and computer instructions that perform a specified function or action.

[0087] While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the disclosure may be devised without departing from the basic scope thereof, and the scope of the disclosure is determined by the claims that follow.

Claims

1. A computer-implemented method for generating a perceived position of a sound source in an acoustic environment, the method comprising: determining a current position of a first virtual audio source relative to a listening area of ​​the acoustic environment; determining a wet signal distribution based on the current position of the first virtual audio source; generating a first virtual audio signal for a first physical sound source included in the acoustic environment, wherein the first virtual audio signal is associated with the first virtual audio source and is based on the wet signal distribution; as well as The first virtual audio signal is transmitted to the first physical sound source to be output by the first physical sound source.

2. The computer-implemented method of claim 1 , further comprising: generating a second virtual audio signal for a second physical sound source included in the acoustic environment, wherein the second virtual audio signal is associated with the first virtual audio source and is based on the wet signal distribution; as well as The second virtual audio signal is transmitted to the second physical sound source to be output by the second physical sound source. 3 . The computer-implemented method of claim 2 , wherein at least a portion of the first virtual audio signal is transmitted to the first physical sound source and at least a portion of the second virtual audio signal is transmitted to the second physical sound source.

4. The computer-implemented method of claim 2, wherein generating the second virtual audio signal based on the wet signal profile comprises: generating a reverberation signal based on the audio signal of the second physical sound source; as well as The reverberation signal is combined with the audio signal based on the wet signal profile.

5. The computer-implemented method of claim 1 , wherein generating the first virtual audio signal based on the wet signal profile comprises: generating a reverberation signal based on the audio signal of the first physical sound source; as well as The reverberation signal is combined with the audio signal based on the wet signal profile. 6 . The computer-implemented method of claim 1 , further comprising determining a boundary of the listening area based on a first position of the first physical sound source in the acoustic environment and a second position of the second physical sound source in the acoustic environment. 7 . The computer-implemented method of claim 6 , wherein determining the wet signal profile comprises setting the wet signal profile to zero when the current position of the first virtual audio source is disposed within the boundary.

8. The computer-implemented method of claim 7, wherein determining the wet signal profile comprises setting the wet signal profile to zero if: The current position of the first virtual audio source is set within the boundary; and The size of the listening area is smaller than a threshold. 9 . The computer-implemented method of claim 7 , wherein determining the wet signal profile comprises setting the wet signal profile to a value greater than zero when the current position of the first virtual audio source is disposed outside of the boundary.

10. The computer-implemented method of claim 9, further comprising: determining a distance between the current position of the first virtual audio source and the boundary; as well as The value greater than zero is determined based on the distance.

11. The computer-implemented method of claim 7, further comprising: determining the first position of the first physical sound source based on one of a first signal received from the first physical sound source or an optically determined position of the first physical sound source; as well as The second position of the second physical sound source is determined based on one of a second signal received from the second physical sound source or an optically determined position of the second physical sound source.

12. The computer-implemented method of claim 6, wherein the boundary of the listening area corresponds to a circular area surrounding the first physical sound source and the second physical sound source.

13. The computer-implemented method of claim 1, wherein the current position of the first virtual audio source corresponds to a current position of a toy or a marking device.

14. A non-transitory computer readable medium comprising a set of instructions that in response to execution by a processor of a computing system causes the processor to simulate a large acoustic space in an acoustic environment by performing the following steps: determining a size of the listening area based on a first position of a first physical sound source in the acoustic environment and a second position of a second physical sound source in the acoustic environment; in response to determining that the size of the listening area exceeds a threshold, determining a wet signal distribution based on the size; generating a first virtual audio signal for the first physical sound source, wherein the first virtual audio signal is associated with a first virtual audio source and is based on the wet signal distribution; and The first virtual audio signal is transmitted to the first physical sound source to be output by the first physical sound source.

15. The non-transitory computer readable medium of claim 14, further comprising instructions that, when executed by a processor, cause the processor to further perform the following steps: generating a second virtual audio signal for the second physical sound source, wherein the second virtual audio signal is associated with the first virtual audio source and is based on the wet signal distribution; and The second virtual audio signal is transmitted to the second physical sound source to be output by the second physical sound source. 16 . The non-transitory computer-readable medium of claim 15 , wherein at least a portion of the second virtual audio signal is transmitted to the second physical sound source while at least a portion of the first virtual audio signal is transmitted to the first physical sound source.

17. The non-transitory computer-readable medium of claim 16, wherein the size comprises a characteristic length of a boundary of the listening area. 18 . The non-transitory computer-readable medium of claim 17 , wherein the characteristic length of the border comprises one of a radius of the border, a width of the border, a length of the border, or a diagonal of the border.

19. The non-transitory computer-readable medium of claim 14, wherein the first virtual audio source corresponds to a toy or a marking device.

20. An audio system comprising: a first physical sound source and a second physical sound source disposed in an acoustic environment; a memory for storing instructions; as well as A processor is communicatively coupled to the memory and is configured to perform the following steps when executing the instructions: determining a size of a listening area based on a first position of the first physical sound source in the acoustic environment and a second position of the second physical sound source in the acoustic environment; in response to determining that the size of the listening area exceeds a threshold, determining a wet signal distribution based on the size; generating a first virtual audio signal for the first physical sound source, wherein the first virtual audio signal is associated with a first virtual audio source and is based on the wet signal distribution; and The first virtual audio signal is transmitted to the first physical sound source to be output by the first physical sound source.

Images