Techniques for rendering audio through multiple audio output devices
By using a microphone to detect the time difference through a computer method, determine the position of the audio output device, and generate position-based audio output, solving the problems of inaccurate positioning and complicated manual calibration when the audio output device renders spatial audio in the prior art, and realizing more accurate audio objects Positioning and trajectory rendering.
Patent Information
- Application Number
- CN202280095747.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-05-09
- Publication Date
- 2024-12-03
AI Technical Summary
When the existing audio output device renders spatial audio, due to the influence of position and distance, the audio positioning is inaccurate, and manual calibration is complicated and it is easy to cause inconsistent audio output.
By a computer implemented method, the microphone detection time difference between multiple audio output devices is used to determine the positions of each other, and a position-based audio output is generated to ensure that the output of each audio object is based on its accurate position and track within the physical space.
More accurate audio object positioning and trajectory rendering are achieved, and the position of the audio output device is automatically and accurately determined, avoiding the complexity and inconsistency of user manual calibration.
Smart Images

Figure CN119072930A_ABST
Abstract
Description
Technical Field
[0001] Various embodiments relate generally to audio output devices, and more particularly to rendering audio through multiple audio output devices. Background Art
[0002] Audio typically includes a mixture of audio objects. For example, the voiceover of a movie may include the speech of one or more characters, sound effects from one or more events, ambient noise from the characters' environment, and background music. As another example, music may include multiple components, such as singing, rhythm guitar, bass guitar, and drums. Audio can be captured by a microphone and rendered live, recorded and then played back, or synthesized by a device such as a computer.
[0003] It is often desirable to output recorded or generated audio through multiple audio output devices, such as multiple sets of wired or wireless speakers. Audio output devices are often positioned at specific locations within a physical space. For example, in a room organized as a home theater, a center speaker is positioned near the center of the front wall of the room, while a left front speaker, a right front speaker, a left rear speaker, and a right rear speaker are each positioned in a corresponding corner of the room. A media device such as a television or computer may transmit a signal to each speaker so that a listener within the physical space hears the combined output of all speakers.
[0004] In some cases, it is desirable to configure an audio output device to output spatial audio, in which audio objects are perceived by a listener as coming from a specific location within a physical space. However, the audio rendered by multiple audio output devices is affected by the distance between each audio output device and the listener. The speed of the audio in the air between the audio output device and the listener affects the timing of the audio perceived by the listener, and the attenuation of the intensity in the air affects the volume of the audio perceived by the listener. In addition, in such cases, the perceived direction of the audio is affected by the angle between each audio output device and the listener. Due to these factors, the effectiveness of spatial audio is affected by the location of the audio output device within the physical space.
[0005] To address these challenges, some audio systems include a user interface for adjusting calibration settings. For example, an audio system may allow a user to set or adjust the volume level and / or delay of each audio output device, and the correct combination of settings may compensate for the variable position of the audio output device. However, the manual calibration process may be complicated and the user may be confused or frustrated. The user may not be able to determine the appropriate settings for the specific arrangement of the audio output device, resulting in the audio positioning being no better than the original or default settings, or even possibly worse. In addition, the manual calibration settings of the audio output device are equally applicable to all audio objects, such as adjusting the intensity and delay of each speaker for all audio objects. As a result, for all audio objects, the output of each speaker is modified, regardless of the different positions of the audio objects, or further, regardless of the different tracks of the audio objects. Therefore, this calibration of the audio output device may result in poor positioning of the audio objects and / or inconsistent audio output of the audio output device for different audio objects.
[0006] As previously mentioned, more efficient techniques are needed for rendering audio through multiple audio output devices. Summary of the invention
[0007] In various embodiments, a computer-implemented method of generating audio includes causing each of a plurality of audio output devices to output an audio sample; for each other of the plurality of audio output devices, determining a detection time of the audio sample from each audio output device by each of two or more microphones included in the other audio output devices; determining a position of each audio output device relative to the other audio output devices based on the detection time of each of the audio samples by each of the audio output devices; and causing each of the plurality of audio output devices to generate an audio output associated with an audio object, wherein the output of each of the audio output devices is based on the position of the audio object and the position of each audio output device relative to the other audio output devices.
[0008] At least one technical advantage of the disclosed technology over the prior art is that, using the disclosed technology, the output of each audio object by each audio output device is based on the position of the audio output device relative to other audio output devices. As a result, the positioning and / or trajectory of each audio object is more accurately rendered by the audio output device based on their position in physical space. In addition, the disclosed calibration technology can determine the position of each audio output device relative to other audio output devices, including determining when the positions of two audio output devices are reversed. In addition, the disclosed calibration technology automatically and accurately determines the position of the audio output device compared to user-based calibration setting adjustments. These technical advantages provide one or more technical improvements over prior art methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] In order to be able to understand in detail the manner in which the above features of various embodiments are implemented, a more particular description of the inventive concept briefly outlined above may be made by reference to various embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the drawings illustrate only typical embodiments of the inventive concept and are therefore not to be considered as limiting the scope in any way and that there are other equally effective embodiments.
[0010] Figure 1 shows apparatus configured according to various embodiments;
[0011] Figure 2 According to various implementation schemes Figure 1 a graphical representation of detection of a first audio output device by a second audio output device;
[0012] Figure 3 According to various implementation schemes Figure 1 an illustration of a portion of a calibration process of a device for a plurality of audio output devices;
[0013] Figure 4 According to various implementation schemes Figure 1 an illustration of another portion of a calibration process of a device for a plurality of audio output devices;
[0014] Figure 5 According to various implementation schemes Figure 1 an illustration of yet another portion of a calibration process of a device for a plurality of audio output devices;
[0015] Figure 6 According to various implementation schemes Figure 1 The device renders a graphical representation of the audio object through a plurality of audio output devices;
[0016] Figure 7 A method for calibrating a Figure 1 A flow chart of method steps for configuring multiple audio output devices.
[0017] Figure 8 Shown according to various embodiments Figure 1 A flowchart of method steps for outputting audio objects to multiple audio output devices. DETAILED DESCRIPTION
[0018] In the following description, numerous specific details are set forth to provide a more thorough understanding of various embodiments. However, it will be apparent to one skilled in the art that the inventive concept may be practiced without one or more of these specific details.
[0019] Figure 1 An apparatus 100 configured to implement one or more aspects of various embodiments is shown. As shown, the apparatus 100 includes, but is not limited to, a processor 102, a memory 104, a storage device 106, and an interconnect bus 108. As shown, the memory 104 includes, but is not limited to, an audio output device positioning engine 114, an audio object 120, and an audio object rendering engine 122.
[0020] Processor 102 may be any suitable processor, such as a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), and / or any other type of processing unit or combination of different processing units, such as a CPU configured to operate in conjunction with a GPU. In general, processor 102 may be any technically feasible hardware unit capable of processing data and / or executing software applications.
[0021] The memory 104 may include random access memory (RAM) modules, flash memory cells, or any other type of memory cells or combinations thereof. The processor 102 is configured to read data from and write data to the memory 104. The memory 104 includes various software programs ( For example , operating system, one or more application programs) and application data associated with the software programs. Storage device 106 may include non-volatile storage devices for applications and data, and may include fixed disk drives or removable disk drives, flash memory devices, and CD-ROM, DVD-ROM, Blu-ray, HD-DVD or other magnetic, optical or solid-state storage devices. Interconnect bus 108 connects processor 102, memory 104, storage device 106 and any other components of device 100.
[0022] The device 100 is coupled to a plurality of audio output devices 110 located in a physical space 112. The plurality of audio output devices 110 may include, for example, a set of speakers in a home theater system. In some audio systems capable of rendering spatial audio ( example like , home theater system), each audio output device 110 corresponds to a specific channel corresponding to a specific location within physical space 112, such as a front left channel, a front right channel, a rear left channel, and a rear right channel. Generally speaking and as shown, the audio output devices 110 are not respectively positioned at the corners of a regular polygon that may correspond to the expected locations of the channels. That is, the audio output device 110-4 is not positioned at the vertices of a square or rectangle, but is positioned at the vertices of an irregular trapezoid. As shown, the device 100 is coupled to four audio output devices 110, but various embodiments are coupled to any number of audio output devices 110 ( For example , two, three, five and / or six or more audio output devices).
[0023] As shown, the audio object rendering engine 122 is a program stored in the memory 104 and executed by the processor 102 to generate an audio output device signal 124 for each of the plurality of audio output devices 110. For example, in an audio system where the first audio output device 110-1 is a left front speaker, the audio object rendering engine 122 sends a first audio output device signal 124-1 to the first audio output device 110-1, the first audio output device signal including a portion of the sound from the audio object 120 that the listener should perceive from the left front corner of the physical space 112. The effectiveness of the rendered spatial audio ( For example , the clarity with which a listener perceives that an audio object 120 is located at a particular location within physical space 112 ) is related to the accuracy with which the position of the audio output device 110 of the rendered audio object 120 matches the actual position of the audio output device 110 within physical space 112 .
[0024] As shown, the audio output device positioning engine 114 is a program stored in the memory 104 and executed by the processor 102 to determine the position 116 of the audio output device 110 in the physical space 112. That is, the device 100 does not define each audio output device 110 as rendering a channel associated with a fixed position ( For example, a left front speaker intended to be positioned in the left front corner of the room), but rather a calibration process is performed to detect the position of each audio output device 110 relative to the positions of the other audio output devices 110 within the physical space. Based on the calibration process, the device 100 stores the position 116 of each audio output device 110. For the audio objects 120, the audio object rendering engine 122 uses the determined position 116 of each audio output device 110 to determine at least a portion of the sound to be generated as the audio object 120 output by each audio output device 110.
[0025] Although Figure 1 1, but in various embodiments, the device 100 renders the audio of the audio object 120 according to a trajectory (such as a line, circle, or arc) that passes through the physical space 112. That is, the position of the audio object 120 within the physical space 112 changes over time. Based on the determined position 116 of the audio output device 110, the audio object rendering engine 122 can adjust the audio output device signal 124 of the corresponding audio output device 110. For example, at a first point in time, the trajectory of the audio object 120 may position the audio object 120 between the first audio output device 110-1 and the second audio output device 110-2. The audio object rendering engine 122 renders the audio output device signal 124, wherein the audio output device signal 124 for the first audio output device 110-1 and the second audio output device 110-2 at the first point in time includes at least a portion of the audio object 120, while the audio output device signal 124 for the third audio output device 110-3 and the fourth audio output device 110-4 does not include a portion of the audio object 120. At the second time point, the trajectory of the audio object 120 may position the audio object 120 near the third audio output device 110-3. The audio object rendering engine 122 renders the audio output device signal 124, wherein the audio output device signal 124 for the third audio output device 110-3 at the second time point includes the audio object 120, while the audio output device signals 124 for the other audio output devices 110-1, 110-2, 110-4 at the second time point do not include a portion of the audio object 120.
[0026] Although Figure 11 , but in various embodiments, the device 100 renders audio for a plurality of audio objects 120, each of which may be associated with a location and / or trajectory within the physical space 112. For example, a first audio object 120 may be associated with a first trajectory that surrounds the plurality of audio output devices 110 in a clockwise direction, while a second audio object 120 may be associated with a second trajectory that surrounds the plurality of audio output devices 110 in a counterclockwise direction. The audio object rendering engine 122 may generate an audio output device signal 124 for each audio output object 120 at each point in time to include a first component corresponding to at least a portion of the first audio object 120 and / or a second component corresponding to a portion of the second audio object 120. That is, the audio output device signal 124 for each audio output device 110 includes a sum of the portions of the corresponding audio objects 120. As a result, the audio output device signal 124 for each audio output device 110 includes a combination or superposition of the audio associated with the corresponding audio object 120 to be rendered by the audio output device 110 at each point in time.
[0027] Figure 2 According to various implementation schemes Figure 1 FIG. 1 is a diagram of a detection of a first audio output device 110-1 by a second audio output device 110-2. As shown in the figure, the second audio output device 110 includes two microphones 202-1, 202-2.
[0028] During calibration, the first audio output device 110-1 transmits an audio sample 206 at a transmission time 208. For example, the audio sample 206 may include a tone of a given frequency, a frequency sweep over a portion of a human audible frequency range and / or a human inaudible frequency range, white noise or pink noise, etc. The second audio output device 110-2 includes two microphones 202-1, 202-2 separated by a distance 204. For example, within the audio output device 110-2, the first microphone 202-1 may be positioned on the left side and the second microphone 202-2 may be positioned on the right side with a spatial separation of 0.3 meters. Due to the speed at which the audio sample 206 propagates in the air, the first microphone 202-1 detects the audio sample 206 at the first detection time 210-1, and the second microphone 202-2 detects the audio sample 206 at the second detection time 210-2. As shown, since the first audio output device 110-1 is positioned to the right of the second audio output device 110-2, the audio sample 206 reaches the second microphone 202-2 to the right of the second audio output device 110-2 before reaching the first microphone 202-1 to the left of the second audio output device 110-2. That is, the second detection time 210-2 occurs before the first detection time 210-1.
[0029] Based on the detection times 210-1, 210-2 and the emission time 208, the audio output device positioning engine 114 can determine the distance 212 between the first audio output device 110-1 and the second audio output device 110-2. In various implementations, the audio output device positioning engine 114 determines the distance 212 based on the following equation:
[0030]
[0031] in,
[0032] represents the distance between the first audio output device 110-1 and the second audio output device 110-2,
[0033] represents the difference between the first detection time 210 - 1 and the emission time 208 of the first microphone 202 - 1 ,
[0034] represents the difference between the second detection time 210-2 and the emission time 208 of the second microphone 202-2, and
[0035] Represents the speed of sound in air.
[0036] Furthermore, based on the detection times 210-1, 210-2 and the emission time 208, the audio output device positioning engine 114 may determine an angle 214 between the first audio output device 110-1 and the second audio output device 110-2 relative to the vector 216. In various implementations, the audio output device positioning engine 114 determines the angle 214 based on the following equation:
[0037]
[0038] in,
[0039] represents the angle between the first audio output device 110-1 and the second audio output device 110-2 relative to an outward vector orthogonal to a line between the first microphone 202-1 and the second microphone 202-2,
[0040] represents the distance between the first microphone 202-1 and the second microphone 202-2 of the first audio output device 110-1,
[0041] represents the difference between the first detection time 210 - 1 and the emission time 208 of the first microphone 202 - 1 ,
[0042] represents the difference between the second detection time 210-2 and the emission time 208 of the second microphone 202-2, and
[0043] Represents the speed of sound in air.
[0044] Although not shown, in various embodiments, the distance 212 and the angle 214 may be determined by the device 100 ( For example , audio output device positioning engine 114), first audio output device 110-1, second audio output device 110-2, and / or any other device capable of evaluating Equation 1 and Equation 2. In various embodiments, distance 212 and / or angle 214 may be determined according to equations other than Equation 1 and Equation 2. For example, in some embodiments, distance 212 and / or angle 214 may be determined in the absence of detected and / or recorded emission time 208, but based on the accumulated detection time 210 of two or more other audio output devices 110 in the plurality of audio output devices 110.
[0045] Although not shown, in various embodiments, the audio output device positioning engine 114 performs calibration, wherein each audio output device 110 of the plurality of audio output devices 110 transmits the audio sample 206 at a different time. Alternatively, although not shown, in various embodiments, at least two audio output devices of the plurality of audio output devices 110 transmit the audio sample 206 simultaneously, such as different audio output devices 110 transmitting the audio sample 206 at different frequencies at a given point in time, or different audio output devices 110 transmitting the audio sample 206 having audio scans over different frequency ranges, scan durations, and / or time periods.
[0046] Figure 3 According to various implementation schemes Figure 1 100 for a portion of a calibration process for a plurality of audio output devices. As shown, this portion of the calibration process includes determining a first position 308 of the plurality of audio output devices 110 within the physical space 112. In various embodiments, Figure 1 The audio output device positioning engine 114 performs this portion of the calibration process.
[0047] As shown, the plurality of audio output devices 110 are associated with a set of detection parameters 302. For each first audio output device 110 and each second audio output device 110, the set of detection parameters 302 includes a determination of the distance 212 between the first audio output device 110 and the second audio output device 110, and a determination of the angle 214 between the first audio output device 110 and the second audio output device 110 ( For example, relative to a vector 216, such as a north vector or other direction within the physical space 112). In various embodiments, the audio output device positioning engine 114 determines a distance 212 and / or angle 214 between each pair of audio output devices 110 based on the detection of each first audio output device 110-1 by each second audio output device 110-2, such as Figure 2 shown.
[0048] As shown, based on the detection parameters, the audio output device positioning engine 114 determines a first position 308 of each audio output device 110 within the first coordinate system 304. In various embodiments, the audio output device positioning engine 114 determines the first position 308 relative to a first origin 306 of the first coordinate system 304. For example, the first origin 306 can be based on the geometric center of the physical space 112 or the center of a rectangular boundary surrounding the plurality of audio output devices 110. In various embodiments, the audio output device positioning engine 114 arbitrarily selects the first origin 306 ( For example , any point inside or outside the polygon at the location of the audio output device 110 with a vertex located therein). Based on the first origin 306 and the detection parameters 302, the audio output device positioning engine 114 determines a first position 308 of each audio output device 110, such as a first coordinate within the first coordinate system 304 relative to the first origin 306. In various embodiments, the first position 308 is consistent with and / or proportional to the detection parameters 302. For example, the geometric distance between each pair of audio output devices 110 within the first coordinate system 304 is consistent with and / or proportional to the distance between each pair of audio output devices 110 within the physical space 112. As shown, the first coordinate system 304 is a Cartesian coordinate system, but various embodiments may include other types of first coordinate systems, such as a polar coordinate system or a spherical coordinate system.
[0049] Figure 4 According to various implementation schemes Figure 1 100 for a plurality of audio output devices 110. As shown, this portion of the calibration process includes segmenting 404 of the plurality of audio output devices 110 within the physical space 112. In various embodiments, Figure 1 The audio output device positioning engine 114 performs this portion of the calibration process.
[0050] As previously discussed, the audio output device positioning engine 114 determines a first position 308 within the first coordinate system 304 for each audio output device 110. In various embodiments, the audio output device positioning engine 114 determines the first position 308 of the audio output device 110 within the first coordinate system 304 and relative to the first origin 306, such as Figure 34. The determined first positions 308 form a polygon 402, wherein the first position 308 of each audio output device 110 corresponds to a vertex of the polygon 402. Based on the first positions 308 and the first origin 306, the audio output device positioning engine 114 performs segmentation 404 of the plurality of audio output devices 110 into a set of partitions 406, wherein each partition 406 is based on the first origin 306 and the first position 308 of a subset of the audio output devices 110 selected in a predetermined order. As shown, each partition 406 includes a triangle having a first vertex corresponding to the first origin 306, and two vertices corresponding to the respective first positions 308 of the first audio output device 110 and the second audio output device 110. As shown, for a given set of four audio output devices 110, the audio output device positioning engine 114 generates four zones 406 corresponding to the following audio output device pairs: (110-1, 110-2), (110-2, 110-3), (110-3, 110-4), and (110-4, 110-1). In various embodiments, the audio output device positioning engine 114 determines the zones 406 in different ways, such as triangles that respectively include vertices corresponding to the first positions 308 of the three audio output devices 110, or quadrilaterals that respectively include a first vertex corresponding to the first origin 306 and three vertices corresponding to the respective first positions 308 of the three audio output devices 110.
[0051] Figure 5 According to various implementation schemes Figure 1 504. In various embodiments, Figure 1 The audio output device positioning engine 114 performs this portion of the calibration process.
[0052] As previously discussed, the audio output device positioning engine 114 determines a zone 406 of one or more audio output devices 110. In various embodiments, the audio output device positioning engine 114 determines the zone 406 based on vertices of the plurality of audio output devices 110 within the first coordinate system 304 and relative to the first origin 306 of the first coordinate system 304, such as Figure 4 As shown. The audio output device positioning engine 114 evaluates each partition 406 to determine the area of each partition 406-1, 406-2, 406-3, 406-4. In various embodiments, when triangles are used, the audio output device positioning engine 114 determines the area of each partition 406 based on the following equation:
[0053]
[0054] in,
[0055] Indicates that a first audio output device is included and a second audio output device Polygon 402 The area of the partition,
[0056] Indicates the first audio output device The x-coordinate in the first coordinate system 304,
[0057] Indicates the first audio output device The y coordinate in the first coordinate system 304,
[0058] Indicates the second audio output device The x-coordinate in the first coordinate system 304, and
[0059] Indicates the second audio output device The y-coordinate in the first coordinate system 304 .
[0060] In various embodiments, the audio output device positioning engine 114 also determines the center of each zone 406 based on the following equation:
[0061]
[0062]
[0063] in,
[0064] Indicates that a first audio output device is included and a second audio output device of The x-coordinate of the center of the partition,
[0065] Indicates the first audio output device The x-coordinate in the first coordinate system 304,
[0066] Indicates the first audio output device The y coordinate in the first coordinate system 304,
[0067] express The y coordinate of the center of the partition,
[0068] Indicates the second audio output device +1 is the x-coordinate in the first coordinate system 304, and
[0069] Indicates the second audio output device The y-coordinate in the first coordinate system 304 .
[0070] The audio output device positioning engine 114 also determines the centroid 118 of the polygon 402 based on the area and center of the partitions 406. As shown, the audio output device positioning engine 114 determines the centroid 118 based on a weighted sum 502 of the partitions 406. The weighted sum 502 includes the sum of the product of the center of each partition 406 and the area of each partition 406. As shown, the device determines that the centroid 118 is located in the first coordinate system 304 In various implementations, the audio output device positioning engine 114 determines the centroid 118 of the polygon 402 based on the following equation:
[0071]
[0072]
[0073] in,
[0074] represents the x-coordinate of the center of mass 118 in the first coordinate system 304,
[0075] represents the number of audio output devices 110,
[0076] Represents polygon 402 The area of the partition,
[0077] express The x-coordinate of the center of the partition,
[0078] represents the y coordinate of the center of mass 118 within the first coordinate system 304, and
[0079] express The y coordinate of the center of the partition.
[0080] Based on the centroid 118, the audio output device positioning engine 114 determines a second position 506 of each audio output device 110 within the second coordinate system 504. In various embodiments, the second coordinate system 504 may be of the same or similar type as the first coordinate system 304. As shown, the second coordinate system 504 is another Cartesian coordinate system in which the first coordinate system 304 is offset by the difference between the coordinates of the first origin 306 and the coordinates of the centroid 118. That is, the second coordinate system 504 is the first coordinate system 304 with the origin translated to the centroid 118. As shown, the audio output device positioning engine 114 also determines each second position 506 within the second coordinate system 504 by subtracting the difference between the coordinates of the first origin 306 within the first coordinate system 304 and the coordinates of the centroid 118 within the first coordinate system 304 from the coordinates of each corresponding first position 308. Although not shown, in various embodiments, the second coordinate system 504 may be of a different type than the first coordinate system 304. For example, the first coordinate system 304 may be a Cartesian coordinate system, and the second coordinate system 504 may be a polar coordinate system or a spherical coordinate system.
[0081] In various embodiments, the audio output device positioning engine 114 determines the second position of each audio output device 110 within the second coordinate system 504 based on the following equation:
[0082]
[0083]
[0084] in,
[0085] Indicates the audio output device The x-coordinate in the second coordinate system 504,
[0086] Indicates the audio output device The x-coordinate in the first coordinate system 304,
[0087] represents the x-coordinate of the center of mass 118 in the first coordinate system 304,
[0088] Indicates the audio output device The y coordinate in the second coordinate system 504,
[0089] Indicates the audio output device y coordinate in the first coordinate system 304, and
[0090] represents the y-coordinate of the center of mass 118 in the first coordinate system 304 .
[0091] As shown, the audio output device positioning engine 114 determines an acoustic impulse response 508 for each audio output device 110. In various embodiments, each acoustic impulse response 508 includes a transfer function that indicates how the audio output device 110 changes the audio emitted by the audio object 120 at various locations. Applying the acoustic impulse response 508 to the representation of the audio object 120 transforms the way each frequency emitted by the audio object 120 is perceived when emitted from, transmitted through, and / or reflected from a second location 506 of the audio output device 110. Thus, the acoustic impulse response 508 determined for each audio output device 110 changes the output of the audio object 120 by the audio output device 110 so that a listener located within the physical space 112 perceives the current location of the audio object 120. In various embodiments, the audio output device positioning engine 114 determines an acoustic impulse response 508 of the audio output device 110 based on a second position 506 of each audio output device 110 within the second coordinate system 504 and an acoustic model (such as (but not limited to) a point source acoustic model, a plane wave acoustic model, etc.). The audio output device positioning engine 114 may store ( For example , stored in the memory 104 or the storage device 106 ) for each audio output device 110 , for use by the audio object rendering engine 122 when rendering the audio object 120 .
[0092] Figure 6 According to various embodiments, multiple audio output devices 110 are connected through Figure 1 The apparatus 100 renders a graphical representation of an audio object. In various embodiments, Figure 1 The audio object rendering engine 122 performs rendering.
[0093] In various embodiments, the audio object 120 follows a trajectory 602 of the plurality of audio output devices 110 within the physical space 112. As shown, the trajectory 602 of the audio object 120 begins at a first position near the first audio output device 110-1, follows a curved path from the first position near the first audio output device 110-1 to a second position near the second audio output device 110-2, and ends at the second position near the second audio output device 110-2. As a result, at each point in time, the audio object 120 is located at a current position 604 along the trajectory. Therefore, at each point in time, the audio object rendering engine 122 sends an audio output device signal 124 to each audio output device 110, wherein the audio object 120 is adjusted based on the current position 604 of the audio object 120 relative to the audio output device position 116 of the audio output device 110. For example, at a first point in time, the audio object rendering engine 122 includes most of the sound from the audio object 120 in the audio output device signal 124 for the first audio output device 110-1. At a second point in time, the audio object rendering engine 122 includes a first portion of the audio object 120 in a first audio output device signal 124-1 for the first audio output device 110-1, and includes a second portion of the audio object 120 in a second audio output device signal 124-2 for the second audio output device 110-2. At a third point in time, the audio object rendering engine 122 includes a majority of the sound from the audio object 120 in the audio output device signal 124-2 for the second audio output device 110-2. At each point in time, each audio output device 110 outputs audio of the audio object 120 as if originating from the location of the audio object 120 within the physical space 112 and reflected from the audio output device 110 before reaching the user.
[0094] In various embodiments, each audio object 120 includes an audio object representation 606 of the audio object 120. For example, each audio object representation 606 may include an audio sample of a sound emitted by the audio object 120. Each audio object representation 606 may also include a source description of the trajectory 602, such as a set of coordinates within the first coordinate system 304 that indicate a current position 604 of the audio object 120 at various points in time. The audio object rendering engine 122 may determine a set of second positions 506 within the second coordinate system 504 for the audio object 120. For example, the audio object rendering engine 122 may subtract the difference between the first origin 306 within the first coordinate system 304 and the center of mass 118 within the first coordinate system 304 from each first coordinate of the source description within the first coordinate system 304, thereby generating the second positions 506 of the source description within the second coordinate system 504. As shown, each audio output device 110 outputs audio of the audio object 120 as if originating from the location of the audio object 120 in the physical space 112 (along a first line 610 ) and reflecting from the audio output device 110 (along a second line 612 ) before reaching the centroid 118 .
[0095] In various embodiments, the audio object rendering engine 122 generates an audio output device signal 124 for each audio output device 110 based on the acoustic impulse response 508 of the audio output device 110 and the audio object representation 606 of the audio object 120. As shown, the audio object rendering engine 122 performs a convolution operation 608 between the audio object representation 606 and the acoustic impulse response 508 of each audio output device 110. In various embodiments, for each audio object 120, the audio object rendering engine 122 performs the convolution operation 608 based on the following equation:
[0096]
[0097] in,
[0098] represents an audio output device among the plurality of audio output devices 110 ,
[0099] Indicates a point in time,
[0100] Indicates the audio output device an audio output device signal 124 for an audio object,
[0101] Indicates the audio output device The acoustic impulse response 508,
[0102] an audio object representation 606 representing the audio object 120, and
[0103] Indicates the audio output device A convolution operation 608 is performed between the acoustic impulse response 508 and the audio object representation 606 of the audio object 120 .
[0104] Each convolution operation 608 adjusts the audio object representation 606 of the audio object 120 to render the sound as if it originated from the position of the audio object 120 within the physical space 112 (along the first line 610) and reflected from the audio output device 110 before reaching the user (along the second line 612). For example, a first convolution operation 608-1 of the audio object representation 606 with the acoustic impulse response 508-1 of the first audio output device 110 causes the audio emitted by the audio object 120 to be perceived as if it was emitted from the current position 604 of the audio object 120, propagated through the air along the first line 610-1 to the second position 506 of the first audio output device 110-1, reflected from the first audio output device 110-1, and propagated downward along the second line 612-1 to the centroid 118. Similarly, other convolution operations 608-2, 608-3, 608-4 of the audio object representation 606 with the acoustic impulse responses 508-2, 508-3, 508-4 of the other audio output devices 110-2, 110-3, 110-4 cause the audio object 120 to be perceived as if emitted from the current position 604 of the audio object 120, propagated through the air along another first line 610-2, 610-3, 610-4 to a second position 506 of one of the other audio output devices 110-2, 110-3, 110-4, reflected from one of the other audio output devices 110-2, 110-3, 110-4, and propagated downward along another second line 612-2, 612-3, 612-4 to the centroid 118. A listener within the physical space 112 perceives the combined audio output of each audio output device 110 , and therefore the audio object 120 , as if it were located at the current position 604 of the track 602 .
[0105] Furthermore, in various embodiments, the audio object rendering engine 122 generates an audio output device signal 124 for each audio output device 110 for the plurality of audio objects 120. For example, each audio output device 110 may simultaneously output rendered audio for each of the plurality of audio objects 120. In various embodiments, the audio object rendering engine 122 generates a component of the audio output device signal 124 for each audio object 120. , and the components of each audio object in the audio object 120 The collection of For example, sum) is sent to the audio output device 110. In various embodiments, for each audio object 120, the audio object rendering engine 122 performs a convolution operation 608 based on the following equation:
[0106]
[0107] in,
[0108] represents an audio output device among the plurality of audio output devices 110 ,
[0109] Indicates a point in time,
[0110] Indicates the audio output device an audio output device signal 124 for the plurality of audio objects 120,
[0111] Indicates the audio output device The acoustic impulse response 508,
[0112] represents the number of audio objects 120,
[0113] express The audio object in the audio object 120 The audio object representation 606, and
[0114] Indicates the audio output device The acoustic impulse response 508 with The audio object in the audio object 120 The audio object represents the convolution operation between 606.
[0115] As shown, the audio object rendering engine 122 sends an audio output device signal 124 generated for the audio output device 110 based on the current position 604 and / or trajectory 602 of each audio object in the one or more audio objects 120 to each audio output device 110. In various embodiments, the audio object rendering engine 122 transmits the audio output device signal 124 generated for the audio output device 110 based on the current position 604 and / or trajectory 602 of each audio object in the one or more audio objects 120. For example , two-wire audio speaker cable) or via wireless signal ( For example , Bluetooth signal, wireless network signal) to send audio output device signal 124. Each audio output device 110 outputs a corresponding audio output device signal 124. As a result, device 100 renders one or more audio objects 120, which the listener perceives as originating from one or more current positions 604 along track 602.
[0116] Figure 7 A method for calibrating a Figure 1 A flowchart of the steps of a method for multiple audio output devices. For example, Figure 7 The method steps can be For example Depend on Figure 1 The audio output device positioning engine 114 is used. Figure 7 The method steps are about Figure 1 The device 100 and Figures 2 to 6 The techniques shown are described herein, but numerous systems configured to perform the method steps in any order are contemplated as falling within the scope of the various embodiments.
[0117] As shown, method 700 begins with step 702 performed for each audio output device in a plurality of audio output devices. At step 704, the audio output device positioning engine causes the audio output device to output an audio sample. For example, the audio output device may output a tone of a given frequency, a frequency sweep on a portion of a human audible frequency range and / or a human inaudible frequency range, white noise or pink noise, etc. At step 706, the audio output device positioning engine determines the detection time of each microphone of two or more microphones included in the other audio output devices for each other audio output device in the plurality of audio output devices. For example, each audio output device may include a left microphone and a right microphone, and the audio output device positioning engine may determine the first detection time of the audio sample by the left microphone of each audio output device and the second detection time of the audio sample by the right microphone of each audio output device. Detection may include determining the distance between the audio output device outputting the audio sample and each other audio output device in the plurality of audio output devices. Detection may include determining the angle of the audio output device outputting the audio sample relative to each other audio output device in the plurality of audio output devices relative to an outward vector orthogonal to a line connecting the microphones of the other audio output devices. In various embodiments, the audio output device positioning engine can determine the distance and / or angle between the corresponding audio output device pairs, as described above with respect to Figure 2 The implementation scheme described.
[0118] At step 708, based on the detection time of each of the two or more microphones of each of the audio output devices for each of the audio samples, the audio output device positioning engine determines a first position of each audio output device relative to a first origin in a first coordinate system. For example, the first origin may be based on a geometric center of the physical space or a center of a rectangular boundary surrounding a plurality of audio output devices, or any point in the physical space. The determination may include determining the coordinates of each audio output device relative to the first origin in the first coordinate system. In various embodiments, the audio output device positioning engine may determine the first position of each audio output device, as described above with respect to Figure 3 as described in the implementation scheme of FIG.
[0119] At step 710, the audio output device positioning engine determines the center of mass of the audio output device. For example, the center of mass can be the center of mass of a polygon that includes vertices based on the first position of the audio output device in the first coordinate system. The audio output device positioning engine can determine the center of mass based on a weighted sum of partitions of the polygon. In various embodiments, the audio output device positioning engine can determine the polygon and the partitions of the polygon, as described above with respect to Figure 4 In various embodiments, the audio output device positioning engine can determine the center of mass of the polygon in the second coordinate system, as described above with respect to Figure 5 as described in the implementation scheme of FIG.
[0120] At step 712, the audio output device positioning engine determines a second position of each audio output device in the second coordinate system based on the centroid. For example, the audio output device positioning engine may subtract the difference between the first origin of the first coordinate system and the determined centroid of the polygon in the first coordinate system from the first position of each audio output device in the first coordinate system. The second position indicates the position of the audio output device in the second coordinate system relative to the centroid. In various embodiments, the audio output device positioning engine may determine the second position of the audio output device in the second coordinate system, as described above with respect to Figure 5 The implementation scheme described.
[0121] At step 714, the audio output device positioning engine determines the acoustic impulse response of each audio output device based on the second position of each audio output device in the second coordinate system and the acoustic model. For example, the audio output device positioning engine may determine the acoustic impulse response of each audio output device based on a point source acoustic model or a plane wave acoustic model. The audio output device positioning engine may store the acoustic impulse response of each audio output device in a memory and / or storage device of the device for use by the audio object rendering engine to render the audio object. In various embodiments, the audio output device positioning engine may determine the acoustic impulse response of each audio output device, as described above with respect to Figure 5 The implementation scheme described.
[0122] Figure 8 Shown according to various embodiments Figure 1 A flowchart of method steps for outputting audio objects to multiple audio output devices. Figure 8 The method steps can be For example Depend on Figure 1 The audio object rendering engine 122 is applied. Although Figure 8 The method steps are about Figure 1 Although described with reference to apparatus 100, many systems configured to perform the method steps in any order are within the scope of the various embodiments.
[0123] As shown, method 800 begins at step 802, where an audio object rendering engine receives an audio object representation of an audio object. For example, the audio object rendering engine may retrieve the audio object representation from a memory and / or storage device of the device, or may receive the audio object representation from another device. The audio object representation may include a source description of the audio object, including a set of one or more first positions of the audio object in a first coordinate system. In various embodiments, the audio output rendering engine receives the audio object representation of multiple audio objects.
[0124] At step 804, the audio object rendering engine performs the steps for each audio output device in a plurality of audio output devices. At step 806, the audio object rendering engine applies a convolution operation to the acoustic impulse response and the audio object representation of the audio object to generate an audio output device signal for the audio output device. For the audio output device, the audio output device signal includes a rendering of audio sent from the current position of the audio object and reflected from a second position of the audio output device in a second coordinate system. In various embodiments, the audio output rendering engine applies a convolution operation to each audio object representation of the plurality of audio objects, applies a convolution operation to each audio object representation in the acoustic impulse response and the audio object representation of the audio output device, and generates an audio output device signal including the sum of the outputs of the convolution operations. At step 808, the audio object rendering engine sends the audio output device signal to the audio output device. The audio object rendering engine may return to step 802 to receive additional audio objects for further rendering by the plurality of audio output devices. In various embodiments, the audio output device positioning engine may render the audio object and cause the audio output device to output spatial audio including the audio object, as described above with respect to Figure 6 The implementation scheme described.
[0125] In summary, the technology for generating audio includes causing each of a plurality of audio output devices to output an audio sample. When each audio output device outputs the audio sample, a detection time of the audio sample is determined for each other of the plurality of audio output devices, and specifically, the detection time of the audio sample is determined by each of two or more microphones included in the other audio output devices. Based on the detection time of each of the audio samples by each of the audio output devices, a distance and angle of each audio output device relative to each other audio output device is determined.
[0126] Each audio output device outputs one or more audio objects. The output of each audio object by each audio output device is based on the position of the audio output device relative to other audio output devices in a second coordinate system. Basing the output of each audio object on the position of each audio output device relative to other audio output devices can improve the positioning of the audio object within the physical space of the audio output devices.
[0127] At least one technical advantage of the disclosed technology over the prior art is that, using the disclosed technology, the output of each audio object by each audio output device is based on the position of the audio output device relative to other audio output devices. As a result, the positioning and / or trajectory of each audio object is more accurately rendered by the audio output device based on their position in physical space. In addition, the disclosed calibration technology can determine the position of each audio output device relative to other audio output devices, including determining when the positions of two audio output devices are reversed. In addition, the disclosed calibration technology automatically and accurately determines the position of the audio output device compared to user-based calibration setting adjustments. These technical advantages provide one or more technical improvements over prior art methods.
[0128] 1. In some embodiments, a computer-implemented method for generating audio includes: causing each of a plurality of audio output devices to output an audio sample; for each other of the plurality of audio output devices, determining a detection time of the audio sample from each audio output device by each of two or more microphones included in the other audio output devices; determining a position of each audio output device relative to the other audio output devices based on the detection time of each of the audio samples by each of the audio output devices; and causing each of the plurality of audio output devices to generate an audio output associated with an audio object, wherein the output of each of the audio output devices is based on the position of the audio object and the position of each audio output device relative to the other audio output devices.
[0129] 2. A computer-implemented method as described in clause 1, wherein determining the position of a first audio output device among the multiple audio output devices relative to the audio output device that outputs the audio sample comprises: determining a difference between a first detection time of the audio sample by a first microphone of the first audio output device and a second detection time of the audio sample by a second microphone of the first audio output device, and based on the difference, determining an angle between the first audio output device and the audio output device that outputs the audio sample.
[0130] 3. A computer-implemented method as described in clause 1 or 2, wherein determining the position of a first audio output device among the multiple audio output devices relative to the audio output device that outputs the audio sample comprises: determining a difference between an emission time of the audio sample and a detection time of the audio sample by at least one microphone of the two or more microphones of the first audio output device, and based on the difference, determining a distance between the first audio output device and the audio output device that outputs the audio sample.
[0131] 4. A computer-implemented method as described in any of clauses 1 to 3, wherein determining the position of each audio output device relative to the other audio output devices comprises: determining the center of mass of the multiple audio output devices in a first coordinate system, and determining the position of each audio output device relative to the center of mass.
[0132] 5. A computer-implemented method as described in any of clauses 1 to 4, wherein determining the position of each audio output device relative to the other audio output devices includes: for one or more pairs of audio output devices among the plurality of audio output devices, determining a triangle that includes an origin of the first coordinate system and a first position of each audio output device in the pair in the first coordinate system, and determining the center of mass of the plurality of audio output devices based on a weighted sum of the centers of each triangle, wherein the weighted sum is based on the area of the triangle.
[0133] 6. A computer-implemented method as described in any of clauses 1 to 5, further comprising: determining a second position of each audio output device in a second coordinate system centered on the centroid of the positions of the plurality of audio output devices, and determining one or more positions of a source description of an audio object in the second coordinate system.
[0134] 7. The computer-implemented method of any of clauses 1 to 6, further comprising determining an acoustic impulse response for each audio output device based on an acoustic model, the acoustic model comprising at least one of a point source acoustic model or a plane wave acoustic model.
[0135] 8. The computer-implemented method of any of clauses 1 to 7, further comprising determining an acoustic impulse response of each audio output device based on a position of each audio output device within the second coordinate system.
[0136] 9. A computer-implemented method as described in any of clauses 1 to 8, wherein generating the audio output for each audio output device also includes applying a convolution operation to the audio representation of the audio object and the acoustic impulse response of the audio output device to generate an audio output device signal for output by the audio output device.
[0137] 10. A computer-implemented method as described in any of clauses 1 to 9, further comprising: combining the audio output of a first audio output device among the multiple audio output devices with a second audio output of a second audio object, wherein the second audio output is based on a position of the second audio object and the position of each audio output device relative to the other audio output devices; and causing the first audio output device to output the combined audio output.
[0138] 11. In some embodiments, a non-transitory computer-readable medium stores instructions that, when executed by a processor, cause the processor to perform the following steps: cause each of a plurality of audio output devices to output an audio sample; for each other of the plurality of audio output devices, determine a detection time of the audio sample from each audio output device by each of two or more microphones included in the other audio output devices; determine a position of each audio output device relative to the other audio output devices based on the detection time of each of the audio samples by each of the audio output devices; and cause each of the plurality of audio output devices to generate an audio output including an audio object, wherein the audio output of each audio output device is based on the position of the audio object and the position of each audio output device relative to the other audio output devices.
[0139] 12. The non-transitory computer-readable medium of clause 11, wherein determining the position of a first audio output device among the plurality of audio output devices relative to the audio output device that outputs the audio sample comprises determining a difference between a first detection time of the audio sample by a first microphone of the first audio output device and a second detection time of the audio sample by a second microphone of the first audio output device, and determining an angle between the first audio output device and the audio output device that outputs the audio sample based on the difference.
[0140] 13. The non-transitory computer-readable medium of clause 11 or 12, wherein determining the position of a first audio output device among the plurality of audio output devices relative to the audio output device that outputs the audio sample comprises determining a difference between an emission time of the audio sample and a detection time of the audio sample by at least one of the two or more microphones of the first audio output device, and determining a distance between the first audio output device and the audio output device that outputs the audio sample based on the difference.
[0141] 14. A non-transitory computer-readable medium as described in any of clauses 11 to 13, wherein determining the position of each audio output device relative to the other audio output devices includes: for one or more pairs of the plurality of audio output devices, determining a triangle including an origin of the first coordinate system and a first position of each audio output device in the pair in the first coordinate system, and determining the center of mass of the plurality of audio output devices based on a weighted sum of the centers of each triangle, wherein the weighted sum is based on the area of the triangle.
[0142] 15. A non-transitory computer-readable medium as described in any of clauses 11 to 14, further comprising: determining a second position of each audio output device in a second coordinate system centered on the centroid of the positions of the plurality of audio output devices, and determining one or more positions of a source description of an audio object in the second coordinate system.
[0143] 16. The non-transitory computer-readable medium of any one of clauses 11 to 15, wherein the step further comprises determining an acoustic impulse response of each audio output device based on an acoustic model, the acoustic model comprising at least one of a point source acoustic model or a plane wave acoustic model.
[0144] 17. The non-transitory computer-readable medium of any of clauses 11 to 16, wherein the steps further comprise determining an acoustic impulse response of each audio output device based on the position of each audio output device within the second coordinate system.
[0145] 18. A non-transitory computer-readable medium as described in any of clauses 11 to 17, wherein generating the audio output for each audio output device also includes applying a convolution operation to the audio representation of the audio object and the acoustic impulse response of the audio output device to generate an audio output device signal for output by the audio output device.
[0146] 19. In some embodiments, a system includes: a memory storing instructions; and one or more processors executing the instructions to perform steps including: causing each of a plurality of audio output devices to output an audio sample; for each other of the plurality of audio output devices, determining a detection time of the audio sample from each audio output device by each of two or more microphones included in the other audio output devices; determining a position of each audio output device relative to the other audio output devices based on the detection time of each of the audio samples by each of the audio output devices; and causing each of the plurality of audio output devices to generate an audio output associated with an audio object, wherein the output of each of the audio output devices is based on the position of the audio object and the position of each audio output device relative to the other audio output devices.
[0147] 20. The system of clause 19, wherein causing each of the plurality of audio output devices to generate the audio output further comprises determining an acoustic impulse response for each audio output device, and applying a convolution operation to the audio representation of the audio object and the acoustic impulse response to generate, via the audio output device, an audio output device signal comprising the audio object.
[0148] Any and all combinations of any of the claim elements as described in any of the claims and / or any of the elements described in this application, in any form, are within the intended scope of the invention and protection.
[0149] Descriptions of various embodiments have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments.
[0150] Aspects of the present embodiment may be embodied as a system, method or computer program product. Therefore, aspects of the present disclosure may take the following forms: a complete hardware implementation, a complete software implementation (including firmware, resident software, microcode, etc.) or an implementation combining software aspects and hardware aspects, which may generally be referred to as a "module", "system" or "computer" in this article. In addition, any hardware and / or software technology, process, function, component, engine, module or system described in the present disclosure may be implemented as a circuit or a collection of circuits. In addition, aspects of the present disclosure may take the form of a computer program product, which is embodied in one or more computer-readable media on which a computer-readable program code is embodied.
[0151] Any combination of one or more computer-readable media may be utilized. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer-readable storage media would include the following media: an electrical connection with one or more wires, a portable computer floppy disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer-readable storage medium may be any tangible medium that may contain or store a program for use by or in connection with an instruction execution system, device, or apparatus.
[0152] Aspects of the present disclosure are described above with reference to the flowchart illustration and / or block diagram of the method, device (system) and computer program product according to the embodiments of the present disclosure.It should be understood that each frame in the flowchart illustration and / or block diagram and the frame combination in the flowchart illustration and / or block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine.These instructions enable the function / action specified in one or more frames of the flowchart and / or block diagram to be realized when executed via the processor of a computer or other programmable data processing device.Such processors can be, but are not limited to, general-purpose processors, special-purpose processors or field programmable gate arrays.
[0153] The flow charts and block diagrams in the accompanying drawings illustrate the architecture, functionality and operation of possible implementations of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a module, section or part of a code, and the code includes one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions proposed in the box may not occur in the order proposed in the accompanying drawings. For example, in fact, depending on the functions involved, the two boxes shown in succession can be executed substantially simultaneously, or sometimes these boxes can be executed in reverse order. It should also be noted that each box illustrated in the block diagram and / or flow chart and the combination of boxes in the block diagram and / or flow chart can be implemented by a system based on special-purpose hardware that performs a specified function or action or by a combination of special-purpose hardware and computer instructions.
[0154] While the foregoing is directed to embodiments of the present disclosure, other and further embodiments of the present disclosure may be devised without departing from the basic scope thereof, the scope of which is determined by the following claims.
Claims
1. A computer-implemented method for generating audio, the method comprising: include: causing each audio output device of a plurality of audio output devices to output an audio sample; for each other audio output device of the plurality of audio output devices, determining a detection time of the audio sample from each audio output device by each microphone of two or more microphones included in the other audio output device; determining a position of each of the audio output devices relative to the other audio output devices based on a detection time of each of the audio samples by each of the audio output devices; as well as Each of the plurality of audio output devices is caused to generate an audio output associated with an audio object, wherein the output of each of the audio output devices is based on a position of the audio object and the position of each audio output device relative to the other audio output devices.
2. The computer-implemented method of claim 1 , wherein determining the position of a first audio output device of the plurality of audio output devices relative to the audio output device outputting the audio sample include: determining a difference between a first detection time of the audio sample by a first microphone of the first audio output device and a second detection time of the audio sample by a second microphone of the first audio output device, and Based on the difference, an angle between the first audio output device and the audio output device outputting the audio sample is determined.
3. The computer-implemented method of claim 1 , wherein determining the position of a first audio output device of the plurality of audio output devices relative to the audio output device outputting the audio sample include: determining a difference between a time at which the audio sample was emitted and a time at which the audio sample was detected by at least one of the two or more microphones of the first audio output device, and Based on the difference, a distance between the first audio output device and the audio output device that outputs the audio sample is determined.
4. The computer-implemented method of claim 1 , wherein determining the position of each audio output device relative to the other audio output devices include: determining the centroids of the plurality of audio output devices in a first coordinate system, and The position of each audio output device relative to the center of mass is determined.
5. The computer-implemented method of claim 1 , wherein determining the position of each audio output device relative to the other audio output devices include: For one or more pairs of audio output devices of the plurality of audio output devices, determining a triangle including an origin of a first coordinate system and a first position of each audio output device in the pair in the first coordinate system, and A center of mass of the plurality of audio output devices is determined based on a weighted sum of the centers of each triangle, wherein the weighted sum is based on an area of the triangle.
6. The computer-implemented method of claim 1, further comprising: include: determining a second position of each audio output device within a second coordinate system centered about the centroid of the positions of the plurality of audio output devices, and One or more locations of a source description of the audio object in the second coordinate system are determined.
7. The computer-implemented method of claim 1, further comprising determining an acoustic impulse response for each audio output device based on an acoustic model, the acoustic model comprising at least one of a point source acoustic model or a plane wave acoustic model.
8. The computer-implemented method of claim 1, further comprising: include: An acoustic impulse response is determined for each audio output device based on the position of each audio output device within the second coordinate system.
9. The computer-implemented method of claim 8, wherein generating the audio output for each audio output device further comprises applying a convolution operation to the audio representation of the audio object and the acoustic impulse response of the audio output device to generate an audio output device signal for output by the audio output device.
10. The computer-implemented method of claim 1, further comprising: include: combining an audio output of a first audio output device of the plurality of audio output devices with a second audio output of a second audio object, wherein the second audio output is based on a position of the second audio object and the position of each audio output device relative to the other audio output devices; as well as The first audio output device is caused to output a combined audio output.
11. A non-transitory computer readable medium storing instructions that, when executed by a processor, cause the processor to perform the following steps: causing each audio output device of a plurality of audio output devices to output an audio sample; for each other audio output device of the plurality of audio output devices, determining a detection time of the audio sample from each audio output device by each microphone of two or more microphones included in the other audio output device; determining a position of each of the audio output devices relative to the other audio output devices based on a detection time of each of the audio samples by each of the audio output devices; as well as Each audio output device of the plurality of audio output devices is caused to generate an audio output comprising an audio object, wherein the audio output of each audio output device is based on a position of the audio object and the position of each audio output device relative to the other audio output devices.
12. The non-transitory computer-readable medium of claim 11, wherein determining the position of a first audio output device of the plurality of audio output devices relative to the audio output device outputting the audio sample include: determining a difference between a first detection time of the audio sample by a first microphone of the first audio output device and a second detection time of the audio sample by a second microphone of the first audio output device, and Based on the difference, an angle between the first audio output device and the audio output device outputting the audio sample is determined.
13. The non-transitory computer-readable medium of claim 11, wherein determining the position of a first audio output device of the plurality of audio output devices relative to the audio output device outputting the audio sample include: determining a difference between a time at which the audio sample was emitted and a time at which the audio sample was detected by at least one of the two or more microphones of the first audio output device, and Based on the difference, a distance between the first audio output device and the audio output device that outputs the audio sample is determined.
14. The non-transitory computer readable medium of claim 11, wherein determining the position of each audio output device relative to the other audio output devices include: For one or more pairs of audio output devices of the plurality of audio output devices, determining a triangle including an origin of the first coordinate system and a first position of each audio output device in the pair in the first coordinate system, determining a center of mass of the plurality of audio output devices based on a weighted sum of the centers of each triangle, wherein the weighted sum is based on an area of the triangle, and The position of each audio output device relative to the center of mass is determined.
15. The non-transitory computer readable medium of claim 11, wherein the step further include: determining a second position of each audio output device within a second coordinate system centered about the centroid of the positions of the plurality of audio output devices, and One or more locations of a source description of the audio object in the second coordinate system are determined.
16. The non-transitory computer readable medium of claim 11, wherein the step further comprises determining an acoustic impulse response for each audio output device based on an acoustic model, the acoustic model comprising at least one of a point source acoustic model or a plane wave acoustic model.
17. The non-transitory computer readable medium of claim 11, wherein the steps further comprise determining an acoustic impulse response for each audio output device based on the position of each audio output device within the second coordinate system.
18. The non-transitory computer-readable medium of claim 17, wherein generating the audio output for each audio output device further comprises applying a convolution operation to the audio representation of the audio object and the acoustic impulse response of the audio output device to generate an audio output device signal for output by the audio output device.
19. A system, wherein include: a memory storing instructions; as well as One or more processors that execute instructions to perform steps including: causing each audio output device of a plurality of audio output devices to output an audio sample; for each other audio output device of the plurality of audio output devices, determining a detection time of the audio sample from each audio output device by each microphone of two or more microphones included in the other audio output device; determining a position of each of the audio output devices relative to the other audio output devices based on a detection time of each of the audio samples by each of the audio output devices; as well as Each of the plurality of audio output devices is caused to generate an audio output associated with an audio object, wherein the output of each of the audio output devices is based on a position of the audio object and the position of each audio output device relative to the other audio output devices.
20. The system of claim 19, wherein causing each of the plurality of audio output devices to generate the audio output further comprises: include: determining the acoustic impulse response of each audio output device, and A convolution operation is applied to the audio representation of the audio object and the acoustic impulse response to generate, via the audio output device, an audio output device signal including the audio object.
