Method and system for spatial audio measurement using extended reality devices

Abstract

Provided herein are system, method, and computer program product embodiments for providing an interactive visual representation of loudspeaker sound coverage in a venue. The embodiments enable temporal, spectral, and spatial audio measurements from digital audio to venue acoustics. The technology can use light to visualize the audio capabilities of one or more loudspeaker arrays and provide a visual grouping of large audio channel outputs. This can allow spatial information to be visually represented by mapping directional sound onto light beams.

Description

[Background technology]

[0001] (CROSS-REFERENCE TO RELATED APPLICATIONS) This application claims priority to U.S. Patent Application No. 18 / 114,089, filed February 24, 2023, which is incorporated herein by reference in its entirety.

[0002] Traditional spatial audio measurement methods for large venues can become more time-consuming and logistically demanding as new spatial audio technologies enable more control options for directional sound. For example, spatial audio measurement in large venues with high channel counts can encounter numerous obstacles to configuring directional sound output from a single location solely by human ear perception. These obstacles arise from current sound production workflows and can be categorized into three groups: temporal measurements (e.g., sound pressure), spectral measurements (e.g., frequency spectrum), and spatial measurements (e.g., coverage). An obstacle for temporal measurements is the lack of confidence in sound pressure across the probe field, studio, and headphones. An obstacle for spectral measurements is the lack of tools for matching spectral content to the output capabilities of loudspeakers or loudspeaker arrays. An obstacle for spatial measurements is the lack of clear visual aids when authoring spatial audio content, adjusting studio-to-venue transformations, and calibrating large venue playback systems. As a result, those skilled in the art face challenges when configuring audio systems and measuring audio performance on-site, particularly during the planning or construction phase of a venue. Summary of the Invention [Means for solving the problem]

[0003] overview Provided herein are system, apparatus, device, method, and / or computer program product embodiments, and / or combinations and subcombinations thereof, for providing an interactive visual representation of loudspeaker sound coverage in a venue. In some embodiments, the techniques described herein can enable temporal, spectral, and spatial audio measurements from digital audio to venue acoustics. The techniques can use light to visualize the audio capabilities of one or more loudspeaker arrays and provide a visual grouping of large audio channel outputs. This can allow spatial information to be visually represented by mapping directional sound to light beams. [Brief explanation of the drawings]

[0004] [Figure 1] FIG. 1 illustrates a graphical representation of an exemplary venue, according to some embodiments.

[0005] [Figure 2A] 2A, 2B, and 3 illustrate example visualizations of a spatial audio measurement system using a mixed reality device for configuration operations, according to some embodiments. [Figure 2B] 2A, 2B, and 3 illustrate example visualizations of a spatial audio measurement system using a mixed reality device for configuration operations, according to some embodiments. [Figure 3] 2A, 2B, and 3 illustrate example visualizations of a spatial audio measurement system using a mixed reality device for configuration operations, according to some embodiments.

[0006] [Figure 4] 4-5 illustrate example visualizations of a spatial audio measurement system that uses a mixed reality device for measurement operations, according to some embodiments. [Figure 5]4-5 illustrate example visualizations of a spatial audio measurement system that uses a mixed reality device for measurement operations, according to some embodiments.

[0007] [Figure 6] FIG. 6 illustrates an exemplary mixed reality user device, according to some embodiments.

[0008] [Figure 7] FIG. 7 illustrates a flowchart of an exemplary method for operating a spatial audio measurement system using an extended reality device, according to some embodiments.

[0009] [Figure 8] FIG. 8 illustrates a simplified block diagram of a computer system for use with the preferred embodiments described herein, according to some exemplary embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0010] Detailed Description of the Invention The following disclosure provides many different embodiments or examples for implementing different features of the provided subject matter. Specific examples of components and arrangements are described below to simplify the disclosure. These are, of course, examples only and are not intended to be limiting. In addition, the disclosure may repeat reference numerals and / or letters in the various examples. This repetition does not, in itself, dictate a relationship between the various embodiments and / or configurations discussed.

[0011] Example Venues FIG. 1 illustrates a graphical representation of an exemplary venue, according to some exemplary embodiments of the present disclosure. In some embodiments, venue 100 represents a location for hosting an event. For example, venue 100 can represent a music venue, e.g., a music theater, music club, and / or concert hall; a sports venue, e.g., an arena, convention center, and / or stadium; and / or any other suitable venue as would be apparent to one skilled in the art without departing from the spirit and scope of the present disclosure. Events can include music events, theatrical events, sporting events, motion pictures, and / or any other suitable events as would be apparent to one skilled in the art without departing from the spirit and scope of the present disclosure. In some embodiments, venue 100 can represent a three-dimensional structure, e.g., a hemispherical structure, also referred to as a hemispherical dome.

[0012] 1, the venue 100 may further include loudspeakers 102.1-102.i for reproducing audio, i.e., sound, associated with the event. In some embodiments, the loudspeakers 102.1-102.i may include a stage entrance array loudspeaker system located at or near a stage entrance of the venue 100, one or more sound effects extension array loudspeaker systems located at or near the stage entrance array loudspeaker system, and / or one or more ambient array loudspeaker systems located throughout the venue 100. In some embodiments, the stage entrance array loudspeaker system, the one or more sound effects extension array loudspeaker systems, and / or the one or more ambient array loudspeaker systems may include one or more line arrays of loudspeakers, which may include one or more super tweeters, one or more tweeters, one or more mid-range speakers, one or more woofers, one or more subwoofers, and / or one or more full-range speakers, to name a few. In these embodiments, the one or more line arrays of loudspeakers may be implemented using audio beamforming techniques to create acoustic waves that provide sound across the venue 100. Generally, the one or more line arrays may direct these acoustic waves across various three-dimensional regions within the venue 100 and provide sound to one or more seating sections 104. In some embodiments, one or more seating sections 104 allow audience members to be seated in various locations within the venue 100 and experience the event.

[0013] As described in more detail below, the venue 100 may implement a spatial audio measurement system to estimate the coverage areas of the loudspeakers 102.1-102.i across the venue 100. In some embodiments, the spatial audio measurement system enables visual estimation to estimate the coverage areas of the loudspeakers 102.1-102.i. Generally, the human eye can detect a range of sound at a resolution of approximately 4×10 14 Hertz (Hz) ~ approx. 8 x 10 14While the human eye can detect light in the visible spectrum, the acoustic waves generated by the loudspeakers 102.1-102.i are in the auditory spectrum, from about 20 Hz to about 20 kHz. Therefore, the acoustic waves generated by the loudspeakers 102.1-102.i typically cannot be detected by the human eye. As described in more detail below, the venue 100 can implement a spatial audio measurement system to project light in the visible spectrum that corresponds to the acoustic waves generated by the loudspeakers 102.1-102.i. This allows for a visual estimation of the coverage areas of the loudspeakers 102.1-102.i across the venue 100. In some embodiments, the spatial audio measurement system can be used to adjust the spatial arrangement of the loudspeakers 102.1-102.i within the venue 100 to accommodate incomplete or missing areas, also referred to as gaps, in the coverage areas of the loudspeakers 102.1-102.i. As part of this spatial audio measurement system, the venue 100 can project light that coincides with the acoustic waves generated by the loudspeakers 102.1-102.i in an extended reality (XR) environment, such as an augmented reality (AR) environment, a virtual reality (VR) environment, and / or a mixed reality (MR) environment. As part of this spatial audio measurement system, the venue 100 can measure the acoustic waves generated by the loudspeakers 102.1-102.i. In some embodiments, the venue 100 can analyze these measurements and modify one or more parameters, characteristics, and / or attributes of the acoustic waves generated by the loudspeakers 102.1-102.i. In these embodiments, the spatial audio measurement system can be used to modify the acoustic waves generated by the loudspeakers 102.1-102.i to provide a similar auditory experience of the event for the audience across the venue 100.

[0014] Exemplary Spatial Audio Measurement System Using a Mixed Reality Device 2A illustrates a graphical representation of an exemplary spatial audio measurement system according to some exemplary embodiments of the present disclosure. In some embodiments, spatial audio measurement system 200 can include speakers 250 and virtual speaker projectors 252. In these embodiments, spatial audio measurement system 200 can map a light beam 256, generated by virtual speaker projector 252, that coincides with an audio beam 254, generated by speaker 250, onto a target area 258. As described in more detail below, one or more real-world users of spatial audio measurement system 200 can compose and play audio, i.e., sounds, associated with an event on speakers 250. As one or more real-world users view the light beam 256, these users can virtually interact with the spatial audio measurement system 200, for example, moving around the virtual event and viewing the light beam 256 at various locations and / or modifying one or more parameters, characteristics, and / or attributes of the speaker 250.

[0015] In some embodiments, the spatial audio measurement system 200 can use a virtual speaker projector 252 to provide a 3D representation of the audio transmission pattern generated by the speaker 250. As described above in FIG. 1 , the audio beam 254 generated by the speaker 250 typically cannot be detected by the human eye. To provide a visual aid for audio detection, the virtual speaker projector 252 can generate a light beam 256 that approximates the properties (e.g., direction, size, shape, etc.) of the audio beam 254 and visualizes those properties of the audio beam 254. The virtual speaker projector 252 can then map the light beam 256, which coincides with the audio beam 254 to visually illustrate the projection of the audio beam 254, onto a target area 258 in a manner suitable for rendering on a device such as a head-mounted display. This allows for a visual estimation of the coverage area of ​​the speaker 250 within the venue.

[0016] In some embodiments, spatial audio measurement system 200 includes microphone 260 and can measure audio beam 254 generated by speaker 250. Microphone 260 can detect sound vibrations in the air, convert the sound vibrations into electronic signals, and provide feedback information to spatial audio measurement system 200. Spatial audio measurement system 200 can analyze these measurements from microphone 260 and modify one or more parameters, characteristics, and / or attributes of audio beam 254 generated by speaker 250.

[0017] 2B illustrates a graphical representation of an exemplary spatial audio metering system according to some exemplary embodiments of the present disclosure. The spatial audio metering system 200 can map light beams 208, corresponding to audio, i.e., sounds, associated with an event, such as a musical event, a theatrical event, a sporting event, and / or a video event, to name a few, onto a target area 210 of a venue 202 for a user to view through a user device 204. As described in more detail below, one or more real-world users of the spatial audio metering system 200 can configure and play selected sound sources in an extended reality (XR) environment, such as an augmented reality (AR), virtual reality (VR), and / or mixed reality (MR) environment, either to test the selected sound sources in the venue 202 or to simulate an event being presented in the venue 202 as a virtual event. When one or more real-world users view the audio visualization, these users can virtually interact with the audio visualization, for example, moving around the virtual event, viewing the audio visualization at various locations, and / or modifying one or more parameters, characteristics, and / or attributes of the sound source.

[0018] In some embodiments, user device 204 may include one or more computing devices, such as one or more desktop computers, one or more mobile phones, one or more mobile computing devices, one or more mobile internet devices, such as tablet computers and / or laptop computers, one or more mobile video game consoles, one or more mobile wearable electronic devices, such as smart watches, and / or any other computing device having one or more processors, as would be recognized by one of ordinary skill in the art without departing from the spirit and scope of the present disclosure. In some embodiments, these one or more computing devices may be communicatively coupled to one or more virtual reality (VR) headsets and / or one or more VR controllers.

[0019] In some embodiments, the user device 204 can process the virtual event, which can be viewed independently or overlaid on a view of the venue 202 through the user device 204. This processing can include tracking each user's three-dimensional position within the three-dimensional space of the virtual event, estimating each user's line of sight at the three-dimensional position, estimating a field of view for each user associated with a line of sight, and / or matching the virtual event view to each user's field of view at the three-dimensional position.

[0020] In some embodiments, a real-world user of a user device 204 can navigate through viewing the virtual event. In some embodiments, these interactions can include having a virtual user corresponding to the real-world user virtually move around within the three-dimensional space of the virtual event and viewing the light beam 208 at various locations. In some embodiments, these various locations can include locations within the three-dimensional space of the virtual event that are not typically available for viewing the event at the venue 202, such as, for example, broadcast film camera locations. In some embodiments, the real-world user can navigate through the venue 202 as the user device 204 view updates in real time, aligning the virtual event overlay with the user's view of the venue 202.

[0021] In some embodiments, the spatial audio measurement system 200 can include light beams 208. These light beams 208 can provide visual representations of real-world audio effects associated with actual events and provide visual support for real-world audio effects that are not typically visible to the human eye. While these various real-world audio effects may be detectable by audio equipment, the exact coverage area may be difficult to understand with respect to the selected content. Additionally, if sound coverage mapping is required within a virtual environment, physical measurements of sound coverage patterns may not be applicable. Therefore, techniques such as those disclosed herein use volumetric light beams within the imaging of the venue 202 to visualize sound coverage parameters, characteristics, and / or attributes.

[0022] In some embodiments, the virtual event can represent real-world audio properties through light beams 208, which can be visual virtual effects such as arrows, lines, or any other visual effect that can be displayed within the virtual event. For example, light beams 208 for directed audio (e.g., beamforming) may depict the direction of the audio from a source to an intended target area 210 and the interaction of the audio with the venue's architecture. As the parameters, characteristics, and / or attributes of these real-world effects are modified (e.g., by the user device 204), the light beams 208 can update to represent the modifications. For example, the updated light beams 208 may represent a new direction or target for the directed audio.

[0023] In some embodiments, the light beam 208 may represent a focused beam sound source that may be configured to be heard by a target area 210 of a seating section within the venue 202. In some embodiments, the sound beam is generated by a piezoelectric or electrostatic transducer (or an array thereof). However, any known method of generating a sound beam for location-specific audio coverage may be substituted without departing from the scope of the technology described herein. Visualization of the real sound coverage pattern would provide a sound engineer, for example, with a quick method for configuring sound coverage for a specific seating section.

[0024] In some embodiments, the light beam 208 may represent a wide-coverage beam sound source that may be configured to be heard by a target area 210 throughout the seating area within the venue 202. In some embodiments, the wide sound beam may be generated by an array of piezoelectric or electrostatic transducers. However, any known method of generating a wide sound beam for location-specific audio coverage may be substituted without departing from the scope of the technology described herein. Visualization of the real sound coverage pattern may provide a sound engineer, for example, a quick way to configure sound coverage for all seating sections.

[0025] In some embodiments, the light beam 208 can represent a diffuse beam sound source that can be configured to be heard by a targeted area 210 of a seating section within the venue 202. Diffusion of sound can be important to avoid dead spots, i.e., areas where sound is weak or cannot be clearly heard. A diffused beam is generated by scattering sound through surface variations of the sound source, such as deflection or scattering surfaces. However, any known method of generating a diffused sound beam for location-specific audio coverage can be substituted without departing from the scope of the technology described herein. Visualization of the real sound coverage pattern would provide a sound engineer, for example, a quick way to configure sound coverage for a focused seating section.

[0026] In some embodiments, the light beam 208, representing the sound source coverage, may have any geometric shape, aperture size, light volume, or other geometric light properties as defined by the source type and / or sound configuration. For example, a focused beam may start from a point and expand into a trapezoidal shape as it extends from the point, while a divergent beam may start as a rectangle and expand into a trapezoidal shape. In another example, a focused beam may start from a point and expand into a cone shape, thus creating a circular target area 210.

[0027] In some embodiments, the light beams 208 can be assigned unique colors that can symbolize certain audio properties (e.g., frequencies) of the corresponding audio sources. Specific color assignments and frequency ranges will be described hereinafter, but the ranges and color assignments can vary without departing from the scope of the technology described herein. For example, other colors, frequencies, and color intensity gradients can be selected, as desired, so long as distinct colors are assigned to defined audio sources, channels, ranges, etc.

[0028] In the first approach, the seven known colors of the visual spectrum are mapped to seven known audio ranges. For example, sunlight is divided into seven colors: violet, indigo, blue, green, yellow, orange, and red (VIBGYOR). In addition, sound is divided into commonly labeled ranges: sub-bass (16-60 Hz), bass (60-250 Hz), low-midrange (250-500 Hz), midrange (500-2 kHz), high-midrange (2-4 kHz), presence (4-6 kHz), and brilliance (6-20 kHz). To illustrate, volume intensity, color saturation, or brightness may be increased proportionally as volume increases. In this approach, different colors are assigned to each audio range. For example, as shown, darkest colors are assigned to lower frequency components and lighter colors to higher frequency components. These specific color and frequency range assignments are for illustrative purposes and may be varied to achieve different visualizations.

[0029] In this first approach, loudspeakers may have dedicated purposes, such as providing bass. In this scenario, each dedicated bass source may be visually mapped using a common color to illustrate the overall bass coverage within the venue. Alternatively, or in addition, each source may generate multiple audio ranges, and the visualization may include one or more audio ranges, using a common color for similar ranges. For example, venue 202 may have 20 audio sources, each providing at least a first and second audio range. Each of the first and second audio ranges may be assigned a unique color, and visualizations may be generated for each audio range or for both ranges in a combined visualization. This approach may be applied to any audio range, combination of ranges, or specific sound effect.

[0030] Alternatively, or in addition, in a second approach, distinct sound sources (e.g., loudspeaker arrays or beams) may be collectively assigned distinct colors, allowing one sound source to be distinguished from another when sound coverage overlap occurs within a venue. In this approach, colors are not assigned to audio ranges.

[0031] Alternatively, or in addition, the two approaches may be combined. Those skilled in the art will appreciate that other approaches or combinations may be implemented using techniques as described herein without departing from the scope of the present disclosure.

[0032] 2 , the user device 204 can compose and play back virtual events to virtually prepare and simulate events being presented in a venue. In some embodiments, these interactions can include virtually modifying the virtual event while the user is viewing the virtual event, either independently or overlaid on the venue 202. In some embodiments, these interactions can include modifying the physical audio system for the venue 202. In these various embodiments, to input modifications, the user can interact with the user configuration interface 206 to change the temporal, spectral, and spatial properties of the audio system for the virtual event and / or the real-world event. In some embodiments, the user configuration interface 206 can allow the user to stop, pause, fast-forward, and / or rewind the virtual event.

[0033] In some embodiments, the user configuration interface 206 for the user device 204 can provide various virtual graphical elements to real-world users, allowing them to modify the virtual event. In these embodiments, the virtual graphical elements of the user configuration interface 206 can outline various interactions, e.g., modifications, available to the real-world user. In these embodiments, the virtual graphical elements of the user configuration interface 206 can include, to name a few, one or more radio buttons, one or more check boxes, one or more text boxes, one or more toggle switches, one or more pop-up menus, one or more lists, and / or any other suitable mechanism that allows a real-world user to interact. For example, the modifications can include removing one or more parameters, characteristics, and / or attributes of the virtual event from the three-dimensional space of the virtual event. As another example, the modifications can include moving the location, e.g., position and / or orientation, of one or more parameters, characteristics, and / or attributes of the virtual event within the three-dimensional space of the virtual event. As a further example, these modifications may include inserting one or more new parameters, new properties, and / or new attributes into the three-dimensional space of the virtual event.In some embodiments, the parameters, characteristics, and / or attributes of the virtual event may include or relate to temporal, spectral, and spatial properties of the audio of the venue, one or more computer-generated digital models of various architectural features of the venue, one or more computer-generated digital models of various objects, one or more computer-generated models of one or more performers, one or more computer-generated models of one or more props associated with the event, and / or other suitable parameters, characteristics, and / or attributes of the virtual event that will be apparent to those skilled in the art without departing from the spirit and scope of the present disclosure.

[0034] In some embodiments, a user may modify parameters, characteristics, and / or attributes of a virtual audio effect via a user configuration interface 206 provided by the user device 204. For example, a user may modify parameters, characteristics, and / or attributes of a virtual event, such as the audio direction of a speaker, on the user device 204. Such modifications may alter the virtual event and display a new representation of the revised audio direction within the venue 202. In this manner, the spatial audio measurement system 200 can simulate the behavior of real-world audio effects within a particular venue and display that simulated behavior as virtual graphical elements. As described below, the spatial audio measurement system 200 can perform various audio configuration tasks.

[0035] In some embodiments, the spatial audio measurement system 200 can perform temporal measurements for audio composition tasks. A user of the spatial audio measurement system 200 can set the volume of a sound source through various input methods using the user configuration interface 206. For example, the user can adjust the volume level of a sound source using voice or gesture input. As the user configures the temporal properties, the user configuration interface 206 can display a sound pressure meter, decibel (dB) values, labels, and / or other indicators of the temporal properties. In response to the configuration input, the user can view a light beam 208 and / or data overlays. Thus, the spatial audio measurement system 200 enables the user to perform composition tasks using in-situ sound pressure visualization, thus reducing logistical demands and time budgets by increasing confidence in sound pressure across a survey site.

[0036] In some embodiments, spatial audio measurement system 200 can perform spectral measurements for audio composition tasks. A user of spatial audio measurement system 200 can use user configuration interface 206 to set the frequency and balance of selected frequencies, referred to as source equalization (EQ), through various input methods. For example, a user can adjust the EQ of a sound source using voice or gesture input. As a user configures spectral properties, user configuration interface 206 can display frequency meters, EQ controls, indicators, and / or other indicators of spectral properties. In response to the configuration input, the user can view light beams 208 and / or data overlays. Thus, spatial audio measurement system 200 enables users to perform composition tasks using in-situ spectral visualization, thereby reducing logistical demands and time budgets by allowing users to match spectral content to the output capabilities of venue speakers.

[0037] In some embodiments, the spatial audio measurement system 200 can perform spatial measurements for audio composition tasks. A user of the spatial audio measurement system 200 can use the user configuration interface 206 to set the coverage area of ​​a sound source through various input methods. For example, a user can use voice or gesture input to adjust the directional sound beam of a sound source and avoid acoustic reflections. In another example, a user can preview a sound rendition or directional sound movement at various playback speeds. In another example, a user can overlay light beams 208 to determine studio-to-venue transformation differences in coverage and overlap due to various speaker layouts. When a user configures spatial properties, the user configuration interface 206 can display coverage areas (e.g., incident audio beams, reflected audio beams, and / or overlapping audio beams), frequency meters, sound pressure meters, signs, and / or other spatial indicators. In response to configuration input, the user can view the light beams 208, a time lapse of the virtual event, a heat map, and / or data overlays. Thus, spatial audio measurement system 200 enables users to perform configuration tasks with in-situ spatial visualization, thus reducing logistical demands and time budgets by providing visual aids for the purposes of authoring spatial audio content, adjusting studio-to-venue transformations, and calibrating large venue playback systems.

[0038] 3 illustrates a graphical representation of an exemplary spatial audio measurement system using multiple MR devices, according to some exemplary embodiments of the present disclosure. In this exemplary embodiment, spatial audio measurement system 300 can enable multiple real-world users to view and compose a virtual event from their respective user devices 304.1-304.n, either from the same location or at different locations within the three-dimensional space of venue 302, through an extended reality (XR) environment, such as an augmented reality (AR), virtual reality (VR), and / or mixed reality (MR) environment. The above discussion of spatial audio measurement system 200 can be applied to spatial audio measurement system 300. Spatial audio measurement system 300 can perform various audio composition tasks, as disclosed above with respect to FIG. 2.

[0039] In some embodiments, the spatial audio measurement system 300 can provide a visual representation of sound coverage from multiple sound sources within the venue 302. As shown in FIG. 3 , a first light beam 308, a second light beam 310, a third light beam 312, and a fourth light beam 314 can be configured to be heard by selected target areas 318, 320, 316, and 322 of the venue 302, respectively. In this approach, one or more of the various individual sound coverages may overlap. In this scenario, visualization of the sound sources will lead to an understanding of where coverage exists or where gaps may exist. Adjustments to the sound source configuration, such as, but not limited to, type, directionality, size, shape, power, location, or spectral properties, may be necessary to fill any gaps or eliminate unnecessary overlaps. Any known sound source can be substituted without departing from the scope of the technology described herein.

[0040] In some embodiments, the spatial audio measurement system 300 can implement multi-zone content configuration using multiple user devices 304.1-304.n. By interacting with the user configuration interface 306, a user can use multiple user devices to play and probe light beams 308-314 in multiple target areas 316-322 in parallel. In some embodiments, a user can monitor crosstalk or signal leakage between adjacent target areas 316-322. For example, a user can listen to audio content, monitor sound pressure level (SPL) and signal leakage, and annotate audio quality in multiple target areas 316-322 simultaneously. As a user configures multi-zone content properties, the user configuration interface 306 can display coverage areas, frequency meters, sound pressure meters, signs, and / or other audio property indicators. In response to configuration inputs, the user can view light beams 308-314 and / or data overlays. Thus, the spatial audio measurement system 300 can reduce logistical demands and time budgets by enabling more than one user to perform configuration tasks in parallel with in-situ visualization and annotation interactions.

[0041] In some embodiments, the spatial audio measurement system 300 can propagate these interactions and modifications across multiple real-world users of the spatial audio measurement system 300, allowing these real-world users to collaboratively interact with the virtual event. A user at user device 304.1 may desire to collaborate with another user at user device 304.2 regarding an interaction with light beams 308-314. User device 304.1 may initiate communication with user device 304.2, which may include modifying the virtual event. For example, user device 304.1 may send a command to user device 304.2 to change the direction of light beam 308. As another example, user device 304.1 may transmit a text-based communication to user device 304.2 that includes an image of the interaction with light beams 308-314. When applying a configuration modification, user device 304.1 can transmit the configuration modification to user devices 304.2-304.n. In response to receiving the modifications, the user devices 304.2-304.n can update their respective views of the virtual event based on the modifications, including displaying light beams 308-314 representing the new audio direction within each user device's 304.n respective virtual event view.

[0042] 4 illustrates a graphical representation of an exemplary spatial audio measurement system using MR devices, according to some exemplary embodiments of the present disclosure. In this exemplary embodiment, spatial audio measurement system 400 may enable one or more real-world users to view and measure a virtual event from their respective user devices 404.1-404.n, either from the same location or at different locations within the three-dimensional space of venue 402, through an extended reality (XR) environment, such as an augmented reality (AR), virtual reality (VR), and / or mixed reality (MR) environment. The above discussion of spatial audio measurement system 200 may apply to spatial audio measurement system 400.

[0043] In some embodiments, the spatial audio measurement system 400 can include one or more microphones 412. The microphones 412 can detect sound vibrations in the air and convert the sound vibrations into electronic signals to provide feedback information to the spatial audio measurement system 400. In some embodiments, the microphones 412 can be stationary. For example, the microphones 412 can be affixed to seating sections within the target area 410. In some embodiments, the microphones 412 can be portable. For example, the microphones 412 can be affixed to an unmanned aerial vehicle (UAV) or user device 404 such that the microphones 412 can be moved throughout the venue 402.

[0044] In some embodiments, the user measurement interface 406 for the user device 404 can provide various virtual graphical elements to the real-world user, allowing them to modify the virtual event. The user device 404 can receive feedback information from the microphone 412 and pass this feedback information to the user measurement interface 406, which can generate virtual graphical elements for the user. These virtual graphical elements of the user measurement interface 406 can outline various types of feedback information, such as volume, frequency, and coverage, of the detected audio in the venue 402. As described in the discussion of FIG. 2, the user measurement interface 406 can allow the user to modify audio properties.

[0045] In some embodiments, the spatial audio measurement system 400 can perform quality control operations through measurement tasks related to the audio system in the venue 402. A user can view an identification tag for each speaker and annotate the speaker's status by using gestures or voice input. For example, a user can use the user device 404 to play a test tone, listen to the live output, and annotate the speaker results. Through annotations, the user can indicate functional and malfunctioning speakers in the venue 402. This feature of the spatial audio measurement system 400 can be useful in large venues where locating the sound origin through listening alone is difficult and it is not efficient for an operator to walk to each speaker for individual inspection. As a result, the spatial audio measurement system 400 can reduce logistical demands and time budgets by enabling users to perform quality control tasks such as measurement, inspection, and annotation using in-situ visualization.

[0046] In some embodiments, the spatial audio measurement system 400 can perform temporal measurements for audio measurement tasks. A user of the spatial audio measurement system 400 can calculate sound pressure using input from the microphone 412, which can act as an SPL meter. For example, the user can compare a digital gain level in the user measurement interface 406 to the SPL measured using the microphone 412. In response to the measurement information, the user can use the user measurement interface 406 to adjust the volume of a sound source through various input methods. For example, the user can adjust the volume level of a sound source using voice or gesture input. When the user measures a temporal property, the user measurement interface 406 can display a sound pressure meter, decibel (dB) values, a sign, and / or other indicators of the temporal property. In response to the measurement input, the user can view a light beam 408 and / or a data overlay. Thus, the spatial audio measurement system 400 enables users to perform measurement tasks with in-situ sound pressure visualization based on input from the microphone 412, thus reducing logistical demands and time budgets by increasing confidence in sound pressure across a survey site.

[0047] In some embodiments, the spatial audio measurement system 400 can perform spectral measurements for audio measurement tasks. A user of the spatial audio measurement system 400 can calculate spectral frequencies using input from a microphone 412, which can act as a spectral analyzer. For example, the user can compare the calculated spectral frequencies in the user measurement interface 406 to a frequency response measured using the microphone 412. In response to the measurement information, the user of the spatial audio measurement system 200 can use the user measurement interface 406 to adjust the frequency and balance of selected frequencies, referred to as the equalization (EQ) of the sound source, through various input methods. For example, the user can adjust the EQ of the sound source using voice or gesture input. When the user measures spectral properties, the user measurement interface 406 can display frequency meters, EQ controls, labels, and / or other indicators of the spectral properties. In response to the measurement input, the user can view a light beam 408 and / or a data overlay. Thus, the spatial audio measurement system 400 enables users to perform measurement tasks using in-situ spectral visualization based on input from the microphone 412, thus reducing logistical demands and time budgets by allowing users to match spectral content to the output capabilities of venue speakers.

[0048] In some embodiments, the spatial audio measurement system 400 can perform spatial measurements for audio measurement tasks. A user of the spatial audio measurement system 400 can use input from the microphones 412 to calculate coverage, sound pressure, or spectral frequency. For example, the user can compare the calculated spectral frequency against frequency responses measured by the microphones 412 in various seating sections to determine the coverage area of ​​a certain frequency. A heat map visualization can provide an overview of such inspection results. In response to the measurement information, the user of the spatial audio measurement system 400 can use the user measurement interface 406 to adjust the coverage area of ​​a sound source through various input methods. For example, the user can use voice or gesture input to adjust the directional sound beam of the sound source to avoid acoustic reflections. In another example, the user can preview sound renditions or directional sound movement at various playback speeds. In another example, the user can overlay light beams 408 to determine studio-to-venue conversion differences in coverage and overlap due to various speaker layouts. When a user measures spatial properties, the user measurement interface 406 can display coverage areas (e.g., incident, reflected, and / or overlapping audio beams), frequency meters, sound pressure meters, signs, and / or other spatial indicators. In response to measurement input, the user can visualize light beams 408, a time lapse of the virtual event, a heat map, and / or data overlays. Thus, the spatial audio measurement system 400 enables users to perform measurement tasks with in-situ spatial visualization based on input from microphones 412, thus reducing logistical demands and time budgets by providing visual aids for purposes of authoring spatial audio content, adjusting studio-to-venue transformations, and calibrating large venue playback systems.

[0049] FIG. 5 illustrates a graphical representation of an exemplary spatial audio measurement system using multiple MR devices, according to some exemplary embodiments of the present disclosure. In this exemplary embodiment, the spatial audio measurement system 500 can enable multiple real-world users to view and measure a virtual event from their respective user devices 504.1-504.n, either from the same location or at different locations within the three-dimensional space of a venue 502, through an extended reality (XR) environment, such as an augmented reality (AR), virtual reality (VR), and / or mixed reality (MR) environment. To improve measurement accuracy, multiple reference microphones 524 can be used in conjunction with the user devices 504.1-504.n. The above discussion of the spatial audio measurement system 400 can be applied to the spatial audio measurement system 500. The spatial audio measurement system 500 can perform various audio measurement tasks, as disclosed above with respect to FIG. 4.

[0050] In some embodiments, the spatial audio measurement system 500 can provide a visual representation of sound coverage from multiple sound sources within a venue 502. As shown in FIG. 5 , a first light beam 508, a second light beam 510, a third light beam 512, and a fourth light beam 514 can be configured to be heard by selected target areas 518, 520, 516, and 522 of the venue 502, respectively. In this approach, one or more of the various individual sound coverages may overlap. In this scenario, visualization of the sound sources will lead to an understanding of where coverage exists or where gaps may exist. Measuring the sound sources, therefore, adjusting the source configuration, such as, but not limited to, type, directivity, size, shape, power, location, or spectral properties, may be necessary to fill any gaps or eliminate unnecessary overlaps. Any known sound source can be substituted without departing from the scope of the technology described herein.

[0051] In some embodiments, the spatial audio measurement system 500 can perform multi-zone content measurements using multiple user devices 504.1-504.n. By interacting with the user measurement interface 506, a user can use multiple user devices to play and probe light beams 508-514 in parallel based on input from microphones 524 in multiple target areas 516-522. In some embodiments, a user can monitor crosstalk or signal leakage between adjacent target areas 516-522 and detect live playback by using the microphones 524. For example, a user can listen to audio content, monitor sound pressure level (SPL) and signal leakage, and simultaneously annotate audio quality in multiple target areas 516-522. When a user measures multi-zone content properties, the user measurement interface 506 can display coverage areas, frequency meters, sound pressure meters, signs, and / or other audio property indicators. In response to the measurement input, the user may view light beams 508-514 and / or data overlays based on input from microphone 524. Thus, spatial audio measurement system 500 may reduce logistical demands and time budgets by enabling more than one user to perform measurement tasks in parallel with in-situ visualization based on input from microphone 524 and annotation interactions.

[0052] In some embodiments, spatial audio measurement system 500 can enable multiple real-world users of spatial audio measurement system 500 to collaboratively interact with a virtual event, propagating these interactions and modifications across these real-world users. A user at user device 504.1 may desire to collaborate with another user at user device 504.2 regarding an interaction with light beams 508-514. User device 504.1 may initiate communication with user device 504.2, which may include modifying the virtual event. For example, user device 504.1 may send a command to user device 504.2 to change the direction of light beam 508. As another example, user device 504.1 may transmit a text-based communication to user device 504.2 that includes an image of the interaction with light beams 508-514. When applying a measurement modification, user device 504.1 can transmit the measurement modification to user devices 504.2-504.n. In response to receiving the modifications, user devices 504.2-504.n can update their respective views of the virtual event based on the modifications, including displaying light beams 508-514 representing the new audio direction within each user device's 504.n view of the virtual event.

[0053] Exemplary Spatial Audio Measurement Mixed Reality User Device 6 illustrates an example user device that may be implemented in an example spatial audio measurement system according to some example embodiments of the present disclosure. In the example embodiment illustrated in FIG. 6, user device 600 can process and analyze one or more audio sources and generate a visual representation of sound coverage that is mapped onto a virtual model of a venue and simulates an event being presented in the venue. When a real-world user of user device 600 is viewing the virtual event on user device 600, the real-world user can use user device 600 to virtually interact with the virtual event, for example, move around within the virtual event, view the virtual event at various locations, and / or modify parameters, characteristics, and / or attributes of the virtual event.

[0054] In some embodiments, user device 600 can be implemented as a stand-alone or discrete device and / or can be incorporated within or coupled to one or more computing devices such as one or more desktop computers, one or more mobile phones, one or more mobile computing devices, one or more headsets designed for virtual reality (VR), augmented reality (AR), and / or mixed reality (MR), one or more mobile internet devices such as tablet computers and / or laptop computers, one or more mobile video game consoles, one or more mobile wearable electronic devices such as smart watches, and / or any other computing device having one or more processors as would be recognized by one of ordinary skill in the art without departing from the spirit and scope of the present disclosure. In some embodiments, user device 600 can represent exemplary embodiments of one or more of the user devices as described in FIGS. 2-5 above. As shown in FIG. 6, the user device 600 can include a user processing device 602, a user display device 604, a user controller device 606, and a virtual speaker projector 608.

[0055] In some embodiments, user processing device 602 can process virtual events and generate a virtual event view 610 that corresponds to a location of a virtual user associated with the real-world user within the virtual event. In some embodiments, user processing device 602 can process virtual events and generate a virtual event view 610 for presentation within a virtual reality (VR) environment. In these embodiments, the virtual event view 610 displayed by user device 600 represents an immersive virtual world. This virtual world effectively immerses the real-world user within the virtual event, giving the real-world user the impression that they are entering the virtual event. In some embodiments, as the real-world user moves, e.g., changes location within the virtual event, and / or moves parts of their body in the real world, e.g., moving their head up and down or left and right, user device 600 can update virtual event view 610, effectively immersing the real-world user within the virtual event. In some embodiments, user processing device 602 can process virtual events and generate a virtual event view in a manner substantially similar to that described in FIGS. 2-5 above. In these embodiments, the processing may include tracking the three-dimensional position of the virtual user within the three-dimensional space of the virtual event, estimating the virtual user's line of sight at the three-dimensional position, estimating the virtual user's field of view associated with the line of sight, and / or matching the virtual event view 610 to the virtual user's field of view at the three-dimensional position. As shown in FIG. 6, the user processing device 602 may include an audio program 615, an audio analyzer 616, a venue database 618, an auditory simulator 619, an audio visualizer 620, a visual representation 622, an interaction module 624, and a venue manipulator 626.

[0056] In some embodiments, the user display device 604 may enable a user to view virtual events generated by the user processing device 602. The user display device 604 may include a stand-alone or discrete device and / or a display of one or more computing devices such as one or more desktop computers, one or more mobile phones, one or more mobile computing devices, one or more headsets designed for virtual reality (VR), augmented reality (AR), and / or mixed reality (MR), one or more mobile internet devices such as tablet computers and / or laptop computers, one or more mobile video game consoles, one or more mobile wearable electronic devices such as smart watches, and / or any other computing device. For example, the user display device 604 may provide a MR experience that combines elements of both augmented reality (AR) and virtual reality (VR), in which the real world and digital objects interact.

[0057] In some embodiments, the user display device 604 may enable a user, using an MR-enabled headset, to visually gauge the spatial characteristics of the sound output from one or more speakers 612 in the venue. A virtual event view 610 appears on the user display device 604, allowing the user to see a visual overlay of sound propagation, virtually visualized as light propagation on physical objects such as seating areas. Specifically, visualization of sound path, volume, coverage, and overlap in MR provides a useful visual aid for audio system calibration and sound rendition preview. The user display device 604 may include one or more cameras to visually identify seating areas, loudspeaker panels, and venue geometry, thus enabling the user processing device 602 to map the virtual loudspeaker layout to the physical venue. Similarly, the user display device may include one or more microphones 614 to identify the acoustic performance of the speakers 612 in the venue. To improve acoustic measurement performance and accuracy, additional reference microphones 614 can be placed throughout the venue to capture real-time audio from the speakers 612 .

[0058] In some embodiments, user controller device 606 represents an input device used by a real-world user to interact with virtual events when using user device 600. In some embodiments, user controller device 606 may include one or more action buttons and / or one or more omnidirectional control sticks or buttons that may be manipulated by the real-world user to interact with virtual events. In some embodiments, the real-world user can use the one or more action buttons and / or one or more omnidirectional control sticks or buttons to perform various actions within the virtual world. For example, a real-world user may use one or more action buttons and / or one or more omnidirectional control sticks to "point and click" and / or "drag and drop" one or more computer-generated digital models of various architectural features of the venue in three-dimensional space, such as performance areas, media surfaces, seating locations, and / or standing locations, to name a few, and / or one or more computer-generated digital models of various objects in the venue in three-dimensional space, such as stage objects associated with the venue and / or stage objects associated with the event, to name a few.

[0059] In some embodiments, each user device 600 may be configured to generate and display a virtual event view 610 representing the viewing height of the virtual event based on the virtual location of the virtual user associated with the user device 600. The virtual speaker projector 608 may provide a 3D representation of audio transmission patterns generated by an audio source, such as those detected by a microphone 614 and output from a speaker 612, suitable for rendering within the virtual event view 610 on the user device 600. The user device 600 may represent interactions with the virtual event view 610, for example, by moving around within the virtual event, viewing the virtual event at various locations, and / or manipulating the virtual event by modifying the virtual event's parameters, characteristics, and / or attributes. The visualization from the virtual speaker projector 608 enables users to observe the results of their interactions with the virtual event. Different interactions may be available via a graphical user interface within the virtual event view 610 of the virtual event based on the type of user device. For example, the user device 600 may include physical interface devices such as a keyboard and mouse. The virtual event view 610 may be customized to include interactions that are more easily entered via such physical interface devices. Examples of such interactions for the user device 600 include modifying a code segment of a virtual event, or any modification of parameters, characteristics, and / or attributes that require text entry. As another example, the virtual event view 610 may be customized to accommodate a VR implementation of the user device 600 and include interactions specific to the user controller device 606 or hands-free gestures. As another example, the virtual event view 610 may be customized to accommodate the mobility of the user device 600. Examples of such interactions for the user device 600 include providing augmented reality (AR)-based interactions.For example, a user device 600 may be physically located within a venue in which an event will occur, while another user device 600 may be physically located remotely from the venue. The virtual event view 610 may combine a real-time view of the venue with the virtual event in an augmented reality format. For example, virtual graphical elements of the virtual event view 610 may be displayed as an overlay over real-world elements of the venue.

[0060] In some embodiments, the audio program 615 can include audio objects and their metadata, which relate to the audio sources of an event. The audio objects can be sounds from various audio sources stored as audio files for output in a channel, along with corresponding spatial metadata that defines position, level, and movement. The audio sources can be libraries of audio content, external devices with stored audio content, streaming sources, or software packages that produce audio signals to be analyzed and visualized. Common examples are digital audio workstation (“DAW”) software packages and audio playback servers. In some embodiments, the audio sources of specific audio content can be employed to generate various different visual mappings for a venue as determined by the spatial metadata. In some embodiments, for example, the mapping of specific audio content that will be part of a live presentation at a venue can be tested or previewed to determine sound coverage for various seating sections of the venue throughout the live presentation. To begin the mapping process, the audio program 615 sends audio object data to the audio analyzer 616 for processing.

[0061] In some embodiments, the audio analyzer 616 receives desired audio content from an audio source as defined by the audio program 615. The audio analyzer 616 can analyze the audio content with respect to its audio signal properties, such as, but not limited to, frequency, wavelength, period, and amplitude (volume), pitch, or modulation. Alternatively, or in addition, a generic speaker array visual mapping can be implemented using generic test audio content to determine a generalized speaker array sound distribution within a venue. For example, the audio signals may be configured as an audio source set of mono audio signals formatted for the specific audio system being visualized.

[0062] In some embodiments, a venue database 618 of loudspeaker and sound beam locations per input channel (i.e., audio system configuration) for a venue provides a mapping of speaker / beam locations and parameters for a specific venue. Parameters can include, but are not limited to, loudspeaker location and orientation, number of speakers, arrangement (e.g., array), power, sound distribution type (e.g., diffuse, directional, etc.), distortion, etc. In one non-limiting example, an audio system configuration for a venue can reflect one or more sets of loudspeakers and beams arranged in one or more arrays of various numbers, sizes, types, and power outputs. In some embodiments, an audio system configuration is a fixed package of data that defines the geometric properties of the speaker system being measured. In a non-limiting example, the geometric properties can be the location and orientation of the individual speakers or beams that make up the system, as well as their capabilities and coverage patterns. This information can be used to initially create audio signals within the audio source and, in a visualization stage, map the resulting signals into three dimensions. Additionally, the venue database 618 can send data defining the geometric properties of the speaker system to an auditory simulator 619 to simulate the speaker 612 output on the headphones of the user device 600.

[0063] In some embodiments, the auditory simulator 619 can render binaural output for headphone monitoring, thereby allowing a real-world user of the user device 600 to compare the digital audio output with the venue acoustic output in situ. The auditory simulator 619 can render the binaural output using information from the audio program 615, the venue database 618, and, when necessary, head-tracking sensors on the user device 600. By leveraging the user's hearing, this simulation of the audio output can allow the user of the user device 600 to test and verify the results of their interaction with the audio system.

[0064] In some embodiments, the audio visualizer 620 can implement a volumetric light renderer and render (i.e., depict) light beams that represent the audio signal. The audio visualizer 620 can render the beams based on various attributes, such as location, color, orientation, type, shape, intensity, or range, to name a few. The shape and location of the light beams are dictated by the audio system configuration. The color and intensity of the light beams are a direct representation of the incoming audio signal properties relative to the sound source. The type of light beam can be based on the type of sound source. In non-limiting examples, the type of sound source may include loudspeaker type (e.g., omnidirectional, directional, etc.), beam, size, array shape, power considerations, etc. One or more microphones 614, installed throughout the venue or attached to the user device 600, can detect the signals produced by the audio sources. The audio visualizer 620 can receive information from the audio analyzer 616, read the audio signal in real time, and emit simplified signals that can be used to drive measurement visualizations, such as the overall signal's instantaneous level and its frequency content. In various embodiments, the audio source can be either the actual audio system being measured or an acoustic model (e.g., a simulation) of this system, such as a binaural renderer.

[0065] In some embodiments, the audio visualizer 620 is configured as a three-dimensional (3D) visualization renderer that synthesizes a 3D representation of the audio system and its measured data. The visual representation 622 is a combination of a static background (e.g., venue seating) and a light-based beam overlay of sound coverage from one or more sound sources within the venue, as shown in Figures 2-5. In some embodiments, to render the static background for the visual representation 622, the audio visualizer 620 uses the audio system configuration to look up the source location and direction of each incoming audio signal and renders a set of visual markers at their respective spatial origins. For each sound, the audio visualizer 620 then uses the audio signal properties to determine specific light visualizations, such as the color and shape of the light beams. The audio visualizer 620 can also render other static background reference points, such as venue walls, seats, and stage entrance openings (i.e., the portion of the theater stage in front of the curtain).

[0066] In some embodiments, the audio visualizer 620 can orient the virtual 3D background around the user by matching the 3D background to motion tracking sensors on the user device 600. These motion tracking sensors can include accelerometers, magnetometers, and gyroscopes. Additionally, the audio visualizer 620 can help the user navigate the virtual 3D background by using the six degrees of freedom (“DOF”) sensing capabilities on the user device 600. Additional global and indoor positioning systems, such as Bluetooth or ultra-wideband (“UWB”) beacons, can also be used to improve positioning performance.

[0067] In some embodiments, the visual representation 622 may be implemented as a 2D or 3D visualization. The visual representation 622 may include a light beam visualization of the path and coverage from an acoustic source (e.g., a speaker or beam). The visual representation 622 may also include other visualization techniques such as data overlays, heat maps, and time lapses. The visual representation 622 may also include a user interface (e.g., coverage area, measurement bars, calculated digital output values, measured SPL values, loudspeaker ID tags, seat section ID tags, etc.). Alternatively, or in addition, the visual representation 622 may add imagery of the venue or at least a portion of the venue as an output. The visual representation 622 may be displayed using any known display technology, such as a display monitor, a mobile computing device with a display, wearable technology (e.g., glasses), or an augmented reality (AR), virtual reality (VR), or mixed reality (MR) headset.

[0068] In some embodiments, the interaction module 624 can provide an interactive environment for interacting with the virtual event. The interaction module 624 can enable a user to move around the virtual event, view audio visualizations at various locations, and / or modify one or more parameters, characteristics, and / or attributes of sound sources, as described in FIGS. 2-5 above. The interaction module 624 can also provide various communication capabilities, e.g., audio, video, and / or data communication, to the user device 600. In these embodiments, the interaction module 624 can request that one or more communication sessions, e.g., audio, video, and / or data communication sessions, be established with other user devices in the spatial audio measurement system, in a manner substantially similar to that described in FIGS. 2-5 above, to enable real-world users of these user devices to communicate therein and interact with the virtual event.

[0069] In some embodiments, the venue manipulator 626 can receive real-world user interactions with the virtual event and apply user modifications to the audio system. The venue manipulator 626 can insert, e.g., overlay, various virtual graphical elements onto the virtual event view to allow the real-world user to interact with the virtual event. In some embodiments, these virtual graphical elements can outline various interactions, e.g., modifications, available to the real-world user. In these embodiments, these virtual graphical elements can include, to name a few, one or more radio buttons, one or more check boxes, one or more text boxes, one or more toggle switches, one or more pop-up menus, one or more lists, and / or any other suitable mechanism that allows the real-world user to interact. In some embodiments, the venue manipulator can recognize speech from the user of the user device 600 and add annotations to the virtual event view. In some embodiments, the venue manipulator 626 can insert, e.g., overlay, a virtual selection tool, such as those illustrated in FIGS. 2-5 above, onto the virtual event view to allow a real-world user to interact with the virtual event. In some embodiments, the interaction module 624 can control the operation of the virtual graphical elements and / or the virtual selection tool. In these embodiments, the interaction module 624 can receive various commands, such as “point and click” and / or “drag and drop,” to name a few. These commands can be entered by the user controller device 606 and / or gesture input. The venue manipulator 626 can then apply these various interactions to modify the audio properties of the virtual event and / or the audio system at the venue by sending modification data to the audio program 615.

[0070] Exemplary Operation of an Exemplary Spatial Audio Measurement System FIG. 7 illustrates a flowchart of an exemplary method of operation of a spatial audio measurement system according to some exemplary embodiments of the present disclosure. This disclosure is not limited to this operational description. Rather, it will be apparent to those skilled in the art that other operational control flows are within the scope and spirit of the present disclosure. The following discussion describes an exemplary operational control flow 700 for using an XR device to configure digital audio for venue acoustics and measure audio performance in the venue. The operational control flow 700 allows a real-world user to interact with a digital representation of an event, which is mapped onto a virtual model of the venue, and apply modifications to the venue. The operational control flow 700 can be executed by one or more computer systems, such as user devices 204 and 304.n as described in FIGS. 2 and 3 above, user devices 404 and 504.n as described in FIGS. 4 and 5 above, and / or user device 600 as described in FIG. 6 above, as described in further detail below.

[0071] At operation 702, operation control flow 700 reads digital audio content from an audio source, such as local computer storage, an external device, a streaming source, or a software package that produces an audio signal for metering. Common examples are digital audio workstation ("DAW") software packages and audio playback servers. Operation 702 reads the audio signal in real time, determines audio signal properties, and emits a simplified signal that can be used to drive metering visualizations, such as the overall signal's instantaneous level and its frequency content.

[0072] At operation 704, the action control flow 700 maps the digital visualization of the audio content onto a virtual model and generates a virtual event. The digital visualization of the event is substantially similar to the digital representation of the event as described in FIGS. 2-6 above. The virtual model of operation 704 may be an embodiment of the virtual model as described in FIGS. 2-6 above. In some embodiments, the action control flow 700 can retrieve and / or generate the virtual model and / or digital representation of the event, including updating the virtual model based on any modifications to parameters, characteristics, and / or attributes associated with the virtual event. In these embodiments, the action control flow 700 can map the digital representation of the event, including parameters, characteristics, and / or attributes associated with any real-world effects, onto the virtual model and generate the virtual event in a manner substantially similar to that described in FIGS. 2-6 above.

[0073] At operation 706, the operational control flow 700 compares multiple visualizations of audio layouts to account for the studio-to-venue transformation. For example, operation 706 can generate a visualization of the studio layout in which the audio was designed and a visualization of the venue layout in which the audio will be played. By overlaying the multiple audio layouts, a user can visualize how the audio is transformed from the studio to the venue. A user can make any modifications to the parameters, characteristics, and / or attributes associated with the virtual event to tailor the event for a specific venue.

[0074] At operation 708, the operational control flow 700 generates a virtual event view that combines a static background of the venue with a light-based beam overlay of a 3D visualization of the audio content at the three-dimensional spatial location of the virtual event from operations 704 and / or 706. Operation 708 determines the source location and audio signal properties of each incoming audio signal and renders a specific light visualization, such as the direction, color, and shape of the light beam. The three-dimensional spatial locations may correspond to physical locations in the venue, and thus the virtual event view at a location corresponds to a real-world view at the corresponding physical location in the venue. In the exemplary embodiment illustrated in FIGS. 2-6, the operational control flow 700 can process the virtual event from operations 704 and / or 706 as the virtual event plays out and provide a virtual event view at that location in a manner substantially similar to that described in FIGS. 2-6 above. In some embodiments, the generated virtual event view may also be based on parameters, characteristics, and / or attributes established or modified by the user device as described in FIGS. 2-6 above. For example, the visual graphical elements of the generated virtual event view may represent the simulated behavior of one or more parameters, characteristics, and / or attributes associated with the virtual event.

[0075] At operation 710, the operation control flow 700 can receive audio-measured interactions in the form of user input from a user device such as described in FIGS. 2-6 above, and represent these interactions in the virtual event view from operation 708. The operation control flow 700 can play back the virtual event from operations 704 and / or 706 to virtually simulate the event being presented at the venue. The operation control flow 700 can receive user input from a user device such as described in FIGS. 2-6 above while the virtual event view from operation 708 is being viewed (e.g., on a display of a user device such as described in FIGS. 2-6 above). In some embodiments, this user input can include instructions (e.g., from an input device of a user device such as described in FIGS. 2-6 above) to virtually move a virtual user around within the three-dimensional space of the virtual event from operation 704 and view a digital representation of the event at one or more locations in the three-dimensional space of the virtual event from operation 704. In some embodiments, the interaction may include virtually modifying one or more parameters, characteristics, and / or attributes of the virtual event from operations 704 and / or 706 when the virtual event view from operation 708 is viewed in a manner substantially similar to that described in FIGS. 2-6 above. The one or more parameters, characteristics, and / or attributes of operation 710 are substantially similar to, and represent one embodiment of, the one or more parameters, characteristics, and / or attributes as described in FIGS. 2-6 above. The modifications may be propagated to the venue's audio system to configure the audio properties. In some embodiments, operation control flow 700 may update the virtual event from operations 704 and / or 706 to reflect the modifications to one or more parameters, characteristics, and / or attributes of the virtual event from operations 704 and / or 706.For example, the action control flow 700 may update one or more virtual graphical elements of the virtual event view to reflect any modifications to one or more parameters, characteristics, and / or attributes of the virtual event. In these embodiments, the action control flow 700 may propagate the modifications to multiple user devices, allowing these multiple user devices to cooperatively interact with the virtual event from acts 704 and / or 706. In some embodiments, instead of broadcasting the updated virtual event, the action control flow 700 may transmit one or more modifications to the user devices as described in Figures 2-6 above. Each user device may then locally update the virtual event based on the received modifications.

[0076] An exemplary computer system that may be utilized to implement electronic devices within an exemplary venue 8 illustrates a simplified block diagram of a computer system suitable for use with the embodiments described herein, according to some exemplary embodiments of the present disclosure. Various electronic devices, such as user devices 204 and 304.n as described in FIGS. 2 and 3 above, user devices 404 and 504.n as described in FIGS. 4 and 5 above, and / or user device 600 as described in FIG. 6 above, can be implemented in hardware, firmware, software, or any combination thereof. The following discussion of FIG. 8 describes an exemplary computer system 800 that may be used for these electronic devices.

[0077] In the exemplary embodiment illustrated in FIG. 8, computer system 800 typically includes at least one processor 804, which communicates with several peripheral devices via bus subsystem 802. Typically, at least one processor 804 can include or be any of a microprocessor, a graphics processing unit, or a digital signal processor, and their electronic processing equivalents, such as an application-specific integrated circuit ("ASIC") or a field-programmable gate array ("FPGA"). As used herein, the term "processor" refers to a tangible data and information processing device that physically transforms data and information using sequence transformations (also referred to as "operations"). Data and information can be physically represented by electrical, magnetic, optical, or acoustic signals that can be stored, accessed, transferred, combined, compared, or otherwise manipulated by the processor. The term "processor" can refer to single processors as well as multi-core systems or multi-processor arrays, including graphics processing units, digital signal processors, digital processors, or combinations thereof. The processor can be, for example, electronic, comprising digital logic circuitry (e.g., binary logic), or analog (e.g., operational amplifiers). The processor may also operate to support performance of associated operations within a "cloud computing" environment or as "software as a service" (SaaS). For example, at least some of the operations may be performed by a group of available processors in a distributed or remote system, which are accessible via a communications network (e.g., the Internet) and via one or more software interfaces (e.g., application program interfaces (APIs)).

[0078] Computer system 800 typically includes an operating system such as Microsoft's Windows®, Sun Microsystems' Solaris®, Apple Computer's Mac OS, Linux®, or UNIX®. Computer system 800 also typically includes a basic input / output system (BIOS) and processor firmware. The operating system, BIOS, and firmware are used by the processor to control the subsystems and interfaces coupled to the processor. Typical processors compatible with these operating systems include the Pentium® and Itanium from Intel, the Opteron and Athlon from Advanced Micro Devices, and the ARM processor from ARM Holdings.

[0079] As illustrated in FIG. 8 , these peripheral devices may include user interface input devices 812, user interface output devices 810, a network interface subsystem 806, and a storage subsystem 814, which includes a file storage subsystem 816 and a memory subsystem 818. The input and output devices enable user interaction with computer system 800. In the exemplary embodiment illustrated in FIG. 8 , network interface subsystem 806 provides an interface to external networks, including an interface to a communications network 808, via which it is coupled to corresponding interface devices in other computer systems or machines. Communications network 808 may comprise many interconnected computer systems, machines, and communications links. These communications links may be wired links, optical links, wireless links, or any other devices for communicating information. Communications network 808 can be any suitable computer network, for example, a wide area network such as the Internet and / or a local area network such as Ethernet. Communications network 808 can be wired and / or wireless, and the communications network can use encryption and decryption methods such as those available with virtual private networks. A communications network uses one or more communications interfaces, which can receive data from other systems and transmit data to them. Embodiments of communications interfaces typically include Ethernet cards, modems (e.g., telephone, satellite, cable, or ISDN), (asynchronous) digital subscriber line (DSL) units, Firewire interfaces, USB interfaces, and the like. One or more communications protocols, such as HTTP, TCP / IP, RTP / RTSP, IPX, and / or UDP, can be used.

[0080] The user interface input devices 812 may include pointing devices such as an alphanumeric keyboard, keypad, mouse, trackball, touchpad, stylus, or graphics tablet, scanner, touch screen integrated into the display, audio input devices such as a voice recognition system or microphone, eye-gaze recognition, electroencephalogram pattern recognition, and other types of input devices. Such devices can be connected to the computer system 800 by wire or wirelessly. In general, the use of the term "input device" includes all possible types of devices and methods intended to input information into the computer system 800 or over the communications network 808. The user interface input devices 812 typically allow a user to select objects, icons, text, and the like, which appear on some type of user interface output device, e.g., a display subsystem.

[0081] The user interface output devices 810 may include a display subsystem, a printer, a fax machine, or a non-visual display such as an audio output device. The display subsystem may include a cathode ray tube (CRT), a liquid crystal display (LCD) such as a flat panel device, a projection device, or some other device for creating a visible image such as a virtual reality system. The display subsystem may also provide a non-visual display, such as via audio output or tactile output (e.g., vibration) devices. In general, use of the term "output device" is intended to include all possible types of devices and methods for outputting information from the computer system 800 to a user or to another machine or computer system.

[0082] The memory subsystem 818 typically includes several memories, including a main random access memory ("RAM") 820 (or other volatile storage device) for storage of instructions and data during program execution, and a read-only memory ("ROM") 822 in which fixed instructions are stored. The file storage subsystem 816 provides persistent storage for program and data files and may include a hard disk drive, a floppy disk drive with associated removable media, a CD-ROM drive, an optical drive, a flash memory, or a removable media cartridge. Databases and modules implementing the functionality of certain embodiments may be stored by the file storage subsystem 816.

[0083] Bus subsystem 802 provides devices for allowing the various components and subsystems of computer system 800 to communicate with each other as intended. Although bus subsystem 802 is shown diagrammatically as a single bus, alternative embodiments of the bus subsystem may use multiple buses. For example, RAM-based main memory may communicate directly with a file storage system using a direct memory access ("DMA") system.

[0084] conclusion The detailed description has referred to the accompanying figures to illustrate exemplary embodiments consistent with this disclosure. References in this disclosure to "an exemplary embodiment" indicate that the described exemplary embodiment may include a particular feature, structure, or characteristic, but that not all exemplary embodiments necessarily include the particular feature, structure, or characteristic. Also, such phrases do not necessarily refer to the same exemplary embodiment. Furthermore, any feature, structure, or characteristic described in connection with an exemplary embodiment may be included independently or in any combination with features, structures, or characteristics of other exemplary embodiments, whether or not explicitly described.

[0085] The Detailed Description is not intended to be limiting. Rather, the scope of the present disclosure is defined solely by the following claims and their equivalents. It is understood that the Detailed Description section, and not the Abstract section, is intended to be used to interpret the claims. The Abstract section may describe one or more example embodiments of the present disclosure, but is not exhaustive, and thus is not intended to limit the present disclosure and the following claims and their equivalents in any way.

[0086] The exemplary embodiments described within this disclosure are provided for illustrative purposes and are not intended to be limiting. Other exemplary embodiments are possible, and modifications may be made to the exemplary embodiments while remaining within the spirit and scope of this disclosure. This disclosure is described with the help of functional components that illustrate implementations of defined functions and their relationships. The boundaries of these functional building blocks are arbitrarily defined herein for convenience of description. Alternative boundaries may be defined so long as the defined functions and relationships are appropriately performed.

[0087] Embodiments of the present disclosure can be implemented in hardware, firmware, a software application, or any combination thereof. Embodiments of the present disclosure can also be implemented as instructions stored on a machine-readable medium, which can be read and executed by one or more processors. A machine-readable medium can include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computing network). For example, a machine-readable medium can include non-transitory machine-readable media, such as read-only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, and others. As another example, a machine-readable medium can include transitory machine-readable media, such as electrical, optical, acoustic, or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.). Furthermore, firmware, software applications, routines, and instructions can be described herein as performing certain actions. However, it should be understood that such description is for convenience only and that such actions actually result from a computing device, processor, controller, or other device executing firmware, software applications, routines, instructions, etc.

[0088] The detailed description of the exemplary embodiments has fully revealed the general nature of the present disclosure, such that others, by applying the knowledge of those skilled in the art, may readily modify and / or adapt such exemplary embodiments for various applications without departing from the spirit and scope of the present disclosure and without undue experimentation. Moreover, such adaptations and modifications are intended to be within the meaning and equivalents of the exemplary embodiments, based on the teaching and guidance presented herein. It is to be understood that the phraseology or terminology used herein is for purposes of description and not of limitation, as the terminology or terminology used herein would be interpreted by one of ordinary skill in the art in light of the teachings herein.

Claims

1. 1. A computer-implemented method comprising: retrieving, by a first user device, audio content from an audio source; mapping, by the first user device, the digital visualization of the audio content onto a virtual model to generate a virtual event, the virtual model being a representation of a venue; and comparing, by the first user device, visualizations of audio layouts to illustrate studio-to-venue transformations; generating, by the first user device, a virtual event view, the generating including: determining a source location and audio signal properties of the audio content; mapping said audio content onto a volumetric light beam of unique direction, color, or shape; overlaying a visualization of the audio content at the venue location onto a static background of the venue in either an extended reality, virtual reality, augmented reality, or mixed reality environment; and receiving, by the first user device, a first user input for audio-measured interaction with the virtual event; updating the virtual event view based on the first user input; 11. A computer-implemented method comprising:

2. the virtual event view corresponds to a location of the virtual event; The computer-implemented method of claim 1 , wherein the location of the virtual event corresponds to a physical location of the venue.

3. The computer-implemented method of claim 1 , further comprising receiving a second user input from a second user device for audio-metered interaction with the virtual event.

4. transmitting any user input to any user device; The computer-implemented method of claim 3 , wherein any user input is configured to update a respective virtual event view at a corresponding user device.

5. Updating the virtual event view further comprises: modifying the volumetric light beam based on the first user input; displaying the modified volumetric light beam within the virtual event view; The computer-implemented method of claim 1 , comprising:

6. The computer-implemented method of claim 1 , further comprising configuring audio signal properties of audio sources at the venue by propagating user input to the venue.

7. The computer-implemented method of claim 1 , further comprising measuring audio signal properties of an audio source at the venue by using a microphone at the venue.

8. The computer-implemented method of claim 1 , wherein one of the audio signal properties of the audio content includes any of a signal volume level, a frequency, a range of frequencies, a signal channel, or a spatial coverage area.

9. The computer-implemented method of claim 1 , wherein receiving the first user input further comprises receiving an annotation visible by the user for propagation across a virtual event view.

10. 1. A system comprising: Memory and a processor, coupled to the memory, configured to perform operations; Equipped with The operation is retrieving, by a first user device, audio content from an audio source; mapping, by the first user device, the digital visualization of the audio content onto a virtual model to generate a virtual event, the virtual model being a representation of a venue; and comparing, by the first user device, visualizations of audio layouts to illustrate studio-to-venue transformations; generating, by the first user device, a virtual event view, the generating including: determining a source location and audio signal properties of the audio content; mapping said audio content onto a volumetric light beam of unique direction, color, or shape; overlaying a visualization of the audio content at the venue location onto a static background of the venue in either an extended reality, virtual reality, augmented reality, or mixed reality environment; and receiving, by the first user device, a first user input for audio-measured interaction with the virtual event; updating the virtual event view based on the first user input; Including, the system.

11. the virtual event view corresponds to a location of the virtual event; The system of claim 10 , wherein the location of the virtual event corresponds to the physical location of the venue.

12. The system of claim 10 , wherein the operations further include receiving a second user input from a second user device for audio-metered interaction with the virtual event.

13. The operations further include transmitting any user input to any user device; The system of claim 12 , wherein any user input is configured to update a respective virtual event view on a corresponding user device.

14. Updating the virtual event view further comprises: modifying the volumetric light beam based on the first user input; displaying the modified volumetric light beam within the virtual event view; The system of claim 10, comprising:

15. The system of claim 10 , wherein the actions further include configuring audio signal properties of an audio source at the venue by propagating user input to the venue.

16. The system of claim 10 , wherein the operations further include measuring audio signal properties of an audio source at the venue by using a microphone at the venue.

17. The system of claim 10 , wherein one of the audio signal properties of the audio content includes any of a signal volume level, a frequency, a range of frequencies, a signal channel, or a spatial coverage area.

18. The system of claim 10 , wherein receiving the first user input further comprises receiving an annotation visible by the user for propagation across a virtual event view.

19. A non-transitory computer-readable device storing instructions that, when executed by at least one computing device, cause the at least one computing device to perform operations, including: retrieving, by a first user device, audio content from an audio source; mapping, by the first user device, the digital visualization of the audio content onto a virtual model to generate a virtual event, the virtual model being a representation of a venue; and comparing, by the first user device, visualizations of audio layouts to illustrate studio-to-venue transformations; generating, by the first user device, a virtual event view, the generating including: determining a source location and audio signal properties of the audio content; Mapping audio content to a unique direction, color, or shape of a volumetric light beam; overlaying a visualization of the audio content at the venue location onto a static background of the venue in either an extended reality, virtual reality, augmented reality, or mixed reality environment; and receiving, by the first user device, a first user input for audio-measured interaction with the virtual event; updating the virtual event view based on the first user input; a non-transitory computer readable device comprising:

20. 20. The non-transitory computer-readable device of claim 19, wherein the actions further include configuring audio signal properties of an audio source at the venue by propagating user input to the venue.

Images