Method and system for spatial audio metrology using augmented reality devices

Abstract

Systems, methods, and computer program product embodiments are provided herein for providing an interactive visual representation of loudspeaker sound coverage in a venue. Embodiments enable temporal, spectral, and spatial audio metrology from digital audio to venue acoustics. The techniques may use light to visualize the audio capabilities of one or more speaker arrays, thereby providing a visualized grouping of large audio channel outputs. This may allow the spatial information to be visually represented by mapping the directional sound to the light beam.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. patent application Ser. No. 18 / 114,089, filed on February 24, 2023, the entire contents of which are incorporated herein by reference. Background Art

[0003] As new spatial audio technologies enable more control options for directional sound, conventional spatial audio metering methods for large venues can become more time-consuming and logistically demanding. For example, spatial audio metering in large venues with high channel counts can be plagued by numerous obstacles, as it's impossible to configure directional sound output from a single location simply by listening. These obstacles stem from current sound production workflows and can be categorized into three groups: time metering (e.g., loudness), spectral metering (e.g., spectrum), and spatial metering (e.g., coverage). A barrier to time metering is a lack of confidence in loudness when reviewing live, in-studio, and with headphones. A barrier to spectral metering is a lack of tools for matching spectral content to the output capabilities of a loudspeaker or speaker array. A barrier to spatial metering is a lack of clear visual aids for creating spatial audio content, adjusting studio-to-venue transitions, and calibrating playback systems for large venues. Consequently, skilled artisans face challenges configuring audio systems and personally measuring audio performance, especially during the planning or construction phases of a venue. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Figure 1 Illustrated is a graphical representation of an exemplary venue in accordance with some embodiments.

[0005] Figure 2A 、 Figure 2B and Figure 3 Illustrated is an example visualization of a spatial audio metering system configured to operate using mixed reality device(s), in accordance with some embodiments.

[0006] Figure 4-Figure 5 Illustrated is an example visualization of a spatial audio metering system using mixed reality device(s) for measurement operations, in accordance with some embodiments.

[0007] Figure 6 An exemplary mixed reality user device is illustrated in accordance with some embodiments.

[0008] Figure 7 A flow chart illustrating an example method of operating a spatial audio metering system using an augmented reality device in accordance with some embodiments is illustrated.

[0009] Figure 8A simplified block diagram of a computer system suitable for use with the embodiments described herein is illustrated, according to some exemplary embodiments of the present disclosure. DETAILED DESCRIPTION

[0010] The following disclosure provides many different embodiments or examples for implementing different features of the provided subject matter. To simplify the disclosure, specific examples of components and arrangements are described below. Of course, these examples are merely illustrative and are not intended to be limiting. In addition, the disclosure may repeat reference numerals and / or letters in various examples. Such repetition itself does not determine the relationship between the various embodiments and / or configurations discussed.

[0011] Overview

[0012] Provided herein are system, apparatus, device, method, and / or computer program product embodiments, and / or combinations and sub-combinations thereof, for providing an interactive visual representation of speaker sound coverage in a venue. In some embodiments, the technology described herein can enable temporal, spectral, and spatial audio metering from digital audio to venue acoustics. The technology can use light to visualize the audio capabilities of one or more speaker arrays, providing a visual grouping of large audio channel outputs. This can allow spatial information to be visually represented by mapping directional sound to light beams.

[0013] Exemplary Sites

[0014] Figure 1 A graphical representation of an exemplary venue according to some exemplary embodiments of the present disclosure is illustrated. In some embodiments, venue 100 represents a location where an event is held. For example, venue 100 may represent a music venue, such as a musical theater, music club, and / or concert hall; a sports venue, such as an arena, convention center, and / or stadium; and / or any other suitable venue that would be apparent to one skilled in the relevant art(s) without departing from the spirit and scope of the present disclosure. Events may include musical events, theatrical events, sporting events, movies, and / or any other suitable event that would be apparent to one skilled in the relevant art(s) without departing from the spirit and scope of the present disclosure. In some embodiments, venue 100 may represent a three-dimensional structure, such as a hemispherical structure, also known as a semispherical dome.

[0015] like Figure 1As shown in , venue 100 may also include speakers 102.1 through 102.i for playback of audio or sounds associated with the event. In some embodiments, speakers 102.1 through 102.i may include a proscenium array speaker system located at or near the proscenium of venue 100, one or more effects extension array speaker systems located at or near the proscenium array speaker system, and / or one or more ambient array speaker systems located throughout venue 100. In some embodiments, the proscenium array speaker system, one or more effects extension array speaker systems, and / or one or more ambient array speaker systems may include one or more line array speakers, which may include one or more super tweeters, one or more tweeters, one or more midrange speakers, one or more woofers, one or more subwoofers, and / or one or more full-range speakers, to provide some examples. In these embodiments, the one or more line array speakers may be implemented using audio beamforming techniques to generate sound waves that provide sound throughout venue 100. Generally speaking, the one or more line arrays may direct these sound waves to various three-dimensional areas within venue 100, thereby providing sound to one or more seating areas 104. In some embodiments, one or more seating areas 104 may allow spectators to sit at various locations within venue 100 to experience the event.

[0016] As will be described in further detail below, venue 100 may implement a spatial audio metering system to estimate the coverage of speakers 102.1 through 102.i across venue 100. In some embodiments, the spatial audio metering system allows for a visual estimation of the coverage of speakers 102.1 through 102.i. Generally speaking, the human eye is able to detect approximately 4x10 14 Hertz (Hz) to 8x10 14While the sound waves generated by speakers 102.1 through 102.i lie in the auditory spectrum, between approximately 20 Hz and 20 kHz. Consequently, the sound waves generated by speakers 102.1 through 102.i are generally undetectable to the human eye. As described in further detail below, venue 100 may implement a spatial audio metering system to project light in the visible spectrum consistent with the sound waves generated by speakers 102.1 through 102.i. This allows for a visual assessment of the coverage of speakers 102.1 through 102.i across venue 100. In some embodiments, the spatial audio metering system may be used to adjust the spatial arrangement of speakers 102.1 through 102.i within venue 100 to accommodate incomplete or defective areas (also referred to as gaps) in the coverage areas of speakers 102.1 through 102.i. As part of this spatial audio metering system, venue 100 can project light consistent with the sound waves generated by speakers 102.1 through 102.i in an extended reality (XR) environment (such as an augmented reality (AR) environment, a virtual reality (VR) environment, and / or a mixed reality (MR) environment). As part of this spatial audio metering system, venue 100 can measure the sound waves generated by speakers 102.1 through 102.i. In some embodiments, venue 100 can analyze these measurements to modify one or more parameters, characteristics, and / or properties of the sound waves generated by speakers 102.1 through 102.i. In these embodiments, the spatial audio metering system can be used to modify the sound waves generated by speakers 102.1 through 102 to provide a similar event auditory experience for audiences at venue 100.

[0017] An exemplary spatial audio metering system using a mixed reality device

[0018] Figure 2A A graphical representation of an exemplary spatial audio metering system according to some exemplary embodiments of the present disclosure is illustrated. In some embodiments, the spatial audio metering system 200 can include a speaker 250 and a virtual speaker projector 252. In these embodiments, the spatial audio metering system 200 can project a beam 256 generated by the virtual speaker projector 252 onto a target area 258 so that it coincides with an audio beam 254 generated by the speaker 250. As will be described in further detail below, one or more real-world users of the spatial audio metering system 200 can configure and play back audio or sounds associated with an event on the speaker 250. While viewing the beam 256, the one or more real-world users can virtually interact with the spatial audio metering system 200, for example, by moving around the virtual event to view the beam 256 at various locations and / or modifying one or more parameters, characteristics, and / or properties of the speaker 250.

[0019] In some embodiments, the spatial audio metering system 200 may use a virtual speaker projector 252 to provide a 3D representation of the audio transmission pattern generated by the speaker 250. Figure 1 As described above, the audio beam 254 generated by the speaker 250 is typically undetectable to the human eye. To provide a visual aid for audio detection, the virtual speaker projector 252 can approximate the properties of the audio beam 254 (e.g., direction, size, shape, etc.) to generate a beam 256 that visualizes these properties of the audio beam 254. The virtual speaker projector 252 can then project the beam 256 onto a target area 258 in a manner suitable for rendering on a device such as a head-mounted display, so that it coincides with the audio beam 254, to visually illustrate the projection of the audio beam 254. This allows for visual estimation of the coverage of the speaker 250 within the venue.

[0020] In some embodiments, the spatial audio meter system 200 may include a microphone 260 for measuring the audio beam 254 generated by the speaker 250. The microphone 260 may detect sound vibrations in the air and convert the sound vibrations into electronic signals, thereby providing feedback information to the spatial audio meter system 200. The spatial audio meter system 200 may analyze these measurements from the microphone 260 to modify one or more parameters, characteristics, and / or properties of the audio beam 254 generated by the speaker 250.

[0021] Figure 2B A graphical representation of an exemplary spatial audio metering system 200, according to some exemplary embodiments of the present disclosure, is illustrated. A spatial audio metering system 200 can project a beam 208 corresponding to audio or sounds associated with an event (such as a musical event, a theatrical event, a sporting event, and / or a film event, to name a few) onto a target area 210 of a venue 202 for viewing by a user via a user device 204. As will be described in further detail below, one or more real-world users of the spatial audio metering system 200 can configure and playback selected sound sources in an extended reality (XR) environment (such as an augmented reality (AR) environment, a virtual reality (VR) environment, and / or a mixed reality (MR) environment) to test the selected sound sources at the venue 202 or to simulate an event presented at the venue 202 as a virtual event. While the one or more real-world users view the audio visualization, the users can virtually interact with the audio visualization, for example, by moving around the virtual event to view the audio visualization at various locations and / or modifying one or more parameters, characteristics, and / or properties of the sound sources.

[0022] In some embodiments, user device 204 may include one or more computing devices, such as one or more desktop computers, one or more mobile phones, one or more mobile computing devices; one or more mobile internet devices, such as tablet computers and / or laptop computers; one or more mobile video game consoles; one or more mobile wearable electronic devices, such as smart watches; and / or any other computing device with one or more processors that one skilled in the relevant art(s) will recognize without departing from the spirit and scope of the present disclosure, to provide some examples. In some embodiments, these one or more computing devices may be communicatively coupled to one or more virtual reality (VR) headsets and / or one or more VR controllers.

[0023] In some embodiments, user device 204 may process a virtual event that may be viewed independently or overlaid on a view of venue 202 by user device 204. This processing may include tracking each user's three-dimensional position in the three-dimensional space of the virtual event; estimating each user's line of sight at the three-dimensional position; estimating each user's field of view associated with the line of sight; and / or matching the virtual event view with each user's field of view at the three-dimensional position.

[0024] In some embodiments, a real-world user of user device 204 can move around in the view of the virtual event. In some embodiments, these interactions can include virtually moving a virtual user corresponding to the real-world user in the three-dimensional space of the virtual event to view beams 208 at various locations. In some embodiments, these various locations can include locations in the three-dimensional space of the virtual event that would not normally be visible from venue 202, such as broadcast cinematic camera locations, to provide examples. In some embodiments, the real-world user can move around venue 202 while the view of user device 204 updates in real time to align the virtual event overlay with the user's view of venue 202.

[0025] In some embodiments, spatial audio metering system 200 may include light beams 208. These light beams 208 can serve as visual representations of real-world sound effects associated with actual events, providing a visual aid for real-world sound effects that are typically invisible to the human eye. While these various real-world sound effects can be detected by audio equipment, the exact coverage areas for the selected content may be difficult to understand. Furthermore, if a sound coverage map within a virtual environment is desired, physical measurements of sound coverage patterns are not applicable. Therefore, techniques as disclosed herein use volumetric light beams within an image of venue 202 to visualize parameters, characteristics, and / or properties of sound coverage.

[0026] In some embodiments, a virtual event can represent real-world audio properties via beams 208, which can be visual virtual effects such as arrows, lines, or any other visual effects that can be displayed within the virtual event. For example, beams 208 for directional audio (e.g., beamforming) can depict the direction of audio from a source to an intended target area 210, as well as the interaction of the audio with the venue's architecture. When parameters, characteristics, and / or properties of these real-world effects are modified (e.g., by user device 204), beams 208 can be updated to represent the modification. For example, an updated beam 208 can represent the new direction or target of the directional audio.

[0027] In some embodiments, light beam 208 may represent a focused beam sound source that can be configured to be heard by a target area 210 of seating areas in venue 202. In some embodiments, the sound beam is generated by a piezoelectric or electrostatic transducer (or array thereof). However, any known method for generating a sound beam for location-specific audio coverage may be substituted without departing from the scope of the technology described herein. For example, visualization of such sound coverage patterns would provide a sound engineer with a way to quickly configure sound coverage for a specific seating area.

[0028] In some embodiments, beam 208 may represent a wide-beam sound source that can be configured to be heard by target area 210 across the entire seating area in venue 202. In some embodiments, the wide sound beam may be generated by an array of piezoelectric or electrostatic transducers. However, any known method for generating a wide sound beam for location-specific audio coverage may be substituted without departing from the scope of the technology described herein. For example, visualization of this sound coverage pattern would provide a sound engineer with a way to quickly configure sound coverage for all seating areas.

[0029] In some embodiments, light beam 208 may represent a diffuse beam sound source that can be configured to be heard by a target area 210 of seating areas in venue 202. Diffusion of sound can be important to avoid dead spots (places where sound is weak or cannot be heard clearly). A diffuse beam is generated by scattering sound through surface variations of the sound source (such as deflecting or scattering surfaces). However, any known method for generating a diffuse beam for location-specific audio coverage may be substituted without departing from the scope of the technology described herein. For example, visualization of this sound coverage pattern would provide a sound engineer with a way to quickly configure sound coverage for a focused seating area.

[0030] In some embodiments, the light beam 208 representing the coverage of the sound source can have any geometric shape, aperture size, amount of light, or other geometric light properties, as defined by the source type and / or the shape of the sound profile. For example, a focused beam can begin at a point and expand in a trapezoidal shape as it extends from that point, while a diffuse beam can begin in a rectangular shape and expand in a trapezoidal shape. In another example, a focused beam can begin at a point and expand in a conical shape, thereby forming a circular target area 210.

[0031] In some embodiments, beams 208 can be assigned unique colors that can symbolize certain audio properties (e.g., frequency) of the corresponding sound source. While specific color assignments and frequency ranges are described below, these ranges and color assignments may vary without departing from the scope of the technology described herein. For example, other colors, frequencies, and color intensity gradients may be selected as desired, as long as individual colors are assigned to defined audio sources, channels, ranges, etc.

[0032] In the first approach, the seven known colors of the visual spectrum are mapped to seven known audio frequency ranges. For example, sunlight is broken down into seven colors: violet, indigo, blue, green, yellow, orange, and red (VIBGYOR). Furthermore, sound is broken down into commonly labeled ranges: sub-bass (16-60Hz), bass (60-250Hz), mid-bass (250-500Hz), mid-range (500-2kHz), mid-treble (2-4kHz), presence (4-6kHz), and bright (6kHz-20kHz). To illustrate volume intensity, color saturation or brightness can increase proportionally with volume. In this approach, each audio frequency range is assigned a different color. For example, as shown, the darkest colors are assigned to low-frequency components, and brighter colors are assigned to high-frequency components. These specific color and frequency range assignments are for illustrative purposes only and can be modified to achieve different visualizations.

[0033] In this first approach, the speakers may have a dedicated purpose, such as providing bass. In this scenario, each dedicated bass source can be visually mapped using a common color to illustrate the overall bass coverage in the venue. Alternatively or additionally, each sound source can generate multiple audio ranges, and the visualization can include one or more audio ranges, using a common color for similar ranges. For example, venue 202 may have 20 sound sources, each providing at least a first audio range and a second audio range. A unique color can be assigned to each of the first and second audio ranges, and a visualization can be generated for each audio range, or for both ranges in a combined visualization. This approach can be applied to any audio range, combination of ranges, or specific sound effects.

[0034] Alternatively or additionally, in a second approach, individual sound sources (e.g., speaker arrays or beams) can be collectively assigned individual colors to allow differentiation of individual sound sources when overlapping sound coverage occurs in the venue. In this approach, no audio ranges are assigned colors.

[0035] Alternatively or additionally, the two approaches can be combined. Those skilled in the art will recognize that other approaches or combinations can be implemented using the techniques described herein without departing from the scope of the present disclosure.

[0036] In the exemplary embodiment shown in Figure 2, user device 204 can configure and playback virtual events to virtually prepare and simulate events presented at the venue. In some embodiments, these interactions can include virtually modifying the virtual event independently or superimposed on venue 202 while the user is watching the virtual event. In some embodiments, these interactions can include modifying the physical audio system of venue 202. In order to input modifications in these various embodiments, the user can interact with user configuration interface 206 to change the time, spectrum and spatial properties of the audio system of the virtual event and / or real-world event. In some embodiments, user configuration interface 206 can allow the user to stop, pause, fast forward and / or rewind the virtual event.

[0037] In some embodiments, the user configuration interface 206 of the user device 204 can provide various virtual graphic elements to the real world user to allow these users to modify the virtual event. In these embodiments, these virtual graphic elements of the user configuration interface 206 can summarize the various interactions available to the real world user, such as modifications. In these embodiments, these virtual graphic elements of the user configuration interface 206 can include one or more radio buttons, one or more check boxes, one or more text boxes, one or more toggle switches, one or more pop-up menus, one or more lists and / or any other suitable mechanism that allows the real world user to interact, only providing some examples. For example, these modifications can include removing one or more parameters, characteristics and / or attributes of the virtual event from the three-dimensional space of the virtual event. As another example, these modifications can include moving one or more parameters, characteristics and / or attributes of the virtual event in the three-dimensional space of the virtual event, such as position and / or orientation. As another example, these modifications can include inserting one or more new parameters, new characteristics and / or new attributes in the three-dimensional space of the virtual event. In some embodiments, the parameters, characteristics and / or attributes of the virtual event may include or relate to: temporal, spectral, and spatial properties of the venue's audio; one or more computer-generated digital models of various architectural features of the venue; one or more computer-generated digital models of various objects; one or more computer-generated models of one or more performers; one or more computer-generated models of one or more props associated with the event; and / or other suitable parameters, characteristics and / or attributes of the virtual event that will be apparent to a person skilled in the relevant art(s) without departing from the spirit and scope of the present disclosure.

[0038] In some embodiments, a user can modify parameters, characteristics, and / or properties of virtual sound effects via a user configuration interface 206 provided by user device 204. For example, a user can modify parameters, characteristics, and / or properties of a virtual event, such as the audio direction of a speaker, at user device 204. Such modifications can alter the virtual event to display a new representation of the revised audio direction within venue 202. In this manner, spatial audio metering system 200 can simulate the behavior of real-world sound effects within a particular venue and display this simulated behavior as a virtual graphical element. As described below, spatial audio metering system 200 can perform various audio configuration tasks.

[0039] In some embodiments, the spatial audio metering system 200 can perform time-based measurement for audio configuration tasks. Users of the spatial audio metering system 200 can set the volume of a sound source through various input methods in the user configuration interface 206. For example, the user can use voice or gesture input to adjust the volume level of a sound source. As the user configures time-based properties, the user configuration interface 206 can display a loudness meter, decibel (dB) values, labels, and / or other indicators of the time-based properties. In response to the configuration inputs, the user can view the beam 208 and / or data overlays. Thus, by enabling users to perform configuration tasks using live loudness visualization, the spatial audio metering system 200 can reduce logistics requirements and time budgets, thereby increasing confidence in loudness at the audit site.

[0040] In some embodiments, the spatial audio metering system 200 can perform spectral metering for audio configuration tasks. Users of the spatial audio metering system 200 can set the frequency of a sound source and the balance of selected frequencies (referred to as equalization (EQ)) through various input methods in the user configuration interface 206. For example, the user can use voice or gesture input to adjust the EQ of a sound source. As the user configures spectral properties, the user configuration interface 206 can display a frequency meter, EQ controls, labels, and / or other spectral property indicators. In response to configuration inputs, the user can view the beam 208 and / or data overlays. Thus, the spatial audio metering system 200 can reduce logistical requirements and time budgets by enabling users to perform configuration tasks using live spectral visualization, thereby allowing users to match spectral content with the output capabilities of the venue's speakers.

[0041] In some embodiments, the spatial audio metering system 200 can perform spatial metering for audio configuration tasks. Users of the spatial audio metering system 200 can configure the coverage of sound sources through various input methods in the user configuration interface 206. For example, a user can use voice or gesture input to adjust the directional sound beam of a sound source to avoid acoustic reflections. In another example, a user can preview the acoustic arrangement or directional sound movement at different playback speeds. In another example, a user can overlay the beams 208 to determine the differences in coverage and overlap resulting from various speaker layouts for studio-to-venue conversions. As the user configures spatial properties, the user configuration interface 206 can display coverage areas (e.g., incident audio beams, reflected audio beams, and / or overlapping audio beams), frequency meters, loudness meters, labels, and / or other spatial indicators. In response to configuration inputs, the user can view the beams 208, a timelapse of the virtual event, a heat map, and / or data overlays. Thus, the spatial audio metering system 200 can reduce logistics requirements and time budgets by enabling users to perform configuration tasks using live spatial visualizations, thereby providing a visual aid for creating spatial audio content, adjusting studio-to-venue conversions, and calibrating large-venue playback systems.

[0042] Figure 3 A diagram illustrates an exemplary spatial audio metering system using multiple MR devices, according to some exemplary embodiments of the present disclosure. In this exemplary embodiment, spatial audio metering system 300 can allow multiple real-world users to view and configure a virtual event from their respective user devices 304.1 through 304.n, either from the same location or from different locations within the three-dimensional space of a venue 302, via an extended reality (XR) environment (such as an augmented reality (AR) environment, a virtual reality (VR) environment, and / or a mixed reality (MR) environment). The above discussion regarding spatial audio metering system 200 can be applied to spatial audio metering system 300. Spatial audio metering system 300 can perform the various audio configuration tasks disclosed above with respect to FIG. 2 .

[0043] In some embodiments, the spatial audio metering system 300 can provide a visual representation of the sound coverage of multiple sound sources in the venue 302. Figure 3 As shown in FIG, first light beam 308, second light beam 310, third light beam 312, and fourth light beam 314 can be configured to be heard by selected target areas 318, 320, 316, and 322, respectively, of venue 302. In such an arrangement, one or more of the respective various sound coverages may overlap. In such a scenario, visualization of the sound sources will help to understand where coverage exists or where gaps may exist. Adjusting the sound source configuration (such as, but not limited to, type, directionality, size, shape, power, location, or spectral properties) may be necessary to fill any gaps or eliminate unnecessary overlaps. Any known sound source may be substituted without departing from the scope of the technology described herein.

[0044] In some embodiments, the spatial audio metering system 300 can perform multi-zone content configuration using multiple user devices 304.1 through 304.n. By interacting with the user configuration interface 306, a user can simultaneously playback and view beams 308 through 314 at multiple target zones 316 through 322 using multiple user devices. In some embodiments, a user can monitor crosstalk or signal leakage between adjacent target zones 316 through 322. For example, a single user can simultaneously listen to audio content, monitor sound pressure level (SPL) and signal leakage, and annotate the audio quality at multiple target zones 316 through 322. As a user configures multi-zone content properties, the user configuration interface 306 can display coverage areas, frequency meters, loudness meters, labels, and / or other audio property indicators. In response to configuration inputs, the user can view beams 308 through 314 and / or data overlays. Thus, the spatial audio metering system 300 can reduce logistics requirements and time budgets by enabling more than one user to simultaneously perform configuration tasks and perform live visualization and annotation interactions.

[0045] In some embodiments, spatial audio metering system 300 can propagate these interactions and modifications to multiple real-world users of spatial audio metering system 300, allowing these real-world users to collaboratively interact with the virtual event. A user at user device 304.1 may wish to collaborate with another user at user device 304.2 regarding interactions with light beams 308 through 314. User device 304.1 can initiate a communication with user device 304.2, which may include modifying the virtual event. For example, user device 304.1 may send an instruction to user device 304.2 to change the direction of light beam 308. As another example, user device 304.1 may transmit a text-based communication to user device 304.2 that includes an image of the interaction with light beams 308 through 314. When applying configuration modifications, user device 304.1 may transmit the configuration modifications to user devices 304.2 through 304.n. Upon receiving the modifications, user devices 304.2 through 304.n may update their respective views of the virtual event based on the modifications. The update includes displaying light beams 308 to 314 representing the new audio direction in the corresponding virtual event view of each user device 304 . n .

[0046] Figure 4 A graphical representation of an exemplary spatial audio metering system using MR devices, according to some exemplary embodiments of the present disclosure, is illustrated. In this exemplary embodiment, spatial audio metering system 400 can allow one or more real-world users to view and measure a virtual event from their respective user devices 404.1 through 404.n, either from the same location or from different locations within the three-dimensional space of a venue 402, via an extended reality (XR) environment (such as an augmented reality (AR) environment, a virtual reality (VR) environment, and / or a mixed reality (MR) environment). The above discussion regarding spatial audio metering system 200 can be applied to spatial audio metering system 400.

[0047] In some embodiments, the spatial audio metering system 400 may include one or more microphones 412. Microphones 412 may detect sound vibrations in the air and convert them into electronic signals to provide feedback information to the spatial audio metering system 400. In some embodiments, microphones 412 may be fixed. For example, microphones 412 may be attached to a seating area within the target area 410. In some embodiments, microphones 412 may be portable. For example, microphones 412 may be attached to an unmanned aerial vehicle (UAV) or user device 404, allowing microphones 412 to be moved throughout the venue 402.

[0048] In some embodiments, the user measurement interface 406 of the user device 404 can provide various virtual graphical elements to the real-world user to allow these users to modify the virtual event. The user device 404 can receive feedback information from the microphone 412 and pass this feedback information to the user measurement interface 406, thereby generating virtual graphical elements for the user. These virtual graphical elements of the user measurement interface 406 can summarize various types of feedback information, such as the volume, frequency, and coverage of the audio detected in the venue 402. As described in the discussion of Figure 2, the user measurement interface 406 can allow the user to modify the audio properties.

[0049] In some embodiments, spatial audio metering system 400 can perform quality control operations by performing full-scale measurement tasks on the audio system of venue 402. Users can view identification marks for each speaker and annotate the speaker's status using gestures or voice input. For example, a user can use user device 404 to play test tones, listen to the real-time output, and annotate the speaker's results. Through annotations, the user can indicate functioning speakers and malfunctioning speakers within venue 402. This feature of spatial audio metering system 400 can be useful in large venues where locating the origin of sounds is difficult by listening alone, and it is inefficient for an operator to walk to each speaker for individual inspection. Thus, spatial audio metering system 400 can reduce logistics requirements and time budgets by enabling users to perform quality control tasks (such as measurement, inspection, and annotation) through on-site visualization.

[0050] In some embodiments, the spatial audio metering system 400 can perform temporal measurement of audio measurement tasks. A user of the spatial audio metering system 400 can use input from microphone 412 (which can function as an SPL meter) to calculate loudness. For example, the user can compare the digital gain level in the user measurement interface 406 with the SPL measured using microphone 412. In response to the measurement information, the user can adjust the volume of the sound source through various input methods in the user measurement interface 406. For example, the user can use voice or gesture input to adjust the volume level of the sound source. When the user measures temporal properties, the user measurement interface 406 can display a loudness meter, decibel (dB) values, labels, and / or other temporal property indicators. In response to the measurement input, the user can view the beam 408 and / or data overlay. Thus, by enabling users to perform measurement tasks using live loudness visualization based on input from microphone 412, the spatial audio metering system 400 can reduce logistics requirements and time budgets, thereby increasing confidence in loudness at the audit site.

[0051] In some embodiments, the spatial audio metering system 400 can perform spectral measurement for audio measurement tasks. A user of the spatial audio metering system 400 can use input from microphone 412 (which can function as a spectral analyzer) to calculate spectral frequencies. For example, the user can compare the spectral frequencies calculated in the user measurement interface 406 with the frequency response measured using microphone 412. In response to the measurement information, the user of the spatial audio metering system 200 can adjust the frequency of the sound source and the balance of the selected frequencies (referred to as equalization (EQ)) through various input methods in the user measurement interface 406. For example, the user can use voice or gesture input to adjust the EQ of the sound source. While the user is measuring spectral properties, the user measurement interface 406 can display a frequency meter, EQ controls, labels, and / or other spectral property indicators. In response to the measurement input, the user can view the beam 408 and / or data overlay. Thus, the spatial audio metering system 400 can reduce logistical requirements and time budgets by enabling users to perform measurement tasks using live spectral visualization based on input from microphone 412, thereby allowing users to match spectral content with the output capabilities of the venue's speakers.

[0052] In some embodiments, the spatial audio metering system 400 can perform spatial metering for audio measurement tasks. A user of the spatial audio metering system 400 can use input from microphone 412 to calculate coverage, loudness, or spectral frequencies. For example, the user can compare the calculated spectral frequencies with the frequency responses measured by microphone 412 at various seating areas to determine the coverage of specific frequencies. A heatmap visualization can provide an overview of the results of such inspections. In response to the measurement information, the user of the spatial audio metering system 400 can adjust the coverage of a sound source through various input methods in the user measurement interface 406. For example, the user can use voice or gesture input to adjust the directional beam of a sound source to avoid acoustic reflections. In another example, the user can preview acoustic arrangements or directional sound movement at different playback speeds. In another example, the user can overlay beams 408 to determine differences in coverage and overlap resulting from various speaker layouts across studio-to-venue transitions. As the user measures spatial properties, the user measurement interface 406 can display coverage areas (e.g., incident audio beams, reflected audio beams, and / or overlapping audio beams), frequency meters, loudness meters, labels, and / or other spatial indicators. In response to the measurement input, the user can view the beam 408, a time-lapse of the virtual event, a heat map, and / or a data overlay. Thus, the spatial audio metering system 400 can reduce logistical requirements and time budgets by enabling users to perform measurement tasks using live spatial visualization based on input from the microphone 412, thereby providing a visual aid for creating spatial audio content, adjusting studio-to-venue conversions, and calibrating large-venue playback systems.

[0053] Figure 5A graphical representation of an exemplary spatial audio metering system using multiple MR devices according to some exemplary embodiments of the present disclosure is illustrated. In this exemplary embodiment, the spatial audio metering system 500 may allow multiple real-world users to view and measure a virtual event from the same location or different locations in the three-dimensional space of a venue 502 from their respective user devices 504.1 to 504.n via an extended reality (XR) environment, such as an augmented reality (AR) environment, a virtual reality (VR) environment, and / or a mixed reality (MR) environment. To enhance measurement accuracy, multiple reference microphones 524 may be used with the user devices 504.1 to 504.n. The above discussion regarding the spatial audio metering system 400 may be applicable to the spatial audio metering system 500. The spatial audio metering system 500 may perform the above discussion regarding Figure 4 Various audio measurement tasks are publicly available.

[0054] In some embodiments, the spatial audio metering system 500 can provide a visual representation of the sound coverage of multiple sound sources in the venue 502. Figure 5 , a first light beam 508, a second light beam 510, a third light beam 512, and a fourth light beam 514 can be configured to be heard by selected target areas 518, 520, 516, and 522, respectively, of venue 502. In such an arrangement, one or more of the respective various sound coverages may overlap. In such a scenario, visualization of the sound sources will help to understand where coverage exists or where gaps may exist. Measuring the sound sources and adjusting the sound source configuration accordingly (such as, but not limited to, type, directionality, size, shape, power, location, or spectral properties) may be necessary to fill any gaps or eliminate unnecessary overlaps. Any known sound source may be substituted without departing from the scope of the technology described herein.

[0055] In some embodiments, the spatial audio metering system 500 can perform multi-zone content measurement using multiple user devices 504.1 through 504.n. By interacting with the user measurement interface 506, a user can use multiple user devices to simultaneously playback and view the beams 508 through 514 at multiple target zones 516 through 522 based on input from a microphone 524. In some embodiments, the user can monitor crosstalk or signal leakage between adjacent target zones 516 through 522 by detecting real-time playback using the microphone 524. For example, a user can simultaneously listen to audio content, monitor sound pressure level (SPL) and signal leakage, and annotate the audio quality at multiple target zones 516 through 522. While the user is measuring multi-zone content properties, the user measurement interface 506 can display coverage areas, frequency meters, loudness meters, labels, and / or other audio property indicators. In response to the measurement input, the user can view the beams 508 through 514 and / or data overlays based on the input from the microphone 524. Thus, the spatial audio metering system 500 may reduce logistical requirements and time budgets by enabling more than one user to simultaneously perform measurement tasks as well as interact with live visualization and annotation based on input from the microphone 524 .

[0056] In some embodiments, spatial audio metering system 500 can propagate these interactions and modifications to multiple real-world users of spatial audio metering system 500, allowing these real-world users to collaboratively interact with the virtual event. A user at user device 504.1 may wish to collaborate with another user at user device 504.2 regarding interactions with light beams 508 through 514. User device 504.1 can initiate a communication with user device 504.2, which may include modifying the virtual event. For example, user device 504.1 may send an instruction to user device 504.2 to change the direction of light beam 508. As another example, user device 504.1 may transmit a text-based communication to user device 504.2 that includes an image of the interaction with light beams 508 through 514. Upon applying the measurement modifications, user device 504.1 may transmit the measurement modifications to user devices 504.2 through 504.n. Upon receiving the modifications, user devices 504.2 through 504.n may update their respective views of the virtual event based on the modifications. The update includes displaying light beams 508 to 514 representing the new audio direction in the corresponding virtual event view of each user device 504 . n .

[0057] Example Spatial Audio Metering Mixed Reality User Device

[0058] Figure 6 FIGURE 1 illustrates an exemplary user device that may be implemented within an exemplary spatial audio metering system according to some exemplary embodiments of the present disclosure. Figure 6In the exemplary embodiment shown in , user device 600 can process and analyze one or more audio sources to generate a visual representation of a sound overlay that is mapped onto a virtual model of a venue to simulate an event being presented at the venue. When a real-world user of user device 600 views the virtual event on user device 600, the real-world user can use user device 600 to virtually interact with the virtual event, for example, moving around the virtual event to view the virtual event at different locations and / or modifying parameters, characteristics, and / or properties of the virtual event.

[0059] In some embodiments, the user device 600 may be implemented as a standalone device or a discrete device, and / or may be incorporated into or coupled to one or more computing devices, such as one or more desktop computers, one or more mobile phones, one or more mobile computing devices, one or more headsets designed for virtual reality (VR), augmented reality (AR), and / or mixed reality (MR), one or more mobile Internet devices (such as tablet computers and / or laptop computers), one or more mobile video game consoles, one or more mobile wearable electronic devices (such as smart watches), and / or any other computing device with one or more processors that one skilled in the relevant art(s) will recognize without departing from the spirit and scope of the present disclosure, to provide only some examples. In some embodiments, the user device 600 may represent the device shown in Figures 2 to 3 above. Figure 5 An exemplary embodiment of one or more user devices as described in. Figure 6 As shown in FIG, user device 600 may include a user processing device 602, a user display device 604, a user controller device 606, and a virtual speaker projector 608.

[0060] In some embodiments, the user processing device 602 may process the virtual event to generate a virtual event view 610 corresponding to the location of the virtual user associated with the real-world user within the virtual event. In some embodiments, the user processing device 602 may process the virtual event to generate the virtual event view 610 for presentation in a virtual reality (VR) environment. In these embodiments, the virtual event view 610 displayed by the user device 600 represents an immersive virtual world. This virtual world effectively immerses the real-world user within the virtual event, giving the real-world user the feeling that they have entered the virtual event. In some embodiments, when the real-world user moves, such as changing location within the virtual event and / or moving a part of his or her body within the real world (e.g., moving his or her head up or down or left or right), the user device 600 may update the virtual event view 610 to effectively immerse the real-world user within the virtual event. In some embodiments, the user processing device 602 may process the virtual event in a manner similar to that described in Figures 2 to 3 above. Figure 5 The virtual event is processed in a manner substantially similar to that described in

[0066] to generate a virtual event view. In these embodiments, the processing may include: tracking the three-dimensional position of the virtual user in the three-dimensional space of the virtual event; estimating the line of sight of the virtual user at the three-dimensional position; estimating the field of view of the virtual user associated with the line of sight; and / or matching the virtual event view 610 with the field of view of the virtual user at the three-dimensional position. Figure 6 As shown in , the user processing device 602 may include an audio program 615, an audio analyzer 616, a venue database 618, an auditory device 619, an audio visualizer 620, a visual representation 622, an interaction module 624, and a venue manipulator 626.

[0061] In some embodiments, the user display device 604 may enable a user to view virtual events generated by the user processing device 602. The user display device 604 may include a display of a standalone or separate device, and / or a device incorporated into or coupled to one or more computing devices, such as one or more desktop computers, one or more mobile phones, one or more mobile computing devices, one or more headsets designed for virtual reality (VR), augmented reality (AR), and / or mixed reality (MR), one or more mobile internet devices (such as tablet computers and / or laptop computers), one or more mobile video game consoles, one or more mobile wearable electronic devices (such as smart watches), and / or any other computing device. For example, the user display device 604 may provide an MR experience that combines elements of both augmented reality (AR) and virtual reality (VR) in which the real world and digital objects interact.

[0062] In some embodiments, a user display device 604 can enable a user using an MR-enabled headset to visually meter the spatial characteristics of the acoustic output of one or more speakers 612 at a venue. A virtual event view 610 is displayed on the user display device 604, allowing the user to see a visual overlay of sound propagation, virtually visualized as light propagating onto physical objects (such as seating areas). Specifically, the visualization of acoustic paths, volumes, overlays, and overlaps in MR provides a helpful visual aid for audio system calibration and acoustic choreography preview. The user display device 604 can include one or more cameras for visually identifying seating areas, speaker panels, and venue geometry, thereby enabling the user processing device 602 to map the virtual speaker layout to the physical venue. Similarly, the user display device can include one or more microphones 614 for identifying the acoustic performance of the speakers 612 in the venue. To enhance acoustic metering performance and accuracy, additional reference microphones 614 can be placed throughout the venue to capture real-time audio from the speakers 612.

[0063] In some embodiments, user controller device 606 represents an input device used by a real-world user to interact with a virtual event while using user device 600. In some embodiments, user controller device 606 may include one or more action buttons and / or one or more omnidirectional joysticks or buttons that can be manipulated by the real-world user to interact with the virtual event. In some embodiments, the real-world user can use the one or more action buttons and / or one or more omnidirectional joysticks or buttons to perform various actions within the virtual world. For example, the real-world user can use the one or more action buttons and / or one or more omnidirectional joysticks to "click" and / or "drag and drop" one or more computer-generated digital models of various architectural features of a venue in three-dimensional space (such as performance areas, media surfaces, seating locations, and / or standing locations, to provide just a few examples), and / or one or more computer-generated digital models of various objects at the venue in three-dimensional space (such as stage objects associated with the venue and / or stage objects associated with the event, to provide just a few examples).

[0064] In some embodiments, each user device 600 may be configured to generate and display a virtual event view 610 that represents the perspective of a virtual event based on the virtual location of the virtual user associated with the user device 600. A virtual speaker projector 608 may provide a 3D representation of the audio transmission pattern generated by an audio source (such as the output of a speaker 612 detected by a microphone 614), suitable for rendering within the virtual event view 610 on the user device 600. The user device 600 may interact with the virtual event by representing interactions with the virtual event view 610, for example, by moving around the virtual event to view the virtual event at different locations and / or modifying parameters, features, and / or properties of the virtual event. The visualization of the virtual speaker projector 608 enables the user to observe the results of their interactions with the virtual event. Depending on the type of user device, different interactions may be provided via a graphical user interface within the virtual event view 610 of the virtual event. For example, the user device 600 may include physical interface devices such as a keyboard and mouse. The virtual event view 610 may be customized to include interactions that are more easily input via such physical interface devices. Examples of such interactions with user device 600 include modifying a code snippet of a virtual event, or any modification of parameters, features, and / or properties that require text input. As another example, virtual event view 610 can be customized to accommodate the VR implementation of user device 600, thereby including interactions specific to user controller device 606 or hands-free gestures. As another example, virtual event view 610 can be customized to accommodate the mobility of user device 600. Examples of such interactions with user device 600 include providing augmented reality (AR)-based interactions. For example, user device 600 may be physically located within an event venue, while another user device 600 may be physically located away from the venue. Virtual event view 610 can combine a real-time view of the venue with the virtual event in an augmented reality format. For example, virtual graphical elements of virtual event view 610 can be displayed as an overlay on real-world elements of the venue.

[0065] In some embodiments, audio program 615 may include audio objects and metadata associated with the audio sources of an event. Audio objects can be sounds from various audio sources stored as audio files for output in specific channels, along with corresponding spatial metadata defining their position, level, and movement. The audio source can be an audio content library, an external device with stored audio content, a streaming source, or a software package that generates audio signals to be analyzed and visualized. Common examples are digital audio workstation ("DAW") software packages and audio playback servers. In some embodiments, the audio source of specific audio content can be used to generate various different visual mappings of the venue, as determined by the spatial metadata. In some embodiments, the mapping of specific audio content can be tested or previewed (e.g., as part of a live presentation of the venue) to determine the sound coverage of various seating areas of the venue throughout the live presentation. To begin the mapping process, audio program 615 sends the audio object data to audio analyzer 616 for processing.

[0066] In some embodiments, the audio analyzer 616 receives the desired audio content from an audio source as defined by the audio program 615. The audio analyzer 616 can analyze the audio signal properties of the audio content, such as, but not limited to, frequency, wavelength, period, amplitude (volume), pitch, or modulation. Alternatively or additionally, a generalized speaker array visual mapping can be implemented using generalized test audio content to evaluate the generalized speaker array sound distribution within a venue. For example, the audio signal can be configured as a collection of audio sources of a single audio signal formatted for the specific audio system to be visualized.

[0067] In some embodiments, a venue database 618 of speaker and beam locations (i.e., audio system configurations) for each input channel of a venue provides a mapping of speaker / beam locations and parameters for a specific venue. Parameters may include, but are not limited to, speaker position and orientation, number of speakers, arrangement (e.g., array), power, sound distribution type (e.g., diffuse, directional, etc.), distortion, and so on. In one non-limiting example, a venue's audio system configuration may reflect one or more sets of speakers and beams arranged in one or more arrays of variable number, size, type, and power output. In some embodiments, an audio system configuration is a fixed data packet that specifies the geometric properties of the speaker system being measured. In a non-limiting example, the geometric properties may include the location and orientation of each speaker or beam in the system, as well as its capabilities and coverage pattern. This information can be used to initially create the audio signal within the audio source and to map the resulting signal into three-dimensional space during the visualization phase. Furthermore, the venue database 618 may send data specifying the speaker system's geometric properties to an auditory device 619 to simulate the output of the speakers 612 on the headphones of the user device 600.

[0068] In some embodiments, the auditory device 619 can render binaural output for headphone listening, allowing the real-world user of the user device 600 to compare the digital audio output with the venue's acoustic output while on-site. The auditory device 619 can use information from the audio program 615, the venue database 618, and the head tracking sensors on the user device 600 to render the binaural output, as needed. By combining this with the user's sense of hearing, this audio output simulation can allow the user of the user device 600 to test and verify the results of their interactions with the audio system.

[0069] In some embodiments, the audio visualizer 620 may implement a volumetric light renderer to render (i.e., draw) light beams representing audio signals. The audio visualizer 620 may render light beams based on various properties, such as location, color, orientation, type, shape, intensity, or range. The shape and location of the light beams are specified by the audio system configuration. The color and intensity of the light beams are a direct representation of the properties of the incoming audio signal from the sound source. The type of light beam may be based on the type of sound source. In non-limiting examples, the type of sound source may include the type of speaker (e.g., omnidirectional, directional, etc.), beam size, array shape, power considerations, and so on. One or more microphones 614 placed throughout the venue or affixed to the user device 600 may detect the signals generated by the sound sources. The audio visualizer 620 may receive information from the audio analyzer 616 to read the audio signal in real time and emit simplified signals that can be used to drive metering visualizations, such as the instantaneous level of the overall signal and its frequency components. In various embodiments, the audio source may be the actual audio system being metered, or it may be an acoustic model (e.g., a simulation) of that system, such as a binaural renderer.

[0070] In some embodiments, the audio visualizer 620 is configured as a three-dimensional (3D) visualization renderer that synthesizes a 3D representation of the audio system and its metering data. The visual representation 622 is a static scene (e.g., venue seats) and a 3D image from FIG. Figure 5 622 . In some embodiments, to render the static scene of visual representation 622 , audio visualizer 620 uses the audio system configuration to find the source location and direction of each incoming audio signal and renders a set of visual markers at the spatial origin of each signal. Audio visualizer 620 then uses the audio signal properties of each sound to determine a specific light visualization, such as the color and shape of the light beam. Audio visualizer 620 can also render other static scene reference points, such as venue walls, seats, and the proscenium opening (i.e., the portion of the theater stage in front of the curtain).

[0071] In some embodiments, the audio visualizer 620 can orient the virtual 3D scene around the user by matching the 3D scene with motion tracking sensors on the user device 600. These motion tracking sensors may include accelerometers, magnetometers, and gyroscopes. In addition, the audio visualizer 620 can help the user navigate the virtual 3D scene by utilizing the 6 degrees of freedom ("DoF") sensing capabilities on the user device 600. Additional global and indoor positioning systems, such as Bluetooth or ultra-wideband ("UWB") beacons, can be used to enhance positioning performance.

[0072] In some embodiments, the visual representation 622 can be implemented as a 2D or 3D visualization. The visual representation 622 can include an acoustic path from a sound source (e.g., a speaker or beam) and an overlaid light beam visualization. The visual representation 622 can include other visualization techniques such as data overlays, heat maps, and time-lapse photography. The visual representation 622 can also include a user interface (e.g., an overlay area, a meter bar, a calculated digital output value, a measured SPL value, a speaker ID label, a seating section ID label, etc.). Alternatively or additionally, the visual representation 622 can add an image of the venue or at least a portion of the venue as an output. The visual representation 622 can be displayed using any known display technology, such as a display monitor, a mobile computing device with a display, wearable technology (e.g., glasses), or an augmented reality (AR), virtual reality (VR), or mixed reality (MR) headset.

[0073] In some embodiments, the interaction module 624 may provide an interactive environment for interacting with the virtual event. The interaction module 624 may allow the user to move around the virtual event to view audio visualizations of various locations and / or modify one or more parameters, characteristics, and / or properties of the sound sources, as shown in FIG. Figure 5 The interaction module 624 may also provide various communication capabilities for the user device 600, such as audio, video, and / or data communication. In these embodiments, the interaction module 624 may request the establishment of one or more communication sessions, such as audio, video, and / or data communication sessions, between other user devices within the spatial audio metering system to allow real-world users of these user devices to interact with the virtual event while also communicating with the user devices described in FIG. 2 through FIG. Figure 5 communicate with each other in a manner substantially similar to that described above.

[0074] In some embodiments, the venue manipulator 626 can receive interactions of a real-world user with the virtual event and apply user modifications to the audio system. The venue manipulator 626 can insert (e.g., overlay) various virtual graphical elements onto the virtual event view to allow the real-world user to interact with the virtual event. In some embodiments, these virtual graphical elements can summarize the various interactions available to the real-world user, such as modifications. In these embodiments, these virtual graphical elements can include one or more radio buttons, one or more check boxes, one or more text boxes, one or more toggle switches, one or more pop-up menus, one or more lists, and / or any other suitable mechanism that allows real-world user interaction, to provide just a few examples. In some embodiments, the venue manipulator can recognize speech by a user of the user device 600 to add annotations to the virtual event view. In some embodiments, the ... a virtual selection tool (such as those shown in Figures 2 to 3 above) into the virtual event view. Figure 5 , which are examples only, are inserted (e.g., superimposed) onto the virtual event view to allow real-world users to interact with the virtual event. In some embodiments, the interaction module 624 can control the operation of the virtual graphical elements and / or virtual selection tools. In these embodiments, the interaction module 624 can receive various commands, such as "click" and / or "drag and drop," to provide some examples. These commands can be input through the user controller device 606 and / or gesture input. The venue manipulator 626 can then apply these various interactions to modify the audio properties of the audio system at the virtual event and / or venue by sending the modification data to the audio program 615.

[0075] Example Operation of an Example Spatial Audio Metering System

[0076] Figure 7 A flow chart illustrating an exemplary method of operation of a spatial audio metering system according to some exemplary embodiments of the present disclosure is illustrated. The present disclosure is not limited to this operational description. Rather, other operational control flows will be apparent to one of ordinary skill in the relevant art(s) and are within the scope and spirit of the present disclosure. The following discussion describes an exemplary operational control flow 700 that uses an XR device to acoustically configure digital audio for a venue and measure audio performance at the venue. The operational control flow 700 allows a real-world user to interact with a digital representation of an event mapped onto a virtual model of the venue and apply modifications to the venue. The operational control flow 700, described in further detail below, may be executed by one or more computer systems, such as the user device 204 and the user device 204 described above in FIG. Figure 3 The user device 304.n described above Figure 4 The user equipment 404 and Figure 5The user equipment 504.n described above, and / or Figure 6 The user equipment 600 described in.

[0077] At operation 702, operational control flow 700 retrieves digital audio content for metering from an audio source (such as local computer storage, an external device, a streaming source, or a software package that generates audio signals). Common examples are digital audio workstation ("DAW") software packages and audio playback servers. Operation 702 reads the audio signal in real time, determines the audio signal's properties, and emits a simplified signal that can be used to drive metering visualizations, such as the instantaneous levels of the overall signal and its frequency components.

[0078] At operation 704, the operational control flow 700 maps the digital visualization of the audio content onto the virtual model to generate a virtual event. The digital visualization of the event is similar to that described in FIG. Figure 6 The digital representation of the events described in FIG. 704 is substantially similar. The virtual model of operation 704 can be the above FIG. Figure 6 In some embodiments, the operational control process 700 can retrieve and / or generate a virtual model and / or digital representation of the event, including updating the virtual model based on any modifications to the parameters, characteristics, and / or attributes associated with the virtual event. In these embodiments, the operational control process 700 can map the digital representation of the event (including the parameters, characteristics, and / or attributes associated with any real-world effects) onto the virtual model to match the above-described FIG2 through FIG2. Figure 6 Generate virtual events in a similar manner as described in .

[0079] At operation 706, the operational control flow 700 compares multiple visualizations of audio layouts to illustrate the transition from studio to venue. For example, operation 706 may generate a visualization of the studio layout in which the audio was designed, and a visualization of the venue layout in which the audio will be played. By overlaying multiple audio layouts, the user can see how the audio will transition from the studio to the venue. The user can make any modifications to the parameters, characteristics, and / or properties associated with the virtual event to customize the event for a specific venue.

[0080] At operation 708, the operational control flow 700 generates a virtual event view that combines a static scene of the venue and a light beam-based overlay of a 3D visualization of the audio content at the location in the three-dimensional space of the virtual event in operations 704 and / or 706. Operation 708 evaluates the source location and audio signal properties of each incoming audio signal to render a specific light visualization, such as the direction, color, and shape of the light beam. The location in the three-dimensional space can correspond to the physical location of the venue, and thus the virtual event view of the location corresponds to the real-world view of the corresponding physical location at the venue. In Figures 2 to 3, the virtual event view of the venue is generated by the light beam-based overlay of the 3D visualization of the audio content at the location in the three-dimensional space of the virtual event in operations 704 and / or 706. Figure 6In the exemplary embodiment shown in FIG. 2 , the operation control flow 700 may process the virtual events in operations 704 and / or 706 as the virtual events are replayed, as described above with reference to FIG. 2 . Figure 6 In some embodiments, the generated virtual event view can also be based on parameters, characteristics and / or attributes established or modified by the user device, as shown in Figures 2 to 3 above. Figure 6 For example, the visual graphical elements of the generated virtual event view may represent simulated behavior of one or more parameters, characteristics, and / or attributes associated with the virtual event.

[0081] At operation 710, the operation control flow 700 may be as described above with respect to FIG. Figure 6 6 above). In some embodiments, the audio metering interactions in the form of user input are received from the user device and represented in the virtual event view in operation 708. The operation control process 700 can play the virtual event in operations 704 and / or 706 to virtually simulate the event being presented at the venue. The operation control process 700 can receive user input from the user device while viewing the virtual event view in operation 708 (e.g., on a display of the user device, as described in Figures 2 to 6 above). In some embodiments, the user input can include instructions (e.g., from an input device of the user device, as described in Figures 2 to 6 above) to virtually move the virtual user around the three-dimensional space of the virtual event in operation 704 to view a digital representation of the event at one or more locations in the three-dimensional space of the virtual event in operation 704. In some embodiments, the interaction can include moving the virtual user in a manner consistent with that described in Figures 2 to 6 above while the virtual event view in operation 708 is being viewed. Figure 6 One or more parameters, characteristics and / or attributes of the virtual event in operation 704 and / or 706 are virtually modified in a manner substantially similar to that described in FIG. Figure 6 One or more parameters, characteristics and / or attributes described in FIG. 2 to FIG. 2 are substantially similar and represent Figure 6, characteristics, and / or properties of the virtual event. The modifications may be propagated to the venue's audio system to configure the audio properties. In some embodiments, the operation control process 700 may update the virtual event in operations 704 and / or 706 to reflect the modifications to one or more parameters, characteristics, and / or properties of the virtual event in operations 704 and / or 706. For example, the operation control process 700 may update one or more virtual graphical elements of the virtual event view to reflect any modifications to one or more parameters, characteristics, and / or properties of the virtual event. In these embodiments, the operation control process 700 may propagate the modifications to multiple user devices to allow these multiple user devices to collaboratively interact with the virtual event in operations 704 and / or 706. In some embodiments, instead of distributing the updated virtual event, the operation control process 700 may transmit one or more modifications to the user devices, as described above in FIG. 2 to FIG. Figure 6 Each user device may then locally update the virtual event based on the received modification(s).

[0082] Example computer systems that can be used to implement electronic devices within an example venue

[0083] Figure 8 FIG2 is a simplified block diagram of a computer system suitable for use with the embodiments described herein according to some exemplary embodiments of the present disclosure. Figure 3 The user equipment 304.n described above Figure 4 The user equipment 404 and Figure 5 The user equipment 504.n described above, and / or Figure 6 The user equipment 600 described in the preceding paragraph may be implemented in hardware, firmware, software, or any combination thereof. Figure 8 The discussion will describe an exemplary computer system 800 that may be used with these electronic devices.

[0084] exist Figure 8In the exemplary embodiment shown in FIG, computer system 800 generally includes at least one processor 804 that communicates with multiple peripheral devices via a bus subsystem 802. Generally, the at least one processor 804 may include or be any microprocessor, graphics processing unit, or digital signal processor, and their electronic processing equivalents, such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). As used herein, the term "processor" refers to a tangible data and information processing device that physically transforms data and information, typically using a sequence of transformations (also called "operations"). Data and information can be physically represented by electrical, magnetic, optical, or acoustic signals that can be stored, accessed, transferred, combined, compared, or otherwise manipulated by the processor. The term "processor" can refer to both single processors and multi-core systems or multi-processor arrays, including graphics processing units, digital signal processors, digital processors, or combinations of these elements. Processors can be electronic, for example, including digital logic circuitry (e.g., binary logic), or analog (e.g., operational amplifiers). Processors can also operate to support the execution of related operations in a "cloud computing" environment or as "software as a service" (SaaS). For example, at least some operations may be performed by a set of processors available at a distributed or remote system, accessible via a communications network (eg, the Internet) and via one or more software interfaces (eg, an application program interface (API)).

[0085] Computer system 800 typically includes an operating system such as Microsoft's Windows, Sun Microsystems' Solaris, Apple Computer's MacOS, Linux, or UNIX. Computer system 800 may also typically include a basic input / output system (BIOS) and processor firmware. The processor uses the operating system, BIOS, and firmware to control subsystems and interfaces coupled to the processor. Typical processors compatible with these operating systems include Intel's Pentium and Itanium processors, Advanced Micro Devices' Opteron and Athlon processors, and ARM processors from ARM Holdings.

[0086] like Figure 8 As shown in , these peripheral devices may include user interface input devices 812, user interface output devices 810, network interface subsystem 806, and storage subsystem 814, which includes file storage subsystem 816 and memory subsystem 818. Input and output devices allow a user to interact with computer system 800. Figure 8In the exemplary embodiment shown in FIG, network interface subsystem 806 provides interfaces with external networks, including an interface with a communications network 808, and is coupled to corresponding interface devices in other computer systems or machines via communications network 808. Communications network 808 may include numerous interconnected computer systems, machines, and communications links. These communications links may be wired, fiber optic, wireless, or any other means for transmitting information. Communications network 808 may be any suitable computer network, such as a wide area network (such as the Internet) and / or a local area network (such as Ethernet). Communications network 808 may be wired and / or wireless, and may utilize encryption and decryption methods, such as those provided by virtual private networks (VPNs). Communications networks utilize one or more communications interfaces that can receive data from or transmit data to other systems. Examples of communications interfaces typically include Ethernet cards, modems (e.g., telephone, satellite, cable, or ISDN), (asynchronous) digital subscriber line (DSL) units, FireWire interfaces, USB interfaces, and the like. One or more communication protocols may be used, such as HTTP, TCP / IP, RTP / RTSP, IPX, and / or UDP.

[0087] The user interface input devices 812 may include an alphanumeric keyboard, a keypad, a pointing device (such as a mouse, trackball, touchpad, stylus, or graphics tablet), a scanner, a touch screen incorporated into a display, an audio input device (such as a voice recognition system or microphone), eye movement recognition, brain wave pattern recognition, and other types of input devices. These devices may be connected to the computer system 800 via a wired or wireless connection. In general, the use of the term "input device" is intended to include all possible types of devices and methods for inputting information into the computer system 800 or onto the communication network 800. The user interface input devices 812 typically allow a user to select objects, icons, text, etc. displayed on some type of user interface output device (e.g., a display subsystem).

[0088] User interface output devices 810 may include a display subsystem, a printer, a fax machine, or a non-visual display (such as an audio output device). The display subsystem may include a cathode ray tube (CRT), a flat-panel device (such as a liquid crystal display (LCD)), a projection device, or some other device for creating visible images (such as a virtual reality system). The display subsystem may also provide a non-visual display, such as via an audio output or a tactile output (e.g., vibration) device. In general, the use of the term "output device" is intended to include all possible types of devices and ways of outputting information from the computer system 800 to a user or another machine or computer system.

[0089] The memory subsystem 818 typically includes a plurality of memories, including a primary random access memory ("RAM") 820 (or other volatile storage device) for storing instructions and data during program execution, and a read-only memory ("ROM") 822 for storing fixed instructions. The file storage subsystem 816 provides persistent storage for program and data files and may include a hard drive, a floppy disk drive and associated removable media, a CD-ROM drive, an optical drive, flash memory, or removable media cartridges. Databases and modules that implement the functionality of certain embodiments may be stored by the file storage subsystem 816.

[0090] The bus subsystem 802 provides a facility for the various components and subsystems of the computer system 800 to communicate with each other as intended. Although the bus subsystem 802 is schematically shown as a single bus, alternative embodiments of the bus subsystem may use multiple buses. For example, a RAM-based main memory may communicate directly with a file storage system using a direct memory access (DMA) system.

[0091] in conclusion

[0092] The detailed description illustrates exemplary embodiments consistent with the present disclosure with reference to the accompanying drawings. References in this disclosure to "exemplary embodiments" indicate that the exemplary embodiments described may include a particular feature, structure, or characteristic, but each exemplary embodiment may not necessarily include the particular feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same exemplary embodiments. In addition, any feature, structure, or characteristic described in conjunction with an exemplary embodiment may, independently or in any combination, include features, structures, or characteristics of other exemplary embodiments, whether or not explicitly described.

[0093] The detailed description is not meant to be limiting. Instead, the scope of the present disclosure is defined solely by the following claims and their equivalents. It should be understood that the detailed description section, and not the abstract section, is intended to be used to interpret the claims. The abstract section may set forth one or more, but not all, exemplary embodiments of the present disclosure and, therefore, is not intended to limit the present disclosure and the appended claims and their equivalents in any way.

[0094] The exemplary embodiments described in this disclosure are provided for illustrative purposes and are not intended to be limiting. Other exemplary embodiments are possible, and modifications may be made to the exemplary embodiments while remaining within the spirit and scope of this disclosure. This disclosure has been described with the aid of functional building blocks that illustrate the implementation of specific functions and their relationships. For ease of description, the boundaries of these functional building blocks have been arbitrarily defined herein. Alternative boundaries may be defined so long as the specified functions and their relationships are appropriately performed.

[0095] Embodiments of the present disclosure may be implemented using hardware, firmware, software applications, or any combination thereof. Embodiments of the present disclosure may also be implemented as instructions stored on a machine-readable medium, which can be read and executed by one or more processors. A machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., computing circuitry). For example, a machine-readable medium may include non-transitory machine-readable media such as read-only memory (ROM); random access memory (RAM); magnetic disk storage media; optical storage media; flash memory devices; and others. As another example, a machine-readable medium may include transitory machine-readable media such as electrical, optical, acoustic, or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.). Furthermore, firmware, software applications, routines, and instructions may be described herein as performing certain actions. However, it should be understood that such descriptions are for convenience only and that such actions are actually performed by a computing device, processor, controller, or other device executing the firmware, software applications, routines, instructions, and the like.

[0096] The specific implementation of the exemplary embodiments sufficiently reveals the general nature of the present disclosure that others can readily modify and / or adapt various applications such as the exemplary embodiments without departing from the spirit and scope of the present disclosure by applying the knowledge of one or more persons skilled in the relevant art(s) without undue experimentation. Therefore, such adaptations and modifications are intended to be within the meaning and multiple equivalents of the exemplary embodiments based on the teachings and guidance presented herein. It should be understood that the terms and terminology herein are for descriptive purposes only and not limiting, so that the terms and terminology of this specification should be interpreted by one skilled in the relevant art(s) in light of the teachings herein.

Claims

1. A computer-implemented method comprising: retrieving, by the first user device, audio content from an audio source; mapping, by the first user device, the digital visualization of the audio content onto the virtual model to generate a virtual event, wherein the virtual model is a representation of the venue; comparing, by the first user device, visualizations of the audio layout to illustrate the transition from studio to venue; Generating a virtual event view by a first user device, wherein the generating includes: Evaluate the source location of audio content and the nature of audio signals; Mapping audio content into a unique direction, color, or shape of volumetric light beams; and overlaying a visualization of the audio content at the location of the venue onto a static scene of the venue in any of an extended reality, virtual reality, augmented reality, or mixed reality environment; Receiving, by a first user device, first user input for audio metering interaction with a virtual event; and The virtual event view is updated based on the first user input.

2. The computer-implemented method of claim 1 , wherein the virtual event view corresponds to a location of the virtual event, and The location of the virtual event corresponds to the physical location of the venue.

3. The computer-implemented method of claim 1 , further comprising receiving second user input from a second user device for audio metering interaction with the virtual event.

4. The computer-implemented method of claim 3, further comprising transmitting any user input to any user device, Any user input is configured to update each virtual event view at the corresponding user device.

5. The computer-implemented method of claim 1 , wherein updating the virtual event view further comprises: modifying the volumetric light beam based on the first user input; as well as Display the modified volumetric beam in the Virtual Event View.

6. The computer-implemented method of claim 1, further comprising configuring audio signal properties of an audio source at the venue by propagating user input to the venue.

7. The computer-implemented method of claim 1, further comprising measuring audio signal properties of an audio source at the venue using a microphone at the venue.

8. The computer-implemented method of claim 1, wherein one of the audio signal properties of the audio content comprises any of: a signal volume level, a frequency, a frequency range, a signal channel, or a spatial coverage area.

9. The computer-implemented method of claim 1, wherein receiving first user input further comprises receiving a user-visible annotation for propagation across the virtual event view.

10. A system comprising: Memory; as well as a processor coupled to the memory and configured to perform operations comprising: retrieving, by the first user device, audio content from an audio source; mapping, by the first user device, the digital visualization of the audio content onto the virtual model to generate a virtual event, wherein the virtual model is a representation of the venue; comparing, by the first user device, visualizations of the audio layout to illustrate the transition from studio to venue; Generating a virtual event view by a first user device, wherein the generating includes: Evaluate the source location of audio content and the nature of audio signals; Mapping audio content into a unique direction, color, or shape of volumetric light beams; and overlaying a visualization of the audio content at the location of the venue onto a static scene of the venue in any of an extended reality, virtual reality, augmented reality, or mixed reality environment; Receiving, by a first user device, first user input for audio metering interaction with a virtual event; and The virtual event view is updated based on the first user input.

11. The system of claim 10, wherein the virtual event view corresponds to a location of the virtual event, and The location of the virtual event corresponds to the physical location of the venue.

12. The system of claim 10, wherein the operations further comprise receiving second user input from a second user device for audio metering interaction with the virtual event.

13. The system of claim 12, wherein the operations further comprise transmitting any user input to any user device, and Any user input is configured to update each virtual event view at the corresponding user device.

14. The system of claim 10, wherein updating the virtual event view further comprises: modifying the volumetric light beam based on the first user input; as well as Display the modified volumetric beam in the Virtual Event View.

15. The system of claim 10, wherein the operations further comprise configuring audio signal properties of an audio source at the venue by propagating user input to the venue.

16. The system of claim 10, wherein the operations further comprise measuring audio signal properties of an audio source at the venue by using a microphone at the venue.

17. The system of claim 10, wherein one of the audio signal properties of the audio content comprises any of: a signal volume level, a frequency, a frequency range, a signal channel, or a spatial coverage area.

18. The system of claim 10, wherein receiving first user input further comprises receiving a user-visible annotation for propagation across the virtual event view.

19. A non-transitory computer-readable device storing instructions that, when executed by at least one computing device, cause the at least one computing device to perform operations comprising: retrieving, by the first user device, audio content from an audio source; mapping, by the first user device, the digital visualization of the audio content onto the virtual model to generate a virtual event, wherein the virtual model is a representation of the venue; comparing, by the first user device, visualizations of the audio layout to illustrate the transition from studio to venue; Generating a virtual event view by a first user device, wherein the generating includes: Evaluate the source location of audio content and the nature of audio signals; Mapping audio content into a unique direction, color, or shape of volumetric light beams; and overlaying a visualization of the audio content at the location of the venue onto a static scene of the venue in any of an extended reality, virtual reality, augmented reality, or mixed reality environment; Receiving, by a first user device, first user input for audio metering interaction with a virtual event; and The virtual event view is updated based on the first user input.

20. The non-transitory computer-readable device of claim 19, wherein the operations further comprise configuring audio signal properties of an audio source at the venue by propagating user input to the venue.

Images