Personalization of audio content within real-world environments

Abstract

Systems, methods, and devices can playback audio-video content having audio and visual content. These systems, methods, and devices can playback the visual content to various viewers within a real-world environment. These systems, methods, and devices can cause the audio content to be played back on various real-world user devices associated with the viewers. These systems, methods, and devices can individualize the audio content played back by the real-world user devices for various spatial locations of the real-world user devices within the real-world environment.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. Patent Application No. 18 / 602,517, filed March 12, 2024, the entire contents of which are incorporated herein by reference. Background Technology

[0003] Spatial audio, also known as 3D audio, can include three-dimensional sound that allows the viewer to perceive the audio content as being delivered from multiple directions and / or multiple distances. Typically, home audio systems achieve spatial audio using advanced speaker systems or soundbars employing technologies such as Dolby Atmos or DTS:X (to provide some examples). These home audio systems create a more immersive listening experience by placing sound in specific locations, making it feel as if the audio is coming from various points around the room. Larger venue systems within spaces, such as music venues (e.g., music theaters, music clubs, and / or concert halls) and sports venues (e.g., arenas, conference centers, and / or stadiums), are often used to enhance events such as those including musical events, theatrical events, sporting events, or movies. This involves strategically placing speakers around the venue to create an immersive sound experience for the audience. However, achieving spatial audio outdoors can be challenging due to the open environment. The just-notifiable difference (JND) of interaural time difference (ITD) refers to the smallest perceptible change in the time delay between arriving sounds, for example, about 10 microseconds. Similarly, in the approximate range of about 2 kHz to 5 kHz, the JND of interaural sound level difference (ILD) is about 1 dB. In some cases, long delays can distort the perception of the direction of a sound source, potentially leading to an unrealistic or disorienting effect. Attached Figure Description

[0004] This disclosure is described with reference to the accompanying drawings. In the drawings, similar reference numerals indicate elements that are identical or functionally similar. Additionally, the leftmost numeral(s) of the reference numeral(s) identifies the drawing in which the reference numeral(s) first appear. In the drawings: Figure 1 A simplified block diagram of an exemplary real-world environment according to some exemplary embodiments of the present disclosure is illustrated; Figure 2 The illustration shows a simplified block diagram of a display device server that can be implemented in an exemplary real-world environment according to some exemplary embodiments of the present disclosure; Figure 3The exemplary operations of modeling an exemplary real-world environment according to some exemplary embodiments of the present disclosure are illustrated graphically. Figures 4A to 4D The illustrations graphically depict one or more types of visual content that can be played back by an exemplary real-world environment according to some exemplary embodiments of the present disclosure; Figure 5 The illustration shows an exemplary operation control flow for an exemplary real-world user device in an exemplary real-world environment, according to some exemplary embodiments of the present disclosure; Figure 6 The illustration shows an exemplary operation control flow for an exemplary display device server in an exemplary real-world environment, according to some exemplary embodiments of the present disclosure; and Figure 7 A simplified block diagram of an exemplary computer system that can be implemented in an exemplary real-world environment according to some exemplary embodiments of the present disclosure is illustrated.

[0005] This disclosure will now be described with reference to the accompanying drawings. Detailed Implementation

[0006] The following disclosure provides numerous different embodiments or examples for implementing various features of the provided subject matter. Specific examples of components and arrangements are described herein to simplify the disclosure. Of course, these are merely examples and are not intended to be limiting. Various aspects of the disclosure are best understood from the following detailed description when read in conjunction with the accompanying drawings. Reference numerals and / or letters may be repeated in various examples. Such repetition, in itself, does not determine the relationship between the various embodiments and / or configurations discussed. Note that, in accordance with industry standard practice, features are not drawn to scale. In fact, the dimensions of features may be arbitrarily increased or decreased for clarity of discussion.

[0007] For ease of description, the following disclosure may include spatial relative terms such as “below,” “under,” “lower,” “above,” “on,” “upper,” etc., to describe the relationship between elements or features illustrated in one or more of the accompanying drawings. These spatial relative terms are intended to cover different orientations of the different embodiments or examples depicted in one or more of the drawings. Different embodiments or examples may be oriented in other ways (rotated 90 degrees or in other orientations), and the spatial relative terms included herein may be interpreted similarly accordingly. Furthermore, the following disclosure may include the terms “about” or “substantially” to indicate that a given quantity can vary based on a particular technique. Based on the technique, the terms “about” or “substantially” may indicate that the value of a given quantity varies within, for example, a range of 1-15% of that value (e.g., ±1%, ±2%, ±5%, ±10%, or ±15% of that value).

[0008] Overview

[0009] Systems, methods, and apparatuses can play back audio and video content that includes both audio and visual elements. These systems, methods, and apparatuses can play back visual content to various viewers within a real-world environment. They can also enable the playback of audio content on various real-world user devices associated with these viewers. Furthermore, these systems, methods, and apparatuses can personalize the audio content played back by these real-world user devices based on their various spatial locations within a real-world environment.

[0010] Exemplary real-world environment

[0011] Figure 1 A simplified block diagram of an exemplary real-world environment according to some exemplary embodiments of the present disclosure is illustrated. Figure 1 In the exemplary embodiment illustrated herein, real-world environment 100 can play back audio and video content having audio and visual content. While the discussion herein may describe real-world environment 100 as performing certain actions, operations, routines, processes, etc., it should be understood that such description is merely for convenience, and such actions, operations, routines, processes, etc., are produced by the operation of one or more electrical, mechanical, and / or electromechanical devices within real-world environment 100, which will be clear to those skilled in the art without departing from the spirit and scope of this disclosure. As will be described herein, real-world environment 100 can play back visual content to various viewers within real-world environment 100. And as will be described herein, real-world environment 100 can play back audio content on various real-world user devices associated with these viewers. In some embodiments, the audio content may include spatial audio content. Spatial audio content, also known as three-dimensional audio content, may include three-dimensional sound that allows viewers within real-world environment 100 to perceive that the audio content is provided from multiple directions and / or multiple distances. In some embodiments, the real-world environment 100 can personalize the audio content played back by these real-world user devices for various spatial locations within the real-world environment 100. In these embodiments, the audio content played back by these real-world user devices at these various spatial locations can be characterized as being similar to one or more types of visual content t1 to t2 to be played back within the real-world environment 100. r Synchronization. For example... Figure 1 As illustrated in the figure, the real-world environment 100 may include a real-world display device 102 and real-world user devices 104.1 to 104. n And / or may be compatible with real-world user devices 104.1 to 104.n One or more associated peripheral devices 106.

[0012] exist Figure 1 In the exemplary embodiments illustrated in the figure, real-world display device 102 represents a three-dimensional media display device capable of replaying visual content throughout a real-world environment 100. In some embodiments, real-world display device 102 may include one or more visual displays, typically referred to as a three-dimensional media plane, distributed across the exterior of real-world display device 102. In these embodiments, the one or more visual displays may include a series of rows and columns of picture elements (also referred to as pixels) in three dimensions, forming real-world display device 102. In these embodiments, pixels may be implemented using one or more light-emitting diode (LED) displays, one or more organic light-emitting diode (OLED) displays, and / or one or more quantum dot (QD) displays (to provide some examples). For example, three-dimensional media plane 102 may include one or more visual displays surrounding the exterior of real-world display device 102 to form a visual display of approximately 580,000 square feet. Figure 1 In the exemplary embodiment illustrated in the figure, the real-world display device 102 can display images on one or more visual displays to real-world user devices 104.1 to 104 within the real-world environment 100. n Each associated viewer replays the visual content. For example... Figure 1 As illustrated, the real-world display device 102 can be implemented as a three-dimensional structure, such as a hemispherical structure (also known as a hemispherical dome) or a hemispherical-like structure. However, this example is not limiting. Rather, those skilled in the art will recognize that the real-world display device 102 can represent any other suitable three-dimensional structure (such as a cube, sphere, cone, pyramid, cuboid, or cylinder, etc., to provide some examples) without departing from the spirit and scope of this disclosure. Furthermore, those skilled in the art will recognize that the real-world display device 102 can even represent any suitable two-dimensional structure (such as a circle, triangle, square, rectangle, pentagon, quadrilateral, hexagon, or octagon, etc., to provide some examples) without departing from the spirit and scope of this disclosure. In some embodiments, such a suitable two-dimensional structure can be connected to one or more architectural and / or non-architectural structures, as will be recognized by those skilled in the art, without departing from the spirit and scope of this disclosure.

[0013] The real-world display device 102 can be used for real-world user devices 104.1 to 104 within the real-world environment 100. nThe audio content is personalized in various spatial locations. In some embodiments, the real-world display device 102 may include or be coupled to one or more electrical, mechanical, and / or electromechanical devices (examples of which will be described herein) to personalize the audio content as described herein. As part of this personalization, the real-world display device 102 may identify real-world user devices 104.1 to 104 within the real-world environment 100. n Spatial location d1 to d n In some embodiments, spatial locations d1 to d n This may include real-world user devices 104.1 to 104 within the real-world environment 100. n Three-dimensional coordinates (e.g., x, y, and z coordinates in a Cartesian coordinate system and / or r, θ, and z coordinates in a spherical coordinate system) (Coordinates, etc.). In these embodiments, spatial positions d1 to d2... n Real-world user equipment 104.1 to 104 within the real-world environment 100 can be represented in two or three dimensions. n The absolute position. Alternatively, spatial positions d1 to d... n Real-world user equipment 104.1 to 104 within the real-world environment 100 can be represented in two or three dimensions. n The relative position of the display device 102 in the real world. Although in Figure 1 Lieutenant General Real-world User Equipment 104.1 to 104. n The diagram shows a space with positions d1 to d2. n The different spatial locations within these, but those skilled in the art (one or more) will recognize, are real-world user devices 104.1 to 104. n One or more devices in the configuration may have spatial locations d1 to d2. n Similar spatial positions are used, without departing from the spirit and scope of this disclosure. As an alternative to or addition to three-dimensional coordinates, spatial positions d1 to d2... n This may include real-world user devices 104.1 to 104 within the real-world environment 100. n The three-dimensional orientation is typically expressed as yaw, pitch, and roll. In some embodiments, the real-world display device 102 can access real-world user devices 104.1 to 104. n Spatial location d1 to d n In these embodiments, the real-world display device 102 may be from real-world user equipment 104.1 to 104. n Searching spatial locations d1 to d n In these embodiments, the real-world display device 102 can dynamically switch between real-world user devices 104.1 and 104.n Searching spatial locations d1 to d n For example, once every millisecond, once every ten milliseconds, once every hundred milliseconds, once per second, once every ten seconds, and / or once every hundred seconds, etc. This dynamic retrieval can facilitate the personalization of audio content to real-world user devices 104.1 to 104 as these real-world user devices move within the real-world environment 100. n .

[0014] Determine spatial locations d1 to d n Subsequently, the real-world display device 102 can target spatial locations d1 to d2. n Personalization will be achieved from real-world user devices 104.1 to 104. n The audio content is being played back. As will be described herein, the real-world display device 102 can identify the audio content and its spatial location d1 to d2. n One or more associated parameters, characteristics, and / or attributes are collectively referred to as audio parameters p1 to p2. m In some embodiments, for spatial locations d1 to d2 n The audio content at the location, audio parameters p1 to p m It may include, or be represented by, sound intensity and / or time delay, etc. In some embodiments, audio parameters p1 to p... m It can also be based on one or more types of visual content t1 to t2 played back on the real-world display device 102. r ,exist Figure 1 The audio parameter p is represented in the middle. 1,1 to p m,r In these embodiments, for spatial locations d1 to d2... n One or more types of visual content t1 to t r Audio parameter p 1,1 to p m,r This may include, or be represented by, sound intensity and / or time delay, etc. In these embodiments, one or more types of visual content t1 to t2... r This may include two-dimensional morphing visual content, three-dimensional volumetric visual content, see-through visual content, and / or augmented reality visual content, each of which will be described herein.

[0015] In some embodiments, the real-world display device 102 can access audio parameters 108 based on spatial location for spatial locations d1 to d2. n One or more types of visual content t1 to t r audio parameters p 1,1 to p m,rIn these embodiments, the spatial location-based audio parameters 108 can represent an organized collection of data, often referred to as a database, which can be accessed through one or more types of visual content t1 to t2. r and / or spatial regions Z1 to Z m The audio parameter p of the index 1,1 to p m,r For example, one or more types of visual content t1 to t2 r Each type of visual content is associated with one or more spatial regions Z1 to Z2. m and audio parameter p 1,1 to p m,r One or more corresponding audio parameter sets are associated, such as Figure 1 The image is illustrated by audio parameters 108 based on spatial location. In some embodiments, the database may include one or more data tables with data values, such as alphanumeric strings, integers, decimals, floating-point numbers, dates, times, binary values, Boolean values, and / or enumerations to provide examples. In some embodiments, the real-world display device 102 may identify one or more types of visual content t1 to t2 corresponding to visual content played back by the real-world display device 102. r In these embodiments, the real-world display device 102 can identify spatial locations d1 to d2 from the location-based audio parameters 108. n Spatial region Z1 to Z m In these embodiments, the real-world display device 102 can access spatial regions Z1 to Z2 from the spatial location-based audio parameters 108. m One or more types of visual content t1 to t r Associated audio parameter p 1,1 to p m,r .

[0016] Accessing audio parameter p 1,1 to p m,r Subsequently, the real-world display device 102 can target spatial locations d1 to d2. n Personalization will be achieved from real-world user devices 104.1 to 104. n The audio content is played back. As will be described herein, this personalization can be performed on the server side, i.e., by the real-world display device 102. In some embodiments, the real-world display device 102 can process the audio content, such as the amplitude and / or frequency of the audio content, to generate a sequence at spatial locations d1 to d2. n The location has audio parameter p 1,1 to p m,r The audio content. For example, the audio parameter p 1,1 to pm,r This can include targets at spatial locations d1 to d n One or more types of visual content t1 to t r From real-world user equipment 104.1 to 104. n The playback audio content includes various sound intensities and / or time delays, etc. In this example, the real-world display device 102 can process audio content, such as the amplitude and / or frequency of the audio content, for spatial locations d1 to d2. n One or more types of visual content t1 to t r This produces audio content with features such as sound intensity and / or time delay. As will be described herein, this applies to real-world user devices 104.1 to 104. n Processed audio content can be played back for spatial locations d1 to d2. n Personalization for one or more types of visual content t1 to t r The audio content. In these embodiments, the real-world display device 102 can package the processed audio content to provide digital audio content packages to real-world user devices 104.1 to 104. n Alternatively or additionally, the real-world display device 102 may provide digital audio content to real-world user devices 104.1 to 104. n Provide audio parameter p 1,1 to p m,r As will be described in this article, real-world user devices range from 104.1 to 104. n It can be based on the audio parameter p in a manner substantially similar to the real-world display device 102 described in this article. 1,1 to p m,r Process audio content (e.g., amplitude and / or frequency of the audio content) to target the spatial location d1 to d2. n One or more types of visual content t1 to t r Personalized audio content.

[0017] exist Figure 1 In the exemplary embodiment illustrated in the figure, real-world user equipment 104.1 to 104. n Audio content can be played back at a spatial location within a real-world environment 100. In some embodiments, real-world user devices 104.1 to 104. nThis can include consumer electronic devices, cellular phones, smartphones, feature phones, tablet computers, wearable computing devices, personal digital assistants (PDAs), pagers, wireless handheld devices, desktop computers, laptop computers, in-vehicle infotainment systems (IVI), in-vehicle entertainment (ICE) devices, instrument clusters (ICs), head-up displays (HUDs), onboard diagnostic (OBD) devices, dashboard top moving equipment (DME), mobile data terminals (MDTs), electronic engine management systems (EEMS), electronic / engine control units (ECUs), electronic / engine control modules (ECMs), embedded systems, microcontrollers, control modules, engine management systems (EMS), connected or "smart" appliances, machine-type communication (MTC) devices, machine-to-machine (M2M) devices, Internet of Things (IoT) devices, and so on. In some embodiments, real-world user equipment 104.1 to 104. n This can include hundreds, thousands, tens of thousands, or even more real-world user devices within a real-world environment 100.

[0018] In some embodiments, real-world user equipment 104.1 to 104. n It can be in spatial location d1 to d n Playback of audio content at spatial locations d1 to d2 n Personalization is achieved from real-world user devices 104.1 to 104. n Playback of visual content t1 to t1 of one or more types r The audio content. In these embodiments, it is provided by real-world user devices 104.1 to 104. n In spatial locations d1 to d n The audio content being played back can be characterized as one or more types of visual content t1 to t2 played back by the real-world display device 102. r Synchronization. As part of this playback, real-world user devices 104.1 to 104. n Spatial positions d1 to d2 can be provided to the real-world display device 102. n In some embodiments, real-world user equipment 104.1 to 104. n Spatial location d1 to d can be provided n For example, once every millisecond, once every ten milliseconds, once every hundred milliseconds, once per second, once every ten seconds, and / or once every hundred seconds, etc. When providing spatial locations d1 to d... n Subsequently, real-world user devices ranged from 104.1 to 104. n It can be in spatial location d1 to d n Playback is performed on one or more types of visual content t1 to t2. rThe audio content. In some embodiments, real-world user devices 104.1 to 104. n The spatial locations d1 to d2 can be recovered from the digital audio content package provided by the real-world display device 102. n For one or more types of visual content t1 to t r The audio content. In these embodiments, real-world user devices 104.1 to 104. n The recovered audio content can be played back for spatial locations d1 to d2. n Personalization for one or more types of visual content t1 to t r The audio content. Alternatively or additionally, real-world user devices 104.1 to 104. n The audio parameters p of the audio content can be accessed from the digital audio content package. 1,1 to p m,r In some embodiments, real-world user equipment 104.1 to 104. n Audio content can be accessed from a real-world display device 102. In these embodiments, real-world user devices 104.1 to 104. n Audio content can be processed in a manner substantially similar to the real-world display device 102 described herein, for spatial locations d1 to d2. n One or more types of visual content t1 to t r Audio content with sound intensity and / or time delay is generated. In these embodiments, real-world user equipment 104.1 to 104. n Processed audio content can be played back for spatial locations d1 to d2. n Personalization for one or more types of visual content t1 to t r The audio content.

[0019] In some embodiments, real-world user equipment 104.1 to 104. n One or more devices in the middle (such as Figure 1 The real-world user equipment 104.3 illustrated in the figure (for example) may be associated with one or more peripheral devices 106. In these embodiments, one or more peripheral devices 106 may include a display, printer, speaker, headset, game controller, virtual reality (VR) headset, VR controller, and / or any other suitable electrical, mechanical, and / or electromechanical device that will be clearly understood by a person skilled in the art to reproduce audio or sound without departing from the spirit and scope of this disclosure. In some embodiments, real-world user equipment 104.1 to 104. nThe processed audio content as described herein may be provided to one or more peripheral devices 106 for playback, either alternatively or additionally.

[0020] Exemplary location-based audio parameters that can be determined by an exemplary real-world environment.

[0021] Figure 2 A simplified block diagram of a display device server that can be implemented in an exemplary real-world environment according to some exemplary embodiments of the present disclosure is illustrated. As described herein, a real-world display device 102 can access spatial regions Z1 to Z2 from location-based audio parameters 108. m One or more types of visual content t1 to t r Associated audio parameter p 1,1 to p m,r The following is about... Figure 2 The discussion will describe exemplary operations of the display device server 202 for determining spatial location-based audio parameters 108. In some embodiments, the display device server 202 may be implemented as a standalone electrical, mechanical, and / or electromechanical device, or a discrete device, and / or may be coupled to or combined with another electrical, mechanical, and / or electromechanical device, or a host device (such as the real-world display device 102 as described herein). As will be described herein, the display device server 202 may model the real-world environment 100 within a virtual environment. In some embodiments, the display device server 202 may simulate playback of audio content within the model of the real-world environment 100 in the virtual environment to determine spatial regions Z1 to Z2. m For one or more types of visual content t1 to t r audio parameters p 1,1 to p m,r This is included within spatial location-based audio parameters 108 as described herein. In these embodiments, the display device server 202 can display spatial regions Z1 to Z2. m For one or more types of visual content t1 to t r audio parameters p 1,1 to p m,r These are stored in the spatial location-based audio parameters 108 as described in this article.

[0022] exist Figure 2 In the exemplary embodiment illustrated in the figure, the display device server 202 can access acoustic parameters 250 based on visual content, which identifies the real-world environment 100 and one or more types of visual content t1 to t2. r One or more associated parameters, characteristics, and / or attributes, in Figure 2 The set of acoustic properties of the real world, a1 to a2, is collectively referred to as the set of acoustic properties of the real world. rIn some embodiments, one or more types of visual content t1 to t r Each type of visual content in Figure 2 For example, in the acoustic parameters based on visual content, the set of real-world acoustic properties a1 to a2 is illustrated in Figure 250. r The set of real-world environmental characteristics is associated with this. In some embodiments, one or more types of visual content t1 to t2 are... r This refers to one or more types of visual content that can be played back by a real-world display device 102. In these embodiments, one or more types of visual content t1 to t2... r This may include two-dimensional morphing visual content, three-dimensional volumetric visual content, see-through visual content, and / or augmented reality visual content, each of which will be described herein.

[0023] Generally speaking, the set of acoustic properties in the real world is a1 to a2. r This can guide the display device server 202 to determine the spatial region Z1 to Z2 as described herein. m For one or more types of visual content t1 to t r Audio parameters p1 to p m In some embodiments, the display device server 202 may use a set of real-world acoustic properties a1 to a2. r In a virtual environment, for one or more types of visual content t1 to t r For spatial regions Z1 to Z m The acoustic properties of the real-world environment 100 within the context are modeled. In these embodiments, the real-world acoustic property set a1 to a2 is used. r This can be related to the physical construction of the real-world display device 102, which can affect the occlusion, reflection, and diffraction of audio content propagating through the real-world environment 100. In these embodiments, the physical construction of the real-world display device 102 may include the physical shape of the real-world display device 102 or the physical three-dimensional surface of the real-world display device 102, such as the geometry of the physical three-dimensional surface and / or the physical material of the physical three-dimensional surface. For example, the set of real-world acoustic properties a1 to a2 r This may include or involve a coordinate system for modeling the real-world environment 100, spatialization of audio content within the real-world environment 100, movement of objects within the real-world environment 100, transparency of objects within the real-world environment 100, and / or acoustic modeling of the real-world environment 100 in terms of occlusion, reflection, and / or diffraction.

[0024] After accessing the visual content-based acoustic parameters 250, the display device server 202 can model the real-world environment 100 in a virtual environment (e.g., based on the visual content-based acoustic parameters 250). In some embodiments, the display device server 202 can execute a simulation tool 204 to model the real-world environment 100 in a virtual environment. In these embodiments, the simulation tool 204 can be implemented as a game engine, or other simulation software that will be clear to those skilled in the art(s) to model the real-world environment 100 in a virtual environment without departing from the spirit and scope of this disclosure. Generally, the display device server 202 can model the acoustic properties of the real-world environment 100 (e.g., occlusion, reflection, and / or diffraction, etc.) in a virtual space (also called a virtual room) within a virtual environment.

[0025] As part of this modeling, the display device server 202 can define the three-dimensional geometry of the virtual space in terms of one or more architectural structures (such as the real-world display device 102) and / or one or more non-architectural structures within, for example, the real-world environment 100. Generally, one or more architectural structures refer to any suitable one or more structures designed for human habitation, and may include one or more residential, industrial, and / or commercial architectural structures to provide examples. Generally, one or more non-architectural structures refer to any suitable one or more structures not designed for human habitation, and may include one or more residential, industrial, and / or commercial non-architectural structures to provide examples. In some embodiments, the three-dimensional geometry of the virtual space can be used to aid in modeling the acoustic properties of the real-world environment 100 (e.g., occlusion, reflection, and / or diffraction, etc.) within the virtual space. In these embodiments, the three-dimensional geometry of the virtual space can be used to model the interaction of audio content with one or more architectural structures and / or one or more non-architectural structures within the real-world environment 100.

[0026] As part of this modeling, the display device server 202 can assign acoustic properties to one or more architectural structures and / or one or more non-architectural surfaces to model the interaction between audio content and these architectural structures and / or these non-architectural structures within the real-world environment 100. In some embodiments, the display device server 202 can assign acoustic properties to the real-world display device 102. In these embodiments, the display device server 202 can assign acoustic properties to the physical shape of the real-world display device 102 in virtual space or to the physical three-dimensional surface of the real-world display device 102, for example, in terms of the geometry and / or physical material of the physical three-dimensional surface. In some embodiments, the display device server 202 can assign acoustic properties to the real-world display device 102 based on acoustic parameters 250 based on visual content. In these embodiments, acoustic properties can be assigned to one or more types of visual content t1 to t2. r The set of real-world acoustic properties a1 to a r As described in this article, it is assigned to the real-world display device 102.

[0027] After modeling the real-world environment 100 in the virtual environment, the display device server 202 can simulate the playback of audio content by the real-world display device 102 within the model of the real-world environment 100 in the virtual environment, thereby determining the spatial region Z1 to Z2. m For one or more types of visual content t1 to t r audio parameters p 1,1 to p m,r In some embodiments, the display device server 202 may use a model of the real-world environment 100 in the virtual environment to simulate the interaction between audio content played back by the real-world display device 102 and the model of the real-world environment 100 in the virtual environment. In some embodiments, the display device server 202 may target one or more types of visual content t1 to t2. r Different types of visual content iteratively simulate audio content played back by real-world display device 102 within a model of real-world environment 100, in order to determine spatial regions Z1 to Z2 as described herein. m For one or more types of visual content t1 to t r audio parameters p 1,1 to p m,r In these embodiments, the display device server 202 can iteratively simulate one or more types of visual content t1 to t2 in terms of occlusion, reflection, and / or diffraction. r The interaction between the audio content and the model of the real-world environment 100 is used to determine the spatial region Z1 to Z2. mFor one or more types of visual content t1 to t r audio parameters p 1,1 to p m,r For example, the display device server 202 can iteratively simulate one or more types of visual content t1 to t2 in terms of occlusion, reflection, and / or diffraction. r The audio content interacts with one or more architectural structures (such as real-world display device 102) and / or one or more non-architectural structures of the model of the real-world environment 100. In these embodiments, the display device server 202 can display spatial regions Z1 to Z2. m For one or more types of visual content t1 to t r audio parameters p 1,1 to p m,r As described herein, the audio parameters are stored in spatial location-based audio parameters 108.

[0028] Exemplary operations for modeling an exemplary real-world environment

[0029] Figure 3 Exemplary operations for modeling an exemplary real-world environment according to some exemplary embodiments of the present disclosure are illustrated graphically. Figure 3 In the exemplary embodiment illustrated in the figure, the display device server 202 can develop a virtual model 300 of the real-world environment 100 in a virtual environment, and can subsequently simulate the played-back audio content within the virtual model 300 in the virtual environment to determine spatial regions Z1 to Z2. m For one or more types of visual content t1 to t r audio parameters p 1,1 to p m,r The following is about... Figure 2 The discussion is to describe exemplary actions, operations, routines, processes, etc., that can be performed by the display device server 202 to determine spatial regions Z1 to Z2. m For one or more types of visual content t1 to t r audio parameters p 1,1 to p m,r .like Figure 3 The diagram shows spatial regions Z1 to Z2. m This represents a concentric three-dimensional volume surrounding the virtual display device 302 in a virtual environment, such as a spherical shell. However, this example is not limiting, and those skilled in the art will recognize that spatial regions Z1 to Z2... m Other three-dimensional volumes (such as cubes, spheres, cones, pyramids, cuboids, or cylinders, etc., as provided as examples) can be used without departing from the spirit and scope of this disclosure.

[0030] like Figure 3 As illustrated herein, display device server 202 can model a virtual model 300 of real-world environment 100 in a virtual environment as described herein. In some embodiments, virtual model 300 may include virtual display device 302, which may be characterized as a computer-generated representation of real-world display device 102 in the virtual environment. In these embodiments, display device server 202 may model the physical shape or physical three-dimensional surface of real-world display device 102 in the virtual environment (e.g., in terms of geometry and / or physical materials of the physical three-dimensional surface) to develop a model of virtual display device 302. In these embodiments, display device server 202 may assign acoustic properties (e.g., occlusion, reflection, and / or diffraction, etc.) to virtual display device 302 in the virtual environment in a manner substantially similar to that described herein to model the interaction between audio content and virtual display device 302 in the virtual environment.

[0031] exist Figure 3 In the exemplary embodiment illustrated in the figure, the display device server 202 can simulate a virtual display device 302 in a virtual environment to virtual user devices 304.1 to 304. m Play back the audio content to determine the spatial region Z1 to Z2. m For one or more types of visual content t1 to t r audio parameters p 1,1 to p m,r .Although Figure 3 The exemplary embodiments illustrated in the figure show virtual user equipment 304.1 to 304. m The virtual model 300 is provided, but this is for illustrative purposes only. Rather, those skilled in the art will recognize that the virtual model 300 may include one or more virtual user devices, which are located in spatial regions Z1 to Z2. m The virtual mobile device is used to simulate virtual user devices 304.1 to 304.m without departing from the spirit and scope of this disclosure.

[0032] In some embodiments, the display device server 202 can target one or more types of visual content t1 to t2. r Iteratively simulate the transition from virtual display device 302 to virtual user devices 304.1 to 304. m Playback of audio content to determine spatial regions Z1 to Z2 m For one or more types of visual content t1 to t r audio parameters p 1,1 to p m,rIn these embodiments, the display device server 202 can iteratively simulate one or more types of visual content t1 to t2 in terms of occlusion, reflection, and / or diffraction. r The audio content interacts with the virtual model 300 to determine the spatial region Z1 to Z2. m audio parameters p 1,1 to p m,r In some embodiments, the display device server 202 can target spatial regions Z1 to Z2. m For one or more types of visual content t1 to t r To identify one or more parameters, characteristics, and / or attributes for the audio content, collectively referred to as audio parameters p 1,1 to p m,r In these embodiments, the audio parameter p 1,1 to p m,r It can include spatial regions Z1 to Z m For one or more types of visual content t1 to t r The sound intensity and / or time delay of the audio content, or its representation. For example, audio parameter p 1,1 to p m,r It can describe spatial regions Z1 to Z m For one or more types of visual content t1 to t r The various sound intensities and / or time delays of the audio content. In some embodiments, the display device server 202 can display spatial regions Z1 to Z2. m For one or more types of visual content t1 to t r audio parameters p 1,1 to p m,r As described herein, the audio parameters are stored in spatial location-based audio parameters 108.

[0033] Generally speaking, the display device server 202 can use spatial regions Z1 to Z2. m For one or more types of visual content t1 to t r audio parameters p 1,1 to p m,r To create a three-dimensional space to spatialize the audio content within the virtual model 300. The display device server 202 can identify the spatial region Z1 to Z2. m For one or more types of visual content t1 to t r audio parameters p 1,1 to p m,r To provide spatial regions Z1 to Z within the virtual model 300 in terms of distance, direction, orientation, and / or location. m Spatialization of audio content. For example, spatial regions Z1 to Z2. mFor one or more types of visual content t1 to t r audio parameters p 1,1 to p m,r It may include a left stereo channel and / or a right stereo channel to spatialize the audio content within the virtual model 300. In some embodiments, spatial regions Z1 to Z2 are defined. m For one or more types of visual content t1 to t r audio parameters p 1,1 to p m,r This can be related to: one or more types of visual content t1 to t2 played back by the virtual display device 302. r Virtual user equipment 304.1 to 304 within virtual model 300. m Three-dimensional coordinates, such as x, y, and z coordinates in a Cartesian coordinate system and / or r, θ, and z coordinates in a spherical coordinate system. Coordinates, etc.; the three-dimensional coordinates of the audio content being played back within the virtual model 300, such as x, y, and z coordinates in a Cartesian coordinate system and / or r, θ, and z coordinates in a spherical coordinate system. Coordinates, etc.; and / or virtual user equipment 304.1 to 304 within the virtual model 300. m The three-dimensional orientation (typically expressed as yaw, pitch, and roll). In these embodiments, the display device server 202 can determine the spatial region Z1 to Z2 based on the following: m audio parameters p 1,1 to p m,r To identify spatial regions Z1 to Z m For one or more types of visual content t1 to t r Various sound intensities and / or time delays of audio content: one or more types of visual content t1 to t r Virtual User Equipment (User Equipment) 304.1 to 304. m The three-dimensional coordinates of the audio content being played back within the virtual model 300, and / or the virtual user devices 304.1 to 304 within the virtual model 300. m The three-dimensional orientation. For example, for two-dimensional deformable visual content, the display device server 202 can approximate these sound intensities, represented as... And / or these time delays, denoted as ,as follows:

[0034] As another example, for three-dimensional volumetric visual content, the display device server 202 can approximate these sound intensities, represented as... And / or these time delays, denoted as ,as follows:

[0035] As a further example, for see-through visual content, the display device server 202 can approximate these sound intensities, represented as... And / or these time delays, denoted as ,as follows:

[0036] As a further example, for augmented reality visual content, the display device server 202 can approximate these sound intensities, represented as... And / or these time delays, denoted as ,as follows:

[0037] In these examples, This represents the intensity of the audio content at a three-dimensional coordinate at a spatial distance L. c Indicates the speed of sound. c m Indicated in medium m The speed of sound in G Indicates gain. ,in The distance from the three-dimensional coordinates of the spatial distance L to the virtual display device 302, This indicates the power level of the audio content played back by the virtual display device 302. , and These represent the reflection, diffraction, and absorption coefficients, respectively. F Represents a filter, and This represents the three-dimensional orientation at a spatial distance L. In some embodiments, this can be from virtual user equipment 304.1 to 304. m Obtain the three-dimensional orientation at a spatial distance L and use it for left / right panning calculations for further personalization, such as:

[0038] Exemplary types of visual content that can be replayed from an exemplary real-world environment

[0039] Figures 4A to 4DThe illustrations graphically depict one or more types of visual content that can be played back by an exemplary real-world environment according to some exemplary embodiments of the present disclosure. As described herein, real-world environment 100 can play back audio-visual content having both audio and visual elements. In some embodiments, one or more types of visual content that can be played back by real-world environment 100 may include two-dimensional morphing visual content, three-dimensional volumetric visual content, see-through visual content, and / or augmented reality visual content, each of which will be described herein.

[0040] like Figure 4A As illustrated, the real-world environment 100 can be displayed on the physical surface of the real-world display device 102 to the real-world user device 406 (e.g., real-world user devices 104.1 to 104). n One or more devices in the real-world display device 102 may play back two-dimensional distorted visual content 400. In some embodiments, the two-dimensional distorted visual content 400 may include visual content 402 (shown using dotted shading) on ​​the physical surface of the real-world display device 102 and audio content 404 that appears to be projected from the physical surface of the real-world display device 102 when played back by the real-world user device 406.

[0041] like Figure 4B As illustrated, real-world environment 100 can play back to real-world user device 406 three-dimensional volumetric visual content 410 that appears to be within the physical surface of real-world display device 102. In some embodiments, the three-dimensional volumetric visual content 410 may include visual content 402 (shown using dotted shading) within the physical surface of real-world display device 102 and audio content 404 that appears to be projected from within the physical surface of real-world display device 102 when played back by real-world user device 406.

[0042] like Figure 4C As illustrated, the real-world environment 100 can play back to the real-world user device 406 visual content 420 that appears to pass through the physical surface of the real-world display device 102. In some embodiments, the visual content 420 may include visual content 402 (shown using dotted shading) passing through the real-world display device 102 and audio content 404 that appears to be projected within the visual content 402 when played back by the real-world user device 406.

[0043] like Figure 4DAs illustrated, the real-world environment 100 can play back augmented reality content 430, which combines real-world 3D visual content and computer-generated 3D visual content, to a real-world user device 406. In some embodiments, the augmented reality content 430 may include visual content 402 (shown using dotted shading) surrounding the real-world display device 102 and audio content 404 that appears to be projected within the visual content 402 when played back by the real-world user device 406.

[0044] Exemplary Operation Control Flow for an Exemplary Real-World Environment

[0045] Figure 5 An exemplary operation control flow for an exemplary real-world user device within an exemplary real-world environment, according to an exemplary embodiment of the present disclosure, is illustrated. The following discussion is for the purpose of describing an exemplary operation control flow 500 for playback of audio content for an exemplary real-world user device as described herein. This disclosure is not limited to these exemplary operation control flows. Rather, it will be apparent to those skilled in the art(s) of the relevant nature that other operation control flows are within the scope and spirit of this disclosure. In some embodiments, operation control flow 500 may be generated by a real-world user device (such as real-world user devices 104.1 to 104 as described herein). n The application, software application, or other similar device may execute the audio content as described herein. In these embodiments, the real-world user device may execute an application, software application, or other similar device to play back audio content as described herein. In these embodiments, the application, software application, or other similar device, when executed by the real-world user device, may functionally cooperate with a real-world display device (such as a real-world display device to be provided as an example) to play back audio content as described herein.

[0046] At operation 502, operation control flow 500 transmits the spatial location of the real-world user equipment to the display device server. In some embodiments, the spatial location of the real-world user equipment may include the three-dimensional coordinates of the real-world user equipment within the exemplary real-world environment described herein, such as x, y, and z coordinates in a Cartesian coordinate system and / or r, θ, and z coordinates in a spherical coordinate system. Coordinates, etc. In some embodiments, the operation control flow 500 may estimate the three-dimensional coordinates of the real-world user equipment using signals such as Global Positioning System (GPS), Wi-Fi, and / or cellular phone signals (to provide some examples), as will be recognized by one or more of those skilled in the art, without departing from the spirit and scope of this disclosure. Alternatively or additionally, the spatial location of the real-world user equipment may include the three-dimensional orientation of the real-world user equipment within the exemplary real-world environment described herein, typically expressed in terms of yaw, pitch, and roll. In some embodiments, the operation control flow 500 may use accelerometers or gyroscopes, etc. (to provide some examples), as will be recognized by one or more of those skilled in the art, without departing from the spirit and scope of this disclosure. In some embodiments, the operation control flow 500 may transmit the spatial location of the real-world user equipment to a display device server, for example, once per millisecond, once per ten milliseconds, once per hundred milliseconds, once per second, once per ten seconds, and / or once per hundred seconds, etc.

[0047] At operation 504, operation control flow 500 may receive audio content from the display device server for playback. In some embodiments, operation control flow 500 may receive one or more digital audio content packets from the display device server, the audio content having been personalized for a spatial location from operation 502 in a manner substantially similar to that described herein. In some embodiments, the display device server may stream one or more digital audio content packets in real-time or near real-time via, for example, the Internet. In these embodiments, one or more digital audio content packets may be streamed according to an audio streaming protocol such as Hypertext Transfer Protocol (HTTP) Live Streaming (HLS), Real-Time Streaming Protocol (RTSP), or HTTP-based Dynamic Adaptive Streaming (ASH), to provide some examples.

[0048] At operation 506, operation control flow 500 can play back the audio content from operation 504. In some embodiments, operation control flow 500 can decompress one or more digital audio content packets from operation 504 to recover the audio content from operation 504. In these embodiments, operation control flow 500 can decompress one or more digital audio content packets from operation 504 according to an audio codec such as Advanced Audio Coding (AAC), MP3, or Opus, to provide some examples. In some embodiments, operation control flow 500 can play back the recovered audio content in real time or near real time. Alternatively or additionally, operation control flow 500 can provide the recovered audio content to a peripheral device (such as peripheral device 106) for playback.

[0049] Figure 6 Exemplary operation control flows for an exemplary display device server within an exemplary real-world environment are illustrated according to some exemplary embodiments of the present disclosure. The following discussion is for the purpose of describing an exemplary operation control flow 600 for an exemplary display device server to provide audio content for playback as described herein. The present disclosure is not limited to these exemplary operation control flows. Rather, it will be apparent to those skilled in the art (one or more) of the relevant art that other operation control flows are within the scope and spirit of this disclosure. In some embodiments, operation control flow 600 may be performed by a real-world user device, such as a real-world user device as described herein.

[0050] At operation 602, operation control flow 600 receives the spatial location of the real-world user equipment. In some embodiments, the spatial location of the real-world user equipment may include the three-dimensional coordinates of the real-world user equipment within a real-world environment as described herein, such as x, y, and z coordinates in a Cartesian coordinate system and / or r, θ, and z coordinates in a spherical coordinate system. Coordinates, etc. Alternatively or additionally, the spatial location of a real-world user equipment may include the three-dimensional orientation of the real-world user equipment within the exemplary real-world environment described herein, typically expressed in yaw, pitch, and roll. In some embodiments, the operation control flow 600 may receive the spatial location of the real-world user equipment, for example, once every millisecond, once every ten milliseconds, once every hundred milliseconds, once per second, once every ten seconds, and / or once every hundred seconds, etc.

[0051] At operation 604, operation control flow 600 personalizes the audio content based on the spatial location from operation 602. In some embodiments, operation control flow 600 may personalize the audio content based on the type of visual content associated with the spatial location from operation 602 (such as one or more types of visual content t1 to t2). r One type of audio parameter (p) accesses audio parameters in a manner substantially similar to that described in this paper. 1,1 to p m,r The audio parameters provided are examples. In these embodiments, the operation control flow 600 can access spatial location-based audio parameters (such as spatial location-based audio parameter 108) in a manner substantially similar to that described herein to identify the audio parameters. In some embodiments, the operation control flow 600 can process audio content based on the audio parameters to personalize the audio content for the spatial location from operation 602. In these embodiments, the operation control flow 600 can process the audio content (e.g., the amplitude and / or frequency of the audio content, etc.) to produce audio content having the audio parameters at the spatial location from operation 602.

[0052] At operation 606, operation control flow 600 transmits personalized audio content from operation 604 to a real-world user device for playback. In some embodiments, operation control flow 600 may compress the personalized audio content from operation 604 to provide one or more digital audio content packets. In these embodiments, operation control flow 600 may compress one or more digital audio content packets according to an audio codec such as Advanced Audio Coding (AAC), MP3, or Opus, to provide some examples. In some embodiments, operation control flow 600 may stream one or more digital audio content packets in real-time or near real-time via a display device server, for example, the Internet. In these embodiments, operation control flow 600 may stream one or more digital audio content packets according to an audio streaming protocol such as Hypertext Transfer Protocol (HTTP) Live Streaming (HLS), Real-Time Streaming Protocol (RTSP), or HTTP-based Dynamic Adaptive Streaming (ASH), to provide examples.

[0053] Exemplary computer systems that can be implemented in an exemplary real-world environment

[0054] Figure 7 The illustration shows a simplified block diagram of an exemplary computer system that can be implemented in an exemplary real-world environment according to some exemplary embodiments of the present disclosure. The following describes... Figure 7 The discussion is for the purpose of describing a computer system 700 that can be implemented within a real-world display device 102 and / or a display device server 202 as described herein.

[0055] exist Figure 7In the exemplary embodiment illustrated in the figure, computer system 700 includes one or more processors 702. In some embodiments, one or more processors 702 may include or be any of a microprocessor, graphics processing unit, or digital signal processor, and their electronic processing equivalents, such as application-specific integrated circuits (“ASICs”) or field-programmable gate arrays (“FPGAs”). As used herein, the term “processor” means a tangible data and information processing device that typically uses sequence transformations (also referred to as “operations”) to physically transform data and information. Data and information may be physically represented by electrical, magnetic, optical, or acoustic signals that can be stored, accessed, transmitted, combined, compared, or otherwise manipulated by a processor. The term “processor” can refer to a single processor and a multi-core system or multi-processor array, including graphics processing units, digital signal processors, digital processors, or combinations of these elements. A processor may be electronic, for example, including digital logic circuit systems (e.g., binary logic), or analog (e.g., operational amplifiers). A processor may also operate to support the performance of related operations in a “cloud computing” environment or as “Software as a Service” (SaaS). For example, at least some operations can be performed by a set of processors available at a distributed or remote system, which can be accessed via a communication network (e.g., the Internet) and via one or more software interfaces (e.g., application programming interfaces (APIs)). In some embodiments, computer system 700 may include an operating system, such as Microsoft Windows, Sun Microsystems' Solaris, Apple Computer's MacOS, Linux, or UNIX. In some embodiments, computer system 700 may also include a basic input / output system (BIOS) and processor firmware. The operating system, BIOS, and firmware are used by one or more processors 702 to control subsystems and interfaces coupled to the one or more processors 702. In some embodiments, the one or more processors 702 may include Pentium and Itanium from Intel, Opteron and Athlon from Advanced Micro Devices, and ARM processors from ARM Holdings.

[0056] like Figure 7As illustrated, computer system 700 may include machine-readable medium 704. In some embodiments, machine-readable medium 704 may further include main random access memory (“RAM”) 706, read-only memory (“ROM”) 708, and / or file storage subsystem 710. RAM 706 may store instructions and data during program execution, and ROM 708 may store fixed instructions. File storage subsystem 710 provides persistent storage for program and data files and may include hard disk drives, floppy disk drives, and associated removable media, CD-ROM drives, optical drives, flash memory, or removable media cartridges.

[0057] Computer system 700 may also include user interface input device 712 and user interface output device 714. User interface input device 712 may include an alphanumeric keypad, a keypad, pointing devices (such as a mouse, trackball, touchpad, stylus, or graphics tablet), a scanner, a touchscreen integrated into a display, audio input devices (such as a speech recognition system or microphone), gaze recognition, brainwave pattern recognition, and other types of input devices to provide some examples. User interface input device 712 may be connected to computer system 700 via wired or wireless means. Generally, user interface input device 712 is intended to include all possible types of devices and methods for inputting information into computer system 700. User interface input device 712 typically allows users to recognize objects, icons, text, etc., appearing on some type of user interface output device (e.g., a display subsystem). User interface output device 714 may include a display subsystem, a printer, a fax machine, or a non-visual display (such as an audio output device). Display subsystem may include a cathode ray tube (CRT), a flat panel device (such as a liquid crystal display (LCD)), a projection device, or some other device for creating visible images (such as a virtual reality system). The display subsystem may also provide non-visual displays, such as via audio output or tactile output (e.g., vibration). Generally, the user interface output device 714 is intended to include all possible types of devices and methods for outputting information from the computer system 700.

[0058] Computer system 700 may also include network interface 716 to provide an interface to external networks, including an interface to communication network 718, and to corresponding interface devices in other computer systems or machines via communication network 718. Communication network 718 may include a number of interconnected computer systems, machines, and communication links. These communication links may be wired links, optical links, wireless links, or any other devices used for information communication. Communication network 718 may be any suitable computer network, such as a wide area network (e.g., the Internet) and / or a local area network (e.g., Ethernet). Communication network 718 may be wired and / or wireless, and the communication network may use encryption and decryption methods, such as those available for virtual private networks. The communication network uses one or more communication interfaces that can receive data from other systems and send data to other systems. Examples of communication interfaces typically include Ethernet cards, modems (e.g., telephone, satellite, cable, or ISDN), (asynchronous) digital subscriber line (DSL) units, FireWire interfaces, USB interfaces, etc. One or more communication protocols may be used, such as HTTP, TCP / IP, RTP / RTSP, IPX, and / or UDP.

[0059] like Figure 7 As illustrated, one or more processors 702, machine-readable medium 704, user interface input device 712, user interface output device 714, and / or network interface 716 may be communicatively coupled to each other using bus subsystem 720. Although bus subsystem 720 is schematically shown as a single bus, alternative embodiments of the bus subsystem may use multiple buses. For example, RAM-based main memory may communicate directly with a file storage system using a direct memory access (“DMA”) system.

[0060] in conclusion

[0061] The detailed description section refers to the accompanying drawings to illustrate exemplary embodiments consistent with this disclosure. References to "exemplary embodiments" throughout this disclosure indicate that the described exemplary embodiments may include specific features, structures, or characteristics, but not every exemplary embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, such phrases do not necessarily refer to the same exemplary embodiments. Further, any feature, structure, or characteristic described in connection with exemplary embodiments may be included independently or in any combination of features, structures, or characteristics in other exemplary embodiments, whether or not explicitly described.

[0062] The detailed description is not intended to be limiting. Rather, the scope of this disclosure is defined only by the following claims and their equivalents. It should be understood that the claims are intended to be interpreted using the detailed description section rather than the abstract section. The abstract section may set forth one or more, but not all, exemplary embodiments of this disclosure, and therefore is not intended to limit this disclosure or the following claims and their equivalents in any way.

[0063] The exemplary embodiments described in this disclosure are provided for illustrative purposes and are not intended to be limiting. Other exemplary embodiments are possible, and modifications may be made to the exemplary embodiments while maintaining the spirit and scope of this disclosure. This disclosure has been described by means of functional building blocks that illustrate the implementation of specified functions and their relationships. For ease of description, the boundaries of these functional building blocks have been arbitrarily defined herein. Alternative boundaries may be defined provided that the specified functions and their relationships are properly performed.

[0064] The embodiments of this disclosure can be implemented using hardware, firmware, software applications, or any combination thereof. Embodiments of this disclosure can also be implemented as instructions stored on a machine-readable medium that can be read and executed by one or more processors. The machine-readable medium can include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computing circuit system). For example, the machine-readable medium can include non-transitory machine-readable media such as read-only memory (ROM); random access memory (RAM); disk storage media; optical storage media; flash memory devices; and other media. As another example, the machine-readable medium can include transient machine-readable media such as electrical, optical, acoustic, or other forms of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.). Further, firmware, software applications, routines, and instructions may be described herein as performing certain actions. However, it should be recognized that these descriptions are merely for convenience, and such actions are actually produced by a computing device, processor, controller, or other device executing the firmware, software application, routine, instructions, etc.

[0065] The specific implementation of exemplary embodiments fully reveals the general nature of this disclosure. Others can readily modify and / or adapt such exemplary embodiments for various applications without undue experimentation by applying the knowledge of one or more persons skilled in the art, without departing from the spirit and scope of this disclosure. Therefore, based on the teachings and guidance set forth herein, such adaptations and modifications are intended within the meaning and various equivalents of the exemplary embodiments. It should be understood that the wording or terminology herein is for descriptive purposes and not for limitation, and that the terminology or terminology of this specification shall be interpreted by one or more persons skilled in the art based on the teachings herein.

Claims

1. A method for delivering audio content to a real-world user device in a real-world environment, the method comprising: The display device server in the real-world environment receives the spatial location of the real-world user device within the real-world environment. The display device server identifies one type of visual content from multiple types of visual content played back by real-world display devices in a real-world environment; The display device server personalizes the audio content based on the type of visual content and the spatial location. as well as The display device server transmits personalized audio content to real-world user devices for playback.

2. The method of claim 1, wherein the personalization further includes Identify a spatial region including the said spatial location from multiple spatial regions within a real-world environment; and Access audio parameters associated with the spatial region and the type of visual content from multiple audio parameters associated with multiple spatial locations and multiple types of visual content.

3. The method of claim 2, wherein the plurality of spatial regions comprise a plurality of concentric three-dimensional volumes surrounding the display device server within a real-world environment.

4. The method of claim 2, wherein the personalization includes processing audio content to generate audio content having the audio parameters at the spatial location.

5. The method of claim 2, further comprising: Develop a virtual model of a real-world environment in a virtual environment, the virtual model including a virtual display device associated with a real-world display device and multiple virtual user devices associated with multiple real-world user devices in the multiple spatial regions; as well as Simulate playback of multiple audio contents from the virtual display device to the plurality of virtual user devices to determine the plurality of audio parameters for the plurality of types of visual content within the plurality of spatial locations.

6. The method of claim 5, wherein developing the virtual model comprises: Define the three-dimensional geometry of virtual space within a virtual environment; Identify the set of real-world acoustic properties associated with the various types of visual content from a set of multiple sets of real-world acoustic properties associated with those types; as well as The set of real-world acoustic properties is assigned to the three-dimensional surface of the virtual display device to model the interaction between the multiple audio contents and the display device server in the real-world environment.

7. The method of claim 1, further comprising playing back visual content associated with the audio content at the spatial location by a real-world display device associated with the display device server.

8. A display device server for transmitting audio content to real-world user devices in a real-world environment, the display device server comprising: The memory is configured to store instructions; as well as A processor is configured to execute the instructions, which, when executed by the processor, configure the processor to: Receive the spatial location of real-world user devices within the real-world environment. Identify one type of visual content from multiple types of visual content played back by real-world display devices within a real-world environment. For the visual content of the aforementioned type, the audio content is personalized based on the spatial location, and Personalized audio content is transmitted to real-world user devices for playback.

9. The display device server of claim 8, wherein the instructions, when executed by the processor, configure the processor to: Identify a spatial region including the said spatial location from multiple spatial regions within a real-world environment; and Access audio parameters associated with the spatial region and the type of visual content from multiple audio parameters associated with multiple spatial locations and multiple types of visual content.

10. The display device server of claim 9, wherein the plurality of spatial regions comprise a plurality of concentric three-dimensional volumes surrounding the display device server within a real-world environment.

11. The display device server of claim 9, wherein the instructions, when executed by the processor, configure the processor to process audio content to generate audio content having the audio parameters at the spatial location.

12. The display device server of claim 9, wherein the instructions, when executed by the processor, further configure the processor: Developing a virtual model of a real-world environment within a virtual environment, the virtual model including a virtual display device associated with a real-world display device and multiple virtual user devices associated with multiple real-world user devices within the multiple spatial regions; and Simulate playback of multiple audio contents from the virtual display device to the plurality of virtual user devices to determine the plurality of audio parameters for the plurality of types of visual content within the plurality of spatial locations.

13. The display device server of claim 12, wherein the instructions, when executed by the processor, configure the processor to: Define the three-dimensional geometry of virtual space within a virtual environment; Identify a set of real-world acoustic properties associated with said types of visual content from a set of multiple sets of real-world acoustic properties associated with said types of visual content; and The set of real-world acoustic properties is assigned to the three-dimensional surface of the virtual display device to model the interaction between the multiple audio contents and the display device server in the real-world environment.

14. The display device server of claim 8, wherein the instructions, when executed by the processor, further configure the processor to play back visual content associated with the audio content at the spatial location on a display device associated with the display device server.

15. A display device for transmitting audio content to a real-world user device in a real-world environment, the display device comprising: A visual display, configured to play back visual content; as well as The display device server is configured as Receive the spatial location of real-world user devices within the real-world environment; Identify one type of visual content from multiple types of visual content played back by real-world display devices in a real-world environment; For the visual content of the aforementioned type, the audio content is personalized based on the spatial location; as well as Personalized audio content associated with visual content is transmitted to real-world user devices for playback.

16. The display device of claim 15, wherein the display device server is configured to: Identify a spatial region including the said spatial location from multiple spatial regions within a real-world environment; and Access audio parameters associated with the spatial region and the type of visual content from multiple audio parameters associated with multiple spatial locations and multiple types of visual content.

17. The display device of claim 16, wherein the plurality of spatial regions comprise a plurality of concentric three-dimensional volumes surrounding the display device server within a real-world environment.

18. The display device of claim 16, wherein the display device server is configured to process audio content to generate audio content having the audio parameters at the spatial location.

19. The display device of claim 16, wherein the display device server is further configured to: Developing a virtual model of a real-world environment within a virtual environment, the virtual model including a virtual display device associated with a real-world display device and multiple virtual user devices associated with multiple real-world user devices within the multiple spatial regions; and Simulate playback of multiple audio contents from the virtual display device to the plurality of virtual user devices to determine the plurality of audio parameters for the plurality of types of visual content within the plurality of spatial locations.

20. The display device of claim 19, wherein the display device server is configured to: Define the three-dimensional geometry of virtual space within a virtual environment; Identify a set of real-world acoustic properties associated with said types of visual content from a set of multiple sets of real-world acoustic properties associated with said types of visual content; and The set of real-world acoustic properties is assigned to the three-dimensional surface of the virtual display device to model the interaction between the multiple audio contents and the display device server in the real-world environment.

Images