DISTRIBUTED AND SYNCHRONIZED CONTROL SYSTEM FOR AMBIENT SIGNALS DURING MULTIMEDIA PLAYBACK

Abstract

A procedure (900) that includes the following: Providing (905, 907, 909) a media data set (210) comprising media content data (211) and environment effect metadata (113) that define a set of environment events (231-233), each corresponding to a media timestamp (221-223) of a plurality of media timestamps; and for each environmental event (231 - 233) in the set environmental events, Identifying (911) a protocol timestamp (252) for a communication protocol, wherein the protocol timestamp (252) corresponds to the media timestamp (221 - 223) of the environment event (231 - 233), Generating (917) a message (250) for transmission according to the communication protocol, wherein the message (250) associates the environment event (260) with the protocol timestamp (252), and Addressing (919) the message (250) to one or more environment effect generators (141 - 145), characterized in that the method (900) further includes the following: for each environment effect generator (141-145) of the one or more environment effect generators, storing (901) an orientation of the environment effect generator (141-145), wherein the orientation of the environment effect generator (141-145) includes a position of the environment effect generator (141-145) and a direction of the environment effect generator (141-145); for each environment event (231-233) in the set of environment events, identifying (913) a subset of the environment effect generators (141-145) for generating the environment event (231-233) based on the orientation of the environment event (231-233) and an orientation of each environment effect generator (141-145) in the one or more environment effect generators; and for each environmental effect generator (141 - 145) in the subset: Computation (915) of a set of one or more instructions (261 – 263) for generating the environment event (231 - 233) in conjunction with other environment effect generators (141 - 145) in the subset based on the orientation of the environment effect generator (141 - 145) and the orientation of the environment event (231 - 233), and Transfer (921) the set of one or more instructions (261 - 263) to the environment effect generator (141 - 145).

Description

RELATED PATENT APPLICATIONS

[0001] This patent application is an international patent application of US patent application number 15 / 921,473, filed on March 14, 2018, which claims priority over provisional US patent application number 62 / 507,650, filed on May 17, 2017. AREA OF INVENTION

[0002] This disclosure relates to the field of multimedia playback devices and, in particular, environmental effects for multimedia playback. STATE OF THE ART

[0003] Home theater systems are audiovisual systems that typically include numerous consumer electronics components, such as a large, high-resolution display device (e.g., television screen, projector, etc.), a multi-channel speaker and audio playback system, and video playback equipment (e.g., set-top boxes, optical disc players, streaming media devices). Such systems allow users to experience movies, music, games, and other media provided by internet-based subscription streaming services, physical media (e.g., digital versatile discs (DVDs) or Blu-ray discs), downloaded files, satellite links, or other sources.

[0004] A common configuration for a modern home theater system includes a large flat-screen HDTV (High-Definition Television) or a video projector and a 5.1 or 7.1-channel surround sound amplifier with multiple speakers positioned at various locations around the room, surrounding the user's viewing position. The speaker system typically includes at least one subwoofer to accurately reproduce the low-frequency sounds of a movie or music track. The TV may be a smart TV, which provides built-in internet and home network access and is capable of running applications and receiving and storing media content from various sources.

[0005] Furthermore, the disclosures in US 2011 / 0 188 832 A1, US 2013 / 0 198 786 A1 and US 2011 / 0 125 789 A1 may be helpful for understanding the present invention.

[0006] US Patent 2011 / 0 188 832 A1 discloses a method and apparatus for generating sensory media. The method comprises: generating sensory effect metadata (SEM) for a sensory effect applied to media; and outputting the SEM. The SEM contains sensory effect declaration information that defines a sensory effect type distinct from a core sensory effect type, and sensory effect representation information for displaying the sensory effect.

[0007] The 2013 / 0 198 786 A1 standard describes systems, methods, software, and data structures that provide an immersive user experience. A video program can be linked to predefined actions performed by one or more dynamic devices located near the video device. Dynamic devices can perform actions synchronously with the media program. Dynamic devices can be controlled wirelessly, and different versions of a media program can be delivered to different receiving devices based on the number of dynamic devices known to be connected to each receiving device.

[0008] US Patent 2011 / 0 125 789 A1 discloses a method and a device for displaying sensory effects, as well as a computer-readable recording medium for storing metadata for sensory commands. The method for displaying sensory effects comprises receiving sensory effect metadata containing sensory effect information, obtaining the sensory effect information by analyzing the sensory effect metadata, and generating sensor device command metadata for controlling sensor devices that correspond to the sensory effect information. The sensor device command metadata includes sensor device command description information for controlling the sensor devices. DESCRIPTION OF THE PRESENT INVENTION

[0009] The present invention relates to a method according to claim 1 and a multimedia device according to claim 9. Advantageous embodiments of the invention may include features of dependent claims. BRIEF DESCRIPTION OF THE FIGURES

[0010] The present revelation is illustrated in an exemplary and non-limiting way in the figures of the accompanying drawings. Fig. Figure 1 is a block diagram of a multimedia system according to one embodiment. Fig. Figure 2A illustrates a media data set that includes environmental effects metadata, according to one embodiment. Fig. Figure 2B illustrates a message for transmitting environmental event data according to one embodiment. Fig. Figure 3 illustrates a block diagram of an environmental effects generator according to one embodiment. Fig. Figure 4 illustrates two views of an ambient effects generator and loudspeaker setup according to one embodiment. Fig. Figure 5A illustrates playback positions for a set of environmental effect generators in viewing spaces according to one embodiment. Fig. Figure 5B illustrates recharging positions for a set of environmental effect generators in viewing spaces according to one embodiment. Fig. Figure 6 illustrates a coordinate system for specifying locations in observation spaces according to one embodiment. Fig. Figure 7 illustrates a spherical coordinate system according to one embodiment. Fig. Figure 8A illustrates media tracks that include an environment effect metadata track, according to one embodiment. Fig. Figure 8B illustrates an ADSR profile (ADSR = Attack, Decay, Sustain, Release) according to one embodiment. Fig. Figure 9 illustrates a process for generating environmental effects according to one embodiment. DETAILED DESCRIPTION

[0011] The following description sets forth numerous specific details, such as examples of specific systems, components, methods, and so on, to provide a good understanding of various embodiments of the claimed subject matter. However, it will be clear to a person skilled in the art that at least some embodiments can be carried out without these specific details. In other cases, well-known components or methods are not described in detail or are presented in a simple block diagram format to avoid unnecessarily obscuring the claimed subject matter. The specific details set forth are therefore merely exemplary. Certain implementations may deviate from these exemplary details and still be considered to be within the scope of the claimed subject matter.

[0012] One embodiment of a home theater system includes hardware that produces ambient lighting effects displayed simultaneously with the presentation of video content on a main display, such as an HDTV screen. One approach to delivering ambient lighting effects involves generating the lighting effects based on the color and intensity of the video image displayed on the main screen and projecting the ambient light from light sources located behind the main screen. Using this approach, the ambient light is confined to the back of the main screen and is not distributed throughout the room. Furthermore, this approach provides no mechanism for the content creator or a third party to input scripts for ambient lighting events or other environmental effects.

[0013] One approach to providing ambient lighting or other environmental effects (e.g., motion actuators, seat shakers, strobe or smoke effects, etc.) distributed throughout the viewing area involves multiple environmental effect generators placed at different locations within the space. However, this increases the number of components in the system; wired connections between the many separate components (i.e., surround sound speakers and environmental effect generators) and the main display or media player can result in a large number of wires running throughout the space.

[0014] In one embodiment, a home theater system that provides visual effects (e.g., ambient lighting) and / or other environmental effects comprises multiple environmental effect generators, such as ambient light sources, integrated into the same enclosures as surround-sound loudspeakers and wirelessly controlled by a multimedia device (e.g., HDTV, set-top boxes, etc.). In such a system, both audio data and instructions for controlling the environmental effect generators are transmitted wirelessly, eliminating the need for signal cabling. The loudspeakers and the environmental effect generators are powered by inductively rechargeable batteries, further eliminating the need for power wiring. The multimedia device also achieves synchronous presentation of environmental effect events with the playback of media content (e.g., videos, videos, etc.).The system achieves a latency of less than one millisecond for music or films by correlating a timestamp associated with the playback of the media content with a timestamp of the wireless communication protocol. Each of the ambient effect generators can be controlled separately, allowing effects to be localized; for example, different ambient lighting effects can be positioned at / in different predefined locations and directions within the space. The system thus provides a mechanism by which ambient effects can be widely distributed across viewing spaces, synchronized with media content, and remotely created for reproduction within those viewing spaces (e.g., in a user's home).

[0015] Fig. Figure 1 illustrates an embodiment of a multimedia system 100 comprising several ambient effect generators 141-145. The multimedia system 100 includes a multimedia device 120 that receives media content (comprising audio and video content) from a media stream 110 or a file system 120. The multimedia device 120 decodes the media content and transmits the video data to be presented on a display 130. The multimedia device 120 extracts metadata 113 from the media stream 110 and / or the ambient effect metadata file 152, which defines ideal virtual locations for ambient effect events, and then generates instructions for generating the specified ambient effects, which are wirelessly transmitted to the ambient effect generators 141-145. As shown in Figure 1, the multimedia device 120 is used to generate the specified ambient effects. Fig. As illustrated in Figure 1, the environmental effect generators 141-144 include configurable ambient light sources, and the environmental effect generator 145 includes a motion actuator that can be used to produce seat-wobble or shake effects or other types of movement. In alternative embodiments, environmental effect generators may include devices that control the temperature, airflow, humidity, scent (e.g., a programmable scent generator), water pressure (e.g., for a fountain), the movement or orientation of an entire environment (e.g., a motion simulator), or other properties that may be perceived in an environment.

[0016] The media data set is transported by a media stream 110 and comprises video and audio data as a sequence of video frames 111 and audio frames 112, which can be played back by the multimedia device 120. The media data set also includes the environment effect metadata 113, which defines environment events and associates each of these events with a corresponding media timestamp. The media timestamp indicates a time during the playback of the media content (i.e., the audio and video frames 111 and 112). In one embodiment, the environment effect metadata 113 is multiplexed with the video frames 111 and the audio frames 112 in a format such as the Moving Picture Experts Group 2 Transport Stream (MPEG2-TS). In one embodiment, the environment effect metadata 113 can be remultiplexed with existing, pre-processed audio and video streams without re-encoding and re-processing of the audio and video streams (e.g.(for media content encoded in MPEG-2 or MPEG-4 format). Media stream 110 is received via internet streaming, a satellite or cable TV connection, an optical disc reader (e.g., Blu-ray or DVD), or another source.

[0017] Accordingly, a creator of original content, such as a film or TV show producer, can use video editing software to generate a sequence of environmental effects encoded in the environmental effects metadata 113, so that this sequence can be played back at predetermined times in the playback timeline along with the video frames 111 and the audio frames 112. This environmental effects track is multiplexed with its associated audio and video content data and can therefore be distributed along with the audio and video content, whether via a data stream (e.g., internet streaming) or a physical medium (e.g., data file or optical disc).

[0018] In one embodiment, the environmental effect metadata 113 is used independently of any video or audio content (e.g., to provide ambient lighting or other effects in an art installation, a live performance, or another environment). In this case, the environmental effect metadata 113 can be generated locally or remotely and downloaded as a file or stream over a network or otherwise obtained. The metadata 113 is synchronized with the local system time and distributed to the environmental effect generators in the environment to produce the desired lighting and / or other environmental effects.

[0019] Environmental effect sequences can also be produced for gaming systems, enabling the dynamic generation of environmental effects for interactive three-dimensional (3D) environments. Different environmental effect sequences can be triggered by different in-game events or by a player's virtual position within a 3D-rendered environment. In one embodiment, the environmental effect metadata need not have been specifically produced for the purpose of generating environmental effects; for example, environmental events could be generated based on haptic metadata or metadata from a control input (e.g., joystick or gamepad). Accordingly, these types of metadata can also be considered environmental effect metadata.

[0020] In one embodiment, the multimedia device 120 also includes a file system 121 for storing data files. Accordingly, the environmental effects can be defined in an environmental effect metadata file 152, which is stored in the file system 121 together with a corresponding audio / video media file 151 that provides the media content for playback. Alternatively, the environmental effect metadata can be stored in the same file as the audio and video content (e.g., multiplexed according to one of the MPEG formats).

[0021] The multimedia device 120 comprises a media input 122, which receives a media data set, comprising the media content data and the environment effect metadata 113, from the media stream 110 (or from files 151 and 152 in the file system 121). The media input 122 provides the media data set to a decoder module 123. The multimedia system 100 comprises a display 130, which is connected to the multimedia device 120 and is capable of displaying videos. The decoder module 123 extracts the video frames 111 from the media stream 110 and transmits a video signal to the display module 130, which presents the video to the user. In one embodiment, the display 130 is an HDTV; alternatively, the display 130 can be implemented using a projector or other display device.In one embodiment, the display 130 is located in the same physical enclosure as the multimedia device 120; in alternative embodiments, the display 130 is located in an enclosure separate from the multimedia device 120, such as when the multimedia device 120 is implemented in a set-top box, a streaming device, a home theater personal computer (HTPC), or another separate device. In one embodiment, the playback of video frames on the display 130 can be delayed to account for any latency that may be added for processing the environment effect metadata 113, so that the environment events stored in scripts can be more easily synchronized with the video playback.

[0022] In one embodiment, the positions of the ambient effect generators 141-145 need not exactly match the orientations (i.e., locations and directions) of the ambient events to be generated, as defined in the ambient effect metadata 113 or the ambient effect metadata file 152. The multimedia device 120 includes a reordering module 126 that computes a set of instructions for controlling the ambient effect generators 141-145 to generate an approximation of the ideal set of ambient effects, as specified in the ambient effect metadata 113 or the ambient effect metadata file 152. For example, the ambient effect metadata 113 may define a virtual ambient light effect that is ideally oriented toward a location where, in fact, no controllable ambient light source (e.g., 141-144) is positioned in the user's premises.To reproduce the required ambient light, the reordering module 126 identifies the ambient light sources that are closest to the defined ideal position and distributes the full intensity of the required light among the closest actual light sources based on their distances to the ideal position.

[0023] In order for the rearrangement module 126 to perform this rearrangement process, the detection module 124 first determines the presence and orientations of each of the ambient effect generators 141-145 that can be controlled by the multimedia device 120. During the detection process, the detection module 124 gathers information about the effect-generating capabilities of the ambient effect generators 141-145. These capabilities can be reported by the ambient effect generators 141-145 themselves via wireless communications, which are received at the wireless communication interface 128 and forwarded to the detection module 124. The detection module 124 also gathers information about the orientation of each of the ambient effect generators 141-145. In one embodiment, each ambient effect generator 141-145 determines its own orientation (including location and direction) and wirelessly reports its orientation to the detection module 124.Alternatively, the detection module 124 can determine the orientations of the environmental effect generators 141-145 without active participation from the environmental effect generators 141-145, or the detection module 124 and the environmental effect generators 141-145 can work together to jointly determine the orientations of all environmental effect generators 141-145 (e.g., by signal triangulation, latency measurements, etc.).

[0024] In one embodiment where each ambient effect generator includes an ambient light source in the same housing as a surround sound loudspeaker, the position of the ambient light source can be derived based on the loudspeaker's function (e.g., front left, front right, center, etc.) within the surround sound system. This information can be provided to the detection module 124 by the ambient effect generators 141-145 themselves or configured by a user.

[0025] Once the detection module 124 has determined an orientation for each of the environment effect generators 141-145, the detection module 124 stores the orientation information in a configuration memory 125. The configuration memory 125 stores an orientation (comprising a position and a direction) for each of the environment effect generators 141-145, which can be used by the reordering module 126 to generate approximations for the environment events stored in scripts.

[0026] During the discovery process for ambient lighting effect generators 141-144, the light control capabilities for each of the ambient effect generators 141-145 are also determined. For example, some ambient light sources may be able to produce light only within a certain color range or with a certain intensity, or be able to project light with a certain minimum or maximum beam angle or within a certain range of directions. These light control capabilities are also used by the reordering module 126, so that actual light sources are not required to generate lighting effects that exceed their respective capabilities. Accordingly, light control capabilities determined during the discovery process are also stored in the configuration memory 125. Capabilities for other types of ambient effect generators (e.g., frequency, intensity, etc.) are also stored.The settings for the motion actuator 145 can also be determined and stored in the configuration memory 125. After the discovery process, the discovery module 124 initiates a calibration of the environmental effect generators 141-145.

[0027] Once the ambient effect generators 141-145 have been located and calibrated, the decoder module 123 begins decoding the data in the media stream 110 or the files 151 and 152. The reordering module 126 processes the ambient effect metadata 113 or the ambient effect metadata file 152 to generate the appropriate instructions for controlling the ambient effect generators 141-145 to produce ambient effects that approximate the ambient effects requested in the ambient effect metadata 113 or the ambient effect metadata file 152. For the ambient light sources 141-144, the reordering module 126 determines which of the ambient light sources 141-144 should be turned on or off and also determines other parameters for generating the ambient light effects, such as light color, intensity, direction, etc.The reordering module 126 can also determine that certain required environmental effects are not rendered because no environmental effect generator is available to generate the effect at the required position or with the required intensity.

[0028] The reordering module 126 thus identifies a subset of the environment effect generators 141-145 to be used for generating each environment event required in the environment effect metadata 113 or the environment effect metadata file 152, along with instructions for controlling the subset of environment effect generators based on the orientation of each of the environment effect generators 141-145 in the multimedia system 100 based on the required orientation of the environment event. The instructions produced by the reordering module 126 are encapsulated, along with timing information, in a message that is to be transmitted wirelessly to the environment effect generators 141-145. The environment effect metadata 113 or 152 associate each environment event with a media timestamp that indicates when the environment event is to be presented during the playback of the audio and video frames 111, 112.For each environmental event identified by the environmental effects metadata, the Decoder Module 123 identifies a protocol timestamp that corresponds to the media timestamp. The protocol timestamp is a timestamp used by a communication protocol; for example, such a timestamp is provided in wireless communication protocols that support Wi-Fi TimeSync™. In various embodiments, the wireless protocol can be a Wi-Fi protocol, a Bluetooth protocol, or another wireless protocol that either supports timestamping or can be modified to support timestamping. In one embodiment, a wired communication protocol (such as an Ethernet protocol) can be used instead of a wireless protocol.

[0029] For each environmental event specified in the environmental effect metadata 113 or 152, the encapsulation module 127 generates one or more messages for transmission according to a wireless communication protocol, associating the environmental event with the specified protocol timestamp. More precisely, in the generated message, the encapsulation module 127 associates the protocol timestamp with the instructions for generating the environmental event (as determined by the reordering module 126).

[0030] In an embodiment where some or all of the ambient effect generators 141-145 are located in the same enclosures as the surround-sound loudspeakers of the multimedia system 100, the encapsulation module 127 includes in the one or more messages one or more audio frames 112, associated with the protocol timestamp, to specify audio elements to be played back concurrently with the ambient event. The encapsulation module 127 directs the resulting message or messages (which comprise instructions and audio frames, each associated with a protocol timestamp) to the one or more ambient effect generators in the subset of ambient effect generators identified by the reordering module 126 based on the orientation of the ambient event and the orientations of the ambient effect generators 141-145 as previously described.

[0031] The wireless communication interface 128 receives the message from the encapsulation module 127 and transmits the message to the ambient effect generators 141-145. The message thus carries the audio frames and a portion of the original ambient effect metadata 113 (in the form of instructions) defining the ambient event to the loudspeakers and ambient effect generators 141-145, respectively, where they are played back synchronously with the video displayed by the display 130. In one embodiment, the message is wirelessly transmitted to each of the ambient effect generators 141-145 according to a wireless communication protocol that supports a protocol timestamping mechanism, such as Wi-Fi TimeSync™.

[0032] Fig. Figure 2A illustrates different types of data in a media data set 210 according to one embodiment. The media data set 210 comprises media content data 211, which represents the content that the user views, listens to, or otherwise consumes. The media content data 211 includes a sequence of video frames 111, representing a video that can be displayed on a monitor, television, projector, etc., and a sequence of audio frames 112, representing audio recordings for playback on a loudspeaker. In general, each of the audio and video media frames includes a set of recordings associated with a media timestamp that identifies a time at which the media frame is to be presented during playback. In one embodiment, the media content data could include speech or other sounds generated in real time (e.g., from a telephone or video conference call) and / or from warnings, messages, alarms, etc.The generated audio elements are included. The media data set 210 includes environment effect metadata 113, which associate each of any number of environment events 231-233 with a respective media timestamp 221-223.

[0033] Fig. Figure 2B illustrates a message 250, generated by the encapsulation module 127 and transmitted via the wireless communication interface 128 to the environmental effect generators 141-145, according to one embodiment. The message 250 comprises a protocol timestamp 252, associated with an audio frame 253 (representing audio recordings to be played back at a time corresponding to the protocol timestamp 252) and an environmental event 260, to be generated at a time corresponding to the protocol timestamp 252. The environmental event 260 is defined in the form of instructions 261-263 for controlling an environmental effect generator. The address 251 identifies one of the environmental effect generators 141-145 to which the message 250 is addressed (e.g., an environmental effect generator capable of generating the environmental event 260).Instructions 261-263 are provided to the environmental effect generator to produce the environmental event 260. For example, instructions 261-263 could specify the color, intensity, direction, etc., for an ambient lighting effect. For a motion actuator, instructions 261-263 could specify the amplitude, frequency, displacement, direction, duration, etc., of a motion to be generated.

[0034] Fig. Figure 3 illustrates an ambient effect generator 141 according to one embodiment. The other ambient effect generators 142-145 comprise similar components to the ambient effect generator 141, except that the ambient effect generator 145 includes a motion actuator instead of a light source 304. The ambient effect generator 141 comprises a loudspeaker 305, a configurable light source 304, and a power circuit (comprising a recharging circuit 310 and an accumulator 311) to power the loudspeaker 305, the configurable light source 304, and other components in the ambient effect generator 141. In one embodiment, the accumulator 311 is a rechargeable accumulator that can be recharged by the recharging circuit 310.In one embodiment, the recharging circuit 310 comprises a coil located near a lower surface of the ambient effect generator 141, so that the accumulator 311 can be inductively recharged by placing the ambient effect generator 141 on an inductive charging surface.

[0035] The ambient effects generator 141 receives messages wirelessly from the multimedia device 120 via a wireless receiver 301. Based on the address 251, the wireless receiver 301 determines whether a received message 250 is addressed to the ambient effects generator 141, and if so, it transmits the received message to a decoder module 302. The decoder module 302 extracts the audio frame 253 and instructions 261-263 from the message 250, then plays the audio elements in the loudspeaker 305 and causes the ambient event 260 to be generated at the configurable light source 304 at the time specified by the protocol timestamp 252, according to instructions 261-263.

[0036] In one embodiment, all of the ambient effect generators 141-145 have device clocks that are synchronized with each other, so that the presentation of audio and ambient events is synchronized across all ambient effect generators 141-145. In one embodiment, the device clock 303 in the ambient effect generator 141 is synchronized with the device clocks in other ambient effect generators using a synchronization mechanism provided by the wireless communication protocol. The device clock synchronization process can thus be based on the periodic adjustment of the device clock 303 based on detected latencies of wirelessly transmitted messages received by the other ambient effect generators, or it can be performed by other methods.

[0037] Upon receiving message 250, the decoder module 302 compares the protocol timestamp 252 with the current protocol time as specified by the device clock 303. If the protocol timestamp 252 specifies a time equal to or later than the current protocol time, the decoder module 302 plays the audio frame 253 in the loudspeaker 305 and causes the configurable light source 304 to generate the environmental event 260 when the protocol timestamp 252 matches the current protocol time (i.e., immediately or after a reasonable delay). If the protocol timestamp 252 specifies a time earlier than the current protocol time, the decoder module 302 discards message 250.

[0038] For an ambient effect generator 141, which includes a configurable light source 304, the calibration process is performed by a calibration module 306 to determine a color correction matrix for compensating for a color space shift due to changes in the environment. During the calibration process, the calibration module 306 causes the configurable light source 304 to emit a known light pattern, which is reflected by the environment. The reflected light is measured, and the calibration module 306 calculates the correction matrix based on the difference between the reflected light and an expected result associated with the known light pattern. The correction matrix is ​​supplied to the decoder module 302. In one embodiment, the decoder module 302 adjusts parameters in instructions 261-263 according to the correction matrix to reproduce the required ambient light effect more accurately.The configurable light source 304 thus generates light based on the color correction matrix and based on the set of instructions 261-263 for generating the ambient light effect.

[0039] Apart from color correction, the calibration process also compensates for the dynamic range of light intensity, which can be influenced by factors such as the reflectivity of the walls in the viewing area. In one embodiment, this calibration procedure includes commanding all available light sources to switch on and off and measuring the maximum and minimum intensity values.

[0040] The environmental effect generator 141, as it appears in Fig. Figure 3 illustrates an ambient light effect generator 141 that includes a loudspeaker 305; however, in alternative embodiments, it may comprise a configurable light source 304 in a housing without the loudspeaker 305. Accordingly, the placement of ambient light effect generators is not necessarily limited to locations where surround-sound loudspeakers are located in the viewing area. Furthermore, some embodiments may include an ambient light effect generator that controls a direct light source (e.g., projecting downwards from the ceiling or towards the user), as opposed to ambient light sources that project light laterally onto walls or other surfaces away from the user.

[0041] Fig. Figure 4 illustrates two different views of an assembly comprising a loudspeaker and an ambient effects generator, according to one embodiment. The assembly includes a base 401 and a housing 403, which is connected to the base 401 by a support 402. The accumulator 311 and the recharging circuit 310 are located in the base 401. The loudspeakers 305 and 404 are mounted in the housing 403. The loudspeaker 305 is a primary loudspeaker for the audio channel and faces the front of the housing 403, while the loudspeaker 404 is an upfiring loudspeaker angled from the top of the housing 403 to reflect sound off the ceiling of the room. The light source 304 is a configurable ambient light source that faces the rear of the housing 403 in the opposite direction to the loudspeaker 305.When the main speaker 305 faces the user, the light source 304 faces away from the user toward a wall or other objects that can reflect ambient light. The light source 304 covers an area curved along multiple axes, allowing light to be projected over a range of different directions. In one embodiment, the light source 304 internally comprises several lighting elements, such as light-emitting diodes (LEDs), facing in different directions, so that the direction of the light effect can be controlled by adjusting the intensities of the different lighting elements. In one embodiment, the light source 304 comprises a matrix of red, green, blue, and white (RGBW) LEDs.

[0042] The Fig. 5A and Fig. Figure 5B illustrates playback and recharge positions for the ambient effect generators 141-144 and their respective loudspeakers according to one embodiment. Fig. In 5A, the ambient effect generators 141-144 are optimally positioned to play media content (i.e., a film or music) with ambient lighting effects and / or other environmental effects embedded in scripts. The discovery and calibration procedures are performed while the ambient effect generators 141-144 are in this position. Fig. Figure 5B illustrates a recharging position for the ambient effect generators 141-144, with each ambient effect generator 141-144 placed on an inductive charging surface of one of the recharging pads 501 and 502. The recharging pads 501 and 502 can be located outside of heavily frequented areas of the premises so that the ambient effect generators 141-144 do not block walkways while being recharged.

[0043] Fig. Figure 6 illustrates a coordinate system for specifying the locations of environmental effects (such as virtual lighting effects) in a home theater system design according to one embodiment. The coordinate system is illustrated as x, y, and z axes extending from an origin point 600. The origin 600 approximates the optimal position from which a user of the home theater system views or listens to the video or audio media content. In one embodiment, the origin 600 is at the same height as the center of the display 130, and one axis (in this case, the y-axis) is aligned with the user's line of sight to the display 130.

[0044] Fig. Figure 6 further illustrates standalone ambient light generators 601 and 602, which are ambient effect generators comprising configurable light sources and not attached to a loudspeaker. Because the ambient light generators 601 and 602 are not attached to loudspeakers, they can be positioned more flexibly in locations other than the designated loudspeaker locations in the surround sound system. Similar to the configurable light source 304, the configurable light sources in the ambient light generators 601 and 602 can be similarly configured and recharged. In one embodiment, the ambient light generators 601 and 602 each comprise a matrix of multiple color LEDs arranged in a curved structure to project light with different colors, directions, and beam widths.

[0045] Fig. Figure 7 illustrates a spherical coordinate system in which a position, as defined by the International Organization for Standardization (ISO) standard ISO 80000-2:2009 and previously ISO 31-11 (1992), is identified by two angles θ and φ and a distance r, according to one embodiment. Using this coordinate system, a location for a virtual light source relative to a fixed position in the viewing space (e.g., the user's optimal viewing point) can be defined as a set of coordinates (r, θ, φ). In one embodiment, the coordinates for virtual light sources are included as part of the environmental effect metadata 113 in a metadata track. Each set of coordinates defining a location of a virtual light source represents part of the metadata that defines an environmental event (e.g., environmental event 231) associated with a time in the metadata track (e.g., 10 ...B. as indicated by the media timestamp 221).

[0046] Fig. Figure 8A illustrates a metadata track 800, an audio track 810, and a video track 820 in a media data set 210 according to one embodiment. Fig. 8A Time progression from left to right. The illustrated sections of metadata track 800, audio track 810, and video track 820 are presented simultaneously over the same time span. Some media formats (e.g., MPEG2-TS) define an absolute time; thus, frames or events in data tracks 800, 810, and 820 have presentation times specified relative to the same absolute timeline. In other words, events 801–803, defined in metadata track 800, occur on the same timeline as the playback of video track 820 and audio track 810, so events 801–803 consistently occur simultaneously with the same audio and video frames when tracks 800, 810, and 820 are played back.In one embodiment, if no external timing track is provided, events 801-803 are directly associated with specific audio or video frame numbers and are thus consistently triggered simultaneously with the same audio and / or video frames.

[0047] As illustrated, metadata track 800 specifies a global setup event 801 and two sets of virtual light setup events 802 and 803. The global setup event 801 and the virtual light setup events 802 and 803 represent environment events in the environment effect metadata 113 in the original media stream 110. Metadata track 800 defines environment events generically and is transformed by the reordering process performed by the reordering module 126 into a specific set of messages for controlling the actually available set of environment effect generators. In one embodiment, the reordered messages maintain the same format as the original messages.

[0048] In one embodiment, the global setup event is defined by a global setup message that specifies parameters for defining the color space and dynamic range of light intensity (e.g., brightness) for lighting effects to be rendered during the multimedia playback sequence. In one embodiment, the global setup message includes the following instructions: transmission time, execution time, color space definition, minimum brightness, and maximum brightness.

[0049] The transmission time specifies the time of delivery of the global setup message (e.g., the wireless transmission of the message), while the execution time specifies the time for executing the message (e.g., the time during which the instructions in the message are carried out in the addressed environmental effects generator). In one embodiment, the execution time is represented by a media timestamp (e.g., 221, 222, 223) that indicates an absolute time in the audio / video playback timeline.

[0050] The color space definition in the global setup message specifies a color space for defining ambient light colors for the ambient effect generators 141-144. Possible color spaces include, but are not limited to, the CIE 1931 XYZ color space (CIE = International Commission on Illumination), the standard RGB color space (RGB = Red Green Blue) (e.g., ITU-R (International Telecommunication Union Radiocommunication Sector) recommendations BT.709, BT.601, and BT.2020), and the HSV and HSL color spaces (HSV = Hue, Saturation, Value; HSL = Hue, Saturation, Lightness). The color space used by the ambient light source 304 need not be the same as the color space used by the display 130. For example, UHDTV displays (UHDTV = Ultra-High-Definition Television) use the BT.The 2020 definition applies, however, the ambient light source 304 could be configured to use a different color space, such as HSV, if the creator of the ambient lighting effects wants more direct control over the brightness. The minimum and maximum brightness values ​​specify a dynamic range for the ambient lighting effects defined in metadata track 800.

[0051] Once the ambient effect generators 141-144 are initially configured according to the global setup message, a subsequent virtual light setup message (e.g., according to 802, 803) is used to generate a light effect. In one embodiment, the virtual light setup message includes the following instructions: transmission time, execution time, a unique light identifier (ID), spherical coordinates, direction, aperture, color coordinates, and an ADSR color pattern (ADSR = Attack, Decay, Sustain, Release) for each color component.

[0052] Similar to the global setup message, the transmission time in the virtual light setup message specifies the time of delivery of the virtual light setup message, while the execution time specifies the time for executing the message (e.g., the time during which the instructions in the message are carried out in the addressed ambient effect generator). In the case of a light setup message, the execution time corresponds to the time during which the light effect is generated by the configurable light source and can be seen by the user. In one embodiment, the execution time is represented by a media timestamp (e.g., 221, 222, 223) that indicates an absolute time in the audio / video playback timeline.

[0053] In one embodiment, each configurable light source (e.g., 304) renders only the specific lighting effect defined in a virtual lighting setup message. If a configurable light source is still rendering a lighting effect when an execution time for another virtual lighting setup message is reached, the more recent virtual lighting setup message overrides the earlier one. Rendering of the previously requested lighting effect is stopped, and the configurable light source 304 begins rendering the newly requested lighting effect as instructed by the most recently received virtual lighting setup message.

[0054] The unique ID is a 64-bit value that uniquely identifies a virtual light object, used to denote the virtual light during playback. The unique ID is consistent throughout the entire multimedia playback sequence, but is not necessarily unique among the IDs for virtual light objects in different multimedia sequences.

[0055] In the virtual lighting setup message, the spherical coordinates are represented as a set of coordinate values ​​(r, θ, φ) that specify a spatial position for the virtual lighting effect within the room space relative to the user's optimal viewing position. This position represents an ideal location for the lighting effect, which can be reordered by the reordering module 126 for the specific set of available configurable light sources. The direction value in the virtual lighting setup message indicates whether the virtual lighting effect should be directed outwards (e.g., towards the walls of the room) or inwards (e.g., towards the viewer). The aperture value specifies the width of the light beam in degrees, allowing for the specification of lighting effects such as spotlights or wide omnidirectional lights.

[0056] The color coordinates in the virtual lighting setup message specify a color for the requested lighting effect using the color system defined by the color space definition in the global setup message. These color coordinates also define the brightness of the virtual lighting effect. The ADSR pattern specified for each color in the virtual lighting setup message indicates how the intensity of each color in the lighting effect changes over time.

[0057] In one embodiment, the configurable light source 304 is capable of displaying a light pattern (e.g., comprising multiple pixels). Accordingly, a virtual light setup message can additionally specify an image for generating a lighting effect that can be projected onto surfaces in the premises or viewed directly. In one embodiment, a specified image has a lower resolution than the video played on the main display 130. In one embodiment, a sequence of such lighting effect images is transported in a subtrack of the main video format (and is thus compressed using the same compression protocol as for the main video data). The subtrack is linked to a queue of outgoing virtual light setup messages, so that when the images are received, they are included in outgoing virtual light setup messages.

[0058] Fig. Figure 8B illustrates an ADSR profile according to one embodiment. In the virtual light setup message, each color component of the color coordinates is associated with four ADSR values: Attack (861), Decay (862), Sustain (863), and Release (864), which define the increase and decrease of the intensity (851) of that color component over time (852). The Attack value (861) specifies the duration over which the intensity (851) of the color component increases, while the Decay value (862) specifies the duration over which the intensity (851) decreases to an intensity level specified by the Sustain value (863). The Release value (864) specifies the duration over which the intensity (851) of the light decreases to zero. If the same ADSR values ​​are defined for each of the red, blue, and green color components of an RGB light, the light changes its brightness while maintaining the same hue.Changes in hue over time can be achieved by using different ADSR values ​​for different color components.

[0059] Fig. Figure 9 illustrates a process 900 for generating environmental effects based on environmental effect metadata according to one embodiment. The operations in the environmental effect generation process 900 are performed by the components of the multimedia system 100, comprising the multimedia device 120 and the environmental effect generators 141-145.

[0060] In block 901, the detection module 124 in the multimedia device 120 performs a detection process to determine the orientations and capabilities of each of the ambient effect generators 141-145 in the system 100. The orientation of each ambient effect generator includes a position and a direction. In one embodiment, the orientation for each ambient effect generator in the space is recorded as coordinates (r, θ, φ) in a spherical coordinate system having an origin 600 at the user's optimal viewing location.

[0061] In block 903, if the media type played in the multimedia system 100 is audio or video media, process 905 continues with block 905. Audio and video media include music, movies, television shows, etc. In block 905, the multimedia device 120 receives a media data set from a media data stream 110 at media input 122. Alternatively, the media data set can be retrieved as files 151 and 152 from a file system 121 of the multimedia device 120. The media data set is received at media input 122, which represents a communication interface (e.g., an internet connection, an internal bus, etc.) at which the media data set can be received.

[0062] The media dataset comprises media content data, which includes media frames (e.g., video frames 111 and audio frames 112) that can be played in sequence to produce audio or video elements. The media dataset also includes environment effect metadata 113, which defines a set of environment events 231-233. Each of the environment events 231-233 corresponds to a respective media timestamp 221-223. Each of the video frames 111 and audio frames 112 is also associated with a media timestamp that indicates the temporal sequence in which the frames should be played back. The media content (comprising the audio and video data) and the environment effect metadata 113 are multiplexed together into a single data stream 110 or a single file using a file format such as MPEG2-TS or MPEG4.In block 907, the decoder 123 of the multimedia device 120 demultiplexes the data stream 110 or the file so that the audio elements 112, the video elements 111 and the environment effect metadata 113 can be processed and presented differently.

[0063] In one embodiment, the multimedia system 100 also supports the computer rendering of interactive three-dimensional (3D) environments, such as video gaming applications. In block 903, if the media type is a game or other 3D-rendered experience, process 900 continues to block 909. In block 909, the multimedia system 100 generates environment effect metadata based on the interactive display of the 3D environment. In one embodiment, the environment effect metadata is generated in a personal computer or gaming console separate from the multimedia device 120. Alternatively, the environment effect metadata can be generated in the multimedia device 120. In one embodiment, the environment effect metadata defines lighting effects and / or other environmental effects that mimic the virtual in-game environment.

[0064] From block 907 or block 909, process 900 continues with block 911. In block 911, decoder module 123 identifies a protocol timestamp that corresponds to a media timestamp for the next environment event 231 in stream 110 (or file 152). The protocol timestamp is a timestamp used by a communication protocol to append timing information to messages.

[0065] In block 913, the reordering module 126 begins a reordering process to enable the set of environment effect generators 141-145 to approximate the environment event 231. In block 913, the reordering module 126 identifies a subset of the environment effect generators 141-145 for generating the environment event 231 based on the orientation of the environment event 231 and also based on the orientation and capabilities of each environment effect generator 141-145. For example, for an environment event that defines a virtual light source where no actual configurable light source is present, the reordering module 126 could identify a subset of environment effect generators capable of producing light and that are closest to the ideal location of the virtual light source.The reordering module 126 can then distribute the light intensity between the nearest environmental effect generators according to their respective distances to the ideal location of the virtual light source.

[0066] In block 915, the reordering module 126 generates new instructions for controlling one or more environmental effect generators in the identified subset, based on reordering the ideal environmental effect locations according to the orientations and capabilities of the actual environmental effect generators in the space. The instructions are commands that cause the environmental effect generators in the identified subset to generate the required environmental event 231 in relation to one another. Instructions can command a light source (e.g., 304) to generate light that has a specific color, intensity, direction, etc.

[0067] In block 917, the encapsulation module 127 generates a message 250 for transmission according to the wireless communication protocol. The generated message 250 associates the environmental event 260 with a protocol timestamp 252 and includes instructions 261-263 to cause an environmental effect generator (e.g., 141) to generate the environmental event 260. If the environmental event is an ambient lighting effect, instructions 261-263 specify parameters for controlling a configurable light source 304 in the ambient effect generators 141. Message 250 also associates at least one audio frame 253 with the protocol timestamp 252. In block 919, the encapsulation module 127 addresses message 250 to the ambient effect generator 141 by specifying an address for the ambient effect generator 141 in the address field 251 of message 250.

[0068] In block 921, the wireless communication interface 128 transmits message 250 wirelessly to the environmental effect generators 141-145. The audio frame 253 and the section of environmental effect metadata 113 defining the environmental event 260 (in the form of the rearranged instructions 261-263) are thus transmitted to the environmental effect generators 141-145. In one embodiment, message 250 is sent to all of the environmental effect generators 141-145.

[0069] In block 923, message 250 is received at the environmental effects generator 141. Message 250 is received at a wireless receiver 301 of the environmental effects generator 141. If message 250 is not addressed to the environmental effects generator 141 in block 924, it is discarded in block 929. If message 250 is addressed to the environmental effects generator 141 in block 924, the process continues with block 925. In block 925, decoder 302 compares the protocol timestamp 252, which is associated with environmental event 260, with a clock value specified by the device clock 303. If in block 927 the protocol timestamp 252 is greater than or equal to the device clock value (i.e., it indicates the same time or a later time), then process 900 continues with block 931.

[0070] In block 931, the decoder 302 executes instructions 261-263 to cause the configurable light source 304 to generate the required lighting effect to produce the environmental event 260. In one embodiment, the wireless protocol timestamp has a sub-millisecond accuracy, so that the rendering of the environmental event at the environmental effect generator 141 occurs within one millisecond before or after the presentation of a media frame (e.g., an audio or video frame) associated with the same media timestamp as the environmental event. Accordingly, lighting effects and other environmental effects can be synchronized with audio and / or video events with sub-millisecond accuracy, so that any timing errors are imperceptible to humans.

[0071] If the log timestamp in block 927 is earlier than the clock value of device clock 303 (i.e., it indicates an earlier time), then message 250 is outdated and is discarded in block 929. This mechanism prevents the environment effect generator 141 from rendering environment effects with a delay.

[0072] In process 900, blocks 911-921 are repeated for each of the environment events 231-233 defined in the environment effect metadata 113, so that the sequence of environment events 231-233 is replicated concurrently with the playback of the media content 211. Each environment event 231-233 can be reordered to generate a single message or multiple messages containing instructions for controlling a specific subset of the environment effect generators 141-145, in order to generate an approximation of the generically defined environment effects. Thus, a created sequence of environment effects can be approximated even if the specific orientations of the environment effect generators are not known at the time of creation.The system thus provides a mechanism by which environmental effects can be widely distributed in the viewing spaces, synchronized with media content, and remotely created in order to be reproduced in the viewing spaces (e.g., a user's home).

[0073] Various modifications can be made to the preceding embodiments; for example, signals described as being activated at a high voltage can instead be activated at a low voltage, or specified components can be replaced by other components that have similar functionality. As described herein, conductive electrodes that are "electrically connected" or "electrically coupled" can be coupled such that a conductive path with a relatively low resistance exists between the conductive electrodes.Quantities, measures, or other values ​​described as "essentially" the same may be nominally equal but need not be exactly the same (with variations due to manufacturing tolerances, environmental conditions, quantization or rounding errors, and / or other factors), or may be sufficiently close to the same to achieve an intended effect or benefit.

[0074] The embodiments described herein include various operations. These operations may be performed by hardware components, software, firmware, or a combination thereof. As used herein, the term "coupled with" may mean direct or indirect coupling through one or more intermediary components. All signals provided via various buses described herein may be time-division multiplexed with other signals and provided via one or more common buses. Additionally, the interconnection between circuit components or blocks may be shown as buses or as individual signal lines. Each of the buses may alternatively be one or more individual signal lines, and each of the individual signal lines may alternatively be a bus.

[0075] Certain embodiments may be implemented as a computer program product, which may include instructions stored on a computer-readable medium. These instructions can be used to program a general-purpose or specialized processor to perform the described operations. A computer-readable medium includes any mechanism for storing or transmitting information in a form (e.g., software, processing application) that can be read by a machine (e.g., a computer). The computer-readable storage medium may, but is not limited to, include: a magnetic storage medium (e.g., a floppy disk); an optical storage medium (e.g., a CD-ROM); a magneto-optical storage medium; a fixed-order memory (ROM); a working memory (RAM); a erasable programmable memory (e.g., a programmable memory).EPROM and EEPROM); a flash memory or other type of medium suitable for storing electronic instructions.

[0076] Additionally, some implementations can be carried out in distributed computing environments where the computer-readable medium is stored on and / or executed by more than one computer system. Furthermore, the information transmitted between computer systems can be transferred either by pull or push over the transmission medium connecting the computer systems.

[0077] Although the operations of the method(s) herein are shown and described in a specific order, the order of operations of each method may be modified so that certain operations may be performed in reverse order or so that a certain operation may be performed at least partially concurrently with other operations. In another embodiment, instructions or sub-operations of individual operations may take place in an intermittent and / or alternating manner.

[0078] In the preceding patent description, the claimed subject matter was described with reference to specific embodiments thereof. However, it will be obvious that various modifications and changes can be made to it without deviating from the broader scope of the invention as set out in the appended claims. Accordingly, the patent description and the drawings are to be regarded in an illustrative rather than a limiting sense.

Claims

[1] A method (900) comprising the following: Providing (905, 907, 909) a media data set (210) comprising media content data (211) and environment effect metadata (113) that define a set of environment events (231-233), each corresponding to a media timestamp (221-223) of a plurality of media timestamps; and for each environmental event (231 - 233) in the set environmental events, Identifying (911) a protocol timestamp (252) for a communication protocol, wherein the protocol timestamp (252) corresponds to the media timestamp (221 - 223) of the environment event (231 - 233), Generating (917) a message (250) for transmission according to the communication protocol, wherein the message (250) associates the environment event (260) with the protocol timestamp (252), and Addressing (919) the message (250) to one or more environment effect generators (141 - 145), characterized by , that the procedure (900) further includes the following: for each environment effect generator (141-145) of the one or more environment effect generators, storing (901) an orientation of the environment effect generator (141-145), wherein the orientation of the environment effect generator (141-145) includes a position of the environment effect generator (141-145) and a direction of the environment effect generator (141-145); for each environment event (231-233) in the set of environment events, identifying (913) a subset of the environment effect generators (141-145) for generating the environment event (231-233) based on the orientation of the environment event (231-233) and an orientation of each environment effect generator (141-145) in the one or more environment effect generators; and for each environmental effect generator (141 - 145) in the subset: Computation (915) of a set of one or more instructions (261 – 263) for generating the environment event (231 - 233) in conjunction with other environment effect generators (141 - 145) in the subset based on the orientation of the environment effect generator (141 - 145) and the orientation of the environment event (231 - 233), and Transfer (921) the set of one or more instructions (261 - 263) to the environment effect generator (141 - 145). [2] Method (900) according to claim 1, wherein: The communication protocol is a protocol for wireless communication. the addressing of the message (250) to one or more environment effect generators (141 - 145) is based on an alignment of the environment event (231 - 233), and The method further includes the wireless transmission (921) of the message (250) for each environmental event (231 - 233) to a set of environmental effect generators (141 - 145) comprising one or more environmental effect generators (141 - 145). [3] Method (900) according to claim 2, wherein: the media content data (211) include a large number of audio frames (112); the message (250) associates one or more of the multiple audio frames (253) with the protocol timestamp (252); and the procedure furthermore the transmission, in the message (250), of one or which includes several audio frames (112) and a section of the environment effect metadata (113) that describes the environment event (231 - 233). [4] Method (900) according to claim 1, wherein the media content data (211) comprise a plurality of media frames (111, 112) and wherein the method further comprises, for each environmental event (231 - 233) in the set of environmental events, generating the environmental event (231 - 233) on the one or more environmental effect generators (141 - 145) within one millisecond before or after presenting a media frame (111, 112) associated with the media timestamp (221 - 223) corresponding to the environmental event (231 - 233). [5] Method (900) according to claim 1, further comprising: Receiving (905) the media data set (210) as a data stream (110) at a communication interface, wherein the media content data (211) includes a plurality of video frames (111), each of which is associated with one of the plurality of media timestamps (221-223), and wherein the plurality of video frames (111) and the environment effect metadata (113) are multiplexed in the data stream (110); and demultiplexing (907) the data stream (110). [6] Method (900) according to claim 1, further comprising generating (909) the environment effect metadata (113) based on an interactive representation of a three-dimensional environment. [7] Method (900) according to claim 1, further comprising for each environmental event (231 - 233) in the set of environmental events: Receiving (923) the message (250) at an environment effect generator (141 - 145) of one or more environment effect generators; Comparing (925) the protocol timestamp (252) associated with the environmental event (231-233) with a clock value generated by a device clock (303) of the environmental effect generator (141-145); and in response to determining (927) that the protocol timestamp (252) is equal to or later than the clock value, generating (931) the environment event (231 - 233) based on one or more instructions (261 - 263) in the message (250). [8] Method according to claim 1, wherein the generated message (250) specifies for each environmental event (231 - 233) in the set of environmental events one or more instructions (261 - 263) for controlling a light source (304) in one of the environmental effect generators (141 - 145), wherein the one or more instructions (261 - 263) specify at least one of a color, intensity and direction of light to be generated by the light source (304). [9] A multimedia device (100) comprising the following: a media input (122) configured to provide a media record (210) including media content data (211) and environment effect metadata (113) defining a set of environment events (231 - 233), each corresponding to a media timestamp (221 - 223) of a plurality of media timestamps; a decoder module (123) configured to identify for each environment event (231 - 233) in the set of environment events a protocol timestamp (252) for a communication protocol, wherein the protocol timestamp (252) corresponds to the media timestamp (221 - 223) of the environment event (231 - 233); an encapsulation module (127) that is configured for each environment event (231 - 233) in the set of environment events for the following: Generating a message (250) for transmission according to the communication protocol, wherein the message (250) associates the environment event (260) with the protocol timestamp (252), and Addressing the message (250) to one or more environment effect generators (141 - 145); and a communication interface (128) configured to transmit the message (250), characterized by , that the multimedia device (100) further includes: a configuration memory (125) configured to store, for each environment effect generator (141-145) of the one or more environment effect generators, an orientation of the environment effect generator (141-145), wherein the orientation of the environment effect generator (141-145) includes a position of the environment effect generator (141-145) and a direction of the environment effect generator (141-145); and a reordering module (126) that is configured for each environment event (231 - 233) in the set of environment events for the following: Identifying a subset of environment effect generators (141-145) for generating the environment event (231-233) based on the orientation of the environment event (231-233) and an orientation of each environment effect generator (141-145) in the one or more environment effect generators (141-145), and calculating a set of one or more instructions (261-263) for generating the environment event (231-233) in the subset of environment effect generators (141-145) based on the orientation of each environment effect generator (141-145) in the subset and the orientation of the environment event (231-233), wherein the communication interface (128) is further configured to provide, for each environment effect generator (141-145) in the subset, the set of one or more instructions (261-263) for generating the environment event (231-233) in the subset. to transmit several instructions (261 - 263) to the environment effect generator (141 - 145). [10] Multimedia device (100) according to claim 9, further comprising a display module (130) configured to display the media content (211), wherein the media content (211) includes video data. [11] Multimedia device (100) according to claim 9, wherein: the communication protocol is a protocol for wireless communication, the encapsulation module (127) is configured to address the message (250) to one or more environment effect generators (141-145) for each environment event (231-233) in the set of environment events based on an orientation of the environment event (231-233), and the communication interface (128) is further configured to wirelessly transmit the message (250) for each environmental event (231 - 233) to a set of environmental effect generators (141 - 145) comprising one or more environmental effect generators (141 - 145). [12] Multimedia device (100) according to claim 11, wherein: the media content data (211) include a large number of audio frames (112); the message (250) associates one or more of the multiple audio frames (253) with the protocol timestamp (252); and the communication interface (128) is further configured to transmit the one or more audio frames (253) and a section of the environment effect metadata (113) describing the environment event (231 - 233) in the message (250). [13] Multimedia device (100) according to claim 9, further comprising one or more ambient effect generators (141 - 145), wherein each ambient effect generator (141 - 145) of the one or more ambient effect generators (141 - 145) comprises the following: a wireless receiver (301) configured to receive the message (250) for each environment event (231 - 233) in the set Environment Events; a device clock (303) that is synchronized with each of one or more device clocks (303) in the other environmental effect generators (141-145) of the one or more environmental effect generators (141-145); and a configurable light source (304) that is configured to generate the environmental event (231 - 233) in response to determining that the protocol timestamp (252) is equal to or later than a clock value generated by the device clock (303). [14] Multimedia device (100) according to claim 9, wherein the generated message (250) specifies for each environmental event (231 - 233) in the set of environmental events one or more instructions (261 - 263) for controlling a light source (304) in one of the environmental effect generators (141 - 145), wherein the one or more instructions (261 - 263) specify at least one of a color, intensity and direction of light to be generated by the light source (304).

Images