Remote live scene control system, method, and technique

Abstract

Apparatus, methods, systems, and techniques are provided for providing special effects wirelessly using devices plugged into a standard electrical outlet. Exemplary embodiments provide an apparatus and associated software application for remote and live control of special effects (hereinafter "Remote Special Effects System" or "RSES") using special effects (SE) devices such as individually addressable LEDs, LED strips, fog and smoke machines, and the like. An exemplary RSES described herein includes one or more SE controller devices, each plugged into a standard electrical outlet and each connected to one or more SE devices. Each SE controller is wirelessly connected to the Internet (or other wide area network) so that it can respond to DMX (or other protocol) commands sent by a remote application to produce synchronized special effects in ad-hoc created zones of the SE controller by issuing corresponding commands specific to the connected SE device.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Patent Application No. 63 / 248,991, entitled “REMOTE LIVE SCENE CONTROL SYSTEM, METHODS, AND TECHNIQUES,” filed September 27, 2021, which is incorporated by reference in its entirety.

[0002] The present disclosure relates to methods, techniques, and systems for special effects technology, and in particular to location-independent methods, techniques, and systems for control of lighting and other special effects such as light wave, electrical, and magnetic device output, haptic feedback, sound, and the like, using wireless communication and a controller device that plugs into an electrical wall outlet. [Background technology]

[0003] There are a myriad of electronic control systems for creating special effects such as lighting effects in theaters, music, and other venues. These systems combine well-known protocols to control all types of theater lighting, fog machines, and other special effects systems. For many years, such special effects systems have been hardwired and controlled by computing systems that allow a person to control the lights to create effects (e.g., flashes, colored lights, gradients, lightning, etc.) or be programmed to automatically control them based on the timing of other factors. The DMX protocol was originally developed for theater lighting, but has become a standard for communicating with DMX-friendly devices to make the devices operate in a certain way, such as turning lights a certain color and flashing, activating a fog machine, or the like. The DMX protocol is to lighting and other DMX special effects devices what "MIDI" is to voice control. The most common lighting control protocols in use today include Art-Net, sACN / E1.31, and DMX512. Today, most lighting or other stage effects equipment can be controlled using these protocols, including moving lights, LED screens, fog and haze machines, and laser displays. Typically, DMX controlled devices are connected together into a universe (of 512 separately addressable channels) using DMX cables. In some scenarios, DMX consoles are being replaced by software running on, for example, a personal computer, connecting via USB to a control device (such as a DMX USB interface), and then communicating via the DMX cable that controls the lighting (or other) devices in that universe.Exemplary DMX software consoles include ArtNetominator (accessible at "https: / / www.lightjams.com / artnetominator"), DMXking (accessible at "https: / / dmxking.com / control-software"), Smart Show (accessible at "http: / / smartshow.lighting / free-dmx-software / "), and DMX-Workshop (accessible at "https: / / art-net.org.uk / resources / dmx-workshop / ").

[0004] Currently, there are several DMX over Ethernet protocols for communicating to such devices over wired Ethernet. Examples of such protocols include Art-Net, PathPort, ShowNet, sACN, and ETC Net2. These protocols essentially wrap DMX packets with IP addresses to address them to lighting fixtures, such as LEDs, in the venue. Art-Net is a UDP-based protocol that generally runs over a local area network, such as Ethernet. This includes functions such as fader levels for individual lights, the position of movable lights, and management functions for managing nodes in the DMX universe. DMX systems such as those described above are often expensive. Similarly, such venues typically require the reliability that often accompanies wired Ethernet connections, so they generally use wired lighting fixtures to connect to a DMX controller (a DMX console, or a DMX control device that connects to a computer running a DMX software console).

[0005] This patent or application document contains at least one drawing executed in color. Copies of this patent or patent application publication containing color drawing(s) will be provided by the Patent Office upon request and payment of the necessary fee. [Brief description of the drawings]

[0006] [Figure 1] 1 is an exemplary remote special effects system that uses multiple special effects controller devices to remotely control special effects devices. [Diagram 2] 1 is a schematic flow diagram illustrating how an exemplary remote special effects system operates according to an exemplary configuration. [Diagram 3] FIG. 2 is a flow diagram of a typical SE-enabled application with an exemplary remote special effects system. [Figure 4A] 1 is an exemplary special effect that can be produced by an exemplary remote special effects system. [Figure 4B] 1 is an exemplary special effect that can be produced by an exemplary remote special effects system. [Figure 4C] 1 is an exemplary special effect that can be produced by an exemplary remote special effects system. [Figure 4D] 1 is an exemplary special effect that can be produced by an exemplary remote special effects system. [Figure 4E] 1 is an exemplary special effect that can be produced by an exemplary remote special effects system. [Figure 5A] 10 is a block diagram of an exemplary remote special effects system that may be used to produce the special effects shown in FIG. [Figure 5B] 10 is a block diagram of an exemplary remote special effects system that may be used to produce the special effects shown in FIG. [Figure 5C] 10 is a block diagram of an exemplary remote special effects system that may be used to produce the special effects shown in FIG. [Figure 6A] 1 provides three different photographs of a printed circuit board implementing an Emanator device, according to an illustrative embodiment. [Figure 6B] 1 provides three different photographs of a printed circuit board implementing an Emanator device, according to an illustrative embodiment. [Figure 6C]1 provides three different photographs of a printed circuit board implementing an Emanator device, according to an illustrative embodiment. [Figure 7A] Provides component layout information for Emanator devices. [Figure 7B] Provides component layout information for Emanator devices. [Figure 7C] Provides component layout information for Emanator devices. [Figure 8] FIG. 2 is a wiring schematic diagram of an exemplary printed circuit board for implementing a special effects controller. [Figure 9] FIG. 1 is an example block diagram of an example computing system that may be used to practice embodiments of the components of the remote special effects system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] The embodiments described herein provide an apparatus and associated software applications, methods, and techniques for remote control of special effects (hereinafter "Remote Special Effects System" or "RSES") using special effects devices such as individually addressable LEDs, LED strips, fog and smoke machines, and the like. While the description refers to LEDs and other lighting devices as examples, it should be understood that other types of devices, such as any IoT (e.g., network addressable) device, can be controlled as well, so long as one of the special effects devices is individually network addressable, either directly or indirectly, and the device is programmed to speak a protocol understood by the IoT device. For example, devices such as lightwave (e.g., therapeutic) devices, wearable device outputs (e.g., haptic feedback, LEDs, audio), and the like, can also be controlled via a remote special effects system using the techniques described herein, regardless of what type of network they are on (e.g., wireless conventional or mesh or wired).

[0008] An exemplary remote special effects system described herein includes one or more special effects controller devices (SE controllers), each of which plugs into a standard (residential or business) electrical outlet and connects to one or more special effects devices (SE devices), such as LED strips, fog machines, horns, wavelength output devices, and the like. Each SE controller is wirelessly connected, directly or indirectly (e.g., via another SE controller), to a network, such as the Internet (or other local area, wide area, or other network), and can respond to DMX (or other protocol) commands sent from an application remotely connected to the SE controller by issuing corresponding commands specific to the connected SE device to generate special effects. Thus, existing AV tools that AV designers use to issue DMX commands can be easily integrated into the RSES to control special effects used by the general public (home, or business, or any use), and no SE controller expertise is required to generate these special effects. For example, if the special effects device is a Neopixel LED strip manufactured by Adafruit Industries (which uses a single wire to control RGB or RGBW LEDs), the SE controller can receive packets containing DMX commands (e.g., using Art-net protocol packets, whether from an existing AV tool or a new application) and then automatically issue corresponding driver instructions (e.g., WS28xx, WS2812, or other driver instructions) to address one or more of the Neopixel LEDs.

[0009] In an exemplary embodiment, the SE controller is a printed circuit board ("PCB") that can be housed in an available enclosure of a size similar to that of an AC adapter. Because these SE controllers are small devices that can utilize a standard electrical outlet as a power source, they can be manufactured inexpensively and can produce theater-quality lighting and other special effects for the home and masses. Moreover, because they are wirelessly connected to a network such as the Internet, they can be remotely controlled by an application, just like most IoT devices (they can act and be controlled as IoT devices). In addition, because they are wirelessly connected, the special effects devices can be located in places where there is no Ethernet, including places where they are controlled by different types of wireless networks, such as traditional (centralized) networks and distributed (mesh) networks. Similarly, the special effects devices being controlled do not need to be DMX-enabled or connected with cumbersome Ethernet cables in a semi-permanent (fixed-time) configuration, as is commonly found in theaters or other professional settings.

[0010] In other exemplary embodiments, the PCB (SE controller) may be powered by a battery contained within, adjacent to, or proximate to the housing. This allows some exemplary SE controllers to be incorporated into devices that communicate wirelessly with remote applications that control, for example, special effects devices managed by the PCB controller (rather than "plugging" into a wall outlet). The SE device may reside on the same device that houses the PCB controller.

[0011] By using a wireless solution rather than a wired or Bluetooth solution, the need to control devices together within a certain range, for example supported by an Ethernet cable or Bluetooth distance requirement, is eliminated. Rather, in some scenarios, SE controllers can join together in an "ad-hoc" (on-the-fly or if available) manner and synchronize (by a separate program) to collaboratively create special effects at discrete and disjointed (not connected) physical locations, at the same physical location, at virtual locations, and / or across one or more devices. For example, in a virtual classroom where each student connects at least one SE controller to a special effects device (such as a light strip or sound generating device), synchronized special effects can be triggered by a teacher via software programmed to communicate with participating SE controllers, for example to ensure that students are awake and paying attention. As another example, a virtual presentation (e.g., delivered via video conference) with slides can be programmed (pre-programmed or controlled on-the-fly) to generate special effects as personalized or synchronized animations at each physical or virtual location (e.g., room, office, home, etc.) where each viewer is participating. Special effects can even be targeted to individual viewers, or used to group participants in crowd situations by generating unique special effects for targeted participants for each intended group. Additionally, in some scenarios participants can collectively control the special effects.

[0012] In other examples, sensor inputs (such as a weather station) can be obtained from an application to generate special effects in response to detecting certain conditions. For example, sensors connected to the human body measuring attributes such as heart rate, sweat, blood flow, and the like can be connected to an application that communicates with an SE controller connected to different special effect (output) devices using an SE controller plug-in. Other types of sensor inputs, such as inputs from wavelength devices, magnetic, electrical, optical, and other devices, can be accommodated as well. Thus, in some scenarios, an SE device can be controlled and operated without direct human interaction, but rather by sensing conditions from sensors that measure some characteristic of someone or the environment.

[0013] In yet other scenarios, different special effects devices may be synchronized and coordinated as may be useful in education or presentations. For example, an audio special effects device may be synchronized to a lighting or haptic feedback special effects device as desired by an application controlling an SE controller attached to these devices. In another example, a proximity sensor may be placed on a device or person or place that triggers an input to an SE controller enabled application. Upon sensing that two people / two devices are in close proximity to each other (or in close proximity to some place), the application sends a command to one or more SE controllers, which then control one or more (output) SE devices to generate synchronized or coordinated special effects.

[0014] Thus, there are many possible work-related, home-related, and / or entertainment-related uses for such SE controllers and associated software.

[0015] In addition to these advantages over existing systems, the SE Controller wirelessly connects to the Internet (or other local or wide area network) either directly or indirectly (e.g., via another SE Controller) and accepts any type of live DMX packet stream over the wireless network (e.g., using Art-Net or other protocols), thus presenting other advantages. For example, animations are not limited to pre-programmed lighting displays, but can provide a new and different experience each time an event is run. In addition, animations, music, or sound can be sourced live from anywhere in the world, as long as the input source is Internet accessible. Furthermore, the SE Controller can execute any of approximately 2.8 million color combinations, limited only by the intelligent lighting devices communicatively connected to the SE Controller.

[0016] Each SE controller is wirelessly connected to the Internet (or other wide area or local network) so that it can respond to DMX (or other protocol) commands sent by an application remotely connected to the SE controller by issuing corresponding commands specific to the connected SE devices to generate special effects. The wireless connection can be a traditional (centralized) wireless network or a distributed (e.g., mesh) wireless network. In addition, while the exemplary embodiments of the RSES are presented for controlling SE devices that use the DMX protocol (by wrapping and forwarding them in Art-net packages), it should be understood that the system architecture and ideas can be used to extend control to other types of devices controlled by other than DMX protocols. Similarly, other packet wrapper protocols other than Art-net can be incorporated into the RSES and can provide the functional advantages described herein.

[0017] FIG. 1 is an exemplary remote special effects system that uses multiple special effects controller devices to remotely control special effects devices, e.g., in a live fashion for discrete locations. As shown in FIG. 1, RSES 100 includes one or more special effects controller devices 120-123 (SE controllers) that are wirelessly connected to a wide area network 110, typically the Internet, and communicatively connected to one or more respective special effects devices 106-109. Each SE controller 120-123 is also connected to a corresponding electrical outlet 105a-105d as a power source. The SE controllers may be located in locations geographically separate from one another (located in different physical residences, businesses, regions, or countries, e.g., having different postal addresses, etc.). For example, SE controller 120 connected to LED strip 106 may be installed at student A's residence, while SE controller 121 connected to LED strip 107 may be installed at student B's residence. Additionally, other SE controllers, such as SE controller 122, may be installed in different states, or potentially different countries, buildings or other structures, or locations not associated with buildings, provided WiFi is available. Additionally, the SE controllers may share the same IP address or may have different IP addresses. The SE controllers 120-123 may be connected to a variety of special effects devices including, for example, individually addressable (RGB) LED strips 106 and 107, Leko lights 108, or a standard (dumb) RGB LED strip 109. Other devices such as fog and smoke machines, audio output devices, haptic feedback devices, other mechanical, electrical, and / or lightwave devices, and a myriad of sensors and devices may be connected as well but are not shown.

[0018] The SE controllers 120-123 wirelessly connect to a special effects analyzer and control program (SE analyzer) 101 or other special effects-enabled programs 130 via the network 110. These programs may run on any type of computing device. In the exemplary embodiment described below, the SE analyzer 101, known as a Remote Live Scene Control or "RLSC" application, is configured to analyze sound and send control data (such as lighting control commands) to each of the SE controllers 120-123 individually, and can synchronize the behavior of these SE controllers 120-123 so that all or some of the connected special effects devices 106-109 (e.g., lights) react / behave in the same or different ways at the same or different times. In this way, the SE controllers 120-123 can cause special effects that are synchronized or otherwise timed to cause similar actions in the audience, even though the audience members are not present in the same physical venue and therefore the SE controllers are located remotely from one another. Of course, the special effects can also be synchronized to one or more special effects devices in the same physical venue. Here, venue refers to a physical address associated with a building, residence, event, open space, field, etc. Thus, the remote special effects system can be used to engage audiences (through special effects) connected through a virtual venue, such as a video conference over web-based video conferencing software, a presentation to a virtual audience connected to the venue through their computers, etc. Additionally, the RSES can be used in a standalone environment to create theater and event quality special effects at home or corporate locations.

[0019] Similarly, SE controllers 120-123 may respond to commands from a single speaker (such as a teacher or speaker), several speakers (e.g., a band), and / or one or more participants, such as an interactive classroom, a family reunion, or a crowd. In addition, any of the effects streamed to SE controllers 120-123 may be pre-designed and thus "played back," or may be generated on the fly. All such combinations are contemplated.

[0020] In one exemplary embodiment, the SE analyzer 101 is used to analyze sounds, such as songs, soundtracks, from a movie, presentation, or any audio track, and use the LED strip to, for example, change color sequentially or randomly to trigger special lighting effects or generate some other type of lighting animation in conjunction with attributes of the sound, such as tone / pitch (e.g., measured as frequency) and volume (e.g., measured as decibels). For example, based on detection of a particular frequency, a corresponding color command can be sent to an individually addressable LED, a group of LEDs, or the entire strip. In the exemplary system shown in FIG. 1, the SE analyzer 101 is specifically programmed to generate commands via Art-Net (DMX over Ethernet), which are sent to the various, independently addressable SE controllers 120-123. Other commands according to other protocols (such as sACN, a streaming protocol using multicast communication technology) can be incorporated as well. When each SE controller 120-123 wirelessly receives an Art-Net (or other protocol) packet, it converts the packet into the appropriate device protocol understood by the special effects device that is communicatively coupled (possibly electronically) to the SE controller. For example, in the case of an RGB pixel-addressable LED strip 109 (such as a Neopixel LED device by Adafruit Industries), the SE controller 123 receives the Art-Net packet and converts the DMX lighting commands into a protocol understood by the ws2812 driver used to control the Neopixel device 109.

[0021] Other third party programs and applications, such as the third party program 130, can use the RSES application programming interface (API) and libraries to generate Art-Net (or other protocol) packets to control the RSES SE controllers 120-123. Thus, mobile or other computer-based applications can be used to create easily accessible special effects, such as playing music with live special lighting effects that are dynamically (on the fly) created in response to an analysis of the audio. Additionally, the same audio may be analyzed differently each time the music is played, allowing for a different special effect experience each time. Alternatively, pre-designed lighting effects, or lighting effects designed by an A / V artist, may be played along with the music as it is played. In some scenarios, the SE analyzer and / or the third party program 130 may incorporate data or stored configurations from the data repository 102.

[0022] As mentioned, the SE controllers 120-123 are location independent and can participate in a special effects "session" on an "ad-hoc" basis. For the purposes of this description, a session is defined as an SE controller that is accessible and addressable to the SE analyzer 101 or special effects enabled application 130 at a particular point in time. The SE controllers 120-123 can be integrated by a special effects service (SE service) 103, e.g., a web service in a more centralized computer architecture scheme, which controls the concept of a session, which are the currently addressable SE controllers that the SE analyzer can control. The SE service maintains a notion of the current "session" of the SE analyzer 101, as SE controllers can be connected and disconnected on the fly. Alternatively, the SE analyzer 101 or any special effects enabled application, such as a third party program 130, can integrate its own (wireless) discovery and registration process, such as using a combination of login functionality, scanning and handshaking protocols, etc. Any type of discovery and registration process can be used by the SE service 103 or by these programs to discover available SE controllers and / or register them as part of the current session. Additionally, a special effects enabled program (such as the SE analyzer 101 or program 130) can present a user interface for configuring the user, the SE controller, the speaker, etc. Thus, the SE analyzer 101 or the special effects enabled program 130 can not only dynamically generate special effects (e.g., while analyzing an audio stream), but can also be "ad-hoc" and control any of the SE controllers 101 currently connected to an applicable session, which is managed by a discovery and registration process. As an alternative to the use of web services, the RSES configuration can be adapted into a peer-to-peer system, where one of the SE controllers 120-123 becomes the server (or "master") and acts to control the session aspects.In addition, the SE controllers that make up the RSES configuration can operate as a mesh network, where each SE controller acts as a node, using protocols and APIs such as, for example, ESP-WIFI-MESH, etc. Other suitable configurations are envisioned, such as those used strictly in intranet scenarios.

[0023] FIG. 2 is a schematic flow diagram illustrating how an exemplary remote special effects system operates according to an exemplary configuration as shown in FIG. 1 to generate synchronized special effects for an ad-hoc "session" (which may also form an ad-hoc network or sub-network) of special effects devices. Here, "synchronized" refers to timed special effects streams (coordinated according to a particular time, order or sequence, relative to each other, and other variations) that are forwarded to one or more SE controllers for coordinating special effects at potentially different physical (mail address or other latitude / longitude designation) locations. The SE controllers may also be in the same physical location, at the same IP address, or at different IP addresses, and any other such combination. Ad-hoc refers to the notion that "zones" of special effects devices are created (through their communicatively connected SE controllers) and made known to the SE analyzer 101 or third party program 130, and special effects instructions are forwarded (e.g., sent, forwarded, multicast, broadcast, etc., based on the application and protocol being used). In block 201, an SE-enabled application such as the SE analyzer 101 or third-party program 130 executes an application in which a special effect is selected (e.g., color scrolling, or a "plasma" effect, or a selected color), programmatically or potentially by a user using a user interface or other tool. In response, the SE-enabled application sends a DMX packet over WiFi (block 202) via Art-Net (or other protocol) to a network address (e.g., IP address, MAC address, etc.) associated with an SE controller registered in the current session (e.g., by the SE service 103, or by an SE-enabled application with such functionality). In blocks 203 and 204, each SE controller that is part of this ad-hoc session receives the packet and converts it into data that the communicatively connected SE device (e.g., LED strip 106) understands.The corresponding SE device (blocks 205 and 206) then receives this packet (typically via a software or firmware driver associated with the respective device) and executes the special effect (e.g., lighting) command issued by the corresponding SE controller. Other types of SE devices and other types of special effects can be handled similarly.

[0024] 3 is a flow diagram of a typical SE-enabled application with an exemplary remote special effects system. The depicted SE-enabled application 300 is, for example, a sound analyzer effect control program 101 (hereinafter referred to as a remote live scene control application) or a third-party RLSC-enabled application such as SE-enabled application 130. Here, in logic block 301, the application discovers or otherwise obtains (e.g., via web services 103) a list of SE controllers currently connected to send special effects. This list forms an "ad-hoc" network of controllers (e.g., a synchronized special effects zone) for synchronized special effects. In block 302, the application receives an indication of the type and attributes of the special effect from a user (or a program such as a slide presentation program). Other characteristics may also be included, such as timing details, target SE controller, and the like. An exemplary application programming interface (API) is described below for use by the SE-enabled application. In block 303, the application generates an Art-Net (or other protocol) packet (e.g., using DMX or other protocol) containing the computer-understandable special effect. In block 304, this special effects packet is transmitted to some or all of the SE controllers in the ad-hoc network.

[0025] These packets are then received and processed by the SE controller as described with reference to FIGS.

[0026] 4A-4E are exemplary special effects that can be produced by an exemplary remote special effects system as described herein, where each figure is a snapshot of a time series of changing color animations generated by an SE controller using LEDs placed around a person's computer screen. Using the RSES, in an environment with multiple participants, each having different physical locations with similarly positioned computer screens, participants can have the same special effect generated simultaneously or in time sequence or similarly around their computer screens (e.g., creating a "wave" effect), etc., by using an application to trigger this special effect across ad-hoc connected SE controllers. Or, the application can select and generate different special effects for one or more of the participants and adjust them in any way (e.g., to focus on a particular participant or other speaker). For this reason, an exemplary implementation of the SE Analyzer is described below as "remote live scene control."

[0027] 5A-5C are additional exemplary special effects that can be produced by an exemplary remote special effects system as described herein, where each figure is a snapshot in time of color rolling through LED strip lights attached to an SE controller. This special effect can be coordinated across multiple virtually connected participants as described in accordance with FIGS. 4A-4E.

[0028] Of note, any of these special effects, applications, SE devices, and SE controllers can also operate within a single physical address, for example, one or more SE controllers in a single (physical or virtual) room, multiple SE controllers connected to different devices in a single home or business, and any other combination. In addition, as new special effect devices that understand protocols such as DMX or other protocols become available, the SE controller can be adapted (e.g., through a firmware update) to translate DMX commands into a language (commands) that the device's driver understands. Thus, the SE controller is not limited to only receiving DMX packets and generating any particular lighting commands (such as ws2812). Thus, as new special effects are created, for example using sensors or other devices, the SE controller can adapt them to generate other or improved special effects.

[0029] Similarly, the current embodiment of the SE Analyzer (RLSC Analyzer) generates and transmits DMX packets using the Art-Net protocol (IP addressable DMX). This Sound Analyzer and other applications can incorporate other protocols during development and as the SE Controller firmware is updated. Thus, the RSES described herein can be extended to incorporate different and other protocols and other devices.

[0030] The exemplary embodiments described herein provide applications, tools, data structures, and other support for implementing a remote special effects system used for live or pre-programmed special effects that are wirelessly delivered in a synchronized manner to specific devices in separate physical locations. Other embodiments of the described technology may be used for other purposes. In this description, numerous specific details are set forth, such as data formats and code sequences, to provide a thorough understanding of the described technology. The described embodiments may also be practiced without some of the specific details described herein, or with other specific details, such as changes with respect to logical order, different logic, and the like. Thus, the scope of the described technology and / or functionality is not limited by the particular order, selection, or decomposition of aspects described with reference to any particular routines, modules, components, and the like.

[0031] Exemplary RSES Implementation An exemplary RSES embodiment having components as described in accordance with Figures 1-3 is directed to generating special effects using lighting and DMX protocols (via the Art-Net protocol). This exemplary embodiment includes one or more SE controllers referred to as Emanator devices. These are devices that plug into a standard electrical outlet and control lighting devices such as LED strips. In operation, participants participating in a session managed by the RLSC application plug their special effects devices (such as intelligent lights or fixtures) into the Emanator device, which in turn plugs into an electrical wall outlet.

[0032] The exemplary RSES embodiment also includes an application, a remote live scene control application (or RLSC application), configured to generate special effects as described in accordance with Figures 2 and 3. The exemplary RLSC generates Art-Net packets using DMX commands, however, may be modified as described above. Additionally, the exemplary RSES embodiment includes an API that can be used to develop other special effects-enabled applications.

[0033] Of course, other example components and embodiments can be developed to generate synchronized lighting effects and other special effects as described herein.

[0034] Remote Live Scene Control Application: An exemplary RLSC application, which may be provided via a mobile, IoT, or wired computing system, is designed to configure and organize one or more Emanator devices and stream effects to each Emanator device or tie all of them into one or more synchronized special effects zones. The application includes user and device discovery features for easy setup, making it easy to organize and tie devices into zones for further custom control. The app offers a variety of pre-programmed effects, including those that react to live audio and generate lighting (or other scene) effects synchronized to the audio. RLSC users can also configure control of various participant special effects devices over the Internet for applications such as live virtual meetings or concerts. In some embodiments, the RLSC application can operate to generate pre-programmed, pre-designed, or pre-recorded effects as well as on-the-fly (dynamic) special effects. In some scenarios, these pre-programmed / designed / recorded special effects can also be obtained from AV designers who created them for theater or film using industry-standard DMX consoles or equivalent.

[0035] In one embodiment, the RLSC application includes an RLSC music analyzer (or with a separate RLSC Music music analyzer) that analyzes sounds based on frequency (or pitch) and volume, as described with reference to Figure 1. The RLSC music analyzer is an example of a third-party application that can be written using the Emanator device API to support special effects. Using the Emanator API, third-party applications can stream and send data in real time to one or more Emanator devices.

[0036] As will be described, applications such as the RLSC application can be used to generate and control special effects for a zone of synchronized special effects across multiple Emanator devices (which can reduce the number of special effects devices connected to one plugged-in Emanator to one). This functionality is referred to as "Remote Live Scene Control." For example, the RLSC application can: ● Run applications with remote live scene control. ● Play movies with remote live scene control. ● Play music with remote live scene control. ● Conduct virtual meetings using remote live scene control; or ● Facilitate group gatherings with remote live scene control.

[0037] Many implementations include the concept of no one missing the event because they cannot physically attend, which is referred to as “virtual live immersion.” The following scenarios are just a few of these possible examples in addition to those summarized above. ● Artist-created lighting effects with music: o Play music with live analyzed lighting effects. Different analyzed lighting effects can provide a different experience every time the music is played. Or 〇 Play music with pre-designed live lighting effects, for example lighting effects designed by A / V artists (MP3 and DMX lighting effects combined) ● Virtual meetings include “home” lighting that is synchronized with presentations and slideshows triggering pre-programmed effects. 〇 Capture your virtual audience's attention with lightning and thunder effects happening in the room, not just on the screen. 〇 With sound, pre-recorded voice or video, live analyzed effects. Overlapping playback may vary with sound analyzer or pre-designed lighting effects. Other hypothetical examples include: 〇 Pending a service (e.g. medical care) 〇 Soothe online patients with music collection / lighting displays to reduce stress scenes. Virtual school with teacher presentations synchronously in students' rooms at home, in the dormitory or anywhere, e.g. ● Synchronize participant lighting with the push of a button to wake up students who fall asleep during class by synchronizing lighting for all participants. Image effects on all participant screens allow all virtual participant screens to be synchronized with the lighting in the room. ● Shine a flashlight on one participant to draw attention or focus on them. Virtual family reunions ● Play family games (trivia, Jeopardy) and analyze the sound, music and noise of the live players (microphones) live and the lighting in the entire family’s home responds accordingly. Virtual church or school choir ● The virtual choir is analysed and live lighting responds across all device screens and in the home. ● Choir with live pre-designed lighting effects. For example, lighting effects designed by A / V artists. (Combined MP3 and DMX lighting effects) Virtual DJ ● Virtual dance music and sounds are analysed and responded to with live lighting on all screens and in every home or location. ● Virtual weddings with pre-programmed lighting effects at each location. No one will miss the event, they will be immersed in it virtually. Every participant can influence and create the event. Virtual Theater ● Virtual theatre where professional dramatic stage effects occur in your home, rented space or other location. ● All virtual participants can influence and shape the event. Virtual theater with pre-designed lighting effects, for example lighting effects designed by A / V artists (MP3 and DMX lighting effects combined) ● Movies come to life at home Experience home lighting or devices that react to the crackles or thunder of a Harry Potter movie or the bright lights of lightning-fast movement in Star Wars. Bringing theatrical special effects to the masses. o Download movies with pre-designed lighting effects, e.g. lighting effects designed by A / V artists (combining video file formats and DMX lighting effects) ● Virtual Concert 〇 Attend a concert and have stage effects happen anywhere, at your home, backyard, etc. Music is analyzed on the fly or pre-designed lighting effects, for example lighting effects designed by A / V artists. ● Games 〇 Not only is there a stagnant LED behind the monitor, but there is also a light that reacts to game events. 〇 Lighting effects based on game music analysis. Pre-designed lighting effects, for example, lighting effects designed by A / V artists and integrated into the game ● Presentation o Video or music included in the presentation synchronized with the lighting in the room Analyzed effects on music or noise from the speaker (useful pauses) or the audience (applause) Pre-designed effects to wake up the audience The sensor is connected ● Use of significant activated lighting 〇 For example, running / exercise ● RSES technology can be used to create a cost-effective personal pace trainer for runners, or an inexpensive alternative for schools or budget training practices. ● Portable waterproof LED pacing machine effect ● Replace existing expensive, lightweight training products with more cost-effective solutions (costing tens of thousands of dollars in equipment and installation costs) to improve runners' pace. For example, patches of lights could be set up around a field or track, and a smartphone or computer could sync up the pace lights, illuminated patches of LEDs, and runners should try to run as close to the lights as possible. ● Grid LED Times Square-like effects for advertising and other purposes

[0038] Emanator (special effects controller): An exemplary RSES embodiment provides an Emanator device as the SE controller. Figures 6A-6C provide three different pictures of a printed circuit board implementing an Emanator device, according to an example described. The device includes a WiFi chip, microprocessor, fan, AP (Access Point) mode header, and various buttons and other components, including firmware. Figures 7A-7C provide component layout information for the Emanator device. Other embodiments powered from nearby battery power may be accommodated as well.

[0039] To summarize operation, 12V of power enters the PCB and goes through some power conditioning to prevent spikes and to extend the life of the LEDs. In the current configuration, the board contains a Wemos D1 Mini that includes an ESP8266 WiFi chip for wireless connectivity. In other configurations (not shown), the board is configured to include an ESP32 chip that is more powerful, dual-core, and provides other features such as Bluetooth for configuration. Other chips can be incorporated as well. The PCB receives Art-Net data packets via WiFi, and firmware on the ESP8266 chip processes it and outputs data that the LEDs can understand (e.g. WS28xx driver commands for Neopixel LEDs). There are terminal block headers that deliver power and data to the LEDs. Additionally, the board includes fan, reset, and AP mode headers. The fan header provides configurable 5V or 12V for an optional cooling fan. The reset header provides a connection to an external reset button that resets the ESP8266 chip. The AP mode button provides a connection to an external button that can force the board into configuration mode to reconfigure the wireless connection.

[0040] The Emanator firmware on the PCB has an Art-Net receiving library (e.g. from public domain software) that takes the Art-Net data broadcast over WiFi and provides raw DMX that is sent to the lights or fixtures to display the requested color / state.

[0041] In the exemplary Emanator device connected to NeoPixel LEDs, the Neopixel library present on the PCB (e.g. from public domain software) then generates ws28xx compatible data (the DMX data sends digital signals on the output pins (0 to N# lights from one pin) in the order that the requested lights should be turned on, which indicates one of millions of RGB values).

[0042] When the PCB receives a Wi-Fi transmission, the firmware processes the packet using the Art-Net receiving library and sends the signal to the light, for example using the Neopixel library. In this way, the Emanator PCB acts as a converter from DMX commands to lighting device compatible commands. Thus, the Emanator firmware turns an LED device (e.g. Neopixel) into a network addressable DMX device. Other conversions can be programmed into the Emanator PCB.

[0043] In the current implementation, the PCB firmware is configurable using a web-based user interface. Config settings include the number of LEDs connected to the board, the pins connected to the LEDs, the two status indicators that can be turned off, the WIFI network it is connected to, the IP address, Art-Net universe information, and the like. The RLSC application or other SE-enabled applications can be programmed to configure the Emanator device.

[0044] FIG. 8 is a wiring schematic of an example printed circuit board for implementing a special effects controller. In summary, power comes in and goes through a capacitor that cleans the power. It then goes through a fuse to prevent it from drawing too much power or damaging the controller. 12V of power is then provided to the LEDs. In one embodiment, the PCB has a fan header to power a 5V or 12V fan, and a reset button header. It also has an AP mode header to put the PCB into access point mode and reconnect the WiFi. The PCB also includes a 12V to 5V step-down to step down power for the WEMOS board with the ESP8266 chip. Other microcontrollers and other WiFi chips can be supported as well. A logic shifter is used to shift the 3.3V coming out of the WEMOS to the 12V required by the LED strip. The PCB includes an LED indicator stack light with two resistors that acts as a status indication for the board. There is also another resistor to clean up the data signal from the WEMOS to the LEDs. The 24×n-bit data signal to the lights (where "n" is the number of lights in the LED strip, and 24 bits provides 24-bit color for a single LED) is run through resistors to purify the signal and reduce noise.

[0045] The majority of the PCB is the power supply for the light and an electrical safety feature. Specifically, the PCB prevents the light from drawing too much power from the power supply, thereby reducing burning out the LEDs and overheating the power supply, which could potentially cause a fire. The PCB contains capacitors to prevent power spikes to the LEDs and fuses to prevent burning out the power supply.

[0046] The PCB can be programmed to control any IoT (Internet of Things) device, any network-addressable device, including another computing device.

[0047] RSES Application Programming Interface (API): An exemplary RSES embodiment also includes an application programming interface, defining functions to facilitate the development and publishing of special effects-enabled applications.

[0048] Table 1 below includes an exemplary set of interfaces. It will be understood that variations are possible and can be incorporated. Additionally, features can be added, modified, or removed. [Table 1] TIFF2024536842000003.tif71169

[0049] Exemplary Computing System 9 is an example block diagram of an example computing system that may be used to practice embodiments of components of the remote special effects system described herein, such as the special effects analyzer and controller (SE analyzer 101 of FIG. 1), or other third-party special effects-enabled applications (applications 130 of FIG. 1). It should be noted that one or more suitably designated general-purpose virtual or physical computing systems, or special-purpose computing systems, may be used to implement the RSES. Furthermore, the components of the RSES may be implemented in software, hardware, firmware, or some combination to achieve the functions described herein.

[0050] It should be noted that one or more general-purpose or special-purpose computing systems / devices may be used to implement the described techniques. However, just because an SE analyzer can be implemented on a general-purpose computing system does not imply that the techniques themselves or the operations required to implement them are conventional or well known.

[0051] The computing system 900 may include one or more server and / or client computing systems and may span distributed locations. In addition, each block shown may represent one or more such blocks as appropriate to a particular embodiment or may be combined with other blocks. Moreover, the various blocks of the SE analyzer / RLSC application 910 may physically reside on one or more machines that communicate with each other using standard (e.g., TCP / IP, UDP / IP) or proprietary inter-process communication mechanisms.

[0052] In the illustrated embodiment, computer system 900 includes computer memory ("memory") 901, a display 902, one or more central processing units ("CPUs") 903, input / output devices 904 (e.g., keyboard, mouse, CRT or LCD display, etc.), other computer readable media 905, and one or more network connections 906. An SE analyzer / RLSC application 910 is shown residing in memory 901. In other embodiments, some of the content, some or all of the components of the SE analyzer / RLSC application 910 may be stored on and / or transmitted via other computer readable media 905. The components of the SE analyzer / RLSC application 910 preferably execute on one or more CPUs 903 and manage the generation of special effects within a synchronization zone across a wireless network, as described herein. Other code or programs 930, RSES web services 940, RSES API definitions and libraries 917, and potentially other data repositories such as data repository 920 are also resident in memory 901 and preferably execute on one or more CPUs 903 as needed. Of note, one or more of the components of Figure 9 may not be present in any particular implementation. For example, some embodiments embedded in other software may not provide a means for user input or display.

[0053] In a typical embodiment, the SE analyzer / RLSC application 910 includes other components such as one or more sound analysis engines 911, one or more DMX or other protocol generation engines (or libraries) 912, a data repository 915 of special effects information, scripts, configuration parameters, etc., and a different special effects engine 913. In at least some embodiments, the other special effects engine 913 is provided external to the SE analyzer / RLSC application and potentially available over one or more networks 950. Other and / or different modules may be implemented. In addition, the SE analyzer / RLSC application 910 may communicate over the network 950 with other special effects enabled applications or client code 955, one or more SE controllers (Emanators) 960, and / or one or more third party information provider systems 965, such as pre-designed AV effects. Also of note, the 915 data repository may be provided external to the SE analyzer / RLSC application, such as in a knowledge base accessible over one or more networks 950.

[0054] In an exemplary embodiment, the components / modules of the SE analyzer / RLSC application 910 are implemented using standard programming techniques. For example, the SE analyzer / RLSC application 910 may be implemented as a "native" executable running on the CPU 103 along with one or more static or dynamic libraries. In other embodiments, the SE analyzer / RLSC application 910 may be implemented as instructions processed by a virtual machine. A range of programming languages ​​known in the art may be employed to implement such exemplary embodiments, including representative implementations of various programming language paradigms, including, but not limited to, object-oriented, functional, procedural, scripting, and declarative.

[0055] The embodiments described above may also use known or proprietary synchronous or asynchronous client-server computing techniques. Also, the various components may be implemented using more monolithic programming techniques, for example as executables running on a single CPU computer system, or alternatively may be decomposed using various structuring techniques known in the art, including but not limited to multiprogramming, multithreading, client-server, or peer-to-peer, to run on one or more computer systems, each having one or more CPUs. Some embodiments may run simultaneously and asynchronously, communicating using message passing techniques. Equivalent synchronous embodiments are also supported.

[0056] Additionally, programming interfaces to the stored data as part of the SE analyzer / RLSC application 910 (e.g., in data 915) may be available through standard mechanisms such as APIs for C, C++, C#, and Java, libraries for accessing files, databases, or other data repositories, scripting languages ​​such as XML, or other types of servers that provide access to the stored data. Data repository 915 may be implemented as one or more database systems, file systems, or any other technology for storing such information, or any combination of the above, including implementations using distributed computing technologies.

[0057] Also, the exemplary SE analyzer / RLSC application 910 may be implemented in a distributed environment including multiple, even heterogeneous, computer systems and networks. Different configurations and locations of programs and data are contemplated for use with the techniques described herein. In addition, the [server and / or client] may be physical or virtual computing systems and may reside on the same physical system. Also, one or more of the modules may be distributed, pooled, or otherwise grouped, such as for load balancing, reliability, or security reasons. Various distributed computing technologies are suitable for implementing the components of the illustrated embodiment in a distributed manner, including, but not limited to, TCP / IP sockets, RPC, RMI, HTTP, Web Services (XML-RPC, JAX-RPC, SOAP, etc.), and the like. Other variations are possible. Also, other functionality may be provided by each component / module, or existing functionality may be distributed among the components / modules in different ways, while still achieving the functionality of the SE analyzer / RLSC application.

[0058] Further, in some embodiments, some or all of the components of the SE analyzer / RLSC application 910 may be implemented or provided in other ways, for example, at least in part, in firmware and / or hardware, including, but not limited to, one or more application specific integrated circuits (ASICs), standard integrated circuits, controllers executing appropriate instructions, and microcontrollers and / or embedded controllers, field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and the like. Some or all of the system components and / or data structures may also be stored as content (e.g., as executable or other machine-readable software instructions or structured data) on a computer-readable medium (e.g., a hard disk, memory, network, other computer-readable medium, or other portable media item that is read by a suitable drive, such as a DVD or flash memory device, or via a suitable connection), enabling the computer-readable medium to execute or otherwise use or provide the content to perform at least some of the described techniques. Some or all of the components and / or data structures may be stored in a tangible, non-transitory storage medium. Some or all of the system components and data structures may also be stored on various computer-readable transmission media as data signals (e.g., by being encoded as part of a carrier wave or by being included as part of an analog or digital propagated signal), which may then be transmitted over wireless-based and wire / cable-based media, and may take a variety of forms (e.g., as part of a single or multiplexed analog signal, or as multiple separate digital packets or frames). Such computer program products may also take other forms in other embodiments. Thus, embodiments of the present disclosure may be practiced with other computer system configurations.

[0059] Example Special Effects Controller 1. An apparatus for controlling a special effects device, comprising: A printed circuit board (PCB), comprising: a wireless network microcontroller; a PCB header configurable to set a network address associated with the device; a terminal connector configured to provide power to the PCB; a data connector configured to be communicatively connected to one or more cables of a special effects device; a printed circuit board (PCB) including a microprocessor containing firmware; a housing configured to house the printed circuit board and enable the PCB to plug into a standard electrical outlet to receive power and to communicatively connect to a special effects device; 16. An apparatus, wherein firmware is configured to wirelessly receive, using a wireless network controller, a stream of data packets, each packet in the stream including data structured according to a special effects device protocol, and wherein, for each packet, the firmware generates one or more corresponding electrical, mechanical, and / or optical signals in response to special effects device protocol instructions included in the packet and transmits them to a communicatively coupled special effects device to cause the special effects device to create a special effect. 2. The apparatus of claim 1, wherein the special effects device protocol is DMX. 3. The apparatus of claim 1, wherein the special effects device is an LED device. 4. The apparatus of claim 3, wherein the special effects device is an LED strip and the special effect is a color change. 5. The apparatus of claim 3, wherein the special effects device is a Neopixel LED device and the corresponding signal is formatted according to a corresponding driver specification. 6. The device of claim 3, wherein the special effect produces a color change, a color gradient, a timed color progression, or a pulsation of a single color or different colors. 7. The apparatus of claim 1, wherein the special effects device is a fog machine or a smoke machine. 8. The apparatus of claim 1, wherein the apparatus generates a special effect in response to receiving a value sensed by the sensor. 9. The apparatus of claim 1, wherein the stream of data packets is organized according to the Art-Net protocol. 10. The apparatus of claim 1, wherein the special effects device is a light wave device, a magnetic device, and / or an electronically controlled device.

[0060] conclusion All of the above U.S. patents, U.S. patent application publications, U.S. patent applications, foreign patents, foreign patent applications, and non-patent literature referenced herein and / or listed in the Application Data Sheet, including, but not limited to, U.S. Provisional Patent Application No. 63 / 248,991, filed September 27, 2021, entitled "REMOTE LIVE SCENE CONTROL SYSTEM, METHODS, AND TECHNIQUES," are hereby incorporated by reference in their entirety.

[0061] From the foregoing, it will be appreciated that, although specific embodiments have been described herein for purposes of illustration, various modifications may be made without departing from the spirit and scope of the present invention. For example, the methods and systems for performing special effects discussed herein are applicable to other special effects devices and for purposes other than those described herein. Also, the methods and systems discussed herein are applicable to different protocols, communication media (optical, wireless, cable, etc.) and devices (wireless handsets, electronic organizers, personal digital assistants, portable email machines, gaming consoles, pagers, navigation devices such as GPS receivers, etc.).

Claims

1. 1. An apparatus for controlling a special effects device, comprising: a wireless network controller; a data connector configured to be communicatively connected to one or more special effects devices; a microprocessor containing firmware; a housing configured to house the data connector and the microprocessor and to be communicatively coupled to the special effects device; the firmware is configured to wirelessly receive, using the wireless network controller, a stream of data packets, each packet of the stream including data structured according to a special effects device protocol, and for each packet, in response to special effects device protocol instructions included in the packet, the firmware generates and transmits one or more corresponding electromagnetic, mechanical, and / or optical signals to one or more of the one or more communicatively connected special effects devices to cause the special effects devices to produce a special effect.

2. The device of claim 1 , wherein the housing is adapted to receive power by being plugged into a standard electrical outlet.

3. The device of claim 1 , wherein the device receives power from a battery device.

4. The apparatus of claim 1 , wherein the special effects device protocol is DMX.

5. 10. The apparatus of claim 1, wherein the special effects device is an LED device and the special effect is color.

6. The device of claim 1 , wherein the special effect causes a color change, a color gradient, a timed color progression, or a pulsation of a single color or different colors.

7. The apparatus of claim 1 , wherein the special effects device is a fog machine or a smoke machine.

8. The device of claim 1 , wherein the device generates the special effect in response to receiving a value sensed by a sensor.

9. The apparatus of claim 1 , wherein the special effects device is a light wave device, a magnetic device, and / or an electronically controlled device.

10. 1. A method for controlling special effects, comprising: wirelessly communicating under control of a plurality of special effects software or hardware controllers, each communicatively connected to a special effects device, to receive one or more data packets and transmit corresponding signals compatible with said connected special effects devices to facilitate generating special effects; Under the control of special effects-enabled code logic, discovering and registering two or more of the plurality of special effects controllers at a plurality of network addresses to create an ad-hoc special effects synchronization zone; receiving an indication of the type and timing of a special effect; and wirelessly delivering a stream of data packets to one or more of the discovered and registered special effects controllers according to the indicated type and timing of the special effects, so that the special effects are executed on each of the special effects devices communicatively connected to a corresponding one or more of the discovered and registered special effects controllers.

11. 11. The method of claim 10, wherein at least one of the plurality of special effects controllers is located at a first network address and at least one other of the plurality of special effects controllers is located at a second network address, the first and second network addresses being in different locations from each other.

12. The method of claim 10 , wherein the received one or more data packets conform to a DMX protocol.

13. The method of claim 10 , wherein the indicated special effect type and timing is associated with an audio stream or portion of audio data.

14. The method of claim 13 , wherein the audio stream or audio data is broadcast live.

15. The method of claim 13 , wherein the audio stream or audio data is pre-recorded, pre-defined, or automatically calculated.

16. 14. The method of claim 13, wherein the stream of data packets wirelessly delivered to one or more of the plurality of discovered and registered special effects controllers is responsive to an analysis of the audio stream or audio data.

17. 11. The method of claim 10, wherein the special effect-enabled application code logic, when executed, is configured to receive an indication of the special effect type by receiving data from a sensor indicative of conditions under which the special effect type is appropriate.

18. 1. A computer-readable storage medium containing instructions that, when executed, control a computer processor to cause a plurality of special effects devices to generate special effects by performing a method comprising: Discovering and registering two or more of a plurality of special effects controllers at a plurality of network addresses to create an ad-hoc special effects synchronization zone, wherein each of the plurality of special effects controllers is communicatively coupled to one or more of the plurality of special effects devices, and receiving an indication of the type and timing of a special effect; wirelessly delivering a stream of data packets to one or more of the discovered and registered special effects controllers according to the indicated type and timing of the special effects, causing the special effects to be executed at each of the special effects devices communicatively connected to a corresponding one or more of the discovered and registered special effects controllers; and facilitating generating a computer-readable storage medium.

19. 20. The computer-readable memory medium of claim 18, wherein at least one of the plurality of special effects controllers is located at a first network address and at least another of the plurality of special effects controllers is located at a second network address, the first and second network addresses being in different locations from each other.

20. 20. The computer-readable memory medium of claim 18, wherein the stream of data packets conforms to a DMX protocol.

21. 20. The computer-readable memory medium of claim 18, wherein the type and timing of the indicated special effect is associated with an audio stream or portion of audio data.

22. 22. The computer-readable memory medium of claim 21, wherein the audio stream or audio data is broadcast live.

23. 22. The computer-readable memory medium of claim 21, wherein the audio stream or audio data is pre-recorded, pre-defined, or automatically calculated.

24. 22. The computer-readable memory medium of claim 21, wherein the stream of data packets wirelessly delivered to one or more of the plurality of discovered and registered special effects controllers is responsive to analysis of the audio stream or audio data.

25. 22. The computer-readable memory medium of claim 21, wherein the method further comprises receiving an indication of the type of special effect by receiving data from a sensor indicative of conditions under which the type of special effect is appropriate.

26. 26. The computer-readable memory medium of claim 25, wherein the sensor detects at least one of proximity, light waves, radio frequency, temperature, respiration, heart rate, and / or pulse.

Images