Real-world beacons that indicate virtual locations

Abstract

Techniques are described for facilitating the coordination of an audio-video (AV) production using multiple actors in respective remote locations such that an integrated AV product can be produced by coordinating the activities of the multiple actors who are far apart. [Solution] A director (200) can indicate a desired position for a stage prop (206) within the virtual world, and can then transmit a beacon (302) toward a real-world location corresponding to the desired position to assist in the placement of the prop.

Description

[Technical Field]

[0001] This application relates generally to technically inventive and unconventional solutions that are necessarily rooted in computer technology and result in specific technical improvements. In particular, this application relates to techniques for enabling collaborative remote action direction across multiple locations. [Background technology]

[0002] Due to health and cost considerations, people are increasingly collaborating from remote locations. As understood herein, collaborative production of films and computer simulations (e.g., computer games) using remote actors can pose unique coordination challenges. This is because a director must direct multiple actors, each of whom may be in their own studio or soundstage, when producing a film and for computer simulation-related activities such as motion capture (MoCap). For example, the way action is coordinated presents challenges in providing physical references for remote actors on individual stages or studios. The present principles provide techniques for addressing some of these coordination challenges. Summary of the Invention

[0003] Thus, the present principles provide an apparatus including at least one processor programmed with instructions for identifying a location within a virtual reality (VR) space where a video prop is desired to be located, and transmitting at least one beacon within a real world (RW) space toward a RW location corresponding to the location within the VR space where the video prop is located.

[0004] In some example implementations, the beacon may include a visible beacon, such as a laser beam. Additionally or alternatively, the beacon may include an audible tone and / or a screen display on the handset.

[0005] Additionally, if desired, instructions may be executable to identify the location of the prop within the RW over time.

[0006] Also, in some exemplary embodiments, instructions may be executable to transmit a beacon based at least in part on a position of the prop within the RW.

[0007] In another aspect, the device includes at least one computer storage that is not a transitory signal, the computer storage including instructions executable by the at least one processor for receiving an indication of a virtual location in a virtual reality (VR) space and for transmitting a beacon into the real world (RW) indicating the virtual location.

[0008] In yet another aspect, a computer-implemented method includes indicating a location in a virtual space where an object is desired to be located, and presenting a beacon in the real world (RW) toward a RW location corresponding to the virtual location.

[0009] The details of the present application, both as to its structure and operation, can best be understood in reference to the accompanying drawings, in which like reference numerals refer to like parts, and in which: [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram of an exemplary system consistent with the present principles. [Figure 2] Shows a director pointing at a virtual reality (VR) display to control activity within a film set. [Figure 3] 1 shows a beacon used to illuminate a movie set according to director commands. [Figure 4] Exemplary logic consistent with the present principles is presented in exemplary flow chart form. [Figure 5] 10 shows a screenshot of an exemplary screen display. [Figure 6]10 is a screenshot of an exemplary director display. [Figure 7] 10 is a screenshot of an exemplary stagehand screen display. [Figure 8] Additional exemplary logic consistent with the present principles is illustrated in exemplary flow chart form. [Figure 9] 10 is yet another screenshot of an exemplary director display. [Figure 10] 1 illustrates an exemplary graphical user interface (GUI) configuration that can be presented on a display consistent with the present principles. DETAILED DESCRIPTION OF THE INVENTION

[0011] Referring now to FIG. 1 , the present disclosure generally relates to a computer ecosystem having aspects of a computer network that may include consumer electronics (CE) devices. The system herein may include a server component and a client component connected via a network such that data may be exchanged between the client component and the server component. The client component may include one or more computing devices, including portable computers such as portable televisions (e.g., smart TVs, Internet-enabled TVs), laptop computers, and tablet computers, as well as smartphones and other mobile devices, including additional examples discussed below. These client devices may operate in a variety of operating environments. For example, some client computers may use, by way of example, Microsoft® operating systems, or Unix® operating systems, or operating systems manufactured by Apple Computer® or Google®. These operating environments may be used to run one or more browsing programs, such as browsers created by Microsoft®, Google®, or Mozilla®, or other browser programs capable of accessing websites hosted by Internet servers, as discussed below.

[0012] The server and / or gateway may include one or more processors that execute instructions that configure the server to receive and transmit data over a network such as the Internet. Alternatively, the client and server may be connected through a local intranet or a virtual private network. The server or controller may be instantiated by a game console such as a Sony PlayStation®, a personal computer, or the like.

[0013] Information may be exchanged between the client and the server over a network. For this purpose and for security, the server and / or client may include firewalls, load balancers, temporary storage, and proxies, as well as other network infrastructure for reliability and security.

[0014] As used herein, instructions refer to computer-implemented steps for processing information in a system. Instructions can be implemented in software, firmware, or hardware and can include any type of programmed step performed by a component of the system.

[0015] The processor may be a general purpose single-chip processor or a general purpose multi-chip processor capable of implementing logic through various lines such as address lines, data lines, and control lines, as well as registers and shift registers.

[0016] The software modules described herein by flowcharts and user interfaces may include various subroutines, procedures, etc. Without limiting the disclosure, the logic specified to be performed by a particular module may be redistributed among other software modules and / or aggregated together in a single module and / or made available in a shareable library. While a flowchart format may be used, it should be understood that the software may also be implemented as a state machine or other logical method.

[0017] The principles described herein may be implemented as hardware, software, firmware, or a combination thereof. Accordingly, the illustrative components, blocks, modules, circuits, and steps are described in terms of their functionality.

[0018] Further, as alluded to above, the logic blocks, modules, and circuits described below may be implemented or performed by a general purpose processor, a digital signal processor (DSP), a field programmable gate array (FPGA), or other programmable logic device, such as an application specific integrated circuit (ASIC), discrete gate or transistor logic, discrete hardware components, or any combination thereof, designed to perform the functions described herein. A processor may be implemented by a controller or state machine, or a combination of computing devices.

[0019] The functions and methods described below, when implemented in software, can be written in a suitable language, such as, but not limited to, C# or C++, and can be stored on or transmitted via a computer-readable storage medium, such as random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), other optical disk storage, such as compact disk read-only memory (CD-ROM) or digital versatile disk (DVD), magnetic disk storage, or other magnetic storage devices, including removable thumb drives, etc. The computer-readable medium can be established by a connection. Such connections can include, by way of example, hardwire cables, including optical fiber and coaxial wire, as well as digital subscriber line (DSL) and twisted pair wire.

[0020] Components included in one embodiment may be used in other embodiments in any suitable combination. For example, any of the various components described herein and / or illustrated in the figures may be combined, interchanged, or omitted from other embodiments.

[0021] "A system having at least one of A, B, and C" (and similarly "a system having at least one of A, B, or C" and "a system having at least one of A, B, C") includes systems having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.

[0022] 1, there is shown an exemplary system 10 in accordance with the present principles that may include one or more of the exemplary devices described above and further below. It should be noted that the computerized devices depicted in the figures herein may include some or all of the components described for the various devices in FIG.

[0023] A first of the exemplary devices included in system 10 is a consumer electronics (CE) device, which is configured as an exemplary primary display device and, in the illustrated embodiment, is an audio-video display device (AVDD) 12, such as, without limitation, an Internet-enabled TV with a TV tuner (equivalently, a set-top box that controls the TV). AVDD 12 may be an Android®-based system. Alternatively, AVDD 12 may also be a computer-controlled Internet-enabled (“smart”) phone, a tablet computer, a notebook computer, an augmented reality (AR) headset, a virtual reality (VR) headset, Internet-enabled glasses or smart glasses, another type of wearable computer-controlled device, such as a computer-controlled Internet-enabled watch, a computer-controlled Internet-enabled bracelet, or other computer-controlled Internet-enabled device; a computer-controlled Internet-enabled music player, computer-controlled Internet-enabled headphones, a computer-controlled Internet-enabled implantable device, such as an implantable skin device, or the like. In any event, it should be understood that AVDD12 and / or other computers described herein are configured to implement the present principles (e.g., communicate with other CE devices using 5G wireless standards to implement the present principles, execute the logic described herein, and perform any other functions and / or operations described herein).

[0024] Accordingly, to implement such principles, AVDD 12 may be established by some or all of the components shown in FIG. 1. For example, AVDD 12 may include one or more displays 14, which may be implemented by flat screens with high or ultra-high resolution (4K) or higher resolution, and which may or may not be touch-enabled for receiving user input signals via touching the display. AVDD 12 may also include one or more speakers 16 for outputting audio in accordance with the present principles and at least one additional input device 18, such as an audio receiver / microphone, for inputting audible commands to AVDD 12 to control AVDD 12. Furthermore, an exemplary AVDD 12 may include one or more network interfaces 20 for communicating over at least one network 22, such as the Internet, a wireless 5G network, another wide area network (WAN), a local area network (LAN), or a personal area network (PAN), under the control of one or more processors 24. Thus, interface 20 may be, without limitation, a Wi-Fi® transceiver, which is an example of a wireless computer network interface, such as, without limitation, a mesh network transceiver. Interface 20 may also be, without limitation, a Bluetooth® transceiver, a Zigbee® transceiver, an IrDA transceiver, a wireless USB transceiver, a wired USB, a wired LAN, Powerline, or MoCA. It will be understood that processor 24 controls AVDD 12 to implement the present principles, including other elements of AVDD 12 described herein, such as, for example, controlling display 14 to present images and receiving input therefrom. Furthermore, it should be noted that network interface 20 may be, for example, a wired or wireless modem or router, or other suitable interface, such as, for example, a wireless telephony transceiver or the Wi-Fi® transceiver described above.

[0025] In addition to the above, AVDD 12 may also include one or more input ports 26, such as a High-Definition Multimedia Interface (HDMI®) port or a USB port for physically connecting to another CE device (e.g., using a wired connection) and / or a headphone port for connecting headphones to AVDD 12 to provide audio from AVDD 12 to a user through the headphones. For example, input port 26 may be connected wired or wirelessly to a cable or satellite source 26a of audio-video content. Thus, source 26a may be, for example, a separate or integrated set-top box, or a satellite receiver. Alternatively, source 26a may be a game console or disc player.

[0026] AVDD 12 may further include one or more computer memories 28, such as non-transitory disk-based or solid-state storage, which may in some cases be embodied within the AVDD chassis as a standalone device, or as a personal video recording device (PVR) or video disc player, or as removable memory media, either internal or external to the AVDD chassis for playing AV programs. In some embodiments, AVDD 12 may also include a position or location receiver, such as, but not limited to, a cellular telephone receiver, a GPS receiver, and / or an altimeter 30, configured to receive geographic location information from at least one satellite or cellular telephone tower and provide that information to processor 24 and / or to determine the altitude at which AVDD 12 is located in conjunction with processor 24. However, it should be understood that another suitable position receiver other than a cellular telephone receiver, a GPS receiver, and / or an altimeter may be used in accordance with the present principles, for example, to determine the position of AVDD 12 in all three dimensions.

[0027] Continuing with the description of AVDD 12, in one embodiment, AVDD 12 may include one or more cameras 32, which may be, for example, a thermal imaging camera, a digital camera such as a webcam, and / or a camera integrated into AVDD 12 and controllable by processor 24 to collect pictures / images and / or video in accordance with the present principles. Also included in AVDD 12 may be a Bluetooth transceiver 34 and other NFC elements 36 for communicating with other devices using Bluetooth and / or near field communication (NFC) technology, respectively. An exemplary NFC element may be a radio frequency identification (RFID) element.

[0028] Furthermore, AVDD 12 may include one or more auxiliary sensors 38 (e.g., motion sensors such as an accelerometer, gyroscope, cyclometer, or the like, or magnetic sensors, infrared (IR) sensors for receiving IR commands from a remote control, optical sensors, speed and / or cadence sensors, gesture sensors (e.g., sensors for detecting gesture commands), etc.) that provide input to processor 24. AVDD 12 may include an over-the-air TV broadcast port 40 for receiving over-the-air (OTA) TV broadcasts that provide input to processor 24. In addition to the above, it should be noted that AVDD 12 may also include an infrared (IR) transmitter and / or IR receiver and / or IR transceiver 42, such as an infrared data association (IRDA) device. A battery (not shown) may be included to power AVDD 12.

[0029] Additionally, in some embodiments, AVDD 12 may include a graphics processing unit (GPU) 44 and / or a field programmable gate array (FPGA) 46. The GPU and / or FPGA may be utilized by AVDD 12 for artificial intelligence processing, such as, for example, training a neural network and performing operations (e.g., inference) of the neural network in accordance with present principles. Note, however, that processor 24 may also be used for artificial intelligence processing, such as in the case where processor 24 may be a central processing unit (CPU).

[0030] 1, in addition to AVDD 12, system 10 may include one or more other types of computing devices that may include some or all of the components shown in AVDD 12. In one example, a first device 48 and a second device 50 are shown, which may include components similar to some or all of the components of AVDD 12. Fewer or more devices than shown may be used.

[0031] System 10 may also include one or more servers 52. Server 52 may include at least one server processor 54, at least one computer memory 56, such as disk-based or solid-state storage, and at least one network interface 58 that, under the control of server processor 54, enables communication with other devices of FIG. 1 over network 22 and, indeed, may facilitate communication between servers, controllers, and client devices in accordance with the present principles. Note that network interface 58 may be, for example, a wired or wireless modem or router, a Wi-Fi® transceiver, or other suitable interface, such as, for example, a wireless telephony transceiver.

[0032] Thus, in some embodiments, server 52 may be an Internet server or may include or perform "cloud" functionality, allowing devices of system 10 to access the "cloud" environment via server 52 in exemplary embodiments. Alternatively, server 52 may be implemented by a game console or other computer located in or near the same room as the other devices shown in FIG. 1 .

[0033] The devices described below may incorporate some or all of the elements described above.

[0034] 2, a director 200 is shown pointing at a virtual reality (VR) display 202 to control activity within a single movie set and / or across multiple movie sets, where several actors may be remotely located. The VR display 202 may be a wall-mounted television, a computer monitor, or even a heads-up display of a VR headset worn by the director 200.

[0035] Consistent with the present principles, images of the director's 200 hands and arms 204 can be collected by one or more cameras that show the director 200 within their field of view, which may include one or more outward-facing cameras on the VR headset itself, if one is being worn by the director 200. Images from the camera(s) can then be used to perform computer vision and / or gesture recognition to track and identify the director 200 as he or she is performing a gesture by pointing with the index finger of his or her right hand, as shown.

[0036] 2, at a first time T1, director 200 points with his index finger at a prop 206 in a real-world (RW) video represented in a VR space presented on a VR display 202, indicating to the VR system that director 200 is providing input directed toward the prop 206. Director 200 may then gradually move his right arm from left to right (or another direction as desired) while pointing with his index finger until a later time T2, as shown, when he stops moving his arm when it points toward a desired virtual location in the VR space of the prop 206. The virtual location may correspond to a RW location on a set, where the prop 206 is placed in the RW for interaction with one or more actors also on the set, translating the RW actor's actions relative to the prop 206 into actions of a virtual space character. In some examples, if the director 200 points to a desired virtual location for a prop, a verbal cue from the director 200, such as "move this prop here," may also be used to move the prop.

[0037] Before moving on to the description of other figures, RW, note that tangible prop 206 can be several things, including real-world inanimate stage objects such as tables, chairs, etc. Prop 206 can also be a "dummy" weapon such as a fake gun or fake sword. Prop 206 can also be any element that corresponds in shape and possibly size to a cutout of a person, a sculpture representing a geographic feature, or a virtual object in a VR space (e.g., a video game object).

[0038] 3 illustrates a beacon device 300 used to illuminate a movie set according to director commands consistent with the present principles. Beacon device 300 may include hardware for transmitting or presenting a visible beacon, such as a laser beam 302. Laser beam 302 may, for example, intersect with a point or location 304 on a surface toward which laser beam 302 is directed.

[0039] The laser beam 302 can be directed based on director commands and using motors or other elements within the device 300 to transmit the laser light toward a RW location within the RW where the RW prop will be placed to correspond to a VR location where the director wants the prop 306 to appear in the VR space. As also shown in FIG. 3 , the prop 306 can include a position sensor 308, such as a GPS transceiver or other suitable position sensor for position tracking over time. The sensor 308 can also include, for example, a Wi-Fi or other wireless signal transceiver, which can be used to triangulate and track the position of the prop 306 over time in all three dimensions based on the known locations of other devices in communication with the transceiver.

[0040] The beeper 310 may also be included on or attached to the prop 306 and may be, for example, a speaker configured to emit an audible "beep," other audible sound, or even audible computerized verbal commands indicating the desired prop placement. A separate handset may be used to track its position relative to the prop 306, for example, carried separately by a stagehand and emit a beep or other sound when initially placing or moving the prop 306. Alternatively, the frequency and / or volume at which the beeper is presented may gradually increase as the prop 306 is tracked over time, for example, as it moves toward the RW position directed by the director where the RW prop 306 will be placed. Conversely, the frequency and / or volume at which the beeper is presented may gradually decrease as the prop 306 is tracked over time, for example, as it moves away from the RW position directed by the director where the RW prop 306 will be placed. Next, when the prop 306 is finally positioned in the desired RW position, a constant noise of the same pitch as the beep may be generated by the beeper 310, and / or a tone of a different pitch may be generated to indicate that the prop 306 is positioned as the desired RW position.

[0041] Additionally or alternatively, in some embodiments, the beeper 310 may further include a video display on which visual directional guidance may be presented, such as a three-dimensional (3D) arrow or other form of visual guidance. However, regardless of whether audible or visual guidance is presented on the beeper 310, it should be understood that the guidance may initially guide a stagehand or other person to position or move the prop 306 toward the RW position, because the prop 306 is also presented in VR space at a desired VR position correlated to the RW position.

[0042] It is further noted that in addition to, or instead of, using the guidance generated by beeper 310, the stagehand may also move prop 306 toward position 304 based on the stagehand visually identifying laser beam 302 as intersecting the surface that establishes position 304.

[0043] Referring now to Figure 4, Figure 4 illustrates exemplary logic in exemplary flowchart form that may be performed, alone or in combination, by one or more devices consistent with the present principles. For example, the logic of Figure 4 may be performed by a server or other device that coordinates recorded actions in a RW space with actions of a character in a VR space and reflects the actions in the VR space. The logic of Figure 4 may also be performed by another device that communicates with the director's VR headset, display 202, beacon 300, and / or beeper 310 to carry out the present principles.

[0044] In any event, the logic begins at block 400, where the device may receive a RW prop position signal, such as from a GPS transceiver or other position transceiver on the prop, indicating the prop's current position. The logic may then proceed to block 402, where the device may receive the prop's position in VR space, as facilitated by a VR simulation engine running on the device or elsewhere, which correlates RW space coordinates captured by one or more cameras in the RW to VR space coordinates, for example.

[0045] After block 402, the logic may proceed to block 404. In block 404, the device may receive a director's movement of the prop within the VR space, such as by the director virtually moving a representation of the prop within the VR space and / or by specifying with a gesture or verbal cue where the VR representation of the prop should be located within the VR space. The logic may then proceed to block 406.

[0046] In block 406, the device may track the RW position of the prop while stationary and / or while moving throughout the RW space. After block 406, the logic may proceed to block 408. In block 408, the device may generate and / or transmit one or both of an audio and a visual signal regarding where the prop should be located in the RW space corresponding to the VR position indicated by the director. For example, in block 408, the device may control device 300 to emit a laser as described above and / or control beeper 310 to provide audio and visual aids as described above.

[0047] Continuing with the detailed description with reference to Figure 5, Figure 5 shows an example screenshot 500 of an example screen display. For example, the screen display may be presented on the display of a handset, such as a dedicated prop device handset or a smartphone. Screenshot 500 may also be presented on the video display of beeper 310, as described above, in embodiments in which beeper 310 includes a video display.

[0048] In any event, as shown in Figure 5, a prop image 502 may be presented on a screen display either using an actual image of the prop from a camera or using a computer-generated representation of the prop, and the screen display may include a position indication 504 represented on the screen display corresponding to the RW stage position to which the prop itself needs to be moved.

[0049] 5, an arrow 506 or other graphical indicator may also be presented as part of the screen display and may be adjusted in real time as the prop moves throughout RW space, indicating where the prop needs to be moved relative to the prop's current position. Thus, arrow 506 may be presented in 3D format on the screen display to guide the stagehand as he moves the prop from the prop's current RW position correlated to image 502 to the RW position of the director's desired prop position correlated to indication 504.

[0050] 6-9 further illustrate enabling a director to command the movement of real-world props (real-world inanimate stage objects, such as tables, chairs, etc.) within one or more studios or stages consistent with the present principles. In this example, it is assumed that the real-world props are tracked by imaging the props using cameras within the studio or by receiving the prop's location information, etc., from a location receiver, such as a global positioning satellite (GPS) receiver, on the props.

[0051] Figure 6 shows an image 600 of a real-world prop in a virtual space on a director computer 604. A director can operate the director computer 604 to move the image 600 using voice commands and / or selector elements 602 presented on the display of the computer 604 to send commands to a stagehand device 700, shown in Figure 7, to place the prop corresponding to the image 600 in the real-world location where it is located. The stagehand device 700 can be a portable computer, such as a mobile phone or tablet, or other suitable device, which may include, for example, an organic light-emitting diode (OLED) display.

[0052] The commands include the prop's position information, from which an image 702 of the prop can be placed in virtual space and presented on the stagework device 700. Director commands are presented graphically and / or textually and / or audibly on the stagework display as indicated at 704, and indicate the desired new position of the prop at 706. As the real-world prop is moved, it is tracked as described herein and its position in virtual space is presented to both the director computer 604 and the stagework device 700, allowing both the director and stageworkers to monitor the prop's movement in virtual space until it is placed in the desired position 706.

[0053] 8 further reflects the above discussion. In block 800, the stage worker device 700 receives director commands from the director computer 604. Moving to block 802, it audibly and / or visually presents an image of the prop on a stage worker display. As the prop moves, in block 804, its real-world position is tracked and translated into virtual space, and in block 806, feedback of the prop movement is sent to the director computer 604, which updates it on the stage worker device 700, such as by updating the virtual positions of the prop images 600, 702 on their respective displays.

[0054] 9 illustrates that a director can choose to provide feedback to the stagehand to position the prop in the real world and reach the director's desired location. The director computer 604 can present an image 900 of the prop in an initial position, an image 902 of the prop in a current position as it moves, and an image 904 of the prop in a desired position, and when input element 906 is provided, the director can input and send commands to the stagehand to enable the stagehand to continue moving the prop in the indicated direction using, for example, up, down, left, and right commands corresponding to the various arrows establishing element 906.

[0055] Referring now to Figure 10, Figure 10 illustrates an exemplary graphical user interface (GUI) 1000 configuration that may be presented on the display of a device or system configured to operate consistent with the present principles. For example, GUI 1000 may be presented on the display of director computer 604, a director's headset or connected laptop computer, a remotely located server, or another device that controls the use of beacons consistent with the present principles. Note that each option described below can be selected by touching or directing cursor input to a checkbox adjacent to the respective option.

[0056] As shown in FIG. 10 , GUI 1000 may include a first option 1002 that may be selectable to configure or enable a VR device or system to perform the present principles. For example, option 1002 may be selected to set or configure one or more device(s) to execute the logic of FIGS. 4 and / or 8 and present or transmit beacons as disclosed herein, as well as perform other functions described above. FIG. 10 also shows that GUI 1000 may include a second option 1004 that may be selectable to configure or enable a device or system to specifically use an audible beacon consistent with the present principles. GUI 1000 may further include a third option 1006 that may be selectable to configure or enable a device or system to specifically use a visual beacon consistent with the present principles, such as a laser and screen display as described herein.

[0057] Thus, based on the description of the preceding figures, it can be understood that VR staging and / or remote filming can be facilitated, allowing a director to view the virtual scene itself from the perspective of the director or game player within the virtual scene and gain a sense of the actors' spatiality in context. In some examples, in addition to or instead of the director using a VR headset to view the VR scene and specify the placement / location of props, the director can also use a remote-controlled RW rig to "hold" a corresponding virtual rig in virtual space, view a corresponding viewpoint within the VR scene, and move the VR rig throughout, thereby moving the director's VR first-person viewpoint throughout the virtual space. In some examples, the director may even "pick up" the VR rig and place it in his desired position, then let go of it, so that while in the specified position, the VR viewpoint presented to the director is shown as the viewpoint of a virtual camera currently stationary on the VR rig.

[0058] In either case, looking at the virtual camera's perspective within the VR scene, the director can communicate with stagehands and other assistants at various remote locations on stage to provide prop placement instructions, with RW action performed on stage being merged into a single VR scene. In some examples, an artificial neural network (ANN) with an input layer, an output layer, and one or more hidden layers in between can be encoded to listen for the director's commands and provide directional assistance to the stagehands based thereon. Thus, voice commands, RW prop positions, and the corresponding VR positions of the props can be provided as inputs to the ANN during training to train the ANN to output directional assistance in the form of audible beeps, lasers, on-screen displays, etc., as disclosed herein.

[0059] Thus, the present disclosure may aid in a director's ability to record or present the director's point of view as well as to record various actor actions that aid in the creation of a VR scene, and may aid in enhancing the director's ability to interact with virtual objects in VR space that correspond to RW props in RW space. The generated data, instructions, etc. may then be made available for post-processing to merge RW scenes recorded from a remotely located stage into a single VR scene.

[0060] In this case, the director's virtual camera (e.g., the director's perspective within the VR scene) can be changed by moving the director's RW camera or by switching to a different RW camera so that the director can see the virtual "set" (e.g., the VR scene itself) while within the VR scene, see the actors' positions within the VR scene as they virtually move throughout the scene based on their tracked RW movement, and even see the positions of props tracked by the VR system so the director knows when the props are in the director's desired virtual position.

[0061] This can be accomplished by the director using his / her RW arm to click or point at the location in the VR scene where he / she wants to place the prop while immersed in the VR scene. The director's assistants on one of the RW physical sets can then guide those sets to the pointed RW stage location that corresponds to the location in the VR scene, possibly compensating for XYZ dimensional offsets between the RW space and the corresponding VR space, and can have auditory and / or visual guides to assist the assistant as they approach the correct RW coordinates that match the VR coordinates of the director's desired prop location. Thus, feedback can be given to the assistant to position the prop at the director's desired location on the RW stage.

[0062] Thus, in some instances of establishing a new resting position for a prop, the feedback may be in the form of a handset with a display screen that indicates where the RW prop needs to be moved to place it where the director wants it, and the assistant may use an audio earpiece or another type of speaker to navigate the prop to the director's desired position throughout the RW space.

[0063] Additionally or alternatively, the drone or other device may, in some instances, autonomously navigate itself through the air in the RW to the desired prop location, and the assistant may then bring the prop itself to a drone location at the desired prop location and swap the drone for the prop at that location.

[0064] Thus, a beacon consistent with the present principles can take various forms to communicate to an assistant (or others) where the coordinates of the director's desired VR position for a prop are located in the corresponding real space of the set. Thus, the director can immerse themselves in the VR scene (e.g., using a VR headset) and look at it, saying something like, "I think this rock needs to be here." The system can use digital assistant, natural language understanding, and / or speech recognition technology to identify what the director is saying, and also identify what is said while the director is clicking, pointing, or otherwise selecting the rock (e.g., selecting it in virtual space using a VR controller). Based on these two inputs, the VR system can instruct another device, such as a handset, possibly located at an offset, to begin buzzing using a vibrator, indicating that the director has established the coordinates for placing the rock. The assistant, noticing this, can pick up the handset, use the arrow or other guide presented therein to pick up the RW rock itself, and navigate to the RW position on the set or stage corresponding to the coordinates established by the director.

[0065] If the rock is not yet perfectly positioned where the director wants it, the director can say "move it left," "move it right," etc., providing additional cues to the handset to further guide the assistant, or again, the director can click or select the virtual space location that corresponds to the RW location where they want to place the rock (or other prop).

[0066] While the present principles have been described with reference to certain exemplary embodiments, it will be understood that these are not intended to be limiting and that various alternative configurations may be used to implement the subject matter claimed herein.

Claims

1. Identifying a desired location for a video prop within a virtual reality (VR) space; transmitting at least one beacon in a real-world (RW) space toward a RW position corresponding to the position in the VR space where the video prop is located, and intersecting the RW position, the beacon including a visible beacon, an audible beacon, or both a visible beacon and an audible beacon; 11. An apparatus comprising: at least one processor programmed with instructions for:

2. The apparatus of claim 1 , wherein the beacon comprises a laser beam.

3. The apparatus of claim 1 , wherein the instructions are executable to identify a position of the video prop within the RW over time.

4. The apparatus of claim 3 , wherein the instructions are executable to transmit the beacon based at least in part on the position of the video prop within the RW.

5. The device of claim 1 , wherein the beacon comprises an audible tone.

6. The device of claim 1 , wherein the beacon comprises a screen display on a handset.

7. 1. A device including at least one computer storage device, The at least one computer storage device may transmit to the at least one processor, rather than a transient signal, receiving an indication of a virtual location within a virtual reality (VR) space; transmitting a beacon, including a visible beacon or an audible beacon, towards a real-world (RW) location corresponding to the virtual location; The device includes instructions to cause the device to execute the steps of:

8. The device of claim 7 , wherein the virtual position is a desired position of a RW prop.

9. The device of claim 7 , wherein the beacon comprises a laser beam.

10. The device of claim 7 , wherein the instructions cause the device to identify a position of a prop within the RW over time.

11. The device of claim 10 , wherein the instructions cause transmitting the beacon based at least in part on the position of the prop within the RW.

12. The device of claim 7 , wherein the beacon comprises an audible tone.

13. The device of claim 7 , wherein the beacon comprises a screen display on a handset.

14. indicating a location in the virtual space where the object is desired to be located; pointing a beacon in the real world (RW) to a RW location corresponding to a location in the virtual space, the beacon comprising a visible beacon or an audio; 11. A computer-implemented method comprising:

15. The method of claim 14 , comprising identifying the location of the object within the RW over time.

16. The method of claim 14 , comprising presenting the beacon based at least in part on the location of the object within the RW.

17. The method of claim 14 , wherein the beacon comprises an audible tone.

Images