Scalable interactive video system and method

Abstract

The present embodiment relates to a distributed control system for an interactive video system. The distributed control system includes a switching fabric network including an input node implemented to collect scan data of a participant. The switching fabric network also includes a set of processing nodes, which are implemented to generate a virtual representation of the participant based at least in part on the scan data of the participant. In addition, the switching fabric network includes an output node, which includes an output device implemented to present a virtual representation of the participant in a virtual environment.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of U.S. Provisional Application No. 62 / 783,544, filed on December 21, 2018, entitled “Scalable Interactive Video Systems and Methods,” which is hereby incorporated by reference in its entirety for all purposes. Background Art

[0003] The present disclosure relates generally to video systems and, more particularly, to interactive video game systems that enable simultaneous multi-player gameplay.

[0004] Video game systems typically enable participants (e.g., players) to control characters in a virtual environment to achieve predefined goals or objectives. Traditional video game systems typically rely on manual input devices, such as joysticks, game controllers, keyboards, etc., to enable participants to control characters within the virtual environment of the game. In addition, some modern video game systems may include cameras that can track the movement of participants, thereby enabling participants to control video game characters based on their movement. However, these systems typically suffer from problems with latency and / or data throughput. For example, these systems may facilitate limited data collection and / or processing, which may limit the experience (e.g., graphics and / or effects) provided by the video game. In addition, the processing time involved in controlling the video game may introduce a delay between the input received from the participant and the resulting output. In addition, in a multi-player video game system, the possibility of latency increases significantly with the number of participants. Therefore, it may be inconvenient to use these systems to provide a smooth and responsive real-time interactive experience for participant interaction. Summary of the invention

[0005] The following summarizes certain embodiments that are commensurate in scope with the originally claimed subject matter. These embodiments are not intended to limit the scope of the present disclosure, but rather these embodiments are intended only to provide a brief overview of certain disclosed embodiments. In fact, the present disclosure may encompass a variety of forms that may be similar or different from the embodiments set forth below.

[0006] The present embodiment relates to a distributed control system for an interactive video system. The distributed control system includes a switching fabric network interconnecting a plurality of nodes. The switching fabric network includes an input node, which includes a sensing unit. The sensing unit is implemented to collect scan data of a participant of the interactive video system. In addition, the switching fabric network includes a set of processing nodes. The set of processing nodes is implemented to generate a virtual representation of the participant based at least in part on the scan data of the participant. The set of processing nodes is communicatively coupled to the input node via a first switch of the switching fabric network. In addition, the switching fabric network includes an output node. The output node includes an output device, which is implemented to present a virtual representation to the participant in a virtual environment. The output node is communicatively coupled to the input node via a second switch of the switching fabric network.

[0007] The present embodiment also relates to a method for operating an interactive video system. The method includes collecting scan data of a participant of the interactive video system at an input node among a plurality of nodes in a switching structure network of a distributed control system of the interactive video system. The method also includes generating a virtual representation of the participant based at least in part on the scan data of the participant at a first set of processing nodes among the plurality of nodes. The first set of processing nodes is communicatively coupled to the input node via a first switch of the switching structure network. In addition, the method includes presenting a virtual representation of the participant in a virtual environment using an output node among the plurality of nodes. The output node is communicatively coupled to the input node via a second switch of the switching structure network. The method also includes determining a set of effects in a game based at least in part on the scan data of the participant at a second set of processing nodes. In addition, the method includes updating the virtual environment based at least in part on the effects in the game using the output node.

[0008] The present embodiment also relates to an interactive video system including a distributed control system, which includes a switching fabric network interconnecting a plurality of nodes. The plurality of nodes include an input node, and the input node includes a sensing unit. The sensing unit is implemented to collect scan data of a participant. The plurality of nodes also include a set of processing nodes. The set of processing nodes is communicatively coupled to the input node via the switching fabric network. The set of processing nodes is implemented to generate a virtual representation of a participant based at least in part on the scan data of the participant. The set of processing nodes is also implemented to determine a set of effects in a game based on the scan data of the participant. The scan data may include one or more in-game actions of the participant. The plurality of nodes additionally include an output node. The output node is communicatively coupled to the input node via the switching fabric network. In addition, the output node is communicatively coupled to the set of processing nodes via the switching fabric network, and the output node includes a set of output devices. The set of output devices is implemented to present a virtual representation of a participant in a virtual environment. The set of output devices is also implemented to update the presentation of the virtual environment based at least in part on the effects in the game. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] These and other features, aspects and advantages of the present disclosure will become better understood when the following detailed description is read with reference to the accompanying drawings, wherein like characters refer to like parts throughout the drawings, and wherein:

[0010] Figure 1 is a schematic diagram of an embodiment of an interactive video system according to the present technology that enables multiple participants to control respective virtual representations by performing actions in a participation area;

[0011] Figure 2 is a block diagram of a switch fabric network according to the present technology;

[0012] Figure 3 is a flow chart illustrating an embodiment of a process for operating an interactive video system according to the present technology; and

[0013] Figure 4 is a flow chart illustrating an embodiment of a process according to the present technology by which Figure 3 Certain actions are indicated in the flowchart. DETAILED DESCRIPTION

[0014] As used herein, "scan data" refers to two-dimensional (2D) or three-dimensional (3D) data collected by sensing (e.g., measuring, imaging, ranging) the visible external surfaces of participants in the participation area. More specifically, "volume scan data" as used herein refers to 3D scan data (such as point cloud data) and can be contrasted with "2D scan data" (such as image data).

[0015] As used herein, a "participant model" is a 2D or 3D model generated from scan data of a participant, which generally describes the external surface of the participant and may include texture data. More specifically, a "volume participant model" or "volume model" as used herein refers to a 3D participant model generated from volume scan data of a participant, and may be contrasted with a "2D participant model" generated from 2D scan data of a participant.

[0016] As used herein, a "shadow model" refers to a textureless volumetric model of a participant generated from scan data of the participant, either directly or through a participant model. Thus, when rendered on a 2D surface such as a display device, the shadow model of the participant has a shape substantially similar to the shadow or silhouette of the participant when lit from behind.

[0017] As used herein, a "skeletal model" refers to a 3D model generated from scanned data of a participant that defines the predicted position and location of certain body parts and / or center of mass (e.g., torso, arms, legs, head) of the participant to describe the position and posture of the participant within the engagement area. Thus, the skeletal model is used to determine the movement and actions of the participant in the engagement area to trigger events in the virtual environment and / or in the engagement area.

[0018] The present embodiment relates to an interactive video system (e.g., a video game system) that enables multiple (e.g., 1 to 12) participants (e.g., players) to perform actions in a physical environment (e.g., a participation area) to control the virtual representation of the participants in a displayed virtual environment (e.g., a presentation area). The disclosed interactive video system includes one or more sensing units, such as volume sensors (e.g., depth cameras and / or light detection and ranging (LIDAR) devices), which are capable of scanning each participant and / or capturing each participant's in-process (e.g., in-game) actions. The interactive video system may also include multiple output devices capable of presenting the virtual environment and / or generating one or more in-process effects, such as display devices, audio devices, physical effect devices, and / or the like. In addition, the interactive video system includes a distributed control system having multiple nodes (e.g., input nodes, processing nodes, database nodes, output nodes). The distributed control system can be implemented to control the output device based in part on input data received from various sensing and input devices in the interactive video system. Therefore, the interactive video system continuously updates the virtual representation of the participant and the virtual environment based on the participant's actions and their corresponding in-process effects. Examples of the present embodiments disclosed herein are described in the context of video games. However, it should be understood that the present embodiments can be applied to and include activities other than gaming activities. In addition, it should be noted that the term video games broadly includes a large number of entertainment activities that utilize video technology.

[0019] In addition, in order to create a real-time interactive experience that is smooth and responsive to participant interactions, the interactive video system may include a switching fabric network. The switching fabric network may include multiple nodes that are communicatively coupled together through a collection of switches and interconnects, which may facilitate low latency, high bandwidth communications. In addition, the switching fabric network may support remote direct memory access (RDMA). The nodes of the switching fabric network may include sensing and input devices, output devices, processing nodes, and / or database nodes. For example, sensing and / or additional input devices in an interactive video system may be implemented as one or more input nodes of a switching fabric network, and output devices may be implemented as one or more output nodes of a switching fabric network. Certain processing nodes may include dedicated processors, such as vector processing units (VPUs) and / or tensor processing units (TPUs), which may be optimized to perform specific functions. Therefore, certain processing nodes may be dedicated to performing certain processing tasks involved in an interactive video system based on their respective capabilities, such as visual (e.g., graphics) processing, audio processing, artificial intelligence (AI) computing, physical computing, and / or the like. In addition, the switching fabric network may divide these processing tasks between a plurality of suitable processing nodes. In addition, the switch fabric network can facilitate remote direct memory access (RDMA). Therefore, using RDMA, data can be transferred between physically separated processing nodes to perform processing tasks with the latency of an internal computer bus, enabling nodes to seamlessly read information from other nodes and write information to other nodes, which are in fact in a large combined memory space that can be considered as all nodes in the switch fabric network. Therefore, the switch fabric network can efficiently process and update the data used in the interactive video system. In addition, the interactive video network can be effectively expanded (scaled) because additional processing nodes, input nodes and / or outputs can be added to the switch fabric network without suffering significant latency losses.

[0020] Considering the foregoing, Figure 1 1 is a schematic diagram of an embodiment of an interactive video system 10 (e.g., a video game system) that enables multiple participants 12 (e.g., participants 12A and 12B) to control corresponding virtual representations 14 (e.g., virtual representations 14A and 14B) by performing actions in a participation area 16. It may be noted that while the present description relates to two participants 12 using the interactive video system 10 for simplicity, in other embodiments, the interactive video system 10 may support more than two (e.g., 6, 8, 10, 12, or more) participants 12.

[0021] Engagement area 16 of interactive video system 10 may have a width (corresponding to x-axis 18), a height (corresponding to y-axis 20), and a depth (corresponding to z-axis 22), wherein system 10 generally monitors the movement of each participant 12 along x-axis 18, y-axis 20, and / or z-axis 22. Interactive video system 10 updates the position of virtual representation 14 presented on display device 24 of presentation area 25 in virtual environment 32 along x-axis 26, y-axis 28, and / or z-axis 35 in response to participant 12 moving within engagement area 16 (e.g., running along x-axis 18, jumping along y-axis 20, running along z-axis 22).

[0022] Figure 1 The embodiment of interactive video system 10 illustrated in FIG. 1 includes a plurality of sensing units 38 disposed within and / or around engagement area 16. More specifically, sensing units 38 may be described as being symmetrically distributed around the perimeter of engagement area 16. In some embodiments, at least a portion of sensing units 38 may be positioned above engagement area 16 (e.g., suspended from a ceiling or suspended from an elevated platform or stand) and directed at a downward angle to image engagement area 16. In other embodiments, at least a portion of sensing units 38 may be positioned near the floor of engagement area 16 and directed at an upward angle to image engagement area 16. In some embodiments, interactive video system 10 may include at least two sensing units 38 for each participant (e.g., participants 12A and 12B) in engagement area 16. Thus, sensing units 38 are appropriately positioned to image a significant portion of potential vantage points around engagement area 16 to reduce or eliminate potential participant occlusion.

[0023] In the illustrated embodiment, each sensing unit 38 includes a corresponding volumetric sensor 40, which may be an infrared (IR) depth camera, a LIDAR device, or another suitable ranging and / or imaging device. For example, in some embodiments, all of the volumetric sensors 40 of the sensing unit 38 are IR depth cameras or LIDAR devices, while in other embodiments, there is a mix of both IR depth cameras and LIDAR devices. It is currently recognized that both IR depth cameras and LIDAR devices can be used to perform a volumetric scan of each of the participants 12, and the collected scan data (e.g., volumetric scan data) can be used to generate various models of the participants, as discussed below. For example, in some embodiments, an IR depth camera can be used to collect data to generate a skeletal model, while the data collected by the LIDAR device can be used to generate a volumetric and / or shadow model of the participant 12. It is also recognized that a LIDAR device that collects point cloud data is generally capable of scanning and mapping larger areas than a depth camera, generally with better accuracy and resolution. Thus, in some embodiments, at least one sensing unit 38 includes a corresponding volume sensor 40 that is a LIDAR device to enhance the accuracy or resolution of interactive video system 10 and / or to reduce the total number of sensing units 38 in interactive video system 10.

[0024] Additionally, in certain embodiments, other sensing devices may be included in and around the engagement area 16. For example, a radio frequency (RF) sensor 45 may be disposed near (e.g., above, below, adjacent to) the engagement area 16. The illustrated RF sensor 45 receives a uniquely identifying RF signal from a wearable device 47, such as a bracelet or headband having a radio frequency identification (RFID) tag worn by each of the participants 12. In response, the RF sensor 45 provides signals regarding the identity and relative location of the participants 12 in the engagement area 16, as described in more detail below.

[0025] in addition, Figure 1 The interactive video system 10 illustrated in FIG. 1 controls the output of stimuli (e.g., audio signals, video signals, lights, physical effects) at a presentation area 25 that is observed and experienced by participants 12 in a participation area 16 as part of a virtual environment 32. Thus, the presentation area 25 may include a plurality of audio devices 50 and / or display devices 24. The display device 24 may be any suitable display device, such as a projector and screen, a flat screen display device, or an array of flat screen display devices, that are arranged and designed to provide a suitable view of the virtual environment 32 to the participants 12 in the participation area 16. In some embodiments, the audio devices 50 may be arranged in an array around the participation area 16 to increase participant immersion during game play.

[0026] Additionally, in one embodiment, the interactive video system 10 may include an interface panel 74 that enables enhanced participant interaction. Figure 1 As illustrated in , the interface panel 74 includes a plurality of input devices 76 (e.g., cranks, wheels, buttons, sliders, blocks) designed to receive input from the participant 12 during game play. The illustrated interface panel 74 also includes a plurality of output devices 78 (e.g., audio output devices, visual output devices, physical stimulation devices) designed to provide audio, visual and / or physical stimulation to the participant 12 during game play. For example, the output devices 78 may include audio devices such as speakers, horns, sirens, etc. The output devices 78 may also include visual devices such as lights or display devices of the interface panel 74. In some embodiments, the output devices 78 of the interface panel 74 include physical effect devices, such as an electronically controlled release valve coupled to a compressed air line that provides a burst of warm or cold air or mist.

[0027] As also illustrated, the interactive video system 10 includes a distributed control system 82. The distributed control system 82 typically receives input information from the various aforementioned input devices, and determines appropriate outputs based in part on the received input information. For example, the distributed control system 82 may receive scan data collected by each volume sensor 40, participant input from the input device 76, signals provided by the RF sensor 45 about the identity and relative positioning of the participant 12 in the participation area 16, and / or the like. The distributed control system 82 may combine the data (which is collectively described herein as "participant data") to determine the identity, location, and / or action of the participant 12 in the participation area 16 during game play. In addition, the distributed control system 82 may determine instructions (e.g., control signals) to control the audio device 50, the display device 24, the output device 78, and / or the like to generate various stimuli for the participant 12 to observe and experience via the presentation area 25. In some embodiments, the distributed control system 82 may determine instructions for controlling the audio device 50, the display device 24, the output device 78, and / or the like based in part on the collected participant data.

[0028] It is currently recognized that processing the collected participant data and / or determining instructions for generating various stimuli in the presentation area 25 can be processor intensive. In addition, the latency of the interactive video system 10 may limit the ability of the interactive video system 10 to provide a smooth and responsive real-time interactive experience for participant interaction. Therefore, in some embodiments, it can be advantageous to split the workload by utilizing multiple processors that are communicatively coupled by a network to process the collected participant data. Therefore, in the illustrated embodiment, the distributed control system 82 includes a plurality of processing nodes 84 and database nodes 86 (e.g., data storage repositories) that are communicatively coupled together via a switching fabric network 90 (such as an INFINIBAND® network). As described in more detail below, the processing node 84 may include a memory 102 and / or a processor 104. As also illustrated, each of the input devices (e.g., volume sensors 40, RF sensors 45, input devices 76) and output devices (e.g., audio devices 50, display devices 24, output devices 78) of the interactive video system 10 is communicatively coupled to the switching fabric network 90. Thus, as described in more detail below, distributed control system 82 may transmit data to and receive data from one or more devices in interactive video system 10 via switch fabric network 90 .

[0029] To help illustrate, Figure 2 , an embodiment of a switching fabric network 90 including a plurality of discrete nodes 96 is shown. In the case of using the switching fabric network 90, the data processing of the interactive video system 10 can be distributed between any suitable number of discrete nodes 96, which can be physically distinct from each other without introducing significant delays or latency. That is, for example, the interactive video system 10 can output stimuli to the participant 12 in real time or near real time and / or provide smooth and responsive movement of the virtual representation 14 in the virtual environment 32. As illustrated, each of the processing nodes 84 of the distributed control system 82 and the devices of the interactive video system 10 (e.g., sensing unit 38, RF sensor 45, input device 76, audio device 50, display device 24, output device 78, and / or the like) can represent different nodes 96 of the switching fabric network 90. ​​The input device (e.g., volume sensor 40, RF sensor 45, input device 76) is represented by an input node 97, and the output device (e.g., audio device 50, display device 24, output device 78) is represented by an output node 98. However, it is to be understood that in other embodiments, each input and / or output device may be coupled to the switch fabric network 90 in a different manner. In other words, the embodiments described herein are intended to be illustrative rather than limiting.

[0030] In order to communicatively couple the nodes 96 to each other, the switch fabric network 90 includes a plurality of switches 92 (e.g., switching devices). The illustrated switch fabric network 90 also includes a plurality of interconnects 94, which may represent wired and / or wireless connections. In one embodiment, the interconnects may include copper cables, optical cables, backplane connectors, and / or the like. The switches 92 may be implemented in combination with the interconnects 94 to couple the devices of the distributed control system 82 and the interactive video system 10 (e.g., sensing units 38, RF sensors 45, input devices 76, audio devices 50, display devices 24, output devices 78, and / or the like) to each other within the switch fabric network 90. ​​For example, in one embodiment, the switch 92 may be a hardware device implemented to route packets (e.g., data packets and / or messages) from a first device (such as a volume sensor 40) to a second device (such as a processing node 84). In addition, with the use of the interconnects 94 and the switches 92, the latency of the switch fabric network 90 may be less than or equal to 1 microsecond (μs). The switch fabric network 90 may also facilitate high bandwidth connectivity. Thus, high volume (eg, throughput) of data may be relayed between devices with low latency, which may facilitate real-time data collection and processing involved with interactive video system 10 .

[0031] The switch fabric network 90 may also facilitate remote direct memory access (RDMA) for both read and write operations between and / or within the nodes 96. Thus, data may be written directly to the memory 102 of the device from any suitable node 96 of the switch fabric network 90. ​​For example, a first node 96 (e.g., a processing node 84) may directly access the memory 102 of a second node 96 (e.g., one of the output nodes 98) to read or update data using a low-level request (e.g., at a network layer or transport layer) without involving a high-level request (e.g., at an application layer or presentation layer) or the additional overhead associated therewith. Furthermore, in some cases, the result that may be calculated at the first node 96 may be written directly to the memory of a slave node 96 (such as a second node 96). Thus, the latency of the switch fabric network 90 and the interactive video system 10 may be minimized (e.g., to less than 5 microseconds (μs), less than 3μs, less than 1μs, less than 0.5μs). Furthermore, the nodes 96 of the switch fabric network 90 may be physically different (e.g., independent) from each other without suffering additional latency. Thus, in some embodiments, each processing node 84 may be housed in a separate chassis 108. For example, a first chassis 108A may include an AI processing node 84A, a second chassis 108B may include a visual processing node 84B, and a third chassis 108C may include a database node 86. In addition, the chassis 108 (e.g., 108A, 108B, 108C) may be remote from each other. For example, the first chassis 108A may be located near the participation area 16 and / or the presentation area 25, while the second chassis 108B may be located at an off-site location remote from the participation area 16 and / or the presentation area 25. Additionally or alternatively, multiple processing nodes 84, input nodes 97, and / or output nodes 98 may be housed in a single chassis 108, or in different chassis located in a common location (e.g., a control area of ​​the interactive video system 10).

[0032] Processing nodes 84 and / or clusters (e.g., groups) of processing nodes 84 may be implemented to handle a certain processing task. Thus, as described above, some of the illustrated processing nodes 84 may include memory 102 and / or processors 104. More specifically, in some embodiments, the processor may be implemented as a central processing unit (CPU) 104A and / or a special-purpose processor, such as a tensor processing unit (TPU) 104B, a graphics processing unit (GPU) 104C, a vector processing unit (VPU) 104D, an application-specific integrated circuit (ASIC) 104E, a field-programmable gate array (FPGA) 104F, and / or the like, which is optimized to perform a processing task.

[0033] Thus, information collected at input node 97 (such as participant data) can be selectively routed to a set of processing nodes 84 that are implemented to perform a task using the information. For example, an artificial intelligence (AI) processing node 84A, which may include a TPU 104B and / or a VPU 104D, can determine the participant data collected at input node 97 by accessing (e.g., reading) information at one of the input nodes 97 via RDMA. In some embodiments, for example, the AI ​​processing node 84A can build a participant model based in part on the collected participant data and a set of inference calculations, predict future actions performed by the participant 12, determine an opponent to be presented to the participant 12, and / or the like. In addition, as described below, the AI ​​processing node 84A can identify the in-game actions of the participant based on the collected participant data, and can determine one or more in-game effects corresponding to the in-game actions. In addition, the visual processing node 84B (which may include a GPU 104C) can determine an output signal so that the display device 24 presents and / or updates the virtual representation 14 of each participant in the virtual environment 32 based in part on the collected participant data and / or in-game effects. Furthermore, audio processing node 84C may determine an output signal based in part on the collected participant data and / or in-game effects to cause audio device 50 to produce a particular audio stimulus. Additionally, physical effect processing node 84D may determine an output signal based in part on the collected participant data and / or in-game effects to cause output device 78 to produce one or more physical effects, such as providing an explosion of air.

[0034] In addition, in some embodiments, the switch fabric network 90 may include a fusion processing node 84E, which may represent a processing node 84 implemented to aggregate data from various sources. For example, in some embodiments, the fusion processing node 84E may aggregate participant data collected from the sensing unit 38 and the RF sensor 45 to determine the location of the participant. In addition, in some embodiments, the fusion processing node 84E may aggregate the output data (such as multiple audio and / or visual signals) before being routed to the output node 98 to be generated by one or more output devices. Therefore, in some embodiments, other processing nodes 84 may receive participant data from the fusion processing node 84E and / or may output data to the output node 98 via the fusion processing node 84E.

[0035] Because nodes 96 can be physically remote from each other, and because the disclosed switch fabric network 90 mitigates latency and supports high data throughput, the interactive video system 10 can be easily expanded. For example, in some embodiments, to accommodate additional participant data that can be generated by additional participants 12, processing can be offloaded to additional processing nodes 84. More specifically, additional processing nodes 84 can be added to the switch fabric network 90 to accommodate the additional participant data. In addition, as described above, any processing node 84 can be located remote from the participation area 16 and / or the presentation area 25. Therefore, the addition of additional processing nodes can be unrestricted by the resources (e.g., area and / or volume) of the participation area 16 and / or the presentation area 25, because another location can accommodate the additional processing node.

[0036] As also illustrated, the switch fabric network 90 may include redundant interconnects 94. For example, certain nodes 96 may be communicatively coupled to a plurality of switches 92 (e.g., switches 92A, 92B, and / or 92C). Additionally or alternatively, switch 92A may be communicatively coupled to a plurality of other switches 92 (e.g., switches 92A, 92B, or 92C). Thus, a plurality of different paths for routing data between switches and / or between nodes 96 may be formed. To this end, the switch fabric network 90 may reduce and / or eliminate undeliverable messages (e.g., data packets) because an alternate routing path may be utilized when a first routing path is unavailable.

[0037] In addition, although the example topology (e.g., layout) of the switching fabric network 90 is illustrated and described herein, it is understood that the switching fabric network 90 can be implemented using any suitable topology. Therefore, in some embodiments, additional or fewer nodes 96 can be included in the switching fabric network 90. ​​For example, as described above, a collection of additional nodes 96 (such as any suitable number of processing nodes 84) can be used to expand the switching fabric network 90 to alleviate the difficulties of a specific data load. Therefore, in some embodiments, the switching fabric network 90 can include any suitable number of visual processing nodes 84B to process higher volumes of visual and / or graphic data. The switching fabric network 90 can include any suitable number of audio processing nodes 84C to process higher volumes of audio information; the switching fabric network 90 can include any suitable number of physical effect processing nodes 84D to process any suitable amount of physical effect data and / or the like. Additionally or alternatively, the interconnectivity of the switching fabric network 90 can be modified. For example, additional or fewer interconnects 94 and / or switches 92 can be included in the switching fabric network 90 to route data between nodes 96. In addition, although the switch fabric network 90 is described herein as an INFINIBAND® network, the switch fabric network 90 can be implemented according to any suitable switch fabric network protocol. For example, in some embodiments, the switch fabric network 90 can be implemented using a high-speed peripheral component interconnect (PCIe) or another suitable high-speed internal communication standard. Therefore, the embodiments described herein are intended to be illustrative rather than limiting.

[0038] According to this technology, Figure 3 Described in and referenced below Figure 1 and / or Figure 2 The elements in FIG. 1 discuss an example of a process 110 for operating the interactive video system 10. It will be appreciated that in other embodiments, certain steps of the illustrated process 110 may be performed in a different order, repeated multiple times, or skipped entirely, in accordance with the present disclosure. In addition, Figure 3 The process 110 illustrated in FIG. 1 may be performed by a node 96 of the switch fabric network 90. ​​More specifically, the process 110 may be performed by a suitable combination of an input node 97, an output node 98, and / or a processing node 84 of the switch fabric network 90.

[0039] The illustrated embodiment of process 110 begins by collecting (block 112) participant data at input node 97. In some embodiments, participants 12 may be scanned or imaged by sensing units 38 positioned around engagement area 16. Additionally or alternatively, collecting participant data may involve determining RF signals at RF sensors 45. For example, in some embodiments, RF sensors may determine a unique identifying RF signal from a wearable device 47, such as a bracelet or headband with a radio frequency identification (RFID) tag worn by each participant 12. In response, RF sensors 45 may provide signals regarding the identity and relative positioning of participants 12 in engagement area 16, which may be captured in the participant data.

[0040] Continuing with the example process 110, next, one or more processing nodes 84 generate (block 114) a corresponding virtual representation for each participant based at least in part on the participant data collected at the input node. For example, in some embodiments, the distributed control system 82 may route the collected scan data from the sensing unit 38 of the input node 97 to the processing node 84 via the switch fabric network 90. ​​The processing node 84 may use the received scan data as a basis to generate a virtual representation of the participant. It will be appreciated that in some embodiments, the virtual representation 14 may have a shape or outline substantially similar to a shadow model of the corresponding participant, which may be generated by the input node (e.g., volume sensor 40) and / or generated at the processing node 84 based in part on the collected participant data. In addition to the shape, the virtual representation 14 may have other attributes that may be modified to correspond to the attributes of the represented participant. For example, participants may be associated with various attributes (e.g., items, status, scores, statistics) that reflect their performance in other gaming systems, their purchases in gift shops, their membership in loyalty programs, and the like. Thus, the properties of the virtual representation (e.g., size, color, texture, animation, presence of virtual items) may be set in response to various properties associated with the corresponding participant, and further modified based on changes in the properties of the participant during game play. For example, in some embodiments, the processing node 84 may request and / or receive additional participant data (such as the above-mentioned properties) from the database node 86 associated with a particular participant based in part on the collected participant data (such as the signals provided by the RF sensor 45). The processing node 84 may then generate a virtual representation of the participant based in part on the additional participant data provided by the database node 86.

[0041] In some embodiments, the distributed control system 82 may use any suitable combination of processing nodes 84 to generate a virtual representation of a participant. For example, as described above, the fusion processing node 84E may aggregate collected participant data from one or more input devices (such as the RF sensor 45 and the sensing unit 38). Based in part on the aggregated collected participant data, the visual processing node 84B may determine an appropriate signal to display a virtual representation via one or more of the output nodes 98 (e.g., at the display device 24). In addition, the database node 86, the AI ​​processing node 84A and / or any additional processing node 84 may be employed to generate a virtual representation and / or provide additional information used by the visual processing node 84B to generate a virtual representation. To this end, as described above, the processing tasks involved in implementing the interactive video system 10 may be distributed among one or more processing nodes 84, which may be dedicated and / or optimized to perform specific processing tasks.

[0042] The illustrated process 110 continues with the interactive video system 10 outputting (e.g., presenting) (block 116) each participant's corresponding virtual representation 14 using the output nodes 98. For example, in some embodiments, the distributed control system 82 may route the virtual representations 14 from the one or more processing nodes 84 to the one or more output nodes 98 via the switch fabric network 90, and the output nodes 98 may display the received virtual representations in the virtual environment 32 on the display device 24. In addition to presenting the virtual representations, in some embodiments, the actions in block 118 may also include presenting other introductory presentations, such as a welcome message or orientation / guidance information, to the participants 12 in the presentation area 25 using the one or more output nodes 98 before the game play begins.

[0043] Once the game play begins, the virtual representations 14 generated in block 114 and presented in block 116 can interact with each other and / or with virtual objects in the virtual environment 32. During the game play, one or more processing nodes 84 typically determine (block 118) in-game actions for each participant 12 in the participation area 16 based on the collected participant data. For example, the in-game actions may include jumping, running, sliding, or otherwise moving the virtual representation 14 within the virtual environment 32. The in-game actions may also include interacting with items (such as virtual objects in the virtual environment 32) (e.g., moving, obtaining, losing, consuming items). Additionally or alternatively, the participants 12 may move, throw, and / or climb on physical props (e.g., balls, toys, climbing structures, and / or the like) that may be included in the participation area 16 and / or purchased at a gift shop to generate in-game actions. The in-game actions may also include completing a goal, defeating another participant, winning a round, or other similar in-game actions. Thus, the collected participant data may include data about the actions of the participants 12 set within the participation area 16. That is, for example, collecting (block 112) participant data may additionally involve collecting data from other devices (e.g., RF sensors 45, input devices 76) to capture the in-game actions of participants 12. Thus, in some embodiments, input nodes 97 may continue to capture participant data during game play.

[0044] In addition, one or more processing nodes 84 may determine in-game effects corresponding to these in-game actions. For example, AI processing node 84A may determine in-game effects that are triggered by and / or associated with in-game actions of participant 12 corresponding to specific game rules. In addition, in some embodiments, certain calculations involved in determining in-game effects may be offloaded to additional processing nodes 84, such as additional AI processing nodes 84A. In addition, in some embodiments, distributed control system 82 may determine appropriate signals to set or modify parameters of virtual environment 32 within presentation area 25 based in part on the in-game actions of the participants. For example, as described in more detail below, modifications may include adjusting house light brightness and / or color, playing game music or game sound effects, adjusting the temperature of a participation area, activating physical effects in a participation area, and the like.

[0045] The distributed control system 82 typically uses the output nodes 98 to update (block 120) the corresponding virtual representations 14 of the participants 12 and / or the virtual environment 32 based on the in-game actions of the participants 12 in the participation area 16 and the corresponding in-game effects determined in block 118. For example, the distributed control system 82 may use RDMA to directly update data stored in a memory of a first node (e.g., at one or more output nodes 98) based on data stored in a memory of a second node (e.g., at a visual processing node 84B, an audio processing node 84C, a physical effect processing node 84D, an AI processing node 84A, and / or the like). The output nodes 98 may then supply appropriate control signals to the associated output devices (e.g., an audio device 50, a display device 24, an output device 78) to update the virtual environment 32 (e.g., update the presentation in the presentation area 25) based on the received signals. As indicated by arrow 122, one or more steps included in process 110 (e.g., block 118 and / or block 120) may be repeated until the game play is completed, e.g., due to one of participants 12 winning the round of game play or due to expiration of an allotted game play time.

[0046] Figure 4 According to the present technology, Figure 3 An example embodiment of a process 140 for determining an in-game effect corresponding to an in-game action of each of the participants 12 in block 118 of FIG. Figure 1 and Figure 2 The process 140 is described with reference to the elements of FIG. 1 . Furthermore, it is to be understood that in other embodiments, certain steps of the illustrated process 140 may be performed in a different order, repeated multiple times, or skipped entirely, in accordance with the present disclosure. In general, the process 140 includes identifying in-game actions of participants, determining in-game effects based on these identified in-game actions, and updating the presentation of the virtual environment in the participating area based on the determined in-game effects. Furthermore, although Figure 4 The process 140 illustrated in is described herein as being performed by a particular node of the distributed control system 82 (eg, one or more processing nodes 84 ), but the process 140 may be performed by any suitable combination of nodes 96 of the distributed control system 82 .

[0047] The illustrated process 140 begins with one or more of the processing nodes 84 (e.g., AI processing node 84A) of the distributed control system 82 identifying (block 142) one or more in-game actions of the corresponding virtual representation 14 of each participant 12 based at least in part on the collected participant data. For example, the in-game action may include jumping, running, sliding, or otherwise moving the virtual representation 14 within the virtual environment 32. Thus, one or more of the processing nodes 84 may identify the in-game action based in part on data received from an input node 97 (e.g., from a sensing unit 38, an RF sensor 45, an input device 76, and / or the like) that is implemented to capture the in-game action of the participant 12.

[0048] Next, the illustrated process 140 continues, where one or more of the processing nodes 84 (e.g., AI processing node 84A) determines (block 144) one or more in-game effects triggered in response to the identified in-game action of each of the participants 12. For example, when the determined in-game action is movement of the participant, then the in-game effect may be a corresponding change in the positioning of the corresponding virtual representation within the virtual environment. When the determined in-game action is jumping, the in-game effect may include moving the virtual representation along the y-axis 20, such as Figure 1 . When the determined in-game action is activating a particular power-up item, the in-game effect may include modifying a state associated with the participant 12 (e.g., health state, power state). When the determined in-game action associated with the participant 12A is the use of a physical prop (such as a foam sword) within the participation area 16, the in-game effect may include updating the virtual representation of the participant 12A in the virtual environment 32 to include a virtual representation corresponding to the physical prop (such as a virtual sword). In addition, in some cases, the movement of the virtual representation 14 may be emphasized or enhanced relative to the actual movement of the participant 12. For example, as discussed above with respect to modifying the appearance of the virtual representation, the movement of the virtual representation of the participant may be temporarily or permanently exaggerated (e.g., able to jump higher, able to jump farther) relative to the actual movement of the participant based on attributes associated with the participant (including items acquired during game play, items acquired during other game play sessions, items purchased in a gift shop, etc.). In addition, the in-game effect may include one or more sound and / or physical effects.

[0049] Continue through Figure 4, the distributed control system 82 may determine appropriate signals corresponding to one or more sounds (block 146), visual effects (block 148), and / or one or more physical effects (block 150) in the rendering area 25 based at least in part on the in-game effects determined in block 144. For example, when the in-game effect is determined to be a particular virtual representation of a participant falling into a virtual swimming pool, the audio processing node 84C may determine appropriate signals to cause an audio device (e.g., audio device 50) associated with one or more of the output nodes 98 to generate an appropriate splash sound. In addition, the visual processing node 84B may determine a visual effect to be presented at one or more of the output nodes 98 (e.g., at the display device 24), such as an updated visual representation of the participant. In some embodiments, the visual processing node 84B may utilize multiple GPUs 104C that may be adjacent to each other or physically distant from each other to determine the visual effect. In addition, the physical effect processing node 84D may determine appropriate signals to cause the output node 98 and the associated output device 78 to generate a blast of fog. Additionally, sounds and / or physical effects may be generated in response to any number of in-game effects, including, for example, gaining a power-up, losing a power-up, scoring a point, or moving through a particular type of environment.

[0050] The technical effects of the present method include an interactive video system that enables multiple participants (e.g., two or more, four or more) to perform actions in a physical participation area to control corresponding virtual representations in a virtual environment presented on a display device in a presentation area near the participation area. More specifically, the system can create a real-time interactive experience that is smooth and responsive to participant interactions, and the system can also be efficiently expanded to facilitate a larger number of participants, higher data processing throughput, and / or the like. To this end, the disclosed system includes a distributed control system implemented in a switching fabric network. The switching fabric network interconnects multiple nodes, such as input nodes (e.g., input devices), output nodes (e.g., output devices), and processing nodes (e.g., computing devices). In addition, the switching fabric network can facilitate low latency, high bandwidth communications and remote direct memory access (RDMA) between its nodes. As a result of low latency communications, and because the processing nodes can include dedicated processors optimized to perform specific computing tasks, the operations involved in the control system can be distributed across multiple processing nodes and / or dedicated processors for faster processing. Furthermore, because the switched fabric network can house processing nodes in physically distinct locations (eg, in separate chassis), additional processing nodes can be efficiently added to the network either on-site or off-site from the system, thereby increasing the scalability of the system.

[0051] Although only certain features of the present technology have been illustrated and described herein, many modifications and changes will occur to those skilled in the art. Therefore, it is to be understood that the appended claims are intended to cover all such modifications and changes as fall within the true spirit of the present technology. In addition, the technology presented and claimed herein is cited and applied to material objects and specific examples of actual nature, which provably improve the present technical field, and is therefore not abstract, intangible or purely theoretical. In addition, if any claim attached to the end of this specification contains one or more elements designated as "means for [performing] ... [function]" or "steps for [performing] ... [function]", it is intended to interpret such elements according to 35 USC112 (f). However, for any claim containing elements designated in any other way, it is intended not to interpret such elements according to 35 USC112 (f).

Claims

1. A distributed control system for an interactive video system, comprising: A switch fabric network interconnecting a plurality of nodes, wherein the switch fabric network comprises: an input node among the plurality of nodes, wherein the input node comprises a sensing unit configured to collect scan data of a participant of the interactive video system; a first set of processing nodes among the plurality of nodes, wherein the first set of processing nodes is configured to generate a virtual representation of the participant based at least in part on the scan data of the participant, wherein the input node is communicatively coupled to the first set of processing nodes via a first switch of the switched fabric network; and an output node of the plurality of nodes, wherein the output node comprises an output device configured to present the virtual representation of the participant in a virtual environment, wherein the output node is communicatively coupled to the input node via a second switch of the switched fabric network.

2. The distributed control system of claim 1, wherein the switch fabric network is implemented using Peripheral Component Interconnect Express connectivity.

3. A distributed control system according to claim 1, wherein the switching fabric network includes a second set of processing nodes among the plurality of nodes, wherein the second set of processing nodes is configured to determine one or more in-game effects for the participant based at least in part on the scan data of the participant. 4 . The distributed control system of claim 3 , wherein the output node is configured to update the virtual environment based at least in part on the one or more in-game effects determined by the second set of processing nodes.

5. The distributed control system of claim 3, wherein the first set of processing nodes is disposed in a first chassis, and the second set of processing nodes is disposed in a second chassis different from the first chassis.

6. The distributed control system of claim 1, wherein the first set of processing nodes comprises a central processing unit, a tensor processing unit, a graphics processing unit, a vector processing unit, an application specific integrated circuit, a field programmable gate array, or a combination thereof.

7. The distributed control system of claim 1, wherein the first set of processing nodes are configured to access the input nodes via remote direct memory access.

8. The distributed control system of claim 1, wherein the scan data includes a participant model, in-game participant motion data, participant identification, or a combination thereof.

9. A distributed control system according to claim 1, wherein the sensing unit includes a depth camera, a light detection and ranging device, or a combination thereof, and wherein the output device includes a display device configured to display the virtual environment to the participant.

10. A distributed control system according to claim 1, wherein the sensing unit is configured to collect the scan data of the participant by measuring the outer surface of the participant, and wherein the output device is configured to present the virtual representation of the participant by displaying the virtual representation in the virtual environment.

11. A method of operating an interactive video system, comprising: collecting scan data of a participant of the interactive video system at an input node of a plurality of nodes in a switched fabric network of a distributed control system of the interactive video system; generating, at a first set of processing nodes among the plurality of nodes, a virtual representation of the participant based at least in part on the scan data of the participant, wherein the first set of processing nodes is communicatively coupled to the input node via a first switch of the switch fabric network; presenting the virtual representation of the participant in a virtual environment using an output node of the plurality of nodes, wherein the output node is communicatively coupled to the input node via a second switch of the switched fabric network; determining, at a second set of processing nodes among the plurality of nodes, a set of in-game effects based at least in part on the scan data of the participant; as well as The virtual environment is updated based at least in part on the set of in-game effects using the output node.

12. The method of claim 11, wherein determining a set of in-game effects based at least in part on the scan data of the participant comprises: identifying, at a first processing node in the second set of processing nodes, an in-game action of the participant based at least in part on the scan data of the participant; determining, at a second processing node in the second set of processing nodes, an in-game effect corresponding to the action in the game; determining, at a third processing node in the second set of processing nodes, an audio effect corresponding to the in-game effect; determining, at a fourth processing node in the second set of processing nodes, a visual effect corresponding to an effect in the game; as well as A physical effect corresponding to the in-game effect is determined at a fifth processing node in the second set of processing nodes.

13. The method according to claim 11, comprising: collecting additional scan data of the participant at the input node; aggregating the additional scan data with the scan data at a third set of processing nodes among the plurality of nodes to generate aggregated scan data for the participant; as well as The virtual representation of the participant is generated at the output node based at least in part on the aggregated scan data of the participant.

14. The method according to claim 11, comprising: receiving, at the first set of processing nodes, additional scan data of the participant from a database node among the plurality of nodes; as well as The virtual representation of the participant is generated at the first set of processing nodes based at least in part on the scan data of the participant and the additional scan data.

15. An interactive video system comprising: A distributed control system comprising a switch fabric network interconnecting a plurality of nodes, wherein the plurality of nodes comprises: an input node comprising a sensing unit configured to collect scan data of a participant of the interactive video system; a set of processing nodes communicatively coupled to the input node via the switch fabric network, wherein the set of processing nodes is configured to: generating a virtual representation of the participant based at least in part on the scanned data of the participant; and determining a set of in-game effects based on the scan data of the participant, wherein the scan data includes one or more in-game actions of the participant; and an output node communicatively coupled to the input node and the set of processing nodes via the switch fabric network, wherein the output node comprises a set of output devices configured to: presenting the virtual representation of the participant in a virtual environment; and The virtual environment is updated based at least in part on the set of in-game effects.

16. The interactive video system of claim 15, wherein the set of in-game effects comprises visual effects, physical effects, audio effects, or a combination thereof in the virtual environment.

17. The interactive video system of claim 15, wherein the input node comprises an interface panel, wherein the interface panel comprises a plurality of input devices configured to receive input from the participants corresponding to the one or more in-game actions.

18. The interactive video system of claim 15, wherein the input node comprises a radio frequency sensor, and wherein the scan data comprises data indicative of an identity, a location, or a combination thereof collected for the participant at the radio frequency sensor.

19. The interactive video system of claim 15, wherein the set of processing nodes is communicatively coupled to the input node via a first switch of the switch fabric network, and wherein the output node is communicatively coupled to the input node via a second switch and to the set of processing nodes via a third switch of the switch fabric network.

20. The interactive video system of claim 15, wherein the plurality of nodes comprises an additional input node comprising an additional sensing unit configured to collect corresponding scan data for each of a plurality of participants of the interactive video system.

21. An interactive video system, comprising: A switched fabric network comprising a plurality of switches interconnecting a plurality of nodes, wherein the plurality of nodes comprises: at least one input node including a sensing unit configured to collect scan data of a participant in a participation area of ​​the interactive video system; at least one artificial intelligence processing node configured to recognize an in-game action of the participant based at least in part on the scan data of the participant and determine an in-game effect corresponding to the in-game action; at least one audio processing node configured to determine an audio effect corresponding to an effect in the game; and at least one audio output node comprising an audio output device configured to present the audio effect to the participant in the participation area of ​​the interactive video system, wherein each of the plurality of nodes is coupled to at least two of the plurality of switches of the switched fabric network.

22. The interactive video system of claim 21, wherein the plurality of nodes comprises: at least one visual processing node configured to generate a virtual representation of the participant based at least in part on the scanned data of the participant; as well as At least one visual output node includes a visual output device configured to present the virtual representation to the participant in the participation area of ​​the interactive video system.

23. An interactive video system according to claim 22, wherein the at least one visual processing node is configured to determine a visual effect corresponding to an effect in the game, and wherein the at least one visual output node is configured to update the presentation of the virtual representation of the participant on the visual output device based on the visual effect.

24. The interactive video system of claim 21, wherein the plurality of nodes comprises: at least one physics effect processing node configured to determine a physics effect corresponding to an effect in the game; as well as At least one physical effect output node includes a physical effect output device configured to present the physical effect to the participant in the participation area of ​​the interactive video system.

25. The interactive video system of claim 24, wherein the at least one physical effects processing node comprises a field programmable gate array.

26. The interactive video system of claim 21, wherein the plurality of nodes comprises: a plurality of input nodes, each comprising a respective sensing unit configured to collect respective portions of the scan data of the participants in the participation area of ​​the interactive video system; as well as An input fusion processing node is configured to aggregate the respective portion of the scan data collected by each respective sensing unit of the plurality of input nodes, and wherein the input fusion processing node comprises an application specific integrated circuit.

27. The interactive video system of claim 21, wherein the plurality of nodes comprises: a plurality of audio output nodes, each comprising a respective audio output device configured to present a respective portion of the audio effect to the participant in the participation area of ​​the interactive video system; as well as An output blending processing node is configured to determine the respective portion of the audio effect to be rendered by the respective audio output device of each of the plurality of audio output nodes.

28. The interactive video system of claim 21, wherein the at least one artificial intelligence processing node is disposed in a first chassis, and the at least one audio processing node is disposed in a second chassis different from the first chassis.

29. An interactive video system according to claim 21, wherein the at least one artificial intelligence processing node comprises a tensor processing unit, a first vector processing unit, or a combination thereof, and wherein the at least one audio processing node comprises a second vector processing unit different from the first vector processing unit.

30. The interactive video system of claim 21, wherein the at least one artificial intelligence processing node is configured to access the at least one input node via remote direct memory access.

31. The interactive video system of claim 21, wherein the switched fabric network is implemented using Peripheral Component Interconnect Express connectivity.

32. A method of operating an interactive video system having a switched fabric network including a plurality of nodes, the method comprising: collecting scan data of participants in a participation area of ​​the interactive video system via at least one input node of the plurality of nodes; identifying, via at least one artificial intelligence processing node among the plurality of nodes, an in-game action of the participant based at least in part on the scanned data of the participant; determining, via the at least one artificial intelligence processing node, an in-game effect corresponding to an action in the game; determining, via at least one audio processing node of the plurality of nodes, an audio effect corresponding to an effect in the game; presenting the audio effect to the participant in the participation area of ​​the interactive video system via an audio output device of at least one audio output node of the plurality of nodes; determining, via at least one physical effect processing node among the plurality of nodes, a physical effect corresponding to an effect in the game; as well as The physical effect is presented to the participant in the participation area of ​​the interactive video system via a physical effect output device of at least one physical effect output node among the plurality of nodes.

33. The method of claim 32, wherein each of the plurality of nodes is coupled to at least two of a plurality of switches of the switched fabric network.

34. The method of claim 32, comprising: generating, via at least one vision processing node of the plurality of nodes, a virtual representation of the participant based at least in part on the scanned data of the participant; as well as The virtual representation is presented to the participant in the participation area of ​​the interactive video system via a visual output device of a visual output node of the plurality of nodes.

35. The method of claim 34, comprising: determining, via the at least one visual processing node, a visual effect corresponding to an effect in the game; as well as The virtual representation of the participant is updated via the visual output device of the visual output node based on the visual effect.

36. An interactive video system comprising: A switching fabric network having a plurality of nodes, wherein the plurality of nodes include: at least one input node including a sensing unit configured to collect scan data of participants in a participation area of ​​the interactive video system; at least one artificial intelligence processing node configured to recognize an in-game action of the participant based at least in part on the scan data of the participant and determine an in-game effect corresponding to the in-game action; at least one visual processing node configured to generate a virtual representation of the participant based at least in part on the scanned data of the participant and determine visual effects corresponding to effects in the game; and at least one visual output node comprising a visual output device configured to present the virtual representation to the participant in the participation area of ​​the interactive video system and to update the presentation of the virtual representation based on the visual effect; at least one audio processing node configured to determine an audio effect corresponding to an effect in the game; and At least one audio output node includes an audio output device configured to present the audio effect to the participant in the participation area of ​​the interactive video system.

37. The interactive video system of claim 36, wherein each of the plurality of nodes is coupled to at least two of a plurality of switches of the switched fabric network.

38. The interactive video system of claim 36, wherein the plurality of nodes comprises: at least one physics effect processing node configured to determine a physics effect corresponding to an effect in the game; as well as At least one physical effect output node includes a physical effect output device configured to present the physical effect to the participant in the participation area of ​​the interactive video system.

39. The interactive video system of claim 36, wherein the at least one input node comprises a radio frequency sensor, and wherein the scan data comprises data indicative of an identity, a location, or a combination thereof collected for the participant at the radio frequency sensor.

40. The interactive video system of claim 36, wherein the at least one input node comprises a plurality of input nodes having a corresponding plurality of sensing units configured to collect corresponding scan data for each of a plurality of participants within the engagement area of ​​the interactive video system.

Images