Capturing and visualizing live venue performances via a game network.

Abstract

The systems, methods, and apparatus disclosed herein can incorporate one or more real-world performers performing in a venue into interactive content. These systems, methods, and apparatus can identify one or more joints or ligaments of one or more real-world performers, for example, the left shoulder, the right knee, etc., from an image or series of images of one or more real-world performers. These joints or ligaments can be represented as one or more performer markers. These systems, methods, and apparatus can generate one or more three-dimensional models of one or more real-world performers in three-dimensional space from one or more performer markers.

Description

Background Art

[0001] Taking some examples, a live performance such as a concert or a play is typically a one-way interaction in which one or more real-world performers appear for the audience. In the conventional audience / performer relationship, the interaction between the performer and the audience only flows from the performer to the audience. Even when interactions from the audience to the performer occur, these are typically minimal interactions among a part of the audience. They can include the audience chanting in response to the demands of the performance, the audience singing the lyrics in tune with the song being performed, the audience holding up lighting lights or glow sticks to illuminate the venue, the audience clapping in response to the performance, filling out a questionnaire following the show, etc. Regarding the audience who selects to watch a live performance, they consume more but are never involved so frequently, which keeps the value of attending a live performance in a more desirable state.

Summary of the Invention

Means for Solving the Problems

[0002] Overview The systems, methods, and apparatus disclosed herein can incorporate one or more real-world performers performing in a venue into interactive content. These systems, methods, and apparatus can identify one or more joints or ligaments of one or more real-world performers, for example, the left shoulder, the right knee, etc., from an image or series of images of one or more real-world performers. These joints or ligaments can be represented as one or more performer markers, as will be described in more detail below. These systems, methods, and apparatus can generate one or more three-dimensional models of one or more real-world performers in three-dimensional space from one or more performer markers. These systems, methods, and apparatus can apply one or more three-dimensional models of one or more real-world performers to one or more virtual characters in three-dimensional space. These systems, methods, and apparatus can render one or more virtual characters from three-dimensional space into the two-dimensional space of interactive content, thereby integrating one or more real-world performers into the interactive content. [Brief explanation of the drawing]

[0003] This disclosure is described with reference to the accompanying drawings. In the drawings, similar reference numbers indicate the same or functionally similar elements. In addition, the leftmost digit of the reference number identifies the drawing in which the reference number first appears.

[0004] [Figure 1] Figure 1 graphically illustrates an exemplary venue for integrating exemplary performers from several exemplary embodiments of this disclosure into exemplary interactive content.

[0005] [Figure 2] Figure 2 graphically illustrates exemplary motion control flows that may be implemented within an exemplary venue to integrate exemplary performers from several exemplary embodiments of the present disclosure into exemplary interactive content.

[0006] [Figure 3] Figure 3 graphically illustrates another exemplary behavior control flow that may be implemented within an exemplary venue to integrate exemplary performers from some exemplary embodiments of the present disclosure into exemplary interactive content.

[0007] [Figure 4] Figure 4 illustrates a simplified block diagram of an exemplary computer system that may be implemented within an exemplary model processing system according to some exemplary embodiments of the present disclosure.

[0008] This disclosure will be described herein with reference to the accompanying drawings. [Modes for carrying out the invention]

[0009] Detailed explanation The following disclosure provides many different embodiments or examples for implementing different features of the subject matter provided. Specific examples of components and arrangements are described below for the sake of simplicity of this disclosure. These are, of course, examples only and are not intended to be limiting. Aspects of this disclosure will be best understood from the following detailed description, when read carefully together with the accompanying figures. This disclosure may repeat reference numbers and / or letters in various embodiments. This repetition does not in itself determine the relationships between the various embodiments and / or configurations discussed. Note that, in accordance with standard practice in this industry, features are not drawn to scale. In fact, the dimensions of features may be enlarged or reduced as appropriate for the sake of clarity of discussion.

[0010] Exemplary venues for integrating exemplary performers into exemplary interactive content. Figure 1 graphically illustrates exemplary venues for integrating exemplary performers into exemplary interactive content, according to some exemplary embodiments of the present disclosure. In the exemplary embodiments illustrated in Figure 1, venue 100 can enable interactive content for members of the audience within venue 100. As will be described in more detail below, venue 100 can incorporate one or more real-world performers performing within venue 100 into the interactive content. Generally, venue 100 represents a place for hosting an event. For example, venue 100 can represent a music venue, e.g., a music theater, a music club, and / or a concert hall; a sports venue, e.g., an arena, a convention center, and / or a stadium, and / or any other suitable venue that will be obvious to those skilled in the art without departing from the spirit and scope of the present disclosure. An event can represent a music event, a theatrical event, a sporting event, a film, and / or any other suitable event that will be obvious to those skilled in the art without departing from the spirit and scope of the present disclosure. In the exemplary embodiment illustrated in Figure 1, the venue 100 can present interactive content 102 to members of the audience 104 within the venue 100.

[0011] In the exemplary embodiment illustrated in Figure 1, the interactive content 102 may request participation from members of the audience 104 to perform actions related to visual-hand coordination and / or reaction time, for example. In some embodiments, the interactive content 102 may include one or more genres of video games, such as action video games, adventure video games, fighting video games, platform video games, puzzle video games, racing video games, role-playing video games, rhythm video games, shooting video games, simulation video games, sports video games, strategy video games, and / or any other suitable genre of video game, which will be obvious to those skilled in the art without departing from the spirit and scope of this disclosure. Alternatively, or in addition, the interactive content 102 may include other interactive content, such as interactive questionnaires and / or interactive surveys, to give some embodiments, to obtain various information from members of the audience 104. As will be described in more detail below, the interactive content 102 may include one or more virtual characters 106, such as one or more player characters, non-player characters, avatars, or equivalents, to give a few examples, which emulate one or more real-world performers 108 in the venue 100. As will be described in more detail below, the venue 100 integrates one or more real-world performers 108 as one or more virtual characters 106 into the interactive content 102. For example, if one or more real-world performers 108 wave to members 104 of the audience in the venue 100, one or more virtual characters 106 will similarly appear to wave to members 104 of the audience in the interactive content 102.

[0012] In the exemplary embodiment illustrated in Figure 1, the venue 100 can capture the movement of one or more real-world performers 108 within the venue 100 through motion capture. In some embodiments, the venue 100 can implement a markerless approach for capturing the movement of one or more real-world performers 108. In these embodiments, the markerless approach can utilize computer vision and / or deep learning techniques, which will be obvious to those skilled in the art without departing from the spirit and scope of this disclosure, for example, to identify and / or analyze one or more real-world performers 108 from images or a series of images visualized as a video. In these embodiments, the venue 100 can capture the movement of one or more real-world performers 108 within the venue 100 from images or a series of images of one or more real-world performers 108. In some embodiments, the venue 100 can capture the movement of one or more real-world performers 108 in real time or near real time. As illustrated in Figure 1, the venue 100 may include one or more cameras 110, a content server 112, and / or portable electronic devices 114. One or more cameras 110 may capture images or a series of images of one or more real-world performers 108. In some embodiments, one or more cameras 110 may include one or more ultra-wide-angle lenses having a field of view of about 100(100) to about 180(180) degrees, which will be recognized by those skilled in the art without departing from the spirit and scope of this disclosure. In some embodiments, one or more cameras 110 may be implemented as one or more television studio cameras that can be erected on the floor of the venue 100, for example, with a pneumatic or hydraulic mechanism that can adjust the height and / or position of one or more cameras 110 within the venue 100. In these embodiments, the content server 112 may include a camera control unit (CCU) to adjust the height and / or position of one or more cameras 110 in real time or near real time in order to track one or more real-world performers 108.In some embodiments, the content server 112 may utilize, for example, computer vision to identify and / or analyze one or more real-world performers 108 from an image or a series of images. In these embodiments, the content server 112 may provide feedback on the location of one or more real-world performers 108 within the venue 100, enabling the CCU to track one or more real-world performers 108 within the venue 100. In these embodiments, the CCU may track one or more real-world performers 108 to a central location within an image or a series of images and constrain one or more virtual characters 106 to a fixed location within the interactive content 102.

[0013] The content server 112 represents one or more computer systems, the exemplary embodiments of which are described in further detail below, and the one or more computer systems facilitate the operation of the venue 100. In some embodiments, the content server 112 can be implemented in hardware, firmware, software, or any combination thereof. Furthermore, firmware, software, routines, instructions, and / or applications can be described herein as performing certain actions. However, it should be understood that such descriptions are for convenience only, and that these actions result from a computing device, processor, controller, or other device running the firmware, software, routines, instructions, and / or applications. As illustrated in Figure 1, the content server 112 can enable interactive content 102 for a portable electronic device 114 for presentation to an audience 104. As will be described in further detail below, the content server 112 can incorporate one or more real-world performers 108 as one or more virtual characters 106 into the interactive content 102. In some embodiments, the content server 112 can capture the movement of one or more real-world performers 108 within the venue 100. In these embodiments, the content server 112 can identify one or more joints or ligaments of one or more real-world performers 108, for example, the left shoulder, the right knee, etc., from images or a series of images of one or more real-world performers 108 captured by one or more cameras 110, as described above. These joints or ligaments can be represented as one or more performer markers. From the above embodiments, the content server 112 can identify one or more joints or ligaments of one or more real-world performers 108 waving to members of the audience 104 within the venue 100.

[0014] In some embodiments, the content server 112 can generate one or more 3D models of one or more real-world performers 108 in 3D space, and one or more 3D models emulate the movement of one or more real-world performers 108 within the venue 100. In these embodiments, the content server 112 can generate one or more 3D models of one or more real-world performers 108 in 3D space from one or more performer markers. From the above embodiments, the content server 112 can generate one or more 3D models waving in a manner substantially similar to one or more real-world performers 108 from one or more performer markers.

[0015] In some embodiments, the content server 112 can apply one or more 3D models of one or more real-world performers 108 to one or more virtual characters 106 in 3D space. In these embodiments, one or more virtual characters 106 emulate the movement of one or more real-world performers 108 within the venue 100. From the above embodiments, the content server 112 can apply 3D models of one or more real-world performers 108 to one or more virtual characters 106 in 3D space and make one or more virtual characters 106 wave their hands in the same way as one or more real-world performers 108. In some embodiments, the content server 112 can render one or more virtual characters 106 from 3D space into the 2D space of interactive content 102 and integrate one or more real-world performers 108 into the interactive content 102.

[0016] In the exemplary embodiment illustrated in Figure 1, a portable electronic device 114 can execute interactive content 102 having one or more virtual characters 106. In some embodiments, the portable electronic device 114 can receive one or more virtual characters 106 in the two-dimensional space of the interactive content 102 from a content server 112. In these embodiments, the portable electronic device 114 can incorporate one or more virtual characters 106 in the two-dimensional space of the interactive content 102 into the interactive content 102 and integrate one or more real-world performers 108 into the interactive content 102. In some embodiments, the portable electronic device 114 can be operated by members 106 of an audience to interact with the interactive content 102. In some embodiments, the portable electronic device 114 may include one or more mobile phone devices such as mobile phones, mobile computing devices, tablet computers and / or laptop computers, mobile internet devices such as game controllers, and / or other suitable mechanical, electrical, or electromechanical input devices, which will be recognized by those skilled in the art without departing from the spirit and scope of this disclosure. In these embodiments, the portable electronic device 114 may include one or more buttons, one or more omnidirectional control sticks or buttons, and / or one or more touch screen interfaces that can be operated by a member of the audience 106 to interact with the interactive content 102.

[0017] Exemplary behavior control flow for integrating exemplary performers into exemplary interactive content Figure 2 graphically illustrates exemplary motion control flows that may be implemented in an exemplary venue to integrate exemplary performers into exemplary interactive content, according to some exemplary embodiments of the present disclosure. The following discussion describes an exemplary motion control flow 200 for integrating performers from the venue, such as one or more real-world performers 108, into interactive content, such as interactive content 102, in one embodiment. The present disclosure is not limited to these exemplary motion control flows. Rather, it will be apparent to those skilled in the art that other motion control flows are also within the scope and spirit of the present disclosure. The motion control 200, which will be described in more detail below, represents one or more modeling tools that, when executed by one or more computer systems, such as a content server 112, as described above, can incorporate performers from the venue into the interactive content. In some embodiments, one or more modeling tools may represent one or more software tools, such as 3D modeling, animation, simulation, and / or rendering software tools, which may be executed by the modeling processing server 102 to integrate performers from the venue into interactive content, as will be described in more detail below.

[0018] In operation 202, the motion control flow 200 can identify one or more joints or ligaments of a real-world performer 206, for example, the left shoulder, the right knee, etc., in particular. As illustrated in Figure 2, these joints or ligaments can be represented as one or more performer markers 204. In some embodiments, the motion control flow 200 can identify one or more joints or ligaments from an image or series of images of the real-world performer 206 in a manner substantially similar to those described above. In these embodiments, the motion control flow 200 can capture an image or series of images of the real-world performer 206 in real time or near real time as the real-world performer 206 is performing in the venue. In these embodiments, the motion control flow 200 can identify one or more joints or ligaments from an image or series of images of the real-world performer 206 using, for example, any preferred image processing, computer vision, and / or deep learning techniques that will be apparent to those skilled in the art without departing from the spirit and scope of this disclosure. In some embodiments, the motion control flow 200 can identify and track the movement of one or more performer markers 204 from an image or a series of images of the real-world performer 206 as the real-world performer 206 moves around the venue.

[0019] In operation 208, the motion control flow 200 can generate a three-dimensional model 210 of a real-world performer 206 in three-dimensional space from one or more performer markers 204. In some embodiments, the three-dimensional model 210 can be implemented using a simple three-dimensional stick model, however, more complex three-dimensional models such as wireframe models, polygonal models, solid models, and / or surface models are also conceivable, as will be apparent to those skilled in the art without departing from the spirit and scope of this disclosure. In some embodiments, the motion control flow 200 can generate the three-dimensional model 210 for movement in three-dimensional space in a manner substantially similar to that of the real-world performer 206.

[0020] In operation 212, the motion control flow 200 can apply the 3D model 210 to a virtual character 214 in 3D space, where the virtual character 214 is, in one embodiment, a virtual bear, etc. In some embodiments, the motion control flow 200 can access the 3D model of the virtual character 214 in 3D space. In these embodiments, the motion control flow 200 can map one or more performer markers 204 of the 3D model 210, for example, the left shoulder, the right knee, onto corresponding character markers 216 of the 3D model of the virtual character 214, for example, the left shoulder, the right knee, etc. In these embodiments, the 3D model of the virtual character 214 can be implemented using a simple 3D stick model, however, more complex 3D models such as wireframe models, polygonal models, solid models, and / or surface models are also possible, as will be apparent to those skilled in the art without departing from the spirit and scope of this disclosure. In some embodiments, this application of the 3D model 210 to the virtual character 214 allows the virtual character 214 to move in 3D space in a manner substantially similar to that of the real-world performer 206. In some embodiments, the motion control flow 200 can graphically render the virtual character 214 from 3D space into the 2D space of the interactive content and integrate the real-world performer 206 into the interactive content.

[0021] FIG. 3 graphically illustrates another exemplary operation control flow that may be implemented within an exemplary venue to integrate exemplary performance into exemplary bidirectional content according to some exemplary embodiments of the present disclosure. The following discussion is for explaining an exemplary operation control flow 300 for integrating a performance from a venue (e.g., one or more persons among one or more real-world performances 108) into bidirectional content (e.g., bidirectional content 102). The present disclosure is not limited to these exemplary operation control flows. Rather, other operation control flows will be apparent to those skilled in the art within the scope and spirit of the present disclosure. The operation control 300, as will be described in more detail below, is executed by a portable electronic device such as one or more of the portable electronic devices 114 described above, and can integrate a performance from the venue into bidirectional content.

[0022] In operation 302, the operation control flow 300 can install the bidirectional content on the portable electronic device. In some embodiments, the portable electronic device can download application software, which has the bidirectional content and is often referred to as an app. In these embodiments, the portable electronic device can download the application software from a content server within the venue, such as content server 112, as described above. Alternatively, or in addition, the portable electronic device can download the application software from a digital distribution platform, also referred to as an app store or app marketplace. After the application software is downloaded by the portable electronic device, in some embodiments, the portable electronic device can install the application software.

[0023] In operation 304, the operation control flow 300 can execute interactive content installed on a portable electronic device. In some embodiments, the operation control flow 300 can receive virtual characters, such as one or more of one or more virtual characters 106, which are rendered on the two-dimensional space of the interactive content in a manner substantially similar to that described above. In these embodiments, the operation control flow 300 can incorporate the virtual characters in the two-dimensional space of the interactive content into the interactive content and integrate a real-world performer into the interactive content. In these embodiments, the portable electronic device may include one or more buttons, one or more omnidirectional control sticks or buttons, and / or one or more touchscreen interfaces that can be operated by members of the audience to interact with the interactive content. For example, the interactive content may be contained within a video game. In this embodiment, a real-world performer can aim at different locations in the venue, such as pointing a weapon at an enemy, as part of the video game. In this embodiment, a member of the audience can operate one or more buttons, one or more omnidirectional control sticks or buttons, and / or one or more touchscreen interfaces to fire a weapon at an enemy. Thus, real-world performers can assist in guiding or aiming weapons at enemies, and members of the audience can fire weapons at enemies within the video game. In another embodiment, interactive content can be incorporated within other forms of interactive applications and / or non-interactive forms of visualization, such as voting systems or navigation.

[0024] Exemplary computer systems that can be implemented within exemplary model processing systems FIG. 4 illustrates a simplified block diagram of an exemplary computer system that may be implemented within an exemplary model processing system according to some illustrative embodiments of the present disclosure. The following discussion of FIG. 4 is for the purpose of explaining a computer system 400 that may be used to implement a content server 112 as described above.

[0025] In the exemplary embodiment illustrated in Figure 4, the computer system 400 includes one or more processors 402. In some embodiments, one or more processors 402 may include, or be, any of the following: a microprocessor, a graphics processing unit, or a digital signal processor, and their electronic equivalents, such as an application-specific integrated circuit ("ASIC") or a field-programmable gate array ("FPGA"). As used herein, the term “processor” typically refers to a tangible data and information processing device that physically transforms data and information using sequence transformations (also referred to as “operations”). Data and information may be physically represented by electrical, magnetic, optical, or acoustic signals that can be stored, accessed, transferred, combined, compared, or otherwise manipulated by the processor. The term “processor” may refer to a single processor and a multicore system or multiprocessor array including a graphics processing unit, a digital signal processor, a digital processor, or a combination of these elements. A processor may be an electronic device comprising, for example, a digital logic network (e.g., binary logic) or analog (e.g., an operational amplifier). Processors may also operate within a "cloud computing" environment or as "software as a service" (SaaS) to support the performance of related operations. For example, at least part of an operation may be performed by a group of processors available in a distributed or remote system, which are accessible via a communication network (e.g., the Internet) and via one or more software interfaces (e.g., application programming interfaces (APIs)).In some embodiments, the computer system 400 may include an operating system such as Microsoft's Windows®, Sun Microsystems' Solaris®, Apple Computer's MacOs, Linux®, or UNIX®. In some embodiments, the computer system 400 may also include a basic input / output system (BIOS) and processor firmware. The operating system, BIOS, and firmware are used by one or more processors 402 to control subsystems and interfaces coupled to one or more processors 402. In some embodiments, one or more processors 402 may include Intel's Pentium® and Itanium, Advanced Micro Devices' Opteron and Athlon, and ARM Holdings' ARM processors.

[0026] As shown in Figure 4, the computer system 400 may include a machine-readable medium 404. In some embodiments, the machine-readable medium 404 may further include a main random access memory ("RAM") 406, a read-only memory ("ROM") 408, and / or a file storage subsystem 410. The RAM 406 can store instructions and data during program execution, and the ROM 408 can store fixed instructions. The file storage subsystem 410 provides persistent storage for program and data files and may include a hard disk drive, a floppy disk drive and associated removable media, a CD-ROM drive, an optical drive, flash memory, or a removable media cartridge.

[0027] The computer system 400 may further include a user interface input device 412 and a user interface output device 414. The user interface input device 412 may, in some embodiments, include pointing devices such as alphanumeric keyboards, keypads, mice, trackballs, touchpads, styluses, or graphics tablets; scanners; touch screens integrated into displays; audio input devices such as voice recognition systems or microphones; gaze recognition; electroencephalogram pattern recognition; and other types of input devices. The user interface input device 412 may be connected to the computer system 400 by wire or wirelessly. Generally, the user interface input device 412 is intended to include all conceivable types of devices and methods for inputting information into the computer system 400. Typically, the user interface input device 412 allows a user to identify objects, icons, text, and equivalents appearing on certain types of user interface output devices (e.g., display subsystems). The user interface output device 414 may include non-visual displays such as display subsystems, printers, fax machines, or audio output devices. The display subsystem may include flat panel devices such as cathode ray tubes (CRTs) and liquid crystal displays (LCDs), projection devices, or other devices for generating visible images, such as virtual reality systems. The display subsystem may also provide non-visual displays via audio output or haptic output (e.g., vibration) devices. In general, the user interface output device 414 is intended to include devices and methods of any type that can be considered as any possibility for outputting information from the computer system 400.

[0028] The computer system 400 may further include a network interface 416 for providing an interface to an external network (including an interface to a communication network 418), which is connected to a corresponding interface device in another computer system or machine via the communication network 418. The communication network 418 may comprise many interconnected computer systems, machines, and communication links. These communication links may be wired links, optical links, wireless links, or any other devices for the transmission of information. The communication network 418 may be any suitable computer network, such as a wide area network like the Internet, and / or a local area network like Ethernet®. The communication network 418 may be wired and / or wireless, and the communication network may use encryption and decryption methods, such as those available using a virtual private network. The communication network may use one or more communication interfaces, one or more of which may receive data from other systems and transmit data to other systems. Embodiments of the communication interface typically include Ethernet® cards, modems (e.g., telephone, satellite, cable, or ISDN), (asynchronous) digital subscriber line (DSL) units, Firewire® interfaces, USB interfaces, and equivalents. One or more communication protocols such as HTTP, TCP / IP, RTP / RTSP, IPX, and / or UDP may be used.

[0029] As illustrated in Figure 4, one or more processors 402, machine-readable media 404, user interface input devices 412, user interface output devices 414, and / or network interfaces 416 can be coupled together to communicate with each other using a bus subsystem 420. Although the bus subsystem 420 is schematically shown as a single bus, alternative embodiments of the bus subsystem may use multiple buses. For example, RAM-based main memory can communicate directly with a file storage system using a direct memory access ("DMA") system.

[0030] conclusion For detailed descriptions, accompanying drawings are used to illustrate exemplary embodiments consistent with this disclosure. The use of the phrase "an exemplary embodiment" in this disclosure indicates that an exemplary embodiment described may include certain features, structures, or characteristics, but not all exemplary embodiments may include such features, structures, or characteristics. Furthermore, such phrases do not necessarily refer to the same exemplary embodiment. In addition, any features, structures, or characteristics described in relation to an exemplary embodiment may be included independently or in any combination with features, structures, or characteristics of other exemplary embodiments, whether expressly described or not.

[0031] The detailed description is not intended to be restrictive. Rather, the scope of this disclosure is defined solely by the following claims and their equivalents. It should be understood that the detailed description section, and not the abstract section, is intended to be used to interpret the claims. The abstract section may describe one or more exemplary embodiments of this disclosure, but not all of them, and is therefore not intended to limit this disclosure and the following claims and their equivalents in any way.

[0032] The exemplary embodiments described herein are provided for illustrative purposes only and are not intended to be limiting. Other exemplary embodiments may be conceivable, and modifications may be made to the exemplary embodiments, while remaining within the spirit and scope of this disclosure. This disclosure is described with the help of functional components illustrating the implementation of the defined functions and their relationships. The boundaries of these functional components are arbitrarily defined herein for the convenience of explanation. Alternative boundaries may be defined, insofar as their defined functions and relationships are adequately implemented.

[0033] Embodiments of the Disclosure may be implemented in hardware, firmware, software applications, or any combination thereof. Embodiments of the Disclosure may also be implemented as instructions stored on a machine-readable medium that can be read and executed by one or more processors. The machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computing network). For example, the machine-readable medium may include non-transient machine-readable media such as read-only memory (ROM), random-access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, and others. In another embodiment, the machine-readable medium may include transient machine-readable media such as electrical, optical, acoustic, or other forms of propagating signals (e.g., carrier waves, infrared signals, digital signals, etc.). Furthermore, firmware, software applications, routines, and instructions may be described herein as performing certain actions. However, it should be understood that such descriptions are for convenience only, and that such actions actually result from computing devices, processors, controllers, or other devices that perform the firmware, software applications, routines, instructions, etc.

[0034] The detailed description of exemplary embodiments fully reveals the general nature of this disclosure that others may readily modify and / or adapt such exemplary embodiments for various uses by applying the knowledge of those skilled in the art, without undue experimentation and without departing from the spirit and scope of this disclosure. Such adaptations and modifications are therefore intended to be within the meaning of the exemplary embodiments and their equivalents based on the teachings and guidance presented herein. It should be understood that the terminology or language used herein is for illustrative purposes only, and not for restrictive purposes, so that it may be interpreted by those skilled in the art in light of the teachings herein.

Claims

1. A content server for integrating real-world performers within a venue into interactive content, wherein the content server is: A memory for storing the video of the aforementioned real-world performer, A processor configured to execute instructions stored in the memory, wherein when an instruction is executed by the processor, To capture the movement of the real-world performer from the aforementioned video, To generate a three-dimensional model of the real-world performer that emulates the movement of the real-world performer, Applying the three-dimensional model of the real-world performer to a virtual character included in the interactive content, and causing the virtual character to emulate the movement of the real-world performer, To provide the virtual character, which will be incorporated into the aforementioned interactive content, to a portable electronic device. The processor is configured to perform the following: A content server equipped with the following features.

2. The content server according to claim 1, wherein, when the instruction is executed by the processor, the processor is further configured to receive the video from a camera of the real-world performer performing in the venue.

3. The content server according to claim 1, wherein, when the instruction is executed by the processor, the processor is configured to capture the movement of the real-world performer using a markerless approach.

4. When the aforementioned instruction is executed by the processor, further, Identifying multiple performer markers of the real-world performer from the aforementioned video, To generate the three-dimensional model of the real-world performer from the plurality of performer markers. The content server according to claim 1, wherein the processor is configured to perform the following:

5. When the aforementioned instruction is executed by the processor, To generate the three-dimensional model of the real-world performer in three-dimensional space, Applying the three-dimensional model of the real-world performer to the virtual character in the three-dimensional space The content server according to claim 1, wherein the processor is configured to perform the following:

6. The content server according to claim 5, wherein, when the instruction is executed by the processor, the processor is configured to further render the virtual character from the three-dimensional space into the two-dimensional space of the bidirectional content.

7. The content server according to claim 6, wherein when the instruction is executed by the processor, the processor is configured to provide the virtual character in the two-dimensional space.

8. A method for incorporating real-world performers into interactive content, wherein the method is Accessing the video of the real-world performer by one or more computer systems, The one or more computer systems capture the movement of the real-world performers within the venue from the video, The one or more computer systems generate a three-dimensional model of the real-world performer that emulates the movement of the real-world performer, The one or more computer systems apply the three-dimensional model of the real-world performer to a virtual character included in the interactive content, and cause the virtual character to emulate the movement of the real-world performer. The one or more computer systems provide the virtual character, which will be incorporated into the interactive content, to a portable electronic device. Methods that include...

9. The method according to claim 8, wherein the access includes receiving the video from a camera of the real-world performer performing in the venue.

10. The method according to claim 8, wherein the capturing includes capturing the movement of the real-world performer using a markerless approach.

11. To cause the aforementioned to occur, Identifying multiple performer markers of the real-world performer from the aforementioned video, To generate the three-dimensional model of the real-world performer from the plurality of performer markers. The method according to claim 8, including the method described in claim 8.

12. The aforementioned generation includes generating the three-dimensional model of the real-world performer in three-dimensional space. The above application includes applying the three-dimensional model of the real-world performer to the virtual character in the three-dimensional space. The method according to claim 8.

13. The method according to claim 12, wherein the provision includes rendering the virtual character from the three-dimensional space into the two-dimensional space of the interactive content.

14. The method according to claim 13, wherein the provision further includes providing the virtual character in the two-dimensional space.

15. A venue for integrating real-world performers into interactive content, the said venue is It is a content server, To capture the movement of the aforementioned real-world performer within the venue from the video of the aforementioned real-world performer, Applying the three-dimensional model of the real-world performer to a virtual character included in the interactive content, and causing the virtual character to emulate the movement of the real-world performer. A content server configured to perform the following actions: Multiple electronic devices, Receiving the virtual character from the content server, Running the aforementioned software application having interactive content, The virtual character is incorporated into the interactive content. Multiple electronic devices configured to perform the following actions A venue equipped with these features.

16. The system further includes a camera for capturing video footage of the real-world performer performing within the venue. The content server is further configured to receive the video from the camera. The venue described in claim 15.

17. The venue according to claim 15, wherein the content server is configured to capture the movement of the real-world performer using a markerless approach.

18. The aforementioned content server further, Identifying multiple performer markers of the real-world performer from the aforementioned video, To generate the three-dimensional model of the real-world performer from the plurality of performer markers. The venue according to claim 15, configured to perform the following.

19. The aforementioned content server is To generate the three-dimensional model of the real-world performer in three-dimensional space, Applying the three-dimensional model of the real-world performer to the virtual character in the three-dimensional space The venue according to claim 18, configured to perform the following:

20. The aforementioned content server further, Rendering the virtual character from the three-dimensional space into the two-dimensional space of the interactive content, To provide the virtual character in the two-dimensional space to the plurality of electronic devices. The method according to claim 19, configured to perform the following:

Images