Performance puppet

Abstract

A physical performance puppet for use in generating a digital character, comprising: a joint network comprising a plurality of joint assemblies and a plurality of transducers configured to provide movement data indicative of a movement of the joint network, wherein the movement data defines a movement of a digital character; a plurality of physique elements distributed around the joint network such that the physical performance puppet has a topography and mass distribution equivalent to the digital character, whereby the movement of the digital character has a substantially one-to-one mapping to the movement of the performance puppet; wherein manipulation of the physical performance puppet corresponds to real-time equivalent movement of the digital character.

Description

[0001] PERFORMANCE PUPPET

[0002] FIELD OF THE INVENTION

[0003] The invention generally relates to a physical performance puppet for use in generating a digital character. In particular, the system allows digitally animated and digitally controlled characters to be performed using the techniques of live physical puppet performance.

[0004] BACKGROUND

[0005] In its traditional form, puppetry uses the performance skills of human puppeteers to directly manipulate characters using manual control systems, producing a realtime character performance with a physical presence, weight, physicality and movement resulting from the interplay of puppeteers and the physicality of the puppets and their environments.

[0006] Puppetry techniques have been expanded to allow the real-time performance of remotely actuated physical puppet characters using indirect manipulation via control apparatus operated by puppeteers remote from the physical character. The control apparatus can pass control movements and signals to the puppet such that the puppet is remotely controlled by the control apparatus. A form of puppet control where the signals driving movement in the puppet are controlled or moderated by (micro)processor(s) is termed digitally controlled characters.

[0007] Physical puppets can be interacted with by other puppeteers, actors, and audience members, and they can be directed live in a manner analogous to working with physical actors. This physical reality limits them to movements that can be physically performed, a limitation most evident with large-scale puppets.

[0008] In contrast with physical puppets (both directly and indirectly controlled), computer-generated (digitally animated) characters are unlimited by physical constraints, and may have additional visual enhancements, animation and simulation applied to any performance. Digitally animated characters do not react to, or interact within, the physical environment inhabited by puppeteers, actors, or audience members. Digitally animated characters require digital visualisation to allow their puppeteers to operate them, and for directors, other cast members, and the audience to see and react to their performances. There are multiple methods to control the performance of digitally animated and digitally controlled characters.

[0009] Indirect control apparatus can be used to control digitally animated characters in real time in a manner that can be thought of as similar to the use of computergame hardware controllers such as joysticks, mice and gamepads in the control of computer-game characters.

[0010] In combination with, or instead of, indirect control apparatus human body and facial motion capture can provide real-time control of digitally animated characters. Motion capture performers are able to be directed and interact with the real world, with enhancement and adjustment possible once these data are applied to digital characters.

[0011] A limitation of human motion capture is that the movement captured is that of the human motion capture performer, inherently limiting the range of captured movement to that physically possible by a human. Additionally, the captured movement data are related to, and associated with, the unique physicality of the performer therefore requiring additional processing and modification to transform the captured movement data into movement suited to the different nominal physicality of a digital character, particularly in cases where that character is not shaped like a human.

[0012] Motion capture technologies may be used to capture animal movement. Typically this only allows capture of movements which may be performed by trained animals, which generally cannot be directed to perform or react during production as a human-performed character could.

[0013] SUMMARY OF INVENTION

[0014] According to a first aspect of the invention there is provided a physical performance puppet for use in generating a digital character. The physical performance puppet comprises a joint network comprising a plurality of joint assemblies and a plurality of transducers configured to provide movement data indicative of a movement of the joint network, wherein the movement data defines a movement of the digital character. Since the digital character’s movement is determined based on the movement of the joint network, and the movement of the joint network is captured by the transducers, it can be considered that the movement data from the transducers defines the movement of the digital character, by way of the joint network. Thus, it may also be said that the movement data is indicative of a movement of the joint network, and the movement of the joint network defines a movement of the digital character. The physical performance puppet further comprises a plurality of physique elements distributed around the joint network such that the physical performance puppet has a topography and mass distribution equivalent to the digital character, whereby the movement of the digital character has a substantially one-to-one mapping to the movement of the performance puppet. Manipulation of the physical performance puppet corresponds to real-time equivalent movement of the digital character.

[0015] In this context, the correspondence between the manipulation of the physical performance puppet and the real-time equivalent movement of the digital character is unidirectional and not bidirectional; movement of the physical performance puppet drives movement of the digital character. Thus, manipulation of the physical performance puppet results in real-time equivalent movement of the digital character. The physique elements of the physical performance puppet, which have a topography and mass distribution equivalent to those of the digital character, ensure that when the physical performance puppet is moved, its physical behaviour mirrors how the digital character would move if it existed in the same physical environment. Since the digital character cannot be directly manipulated, the physical performance puppet serves as a physical proxy. The matched physical characteristics enable a natural and accurate one-to-one mapping between the physical puppet’s movement and the digital character’s movement, ensuring realistic and consistent movement of the digital character in real time. Throughout this disclosure, topography and morphology are intended to describe the shape, and form of features, particularly surface features. Both terms, topography and morphology, can be used interchangeably. Thus, the topography or morphology of a physical performance puppet refers to the shape, texture, and arrangement of the physical puppet’s visible elements, external features, and surface contours. A physical puppet’s topography / morphology includes but is not limited to surface details (such as facial contours and body contours), material texture (including hair and fur), structural features (including joints, hinges, and other movable parts), spatial arrangement of parts, and decorative features (such as clothing and accessories). A physical puppet’s topography / morphology helps accurately depict the puppet's physical form for animation, capturing the puppet’s physical details and arrangement in a descriptive and measurable way.

[0016] In the context of the disclosure, the topography / morphology is understood to mean an anatomical topography / morphology.

[0017] Movement of the physical performance puppet is captured by the transducers and accurately transferred to the digital character such that real-time manipulation of the physical performance puppet results in equivalent real-time movement of the digital character. The physical performance puppet is topographically equivalent to the digital character (which may also be referred to as a digital replica or digital twin) such that the physicality of the physical performance puppet is substantially the same as the physicality of the digital character. Movements resulting from live performance and environmental interaction of the physical performance puppet analogue are transferred onto the digital character.

[0018] In some examples, one or more transducers may be configured such that one or more surfaces or objects external to the performance puppet and being in proximity to or in contact with the surface of one or more physique elements results in movement data indicative of the movement of the physique element surface(s) relative to the external surface(s) or object(s).

[0019] Throughout this disclosure, the term “physicality” includes overall body shape, distribution of mass and volumes, and limb and body part jointing and movement. In this way, the construction of the physical performance puppet is such that, at least topographically, the physicality of the performance puppet is equivalent to that of the digital character or would be equivalent if the digital character had a physical existence.

[0020] The joint network can be thought of as a matrix of rotation and / or translation centres that have a known, and in some cases fixed, relationship with respect to one another which may provide similar or identical topography and movement to the structure of the digital character being animated. The transducers may be attached to the joint network such that transducer data values are generated by the movements (e.g., rotation, translation) of the joint network components. The joint network may be attached to and I or within the performance puppet. Thus, in some examples the joint network may be an internal structure, in some other examples the joint network may be an external structure, and in further examples the joint network may be both an internal and external structure.

[0021] The physical performance puppet provides a means of transferring the performance, movement, and elements of physicality of a live-performed physical puppet character onto a digital or digitally controlled character. The plurality of transducers may be attached to and I or built into the physical performance puppet. The transducers output data for movements of individual regions of the joint network of the physical performance puppet which result both from the movement of the physical performance puppet by a puppeteer (which may also be referred to herein as a human performer) and from the physical performance puppet’s interaction with its physical environment.

[0022] The movement data for the different regions of the physical performance puppet are input to and collated by a computing device comprising a (micro)processor. The movement data may be converted in real time onto equivalent regions of the digital or digitally controlled character having a topographically equivalent structure to the physical performance puppet. In some cases, the movement data may be stored in memory. The movement of different regions of the joint network measured by the transducers allows the real time transfer of a live physical puppet performance to digital or digital-controlled characters, with characteristic movements resulting from the weight, physicality, and interaction of the physical performance puppet with its physical environment.

[0023] The live and stored movement data may be directly used, further enhanced, edited, combined with other data, analysed, or used to train machine learning models, to be used in the generation of new digital or digital-controlled character performances.

[0024] The physique elements may comprise a plurality of mass elements and I or volume elements. Each physique element in the plurality of physique elements may comprise at least one mass element having a volume. Each physique element in the plurality of physique elements may comprise at least one volume element having a mass. At least one physique element may be designed to represent flesh components of the digital character. At least one physique element may be designed to represent digits (including fingers and toes) of the digital character. At least one physique element may be designed to represent additional appendages (including tails, horns, antlers, ears, etc) of the digital character. In this way, each physique element may be designed to correspond to a part of a body of the digital character such that at least one of the mass distribution, volume, and topography of the physique element corresponds to the equivalent mass distribution, volume, and I or topography of the part of the digital character being represented. A plurality of physique elements may be used to represent each limb of the digital character. One or more physique elements may be used to represent the main trunk of the digital character. One or more physique elements may be used to represent the head of the digital character.

[0025] The physical performance puppet may comprise an external surface which substantially realistically represents skin and / or costume of the digital character.

[0026] Manipulation of the physical performance puppet may be direct manipulation (which may also be referred to as primary manipulation) or indirect manipulation (which may also be referred to as secondary manipulation), or a combination of both direct and indirect manipulation. Movement of some part of the physical performance puppet (e.g., ears, one or more limbs etc) may be generated as a result of direct manipulation of another part of the physical performance puppet (e.g., main body, one or more limbs). Direct manipulation may include direct contact by a puppeteer with part of the performance puppet e.g. part of an external surface of the performance puppet, part of the joint network, and I or one or more of the plurality of physique elements. Direct manipulation may also include contact by a puppeteer with a designated contact point on the performance puppet e.g. a handle. Indirect manipulation may include movement as a result of environmental forces applied to at least a part of the performance puppet e.g., wind, fluid resistance, or gravity acting on a part of the performance puppet, or reaction to contact with a surface. The physical performance puppet may be manipulated by one or more operators (also referred to herein as puppeteers) to produce a live character performance in real time. The physical performance puppet may be manipulated by one or more puppeteers using manual control systems.

[0027] Each joint assembly in the joint network may be associated with at least one transducer. The transducer may be attached to part of the joint assembly. The transducer is preferably configured to measure movement of the joint assembly and convert the measured movement of the joint assembly into digital movement data for processing by an external computing device.

[0028] At least one joint assembly in the plurality of joint assemblies may preferably be a multi-axis joint assembly comprising a plurality of axes of rotation. The multi-axis joint assembly may be a two-axis, three-axis, or four-axis rotation joint assembly. In this way, the joint assembly may provide a large range of motion around a plurality of rotation axes.

[0029] The multi-axis joint assembly may comprise at least two axes of rotation that intersect. The multi-axis joint assembly may comprise at least one axis of rotation that does not intersect with one or more other axes of rotation. The multi-axis joint assembly may comprise at least one axis of rotation that does not intersect with any other axis of rotation. An advantage of a multi-axis joint assembly comprising at least one axis of rotation that does not intersect with any other axes of rotation is that the joint assemblies may provide motion (e.g. rotations) equivalent to that of a three-axis rotation joint, such as a ball joint, whilst having the components of the joint assembly that do the rotating physically separated from each other onto one or more individual singleaxis joints.

[0030] The joint assembly may comprise at least two single-axis rotation joints, wherein the at least two single-axis rotation joints are physically separate from each other in space but share a common intersection point of their axes of rotation.

[0031] Preferably, at least one of the single-axis rotation joints is displaced from a common centre of rotation of the joint assembly. In some examples, at least one of the single-axis rotation joints may be displaced along a primary axis from a common centre of rotation of the joint assembly. Alternatively or in addition, in some examples at least one of the single-axis rotation joints may be displaced along a secondary axis from a common centre of rotation of the joint assembly.

[0032] Two or more of the single-axis rotation joints may be displaced from a common centre of rotation of the joint assembly. Joint assemblies comprise one or more joints that are displaced from a common centre of rotation of the joint assembly have an effective common centre of rotation of the joint assembly, which is considered as the effective centre point about which rotations originate. In this way, different single-axis rotation joints are displaced in space but share an effective common intersection point of their axes of rotation. The joint assembly may comprise an axis of rotation along which a joint may be offset a considerable distance from the common movement intersection point of the joint assembly without affecting the movement of the overall joint assembly. The physical separation of complex rotation movement into multiple discrete single axis rotations can help to simplify rotation measurement. For example, the same position of a ball in space can be reached by several different combinations of x-, y-, and z-axis rotations but this combination will not necessarily be the same each time this position is reached. Whilst this is generally not a problem for direct live feeds, this can present an issue when trying to edit or clean up the measured and captured movement data. Thus, having a combination of single axis mechanical movements helps to keep the digital xyz representation consistent.

[0033] The joint network may comprise joint assembles that have pairs of single axis rotation joints having a slight displacement between the joints’ axes of rotation and what would be the effective common intersection point. Here, the displacement distance is sized such that this displacement does not result in a significant effect on the overall movement of the joint assembly at the scale of the digital character.

[0034] In some examples, at least one axial rotation joint (also referred to as an on-axis rotation joint) may be offset from single multi-axis centres of rotation, which results in joint assemblies that allow transducers to measure multi-axis rotations around a single effective centre of rotation with large angles of rotation and small joint assembly volumes and masses. The presence of this offset allows for freer movement and wider rotation of the performance puppet structures around the transducers. Additionally, offsetting individual joint rotation centres away from the common intersection point of a joint assembly allows space within the joint network for the placement of the transducers whilst maintaining a sufficient range of motion to fully move the performance puppet.

[0035] In some examples, a multi-axis joint assembly may comprise a ball joint. Movement of the ball joint may result in the movement of at least one single axis rotation joint. In some examples, a ball joint may be associated with at least one transducer. The at least one transducer may be positioned to allow measurement of at least one single component (e.g., x, y, z) and I or double component (e.g., xy, xz, yz) and I or triple component (e.g., xyz) axis rotations of the ball joint.

[0036] The joint assemblies of the joint network are positioned relative to the performance puppet such that rotation or translation of regions of the performance puppet will result in rotation or translation of the corresponding joint assemblies of the joint network. Each region of the performance puppet may have an equivalent region on the digital character, preferably including equivalent axes of rotation and / or centres of movement to the corresponding physical performance puppet region. Movement (e.g. rotation and / or translation) of the performance puppet causes movement of the joint assemblies in the joint network, and so it can be thought that movement of the performance puppet drives movement of the joint network. Movement of the joint network, in particular movement of the joint assemblies, results in movement data being generated and signals being sent from the transducer. The signals from the transducer comprise the movement data. Each transducer preferably generates signals, corresponding to the measured movement data, that are proportionate to the movement of the joint assembly in real time. The movement data can include translation date and / or rotation data. In this way, each transducer is measuring, in real time, movement of the performance puppet.

[0037] The signals from the transducers, comprising movement data, may be input into a computing device comprising a (micro)processor. The movement data comprises data values representing movement in one or more directions for a particular joint assembly of the joint network.

[0038] In some examples, the data values are stored for subsequent processing. Alternatively, or in addition, the data values may be input into a software application which is configured to create motion (e.g. rotation and / or translation) in a digital animation or digitally controlled character. In the case of the digitally controlled character, the data values, representing movement data, may be input into a control system and used to instruct the control system to move the digitally controlled character in an analogous manner to the performance puppet. It should be noted that both a digital animation and a digitally controlled character are examples of a digital character. Movement of the digital animation or digitally controlled character preferably happens substantially in real time, this movement driven by the data values generated as a result of movement of the physical performance puppet.

[0039] In the example of a digitally controlled character, the movements of the control system cause the digital character regions to move substantially in real time in tandem with the data generated from movement of analogous regions of the physical performance puppet. For the purpose of this disclosure, “substantially in real time” means that there will be a negligible difference in time between the two events such as movement of the puppet and the initiation of the control of the digitally controlled character using the control system or the movement of the digital animation. In other words, any time difference between the two events will essentially not be perceptible to a human. In some examples, a time difference of less than 1 second, preferably less than 0.5 seconds, more preferably less than 0.25 seconds, and more preferably less than 0.1 seconds between two events is considered to be in real-time.

[0040] The data values, after input into a computing device and / or software application, may be further processed, enhanced, edited, combined with other data, or analysed. In some examples, the data values may be used to train a machine learning model for use in the generation of new digital animation or digital- controlled characters.

[0041] The joint network and transducers are suitably small and lightweight such that their presence does not affect or restrict the ability of the puppeteer to move and operate the performance puppet or movement of the performance puppet resulting from the performance puppet’s interaction with the physical environment.

[0042] As the physical puppets are preferably designed to have equivalent physicalities to a physical analogue of the digital character, the digital character has substantially realistic movement features for its nominal physicality and environment.

[0043] A real-time visualisation of the digital character(s), which may include representations of other characters, puppeteers, performers, and the environment, may be displayed for example to puppeteers, directors, other participants and I or an audience during the physical puppet performance. This visualisation may take the form of media presentations including display screens or projections, and the visualisation may be presented as overlays co-located with the physical performance puppet in the field of view of the puppeteers, including on viewing headsets or glasses. The physical performance puppet may comprise a communication device configured to communicate, preferably wirelessly, with an external computing device. The communication device may be a transmitter or a transceiver. The communication device may send data from the performance puppet to the external computing device.

[0044] The communication device, such as a transceiver, may be configured to transmit movement data to an external computing device. The movement data may be generated by the performance puppet and measured by the transducer. The transducer may be in communication with the communication device such that data measured by the transducer can be sent to the communication device, for subsequent transmission to the external computing device.

[0045] The communication device may be configured to transfer data wirelessly, preferably using Wi-Fi. This reduces the number of cables that need to be attached between the performance puppet and the external computing device.

[0046] Each joint assembly may be connected to one or more other joint assemblies in the joint network. In some examples, a joint assembly may not be connected to any other joint assembly in the joint network.

[0047] There may be provided a system comprising the physical performance puppet as described above and a computing device configured to generate real-time movement of a digital character, the real-time movement corresponding to movement of the physical performance puppet.

[0048] In some examples, at least one transducer may be a single axis transducer configured to measure movement data in one axis only. Alternatively, or in addition, one or more of the transducers may be multi-axis transducers configured to measure movement data in more than one axis.

[0049] The plurality of transducers may comprise more than one type of transducer. The plurality of transducers may comprise a single type of transducer. In some examples, one or more of the transducers may take the form of any one or more of the following: optical pattern sensors, accelerometers, ultrasonic sensors, RFID sensors, potentiometers, gyroscopes, magnetometers, and I or magnetic rotation sensors. More than one sensor type can be combined. As an example, magnetic rotation sensors may be small in size whilst providing accurate data measurements.

[0050] The physical performance puppet may facilitate the real time transfer, for direct use, storage or further processing, of a live physical puppet performance onto a digital or digital-controlled characters, with characteristic movement resulting from the weight, physicality and interaction of the physical puppet with its physical environment and other physical characters, performers, and puppeteers present.

[0051] The physical performance puppet may comprise a matrix-like structure of rotation and / or translation centres, the structure having a similar or identical topography and movement to the internal structure of the digital character being replicated, with attached transducer data values generated by the movements (for example rotation, translation) of the matrix-like structure.

[0052] At least one transducer may be arranged such that contact between at least one physique element and at least one external object or surface is detected. The transducer is able to detect movement of the physique element caused by contact of the physique element with an external object or surface, and so it can be inferred that contact between the physique element and the external object or surface has occurred through detection of appropriate movement data.

[0053] At least one transducer may be arranged such that proximity to one external object or surface from at least one physique element is detected.

[0054] In some examples, a weight of the physical performance puppet may be at least partially supported by suspension of the physical performance puppet from one or more fixed or moveable external point(s). Suspension of the physical performance puppet may help relieve any compressions or tensions which the joint assemblies may otherwise experience. Suspension of the physical performance puppet may also help reduce potential injury to puppeteers from prolonged use.

[0055] There may be provided a joint network for use in a physical performance puppet used in generating a digital character, the joint network comprising: a plurality of joint assemblies; and a plurality of transducers; wherein each joint assembly in the joint network is associated with at least one transducer, wherein the at least one transducer is configured to measure movement of the joint assembly and convert the measured movement of the joint assembly into digital movement data for processing by an external computing device. The digital movement data may define a movement of the digital character.

[0056] At least one joint assembly may comprise at least two single-axis rotation joints. The at least two single-axis rotation joints may be physically separate from each other in space but share a common intersection point of their axes of rotation.

[0057] At least one of the single-axis rotation joints may be displaced from a common centre of rotation of the joint assembly. In some examples, at least one of the single-axis rotation joints may be displaced along a primary axis from a common centre of rotation of the joint assembly. Alternatively or in addition, at least one of the single-axis rotation joints may be displaced along a secondary axis from a common centre of rotation of the joint assembly.

[0058] Two or more of the single-axis rotation joints may be displaced from a common centre of rotation of the joint assembly. Two of the single axis rotation joints may have a common axis of rotation. The multi-axis joint assembly may comprise a ball joint, preferably wherein movement of the ball joint results in the movement of at least one single axis rotation joint.

[0059] There may be provided a joint assembly for use in a physical performance puppet used in generating a digital character, the joint assembly comprising at least two single-axis rotation joints, wherein the at least two single-axis rotation joints are physically separate from each other in space but share a common intersection point of their axes of rotation. At least one of the single-axis rotation joints may be displaced from a common centre of rotation of the joint assembly. The joint assembly may further comprise at least one transducer, wherein the at least one transducer is configured to measure movement of the joint assembly and convert the measured movement of the joint assembly into digital movement data for processing by an external computing device. The digital movement data may define a movement of the digital character.

[0060] There may be provided a joint assembly for use in a physical performance puppet used in generating a digital character, the joint assembly comprising a multi-axis joint assembly comprising a plurality of axes of rotation, wherein the multi-axis joint assembly comprises a ball joint, wherein movement of the ball joint results in the movement of at least one single axis rotation joint. The joint assembly may further comprise at least one transducer, wherein the at least one transducer is configured to measure movement of the joint assembly and convert the measured movement of the joint assembly into digital movement data for processing by an external computing device. The digital movement data may define a movement of the digital character.

[0061] There may be provided a joint assembly for use in a physical performance puppet used in generating a digital character, the joint assembly comprising: at least two single-axis rotation joints, wherein the at least two single-axis rotation joints are physically separate from each other in space but share a common intersection point of their axes of rotation; and wherein at least one of the single-axis rotation joints is displaced from a common centre of rotation of the joint assembly.

[0062] There may be provided a joint assembly for use in a physical performance puppet used in generating a digital character, the joint assembly comprising: a multi-axis joint assembly comprising a plurality of axes of rotation; wherein the multi-axis joint assembly comprises a ball joint, wherein movement of the ball joint results in the movement of at least one single axis rotation joint.

[0063] It will be understood that any joint assembly described herein can be combined to form a joint network, and any joint assembly or joint network can be used in a performance puppet. Thus, all the concepts described herein can be combined in a suitable manner to form joint assemblies and joint networks for use in physical performance puppets.

[0064] The physical performance puppet of the claimed invention may help provide a production technique equivalent to human motion capture as applied to human digital characters, for application to digital non-human characters, and digital humanoid characters which are performing actions difficult or dangerous for a real human performer. The physical performance puppet may facilitate providing a production technique in which motions resulting from live performance and environmental interaction of the physical puppet analogues may be transferred to their digital character counterparts.

[0065] BRIEF DESCRIPTION OF DRAWINGS

[0066] Embodiments of the present invention will be described with reference to the accompanying drawings in which:

[0067] Figure 1 shows an example physical performance puppet;

[0068] Figure 2 shows an example joint assembly;

[0069] Figure 3 shows an example joint assembly;

[0070] Figure 4 shows an example joint assembly;

[0071] Figures 5a - 5j show example joint assemblies;

[0072] Figure 6 shows an example physical performance puppet;

[0073] Figure 7 shows an example physical performance puppet; and

[0074] Figure 7a shows an example physical performance puppet. DETAILED DESCRIPTION

[0075] For stop-motion animation puppets, and puppets derived from these, the term “armature” is used to refer to an articulated skeleton, typically made of metal or plastic, that can be repositioned for each frame of the animation. The armature allows the puppet to maintain its shape and be manipulated in a controlled manner by the stop-motion animator. Within this disclosure we shall also use the term “armature” to refer to any internal framework or skeleton that supports and allows movement of a puppet. For example, for string puppets (marionettes), rod puppets, and tabletop (equiplane) puppets, the armature may be considered the jointed framework of wood, metal, or plastic that supports and allows movement of the body and limbs of the puppet. Strings, rods, and handles are attached to this armature to control the movements. For glove puppets the armature may be considered as the human hand inside it in along with the fabric connecting the body to any limbs controlled by rods. In some examples, it may be the case that the armature is made of flexible or posable materials like aluminium wire that allow the puppet's body and limbs to bend and I or be posed by the puppeteer.

[0076] Within the disclosure we shall use the term “rig” to refer to the entire mechanical system that makes up the puppet, including the armature, but also the attached control mechanisms like handles, strings, rods, levers, etc. that the puppeteer uses to manipulate the puppet's movements. While the armature can be thought of as the "bones" of the puppet, the rig is the armature plus all the control components like strings, rods, etc. that comprise the full operating system. Generally, the armature alone does not allow full manipulation but instead it needs the attached rig components to make it a fully functional puppet. Whilst an armature and a rig are related but different concepts, (an armature being part of a rig and a rig also including additional control elements beyond just the armature framework), throughout this disclosure we will use the terms armature and rig interchangeably to refer to the posable internal structure of a puppet.

[0077] A performance puppet refers to a puppet that is designed and constructed for use in live performances, for example, stage shows, theatre productions, television broadcasts or filmed performances. As such, performance puppets are built to be easily manipulable by the puppeteer(s) during a live performance. Performance puppets must have well-designed armatures, rigs and control mechanisms that allow for smooth, lifelike movements and expressions during the performance. Performance puppets generally also have detailed external designs, such as intricate costumes and character and body features. A performance puppet’s design and rigging system allows for efficient manipulation by one or more puppeteers during the performance.

[0078] A key difference between performance puppets and armatures is that the performance puppets can replicate the “full character”, whereas an armature only replicates the underlying movement of the character. In other words, a person watching a live performance of a performance puppet sees the full overall appearance and motion of the character, whereas a person watching a live performance of an armature sees only the motion of the underlying skeleton.

[0079] The present invention relates to performance puppets. The invention described herein generally relates to a system for performing digitally animated and digitally controlled characters in real time using a physical puppet performance. The inventive system offers particular advantages over existing techniques for the performance of real time non-human digital characters.

[0080] Figure 1 shows a physical performance puppet 2 for use in generating a digital character 4. The performance puppet 2 includes a joint network 6 which is made up of a plurality of joint assemblies 8, and a plurality of transducers 10 configured to provide movement data indicative of a movement of the joint network 6. The movement data defines a movement of the digital character 4.

[0081] The performance puppet 2 also includes a plurality of physique elements 12 distributed around the joint network 6 such that the physical performance puppet 2 has a topography, volume, and mass distribution substantially equivalent to the topography, volume, and mass distribution of the digital character 4. This has the effect that the movement of the digital character 4 has a substantially one-to-one mapping to the movement of the performance puppet 2. As will be discussed in more detail, manipulation of the physical performance puppet 2 corresponds to real-time substantially equivalent movement of the digital character 4.

[0082] The performance puppet 2 of the invention is therefore distinct from an armature in that it is not just an articulated skeletal structure. Instead, the performance puppet 2 additionally comprises physique elements 12 arranged to mimic the size, shape, and mass of corresponding physical elements of the digital character 4. For example, as can be seen in Figure 1 , the performance puppet 2 comprises three physique elements 12a, 12b, 12c corresponding to the upper arm, forearm, and hand of the digital character 4. These three physique elements 12a, 12b, 12c have properties (including topography, volume, and mass distribution) that mean that the joint assemblies 8 in the arm portion of the joint network 6 move in a manner that would be expected of the digital character 4 such that the arm of the performance puppet 2 moves and behaves (e.g., reacts to external forces including manipulation by a puppeteer) in a substantially realistic manner, which then results in equivalent realistic movement of the digital character 4. As an example, the arm of an adult would be bigger (in terms of both size and mass) than the arm of an infant such that if the same object (e.g. a basketball) was thrown at both arms (or a puppeteer moved the arms with the same force), a greater response (e.g., in terms of range of movement) would be expected from the smaller and lighter infant’s arms compared to the larger and heavier adult’s arm. As a further example, the performance puppet 2 comprises two physique elements 12d, 12e corresponding to the belly and buttocks of the digital character 4. These two physique elements 12d, 12e have properties (including topography, volume, and mass distribution) that mean that the joint assemblies 8 in the lower abdomen portion of the joint network 6 move in a manner that would be expected of the digital character 4 such that the abdomen of the performance puppet 2 moves and behaves (e.g., reacts to external forces, including the manipulation by a puppeteer) in a substantially realistic manner, which then results in equivalent realistic movement of the digital character 4. In this example, the abdomen of an obese character would be bigger (in terms of both volume and mass) than the abdomen of an underweight character, such that if the two characters were to both to walk forward at a similar pace, the movement of the lower abdomen, belly, and buttocks would have characteristic and different responses between the two characters, which would further inform and influence the gait and upper body movements of each character.

[0083] The physique elements 12 on the performance puppet 2 are designed to reflect this difference in response behaviour such that the movement of the digital character 4 is realistic for the shape and size of the character in question. In addition, the provision of the physique elements 12 helps the puppeteer determine where the boundaries of the physical character 2 are without having to rely on the digital performance.

[0084] Prior art armature systems were designed to control the movement of a digital character, with data processed and modified in real time to generate an on-screen digital performance. Puppeteer real time feedback is based on this on-screen performance. Importantly, in armature systems, the physicality of the controlled physical character does not have to accurately reflect that of the digital character as long as the puppeteers can manipulate the armature of the physical character into positions to make it look like the digital character they are performing is reacting "correctly".

[0085] Movement of some portions of the armature is interpolated by a computing system because the anatomy of the physical rig and the digital character do not have a one-to-one mapping with each other. Putting a skin (including suitable weightings and volumes for the required digital character), onto an armature would result in physical movement that does not fully match the nominal anatomy of the digital character. Additionally, the armature is greater in mass than the joint network 6 of the performance puppet 2 and so the armature does not react to the environment as the nominal physicality of the corresponding digital character would be expected to.

[0086] In contrast, the system described herein focuses on duplicating and digitising the performance, physicality, interactions, and movement of the physical performance puppet 2 onto equivalent movement of a digital twin (i.e. , the digital character). In this system, the puppeteer real time feedback is provided directly to the puppeteer from the physical performance puppet 2, with some reference to the resulting digital performance. The physicality of the physical performance puppet 2 reacts with "correct" movements in response to puppeteer manipulation and interaction as a result of the physique elements 12 placed around the performance puppet 2.

[0087] In particular, the performance puppet 2 moves "correctly" (which can be understood to mean accurately or realistically) under external forces applied to or experienced by the performance puppet 2, including gravity, wind, external contact, etc. The performance puppet 2 having the physicality of the digital character 4 means that the puppeteer is able to tell where the character’s boundaries are without having to watch the digital visualisation. This can help avoid boundary clashes by the puppeteer e.g., the puppeteer moving the character’s hand through a nose, ear, limb, etc of the digital character.

[0088] In summary, the physicality, interaction, and behaviour of a motion armature capture system is not the same as or equivalent to the nominal physicality of the digital character, whereas in the system described herein the physical performance puppet 2 is topographically equivalent to a digital replica such that the physicality and movement of the physical performance puppet 2 has a one-to- one mapping to equivalent movement of the replica digital character 4. In the system described herein, components are as light and small as possible, which reduces injury to puppeteers from prolonged and repetitive use.

[0089] An important aspect of the performance puppet’s 2 ability to accurately and realistically move in response to forces applied to it are the joint assemblies 8 of the joint network 6. In particular it is important that the joint assemblies 8 allow large rotation ranges, preferably in multiple axes, with small joint assembly volumes.

[0090] Various joint assemblies 8 will now be described. It should be noted that these figures are schematics designed to show the type of motion possible in different joint assemblies 8 and are not necessarily accurate representations of joint assemblies 8. Figure 2 shows an exemplary joint assembly 8. The joint assembly 8 comprises at least one individual joint 14 made up of a pair of cylinders 14a, 14b. The example in Figure 2 shows four individual joints 14. Individual joints 14 are connected together using one or more links 16’. The joint assembly 8 also comprises at least one rod 16 arranged to move with the individual joint 14. For reference, in the Figures provided, the shaded cylinders and links 16’ can be considered as “fixed” components whilst the unshaded rods 16 and white cylinders can be considered as “moveable” components. Considering the left most individual joint 15 in Figure 2, which is configured to rotate about a longitudinal axis passing through the pair of cylinders 15a, 15b, the rod 16a and first cylinder 15a move (i.e., rotate) relative to the link 16b and second cylinder 15b which remain stationary relative to the rod 16a and first cylinder 15a. In each individual joint 14, there is a moving portion and a portion that remains stationary relative to that moving portion. It is noted that stationary portions in a first individual joint 14 may move relative to both a stationary and moving portion of a second separate individual joints, as is the case in Figure 4.

[0091] The combination of the individual joints 14, rods 16, and links 16’ can act as a single point of movement on the digital character 4. In other words, a single joint assembly 8 can be represented by a combination of at least one individual joint 14 and at least one link 16’ (where there is more than one individual joint 14). Each joint assembly 8 in the performance puppet 2 corresponds to a single joint in the digital puppet 2 and so the joint assembly 8 acts as a single effective joint. As an example, the shoulder joint in the digital character 4, which is a single point of movement i.e. one “real” joint, can be represented by a joint assembly 8 in the performance puppet 2 comprising three individual joints 14 and corresponding links 16’ between the three individual joints 14, wherein two individual joints provide “hinge” type motion (at right angles to each other) at the shoulder and the other individual joint provides “swivel” rotation type motion at the shoulder.

[0092] The joint assembly 8 of Figure 2, is a multi-axis joint assembly 8 comprising a plurality of axes of rotation. This exemplary joint assembly 8 is an example of a 3- axis joint assembly, which allows for rotational movement around three different axes, namely the x-, y-, and z- axes. In this example, the primary rotational axis is the x-axis allowing full 360-degree rotational movement. The other two axes (i.e. , y- and z- axes) are considered the secondary, non-axial rotation axes, referred to as off-axis rotation, allowing for approximately 240-270 degree rotational movement. The secondary, non-axial rotation axes provide additional rotational movements that are not aligned with the primary rotational axis. The non-axial rotation axes therefore provide additional degrees of freedom and flexibility in positioning or orienting the joint assembly 8. For example, they may enable movements such as tilting, swivelling, or adjusting the orientation of the joint relative to the primary axis of rotation. By combining the primary rotational axis and the two non-axial rotation axes, the 3-axis joint assembly such as that illustrated in Figure 2 can achieve a wide range of positions and orientations, enabling the joint assembly 8 to accurately mimic the dexterity of human-like movements.

[0093] The multi-axis joint assembly 8 in Figure 2 has a single centre of rotation 18, wherein all three axes of rotation intersect with each other at a single point. This can also be seen in Figure 3, which is an expanded view of the joint assembly 8 of Figure 2, more clearly illustrating the individual joints 14, links 16’, and rods 16. The joint assembly shown in Figure 2 comprises two offset joints along the primary x-axis (the left and right most joints), and it can be seen that there is a physical separation between the two x-axis rotation joints and the common centre of rotation 18.

[0094] Throughout the disclosure, the term “off-centre” will generally be used to refer to hinge-type joints (i.e. joints which do not allow rotation about the primary x-axis) being displaced from a common centre of rotation such that there is an effective common centre of rotation. Throughout the disclosure, the term “offset” will generally be used to refer to swivel-type joints (i.e. joints which allow rotation about the primary x-axis) being displaced, along the primary axis of rotation, from the common centre of rotation such that there is an effective common centre of rotation. The difference between off-centre and offset joints is whether the joint in question is displaced along its rotational axis, which in many examples is the primary axis i.e., x-axis (“offset”) or is displaced along at least one of its non- rotational axes, which in many examples are joints rotating on the non-primary axes (aka secondary axes) i.e. y- or z- axes (“off-centre”).

[0095] Both off-centre joint assemblies and offset joint assemblies can collectively be referred to a displaced joint assemblies.

[0096] Although a single centre of rotation provides wide-range, multi-axis rotations around a single centre, it has been found that slightly off-centre joint assemblies provide a more physically robust solution, as well as providing more symmetric movement. Figure 4 shows an exemplary joint assembly 8 comprising an effective centre of rotation 20. This joint assembly 8 comprises three axes of rotation (similar to the joint assembly in Figure 2), but in this example the three axes do not all intersect at a single point. As can be seen, the x-axis and y-axis intersect and the x-axis and z-axis intersect, but the y-axis and z-axis do not intersect.

[0097] In some off-centre joint assemblies 8, there can be a significant displacement between these two rotational centres. It should be noted that in this context “significant” is intended to mean large enough that there is a measurable distance between the two centres of rotation (i.e. the displacement can be measured and assigned a length value) but not so large that it causes the two centres of rotation to be considered as being part of different joints, so the digitised motion of the digital character 4 is not affected. For example, in some arrangements there may be displacements of up to 11 mm from the effective common centre (equivalent to up to 22 mm between two joints).

[0098] The joint assembly in Figure 4 has an effective centre of rotation 20 which can be considered the point about which all movements and rotations originate from. Importantly, Figure 4 utilises a plurality of joint assemblies that are offset from each other (the two x-axis joints), as well as joint assemblies that are off-centre (the y- and z-axis joints).

[0099] The off-centre configuration, such as that in Figure 4, provides joint assemblies 8 which allow large angles of rotation (and a high range of rotation) in multiple axes around an effective common centre point 20, whilst keeping the joint assemblies 8 relatively small in size and weight, which results in a joint assembly 8 that is analogous to a 3-axis ball joint, with multi-axis rotational movement, in a relatively small volume. The off-centre arrangement allows wide movement ranges mimicking realistic anatomical joint movement of e.g. hips, shoulders, heads, rather than relying on approximations from a series of displaced single axis rotation joints which would take up more space.

[0100] Displacing one or more rotation axes from the others still provides for a high amount of dexterity whilst accommodating physical constraints imposed by the joint network 6 and joint assemblies 8. This also provides more room to accommodate multiple transducers 10 in the joint assembly 8. Joints rotating on the primary rotation axis (which may also be referred to as on-axis rotation) can be considerably offset from the multi-axis centre of rotation 18 or the common effective centre point 20. In this context “considerably offset” can include displacements that are large enough to be measurable and noticeable by a human, but not so large that it causes the offset joint to be considered as being part of different joint assemblies and so the digitised motion of the digital character 4 is not affected.

[0101] Having on-axis rotation that is offset from a single multi-axis centre of rotation means that the axis of rotation for the on-axis swivel (i.e. , axial rotation) is only aligned with a single axis of the common centre of rotation for the other axes in the multi-axis joint assembly 8. The joint assembly 8 compensates for this offset and enables accurate measurement of the combined multi-axis rotations around a single centre or effective centre of rotation, while still allowing for a wide range of motion and maintaining a compact and lightweight joint assembly 8. This consideration is important in the design of multi-axis rotation devices to ensure precise control, measurement, and optimization of the overall system's performance and size / weight characteristics.

[0102] The joint assemblies 8 (including the mechanical structures connecting the rotating components together) are designed in a way that allows the transducers 10 to accurately measure the different rotations and at the scale of the digital character 4, the movement of any off-centre joint assemblies 8 is perceived to originate from a single, effective common centre of rotation, despite any displacement or offset of the on-axis rotation in the physical performance puppet 2.

[0103] Various different types of joint assembly 8 can be used within the performance puppet 2, depending on the type of movement a given joint assembly 8 is designed to replicate (e.g. “knees” based on hinge-based joints, “shoulders” based on ball- and-socket joints, etc). The different joint assemblies 8 allow for at least partial rotation about one or more axes. Figures 5a-5j illustrate some of the different types of joint assemblies 8 that may be used within a performance puppet 2.

[0104] Figure 5a illustrates a joint assembly 8 comprising two individual joints 14 which provide on-axis rotation along the x-axis and off-axis rotation in the y-direction. The joint assembly 8 in Figure 5b comprises two individual joints 14 which share a common cylinder. Each of the individual joints 14 provides on-axis rotation about the x-axis. Figure 5c shows a joint assembly 8 comprising one individual joint 14 which allows for off-axis rotation about the y-axis. Figure 5d is similar to Figure 4 and shows a joint assembly 8 comprising 3 individual joints 14 providing rotation about the x-, y-, and z-axes. As with Figure 5a, this joint assembly in Figure 5d provides on-axis and off-axis rotation. Figure 5e illustrates a joint assembly 8 comprising two individual joints 14 which allow off axis rotation about the y- and z-axes.

[0105] Figure 5f illustrates a joint assembly 8 comprising a single central ball joint and three single-axis joints offset from the centre of the ball. The central ball joint is positioned such that the three individual single-axis joint rotations are driven by the respective x, y and z axis rotation components of the central ball joint around its single centre of rotation 18.

[0106] Figure 5g illustrates a joint assembly 8 comprising a single central ball joint and three single-axis joints which are all offset from the centre of the ball. This arrangement is an alternative to that shown in Figure 5f, but the individual single- axis joint rotations are still driven by the respective x, y and z axis rotation components of the central ball joint around its single centre of rotation 18.

[0107] Figure 5h illustrates a joint assembly 8 with a plurality of transducers 10. The joint assembly 8 comprises a single central ball joint and three individual transducers 10 located about the central ball joint and positioned such that the three transducers 10 are able to measure the individual respective x, y and z axis rotation components of the central ball joint around its single centre of rotation 18.

[0108] Figure 5i illustrates a joint assembly 8 with a plurality of transducers 10. The joint assembly 8 comprises a single central ball joint and three individual transducers 10 arranged in a different manner about the central ball joint compared to Figure 5h, However, in this arrangement, the three transducers 10 are still able to measure the individual respective x, y and z axis rotation components of the central ball joint around its single centre of rotation 18.

[0109] Figure 5j illustrates another joint assembly 8 with a plurality of transducers 10. This joint assembly 8 comprises a single central ball joint and three individual transducers 10 located about the central ball joint and positioned such that the three transducers 10 are able to measure the pairs of respective xy, xz and yz double axis rotation components of the central ball joint around its single centre of rotation 18.

[0110] It can be seen that Figures 5a-5c have a single, common centre of rotation 18 whilst Figures 5d-5e have an off-centre configuration with a single effective centre of rotation 20. Figures 5f-5j have a single, common centre 18 of the component x, y, z rotations.

[0111] It should be noted that the joint assemblies 8 in Figures 5a-5j are just some examples of joint assemblies 8 that may be used are not an exhaustive listed. Other joint assemblies may be used which may be formed of combinations of one or more of the joint assemblies shown in Figures 5a-5j. Alternatively or in addition, joint assemblies maybe formed by removing or adding an individual joint to any of the illustrated joint assemblies to form a new joint assembly. Alternatively, or in addition, joint assemblies may be formed by moving an individual single axis joint to a functionally equivalent position on the joint assemblies shown in Figures 5f-g to form a new joint assembly, or by moving, removing, or adding an individual transducer to a functionally equivalent position on the joint assemblies shown in Figures 5f-5i.

[0112] As mentioned above, the joint network 6 is made up of a plurality of joint assemblies 8 and a plurality of transducers 10 configured to provide movement data indicative of a movement of the joint network 6. In particular, each joint assembly 8 in the joint network 6 is associated with at least one transducer 10, depending on how many individual joints 14 make up the joint assembly 8. Preferably there is at least one transducer 10 associated with each individual joint 14 in the joint assembly. In this way, each transducer 10 is configured to measure movement components (e.g., x, y, z components) of a particular joint 14 in the joint assembly 8 and convert the measured movement into digital movement data for processing by an external computing device.

[0113] When the joint assembly 8 is a multi-axis joint assembly having a single common centre of rotation 18, such as that of Figures 2, 3, 5a-c, and 5f-j, the movement data measured by each transducer 10 for each individual joint 14 is associated with the single common centre of rotation 18 of the joint assembly 8. When the joint assembly 8 is a (multi-axis) joint assembly having an off-centre configuration, such as that of Figures 4 or 5d or 5e, the respective transducer(s) considers movement of the individual joint(s) 14 in the joint assembly 8 to originate from an effective common centre of rotation 20 of the joint assembly 8.

[0114] In both cases, when the measured data is converted into digital movement data, the digital movement data defines movement of a single joint of the digital character 4 represented by the joint assembly 8. In other words, the presence of the off-centre configuration in some joint assemblies 8 does not affect how the movement joint network 6 is converted into movement of the digital puppet 4.

[0115] The joint assemblies 8 of the performance puppet 2 allow the rotations around specific rotation centres (either an actual common centre or an effective common centre) of the performance puppet 2, to be digitised by transducers 10 connected to the joint assemblies 8. The translation in world space of one or more datum points associated with the performance puppet 2 may also be digitised, by use of transducers 10, but non-moving datum points are also possible. Any rotations and translations of the joint assemblies 8 in the joint network 6 are converted into movement of equivalent sections of the digital puppet 4.

[0116] In some developments, the joint assemblies 8 making up the joint network 6 do not have to be connected to each other to form a complete skeleton. What is an important requirement for the joint network 6 is that the translational and / or rotational displacements and / or relationships between the various rotation centres of the joint assemblies 8 remains constant between each other. In other words, the various different joint assemblies 8 can be individually attached to the performance puppet 2 but they do not have to be connected or attached to each other. As an example, three separate joint assemblies 8 representing a shoulder, elbow, and wrist joint may be individually attached to a performance puppet 2 in appropriate locations on the performance puppet 2, but these three joint assemblies 8 do not need to be connected to each other. In some examples, a joint network 6 may comprise some joint assemblies 8 that are connected to one or more other joint assemblies as well as some joint assemblies 8 that are not connected to any other joint assembly 8. In some examples, the joint network 6 may only comprise joint assemblies 8 that are not connected to any other joint assembly 8.

[0117] As can be seen, the performance puppet 2 described herein includes a joint network 6 and physique elements 12 (in the form of volumes and masses) which together result in a characteristic movement of the performance puppet 2, with transducers 10 to digitise the movement which is transferred to a digital equivalent of the performance puppet 2.

[0118] The particular construction and arrangement of joint assemblies 8 allows the required network of rotation and translation centres to exist within the volumes and masses of the performance puppet 2, with a sufficiently large range of movement, that the performance puppet 2 can be moved, performed, and interacted with in an equivalent and characteristic manner to, and can physically resemble, a non-digitising puppet. The digital character 4 is able to have a one- to-one mapping of digitised movements from the physical performance puppet 2, with multi-axis rotations around single (or effectively single) centre points of rotation in an anatomically accurate manner, and move as a duplicate of the physical puppet 2.

[0119] The joint network 6 of the performance puppet 2 avoids the need to have a series of single axis rotation joints, that do not combine multiple rotations around single (actual or effective) rotation centres.

[0120] As mentioned above, the performance puppet 2 includes a plurality of physique elements 12 distributed around the joint network 6 arranged to mimic the size, shape, and mass distribution of corresponding physical elements of the digital character 4. The physique elements 12 can include mass elements, volume elements or a combination of both mass and volume elements.

[0121] The physique elements 12, including mass elements and I or volume elements, means that the performance puppet 2 has an anatomy that is topographically equivalent to the anatomy of the digital character 4. In particular, the volumes, distribution of relative masses, translation and rotation centres, and rotation limits for all the various difference sections of the performance puppet 2 together represent the physicality of the digital character 4.

[0122] The use of the term "physicality" is not intended to be specific or limiting about which physical characteristics of the digital character 4 the term refers to. The digital character 4 has a nominal anatomy which can be broken down into, and represented by, different volumes, distribution of relative masses, translation and rotation centres, and rotation limits for different sections of the body, and it is these features and characteristics which forms the character’s “physicality”. These are the features that are important to represent equivalently in the performance puppet 2 because it is these features and characteristics which influence the behaviour and movement of the digital character 4 resulting from the performance puppet’s 2 response to forces within its environment (e.g. forces including but not limited to gravity, contact with surfaces, wind / fluid flow), and manipulation by puppeteers.

[0123] In some situations, it may be desirable to have a more realistic performance puppet 2 including one or more of an outer skin, clothes, an exoskeleton, talons, claws, horns, antlers etc instead of just simple volumes and masses distributed around the joint network 6. As an example, Figure 6 illustrates a performance puppet 2 comprising an outer skin 22 which resembles the outer skin of the digital character 4. In this example, the outer skin 22 at least partly, and in some cases substantially, covers the physique elements 12 such that at least some of, and in some cases substantially all of, the individual physique elements 12 cannot be seen. This more realistic depiction is in contrast with the performance puppet 2 illustrated in Figure 1 which does not have any form of outer skin or covering and uses simple shapes and volumes for the corresponding physique elements 12. The performance puppet 2 in Figure 6 can be thought of as having a more complex outer appearance than the performance puppet 2 in Figure 1 . Regardless of the complexity of the outer appearance of the performance puppet 2, the effect of both types of performance puppet 2 is that the puppeteer and any other performers interacting with the character are able to tell where the boundaries of the performance puppet 2 are and they and any others can see the performance, without having to watch the digital character 4 on a screen.

[0124] Features which may classically fall under “physicality” or “physical characteristics” but that do not impact how a digital character 4 would respond to a force applied to it (such as hair, fur, skin texture, etc) do not need to be equivalently represented in the performance puppet 2.

[0125] It should be noted that whilst the performance puppet 2 has a topography and mass distribution that is equivalent to that of the digital character 4, the scale may be different such that the performance puppet 2 is larger or smaller than the corresponding digital character 4. For some digital characters 4 that are very small in size, e.g. small rodents, it may be beneficial to “scale-up” the performance puppet 2, such that the performance puppet 2 is larger than its digital replica, in order to make it easier for puppeteers to move and interact with the performance puppet 2. Similarly, for some digital characters 4 that are very large in size, e.g. large dinosaurs, it may be beneficial to “scale-down” the performance puppet 2, such that the performance puppet 2 is smaller than its digital replica, in order to make it easier for puppeteers to move and interact with the performance puppet 2.

[0126] We have seen that the performance puppet 2 can be thought of as a combination of contact surface volumes and distributed masses 12, individual (small volume and mass) rotation joint assemblies 8, with connected transducers 10, which creates a puppet character that can be manipulated in a live performance as though it were a non-digitising puppet, with characteristic movement when interacting with the environment or moved under manipulation, which is digitised and transferred to a topographically (and in some cases anatomically) equivalent digital version, for direct visualisation and I or further processing and enhancement.

[0127] The manipulation of the performance puppet 2 can be direct manipulation by the puppeteer with the performance puppet 2, for example by either directly contacting and moving parts of the performance puppet 2 (as shown in Figure 1), or by using one or more contact points 24 (e.g. handles 24) attached to the performance puppet 2 (as shown in Figure 7). For some performance puppets 2, the manipulation by the puppeteer may be a combination of contact points 24 and manipulating the puppet itself (as shown in Figure 7). Figure 7a shows a slight modification to Figure 7 in which the performance puppet 2 additionally comprises transducers in the soles of the feet. These transducers are able to measure data as a result of contact of the performance puppet 2 with a surface such as a floor, rather than measuring joint movement data.

[0128] In some situations, manipulation of the performance puppet 2 does not result from direct manipulation by the puppeteer but could instead be secondary movement generated by interaction of the physique elements 12 with the external environment. For example, if an object collides with a shoulder portion of the performance puppet 2 this may result in subsequent movement of the arms and I or legs. The collision between the object and the performance puppet may be considered direct manipulation resulting in primary movement. The subsequent movement of the arms, legs, or any other portion of the performance puppet as a result of this collision may be considered as simply “manipulation” or indirect manipulation resulting in secondary movement. Within this disclosure, the term “manipulation” is intended to include both direct and indirect manipulation unless otherwise specified.

[0129] Movement created as a result of manipulation of the performance puppet 2 is measured by one or more transducers 10. The performance puppet 2 comprises a communication device, such as a transmitter or transceiver, which is configured to transmit the movement data determined by the transducer(s) 10 to an external computing device for further processing. In some examples, the communication device wirelessly communicates with the external computing device, for example using Wi-Fi. In other examples the communication device may communicate via physical connection with the external computing device.

[0130] In some examples, the communication device is powered using one or more umbilicals to transfer power from a power supply to the communication device. However, in other examples, the performance puppet 2 may comprise one or more power sources, such as a battery, which can be used to at least partially power the communication device.

[0131] As will be appreciated, some modifications can be made to the above-described performance puppet 2 without departing from the inventive concept.

[0132] Figure 4 shows an exemplary joint assembly 8 having one axis of rotation displaced from the other axes of rotation. However, in some joint assemblies all the axes of rotation (e.g. all three axes of rotation) could be displaced from each other. In this case, there would not be a point of intersection between any axes of rotation. What is important is that the joint assembly comprising one or more displaced axes behaves in a manner such that movement of the joint assembly is considered to originate from an effective common centre point and that movement of the performance puppet 2 maps correctly to movement of the digital character 4.

Claims

CLAIMS1. A physical performance puppet for use in generating a digital character, comprising: a joint network comprising a plurality of joint assemblies and a plurality of transducers configured to provide movement data indicative of a movement of the joint network, wherein the movement data defines a movement of the digital character; a plurality of physique elements distributed around the joint network such that the physical performance puppet has a topography and mass distribution equivalent to the digital character, whereby the movement of the digital character has a substantially one-to-one mapping to the movement of the performance puppet; wherein manipulation of the physical performance puppet corresponds to real-time equivalent movement of the digital character.

2. The physical performance puppet of claim 1 , wherein each joint assembly in the joint network is associated with at least one transducer, and wherein the transducer is configured to measure movement of the joint assembly and convert the measured movement of the joint assembly into digital movement data for processing by an external computing device.

3. The physical performance puppet of claim 1 or claim 2, wherein at least one joint assembly in the plurality of joint assemblies is a multi-axis joint assembly comprising a plurality of axes of rotation.

4. The physical performance puppet of claim 3, wherein the multi-axis joint assembly comprises at least two axes of rotation that intersect.

5. The physical performance puppet of claim 3 or 4, wherein the multi-axis joint assembly comprises at least one axis of rotation that does not intersect with any other axis of rotation.

6. The physical performance puppet of any preceding claim, wherein the joint assembly comprises at least two single-axis rotation joints, wherein the at leasttwo single-axis rotation joints are physically separate from each other in space but share a common intersection point of their axes of rotation.

7. The physical performance puppet of claim 6, wherein at least one of the single-axis rotation joints is displaced from a common centre of rotation of the joint assembly.

8. The physical performance puppet of claim 7, wherein at least one of the single-axis rotation joints is displaced along a primary axis from a common centre of rotation of the joint assembly.

9. The physical performance puppet of claim 7 or claim 8, wherein at least one of the single-axis rotation joints is displaced along a secondary axis from a common centre of rotation of the joint assembly.

10. The physical performance puppet of any of claims 6-9, wherein two or more of the single-axis rotation joints are displaced from a common centre of rotation of the joint assembly.

11. The physical performance puppet of any preceding claim, wherein the physique elements comprise mass elements and / or volume elements.

12. The physical performance puppet of any preceding claim, wherein the physical performance puppet comprises an external surface which realistically represents skin of the digital character.

13. The physical performance puppet of any preceding claim, wherein the physical performance puppet comprises an external surface which realistically represents at least part of a costume of the digital character.

14. The physical performance puppet of any preceding claim, wherein the physical performance puppet comprises a transceiver configured to communicate, preferably wirelessly, with an external computing device.

15. The physical performance puppet of claim 14, wherein the transceiver is configured to transmit movement data to an external computing device.

16. The physical performance puppet of claim 15, wherein the physical performance puppet is configured to transfer data wirelessly, preferably using WiFi.

17. The physical performance puppet of any preceding claim, wherein a weight of the physical performance puppet is at least partially supported by suspension of the physical performance puppet from one or more fixed or moveable external point(s).

18. The physical performance puppet of claim 3, wherein the multi-axis joint assembly comprises a ball joint, preferably wherein movement of the ball joint results in the movement of at least one single axis rotation joint.

19. The physical performance puppet of claim 18 wherein the ball joint is associated with at least one transducer, and wherein the at least one transducer is positioned to allow measurement of at least one single component rotation of the ball joint and I or double component rotation of the ball joint and I or triple component rotation of the ball joint.

20. A joint network for use in a physical performance puppet used in generating a digital character, the joint network comprising: a plurality of joint assemblies; a plurality of transducers, wherein each joint assembly in the joint network is associated with at least one transducer, wherein the at least one transducer is configured to measure movement of the joint assembly and convert the measured movement of the joint assembly into digital movement data for processing by an external computing device.

21. The joint network of claim 20, wherein at least one joint assembly comprises at least two single-axis rotation joints, wherein the at least two single-axis rotation joints are physically separate from each other in space but share a common intersection point of their axes of rotation.

22. The joint network of claim 21 , wherein at least one of the single-axis rotation joints is displaced from a common centre of rotation of the joint assembly.

23. The joint network of claim 22, wherein at least one of the single-axis rotation joints is displaced along a primary axis from a common centre of rotation of the joint assembly.

24. The joint network of claim 22 or claim 23, wherein at least one of the singleaxis rotation joints is displaced along a secondary axis from a common centre of rotation of the joint assembly.

25. The joint network of any of claims 21 -24, wherein two or more of the singleaxis rotation joints are displaced from a common centre of rotation of the joint assembly.

26. The joint network of claim 21 , wherein two of the single axis rotation joints have a common axis of rotation.

27. The joint network of claim 20, wherein at least one joint assembly in the plurality of joint assemblies is a multi-axis joint assembly comprising a plurality of axes of rotation.

28. The joint network of claim 27, wherein the multi-axis joint assembly comprises a ball joint, preferably wherein movement of the ball joint results in the movement of at least one single axis rotation joint.

29. A joint assembly for use in a physical performance puppet used in generating a digital character, the joint assembly comprising: at least two single-axis rotation joints, wherein the at least two single-axis rotation joints are physically separate from each other in space but share a common intersection point of their axes of rotation;wherein at least one of the single-axis rotation joints is displaced from a common centre of rotation of the joint assembly; the joint assembly further comprising at least one transducer, wherein the at least one transducer is configured to measure movement of the joint assembly and convert the measured movement of the joint assembly into digital movement data for processing by an external computing device.

30. A joint assembly for use in a physical performance puppet used in generating a digital character, the joint assembly comprising: a multi-axis joint assembly comprising a plurality of axes of rotation; wherein the multi-axis joint assembly comprises a ball joint, wherein movement of the ball joint results in the movement of at least one single axis rotation joint; the joint assembly further comprising at least one transducer, wherein the at least one transducer is configured to measure movement of the joint assembly and convert the measured movement of the joint assembly into digital movement data for processing by an external computing device.

31. A joint assembly for use in a physical performance puppet used in generating a digital character, the joint assembly comprising: at least two single-axis rotation joints, wherein the at least two single-axis rotation joints are physically separate from each other in space but share a common intersection point of their axes of rotation; and wherein at least one of the single-axis rotation joints is displaced from a common centre of rotation of the joint assembly.

32. A joint assembly for use in a physical performance puppet used in generating a digital character, the joint assembly comprising: a multi-axis joint assembly comprising a plurality of axes of rotation; wherein the multi-axis joint assembly comprises a ball joint, wherein movement of the ball joint results in the movement of at least one single axis rotation joint.

Images