Tracking system

Abstract

The invention relates to a tracking system (100) for illuminating a performer (12) at a tracked position (T) of a performance environment (10) with a light beam from a directable light beam generator, comprising: a camera input (120) for receiving a camera stream (125) from a camera arranged to observe the performance environment, where the camera stream represents a 2D view (127) of the performance environment presented by the camera; a display output (130) for outputting a display stream (125); and a controller (110) arranged to: acquire (210) a camera placement (30) of the camera relative to the performance environment; acquiring (215) a camera orientation ([theta]) of the camera relative to the performance environment; acquiring (255) an orientable beam generator placement (20, 20 ') of the orientable beam generator relative to the performance environment; acquiring (260) an orientable beam generator orientation ([theta]) of the orientable beam generator relative to the performance environment; receiving (230) a camera stream from a camera input; acquiring a tracked position; calculating (265) a 2D projection of the light beam at the tracked location based on the tracked location, the camera placement, the camera orientation, the orientable beam generator placement, and the orientable beam generator orientation; inserting (270) a beam indicator (190, 190 ', 191) for indicating the 2D projection of the beam in the camera stream to generate a display stream, and providing (250) the display stream to a display output.

Description

Technical Field

[0001] This invention relates to a tracking system for tracking performers in a performance environment. More specifically, this invention relates to additional indicators in the display stream, and / or the use of a pan-tilt-zoom camera stream provided to the tracking system to improve the tracking of performers. Background Technology

[0002] To enhance the visual impact of a performer, spotlights on stage can be used to place the performer in a beam of light while illuminating little or no of the stage background. Since performers typically move around on stage, the spotlight beam may need to follow their movement. This type of spotlight that follows the performer on stage is called a follow spot. Follow spots can be operated manually.

[0003] Another solution is to track performers on stage using cameras and screens. While this type of system has been known in the art since at least 2012, a more recent disclosure can be found in EP3868137 (A1). EP3868137 (A1) discloses a computer-implemented system and method for controlling changes in the output of media sources and / or other sources, wherein the input and output of the sources and media can be changed based on the acquisition of the physical properties of the target. The target can be acquired passively, actively, visually, remotely, or through a variety of other means, thereby allowing the output of sources, including but not limited to lighting, audio, video, projection, laser, media, mechanical devices, electronic devices, or other types of sources, to change based on location acquisition.

[0004] US2018 / 0292809A1 discloses a follow spot controller and method. The follow spot controller and method provide the following: storing a first set and a second set of separate pan and tilt parameters for each of a plurality of automated lighting devices, and calibrating a 3-D model of the surface of a performance area and the position and mounting orientation of the automated lighting device relative to the performance area based on the stored sets of separate pan and tilt parameters. This follow spot controller is sensed in physical orientation and sends the operator's pan and tilt parameters to one of the automated lighting devices. The operator's pan and tilt parameters are determined based on the physical orientation of the follow spot controller. Individually calculated pan and tilt parameters are sent to each of the remaining automated lighting devices. These individually calculated pan and tilt parameters are determined based on the operator's pan and tilt parameters and the 3-D model.

[0005] US2020 / 184222A1 discloses a method for controlling lighting fixtures in a location using augmented reality on a user device, which displays virtual elements on an image of the lighting fixtures and / or a scene of the location. User input for controlling the lighting fixtures is received via these virtual elements, and the user device sends signals to change the lighting fixtures in the location based on the user input. The virtual elements on the display change to reflect changes in the actual state of the lighting fixtures. Changing the lighting fixtures includes changing the brightness, color, or focus of the light, or changing the position of the lighting fixtures. The virtual elements may include selection boxes surrounding the lighting fixtures, manufacturer data, channel numbers, DMX addresses, diagnostic information, sliders, switches, knobs, buttons, virtual lighting shutters, pan / tilt axes, movable virtual beams, or scene elements.

[0006] US2019 / 364642A1 discloses a user-operated spotlight system and method for illuminating performers on a stage or performance space. The user-operated spotlight system includes a screen for displaying a stage image and a cursor, a screen cursor locator adapted to be operated to move the cursor on the screen, a processor connected to the screen, and a plurality of controllable spotlights connected to the processor, which can be moved by the user moving the cursor on the screen. An advantage of providing such a user-operated spotlight system is that a single user can operate multiple spotlights.

[0007] The disclosed system and method have the following drawbacks: Changes in the mouse cursor on the screen are not proportional to changes in the performer's position on stage, making these changes dependent on the performer's location. This results in unnatural control. Furthermore, changes in the X-direction on the screen do not correspond to comparable changes in the X and Y directions on the stage, as equivalent in magnitude to changes in the Y-direction. Additionally, variations in screen resolution, depending on the camera's position, affect the precision with which the mouse can point to a target location. One or more of these drawbacks become more pronounced when the camera is viewed from a corner of the stage. Summary of the Invention

[0008] The purpose of this invention is to overcome one or more of the disadvantages mentioned above.

[0009] According to a first aspect of the invention, a tracking system for illuminating a tracked position of a performer in a performance environment using a directional beam generator, wherein the performer has a performer height; the tracking system includes: a camera input for receiving a camera stream from a camera arranged to observe the performance environment, wherein the camera stream represents a 2D view of the performance environment presented by the camera; a display output for outputting a display stream; and a controller arranged to: acquire the camera placement of the camera relative to the performance environment; acquire the camera orientation of the camera relative to the performance environment; acquire the tracked position; acquire a height offset based on the performer height; receive the camera stream from the camera input; calculate a reference position based on the tracked position and the height offset; calculate a 2D reference position based on the camera placement, camera orientation, and reference position; insert a reference indicator for indicating the 2D reference position in the camera stream to generate the display stream; and provide the display stream to the display output.

[0010] A tracking system is a system used to track a position within a performance environment. The performance environment includes a set and a stage. The set may include sloping or vertical sections of the performance environment. The set typically includes parts of the performance environment that performers may not or cannot walk on. Typically, the performer is positioned at the tracked location on the stage. The stage may include flat surfaces, raised sections, steps, movable sections (e.g., lifting equipment), and / or sloping sections that performers can typically walk on.

[0011] Tracking systems are typically used to control directional beam generators. A tracking system can be configured to control multiple directional beam generators, such that these beam generators focus or direct at a tracked location from different angles and / or different positions. The tracking system can be arranged to control multiple directional beam generators to track multiple tracked locations in a performance environment. The directional beam generator generates a directional beam of light to illuminate a specific location in 3D space. The directional beam generator can be a follow spot, automated lighting fixture, and / or projection device.

[0012] Camera inputs are arranged to receive camera streams from cameras. Camera inputs can be custom-type or standardized-type inputs, such as SDI (more specifically, SDI via a coaxial cable with a BNC connector), HDMI, Ethernet, USB, or COM ports. Camera streams can conform to custom or standardized protocols. Camera streams can conform to video streaming protocols such as SMPTE, H.264, MPEG, ONVIF, HLS, RTMP, WebRTC, SRT, RTSP, or MPEG-DASH. The camera arrangement providing streams to camera inputs is for observing the performance environment. The camera stream represents a 2D view of the performance environment presented by the camera. The camera stream can present a view of the stage, such as from a tilted perspective. The camera stream can present the performance environment without presenting the stage area.

[0013] The display outputs are arranged to display a streaming output. The display outputs can be custom-type inputs or standardized-type inputs, such as HDMI, Ethernet, USB, or COM ports. The streaming output can be based on custom or standardized protocols. The streaming output can be based on video streaming protocols such as H.264, MPEG, ONVIF, HLS, RTMP, WebRTC, SRT, RTSP, or MPEG-DASH. The monitors that receive the streaming outputs are arranged to display the streaming output.

[0014] The controller is configured to perform multiple steps. The controller can be a custom controller or integrated into, for example, a general-purpose computer with appropriate processing capabilities to accept and generate different streams. Unless other requirements specify that the steps must be performed in a particular order, the controller is configured to execute the steps in parallel and / or in a different order. The steps the controller is configured to perform are: acquiring camera placement; acquiring camera orientation; acquiring the tracked position; acquiring height offset; receiving camera streams; calculating a reference position; calculating a 2D reference position; inserting a reference indicator; and providing a display stream.

[0015] Camera placement refers to the orientation of the camera relative to the performance environment. Typically, a reference point and / or reference axis is set up somewhere in the performance environment, especially on the stage. For example, the center of the stage or the middle of the front edge of the stage facing the audience can be chosen as the reference point. The reference axis can be selected as follows: the X-axis is parallel to the front of the audience / stage and points to the right of the audience; the Y-axis extends directly from the stage away from the audience and perpendicular to the stage; and the Z-axis extends vertically upwards from the stage.

[0016] Camera orientation is the direction in which a camera faces relative to the performance environment. Camera orientation is typically specified relative to a reference axis. It can be expressed using translation and tilt values.

[0017] The tracked position is the location illuminated by the beam of a directional beam generator. The beam of a directional beam generator typically has a centerline. The centerline can be the axis of symmetry of the beam or a partial axis of symmetry. The centerline usually coincides with the tracked position. The beam is typically conical, and the centerline is usually the line of symmetry of the cone. The tracked position is typically not positioned directly above the performer's head, but rather slightly below it. This provides illumination of the performer's face or head. This can further provide partial illumination of the performer's shoulders. Furthermore, this minimizes the amount of light from the directional beam generator that does not reach the performer. Minimizing the amount of light that does not reach the performer has the advantage of increasing the contrast between the performer (especially the performer's face) and the performance environment (e.g., the stage and / or set). Increased contrast provides the advantage of improving the performer's presentation within the performance environment, particularly improving the performer's presentation on stage.

[0018] The height offset is based on the performer's height. The height offset is typically or based on the distance between the tracked position and the performance environment (e.g., a stage). The height offset is slightly below the performer's height to provide the added contrast benefit described above. The performer's height is the height from their toes to the top of their head when they are standing upright.

[0019] The reference position is based on the tracked position and the height offset. In other words, the reference position, the tracked position, and the height position are interrelated. Typically, the third can be derived when two of the three are known. The reference position is usually where the performer's feet touch the stage, more precisely, the midpoint between the feet, or the point where the vertical line of the performer's center of gravity intersects the stage.

[0020] A 2D reference position is based on camera placement, camera orientation, and a reference location. Based on camera placement and orientation, the reference position can be converted to a 2D reference position by projecting the 3D performance environment onto a 2D display surface. For example, when the reference position is represented by a 3D vector and the 2D reference position by a 2D vector, the conversion can be represented by multiplying the reference position by a 3×2 matrix to obtain the 2D reference position. The 3×2 matrix is ​​based on camera placement, camera orientation, and the reference location. Object placement can include the object's pose, such as angular deviations or rotations of the Cartesian coordinate axes relative to the origin at the object's specified location, and / or the object's position, such as the object's X, Y, Z position relative to another location (typically the origin of the Cartesian coordinate axes). Object orientation typically includes two of the three axes that determine the object's pose. Object orientation can include the object's tilt and translation values.

[0021] A reference indicator is used to indicate a 2D reference position in a camera stream to generate a display stream. Reference indicators can have any shape, form, and / or size. Reference indicators are typically detectable by the operator of the tracking system. The reference indicator provides the operator with reference information. This reference information is relative to the tracked position. The reference position is usually associated with the stage where the performer is located, providing a reference or projection of the tracked position onto the stage.

[0022] When using a directional beam generator to track a performer, the operator of the tracking system may find it difficult to manipulate the tracked position to follow or track the performer because the tracked position is a floating point in the 3D space of the performance environment.

[0023] Current tracking systems advantageously allow operators to associate a reference point with a point in space, typically on the stage where the performer is positioned, such as the location of the performer's feet on the stage. As the tracked position is manipulated, the associated reference position changes accordingly, thus advantageously enhancing intuitive control over the tracked position. Practice has shown that without a reference indicator, operators are more likely to overshoot and / or undershoot when tracking a performer. This intuitive control over the tracked position becomes even more pronounced when the stage has varying heights, such as sloping surfaces or stepped sections. When overshoot or undershoot occurs while tracking a performer, the tracked position and therefore the beam of the directional beam generator may change in the Z-direction due to the varying heights on the stage, resulting in the performer being unilluminated or partially unilluminated.

[0024] Improved tracking of the performer reduces overshoot and / or undershoot, allowing for a reduction in the cross-sectional size of the beam from the directional beam generator. This reduced cross-sectional size, while still illuminating the performer, provides the advantage of illuminating the performance environment with less light. As previously mentioned, illuminating the performer's body, rather than or minimally illuminating the performance environment, provides contrast. The reduction in the amount of light illuminating the performance environment has the technical effect of increasing the contrast between the performer and the environment.

[0025] According to another aspect of the invention, a tracking component includes: a tracking system according to any embodiment or other aspect of the invention; a camera capable of being coupled to a camera input of the tracking system; and a display capable of being coupled to a display output of the tracking system. The tracking component provides the same advantages as described with respect to other aspects or embodiments of the invention.

[0026] According to another aspect of the invention, a computer-implemented method for illuminating a performer at a tracked position in a performance environment using a directional beam generator, wherein the performer has a performer height, the method comprising: acquiring a camera placement of cameras arranged for observing the performance environment, wherein the camera placement is relative to the performance environment; acquiring the camera orientation of the cameras relative to the performance environment; acquiring the tracked position; acquiring a height offset based on the performer height; receiving a camera stream from the cameras, wherein the camera stream represents a 2D view of the performance environment presented by the cameras; calculating a reference position based on the tracked position and the height offset; calculating a 2D reference position based on the camera placement, camera orientation, and reference position; inserting a reference indicator for indicating the 2D reference position in the camera stream to generate a display stream; and providing the display stream to a display. This computer-implemented method provides the same advantages as described with respect to other aspects or embodiments of the invention.

[0027] According to another aspect of the invention, a tracking system for illuminating a performer at a tracked location in a performance environment using a directional beam generator includes: a camera input for receiving a camera stream from a camera arranged for observation in the performance environment, wherein the camera stream represents a 2D view of the performance environment presented by the camera; a display output for outputting a display stream; and a controller arranged to: acquire the camera placement of the camera relative to the performance environment; acquire the camera orientation of the camera relative to the performance environment; acquire the directional beam generator placement of the directional beam generator relative to the performance environment; acquire the directional beam generator orientation of the directional beam generator relative to the performance environment; receive the camera stream from the camera input; calculate a 2D projection of a beam based on the camera placement, camera orientation, directional beam generator placement, and directional beam generator orientation; insert a beam indicator for indicating the 2D projection of the beam in the camera stream to generate the display stream; and provide the display stream to the display output.

[0028] In this aspect of the invention, the controller is configured to perform the following steps: obtaining camera placement; obtaining camera orientation; obtaining directional beam generator placement; obtaining directional beam generator orientation; receiving camera stream; calculating 2D projection; inserting beam indicator; and providing display stream.

[0029] Oriented beam generator placement refers to the orientation of the directional beam generator relative to the performance environment. Typically, a reference point and / or reference axis is set up somewhere in the performance environment, particularly on the stage. This reference point and / or reference axis is the same for both the camera and the directional beam generator.

[0030] The orientation of a directional beam generator is the orientation of the directional beam generator relative to the performance environment. The orientation of a directional beam generator is typically specified relative to a reference axis. The orientation of a directional beam generator can be expressed using translation and tilt values.

[0031] The 2D projection of a light beam is based on camera placement, camera orientation, and the placement and orientation of the directional beam generator. By projecting the 3D performance environment onto a 2D display surface, the 3D representation of the light beam can be converted into a 2D beam. For example, when a light beam is represented by a set of 3D vectors and a 2D light beam is represented by a set of 2D vectors, the conversion can be achieved by multiplying the set of 3D vectors by a 3×2 matrix to obtain the 2D light beam.

[0032] A beam indicator is used to indicate the 2D projection of a beam in a camera stream to generate a display stream. Beam indicators can have any shape, form, and / or size. Beam indicators are typically detectable by the operator of the tracking system. The beam indicator provides the operator with reference information about the actual position of the beam.

[0033] When using a directional beam generator to track a performer, the operator of the tracking system may find it difficult to manipulate the tracked position to follow or track the performer because the tracked position is a floating point in the 3D space of the performance environment.

[0034] Current tracking systems advantageously allow operators to associate a beam indicator with a point in space, typically relative to the stage where the performer is positioned, such as the portion of the performer illuminated by the beam. As the tracked position is manipulated, the associated beam indicator changes accordingly, thus advantageously improving intuitive control over the tracked position. Practice has shown that without a beam indicator, operators are more likely to overshoot and / or undershoot when tracking a performer. This intuitive control over the tracked position becomes even more pronounced when the stage has varying heights, such as sloping surfaces or stepped sections. When overshoot or undershoot occurs while tracking a performer, the tracked position and therefore the beam of the directional beam generator may change in the Z-direction due to the varying heights within the stage, resulting in the performer being unilluminated or partially unilluminated.

[0035] Improved tracking of the performer reduces overshoot and / or undershoot, allowing for a reduction in the cross-sectional size of the beam from the directional beam generator. This reduced cross-sectional size, while still illuminating the performer, provides the advantage of illuminating the performance environment with less light. As previously mentioned, illuminating the performer's body, rather than or minimally illuminating the performance environment, provides contrast. The reduction in the amount of light illuminating the performance environment has the technical effect of increasing the contrast between the performer and the environment.

[0036] Furthermore, inserting a beam indicator into the display stream provides the advantage of showing where the beam will be when the light from the directional beam generator is off. With the help of the beam indicator, the operator gains perception of the effect and / or illuminated area when the directional beam generator may be turned off. Therefore, the beam indicator has the technical effect of allowing the operator to improve tracking of the performer before the beam of the directional beam generator is turned on, and more specifically, to improve the localization of the tracked position.

[0037] Furthermore, inserting a beam indicator into the display stream provides the advantage of showing where the beam will be in a simulated performance environment. In a simulated environment, the beam indicator can advantageously allow operators to practice using a tracking system and / or allow light designers to refine their expectations of the beam effect of the directional beam generator. The beam effect can include practicing in the simulation to prevent illumination of certain areas or objects that should remain dark, receiving no light from the beam or at least receiving as little light as possible from the beam.

[0038] According to another aspect of the invention, a computer-implemented method for illuminating a tracked position of a performer in a performance environment using a beam from a directional beam generator includes: acquiring a camera placement for observing the performance environment using cameras, wherein the camera placement is relative to the performance environment; acquiring the camera orientation of the cameras relative to the performance environment; acquiring the directional beam generator placement relative to the performance environment; acquiring the directional beam generator orientation relative to the performance environment; receiving a camera stream from the cameras, wherein the camera stream represents a 2D view of the performance environment presented by the cameras; calculating a 2D projection of the beam based on the camera placement, camera orientation, directional beam generator placement, and directional beam generator orientation; inserting a beam indicator to indicate the 2D projection of the beam in the camera stream to generate a display stream; and providing the display stream to a display. This computer-implemented method provides the same advantages as described with respect to other aspects or embodiments of the invention.

[0039] According to another aspect of the invention, a tracking system for tracking a performer located at a tracked position in a performance environment using a camera arranged for observing the performance environment, comprising: a camera command output for controlling the camera; a position tracker input for receiving a change in the tracked position from the position tracker; and a controller arranged to: acquire the camera placement of the camera relative to the performance environment; acquire the camera orientation of the camera relative to the performance environment; acquire the tracked position; receive a change in the tracked position from the position tracker input; change the tracked position based on the change in the tracked position; calculate the camera orientation relative to the performance environment based on the tracked position; and provide the camera orientation to the camera command output for orienting the camera such that the camera follows the tracked position.

[0040] Camera command outputs are used to control cameras that can be coupled to them. Camera control includes controlling the camera's orientation. Camera orientation is typically expressed as tilt and pan values. Camera command outputs can typically transmit commands including tilt and / or pan settings. Camera command outputs can be custom-type outputs or standardized type outputs, such as HDMI, Ethernet, USB, or COM ports.

[0041] A position tracker input is used to receive tracked position changes from the position tracker. The position tracker input can be coupled to the position tracker. The position tracker input is arranged to receive tracked position changes from the position tracker. The position tracker typically identifies changes as relative changes. Alternatively, the position tracker can detect absolute changes. Tracked position changes from the position tracker can be read to include absolute changes (e.g., absolute position). The position tracker can be a mouse, and the position tracker input can be a mouse console input. Alternatively, the position tracker can be a device placed on a performer that generates wireless signals to track the performer's position on stage. Alternatively, the position tracker can be a device placed on a performer that generates invisible signals (e.g., infrared signals) to track the performer's position on stage.

[0042] In this aspect of the invention, the controller is configured to perform the following steps: acquiring camera placement; acquiring camera orientation; acquiring tracked position; receiving changes in tracked position; changing tracked position; calculating camera orientation; and providing camera orientation.

[0043] The tracked position change is received from the position tracker input. The tracked position change is typically associated with a change input to the position tracker. A change provided by the position tracker could be moving the position tracker a distance across a surface. Such a change could be manipulating a joystick and / or trackball.

[0044] The tracked position is changed based on the change in the tracked position. This change in tracked position is typically scaled and / or added to the tracked position. The change in tracked position may also be rotated before being added and / or scaled. For example, the change in tracked position might indicate that the position tracker changes by 10 millimeters in a specific direction relative to the position tracker's reference axis. The change in the tracked position relative to the reference axis in the relevant specific direction could be 1 meter.

[0045] Camera placement refers to the orientation of the camera relative to the performance environment. Typically, a reference point and / or reference axis is positioned somewhere within the performance environment, more specifically, somewhere on the stage. For example, the center of the stage or the middle of the front of the stage facing the audience could be chosen as the reference point. The reference axis can be selected as follows: the X-axis parallel to the front of the audience / stage and pointing to the right of the audience; the Y-axis pointing directly from the stage away from the audience and also directly towards the stage; and the Z-axis extending vertically upwards from the stage.

[0046] Camera orientation is the direction in which a camera faces relative to the performance environment. Camera orientation is typically specified relative to a reference axis. Camera orientation can be represented by translation and tilt values. It is calculated based on the tracked position relative to the performance environment. Camera orientation is further based on camera placement. Camera orientation can be considered as the orientation and / or direction of a vector that starts at the camera's placement position and ends at the tracked position. This camera vector's orientation and / or direction can be represented by the translation and tilt settings of the camera controlled by the tracking system.

[0047] A camera can track a location being tracked, and thus track a performer. When a camera tracks a performer, its field of view does not need to cover the entire performance environment, typically not the entire stage. Choosing to narrow the field of view provides the advantage of more precise tracking of performers on stage. Narrowing the field of view also allows the camera to zoom in on the tracked location to provide a higher-resolution image of the portion of the performance environment where the tracked location is located, thereby providing a higher-resolution image of the performer.

[0048] According to another aspect of the invention, a computer-implemented method for tracking a performer located at a tracked position within a performance environment using cameras arranged for observing the performance environment includes: acquiring the camera placement of the cameras relative to the performance environment; acquiring the camera orientation of the cameras relative to the performance environment; acquiring the tracked position; receiving a change in the tracked position from a position tracker; changing the tracked position based on the change in the tracked position; calculating the camera orientation relative to the performance environment based on the tracked position; and providing the camera orientation to the cameras for orienting the cameras such that the cameras follow the tracked position. This computer-implemented method provides the same advantages as described with respect to other aspects or embodiments of the invention.

[0049] According to another aspect of the invention, a tracking system for illuminating a tracked position of a performer in a performance environment using a beam from a directional beam generator, the tracking system comprising: a position tracker input for receiving changes in the tracked position from the position tracker; a directional beam generator command output for controlling the directional beam generator; a camera input for receiving a camera stream from a camera arranged to observe the performance environment, wherein the camera stream represents a 2D view of the performance environment presented by the camera; a display output for outputting a display stream; and a controller arranged to: acquire the directional beam generator placement relative to the performance environment; acquire the camera placement relative to the performance environment; acquire the camera orientation relative to the performance environment; acquire the tracked position; and acquire a height offset based on the performer's performer height, wherein the height offset indicates the tracked position relative to the performance environment. The system calculates the target position relative to the performance environment based on the performer's height, where the target offset indicates the height difference between the performer's head and the performance environment; receives tracked position changes from position tracker input; changes the tracked position based on the tracked position changes and the height offset; calculates the orienting of the orienting beam generator relative to the performance environment based on the orienting beam generator placement and the tracked position; provides the orienting beam generator orientation to the orienting beam generator command output for orienting the orienting beam generator so that it illuminates the tracked position in the performance environment; calculates the target position based on the tracked position and the target offset; calculates the 2D target position based on the camera placement, camera orientation, and target position; receives a camera stream from camera input; inserts a target indicator to indicate the 2D target position in the camera stream to generate a display stream; and provides the display stream to the display output.

[0050] The directional beam generator command output can be coupled to the directional beam generator. The directional beam generator command output is arranged to control the directional beam generator. The directional beam generator is typically oriented using tilt and translation settings.

[0051] In this aspect of the invention, the controller is configured to perform the following steps: acquiring the position of an orientable beam generator; acquiring camera placement; acquiring camera orientation; acquiring the tracked position; acquiring height offset; acquiring target offset; receiving the tracked position; calculating the orientable beam generator orientation; providing the orientable beam generator orientation; calculating the target position; calculating the 2D target position; receiving a camera stream; inserting a target indicator; and providing a display stream.

[0052] The height offset is based on the performer's height, indicating the difference between the tracked position and the performance environment. The target offset is based on the performer's height, indicating the height difference between the performer's head and the performance environment. The tracked position is changed based on the tracked position change and the height offset. This tracked position change is typically scaled and / or added to the tracked position. The tracked position change may also be rotated before being added and / or scaled. For example, the tracked position change might indicate that the position tracker changes by 10 mm in a specific direction relative to the position tracker's reference axis. The change in the tracked position relative to the reference axis of the tracked position in the relevant specific direction could be 1 meter. Typically, the height offset is added to or subtracted from the tracked position in a predefined direction (e.g., vertically).

[0053] Based on the placement of the directional beam generator and the position being tracked, the orientation of the directional beam generator relative to the performance environment is calculated. The orientation of the directional beam generator can include translation and tilt values. This calculation can be viewed as involving two steps. Step 1: Calculate a vector in 3D space that starts at the directional beam generator position and terminates at the tracked position. Step 2: Calculate the translation and tilt values ​​of the directional beam generator to orient it, thereby directing the beam to the tracked position.

[0054] The directional beam generator is directed to the directional beam generator command output for orienting the directional beam generator so that it illuminates the tracked position in the performance environment.

[0055] Target position calculation is based on the tracked position and target offset. The target position is typically slightly higher than the performance environment (e.g., stage). 2D target position calculation is based on camera placement, camera orientation, and target position. The target offset is typically added to or subtracted from the tracking position in a predetermined direction (e.g., vertically). A target indicator is inserted to indicate the 2D target position in the camera stream to generate the display stream. The target indicator can have any shape, form, and / or size.

[0056] The tracked position is the location illuminated by the beam of a directional beam generator. The beam of a directional beam generator typically has a centerline. The centerline can be the axis of symmetry of the beam or a partial axis of symmetry. The centerline usually coincides with the tracked position. The beam is typically conical, and the centerline is usually the line of symmetry of the cone. The tracked position is typically not positioned directly above the performer's head, but rather slightly below it. This provides illumination of the performer's face or head, and often also illuminates the shoulders and / or part of the body. This minimizes the amount of light from the directional beam generator that does not reach the performer. Minimizing the amount of light that does not reach the performer has the advantage of increasing the contrast between the performer (especially the performer's face) and the performance environment (e.g., the stage and / or set). Increased contrast provides the advantage of improving the performer's presentation within the performance environment, particularly improving the performer's presentation on stage.

[0057] Furthermore, the operator uses a target indicator to change the tracked position, positioning the target indicator above the performer's head or face. The performer's head or face is easier to track than a typical tracking position around the performer's throat. Therefore, positioning the target indicator in an easily identifiable area of ​​the performer, while simultaneously positioning the beam at the tracked position, illuminates the performer while minimizing background illumination. This provides the technical effect of allowing the operator to track the performer more efficiently (e.g., more accurately, with less undershoot and / or overshoot), thus improving contrast. In theatrical performances, typically only the head or face is illuminated. Alternatively, the tracked position can be chosen so that the entire performer is within the beam. This alternative is commonly used in rock performances. Another advantage is that keeping the target indicator above the performer's head provides a relatively easily identifiable and / or trackable target, while maintaining the tracked position so that a portion or the entire performer is illuminated. In another alternative embodiment, the target indicator identifies the 2D position in the display stream of the tracked position. In this alternative, the tracking position and target position are identical in 3D, and therefore identical in 2D as well. Furthermore, in this specific case, the centerline of the beam coincides with the tracked position, and thus with the target position.

[0058] According to another aspect of the invention, a computer-implemented method for illuminating a tracked position of a performer in a performance environment using a beam from a directional beam generator includes: acquiring the directional beam generator placement relative to the performance environment; acquiring the camera placement relative to the performance environment; acquiring the camera orientation relative to the performance environment; acquiring the tracked position; acquiring a height offset based on the performer's height, wherein the height offset indicates the difference between the tracked position and the performance environment; acquiring a target offset based on the performer's height, wherein the target offset indicates the height difference between the performer's head and the performance environment; receiving a change in the tracked position from a position tracker; and based on the change in the tracked position and... The method involves: adjusting the height offset to change the tracked position; calculating the orientation of the directional beam generator relative to the performance environment based on the placement of the directional beam generator and the tracked position; providing the directional beam generator orientation for orienting the directional beam generator such that it illuminates the tracked position in the performance environment; calculating the target position based on the tracked position and the target offset; calculating the 2D target position based on the camera placement, camera orientation, and the target position; receiving a camera stream from cameras arranged to observe the performance environment, wherein the camera stream represents a 2D view of the performance environment presented by the cameras; inserting a target indicator to indicate the 2D target position in the camera stream to generate a display stream; and providing the display stream to a display. This computer-implemented method provides the same advantages as described with respect to other aspects or embodiments of the invention.

[0059] According to another aspect of the invention, a computer program product includes instructions that, when executed by a suitable processor, cause the processor to perform the methods described in claims, embodiments, or the specification. This computer program product provides the same advantages as described with respect to other aspects or embodiments of the invention.

[0060] Detailed Implementation of Exemplary Examples

[0061] In embodiments of the tracking system, a reference indicator advantageously indicates the area of ​​the performer's feet within the performance environment. The reference indicator may be a line on a screen indicating the distance between the feet. The reference indicator may be circular or elliptical, used to indicate the area where the performer's feet may or should contact the performance environment (e.g., the stage). In another embodiment of the tracking system, the reference position advantageously coincides with the performance environment.

[0062] In embodiments of the tracking system, the reference indicator is a surface, such as a square or a circle. In another embodiment, the surface of the reference indicator is aligned with or coincides with a portion of the performance environment (e.g., the stage). Furthermore, the surface of the reference indicator is preferably presented in a perspective view in the display stream. In yet another embodiment of the tracking system, the reference indicator is a line segment, wherein preferably, the line segment is displayed in the display stream as a horizontal line segment.

[0063] In an embodiment of the tracking system, the controller is arranged to: calculate the target position based on the tracked position; calculate the 2D target position based on camera placement, camera orientation, and the target position; and insert a target indicator to indicate the 2D target position in the display stream. Distinguishing the target position from the tracked position provides the advantage that the operator can track the more easily tracked head of a performer by placing the target indicator above the performer's head, while the tracked position and thus the beam of light are positioned such that a portion or all of the performer is illuminated while the background remains unilluminated, thereby enhancing the contrast between the portion or all of the performer and the background.

[0064] In embodiments of the tracking system, the target indicator is a crosshair. The crosshair can advantageously leave space in the middle so that the performer's face or head is not obstructed by the target indicator and remains visible.

[0065] In embodiments of the tracking system, the controller is arranged to insert an association indicator for associating a reference indicator with a target indicator in the display stream. The association indicator may be a line segment connecting the reference indicator and the target indicator. Visually associating the reference indicator with the target indicator provides the advantage of reducing the likelihood of operator confusion, typically when tracking multiple performers on the same display.

[0066] In one embodiment, the tracking system includes a position tracker input for receiving a tracked position change from the position tracker; wherein the controller is arranged to: receive the tracked position change from the position tracker input; and change the tracked position based on the tracked position change, thereby providing the advantage of actively following the performer.

[0067] In another embodiment of the tracking system, changing the tracked position involves changing the tracked position on a surface parallel to the performance environment and based on the change in the tracked position. If the performance environment is flat, the surface is a plane parallel to the performance environment. If the performance environment (such as a stage) has stepped or sloping areas, the parallel surface follows the stepped or sloping areas. In this way, less light beam is allowed to fall on the background, thereby advantageously enhancing the contrast between the performer and the background. In another embodiment of the tracking system, the change in the tracked position is advantageously performed such that the reference position remains aligned with the performance environment. In another embodiment of the tracking system, the controller is arranged to: retrieve a model of the performance environment; and the change in the tracked position is also advantageously based on the model of the performance environment.

[0068] In another embodiment of the tracking system, changing the tracked position includes altering the tracked position vertically based on a model of the performance environment. This can be advantageous when the performer bends over or jumps. It is particularly advantageous when multiple directional beam generators follow the same tracked position from different angles. In another embodiment of the tracking system, changing the tracked position vertically includes maintaining the tracked position at a vertical height offset relative to the performance environment. This embodiment is also advantageous for performance environments with vertically moving elements (e.g., lifts) on which the performer may be located.

[0069] In embodiments of the tracking system, the tracked position is a 3D position. The 3D position can be represented using Cartesian, polar, or spherical coordinates. The 2D position can be represented using Cartesian or polar coordinates. The 3D direction can be represented using translation and tilt settings. The 3D direction can be represented using 3D vectors.

[0070] In embodiments of the tracking system, calculating the 2D projection of the beam includes: virtually arranging a shape that at least partially represents the beam based on the placement and orientation of the directional beam generator; and calculating the 2D projection of the shape based on the placement and orientation of the camera. The beam shape, which at least partially represents the beam of the directional beam generator, can have any shape, advantageously providing the operator with information about the beam, such as information about its location and / or what is being illuminated. The shape may include showing an illuminated surface, such as a portion of an illuminated performer and / or a portion of the performance environment. The shape may also include at least partially showing the beam between the directional beam generator and the surface illuminated by the light.

[0071] In another embodiment of the tracking system, the shape is an extended shape with an extending axis; and the tracked position is located on this extending axis. This extending axis is typically the central axis and / or axis of symmetry of the beam. Furthermore, the tracked position is typically a point on the central axis and / or axis of symmetry. In another embodiment of the tracking system, the shape is advantageously a symmetrical shape with an axis of symmetry; and the tracked position is located on the axis of symmetry. In another embodiment of the tracking system, the shape is a cone, a truncated cone, a cylinder, a pyramid, or a truncated pyramid. This shape generally advantageously mimics the shape of the beam to improve the operator's insight into the beam's effect.

[0072] In another embodiment of the tracking system, the shape has a volume; and the tracked position lies within this volume. Therefore, the shape advantageously displays and / or identifies the tracked position. In another embodiment, an indicator is inserted into the display stream to identify the tracked position adjacent to the beam indicator. In another embodiment of the tracking system, the shape at least partially coincides with the boundary of the beam. Therefore, the shape advantageously displays and / or identifies the boundary of the beam, providing the operator with improved information about the beam. In another embodiment of the tracking system, the shape includes an edge, and this edge advantageously coincides with the edge of the beam.

[0073] In another embodiment of the tracking system, the shape is spherical, having a center equal to the tracked location and a radius such that the sphere and the beam share a circle. This shape advantageously provides the operator with insight into which part of the performer (typically located at the tracked location) is illuminated.

[0074] In one embodiment of the tracking system, the controller is arranged to: acquire the tracked position; and calculate a 2D projection also based on that tracked position. Since the performer is typically located at or near the tracked position, the 2D projection can provide improved insight into which part of the performer is illuminated by the beam. In another embodiment of the tracking system, calculating the 2D projection of the beam advantageously includes calculating the 2D projection at the tracked position.

[0075] In embodiments of the tracking system, computational 2D projection involves virtually projecting a beam onto the performance environment. Virtual projection offers the advantage of showing where the beam will be when the light from the directional beam generator is turned off. With the aid of virtual projection of the beam indicator, the operator gains a perception of the effect and / or illuminated area when the directional beam generator may be turned off. Therefore, virtual projection has the technical effect of allowing the operator to improve the tracking of the performer before the beam of the directional beam generator is turned on, more specifically, to improve the localization of the tracked position. Furthermore, virtual projection offers the advantage of displaying the beam position when simulating a performance environment. In a simulated environment, the beam indicator can advantageously allow the operator to practice the tracking system and / or allow the light designer to anticipate improvements in the beam effect of the directional beam generator.

[0076] In another embodiment of the tracking system, the controller is arranged to: acquire a model of the performance environment; and calculate the 2D projection of the light beam based on the model of the performance environment. The model of the performance environment may include a stage surface. The model of the performance environment may include a set and / or background. The model may include a grid structure for describing the stage surface, set, and / or background. Advantageously, the model specifies that the light beam falling on the performance environment but not illuminating the performer is displayed in the display stream.

[0077] In embodiments of the tracking system, the controller is arranged to: acquire the optical parameters of the directional beam generator; and calculate the 2D projection of the beam based on these optical parameters. The optical parameters of the directional beam generator may include zoom parameters, focus parameters, aperture parameters, shutter blade parameters, and / or shutter parameters. Focus parameters typically affect or control the lens setup in the directional beam generator to adjust the beam focus. Zoom parameters typically affect or control the opening angle of the luminaire or directional beam generator. Aperture parameters typically affect the aperture size of the directional beam generator to adjust the amount of light from the directional beam generator. Shutter blade parameters typically affect the shutter blades or frame blades of the directional beam generator to adjust the beam shape. A beam indicator with improved beam representation, more specifically, a 2D projection of the beam, advantageously provides the operator with more information to improve the arrangement of the beam and / or the tracked position relative to the performer.

[0078] In one embodiment of the tracking system, the controller is arranged to: acquire the placement of a second directional beam generator relative to the performance environment, wherein the second directional beam generator illuminates the performer located at the tracked position using a second beam; acquire the orientation of the second directional beam generator relative to the performance environment; calculate a 2D projection of the second beam based on camera placement, camera orientation, the placement of the second directional beam generator, and the orientation of the second directional beam generator; and insert a second beam indicator to indicate a second 2D projection of the second beam in the display stream. This embodiment allows for simplified tracking using multiple directional beam generators. In another embodiment, the beam indicator and / or the second beam indicator can be turned on and off independently to advantageously allow the operator to obtain improved perception of what is illuminated by the beam and / or the second beam, thereby improving the tracking of the performer. In another embodiment of the tracking system, the beam indicator is different from the second beam indicator. This embodiment provides the advantage of having two indicators in the display stream, while the operator can distinguish between the two indicators, allowing the beam effects to be presented either separately or in combination.

[0079] In an embodiment of the tracking system, the controller is configured to: acquire the tracked position; acquire a height offset based on the performer's height; calculate a reference position based on the tracked position and the height offset; calculate a 2D reference position based on camera placement, camera orientation, and the reference position; and insert a reference display to indicate the 2D reference position in the display stream. This embodiment advantageously combines the insertion of a reference indicator with a beam indicator to enhance their respective advantages and / or improve the tracking of the performer.

[0080] In embodiments of the tracking system, calculating the camera orientation includes applying a control loop that keeps the tracked position within the field of view. The control loop provides the advantage of smoothly following the tracked position. The control loop may include a PID controller.

[0081] In embodiments of the tracking system, calculating the camera orientation includes applying a camera control loop that keeps the tracked position centered in the field of view. The control loop provides the advantage of smoothly following the tracked position. The control loop may include a PID controller.

[0082] In an embodiment of the tracking system, the controller is arranged to acquire a threshold for the change in the tracked position; and to calculate the camera orientation, including maintaining the camera orientation unchanged when the change in the tracked position is below the threshold. This threshold provides the advantage of advantageously preventing jitter and / or uncontrolled small-amplitude changes in camera orientation when the tracked position is stable, substantially stable, or undergoing small movements.

[0083] In an embodiment of the tracking system, the tracking system includes a directional beam generator command output for controlling a directional beam generator to track a tracked position using a beam; and a controller is arranged to: acquire the directional beam generator placement relative to the performance environment; calculate the directional beam generator orientation relative to the performance environment based on the directional beam generator placement and the tracked position; and provide the directional beam generator orientation to the directional beam generator command output for orienting the directional beam generator such that it illuminates the tracked position in the performance environment. In addition to controlling the camera orientation, this tracking system advantageously controls directional beam generators, thereby orienting them toward the tracked position.

[0084] In another embodiment of the tracking system, calculating the orientation of the directional beam generator includes applying a directional beam generator control loop that keeps the tracked position centered on the beam. The directional beam generator control loop offers the advantage of smoothly following the tracked position. The directional beam generator control loop may include a PID controller.

[0085] In another embodiment of the tracking system, the directional beam generator control loop advantageously tracks the tracked position in a different manner than the camera control loop. In another embodiment, the directional beam generator control loop tracks the tracked position more aggressively than the camera control loop. The directional beam generator control loop can follow the tracked position faster to keep the performer always in the light at the tracked position, whereas the camera control loop can follow the tracked position more slowly to simply keep the tracked position in the camera's field of view and provide the operator with a more stable or slowly changing image to improve the tracking of the performer, who is essentially in the tracked position or remains in the tracked position by changing the tracked position.

[0086] In an embodiment of the tracking system, the tracking system includes: a display output for outputting a display stream; and a camera input for receiving a camera stream from a camera, wherein the camera stream represents a 2D view of at least a portion of the performance environment; and a controller arranged to: receive the camera stream from the camera input; calculate a 2D tracked position based on camera placement, camera orientation, and the tracked position; insert a tracked indicator for indicating the 2D tracked position in the camera stream to generate a display stream; and provide the display stream to the display output. This embodiment advantageously combines the step of inserting the tracked indicator with the operation of the camera tracking the tracked position, thereby enhancing their respective advantages and / or providing the advantage of providing a high-resolution form of tracked position identification due to the reduced field of view of the camera.

[0087] In another embodiment of the tracking system, the controller is configured to: acquire tracked position filter parameters for low-pass filtering of tracked position changes; acquire a tracked position change threshold; calculate a change indicator by applying the low-pass filter configured with the tracked position filter parameters and the tracked position change threshold; calculate a camera zoom setting based on the change indicator; and provide the camera zoom setting to a camera command output for controlling the camera's zoom. This embodiment provides the advantages of an auto-zoom camera, allowing the field of view to decrease during periods when the tracked position does not change significantly, and to increase the field of view during periods when the tracked position changes significantly. This approach allows the operator to track the performer with higher precision by adjusting the tracked position, whether the performer's movement is small or large.

[0088] In another embodiment of the tracking system, when the change indicator indicates a low change level, the camera zoom setting instructs the camera to zoom in; and when the change indicator indicates a high change level, the camera zoom setting instructs the camera to zoom out. This embodiment advantageously adjusts the zoom setting according to changes in the tracked position, and thus adjusts the camera's field of view.

[0089] In another embodiment of the tracking system, the controller is arranged to: acquire the orientation of the directional beam generator relative to the performance environment; acquire the orientation of the directional beam generator relative to the performance environment; calculate the 2D projection of the beam based on the camera orientation, camera orientation, directional beam generator orientation, and directional beam generator orientation; and insert a beam indicator to indicate the 2D projection of the beam in the display stream. This embodiment advantageously combines the beam indicator insertion step with the camera tracking operation of the tracked position, thereby enhancing their respective advantages, and / or improving the tracking effect by allowing the tracked position to be displayed at a higher resolution due to the reduced field of view of the camera.

[0090] In another embodiment of the tracking system, the controller is arranged to: acquire a height offset based on the performer's height; calculate a reference position based on the tracked position and the height offset; calculate a 2D reference position based on the camera placement, the camera orientation, and the reference position; and insert a reference indicator to indicate the 2D reference position in the display stream to generate the display stream. This embodiment advantageously combines the reference indicator insertion step with the camera tracking the tracked position, thereby enhancing the advantages of both, and / or improving the tracking performance by allowing the reference position to be displayed at a higher resolution due to the reduced field of view of the camera.

[0091] In embodiments of the tracking system, the height offset is less than the target offset. Performers typically require the beam to illuminate part or all of the performer's head and body. Therefore, the beam's center is usually positioned at the performer's throat or chest to illuminate the performer and less of the background to enhance contrast between the performer and the background. The target indicator is advantageously kept focused on or oriented towards the head, as this allows the operator to keep the target indicator on the performer's body (e.g., on the performer's head) while the beam's center remains at the target location. The difference between the height offset and the target offset (especially if the height offset is less than the target offset) allows for better tracking convenience for the operator while enhancing contrast.

[0092] In embodiments of the tracking system, the target indicator is a crosshair. This advantageously allows for relatively easy tracking of performer body parts (e.g., the performer's head). The crosshair may include a central opening to improve the visibility of the head located in the center of the crosshair.

[0093] In embodiments of the tracking system, the target offset and height offset are selected such that when the target indicator is pointed at the performer's head, the tracked position allows the directional beam generator to illuminate at least the performer's head and shoulders, and preferably further illuminate at least a portion of the performer's chest. Presenting a performer typically requires the beam to illuminate part or all of the performer's head and body. Therefore, the center of the beam is usually maintained at the performer's throat or chest to illuminate the performer while illuminating less of the background, thus enhancing the contrast between the performer and the background. Because it is easy for the operator to hold the target indicator at a part of the performer's body (e.g., the performer's head), the target indicator is advantageously kept focused on or oriented towards the head. The difference between the height offset and the target offset (especially if the height offset is less than the target offset) allows for better tracking convenience for the operator while enhancing contrast.

[0094] In embodiments of the tracking system, the controller is arranged to: acquire a beam size indication of the directional beam generator; and, based on the beam size indication and a height offset, change a target offset. The top of the beam is preferably positioned directly above the performer's head. The center of the beam remains at the tracked position. Therefore, depending on the beam size and the performer's height, the tracked position can be at different heights relative to the performance environment (e.g., a stage) to illuminate the performer. The current embodiment advantageously allows adjustment of the height of the tracked position based on the size of the beam (typically the shape and / or diameter of the beam).

[0095] In embodiments of the tracking system, the height offset and / or target offset are advantageously offsets in the vertical direction, preferably offsets in the upward direction.

[0096] In an embodiment of the tracking system, the tracking system is arranged to illuminate a second performer at a second tracked position in a performance environment using a second beam from a second directional beam generator. The tracking system includes: a second position tracker input for receiving changes in the second tracked position from the second position tracker; a second directional beam generator command output for controlling the second directional beam generator; and a controller arranged to: acquire the placement of the second directional beam generator relative to the performance environment; acquire the second tracked position; acquire a second height offset based on the second performer's second performer height, wherein the second height offset indicates the difference between the second tracked position and the performance environment; and acquire a second target offset based on the second performer's height, wherein the second target offset indicates the distance between the second performer's head and the performance environment. The height difference between the performance environment; receiving a second tracked position change from a second position tracker input; changing the second tracked position based on the second tracked position change and a second height offset; calculating the orientation of a second directional beam generator relative to the performance environment based on the placement of a second directional beam generator and the second tracked position; providing the second directional beam generator orientation to a command output for orienting the second directional beam generator such that it illuminates the second tracked position in the performance environment; calculating a second target position based on the second tracked position and a second target offset; calculating a second 2D target position based on camera placement, camera orientation, and the second target position; and inserting a second target indicator to indicate the second 2D target position in the display stream. The second performer is advantageously illuminated by the second directional beam generator. This embodiment advantageously allows tracking two or more performers with the same tracking system, reducing hardware usage. In another embodiment of the tracking system, the second height offset is smaller than the second target offset to provide the same advantages as the height offset and target offset described above.

[0097] In another embodiment of the tracking system, the target indicator differs from the second target indicator, preferably in color, appearance, and / or shape. Different target indicators advantageously allow for easy tracking of multiple performers with the same display stream.

[0098] In an embodiment of the tracking system, the system is arranged to illuminate a performer at a tracked location in the performance environment using a third beam from a third directional beam generator. The tracking system includes a third directional beam generator command output for controlling the directional beam generator. The controller is arranged to: acquire the placement of the third directional beam generator relative to the performance environment; calculate the orientation of the third directional beam generator relative to the performance environment based on the placement and the tracked location; and provide the third directional beam generator orientation to the third directional beam generator command output for orienting the third directional beam generator such that it illuminates the tracked location in the performance environment. This embodiment advantageously allows for the illumination of the performer using multiple directional beam generators to improve the illumination of the performer, thereby increasing contrast with the performance environment (e.g., background).

[0099] In an embodiment of the tracking system, the controller is arranged to: calculate the directional beam generator light parameters of the directional beam generator based on the height offset and the target offset, to optimize the ratio of the light intensity of the beam on the performer to the light intensity of the beam in the performance environment; and provide the directional beam generator light parameters to the directional beam generator command output. The directional beam generator light parameters may include shutter parameters for changing the shape and / or size of the beam. As an example, shutter parameters can shape the beam into a square pyramid, a quadrangular pyramid, or a cone. For example, shutter parameters can limit the beam to a specific size of square, rectangular, or circular diameter. Optimizing this ratio advantageously improves the contrast between the performer and the performance environment (e.g., the stage background).

[0100] In an embodiment of the tracking system, the controller is configured to: acquire the orientation of the directional beam generator relative to the performance environment; calculate the 2D projection of the beam based on camera placement, camera orientation, directional beam generator placement, and directional beam generator orientation; and insert a beam indicator to indicate the 2D projection of the beam in the display stream.

[0101] In an embodiment of the tracking system, the controller is arranged to: calculate a reference position based on the tracked position and height offset; calculate a 2D reference position based on camera placement, camera orientation, and the reference position; and insert a reference indicator to indicate the 2D reference position in the display stream to generate the display stream. The indicated position advantageously provides the relationship between the tracked position and the performance environment, and more specifically, the relationship between the tracked position and the stage.

[0102] In an embodiment of the tracking system, the tracking system includes a camera command output for controlling a camera; and a controller arranged to: calculate the camera orientation relative to the performance environment based on the tracked position; and provide the camera orientation to the camera command output for orienting the camera so that it follows the tracked position. This allows the camera's field of view to not necessarily cover the entire performance environment, particularly not the entire stage. The camera can then zoom in to improve the tracking of the performer. Therefore, it is advantageous to select a smaller beam to enhance the contrast between the performer and the performance environment.

[0103] In another embodiment of the tracking system, the camera orientation is calculated based on a second tracked position; and the controller is arranged to calculate a camera zoom setting based on the tracked position and the second tracked position; and the camera orientation and zoom setting are calculated in such a way that the tracked position and the second tracked position are within the camera's field of view. The camera can zoom in to improve the tracking of the performer. Therefore, it is advantageous to select a smaller beam to enhance the contrast between the performer and the performance environment. Attached Figure Description

[0104] The present invention will be further apparent and illustrated by means of embodiments described by way of example in the following specification, and in conjunction with the accompanying drawings, wherein:

[0105] Figure 1 schematically illustrates a performance environment; Figure 2 schematically illustrates a controlled object; Figure 3 schematically illustrates a tracking system according to the present invention; Figure 4 schematically illustrates a 2D view of the performance environment; Figure 5 schematically illustrates a method for the tracking system; Figure 6 schematically illustrates a method for the tracking system; Figure 7 schematically illustrates a method for the tracking system; Figure 8 schematically illustrates a method for the tracking system; and Figure 9 schematically illustrates an embodiment of a computer program product, computer-readable medium, and / or non-transitory computer-readable storage medium according to the present invention. The figures are schematic only and not drawn to scale. In the figures, elements corresponding to the described elements may have the same reference numerals.

[0106] List of reference numerals

[0107]

[0108]

[0109]

[0110] Detailed Implementation

[0111] Figure 1 schematically illustrates a performance environment 10. A performance environment typically includes a stage 11. A stage is an area on which performers can walk. Although a flat stage is shown, it may include raised sections, steps, lifts, or other elements that provide static or dynamic height differences.

[0112] The dots in the diagram represent different locations. Dashed lines are used to indicate how these locations are perceived in 3D relative to the performance environment. The center front of the stage is the reference point (R). The front edge of the stage is typically used as the reference direction for the diagram. The X-direction is parallel to the front of the audience / stage and points to the right of the audience; the Y-direction is the direction directly from the stage away from the audience and directly towards the stage; the Z-direction is perpendicular to the stage and upwards. Alternative directions may be used.

[0113] An exemplary performance environment includes two directional beam generators (e.g., two follow spots) placed at different directional beam generator positions 20, 20', and a camera arranged in a camera position 30. As an example, the tracked position T and the target position Tar are shown.

[0114] Figure 2 schematically illustrates the controlled object 40. The controlled object can be a follow spot, a camera, or any other object whose direction can be controlled within the context of the performance environment.

[0115] In the current example, the camera orientation in Figure 1 can be shown. The camera has a field of view that typically extends across the entire stage. The camera typically rotates around its center point C. The controlled object can rotate around an angle relative to the horizontal plane H. Rotation is usually denoted as translation. A controlled object can rotate about an angle θ relative to the vertical axis V, usually denoted as tilt. Translation and tilt together provide the orientation of the controlled object. The combination of the position of the controlled object (e.g., a camera or follow spot in Figure 1) and the orientation shown in Figure 2 provides the attitude of the controlled object.

[0116] Figure 3 schematically illustrates a tracking system 100 according to the present invention. The tracking system includes a controller 110. The controller is typically arranged to execute steps or software including steps, such as steps from a specified method.

[0117] The tracking system may include a camera input 120. The camera input is arranged to receive a camera stream 125 from a camera. The camera stream typically represents the content within the camera's field of view in digital form. The camera input provides the camera stream as an internal camera stream 126 to the controller.

[0118] The tracking system may include a position tracker input 150. The position tracker is arranged to receive tracked position changes 155. The position tracker may be a mouse, joystick, trackball, or any other computer interface device capable of recording changes. The tracked position changes may be associated with position changes of the position tracker or other inputs of the position tracker. The position tracker input provides the tracked position changes as internal tracked position changes 156 to the controller.

[0119] The tracking system may include a display output 130. The display output may be HDMI, DisplayPort, VGA, or any standardized display output. The display output is arranged to provide a display stream 135. The display stream is typically represented digitally as a camera stream processed by the controller. The display output receives the display stream from the controller as an internal display stream 136.

[0120] The tracking system may include a camera command output 140. The camera command output is arranged to provide camera commands 145 to the camera. The camera commands may include pan and tilt settings for the camera. In some embodiments, the camera input and camera command output may share a single physical port, allowing the tracking system to be linked to the camera via a single communication cable. The camera command output may be a USB or Ethernet port. The camera command output receives camera commands as internal camera commands 146 from the controller.

[0121] The tracking system may include a directional beam generator command output 157. The directional beam generator command output may be a USB or Ethernet port. The directional beam generator command output is arranged to provide directional beam generator commands 158. The directional beam generator commands may include translation and tilt settings for the directional beam generator. The directional beam generator command output receives directional beam generator commands as internal directional beam generator commands 159 from the controller.

[0122] Multiple ports (such as input and output ports) can be combined on a single port (such as a single Ethernet port or USB port) to reduce the number of communication cables coupled to the tracking system.

[0123] Figure 4 schematically illustrates a 2D view 127 of the performance environment. The figure shows a performer 12 with a head 13. The performer has a height of 160.

[0124] Furthermore, the 2D tracked position T-2D, the 2D target position Tar-2D, and the 2D reference position B are shown as points. These points are typically not displayed in the display stream during normal operation. The 2D target position is the center of the target indicator 180. The 2D target position is a 2D projection of the target position (e.g., in Figure 1). The 2D tracked position is a 2D projection of the tracked position (e.g., in Figure 1). The reference position is typically a spatial point where the stage intersects with a vertical axis passing through both the tracked and target positions. The 2D reference position is a 2D projection of this intersection point.

[0125] The target indicator is used to allow the operator to easily overlay it onto the performer, typically onto the performer's head. The position of the target indicator is usually manipulated by providing the positional changes being tracked to the position tracker input; this manipulation is particularly indirect. The performer's spatial point on the stage can be indicated by the reference indicator 185.

[0126] The tracked position is positioned at a height of 170° relative to the reference position or a height offset of 170°. The target position is positioned at a target height of 165° relative to the reference position. The target offset of 175° is defined as the difference between the target position and the tracked position. The tracked position is typically based on the performer's height. The tracked position is typically the centerline of the beam directed from the directional beam generator to the tracked position. The tracked position can be configured such that the beam illuminates only the head and shoulders, as needed. Alternatively, the tracked position can be configured such that the beam illuminates the performer from head to toe. Many alternative configurations are possible, all of which depend on the performer's height. As shown in the 2D projection of Figure 4, the target position typically coincides with, or at least follows, the performer's head. Other body parts can be selected, or even the entire performer can be placed within the 2D projection of the target indicator.

[0127] Figure 4 also shows beam indicators 190, 190', for example, indicators corresponding to the respective directional beam generators at the positions of the directional beam generators shown in Figure 1. Specifically, the first beam indicator 190 may indicate the reflection position of the beam of the first directional beam generator at the first directional beam generator position 20 on the stage, and the second beam indicator 190' may indicate the reflection position of the beam of the second directional beam generator at the second directional beam generator position 20' on the stage. Although not shown in Figure 4, alternative beam indicators may be used at other locations according to the claims and description. Beam indicator 191 may indicate the portion of the performer that is illuminated. Specifically, in the simulation, the beam indicator may advantageously show which part of the performer is illuminated before the lights are actually turned on. As shown in Figure 4, beam indicator 191 may be displayed as a shaded circle or a circle with a pattern fill. Optionally, beam indicator 191 may be displayed as a pattern only on the performer. For this embodiment, the system may include object tracking that displays a pattern only on the performer. The beam indicator 191 can display a combination of beams from multiple directional beam generators pointing to the same performer or tracked location.

[0128] Figure 5 schematically illustrates method 200 of the tracking system. This method is a computer-implemented approach for illuminating a tracked location of a performer within a performance environment using a directional beam generator. The performer has a height. The method comprises multiple steps. Depending on the interdependencies between the steps, these steps can be performed in parallel, in different orders, and / or cyclically multiple times.

[0129] The first step is to acquire the camera placement 210, which is the camera arrangement for observing the performance environment. Camera placement is relative to the performance environment. The next step is to acquire 215, the camera orientation relative to the performance environment. The next step is to acquire 220, the tracked position. The next step is to acquire 225, the height offset based on the performer's height. The next step is to receive 230, the camera stream from the cameras. The camera stream represents a 2D view of the performance environment presented by the cameras. The next step is to calculate 235, the reference position based on the tracked position and the height offset. The next step is to calculate 240, the 2D reference position based on the camera placement, camera orientation, and reference position. The next step is to insert 245, a reference indicator to indicate the 2D reference position in the camera stream, thereby generating a display stream. The next step is to provide 250, the display stream, to the display.

[0130] In an alternative embodiment, the steps of acquiring the tracked position and calculating the 235 reference position can be interchanged to achieve the same effect. Therefore, the method includes the step of acquiring the reference position, and the step of calculating the tracked position based on the reference position and the height offset.

[0131] Furthermore, changes in the tracked position from the position tracker can be indirectly altered through the following steps, rather than directly changing the tracked position: receiving the position change input from the position tracker; changing the reference position based on the position change; and calculating the tracked position based on the reference position and a height offset. Additionally, changing the reference position can also be based on a model of the performance environment.

[0132] Figure 6 schematically illustrates a method 400 for a tracking system. This method, implemented by a computer, illuminates a performer at a tracked location within a performance environment using a beam from a directional beam generator. The method comprises multiple steps. Depending on the interdependencies between the steps, these steps can be performed in parallel, in different orders, and / or cyclically multiple times.

[0133] The first step is to obtain the camera placement (210) of the cameras arranged to observe the performance environment. Camera placement is relative to the performance environment. The next step is to obtain the camera orientation (215) of the cameras relative to the performance environment. The next step is to obtain the orientation (255) of the directional beam generators relative to the performance environment. The next step is to obtain the orientation (260) of the directional beam generators relative to the performance environment. The next step is to receive the camera stream (230) from the cameras. The camera stream represents a 2D view of the performance environment presented by the cameras. The next step is to calculate the 2D projection of the beam (265) based on the camera placement, camera orientation, directional beam generator placement, and directional beam generator orientation. The next step is to insert a beam indicator (270) to indicate the 2D projection of the beam in the camera stream to generate a display stream. The next step is to provide the display stream (250) to the monitor.

[0134] Figure 7 schematically illustrates a method 500 for a tracking system. This method, implemented by a computer, tracks a performer at a tracked location within a performance environment using cameras arranged to observe the environment. The method comprises multiple steps. Depending on the interdependencies between the steps, these steps can be performed in parallel, in different orders, and / or cyclically multiple times.

[0135] The first step is to obtain the camera placement of camera 210 relative to the performance environment. The next step is to obtain the camera orientation of camera 215 relative to the performance environment. The next step is to obtain the tracked position 220. The next step is to receive the tracked position change 270 from the position tracker. The next step is to change the tracked position 275 based on the tracked position change. The next step is to calculate the camera orientation 280 relative to the performance environment based on the tracked position. The next step is to provide the camera orientation 285 to the camera for orientation, so that the camera follows the tracked position.

[0136] Figure 8 schematically illustrates method 600 of the tracking system. This method, implemented by a computer, illuminates a performer at a tracked location within a performance environment using a beam from a directional beam generator. The method comprises multiple steps. Depending on the interdependencies between the steps, these steps can be performed in parallel, in different orders, and / or cyclically multiple times.

[0137] The first step is to acquire the directional beam generator placement (255) relative to the performance environment. The next step is to acquire the camera placement (210) relative to the performance environment. The next step is to acquire the camera orientation (215) relative to the performance environment. The next step is to acquire the tracked position (220). The next step is to acquire the height offset (225) based on the performer's height. The height offset indicates the difference between the tracked position and the performance environment. The next step is to acquire the target offset (290) based on the performer's height. The target offset represents the height difference between the performer's head and the performance environment. The next step is to receive the tracked position change (270) from the position tracker. The next step is to change the tracked position (275) based on the tracked position change and the height offset. The next step is to calculate the directional beam generator orientation (300) relative to the performance environment based on the directional beam generator placement and the tracked position. The next step is to provide the directional beam generator orientation (305) for orienting the directional beam generator so that it illuminates the tracked position in the performance environment. The next step is to calculate the target position (310) based on the tracked position and the target offset. The next step is to calculate the 315-2D target position based on camera placement, camera orientation, and target location. The next step is to receive 230 camera streams from cameras positioned to observe the performance environment. The camera streams represent a 2D view of the performance environment as presented by the cameras. The next step is to insert 320 target indicators to indicate the 2D target position in the camera streams, thereby generating a display stream. The next step is to provide 250 display streams to the display.

[0138] Note that changing the tracked position can also be done based on the camera's field of view and / or zoom.

[0139] Figure 9 schematically illustrates an embodiment of a computer program product 1000, a computer-readable medium 1010, and / or a non-transitory computer-readable storage medium including computer-readable code 1020 according to the present invention. The composite system typically includes a controller arranged to perform one or more methods specified in the specification and claims, which are typically coded in software.

[0140] Examples, embodiments, or optional features, whether or not indicated as non-limiting, should not be construed as limiting the claimed invention. It should be noted that these drawings are purely schematic and not drawn to scale. In the drawings, elements corresponding to those already described may have the same reference numerals.

[0141] Those skilled in the art will understand that the term "substantially" as used herein refers to, for example, "substantially all emissions" or "substantially constitutes...". The term "substantially" may also include embodiments having connotations such as "completely," "entirely," "whole," etc. Therefore, in embodiments, the adjective "substantially" may also be omitted. Where applicable, the term "substantially" may also refer to 90% or higher, such as 95% or higher, particularly 99% or higher, even more particularly 99.5% or higher, including 100%. The term "comprise" also includes its use to mean "consists of."

[0142] The term "functionally" is clearly understood by those skilled in the art. The terms "substantially" and "functionally" can also include embodiments having meanings such as "completely," "entirely," "holistically," etc. Therefore, in embodiments, the adjective "functionally" may be omitted. When used, for example, in "functionally parallel," those skilled in the art will understand that the adjective "functionally" includes the terms substantially as described above. Specifically, "functionally" should be understood to include feature configurations that allow these features to function as if the adjective "functionally" were not present. The term "functionally" is intended to cover variations of the feature it refers to, as long as these variations enable the combination of features to operate or function in the functional use of the feature (possibly in combination with other features associated with it in this invention). For example, if an antenna is functionally coupled or functionally connected to a communication device, the electromagnetic signals received by the antenna can be used by that communication device. The use of the term "functionally," for example in "functionally parallel," is intended to cover cases of complete parallelism, as well as embodiments covered by the above explanation of the term "substantially." For example, "functionally parallel" refers to some embodiments that behave in operation as if these components were, for example, parallel. This covers those embodiments that are obvious to those skilled in the art and that operate as parallels in the intended field of use.

[0143] Furthermore, the terms first, second, third, etc., used in the specification and claims are used to distinguish similar elements and are not necessarily used to describe an order or chronological sequence. It should be understood that such terms are interchangeable where appropriate, and that embodiments of the invention described herein can operate in orders other than those described or shown herein.

[0144] The apparatus or device described herein is at least partially in an operational manner. Those skilled in the art will understand that the invention is not limited to methods or apparatuses in an operational state.

[0145] It should be noted that the above embodiments are illustrative and not intended to limit the invention, and those skilled in the art will be able to devise many alternative embodiments without departing from the scope of the appended claims. Any reference numerals in parentheses within the claims should not be construed as limiting the claims. The use of the verb "comprising" and its variations does not exclude the presence of elements or steps not stated in the claims. The article "a" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by hardware comprising a plurality of different elements and by a suitably programmed computer. In device or apparatus claims enumerating multiple units, the plurality of devices or apparatuses among these units can be implemented by the same hardware. The fact that certain technical features are separately recited in different dependent claims does not mean that these features cannot be combined and simultaneously produce beneficial effects.

[0146] The present invention is also applicable to apparatus or devices that include one or more features described in the specification and / or shown in the drawings. The present invention also relates to methods or processes that include one or more features described in the specification and / or shown in the drawings.

[0147] The various aspects discussed in this patent can be combined with each other to provide additional advantages. Furthermore, some technical features can form the basis for subsequent divisional applications.

Claims

1. A tracking system (100) for illuminating a tracked position (T) of a performer (12) in a performance environment (10) using a beam from a directional beam generator, comprising: - Camera input (120) for receiving a camera stream (125) from a camera arranged to observe the performance environment, wherein the camera stream represents a 2D view (127) of the performance environment presented by the camera. - Display output (130) is used to output the display stream (125); and - Controller (110), which is arranged as follows: (210) Obtain the camera placement (30) of the camera relative to the performance environment; Obtain (215) the camera orientation relative to the performance environment. ,θ); Obtain (255) the orientation of the directional beam generator relative to the directional beam generator placement (20, 20') of the performance environment; Obtain (260) the orientation of the directional beam generator relative to the performance environment. ,θ); Receive (230) the camera stream input from the camera; Obtain the tracked position; Based on the tracked position, the camera placement, the camera orientation, the orientation of the directional beam generator, and the orientation of the directional beam generator, calculate (265) the 2D projection of the beam at the tracked position; Insert (270) beam pointers (190, 190', 191) to indicate the 2D projection of the beam in the camera stream to generate the display stream; as well as Provide the display stream (250) to the display output.

2. The tracking system according to the preceding claim, wherein calculating the 2D projection of the beam comprises: Based on the placement and orientation of the directional beam generator, a shape that at least partially represents the beam is virtually arranged; as well as The 2D projection of the shape is calculated based on the camera placement and camera orientation.

3. The tracking system according to the preceding claim, The shape described therein is an extended shape having an extending axis; and The tracked position is located on the extended axis.

4. The tracking system according to any one of the preceding claims, wherein the beam indicator indicates a 2D projection of the beam.

5. The tracking system according to any one of the preceding claims, wherein the controller is further arranged to insert a tracked indicator in the camera stream for indicating the tracked location.

6. The tracking system according to any one of the preceding claims, The shape described therein is a symmetrical shape with an axis of symmetry; and The tracked position is located on the axis of symmetry.

7. The tracking system according to any one of the preceding claims, wherein the shape is a cone, a truncated cone, a cylinder, a pyramid, or a truncated pyramid.

8. The tracking system according to any one of the preceding claims, The shape described therein has volume; and The tracked location is located within the volume.

9. The tracking system according to any one of the preceding claims, wherein the shape at least partially coincides with the boundary of the light beam.

10. The tracking system according to any one of the preceding claims, The shape described includes one side; and The edge thereon coincides with the edge of the beam.

11. The tracking system according to any one of the preceding claims, wherein the shape is spherical, the sphere having a center equal to the tracked position and having a radius such that the sphere shares a circle with the light beam.

12. The tracking system according to any one of the preceding claims, wherein calculating the 2D projection includes virtually projecting the light beam in the performance environment.

13. The tracking system according to the preceding claim, The controller is configured to acquire a model of the performance environment; and The calculation of the 2D projection of the beam is also based on a model of the performance environment.

14. The tracking system according to any one of the preceding claims, The controller is arranged to acquire the directional beam generator optical parameters of the directional beam generator; and The calculation of the 2D projection of the beam is also based on the optical parameters of the directional beam generator.

15. The tracking system according to any one of the preceding claims, wherein the controller is arranged as follows: The placement of a second directional beam generator relative to the performance environment is determined, wherein the second directional beam generator uses a second beam to illuminate the performer located at the tracked position; Obtain the orientation of the second directional beam generator relative to the performance environment; Based on the camera placement, camera orientation, second directional beam generator placement, and second directional beam generator orientation, calculate the 2D projection of the second beam; as well as Insert a second beam indicator to indicate a second 2D projection of the second beam in the display stream.

16. The tracking system according to the preceding claim, wherein the beam indicator is different from the second beam indicator.

17. The tracking system according to any one of the preceding claims, wherein the controller is arranged as follows: Obtain the tracked position; The height offset is obtained based on the performer's height. Calculate the reference position based on the tracked position and the height offset; Calculate the 2D reference position based on the camera placement, camera orientation, and reference position; as well as Insert a reference indicator to indicate the 2D reference position in the display stream.

18. The tracking system according to any one of the preceding claims, The performer described therein has a performer height; and The controller (110) is further arranged as follows: Obtain the tracked position (220); Based on the performer's height, obtain (225) the height offset (175); Based on the tracked position and the height offset, calculate the (235) reference position; Based on the camera placement, the camera orientation, and the reference position, calculate (240) 2D reference position (B). as well as Insert (245) a reference indicator (185) for indicating the 2D reference position in the camera stream to generate the display stream.

19. The tracking system according to any one of the preceding claims, The tracking system further includes: - Camera command output (145) for controlling the camera; - Position tracker input (150) for receiving tracked position changes (155) from the position tracker; and The controller (110) is further arranged as follows: Obtain the tracked position (220); Receive (270) the tracked position change from the input of the position tracker; Based on the change in the tracked position, change (275) the tracked position; Based on the tracked position, calculate (280) the camera orientation relative to the performance environment. ,θ); as well as The camera is directed to the camera via a command output (285) to orient the camera so that it follows the tracked position.

20. The tracking system according to any one of the preceding claims, The tracking system further includes: - Position tracker input (150) for receiving tracked position changes (155) from the position tracker; - A directional beam generator command output (157) for controlling the directional beam generator; and The controller (110) is further arranged as follows: Obtain the tracked position (220); Based on the performer's performer height (160), a height offset (175) is obtained (225), wherein the height offset indicates the difference between the tracked position and the performance environment; Based on the performer's height, a target offset (176) is obtained (290), wherein the target offset indicates the height difference between the performer's head (13) and the performance environment; Receive (270) the tracked position change from the input of the position tracker; Based on the change in the tracked position and the height offset, change (275) the tracked position; Based on the placement and tracked position of the directional beam generator, the orientation of the directional beam generator relative to the performance environment is calculated (300). ,θ); The directional beam generator is directed to the output of a command (305) to orient the directional beam generator so that the directional beam generator illuminates the tracked position in the performance environment. Based on the tracked position and the target offset, calculate (310) the target position (Tar). Based on the camera placement, the camera orientation, and the target position, calculate (315) 2D target position (Tar-2D). as well as Insert (320) a target indicator (180) to indicate the position of the 2D target in the camera stream to generate the display stream.

21. A computer-implemented method (400) for illuminating a tracked position of a performer in a performance environment using a beam from a directional beam generator, comprising: Obtain a camera placement arrangement for cameras used to observe the performance environment, wherein the camera placement is relative to the performance environment. Obtain the camera orientation relative to the performance environment; The orientation of the directional beam generator relative to the performance environment is determined; Obtain the orientation of the directional beam generator relative to the performance environment; Receive a camera stream from the camera, wherein the camera stream represents a 2D view of the performance environment presented by the camera; Obtain the tracked position; Based on the tracked position, the camera placement, the camera orientation, the orientation of the directional beam generator, and the orientation of the directional beam generator, calculate the 2D projection of the beam at the tracked position; Insert beam indicators (190, 190', 191) to indicate the 2D projection of the beam in the camera stream to generate the display stream; as well as The display stream is provided to the display.

22. A computer program product comprising instructions that, when executed by a suitable processor, cause the processor to perform the method of claim 21.

23. A tracking system (100) for illuminating a tracked position (T) of a performance environment (10) using a directional beam generator, wherein the performer has a performer height; The tracking system includes: - Camera input (120) for receiving a camera stream (125) from a camera arranged for observation in the performance environment, wherein the camera stream represents a 2D view (127) of the performance environment presented by the camera. - Display output (130) is used to output the display stream (125); and - Controller (110), which is arranged as follows: (210) Obtain the camera placement (30) of the camera relative to the performance environment; Obtain (215) the camera orientation relative to the performance environment; Obtain the tracked position (220); Based on the performer's height, obtain a height offset of (225) (175); Receive (230) the camera stream input from the camera; Based on the tracked position and the height offset, calculate the (235) reference position; Based on the camera placement, the camera orientation, and the reference position, calculate (240) 2D reference position (B). Insert (245) a reference indicator (185) for indicating the 2D reference position in the camera stream to generate the display stream; and Provide the display stream (250) to the display output.

24. The tracking system according to the preceding claim, wherein the reference indicator indicates the performer's foot area in the performance environment.

25. The tracking system according to the preceding claim, wherein the reference position coincides with the performance environment.

26. The tracking system according to any one of claims 23-25, wherein the reference indicator is a surface, such as a square or a circle, wherein preferably, the surface is aligned with or overlaps with a portion of the performance environment, and wherein preferably, the surface is displayed in the display stream in a perspective view.

27. The tracking system according to any one of claims 23-26, wherein the reference indicator is a line segment, wherein preferably, the line segment is displayed as a horizontal line segment in the display stream.

28. The tracking system according to any one of claims 23-27, wherein the controller is arranged as follows: Calculate the target position based on the tracked position; Calculate the 2D target position based on the camera placement, camera orientation, and target position; as well as Insert a target indicator to indicate the location of the 2D target in the display stream.

29. The tracking system according to the preceding claim, wherein the target indicator is a crosshair.

30. The tracking system according to any one of claims 28-29, wherein the controller is arranged to insert an association indicator for associating the reference indicator with the target indicator in the display stream, wherein preferably, the association indicator is a line segment connecting the reference indicator and the target indicator.

31. The tracking system according to any one of claims 23-30, comprising a position tracker input for receiving a tracked position change from a position tracker; The controller is arranged as follows: Receive the tracked position change from the input of the position tracker; as well as Based on the change in the tracked position, change the tracked position.

32. The tracking system according to the preceding claim, wherein changing the tracked position comprises: Based on the change in the tracked position, the tracked position is changed in a plane parallel to the performance environment.

33. The tracking system according to any one of claims 31-32, wherein the tracked position is changed in such a way that the reference position remains aligned with the performance environment.

34. The tracking system according to any one of claims 31-33, The controller is configured to retrieve a model of the performance environment; and The change in the tracked position is also based on a model of the performance environment.

35. The tracking system according to the preceding claim, wherein changing the tracked position comprises: The model based on the performance environment changes the tracked position in the vertical direction.

36. The tracking system according to the preceding claim, wherein changing the tracked position in the vertical direction comprises: The tracked position is kept at the height offset relative to the performance environment in the vertical direction.

37. The tracking system according to any one of claims 23-36, wherein the tracked position is a 3D position.

38. A tracking component comprising: - A tracking system according to any one of claims 23-37; - A camera capable of being coupled to the camera input of the tracking system; and - A display capable of being coupled to the display output of the tracking system.

39. A computer-implemented method (200) for illuminating a tracked position of a performance environment using a directional beam generator, wherein the performer has a performer height, the method comprising: Obtain a camera placement arrangement for cameras used to observe the performance environment, wherein the camera placement is relative to the performance environment. Obtain the camera orientation relative to the performance environment; Obtain the tracked position; Based on the performer's height, obtain the height offset; Receive a camera stream from the camera, wherein the camera stream represents a 2D view of the performance environment presented by the camera; Calculate the reference position based on the tracked position and the height offset; Calculate the 2D reference position based on the camera placement, camera orientation, and reference position; Insert a reference indicator to indicate the 2D reference position in the camera stream to generate a display stream; and The display stream is provided to the display.

40. A computer program product comprising instructions that, when executed by a suitable processor, cause the processor to perform the method of claim 39.

41. A tracking system (100) for tracking a performer (12) located at a tracked position (T) in the performance environment (10) using cameras arranged for observing the performance environment, comprising: - Camera command output (145) for controlling the camera; - Position tracker input (150) for receiving tracked position changes (155) from the position tracker; and - Controller (110), which is arranged as follows: (210) Obtain the camera placement (30) of the camera relative to the performance environment; Obtain (215) the camera orientation relative to the performance environment. ,θ); Obtain the tracked position (220); Receive (270) the tracked position change from the input of the position tracker; Based on the change in the tracked position, change (275) the tracked position; Based on the tracked position, calculate (280) the camera orientation relative to the performance environment. ,θ); as well as The camera is directed to the camera via a command output (285) to orient the camera so that it follows the tracked position.

42. The tracking system according to the preceding claim, wherein calculating the camera orientation includes applying a control loop that keeps the tracked position within the field of view.

43. The tracking system according to any one of claims 41-42, wherein calculating the camera orientation includes applying a camera control loop such that the tracked position is kept at the center of the field of view.

44. The tracking system according to any one of claims 41-43, The controller is configured to acquire a threshold value for changes in the tracked location; and The calculation of camera orientation includes keeping the camera orientation unchanged when the change in the tracked position is less than the threshold of the change in the tracked position.

45. The tracking system according to any one of claims 41-45, The tracking system includes a directional beam generator command output for controlling the directional beam generator to track the tracked position using a beam; and The controller is arranged as follows: The orientation of the directional beam generator relative to the performance environment is determined; Based on the placement of the directional beam generator and the tracked position, calculate the orientation of the directional beam generator relative to the performance environment; and The direction of the directional beam generator is provided to the command output for orienting the directional beam generator so that the directional beam generator illuminates the tracked position in the performance environment.

46. ​​The tracking system according to the preceding claim, wherein calculating the orientation of the directional beam generator includes applying a directional beam generator control loop such that the tracked position is kept at the center of the beam.

47. The tracking system according to the preceding claim and dependent on claim 43, The directional beam generator control loop and the camera control loop track the tracked position in different ways; and Preferably, the directional beam generator control loop tracks the tracked position more aggressively than the camera control loop.

48. The tracking system according to any one of claims 41-47, The tracking system includes: - Display output, used to output the display stream; and - Camera input for receiving a camera stream from the camera, wherein the camera stream represents a 2D view of at least a portion of the performance environment; and The controller is arranged as follows: Receive the camera stream input from the camera; Calculate the 2D tracked position based on the camera placement, camera orientation, and tracked position; Insert a tracked indicator to indicate the 2D tracked position in the camera stream to generate the display stream; as well as The display stream is provided to the display output.

49. The tracking system according to the preceding claim, wherein the controller is arranged as follows: Obtain the filter parameters of the tracked position, and use them to perform low-pass filtering on the changes in the tracked position; Obtain the threshold for changes in the tracked location; A change indicator is calculated by applying a low-pass filter configured with the filter parameters of the tracked position and the change threshold of the tracked position; Calculate the camera zoom setting based on the change indicator; as well as The camera zoom settings are provided to the camera command output for controlling the camera's zoom.

50. The tracking system according to the preceding claim, When the change indicator indicates a low change level, the camera zoom setting instructs the camera to zoom in; and When the change indicator indicates a high change level, the camera zoom setting instructs the camera to zoom in and reduce its size.

51. The tracking system according to any one of claims 48-50, wherein the controller is arranged as follows: The orientation of the directional beam generator relative to the performance environment is determined; Obtain the orientation of the directional beam generator relative to the performance environment; The 2D projection of the beam is calculated based on the camera placement, camera orientation, directional beam generator placement, and directional beam generator orientation. as well as Insert beam indicators (190, 190', 191) to indicate the 2D projection of the beam in the display stream.

52. The tracking system according to any one of claims 48-51, wherein the controller is arranged as follows: Based on the performer's height, obtain the height offset; Calculate the reference position based on the tracked position and the height offset; Calculate the 2D reference position based on the camera placement, camera orientation, and reference position; as well as Insert a reference indicator to indicate the 2D reference position in the display stream to generate the display stream.

53. A computer-implemented method (500) for tracking a performer (12) located at a tracked position (T) in the performance environment (10) using cameras arranged for observing the performance environment, comprising: Obtain the camera placement relative to the performance environment; Obtain the camera orientation relative to the performance environment; Obtain the tracked position; Receive the tracked position change from the position tracker; Based on the change in the tracked position, change the tracked position; Based on the tracked position, calculate the camera orientation relative to the performance environment; as well as The camera orientation is provided to the camera for orientation, so that the camera follows the tracked position.

54. A computer program product comprising instructions that, when executed by a suitable processor, cause the processor to perform the method of claim 53.

55. A tracking system (100) for illuminating a tracked position (T) of a performance environment (10) using a beam from a directional beam generator, the tracking system comprising: - Position tracker input (150) for receiving tracked position changes (155) from the position tracker; - Directional beam generator command output (157) for controlling the directional beam generator; - Camera input (120) for receiving a camera stream (125) from a camera arranged to observe the performance environment, wherein the camera stream represents a 2D view (127) of the performance environment presented by the camera. - Display output (130) is used to output the display stream (125); and - Controller (110), which is arranged as follows: Obtain (255) the orientation of the directional beam generator relative to the directional beam generator placement (20, 20') of the performance environment; (210) Obtain the camera placement (30) of the camera relative to the performance environment; Obtain (215) the camera orientation relative to the performance environment. ,θ); Obtain the tracked position (220); Based on the performer's performer height (160), a height offset (175) is obtained (225), wherein the height offset indicates the difference between the tracked position and the performance environment; Based on the performer's height, a target offset (176) is obtained (290), wherein the target offset indicates the height difference between the performer's head (13) and the performance environment; Receive (270) the tracked position change input from the position tracker; Based on the change in the tracked position and the height offset, change (275) the tracked position; Based on the placement of the directional beam generator and the tracked position, calculate (300) the orientation of the directional beam generator relative to the performance environment. ,θ); The directional beam generator is directed to the output of a command (305) to orient the directional beam generator so that the directional beam generator illuminates the tracked position in the performance environment. Based on the tracked position and the target offset, calculate (310) the target position (Tar). Based on the camera placement, the camera orientation, and the target position, calculate (315) 2D target position (Tar-2D). Receive (230) the camera stream input from the camera; Insert (320) a target indicator (180) to indicate the position of the 2D target in the camera stream to generate a display stream; as well as Provide the display stream (250) to the display output.

56. The tracking system according to the preceding claim, wherein the height offset is less than the target offset.

57. The tracking system according to any one of claims 55-56, wherein the target indicator is a crosshair.

58. The tracking system according to any one of claims 55-57, wherein the target offset and the height offset are selected such that when the target indicator is pointed at the performer's head, the tracked position causes the directional beam generator to illuminate at least the performer's head and shoulders, and preferably further illuminate at least a portion of the performer's chest.

59. The tracking system according to any one of claims 55-58, wherein the controller is arranged as follows: Obtain the beam size indication of the directional beam generator; and The target offset is changed based on the beam size indication and the height offset.

60. The tracking system according to any one of claims 55-59, wherein the height offset and / or the target offset is an offset in the vertical direction, preferably an offset in the upward direction.

61. The tracking system according to any one of the preceding claims, wherein the tracking system is arranged to illuminate a second performer at a second tracked position in the performance environment using a second beam from a second directional beam generator, the tracking system comprising: - Second position tracker input, used to receive a second tracked position change from the second position tracker; - Second directional beam generator command output, used to control the second directional beam generator; and - The controller, which is arranged as follows: The placement of the second directional beam generator relative to the performance environment is determined. Obtain the second tracked position; A second height offset is obtained based on the second performer's second performer height, wherein the second height offset indicates the difference between the second tracked position and the performance environment; Based on the height of the second performer, a second target offset is obtained, wherein the second target offset indicates the height difference between the head of the second performer and the performance environment; Receive a second tracked position change from the input of the second position tracker; Based on the change in the second tracked position and the second height offset, change the second tracked position; Based on the placement of the second directional beam generator and the second tracked position, the orientation of the second directional beam generator relative to the performance environment is calculated; The command output provides the orientation of the second directional beam generator to the second directional beam generator for orienting the second directional beam generator so that the second directional beam generator illuminates the second tracked position of the performance environment; Calculate the second target position based on the second tracked position and the second target offset; Calculate the position of the second 2D target based on the camera placement, the camera orientation, and the position of the second target. as well as Insert a second target indicator to indicate the location of the second 2D target in the display stream.

62. The tracking system according to the preceding claim, wherein the second height offset is less than the second target offset.

63. The tracking system according to any one of claims 61-62, wherein the target indicator is different from the second target indicator, preferably in terms of color, appearance and / or shape.

64. The tracking system according to any one of claims 55-63, The tracking system is arranged such that the performer at the tracked location in the performance environment is illuminated by a third beam from a third directional beam generator; The tracking system includes a third directional beam generator command output for controlling the directional beam generator; and The controller is arranged as follows: The placement of the third directional beam generator relative to the performance environment is determined. Based on the placement of the third directional beam generator and the tracked position, calculate the orientation of the third directional beam generator relative to the performance environment; and The command output provides the orientation of the third directional beam generator for orienting the third directional beam generator so that the third directional beam generator illuminates the tracked position in the performance environment.

65. The tracking system according to any one of claims 55-64, wherein the controller is arranged as follows: Based on the height offset and the target offset, the optical parameters of the directional beam generator are calculated to optimize the ratio of the light intensity of the beam on the performer to the light intensity of the beam in the performance environment; and The optical parameters of the directional beam generator are provided to the command output of the directional beam generator.

66. The tracking system according to any one of claims 55-65, wherein the controller is arranged as follows: Obtain the orientation of the directional beam generator relative to the performance environment; Based on the camera placement, camera orientation, directional beam generator placement, and directional beam generator orientation, calculate the 2D projection of the beam; and Insert beam indicators (190, 190', 191) to indicate the 2D projection of the beam in the display stream.

67. The tracking system according to any one of claims 55-66, wherein the controller is arranged as follows: Calculate the reference position based on the tracked position and the height offset; Calculate the 2D reference position based on the camera placement, camera orientation, and reference position; as well as Insert a reference indicator to indicate the 2D reference position in the display stream to generate the display stream.

68. The tracking system according to any one of claims 55-67, The tracking system includes a camera command output for controlling the camera; and The controller is arranged as follows: Based on the tracked position, calculate the camera orientation relative to the performance environment; and The camera is directed to the camera via a command output, which is used to orient the camera so that it follows the tracked position.

69. The tracking system according to the preceding claim and dependent on claim 61, The camera orientation is calculated based on the second tracked position; and The controller is arranged as follows: Based on the tracked position and the second tracked position, calculate the camera zoom setting; and The method for calculating the camera orientation and the camera zoom setting is such that the tracked position and the second tracked position are within the field of view of the camera.

70. A computer-implemented method (500) for illuminating a tracked position of a performer in a performance environment using a beam from a directional beam generator, comprising: The orientation of the directional beam generator relative to the performance environment is determined; Obtain the camera placement relative to the performance environment; Obtain the camera orientation relative to the performance environment; Obtain the tracked position; Based on the performer's height, a height offset is obtained, wherein the height offset indicates the difference between the tracked position and the performance environment; Based on the performer's height, a target offset is obtained, wherein the target offset indicates the height difference between the performer's head and the performance environment; Receive tracked position changes from the position tracker; The tracked position is changed based on the change in the tracked position and the height offset. Based on the placement of the directional beam generator and the tracked position, calculate the orientation of the directional beam generator relative to the performance environment; Provide the orientation of the directional beam generator for orienting the directional beam generator so that the directional beam generator illuminates the tracked position in the performance environment; Calculate the target position based on the tracked position and the target offset; Calculate the 2D target position based on the camera placement, camera orientation, and target position; Camera streams are received from cameras positioned to observe the performance environment, wherein the camera streams represent a 2D view of the performance environment presented by the cameras. Insert a target indicator to indicate the location of the 2D target in the camera stream to generate the display stream; as well as The display stream is provided to the display.

71. A computer program product comprising instructions that, when executed by a suitable processor, cause the processor to perform the method of claim 70.

Images