Studio set-up

Abstract

A method for configuring a filming environment (e.g. studio) comprises: providing in the filming environment an item of filming equipment (e.g. camera 1, 2, light, 3, 4, reflector, diffuser or microph

Description

This invention relates to simulating the operation of equipment in a filming location such as a studio, and automatically adjusting the position and / or orientation of equipment in a filming location. When filming is to take place in a studio, the operator of the studio sets up the locations and orientations of lighting units and cameras so as to provide suitable coverage of a subject. Conventionally, this is done by adjusting the lighting units and the cameras in the studio itself until a desired effect is achieved. This can take a considerable time to achieve, during which the studio is unavailable to be used for other filming. Also, if at a later time the equipment has been moved it is difficult to replace it where it was before, for example to re-film parts of a scene. It would be desirable to make it easier to determine a desired location and / or orientation for equipment in a studio. It would be desirable to automatically position equipment in a studio rather than doing so manually. SUMMARY OF THE INVENTION According to the present invention there is provided a method for configuring a filming environment, the method comprising: providing in the filming environment an item of filming equipment mounted on a poseable support; in a modelling system having one or more processors configured to execute code for forming a three-dimensional model, modelling the appearance of the filming environment and the performance of the item of filming equipment, and simulating the performance of the filming environment with the item of filming equipment in a determined location and / or orientation relative to the filming environment; repeatedly receiving at the modelling system a sensed position and / or orientation of the item of filming equipment; and transmitting from the modelling system, in dependence on the sensed position, a command for causing the item of filming equipment to be moved on the poseable support to the and / or orientation. The poseable support may comprise a powered motion element. The command comprises a command for the motion element to adopt a specified configuration. The motion element may be a poseable joint. It may be a revolute joint or a prismatic joint. A motor may be provided for driving motion at the joint. The poseable support may be an articulated boom or a tripod. The poseable support may be manually adjustable. The method may comprise providing at the item of filming equipment an indicator for indicating a desired motion direction and / or axis of the item of filming equipment. The command may comprise a command for causing the indicator to indicate a direction in which the item of filming equipment can be moved to reach the determined position and / or location. The poseable support may comprise one or more sensors for sensing the pose of the support. The method comprises transmitting data from the sensors to the modelling system. The method may comprise providing at the item of filming equipment a sensor apparatus capable of visually sensing a field of irregularly positioned indicia and estimating therefrom a position and / or orientation of the item of filming equipment, and transmitting data from the sensor apparatus to the modelling system. The method may comprise recalling at the modelling system a stored, previously determined location and / or orientation and transmitting from the modelling system a command for causing the item of filming equipment to be moved on the poseable support to the previously determined location and / or orientation. The item of filming equipment may be one of a camera, a light, a reflector, a diffuser and a microphone. The method may comprise receiving the position and / or orientation of a first item of equipment moved manually, and causing a second item of filming equipment which interacts with the first item to be moved automatically to preserve an operational relationship between the first and second items of equipment. The method may comprise sensing a subject and causing an item of equipment to be moved when the subject moves to preserve an operational relationship with the subject. According to a second aspect there is provided a filming environment or a filming system, comprising: an item of filming equipment mounted on a poseable support; a modelling system having one or more processors configured to execute code for forming a three-dimensional model, modelling the appearance of the filming environment and the performance of the item of filming equipment, and simulating the performance of the filming environment with the item of filming equipment in a determined location and / or orientation relative to the filming environment; wherein the modelling system repeatedly receives a sensed position and / or orientation of the item of filming equipment; and the modelling system transmits, in dependence on the sensed position, a command for causing the item of filming equipment to be moved on the poseable support to the determined location and / or orientation. The system may receive the position and / or orientation of a first item of equipment moved manually. In response to that it may causes a second item of filming equipment which interacts with the first item to be moved automatically to preserve an operational relationship between the first and second items of equipment. The system may sense a subject, for example the position and / or orientation of the subject, and may cause an item of equipment to be moved automatically when the subject moves to preserve an operational relationship with the subject. DESCRIPTION OF THE DRAWINGS The present invention will now be described by way of example with reference to the accompanying drawings. In the drawings: Figure 1 shows a studio environment. Figure 2 shows a direction indicator. DETAILED DESCRIPTION The following description is presented to enable any person skilled in the art to make and use the invention and is provided in the context of a particular application. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art. The general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present invention. Thus, the present invention is not intended to be limited to the embodiments shown, but is to be accorded the widest scope consistent with the principles and features disclosed herein. In the description below, filming equipment such as lights and cameras will be described as being used in a studio. The principles described below could be applied in any location, without limitation to filming studios: for example in outdoor locations and in indoor locations that are not dedicated studios. The term filming will be used to refer to the capture of video. The video may be recorded on film or in data memory. Figure 1 shows a studio environment. The studio comprises cameras 1,2, lights 3, 4, microphones 5, 6 and a subject 7. The intention is that the lights 3, 4 illuminate the subject while the cameras 1,2 capture images of the subject and the microphones 5, 6 capture audio. The subject may be or comprise a human, anil object. The cameras, lights and microphones are mounted on automated supports 8. Each automated support can move and direct the item of equipment that is mounted on it. The automated support is selected so as to be appropriate for the weight of the equipment mounted on it, and to provide a suitable range of motion forthat equipment. In one example, the automated support could be a motorised tripod. The tripod may have motorised wheels by which it can be driven across the studio floor, a motorised lifting pillar for raising equipment mounted on the tripod, and a motorised pan / tilt head for directing the equipment mounted on the tripod. In another example, the automated support could be an articulated boom. The boom may be fixed in position on a wall or floor, or it may be provided with motorised wheels like a tripod may. The boom has a series of rigid links, interconnected by motorised revolute joints. These enable the boom to move equipment mounted on the end of the boom to a suitable position and optionally to adjust the direction of that equipment. The boom may have a motorised pan / tilt head at its distal end for directing equipment mounted on it. Other studio equipment may be mounted in a similar way and controlled and simulated in the manner described below. Examples include reflectors, diffusers and effects generators. The system may be capable of sensing the location and / or direction of equipment carried by the automated supports. One way to do this is to provide the supports with sensors that detect the position of their movable elements: for example the position of boom joints or the rotation of ground wheels. With this data, and if necessary with knowledge of the original position of the support - which may be provided by an operator - the current position and / or direction of a mount on the end of the support can be calculated. Another approach is to use a system such as Startracker TM, which is available from Mo-Sys Engineering Limited. Such a system is described in EP 2 962 284. In such a system optically detectable markers are positioned at irregular loc conveniently on a ceiling of the studio. A sensor, for example a positioning camera, is attached to an object whose location and / or direction is to be determined. The sensor images the field of markers and provides the sensed data to a processor. The processor is programmed to initially learn the positions of the markers. Subsequently, as the sensor moves it can infer its position and orientation from the view it has of the field of markers. In figure 1, camera 2 is provided with such a system. A positioning sensor 10 is attached to camera 2 so as to face upwards at the ceiling of the studio. Markers 11 are positioned irregularly on the ceiling. A processor 12 receives data from the sensor. Processor 12 executes program code stored in non-transient form in memory 13 so as to learn the configuration of the marker field and then estimate the position and orientation of camera 2. That information may be transmitted to a studio controller 20. The sensor 10 senses the surroundings of the object. The sensor images the ceiling and the markers. Processor 12, which may be integrated with the sensor detects the markers in the sensed image data. The markers may be detected by virtue of their contrast relative to the background and / or their shape. Once the markers are detected in the image data their vectors relative to the sensor can be determined. This can be done by processing capabilities integrated in the sensor or by the processor. The processor stores data defining the positions of the markers in two or three dimensions. The processor determines a three-dimensional position that provides a best fit between the determined vectors and the expected vectors that would be seen when the pattern is imaged from the position. This is adopted as a position of the object. Similarly, the orientation that provides a best fit between the determined vectors and the expected vectors that would be seen when the pattern is imaged in that orientation is adopted as the orientation of the sensor. The markers may be detected by virtue of their contrast relative to the background and / or their shape. The markers are preferably of an appearance that is readily distinguishable from the environment. For example, they may be optically illuminated, or of very high reflectivity (e.g. of retroreflective material), or of (e.g. having a matt black surface coating), or they may be of a define colour (e.g. a specific green). When the markers are of high reflectivity, preferably each one is of a material that reflects preferentially in a direction orthogonal to its major plane, as may be the case with dedicated retroreflective materials. The markers are preferably flat: for example, they may be in the form of laminar stickers applied to the surfaces of the environment. This can make them easy to apply in the environment. The markers preferably bear no surface markings (e.g. numbers or bar codes) by which each one can be distinguished from the others. This can make the task of applying the markers in the environment easier. The markers may all have the same outline (e.g. round or square) or they may have different outlines. The markers may have a retroreflective surface. The markers are positioned in an irregular pattern. The pattern is preferably nonrepeating. This may be achieved by randomly positioning the markers in the environment. Positioning the markers in an irregular pattern can make the task of applying the markers easier and also facilitates locating the object in the environment. In summary, in a preferred arrangement the markers are provided by substantially identical retroreflective stickers which are applied to the environment in an irregular or random pattern. The markers may all be of the same size, which may help their range to be determined, or of different sizes. A mixture of large and small markers may be used in a single area. The markers may be located on upwards-facing, downwards-facing or sidewaysfacing surfaces of the environment. It is preferred that at least some of the markers are located on downward-facing surface, e.g., a ceiling. Such a downward-facing surface may be above the subject. Such a downward-facing surface may be above the camera. Such a downward-facing surface may be above the lighting device. Visibility of markers located above these objects is typically better than markers located sideways of or below the object(s) because it is less likely to be obstructed by other objects or people. Markers in such a position may remain cl more easily distinguished from a backing than a marker on the floor. The positioning sensor may be an imaging device such as a camera, for example an optical camera, CCD, infrared camera or UV camera. The object may have more than one sensor. The sensor may be positioned on the upward face of the object, such that it is below the markers in operation. The sensor may also be positioned on the side or underside of the object, for manoeuvring and altering the orientation of the object. An upwards facing sensor may be advantageous for detecting markers positioned on the ceiling of a room. Markers in such a position may remain cleaner and therefore more easily distinguished from a backing than a marker on the floor. The sensor may be mounted to the object in an orientation such that in normal orientation, the centre of the imaging device’s field of view is directed upwards. Some of the irregularly positioned markers may be obscured. In this situation the sensor senses the markers that are visible and uses their relative positions to determine the position of the object. Other positioning systems may be used, for example radio-based triangulation or trilateration systems. The operation of the studio controller 20 will now be described. The studio controller comprises on or more processors 21, and a memory 22. The memory stored in non-transient form instructions that are executable by the processor(s) to perform the functions described of the studio controller herein. The studio controller is capable of implementing a three-dimensional model of the studio. This may be done using a commercially available product such as Unreal Engine. A model of the studio, the subject and the equipment in it is stored in the memory 22. The model of the studio and the subject may include the walls and floor of the studio and any props. In the case of an outdoor location the model may include trees, building and the like. The model of the subject may dimensional shape and the appearance of the subject. The subject may be positioned in the model at a desired location in the modelled studio. The model of a camera may include parameters such as the lens being used on the camera with its field of view and any lens distortions it imposes. The model of a light may include its luminance and the pattern of its light spread. The model of a diffuser may include its size and its diffusive effect on incident light. The model of a microphone may include its sensitivity and directionality. These are examples, and other equipment could be modelled in analogous ways. An operator of the studio controller may define the model in advance of filming. The operator may then use the model to simulate set-ups of the studio, with the equipment in a range of positions and orientations. The studio controller uses the stored data about the studio and the performance of the equipment to simulate the images that would be captured by the cameras with the studio in that configuration. The operator can then view the simulated images and adjust the position and direction of the equipment until a desired effect is achieved. This can be considerably quicker than doing the same process in the actual studio. The operator need not even be in the studio to undertake this process. The actual studio could be being used for other purposes at this time. The operator may cause the production controller to store the locations and / or directions of the equipment in the selected set-up. The production controller can communicate with the automated supports so as to control their operation. The production controller can also receive information from sensors on the supports and / or from devices such as Startracker devices on the equipment so it has knowledge of the current location of the equipment. The data involved can be communicated over any suitable link: for example a wired or a wireless link. In one mode of operation, the operator can select a position and / or orientation / direction for an item of equipment in the studio. Then the operator can cause the production controller to signal an automated support that is carrying that equipment to cause the equipment to adopt the selected positi This may be done with feedback of the sensed actual position and / or direction of the equipment. This process may be followed for multiple items of equipment: for example multiple cameras and lighting units. The position(s) and / or direction(s) may be stored in memory accessible to the production controller, and recalled so that they can be readily re-adopted after equipment has been moved. In another mode of operation, an operator in the studio can move an item of equipment manually to adopt a desired location and / or orientation. This might achieve a desired visual effect. The position and / or orientation of the equipment can then be transmitted to the production controller and stored as described above. The equipment can later be moved automatically under the control of the production controller to adopt the same position and / or direction. In the examples described above, the equipment is moved automatically. This may be done by electric motors, hydraulic actuators, pneumatic actuators or other drivers attached to reconfigurable and / or mobile equipment supports. In an alternative approach, an item of equipment may be equipped with a direction indicator, for example as illustrated in figure 2. The direction indicator 30 includes lights 31 or other selectively actuable indicia which can be controlled to indicate the direction in which the direction indicator would need to be moved to reach an intended position and / or orientation. The indicators may refer to three orthogonal translation directions and / or three orthogonal rotation axes, each in two directions. The direction indicator can be attached to an item of equipment. It can receive information about its current position and / or orientation from a Startracker associated with the item of equipment or from sensors associated with a mobile support for the item of equipment. The production controller can transmit to the direction indicator a desired position and / or direction. The direction indicator compares the desired and current positions and / or orientations and provides an output, e.g. using lights 31, to signal an operator the direction in which the equipment is to be moved to get to the desired position. The comparison could alternatively be done at the production controller, which could then provide instructions for the direction indicator to provide suitable outputs for the operator. With this system the equipment can readily be moved to a desired position and / or need for an automated support. A hybrid system may be provided, in which some motions of an equipment support are automated and some are done manually under the command of a direction indicator. In the examples described above, an operator of the 3D modelling system can select suitable locations for a camera and a light so as to achieve a desired effect when the subject is filmed. In practice, events in the studio may affect the preferred position of an item of studio equipment. For instance, it may be desirable to move a camera from the modelled position so as to accommodate cabling or an unforeseen item of furniture. If the camera by itself is moved in this way then the anticipated lighting effect may be lost. One way in which this can be mitigated will now be described. The modelling system stores desired locations and / or orientations of two interacting items of studio equipment. These may, for example be a light and a camera or a diffuser and a camera or a sound absorber and a microphone. The modelling system receives actual positions and / or orientations of the items of equipment, in the manner described above. If an operator changes the actual position or orientation of an item of equipment, that will be received by the modelling system. The modelling system may then cause another item of equipment which interacts with the manually moved item of equipment to be moved automatically, for example to preserve an operational relationship between them. The motion needed to preserve that operational relationship may depend on the nature of the equipment. In one example, the second item of equipment may be moved in a way such that it rotates about the subject in the same direction and by the same angle as the manually moved item of equipment as been rotated about the subject. In another example, the second item of equipment may be moved so as to scale about the subject in a proportional way to the manually moved item of equipment. In this way, the operational relationship between the items of equipment may be preserved without the need for an operator to manually move the second item of equipment or determine a suitable position for the second item of equipment. In another arrangement, the location of the subject may be sensed, for example using a mechanism as described above. Then when the subject moves one or more items of equipment may be moved to preserve an operational relationship with the subject and / or with each other. In the cases described above, the items of studio equipment may be moved automatically or under the guidance of an indicator system as described above. The modelling system may be separated from the studio control system. The applicant hereby discloses in isolation each individual feature described herein and any combination of two or more such features, to the extent that such features or combinations are capable of being carried out based on the present specification as a whole in the light of the common general knowledge of a person skilled in the art, irrespective of whether such features or combinations of features solve any problems disclosed herein, and without limitation to the scope of the claims. The applicant indicates that aspects of the present invention may consist of any such individual feature or combination of features. In view of the foregoing description it will be evident to a person skilled in the art that various modifications may be made within the scope of the invention.

Claims

1. A method for configuring a filming environment, the method comprising:providing in the filming environment an item of filming equipment mounted on a poseable support;in a modelling system having one or more processors configured to execute code for forming a three-dimensional model, modelling the appearance of the filming environment and the performance of the item of filming equipment, and simulating the performance of the filming environment with the item of filming equipment in a determined location and / or orientation relative to the filming environment;repeatedly receiving at the modelling system a sensed position and / or orientation of the item of filming equipment; andtransmitting from the modelling system, in dependence on the sensed position, a command for causing the item of filming equipment to be moved on the poseable support to the determined location and / or orientation.

2. A method as claimed in claim 1, wherein the poseable support comprises a powered motion element and the command comprises a command for the motion element to adopt a specified configuration.

3. A method as claimed in claim 2, wherein the poseable support is an articulated boom.

4. A method as claimed in claim 2, wherein the poseable support is a tripod.

5. A method as claimed in claim 1, wherein the poseable support is manually adjustable, the method comprises providing at the item of filming equipment an indicator for indicating a desired motion direction and / or axis of the item of filming equipment, and the command comprises a command for causing the indicator to indicate a direction in which the item of filming equipment can be moved to reach the determined position and / or location.

6. A method as claimed in any preceding claim, wherein the poseable support comprises one or more sensors for sensing the pose of the support, and the method comprises transmitting data from the sensors to the modelling system.

7. A method as claimed in any preceding claim, comprising providing at the item of filming equipment a sensor apparatus capable of visually sensing a field of irregularly positioned indicia and estimating therefrom a position and / or orientation of the item of filming equipment, and transmitting data from the sensor apparatus to the modelling system.

8. A method as claimed in any preceding claim, comprising recalling at the modelling system a stored, previously determined location and / or orientation and transmitting from the modelling system a command for causing the item of filming equipment to be moved on the poseable support to the previously determined location and / or orientation.

9. A method as claimed in any preceding claim, wherein the item of filming equipment is one of a camera, a light, a reflector, a diffuser and a microphone.

10. A method as claimed in any preceding claim, wherein the method comprises receiving the position and / or orientation of a first item of equipment moved manually, and causing a second item of filming equipment which interacts with the first item to be moved automatically to preserve an operational relationship between the first and second items of equipment.

11. A method as claimed in any preceding claim, wherein the method comprises sensing a subject and causing an item of equipment to be moved when the subject moves to preserve an operational relationship with the subject.

12. A filming system, comprising:an item of filming equipment mounted on a poseable support;a modelling system having one or more processors configured to execute code for forming a three-dimensional model, modelling the appearance of the filming environment and the performance of the item of filming equipment, and simulating the performance of a filming environment with the item of filming equipment in a determined location and / or orientation relative to the filming environment;wherein the modelling system repeatedly receives a sensed position and / or orientation of the item of filming equipment; andthe modelling system transmits, in dependence on the sensed position, a command for causing the item of filming equipment to be moved on the poseable support to the determined location and / or orientation.

13. A filming system as claimed in claim 12, wherein the modelling system receives the position and / or orientation of a first item of equipment moved manually, and causes a second item of filming equipment which interacts with the first item to be moved automatically to preserve an operational relationship between the first and second items of equipment.

14. A filming system as claimed in claim 12, wherein a subject is sensed and the modelling system causes an item of equipment to be moved when the subject moves to preserve an operational relationship with the subject.

Images