Capturing metadata with a production tool

Abstract

Examples described herein provide devices, systems, and methods for capturing metadata using a production tool. One example production tool includes a camera for capturing a spherical 360˚ image of an environment and a sound sensor for capturing an audio footprint of the environment. The sound sensor is a spatial sensor including two or more microphones. An electronic processor is configured to extract light metadata from the spherical 360˚ image, extract audio metadata from the audio footprint, and generate a data representation package that includes the light metadata and the audio metadata. The production tool includes a graphical user interface configured to display the data representation package.

Description

CAPTURING METADATA WITH A PRODUCTION TOOEBACKGROUND1. Cross-Reference to Related Applications

[0001] This application claims the benefit of priority from U.S. Provisional Application No. 63 / 667,571 filed on 3 July 2024, and European Application No. 24203664.8 filed on 30 September 2024, each of which is incorporated by reference herein in its entirety.2. Field of the Disclosure

[0002] This application relates generally to a system utilized by a production tool and, particularly, to capturing metadata associated with an environment using the production tool.3. Description of Related Art

[0003] Clapperboards are a key production tool used in television and film production to identify scenes and takes as they are filmed. Traditionally, clapperboards have provided an audio-visual synchronization, along with information about the production, to assist in editing. More recently, electronic boards and applications, such as MovieSlate® and Timecode+, have been developed for displaying, capturing, storing, and transmitting metadata pertaining to a video clip.

[0004] Further prior techniques provide methods of capturing images of scenes taken using a clapperboard which has a color table. Images are separated into action images, which are used to make a film, CF, or TV program, and reference images of the clapperboard. Color information of each color plate is extracted and converted into a digital value.BRIEF SUMMARY OF THE DISCLOSURE

[0005] Enhancing the clapperboard with various sensors and a display enables considerably more information to be captured on set and presented within the rushes, or the raw footage before postproduction, in such a manner that is machine readable and easily accessible. This information can then be used to streamline post-production of the content, especially in the use cases where volumetric assets are being created for Extended Reality (XR) presentation.

[0006] Various aspects of the present disclosure relate to a production tool including a camera for capturing a spherical 360° image of an environment and a sound sensor for capturing an audio footprint of the environment. The sound sensor may be a spatial sensor including two or more microphones. An electronic processor is configured to extract light metadata from the spherical 360° image, extract audio metadata from the audio footprint, and generate a data representation package that includes the light metadata and the audio metadata. The production tool includes a graphical user interface configured to display the data representation package.

[0007] According to some aspects, the present disclosure relates to a method of sensing an environment. The method includes capturing a visual footprint of the environment with at least one visual sensor and displaying the visual footprint on a graphical user interface. The visual footprint includes a spherical 360° image of the environment. The graphical user interface includes a color reference. The method further includes capturing an audio footprint of the environment with at least one sound sensor. The sound sensor includes two or more microphones. The method includes extracting light metadata from the visual footprint and extracting audio metadata from the audio footprint. The method includes generating an extensible markup language (XML) package including the light metadata and the audio metadata and displaying, on a user interface, a representation of the XML package.

[0008] In this manner, various aspects of the present disclosure provide for capturing metadata associated with an environment using a clapperboard, and effect improvements in at least the technical fields of image projection, imaging capturing, video recording, and the like.DESCRIPTION OF THE DRAWINGS

[0009] These and other more detailed and specific features of various embodiments are more fully disclosed in the following description, reference being had to the accompanying drawings, in which:

[0010] FIG. 1 illustrates a schematic of a production tool for capturing metadata according to some aspects of the disclosure herein.

[0011] FIG. 2 illustrates the production tool as a clapperboard according to some aspects of the disclosure herein.

[0012] FIG. 3 illustrates the clapperboard of FIG. 2 and example set during operation in some aspects of the disclosure herein.

[0013] FIG. 4 illustrates an example metadata capturing system for controlling one or more clapperboards such as the clapperboard of FIG. 2.

[0014] FIG. 5 illustrates an exemplary use of the clapperboard of FIG. 2 according to some aspects of the disclosure herein.

[0015] FIG. 6 illustrates a schematic illustrating movement of visual and audio footprints obtained during operation.

[0016] FIG. 7 is a flowchart illustrating a method of sensing information associated with an environment according to some aspects of the present disclosure.DETAILED DESCRIPTION

[0017] This disclosure and aspects thereof can be embodied in various forms, including hardware, devices or circuits controlled by computer-implemented methods, computer program products, computer systems and networks, user interfaces, and application programming interfaces; as well as hardware-implemented methods, signal processing circuits, memory arrays, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), and the like. The foregoing is intended solely to give a general idea of various aspects of the present disclosure, and does not limit the scope of the disclosure in any way.

[0018] In the following description, numerous details are set forth, such as device configurations, timings, operations, and the like, in order to provide an understanding of one or more aspects of the present disclosure. It will be readily apparent to one skilled in the art that these specific details are merely exemplary and not intended to limit the scope of this application.

[0019] Moreover, while the present disclosure focuses mainly on examples in which the various embodiments are used in a tablet, it will be understood that this is merely one example of an implementation. It will further be understood that the disclosed systems and methods can be used in any device in which there is a need to identify information associated with a surrounding environment; for example, in other industries including medical, construction, and the like.

[0020] FIG. 1 illustrates a schematic of a production tool 100 for capturing metadata according to some aspects of the disclosure herein. The production tool 100 includes at least one sensor 125. The at least one sensor 125 may include an optical sensor 126, such as a camera for capturing a visual footprint 134 of a surrounding environment 138. The at least one sensor 125 may further include a sound sensor 128, such as a microphone for capturing an audio footprint 135 of the surrounding environment 138. The sound sensor 128 may be, for example, a spatial microphone that includes two or more microphones that gather audio directionally. Additional sensors may include Light Detection and Ranging (LiDaR) sensors, photodiodes, phototransistors, photoelectric sensors, infrared sensors, microphones, piezoelectric transducers, ultrasonic sensors, acoustic emission sensors, humidity sensors, thermometers, laser sensors, and the like, for capturing information associated with the surrounding environment 138, including but not limited to, sound, light, temperature, time of day, and time of year. The term “footprint” as used herein includes any data that is captured using the sensors disclosed herein. “Metadata” as used herein refers to data describing the footprint, such as the type of sensors used for capturing the visual footprint 134 and / or the audio footprint 135, camera settings for a camera that captured the visual footprint 134, a white point value, grading monitor parameters, a type of lens used by the camera, a framerate used for capturing the visual footprint 134 and / or the audio footprint 135, and the like.

[0021] The production tool 100 may include a graphical user interface (GUI) 124, for example via a display. In some instances, the display providing the GUI 124 may be a liquid-crystal display (LCD) touch screen configured to display images. In other instances, the display may be a quantum dot (QLED) display, a plasma display, a light emitting diode (LED) display, an organic LED (OLED) display, or the like. The GUI 124 may provide an emissive reference color chart 130 (e.g., a color reference chart), described below in more detail. Additionally, in some instances, a reflective color chart 131 may be implemented with the production tool 100 and provided via the GUI 124 for providing a reflective reference based on the actual lighting of a scene being captured. The GUI 124 may be associated with a user interface, such as a touch screen, a keypad, a mouse, a trackpad, or some other form of user input.

[0022] The GUI 124 may further display scene information 132 (for example, roll number, date, take number, and shot number) associated with the scene being captured. Further, the GUI 124 may display the visual footprint 134 and a QR code 136 associated with at least the captured visual andaudio footprints 134, 135. It is contemplated that the GUI 124 may display more or less of that described herein or be capable of toggling between different items for display.

[0023] In some instances, the production tool 100 may be implemented as a clapperboard device for use on a film set. FIG. 2 illustrates the production tool 100 as a clapperboard 200 according to some aspects of the disclosure herein. The clapperboard 200 may include features different than or in addition to those already discussed with regards to the production tool 100. Any features which are common to both may be omitted from the following description, and so it should be assumed that features of the production tool 100 previously described are or at least can be implemented in the clapperboard 200. The clapperboard 200 may be integral with a tablet 222 (e.g., a tablet device) having a screen 224 for displaying the GUI 124. The tablet 222 may also include the at least one sensor 125 including the optical sensor 126 and the sound sensor 128.

[0024] The optical sensor 126 may include a spherical camera capable of capturing a 360° view (for example, a spherical 360° view) of the surrounding environment 138 to define the visual footprint 134 in one spherical 360° image 234. In another example, the optical sensor 126 includes a camera integral with the tablet 222 which is rotated to capture multiple images to define the spherical 360° image 234. In some aspects, the optical sensor 126 may include a physically separate camera that remotely sends the tablet 222 the visual footprint 134. The physically separate camera may be removable and mountable to the tablet 222. According to another aspect, the optical sensor 126 includes multiple cameras working together to capture the visual footprint 134.

[0025] The optical sensor 126 may include an LiDaR sensor. The LiDaR sensor may be provided in place of or in addition to a camera as previously described. Similarly, the LiDaR sensor may be integral with the tablet 222. In another aspect, the LiDaR sensor may be physically separate from the tablet 222. According to another aspect, multiple LiDaR sensors are provided.

[0026] The sound sensor 128 may include a microphone integral with the tablet 222. In some aspects, the sound sensor 128 includes a physically separate microphone that remotely sends the tablet 222 audio captured from the surrounding environment 138 of the clapperboard 200.According to another aspect, the sound sensor 128 includes multiple microphones working together to capture audio inputs from the surrounding environment 138. It is further contemplated that the sound sensor 128 may include a speaker integral with or physically separate from the tablet 222.

[0027] The screen 224 may display the emissive reference color chart 130, e.g., a Macbeth chart or other color reference chart. In the example of FIG. 2, the different colors of the emissive reference color chart 130 are represented as different patterns. The emissive reference color chart 130 does not alter under different lighting and may be used to categorize cameras (for example, organize, select or de-select cameras for processing) ahead of a post-production workflow 624 (see FIG. 6). For example, a physical color chart is affected by lighting conditions on set and reflects light. As the emissive reference color chart 130 emits light, the emissive reference color chart 130 provides a constant color reference. In some instances, the colors shown on the emissive reference color chart 130 may be reduced in intensity to emit luminance levels that, when captured, fall into the signal to noise ratio of the camera, avoiding any potential signal crushing or clipping. Additionally, in some instances, the reflective color chart, e.g., a traditional Macbeth chart, may be attached to or proximate the tablet 222 to provide a reflective reference based on the actual lighting of a scene being captured. The reflective color chart provides practitioners such as engineers and colourists with a quick reference to establish an initial look for the scene being captured and may be utilized during the post-production workflow 624 (see FIG. 4) for providing accurate color information.

[0028] The screen 224 may further display the scene information 132, the visual footprint 134, e.g., the spherical 360° image 234 and the QR code 136. It is contemplated that the screen 224 may display more or less of that described herein, or be capable of toggling between different items for display.

[0029] While clapperboards are commonly used on film and television sets for identifying shots and takes, example clapperboards described herein provide additional means for displaying, capturing, storing, and transmitting data and metadata associated with such film and television sets. For example, as the clapperboard 200 is equipped with a variety of sensors, such as the optical sensor 126 and the sound sensor 128, the clapperboard 200 may assist with monitoring a status of the film set (e.g., the audio and visual footprints) and detecting changes within the set. Changes may include movement of lighting fixtures, changes in weather and ambient lighting, and the like. The detected status and changes may be accounted for during post-production by content creators as they edit the captured audio and video content, as described below in more detail.

[0030] FIG. 3 illustrates an example set 300 (e.g., a film set or a television set) for use during filming of a movie or television show including a plurality of set tools, such as a set camera 312, aset light 314, a set microphone 316 and the clapperboard 200. The set 300 may be located indoors, outdoors, on a sound stage, include a green screen, or the like. The set camera 312 may be used to take pictures (or images) of the set 300. The set camera 312 may be any image acquisition device configured to take pictures from a scene, and may be digital or analog (e.g., photosensitive chemical film-based). The set light 314 may be any type of lighting source for illuminating the set 300 or may be part of a scene being captured on the set. The set microphone 316 may be any type of sound capturing device for capturing sound on the set 300. The set microphone 316 may be a plurality of microphones and may include omnidirectional, unidirectional, and / or bidirectional microphones.

[0031] A production system 330 includes, but is not limited to, the clapperboard 200, a computer 332, a network 334, a server 336, providing external storage, and a tablet memory 338, providing local storage. As is illustrated the network 334 provides a communication link between the clapperboard 200 and the server 336 and / or the computer 332. The network 334 may be, for example, a Long-Term Evolution (LTE) network, a Bluetooth™ network, a Wi-Fi network, or other similar communication networks. The server 336 can be separate from the computer 332 as illustrated. It is also contemplated that the computer 332 provides additional external storage.

[0032] The clapperboard 200 may be in direct or wireless communication, e.g. via the network 334, with the set camera 312, the set light 314, and / or the set microphone 316. Audio and visual content collected from the set camera 312, the set light 314, and / or the set microphone 316 may be stored in the server 336. Further, audio and visual content collected from the clapperboard 200 may also be stored in the server.

[0033] The production tool 100 may include a control system. FIG. 4 provides a block diagram 400 representing a control system of the clapperboard 200. The clapperboard 200 includes, for example, the optical sensor 126, the sound sensor 128, and a controller 402. A controller 402 includes, among other things, an electronic processor 404, a memory 406, and a transceiver 414. The electronic processor 404, the memory 406, and the transceiver 414 communicate over one or more control and / or data buses. FIG. 2 illustrates only one example of the controller 402. The controller 402 may include more or fewer components and may perform functions other than those explicitly described herein. In some examples, the electronic processor 404 is implemented as a microprocessor with separate memory 406. In other examples, the electronic processor 404 is implemented as a microcontroller, where the memory 406 is on the same chip. The electronic processor 404 may beimplemented with multiple processors, and may be implemented partially or entirely as, for example, a field-programmable gate array (FPGA) or an applications specific integrated circuit (ASIC).

[0034] The memory 406 includes non-transitory, computer-readable memory that stores instructions that are received and executed by the electronic processor 404 to carry out the functionality of the clapperboard 200 described herein. The memory 406 may include, for example, combinations of different types of memory, such as read-only memory and random-access memory.

[0035] The transceiver 414 allows the controller 402 to perform wired and / or wireless communications with various set tools and storage locations as previously described herein (see FIG. 3) over the network 334. The transceiver 414 may also handle communication with various components and devices of the clapperboard 200 connected to the controller 402, such as the optical sensor 126, and the sound sensor 128, the set camera 312, the set light 314, and the set microphone 316.

[0036] The controller 402 receives and processes data from connected components to operate the clapperboard 200. For example, the controller 402 receives image and / or video of the surrounding environment 138 from the optical sensor 126. The controller 402 receives sound information regarding the surrounding environment including speech and other sound present from the sound sensor 128 and / or the set microphone 316.

[0037] In some instances, the clapperboard 200 is configured to capture an impulse response of the set 300. For example, as shown in FIG. 5, the sound sensor 128 may include a speaker 500 and a microphone 510. The speaker 500 may emit a test signal 512 into the set 300. The test signal 512 travels through the set 300 and may reflect off objects and props situated within the set 300. The microphone 510 detects the test signal 512 and corresponding reflections. The received test signal 512 and the corresponding reflections may be used by the controller 400 to generate a room impulse response (RIR) 514 of the surrounding set 300. In some instances, the optical sensor 126 may capture a visual representation of the set 300 (e.g., the visual footprint) in addition to the speaker 500 emitting the test signal 512. The controller 402 may identify the distance between objects and props situated within the set 300 based on the visual footprint. When the sound sensor 128 is used in combination with the optical sensor 126, the controller 402 generates the RIR 514 based on the distance between objects within the set 300 and the detected reflections of the test signal 512. Whensound effects and dialogue are added during post-production, the RIR 514 may be used in the postproduction workflow 624 (FIG. 6) to apply shaping to added sounds to simulate the sound on the set 300 during filming.

[0038] The optical sensor 126 may include a spherical camera 520 for capturing the spherical 360° image 234. The spherical 360° image 234 may be processed to produce the visual footprint 134 in the form of an image-based lighting model (for example, an input for image-based lighting applications). The image-based lighting model identifies positions of the lighting sources, e.g., the set light 314. The image-based lighting model may be used by the controller 402 to generate a computer-generated imagery (CGI) model of the set 300 or objects, props, and actors within the set 300. In some implementations, the image-based lighting model is taken at every take, and therefore provides a map of any lighting changes or failures during filming. Accordingly, in the event a significant change has occurred to any lighting on the set 300, the production team may be alerted (for example, via the display) to adjust the lighting. Examples of a significant change in lighting include a light changing color profile from a burn out or a change in location of reflectors or lighting (not illustrated). In situations where the optical sensor 126 includes a LiDaR sensor, the results of the LiDaR sensor may be used to produce a depth-map. The depth-map may be implemented by the controller 402 when generating the CGI model of the set 300.

[0039] Once captured, data and metadata associated with the film and television set described herein must be packaged and transported for unpackaging during post-production. The clapperboard 200 described herein is an example of the production tool 100 capable of collecting the data and metadata. While clapperboards are commonly used on film and television sets, it should be understood that the production tool 100 need not be a clapperboard, nor look like a clapperboard. For example, operations described with respect to the production tool 100 and the clapperboard 200 may instead be performed by a mobile device, such as a tablet. The production tool 100 described herein is capable of capturing the data and metadata needed and either packaging the necessary data or sending the necessary data to a processor for packaging for later use during rushes.

[0040] Turning to FIG. 6, packaging and transporting the data and meta data for unpackaging is depicted in a schematic illustrating the movement of the visual and audio footprints 134, 135 from filming to the post- production workflow 624 is illustrated. The visual footprint 134 may be an image 600, e.g., the spherical 360° image 234, captured by the optical sensor 126 at a rate from 24 to 30frames per second (fps). Each frame 610a, 610b, 610n from the image 600 includes a block of data 612a, 612b, 612n, where “n” is the total number of frames. The blocks of data 612a, 612b, 612n include distinct light metadata “Lm” (e.g., illumination and light source related metadata, light and illumination intensity, color and distribution metadata) and audio metadata “Am” (e.g., audio and acoustic metadata) associated with the set 300. The light metadata Lmincludes data associated with the visual footprint 134, such as a type of the optical sensor 126, a lens associated with the optical sensor 126, a time of day (associated with ambient lighting), a location of the set 300, a framerate of the optical sensor 126, characteristics of the set camera 312, and the like. The audio metadata Amincludes data associated with the audio footprint 135, such as a type of the sound sensor 128, a position of the sound sensor 128 within the set 300, data associated with the RIR of the set 300, and the like. The light metadata Lmand the audio metadata Ammay be extracted from the visual footprint 134 and the audio footprint 135, respectively, by the controller 402. For example, the controller 402 may implement a visual analysis algorithm on the visual footprint 134 to extract the light metadata Lm. In other instances, at least portions of the light metadata Lmmay be provided by a user via a user input (for example, the selection of a particular camera, the input of the time of day, and the like). Similarly, the controller 402 may implement an audio analysis algorithm on the audio footprint 135 to extract the audio metadata Am, may receive the audio metadata Amvia a user input, or a combination thereof.

[0041] The light metadata Lmand audio metadata Ammay be processed by the electronic processor 404 (or one or more processors, such as a central processing unit, CPU) to generate a data representation package, such as an extensible markup language (XML) package 622 (or some other data representation format). The contents of XML package 622 may be converted to the QR code 136 and displayed on the GUI 124. The QR code 136 link directly to the XML package 622 or may be a URL linked to the XML package 622. The XML package 622 enables the information from the blocks of data 612a, 612b, 612n to be machine readable for extracting the information automatically in the rushes. As this is captured visually, unlike manual notes, it cannot be lost or separated from the content.

[0042] Additional information from the set 300 may be captured in the same XML package 622, for example, a visual footprint from a drone image from above. Traditional metadata may also be included in the XML package 622, such as GPS information, temperature, cast and crew details,shot number, actors in vision, and the like. The XML package 622 may be stored locally, e.g., in the tablet memory 338 of the tablet 222 to provide immediate feedback to the crew during filming.

[0043] The XML package 622 includes information significantly valuable to extract during the postproduction workflow 624. The post-production workflow 624 may include, for example, adjusting or modifying colors or brightness in particular areas of an image to enhance the image quality or achieve a particular appearance for the image. Such post-production editing may be referred to as “color timing” or “color grading.” Other editing (e.g., scene selection and sequencing, image cropping, addition of computer-generated visual special effects, etc.) may be performed to produce a final product for distribution. During the post-production workflow 624, content for editing may be viewed on a reference display 626.

[0044] FIG. 7 is a flowchart illustrating a method 700 of sensing information associated with the surrounding environment 138, e.g., the set 300. The method 700 may be performed by, for example, the controller 402. Additionally, the steps provided within FIG. 7 are merely examples, and may instead be conducted in a different order or simultaneously. The method 700 may be performed on the set 300 using the production tool 100, and more specifically the clapperboard 200, described herein.

[0045] At block 710, the method 700 includes capturing the visual footprint 134 of the surrounding environment 138 with at least one optical sensor 126. For example, the controller 402 captures the spherical 360° image 234 using the spherical camera 520.

[0046] At block 720, the method 700 includes displaying the visual footprint 134 on the GUI 124, for example on the screen 224. The screen 224 also displaying the emissive reference color chart 130.

[0047] At block 730, the method 700 includes capturing the audio footprint 135 of the surrounding environment 138 with at least one sound sensor 128. For example, the controller 402 captures the RIR 514 with the microphone 510.

[0048] At block 740, the method 700 includes extracting light metadata Lmfrom the visual footprint 134. For example, the controller 402 may implement a visual analysis algorithm on the visual footprint 134, may receive a user input indicating the light metadata Lm, or a combination thereof.

[0049] At block 750, the method 700 incudes extracting audio metadata Amfrom the audio footprint 135. For example, the controller 402 may implement an audio analysis algorithm on the audio footprint 135, may receive a user input indicating the audio metadata Am, or a combination thereof.

[0050] At bock 760, the method 700 includes generating the extensible markup language (XML) package 622 including the light metadata Lmand the audio metadata Amassociated with the surrounding environment 138.

[0051] The method 700 may further include storing the XML in the QR code 136 or in the server 336. Further, at least a portion of the XML package 622 may be stored locally in the tablet memory 338 to provide immediate feedback.

[0052] The method 700 may further include communicating with at least one light source, e.g., the set light 314 in the surrounding environment 138. Further, the method may include converting the audio footprint 135 to the RIR 514.

[0053] Systems, methods, and devices in accordance with the present disclosure may take any one or more of the following configurations.

[0054] Clause 1. A production tool comprising: a camera for capturing a spherical 360° image of an environment; a sound sensor for capturing an audio footprint of the environment, wherein the sound sensor is a spatial sensor including two or more microphones; an electronic processor configured to: extract light metadata from the spherical 360° image, extract audio metadata from the audio footprint, and generate a data representation package that includes the light metadata and the audio metadata; and a graphical user interface configured to display the data representation package.

[0055] Clause 2. The production tool according to clause 1, wherein the camera includes a spherical 360° camera.

[0056] Clause 3. The production tool according to clause 1, wherein the camera includes multiple cameras.

[0057] Clause 4. The production tool according to any one of clauses 1 to 3, wherein the sound sensor detects a direction of received audio, wherein the direction is indicated in the audio metadata.

[0058] Clause 5. The production tool according to any one of clauses 1 to 4, wherein the graphical user interface is further configured to display a color reference chart.

[0059] Clause 6. The production tool of clause 5, wherein the graphical user interface comprises an emissive display configured to display the color reference chart.

[0060] Clause 7. The production tool according to any one of clauses 1 to 6, wherein the data representation package is an extensible markup language (XML) package converted to a QR code.

[0061] Clause 8. The production tool according to any one of clauses 1 to 7, wherein the data representation package is stored in a server.

[0062] Clause 9. The production tool according to any one of clauses 1 to 8, wherein at least a portion of the data representation package includes stored locally on the production tool.

[0063] Clause 10. The production tool according to any one of clauses 1 to 9, further comprising a tablet device, wherein the camera is removable and mountable to the tablet.

[0064] Clause 11. The production tool according to clause 10, further comprising a plurality of light sources are configured to communicate characteristics associated with each of the plurality of light sources through wireless communication with the tablet device and / or at least one camera.

[0065] Clause 12. The production tool according to any one of clauses 1 to 11, wherein the electronic processor is configured to convert the audio footprint to a room impulse response (RIR).

[0066] Clause 13. The production tool according to any one of clauses 1 to 12, wherein the light metadata includes data characterizing the spherical 360° image.

[0067] Clause 14. The production tool according to any one of clauses 1 to 13, wherein the audio metadata includes data characterizing the audio footprint.

[0068] Clause 15. A method of capturing metadata with a production tool, the method comprising: capturing a visual footprint of the environment with a visual sensor, wherein the visual footprint includes a spherical 360° image of the environment; displaying the visual footprint on a graphical user interface, the graphical user interface including a color reference; capturing an audio footprint of the environment with a sound sensor, wherein the sound sensor includes two or more microphones; extracting light metadata from the visual footprint; extracting audio metadata from theaudio footprint; generating an extensible markup language (XML) package including the light metadata and the audio metadata; and displaying, on a user interface, a representation of the XML package.

[0069] Clause 16. The method according to clause 15, further comprising converting and storing the content of the data representation package in a QR code and wherein displaying, on the user interface, the representation of the XML package comprises displaying, on the user interface, the QR code.

[0070] Clause 17. The method according to clause 15, further comprising storing the XML package in a server.

[0071] Clause 18. The method according to any one of clauses 15 to 17, further comprising storing at least a portion of the XML package locally on the production tool.

[0072] Clause 19. The method according to any one of clauses 15 to 18, further comprising converting the audio footprint to a room impulse response (RIR).

[0073] Clause 20. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method according to any one of clauses 15 to 19.

[0074] With regard to the processes, systems, methods, heuristics, etc. described herein, it should be understood that, although the steps of such processes, etc. have been described as occurring according to a certain ordered sequence, such processes could be practiced with the described steps performed in an order other than the order described herein. It further should be understood that certain steps could be performed simultaneously, that other steps could be added, or that certain steps described herein could be omitted. In other words, the descriptions of processes herein are provided for the purpose of illustrating certain embodiments, and should in no way be construed so as to limit the claims.

[0075] Accordingly, it is to be understood that the above description is intended to be illustrative and not restrictive. Many embodiments and applications other than the examples provided would be apparent upon reading the above description. The scope should be determined, not with reference to the above description, but should instead be determined with reference to the appended claims,along with the full scope of equivalents to which such claims are entitled. It is anticipated and intended that future developments will occur in the technologies discussed herein, and that the disclosed systems and methods will be incorporated into such future embodiments. In sum, it should be understood that the application is capable of modification and variation.

[0076] All terms used in the claims are intended to be given their broadest reasonable constructions and their ordinary meanings as understood by those knowledgeable in the technologies described herein unless an explicit indication to the contrary in made herein. In particular, use of the singular articles such as “a,” “the,” “said,” etc. should be read to recite one or more of the indicated elements unless a claim recites an explicit limitation to the contrary.

[0077] The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments incorporate more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims arc hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.

Claims

CLAIMSWhat is claimed is:

1. A production tool comprising: a camera for capturing a spherical 360° image of an environment; a sound sensor for capturing an audio footprint of the environment, wherein the sound sensor is a spatial sensor including two or more microphones; an electronic processor configured to: extract light metadata from the spherical 360° image, extract audio metadata from the audio footprint, and generate a data representation package that includes the light metadata and the audio metadata; and a graphical user interface configured to display the data representation package.

2. The production tool of claim 1, wherein the camera includes a spherical 360° camera.

3. The production tool of claim 1, wherein the camera includes multiple cameras.

4. The production tool of claim 1, wherein the sound sensor is configured to detect a direction of received audio, wherein the direction is indicated in the audio metadata.

5. The production tool according to any one of claims 1 to 4, wherein the graphical user interface is further configured to display a color reference chart.

6. The production tool of claim 5, wherein the graphical user interface comprises an emissive display configured to display the color reference chart.

7. The production tool according to any one of claims 1 to 6, wherein the data representation package is an extensible markup language, XML, package converted to a QR code.

8. The production tool according to any one of claims 1 to 7, wherein the data representation package is configured to be stored in a server.

9. The production tool according to any one of claims 1 to 8, wherein at least a portion of the data representation package is stored locally on the production tool.

10. The production tool according to any one of claims 1 to 9, further comprising a tablet device, wherein the camera is removable and mountable to the tablet device.

11. The production tool according to any one of claims 1 to 10, further comprising a speaker for emitting a test signal into the environment, wherein when the sound sensor is used in combination with the camera, the electronic processor is configured to generate a room impulse response, RIR, based on the distance between objects within the environment and detected reflections of the test signal.

12. The production tool of any one of claims 1 to 11 , wherein the light metadata includes data characterizing the spherical 360° image.

13. The production tool of any one of claims 1 to 12, wherein the audio metadata includes data characterizing the audio footprint.

14. A method of capturing metadata with a production tool, the method comprising: capturing a visual footprint of the environment with a visual sensor; displaying the visual footprint on a user interface, the user interface including a color reference; capturing an audio footprint of the environment with a sound sensor; extracting light and illumination metadata from the visual footprint; extracting audio metadata from the audio footprint; generating an extensible markup language, XML, package including the light metadata and the audio metadata; and displaying, on a user interface, a representation of the XML package.

15. The method of claim 14, further comprising converting and storing the XML package in a QR code and wherein displaying, on the user interface, the representation of the XML packagecomprises displaying, on the user interface, the QR code.

16. The method of claim 14 or 15, further comprising storing the XML package in a server.

17. The method of any one of claims 14 to 16, further comprising storing at least a portion of theXML package locally on the production tool.

18. The method of any one of claims 14 to 17, further comprising emitting a test signal into the environment, wherein when the sound sensor is used in combination with the visual sensor, the electronic processor is configured to generate a room impulse response, RIR, based on the distance between objects within the environment and detected reflections of the test signal..

19. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 14

Images