Plenoptic Camera Measurement and Calibration for Head Mounted Displays

Abstract

A method for measuring the performance of a head-mounted display module, the method including: arranging the head-mounted display module relative to a plenoptic camera assembly so that an exit pupil of the head-mounted display module coincides with a pupil of the plenoptic camera assembly; emitting light from the head-mounted display module while the head-mounted display module is arranged relative to the plenoptic camera assembly; filtering the light at the exit pupil of the head-mounted display module; obtaining one or more brightfield images projected from the head-mounted display module together with the filtered light using the plenoptic camera assembly; and determining information about the performance of the head-mounted display module based on the obtained brightfield images.

Description

[Technical field]

[0001] (Priority Claim) This application claims priority under 35 U.S.C. §119(e) to U.S. patent application Ser. No. 63 / 048,331, filed July 6, 2020, the entire contents of which are incorporated by reference herein.

[0002] FIELD OF THE DISCLOSURE The present disclosure relates to the measurement and calibration of head mounted displays, and more particularly to the use of a plenoptic camera for the calibration of eyepieces used in head mounted displays. [Background technology]

[0003] (background) Wearable head mounted display systems include one or more eyepieces through which a user views the outside world. The eyepieces are typically formed from a transparent, highly refractive material so that information can be projected to the user through the eyepieces while simultaneously transmitting a view of the outside world. Often, the eyepieces in the head mounted displays undergo a calibration process to normalize the bright field displayed to a user of the wearable display system and ensure consistent images between head mounted display systems.

[0004] In some cases, the calibration process involves evaluation of multiple white field images projected through the eyepiece across a large dynamic range. These images can be obtained using a camera and various neutral density filters. A similar process can be used to evaluate red, green, and blue color balance, as well as several offsets and positions from the eyepiece, simulating multiple user pupil locations and interpupillary distances. From these images, the calibration process can include adjusting the head mounted display to match the image field to an external standard. Typically, these steps must be performed for each head mounted display system, resulting in a time-consuming and costly process. Summary of the Invention [Means for solving the problem]

[0005] (summary) This disclosure describes a method and system for calibrating a head-mounted display using a plenoptic camera. A plenoptic camera, also called a bright-field camera, captures information about the bright field emanating from a scene. The bright field refers to both the intensity of the light from the scene and also the direction in which the light rays travel in space. In contrast, a normal camera captures only intensity information about the scene.

[0006] In the systems disclosed herein, the optical system for the plenoptic camera features a pupil that is physically accessible to array spatially discrete filters across the pupil and enable multiplexed measurements of different light properties (e.g., luminance, polarization, spectral content) across the pupil. For example, a plenoptic camera assembly in a measurement and calibration system can include an entrance pupil that is external to the lens of the camera assembly, providing sufficient physical space for one or more filter arrays. The system can include a stage that positions the head-mounted display under test such that the exit pupil of the display is co-located with the entrance pupil for the plenoptic camera.

[0007] In some embodiments, the measurement and calibration system can include an optical assembly that provides views from several pupil positions in a single image, allowing calibration of several user pupil positions in one capture, saving measurement and calibration time.

[0008] Various aspects of the invention are summarized as follows.

[0009] In general, in a first aspect, the invention features a method for measuring performance of a head-mounted display module, the method including aligning a head-mounted display module with a plenoptic camera assembly such that an exit pupil of the head-mounted display module coincides with a pupil of the plenoptic camera assembly, emitting light from the head-mounted display module while the head-mounted display module is aligned with the plenoptic camera assembly, filtering the light at the exit pupil of the head-mounted display module, obtaining one or more brightfield images projected from the head-mounted display module along with the filtered light using the plenoptic camera assembly, and determining information about performance of the head-mounted display module based on the obtained brightfield images.

[0010] An embodiment of the method for displaying an image using a wearable display system may include one or more of the following features: The light may be filtered by a plurality of spatially discrete filters positioned at a pupil of the plenoptic camera assembly. The spatially discrete filters may include color filters. The color filters may include X, Y, Z color matching function color filters. The spatially discrete filters may include polarizing filters. The spatially discrete filters may include neutral density filters.

[0011] The light can be filtered by a first set of spatially discrete filters and a second set of spatially discrete filters that overlap the first set, the first and second sets of spatially discrete filters filtering different properties of the light. The property of the light can be selected from the group consisting of color, polarization, and intensity.

[0012] Obtaining one or more bright field images may include re-imaging a real image from the head mounted display module onto a multi-element sensor using a microlens array.

[0013] The microlens array can sample a portion of the exit pupil of the lens of the plenoptic camera assembly to provide different angular views of the real image from the head-mounted display module.

[0014] The plenoptic lens assembly can define an optical path from the pupil to the sensor, and the plenoptic lens assembly can include a camera lens assembly in the optical path that defines a pupil of the plenoptic camera assembly and defines an image plane, and the plenoptic camera assembly can further include an array of focusing elements in the optical path between the image plane and the sensor array.

[0015] The information about the performance of the head mounted display module may include information about at least one of the performance parameters selected from the group consisting of radiance, luminance, color, geometric distortion, virtual image distance, and field curvature.

[0016] Determining information about the performance of the head mounted display module can include computing two-dimensional images at a number of different depths across a three-dimensional volume of interest of the head mounted display module.

[0017] Determining the information may further include determining information about one or more properties of each of the two-dimensional images.

[0018] The method may further include combining light emitted from multiple different locations of an exit pupil of the head-mounted display module to form multiple overlapping images at a sensor of the plenoptic camera assembly, each of the multiple overlapping images corresponding to a different user view for the head-mounted display module.

[0019] In a second aspect, the invention features a method for calibrating a head mounted display, the method including measuring performance of the head mounted display using a method for measuring performance of a head mounted display module, and adjusting operation of the head mounted display based on the measured performance.

[0020] In a third aspect, the invention features a system including a plenoptic camera assembly including a camera lens defining an image plane, a camera sensor, and a microlens array arranged to image light at the image plane to the camera sensor; a stage for receiving a head mounted display and positioning the head mounted display relative to the plenoptic camera assembly such that an exit pupil of the head mounted display module coincides with a pupil of the plenoptic camera assembly; one or more filter arrays positioned at the pupil of the plenoptic camera assembly, each of the one or more filter arrays including a plurality of spatially discrete filters extending across an aperture of the camera lens; and a system controller in communication with the plenoptic camera assembly and programmed to obtain one or more brightfield images projected from the head mounted display module during operation of the system and to determine information about performance of the head mounted display module based on the obtained brightfield images.

[0021] Embodiments of the system can include one or more of the following features: One of the filter arrays can include spatially discrete color filters. The spatially discrete color filters can include X, Y, Z tristimulus color filters.

[0022] One of the filter arrays can include spatially discrete polarizing filters.

[0023] The filter array can include one of spatially discrete neutral density filters.

[0024] The one or more filter arrays can include a first array of spatially discrete color filters and a second array of spatially discrete filters overlapping the first set, where the first and second arrays of spatially discrete filters filter different properties of light, the properties of light being selected from the group consisting of color, polarization, and intensity.

[0025] The system further includes an optical assembly arranged at the pupil of the plenoptic camera assembly and configured to combine light emitted from multiple different locations of the exit pupil of the head-mounted display module and form multiple overlapping images at a sensor of the plenoptic camera assembly, each of the multiple overlapping images corresponding to a different user view for the head-mounted display module.

[0026] The optical assembly may include one or more polarizing beam splitters arranged at different locations within an exit pupil of the head-mounted display module, and a beam combiner arranged within the pupil of the plenoptic camera assembly arranged to receive light from each of the polarizing beam splitters and direct the light from each polarizing beam splitter along a common path to a camera lens.

[0027] Other features and advantages will become apparent from the description, the drawings, and the claims. [Brief description of the drawings]

[0028] [Figure 1] FIG. 1 is a schematic diagram of an exemplary measurement and calibration system for a wearable headset display.

[0029] [Figure 2A] FIG. 2A is a plan view of an exemplary color filter array for use in the system shown in FIG.

[0030] [Figure 2B] FIG. 2B is a plan view of an exemplary neutral density filter array for use in the system shown in FIG.

[0031] [Figure 2C] FIG. 2C is a plan view of an exemplary polarizing filter array for use in the system shown in FIG.

[0032] [Diagram 3] FIG. 3 is a schematic diagram of an exemplary optical assembly that extends the entrance pupil of a plenoptic camera to encompass multiple user pupil positions.

[0033] [Figure 4] FIG. 4 is a schematic diagram illustrating an embodiment of a wearable headset display system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] In the drawings, like symbols indicate like elements.

[0035] (Detailed Description) 1, an exemplary measurement and calibration system 100 for a wearable headset display 150 (e.g., an eyepiece) includes a plenoptic camera assembly 101, a filter module 120, and a stage 130 for supporting the display 150 and positioning it relative to the plenoptic camera assembly 101. Specifically, the display 150 is arranged such that an exit pupil of the display 150 coincides with an entrance pupil 103 of the plenoptic camera assembly 101. A system controller 110 (e.g., a computer controller), in communication with the plenoptic camera assembly 101 and the display 150, is programmed to control the operation of both the camera assembly 101 and the display 150, and to obtain and analyze images from the display to calibrate the display.

[0036] Generally, entrance pupil 103 is in a location that is accessible to other optical components that are not part of the imaging optics of system 100. For example, as depicted in FIG. 1, entrance pupil 103 is external to plenoptic camera assembly 101 in the space between the camera assembly and display 150.

[0037] A filter module 120 is positioned at the entrance pupil 103. The filter module 120 includes three spatially discrete filters 120a, 120b, and 120c, each co-located with respect to the axis 102. The filters are spatially discrete because they occupy non-overlapping areas of the entrance pupil 103. A mounting 121 positions the positional filter array 120 at the appropriate location along the axis 102.

[0038] Plenoptic camera assembly 101 includes a sensor 106, a microlens array 105, and a camera lens 104, arranged in sequence along a camera assembly axis 102. Sensor 106, microlens array 105, and camera lens 104 are housed within a common housing that features mounting elements to maintain the relative positions of each component and protect each from the environment.

[0039] A stage 130 supports the display 150 facing the plenoptic camera assembly 101 such that light 160 from an image projected by the display 150 is received by the camera assembly along an axis 102. The camera lens 104 images the display 150 to an image plane 108. Each microlens of the microlens array 105 relays an image of the display to the sensor, however, each of the images at the sensor 106 captures information about the direction of the light in addition to the intensity. It should be noted that the sensor generally has many more pixels than the microlens array 105 has lenses, allowing each microlens to form an image on a different area of ​​the sensor. The sensor 106 is typically a solid-state image sensor device that includes a regular array of pixels. For example, the sensor 106 can be a charge-coupled device (CCD) or an active pixel sensor (CMOS). The resulting image read from sensor 106 corresponds to an array of micro-images, each corresponding to a slightly different angular viewing height of the object being imaged (in this case, display 150).

[0040] Although camera lens 104 is depicted as a single lens element, typically camera lens 104 is a compound lens including two or more lens elements that collectively image the display to image plane 108. Generally, camera lens 104 can include spherical, aspheric, conical, or anamorphic lens elements, or any combination thereof, to provide sufficiently low aberration imaging suitable for system 100.

[0041] During operation, the plenoptic camera assembly 101 captures brightfield images projected from the display 150 of the head mounted display module for processing by the system controller 110. The system controller 110 can be implemented using digital electronic circuitry or in computer software, firmware, or hardware, or in a combination of one or more of them. For example, in some cases, the system controller 110 can be implemented, at least in part, as one or more computer programs (e.g., one or more modules of computer program instructions encoded on a computer storage medium for execution by or to control the operation of a data processing apparatus). The computer storage medium can be or can be included in a computer readable storage device, a computer readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. The term "processing apparatus" encompasses any kind of apparatus, device, and machine for processing data, including, by way of example, a programmable processor, a computer, a system on a chip, or a plurality or combination of the foregoing. The apparatus can include special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit). In addition to hardware, an apparatus may also include code that creates an execution environment for the computer program, such as code that constitutes a processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The apparatus and execution environment may implement a variety of different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures.

[0042] In general, a variety of different types of filter arrays can be used as the filter module 120. For example, in some embodiments, the filters 120a, 120b, and 120c are neutral density filters, each having a different attenuation. For example, the filters 120a, 120b, and 120c can attenuate different amounts of light from the display 150. Thus, in a single capture, the sensor 106 can obtain an image volume from the display 150 across a high dynamic range without having to capture different intensity levels that would accompany different volume captures.

[0043] Although the filter module 120 depicted in FIG. 1 includes an array of three different filters, other filter array arrangements are also possible. Furthermore, in some embodiments, multiple filter arrays can be arranged in a sequence along the axis 102. For example, referring to FIGS. 2A and 2B, the color filter array 220 and the neutral density filter array 222 are designed to be arranged in a sequence along the axis so that the measurement and calibration system can obtain bright-field images from the display that include information for different spectral profiles and across a high dynamic range, as per spectral profile information. In this example, the filter array 220 is a color filter array with three different color filters 220a, 220b, and 220c. For example, the filter 220a can be a filter for CIE1931 color matching function X, the filter 220b can be a filter for CIE1931 color matching function Y, and the filter 220c can be a filter for CIE1931 color matching function Z. Other spectral filter arrays are also possible. The filter array is depicted in plan view and arranged in the system 100 such that axis 102 is perpendicular to the plane of the page.

[0044] Filter array 222 consists of three neutral density filters (e.g., 50% or greater attenuation, 80% or greater attenuation, 90% or greater attenuation) 222a, 222d, and 222e, and three transparent windows 222b, 222c, and 222f (i.e., providing little or no light attenuation). Filter arrays 220 and 222 are sized and shaped such that, when positioned within the filter module of system 100, color filter 220a overlaps neutral density filter 222a and clear window 222b. Similarly, color filter 220b overlaps neutral density filter 222d and clear window 222c, and color filter 220c overlaps neutral density filter 222e and clear window 222f.

[0045] Thus, when arranged together in system 100, filter arrays 220 and 222 provide bright field capture in which a portion of the entrance pupil samples light with three different spectral compositions across a high dynamic range such as that provided by neutral density filters.

[0046] 2C shows a further example of a filter array that may be used in system 100. Filter array 224 includes four filters 224a, 224b, 224c, and 224d, each consisting of a linear polarizer with a different pass axis orientation. Alternatively, in some embodiments, the four polarizing filters may include two linear polarizers with orthogonal axes and two circular polarizers for orthogonal circular polarization states. Brightfield images obtained using a polarizing filter array may be useful in assessing properties such as birefringence in different components of a display under test.

[0047] Other filter array configurations are also possible. For example, a polarizing filter array could include 24 filters arranged such that four polarizing filters overlap each filter in filter array 222. With such an arrangement, a single brightfield image can contain color and polarization information across a high dynamic range.

[0048] As an alternative or in addition to a filter module, other components can be positioned within the entrance pupil of the plenoptic camera assembly 101 to provide additional functionality to the system 100. For example, in some embodiments, a multiplexing assembly can be used to capture bright fields across multiple user pupil positions within a single bright field capture. An example of such an assembly is shown in FIG. 3, which depicts an optical assembly 300 aligned with the external entrance pupil 103 of the plenoptic camera assembly 101, extending the pupil plane to capture a larger section of the exit pupil 320 of the display under test. The optical assembly 300 includes two polarizing beam splitters (PBS) 304a and 304b affixed on opposite sides of a crossed prism 302. Two high retardation films 306a and 306b are positioned between the two polarizing beam splitters 304a and 304b, respectively, and the display under test. Optionally, a linear polarizer 308 is positioned between the crossed prism 302 and the camera assembly 101 .

[0049] In general, the head mounted display's exit pupil 320 is significantly larger than the user's pupil and can accommodate multiple different user pupil positions corresponding to the user's eye movements. The assembly 300 shown in FIG. 3 expands the entrance pupil of the plenoptic camera assembly 101 to be coextensive with three different user pupil positions. In particular, arrows 310a, 310b, and 310c represent light from the display at three different pupils. The light 310a passes through a retardation film 306a before the PBS 304a directs the component of this light having a first polarization state toward the crossed prism 302. Similarly, the light 310c passes through a retardation film 306b before the PBS 304b directs the component of this light having a second polarization state toward the crossed prism 302. The retardation films 306a and 306b can have the effect of randomizing the polarization of the light from the display under test. The first and second polarization states can be identical. The light 310b can be a combination of the first and second polarization states, or can be orthogonal.

[0050] The crossed prism 302 combines the light from the PBSs 304a and 304b with the light 310b and directs this light towards the plenoptic camera assembly 101. The crossed prism 302 surfaces may be 50% reflective (e.g., 50% transmissive) or may be adjusted to create similar transmission through the three optical paths. Note that the reflective surfaces of the PBS and crossed prisms are planar surfaces and preserve the directional nature of the light rays representing the bright field at the exit pupil 320. Thus, the light entering the camera assembly is composed of light from three different pupils, and a bright field image captured from this light will contain information from each of these pupils. The performance of the sum of the three pupil positions can therefore be evaluated based on a single bright field image.

[0051] The assembly 300 is typically positioned in the optical path between the display and the plenoptic camera assembly by a mounting device (e.g., an opto-mechanical mount) that allows precise positioning of the assembly at the entrance pupil 103. In some embodiments, one or more actuators can be used to automatically switch various optical components at the entrance pupil 103. An example is a filter wheel, which can rotate different filter arrays in and out of the optical path. Manual replacement of these components is also a possibility.

[0052] In general, system 100 can include additional components in combination with those described above. For example, in some embodiments, the system can include an afocal optical relay system between the stage and the plenoptic camera assembly. Such a relay system can provide additional space in the optical path without significantly affecting the imaging properties of the optical system. Alternatively, or additionally, one or more folding mirrors can be used to fold the optical path of the system, for example, to provide a more compact form factor for the system.

[0053] In general, system 100 can be used to characterize a variety of different performance parameters associated with a display. These performance parameters can include, but are not limited to, radiance, luminance, color, geometric distortion, virtual image distance, and field curvature. Traditional bright field analysis techniques can also be used to characterize the performance of a display.

[0054] In some implementations, super-resolution techniques are used to improve the accuracy of pixel locations. For example, geometric correction may require very accurate pixel location measurements and may benefit from the use of super-resolution techniques.

[0055] Display calibration can be performed by comparing the uniformity of one or more performance parameters across the exit pupil of the display and / or comparing the performance parameters to a standard. Operation of the display can be adjusted based on the measurements and additional measurements can be performed to assess the adjustments. The measurement and adjustment sequence can be repeated until the performance parameters are within pre-established thresholds.

[0056] System 100 can be used to measure performance parameters and calibrate displays suitable for an augmented reality head mounted display system. An example of such a system is shown in FIG. 4, which shows an exemplary head mounted display system 60 including a display (or eyepiece) 70 and various mechanical and electronic modules and systems to support the functionality of the display 70. The display 70 may be housed in a frame 80, which is wearable by a display system user 20 and configured to position the display 70 in front of the user's 20 eyes. The display 70 may be considered eyewear in some embodiments. A world-viewing (e.g., facing the user's environment and having a similar field of view) camera 81 is mounted in the frame 80. In some embodiments, a speaker 90 is coupled to the frame 80 and positioned adjacent to the ear canal of the user 20. The display system may also include one or more microphones 112 to detect sound. The microphone 112 may enable the user to provide input or commands to the system 60 (e.g., voice menu command selections, natural language queries, etc.) and / or audio communication with others (e.g., other users of a similar display system). The microphone 112 may also collect audio data (e.g., sounds from the user and / or the environment) from around the user. In some embodiments, the display system may also include an ambient sensor 122a, which may be separate from the frame 80 and attached to the body of the user 20 (e.g., on the head, torso, limbs, etc.). The ambient sensor 122a, in some embodiments, may obtain data characterizing a physiological state of the user 20.

[0057] The display 70 is operably coupled by a communication link 132, such as wired or wireless connectivity, to a local data processing module 140, which may be mounted in a variety of configurations, such as fixedly attached to the frame 80, fixedly attached to a helmet or hat worn by the user, built into headphones, or otherwise removably attached to the user 20 (e.g., in a backpack configuration or in a belt-connected configuration). Similarly, the sensor 122a may be operably coupled to the local processor and data module 140 by a communication link 122b (e.g., wired or wireless connectivity). The local processing and data module 140 may include a hardware processor and digital memory, such as non-volatile memory (e.g., flash memory or hard disk drive), both of which may be utilized to aid in processing, caching, and storing data. The data may include 1) data captured from sensors (e.g., which may be operatively coupled to the frame 80 or otherwise attached to the user 20), such as image capture devices (e.g., cameras), microphones, inertial measurement units, accelerometers, compasses, GPS units, wireless devices, gyroscopes, and / or other sensors disclosed herein, and / or 2) data acquired and / or processed using the remote processing module 152 and / or remote data repository 162 (including data related to virtual content), possibly for passing to the display 70 after processing or retrieval. The local processing and data module 140 may be operatively coupled to the remote processing module 152 and the remote data repository 162 by communications links 170, 180, such as via wired or wireless communication links, such that these remote modules 152, 162 are operatively coupled to each other and available as resources to the local processing and data module 140.In some embodiments, local processing and data module 140 may include one or more of an image capture device, a microphone, an inertial measurement unit, an accelerometer, a compass, a GPS unit, a wireless device, and / or a gyroscope. In some other embodiments, one or more of these sensors may be attached to frame 80 or may be independent devices that communicate with local processing and data module 140 by wired or wireless communication paths.

[0058] The remote processing module 152 may include one or more processors for analyzing and processing data such as image and audio information. In some embodiments, the remote data repository 162 may be a digital data storage facility, which may be available through the Internet or other networking configurations in a "cloud" resource configuration. In some embodiments, the remote data repository 162 may include one or more remote servers that provide information (e.g., information for generating augmented reality content) to the local processing and data module 140 and / or the remote processing module 152. In other embodiments, all data is stored and all calculations are performed in the local processing and data module, allowing for fully autonomous use from the remote modules.

[0059] Other embodiments are in the following claims.

Claims

1. 1. A system comprising: a plenoptic camera assembly including a camera lens defining an image plane, a camera sensor, and a microlens array arranged to image light at the image plane onto the camera sensor; a stage for receiving a head mounted display module and positioning the head mounted display module relative to the plenoptic camera assembly such that an exit pupil of the head mounted display module coincides with a pupil of the plenoptic camera assembly; one or more filter arrays positioned at a pupil of the plenoptic camera assembly, each of the one or more filter arrays comprising a plurality of spatially discrete filters extending across an aperture of the camera lens; a system controller in communication with the plenoptic camera assembly, the system controller being programmed to: acquire one or more brightfield images projected from the head mounted display module during operation of the system; and determine information about a performance of the head mounted display module based on the acquired brightfield images; A system comprising:

2. The system of claim 1 , wherein one of the filter arrays comprises spatially discrete color filters.

3. The system of claim 2 , wherein the spatially discrete color filters comprise X, Y, Z tristimulus color filters.

4. The system of any one of claims 1-3, wherein one of the filter arrays comprises spatially discrete polarizing filters.

5. The system of any one of claims 1-4, wherein one of the filter arrays comprises spatially discrete neutral density filters.

6. 6. The system of claim 1, wherein the one or more filter arrays comprise a first array of spatially discrete color filters and a second array of spatially discrete filters overlapping the first set, the first and second arrays of spatially discrete filters filtering different properties of the light.

7. The system of claim 6 , wherein the light property is selected from the group consisting of color, polarization, and intensity.

8. The system of any one of claims 1-7, further comprising an optical assembly arranged at the pupil of the plenoptic camera assembly and configured to combine light emitted from multiple different locations of the exit pupil of the head-mounted display module to form multiple overlapping images at a sensor of the plenoptic camera assembly, each of the multiple overlapping images corresponding to a different user view for the head-mounted display module.

9. 9. The system of claim 8, wherein the optical assembly comprises one or more polarizing beam splitters arranged at different locations within an exit pupil of the head-mounted display module, and a beam combiner arranged within a pupil of the plenoptic camera assembly arranged to receive light from each of the polarizing beam splitters and direct light from each polarizing beam splitter along a common path to the camera lens.

10. 1. A method for measuring performance of a head mounted display module, the method comprising: aligning the head mounted display module with respect to a plenoptic camera assembly such that an exit pupil of the head mounted display module coincides with a pupil of the plenoptic camera assembly; emitting light from the head mounted display module while the head mounted display module is aligned with the plenoptic camera assembly; filtering the light at an exit pupil of the head mounted display module; obtaining one or more bright field images projected from the head mounted display module along with the filtered light using the plenoptic camera assembly; determining information about the performance of the head mounted display module based on the acquired bright field image; A method comprising:

11. The method of claim 10 , wherein the light is filtered by a plurality of spatially discrete filters positioned at a pupil of the plenoptic camera assembly.

12. The method of claim 11 , wherein the spatially discrete filters comprise color filters.

13. The method of claim 12 , wherein the color filters comprise X, Y, Z color matching function color filters.

14. The method of claim 11 , wherein the spatially discrete filters comprise polarizing filters.

15. The method of claim 11 , wherein the spatially discrete filter comprises a neutral density filter.

16. 16. A method according to any one of claims 10-15, wherein the light is filtered by a first set of spatially discrete filters and a second set of spatially discrete filters that overlap the first set, the first and second sets of spatially discrete filters filtering different properties of the light.

17. The method of claim 16 , wherein the light property is selected from the group consisting of color, polarization, and intensity.

18. 18. The method of any one of claims 10-17, wherein obtaining the one or more bright field images comprises re-imaging a real image from the head mounted display module onto a multi-element sensor using a microlens array.

19. 20. The method of claim 18, wherein the microlens array samples a portion of an exit pupil of a lens of the plenoptic camera assembly to provide different angular views of the real image from the head mounted display module.

20. A method according to any one of claims 10-19, wherein the plenoptic camera assembly defines an optical path from the pupil to a sensor array, the plenoptic camera assembly comprising a camera lens assembly in the optical path that defines a pupil of the plenoptic camera assembly and defines an image plane, the plenoptic camera assembly further comprising an array of focusing elements in the optical path between the image plane and the sensor array.

21. A method according to any one of claims 10-20, wherein the information about the performance of the head mounted display module comprises information about at least one of the performance parameters selected from the group consisting of radiance, luminance, color, geometric distortion, virtual image distance, and field curvature.

22. A method according to any one of claims 10-21, wherein determining the information about the performance of the head mounted display module comprises calculating two-dimensional images at a number of different depths across a three-dimensional volume of interest of the head mounted display module.

23. The method of claim 22 , wherein determining the information further comprises determining information about one or more properties of each of the two-dimensional images.

24. A method according to any one of claims 10-23, further comprising combining light emitted from multiple different locations of an exit pupil of the head-mounted display module to form multiple overlapping images at a sensor of the plenoptic camera assembly, each of the multiple overlapping images corresponding to a different user view for the head-mounted display module.

25. 1. A method for calibrating a head mounted display, comprising: Measuring the performance of the head mounted display using a method according to any one of claims 10-24; adjusting operation of the head mounted display based on the measured performance; and A method comprising:

Images