3D display using electromagnetic field calculations

Abstract

To provide methods, apparatus, devices, and systems for three-dimensional (3D) displaying objects.SOLUTION: In one aspect, a method includes: acquiring data including each piece of primitive data for primitives corresponding to an object; determining an electromagnetic (EM) field contribution to each element of a display for each of the primitives by calculating an EM field propagation from the primitive to the element; generating a sum of the EM field contributions from the primitives for each of the elements; transmitting each control signal for modulating at least one property of the element to each of the elements on the basis of the sum of the EM field contributions; and transmitting a timing control signal to an illuminator to activate the illuminator to illuminate light on the display such that the light is caused by the modulated elements of the display to form a volumetric light field corresponding to the object.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] (Incorporated by reference) This application claims priority under 35 U.S.C. §119 to U.S. Patent Application Publication No. 62 / 618,054, filed January 16, 2018, entitled "Three-Dimensional Displays Using Electromagnetic Field Computations," the entire contents of which are incorporated herein by reference.

[0002] This disclosure relates to three-dimensional (3D) displays, and more particularly to 3D displays using computational techniques. [Background technology]

[0003] Advances in traditional two-dimensional (2D) projection and 3D rendering have led to new methods of 3D display, including a number of hybrid techniques that blend head and eye tracking with traditional display devices for virtual reality (VR), augmented reality (AR), and mixed reality (MR). These techniques attempt to recreate the experience of holographic images, combined with tracking and measurement-based calculations to simulate the stereoscopic images or in-eye light fields that may be represented by actual holograms. Summary of the Invention

[0004] This disclosure describes methods, apparatus, devices, and systems for using electromagnetic (EM) field computations in three-dimensional (3D) displays.

[0005] The present disclosure provides technology that can overcome limitations present in known technologies. As one example, the technology disclosed herein can be implemented without the use of cumbersome wearable devices such as “3D glasses.” As another example, the technology disclosed herein can optionally be implemented without being limited by the accuracy of tracking mechanisms, the quality of display devices, relatively long processing times, and / or relatively high computational demands, and / or the inability to simultaneously display objects to multiple viewers. As yet another example, the technology can be implemented without the use of specialized tools and software to develop content that extends beyond the tools and software used in traditional 3D content creation. Various embodiments can exhibit one or more of the aforementioned advantages. For example, certain implementations of the present disclosure can generate real-time, full-color, true 3D images that appear like real-world 3D objects and can be viewed unobstructed by multiple viewers simultaneously from different positions.

[0006] One aspect of the present disclosure features a method that includes determining, for each of a plurality of primitives corresponding to objects in three-dimensional (3D) space, an electromagnetic (EM) field contribution to each of a plurality of elements of a display by calculating, in a 3D coordinate system, a propagation of the EM field from the primitive to the element, and generating, for each of the plurality of elements, a sum of the EM field contributions from the plurality of primitives to the element.

[0007] The EM field contribution may include at least one of a phase contribution or an amplitude contribution. The primitive may include at least one of a point primitive, a line primitive, or a polygon primitive. The primitive may include a line primitive including at least one of a gradient color, a textured color, or an arbitrary surface shading effect. The primitive may also include a polygon primitive including at least one of a gradient color, a textured color, or an arbitrary surface shading effect. The multiple primitives may be indexed in a particular order.

[0008] In some implementations, the method further includes acquiring respective primitive data for each of the plurality of primitives. The respective primitive data for each of the plurality of primitives may include respective color information for the primitive, and the determined EM field contribution for each of the elements includes information corresponding to the respective color information for the primitive. The color information may include at least one of a texture color or a gradient color. The respective primitive data for each of the plurality of primitives may include texture information for the primitive. The respective primitive data for each of the plurality of primitives may include shading information on one or more surfaces of the primitive. The shading information may include modulation for at least one of color or brightness on one or more surfaces of the primitive.

[0009] In some implementations, the primitive data for each of the plurality of primitives includes coordinate information for the primitive in a 3D coordinate system. The coordinate information for each of the plurality of elements in the 3D coordinate system can be determined based on the coordinate information for the plurality of primitives in the 3D coordinate system. The coordinate information for each of the elements can correspond to a logical memory address of the element stored in memory.

[0010] For each of the plurality of primitives, determining the EM field contribution to each of the plurality of elements may include determining, within a 3D coordinate system, at least one distance between the element and the primitive based on the coordinate information of each of the elements and the coordinate information of each of the primitives. In some examples, determining the EM field contribution to each of the plurality of elements for each of the plurality of primitives includes determining a first distance between a first primitive of the plurality of primitives and a first element of the plurality of elements based on the coordinate information of the first primitive and the coordinate information of each of the first elements, and determining a second distance between the first primitive and a second element of the plurality of elements based on the first distance and the distance between the first element and the second element. The distance between the first element and the second element may be predetermined based on the pitch of the plurality of elements of the display.

[0011] In some examples, at least one of the plurality of primitives is a line primitive including first and second endpoints, and determining at least one distance between the element and the primitive includes determining a first distance between the element and a first endpoint of the line primitive and determining a second distance between the element and a second point of the line primitive. In some examples, at least one of the plurality of primitives is a triangle primitive including first, second, and third endpoints, and determining at least one distance between the element and the primitive includes determining a first distance between the element and a first endpoint of the triangle primitive, determining a second distance between the element and a second point of the triangle primitive, and determining a third distance between the element and a third point of the triangle primitive.

[0012] In some implementations, determining, for each of the plurality of primitives, the EM field contribution to each of the plurality of elements includes determining the EM field contribution from the primitive to the element based on a predetermined equation for the primitive and at least one distance. In some cases, the predetermined equation is determined by analytically calculating the propagation of the EM field from the primitive to the element. In some cases, the predetermined equation is determined by solving Maxwell's equations. The Maxwell's equations may be solved by providing boundary conditions defined at a surface of the display. The boundary conditions may include Dirichlet boundary conditions or Cauchy boundary conditions. The plurality of primitives and the plurality of elements may exist in a 3D space, and the surface of the display may form part of a boundary surface of the 3D space. In some cases, the predetermined equation includes at least one of a function including a sine function, a cosine function, or an exponential function, and determining the EM field contribution includes identifying a value of at least one of the functions in a table stored in memory.

[0013] In some implementations, for each of the plurality of primitives, determining an EM field contribution to each of the plurality of elements and generating a sum of the field contributions for each of the plurality of elements includes determining a first EM field contribution from the plurality of primitives to a first element of the plurality of elements and summing the first EM field contributions to the first element, and determining a second EM field contribution from the plurality of primitives to a second element of the plurality of elements and summing the second EM field contributions to the second element. Determining the first EM field contribution from the plurality of primitives to the first element may include determining the EM field contribution from a first primitive of the plurality of primitives to the first element in parallel with determining the EM field contribution from a second primitive of the plurality of primitives to the first element.

[0014] In some implementations, for each of the plurality of primitives, determining the EM field contribution to each of the plurality of elements includes determining a first respective EM field contribution from a first primitive of the plurality of primitives to each of the plurality of elements and determining a second respective EM field contribution from a second primitive of the plurality of primitives to each of the plurality of elements, and generating a sum of the field contributions for each of the plurality of elements includes accumulating the EM field contributions to the element by adding the second respective EM field contribution to the first respective EM field contribution to the element. Determining the first respective EM field contribution to each of the plurality of elements from the first primitive may be performed in parallel with determining the second respective EM field contribution to each of the plurality of elements from the second primitive.

[0015] For each of the plurality of primitives, determining the EM field contribution to each of the plurality of elements may include determining a first EM field contribution from a first primitive of the plurality of primitives to a first element of the plurality of elements in parallel with determining a second EM field contribution from a second primitive of the plurality of primitives to the first element.

[0016] In some implementations, the method further includes generating, for each of the plurality of elements, a respective control signal based on a sum of EM field contributions from the plurality of primitives to the element, wherein each control signal is present to modulate at least one characteristic of the element based on the sum of EM field contributions from the plurality of primitives to the element. The at least one characteristic of the element may include at least one of a refractive index, an amplitude index, a birefringence, or a phase difference. Each control signal may include an electrical signal, an optical signal, a magnetic signal, or an acoustic signal. In some cases, the method further includes multiplying the sum of the field contributions for each of the elements by a scale factor to obtain a scaled sum of the field contributions, wherein each control signal is generated based on the scaled sum of the field contributions for the element. In some cases, the method further includes normalizing the sum of the field contributions for each of the elements, wherein each control signal is based on the normalized sum of the field contributions for the element. The method may also include transmitting each control signal to the element.

[0017] In some implementations, the method further includes transmitting a control signal to the light emitter, the control signal indicating to turn on the light emitter so that the light emitter emits light on the display. The control signal may be transmitted in response to determining completion of obtaining the sum of the field contributions for each of the plurality of elements. The modulation elements of the display may propagate the light in different directions to form a volumetric light field corresponding to an object in 3D space. The volumetric light field may correspond to a solution of Maxwell's equations with boundary conditions defined by the modulation elements of the display. The light may include white light, and the display may be configured to diffract the white light into light having different colors.

[0018] In some implementations, the method further includes representing the values ​​using a fixed-point representation during the calculation, each of which may be represented as an integer with an implicit scale factor.

[0019] In some implementations, the method further includes performing a mathematical function using a fixed-point representation. The mathematical function can include at least one of sine, cosine, and arctangent. Performing the mathematical function can include receiving an expression in a first fixed-point format and outputting a value in a second fixed-point format having a different level of precision than the precision of the first fixed-point format. Performing the mathematical function can include looking up a table for calculation of the mathematical function, the table including at least one of a fully enumerated lookup table, an interpolated table, a semi-table based on a polynomial function, and a semi-table based on a full minimax polynomial. Performing the mathematical function can include applying a special range reduction to the input. Performing the mathematical function can include converting trigonometric calculations in the range [-π,π] to signed two's complement representation in the range [-1,1].

[0020] Another aspect of the present disclosure features a method that includes acquiring primitive data for a plurality of primitives corresponding to an object in three-dimensional (3D) space, calculating a first respective electromagnetic (EM) field contribution from a first primitive of the plurality of primitives to each of a plurality of elements of a display, and calculating a second respective EM field contribution from a second primitive of the plurality of primitives to each of the plurality of elements of the display, wherein calculating the first respective EM field contribution from the first primitive is at least partially parallel to calculating the second respective EM field contribution from the second primitive.

[0021] In some implementations, calculating a first EM field contribution from a first primitive to a first element of the plurality of elements is in parallel with calculating a second EM field contribution from a second primitive to the first element of the plurality of primitives. The method can include calculating each EM field contribution from each of the plurality of primitives to each of the plurality of elements. The calculation of each EM field contribution can be without at least one of expanding the shape of the object to the plurality of elements, applying a visibility test before assembling the wavefront, and decision-making or communication between parallel calculations for different primitives. The calculation of each EM field contribution can be configured to cause at least one of: tuning the parallel calculations of different primitives for speed, cost, size, or energy optimization; reducing latency before starting drawing and ready to display results; improving precision using fixed-point representation; and optimizing calculation speed by optimizing mathematical functions.

[0022] In some implementations, the method further includes representing the value using a fixed-point representation during the calculation. Representing the value using a fixed-point representation can avoid using denormalized floating-point numbers for gradual underflow, handling NaN results from operations involving division by zero, changing floating-point rounding modes, and raising floating-point exceptions to an operating system.

[0023] In some implementations, the method includes, for each of the plurality of elements, determining a second respective EM The E It further includes accumulating the M field contributions.

[0024] In some implementations, the method further includes generating, for each of the plurality of elements, a respective control signal based on a sum of EM field contributions from the plurality of primitives to the element, each control signal being present to modulate at least one characteristic of the element based on the sum of EM field contributions from the plurality of primitives to the element.

[0025] In some implementations, the method further includes scaling a first primitive adjacent to a second primitive by a predetermined factor so that reconstruction of the first primitive does not overlap reconstruction of the second primitive. The predetermined factor may be determined at least in part based on a display resolution. The method may further include acquiring respective primitive data for each of a plurality of primitives, the respective primitive data for each of the plurality of primitives including respective coordinate information for the primitive in a 3D coordinate system, and determining new respective coordinate information for the first primitive based on the respective coordinate information for the first primitive and the predetermined factor. The method may further include determining an EM field contribution from the first primitive to each of the plurality of elements based on the respective new coordinate information for the first primitive. The method may further include scaling the second primitive by the predetermined factor. The first primitive and the second primitive can share a common portion, and resizing the first primitive can include resizing the common portion of the first primitive. Resizing the first primitive can include resizing the first primitive in a predetermined direction.

[0026] Another aspect of the present disclosure features a method that includes obtaining primitive data for each of a plurality of primitives corresponding to an object in three-dimensional (3D) space; using the primitive data for a first primitive and a second primitive to rescale the first primitive adjacent to the second primitive by a predetermined factor; and updating the primitive data for the first primitive based on the results of the rescaling.

[0027] In some implementations, the respective primitive data for each of the plurality of primitives includes respective coordinate information for the primitive in a 3D coordinate system, and updating the respective primitive data includes determining new respective coordinate information for the first primitive based on the respective coordinate information for the first primitive and a predetermined coefficient.

[0028] In some implementations, the pre-determined coefficients are determined such that the reconstruction of the first primitive does not overlap the reconstruction of the second primitive in 3D space.

[0029] In some implementations, the resizing is performed so that the gap between the reconstruction of the first primitive and the reconstruction of the second primitive in 3D space is large enough to separate the first and second primitives and minimize the effects of overlap, and small enough to make the reconstructions appear seamless.

[0030] In some implementations, the pre-determined coefficients are determined based at least in part on the resolution of the display.

[0031] In some implementations, the method further includes storing the updated primitive data of the first primitive in a buffer.

[0032] In some implementations, resizing is performed during the object rendering process to obtain primitive data for each of the multiple primitives.

[0033] In some implementations, the method further includes transmitting updated primitive data for the plurality of primitives to a controller, the controller being configured to determine a respective electromagnetic (EM) field contribution from each of the plurality of primitives to each of the plurality of elements of the display based on the updated primitive data for the plurality of primitives.

[0034] In some implementations, the method further includes determining an EM field contribution from the first primitive to each of a plurality of elements of the display based on the updated primitive data for the first primitive.

[0035] In some implementations, the method further includes scaling the second primitive by a predetermined factor.

[0036] In some implementations, the first primitive and the second primitive share a common portion, and resizing the first primitive includes resizing the common portion of the first primitive.

[0037] In some implementations, resizing the first primitive includes resizing the first primitive in a predetermined direction.

[0038] In some implementations, resizing the first primitive includes resizing a first portion of the first primitive by a first predetermined coefficient and resizing a second portion of the second primitive by a second predetermined coefficient, where the first predetermined coefficient differs from the second predetermined coefficient.

[0039] Another aspect of the present disclosure features a method that includes obtaining a plurality of discrete cosine transform (DCT) weightings of an image that is mapped to a specified surface of a particular primitive of a plurality of primitives that correspond to an object in three-dimensional (3D) space, and determining each EM field contribution from the particular primitive to each of a plurality of elements of a display by considering the effect of the plurality of DCT weightings of the image.

[0040] In some implementations, the method further includes determining a resolution of an image to be mapped to a specified surface of a particular primitive, and determining a plurality of DCT weights of the image based on the resolution.

[0041] In some implementations, the method further includes decoding the DCT weights of the image to obtain respective DCT amplitudes for each pixel of the image.

[0042] In some implementations, the method further includes storing a value associated with each DCT amplitude of the pixels of the image along with the primitive data for the particular primitive. Determining each EM field contribution can include calculating each EM field contribution from the particular primitive to each of the plurality of elements using the value associated with each DCT amplitude of the pixels of the image.

[0043] In some implementations, the method further includes selecting particular DCT terms to be included in determining the contribution of each EM field, each of the particular DCT terms including a respective DCT weighting greater than a predetermined threshold.

[0044] Another aspect of the present disclosure features a method that includes obtaining occluder information for a particular primitive and the particular primitive, the particular primitive being within a plurality of primitives corresponding to an object in three-dimensional (3D) space, and determining one or more particular elements of a plurality of elements of a display that do not contribute to the reconstruction of the particular primitive due to the influence of the occluder.

[0045] In some implementations, the method further includes storing information for the particular element along with information for the particular primitive and occluder.

[0046] In some implementations, this determining is performed during the object rendering process to obtain primitive data for multiple primitives.

[0047] In some implementations, the method further includes transmitting the stored information of the particular element, along with information of the particular primitive and occluder, to a controller configured to calculate electromagnetic (EM) contributions of the multiple primitives to the multiple elements of the display.

[0048] In some implementations, the method further includes generating, for each particular element, a sum of electromagnetic (EM) field contributions from the plurality of primitives to one of the particular elements by excluding the EM field contribution from the particular primitive to the one of the particular elements.

[0049] In some implementations, the method further includes generating, for each of the plurality of elements other than the particular element, a respective sum of EM field contributions to the element from the plurality of primitives.

[0050] In some implementations, the method further includes masking the EM field contribution of a particular element to a particular primitive.

[0051] In some implementations, determining one or more particular elements includes connecting a particular primitive to an end point of an occluder, extending this connection to a display and determining intersections between this connection and the display, and determining that a particular range defined by these intersections is a particular element that does not contribute to the reconstruction of the particular primitive due to the influence of the occluder.

[0052] Another aspect of the invention features a method that includes obtaining occluder information for a particular primitive and the particular primitive, the particular primitive being within a plurality of primitives corresponding to objects in three-dimensional (3D) space, and determining, for each of a plurality of elements of a display, portions of the particular primitive that do not contribute an electromagnetic (EM) field to the element due to the occluder.

[0053] In some implementations, the method further includes storing information for each portion of the particular primitive along with information for the particular primitive and occluder.

[0054] In some implementations, this determining is performed during the object rendering process to obtain primitive data for multiple primitives.

[0055] In some implementations, the method further includes transmitting the stored information for each portion of the particular information, along with the particular primitive and occluder information, to a controller configured to calculate electromagnetic (EM) contributions of the multiple primitives to the multiple elements of the display.

[0056] In some implementations, the method further includes masking the EM field contribution of each of the plurality of elements to each portion of the particular primitive.

[0057] In some implementations, the method further includes generating, for each of the plurality of elements, a sum of the EM field contributions from the plurality of primitives to the element by excluding the EM field contribution from each portion of the particular primitive to the element. Generating the sum of the EM field contributions from the plurality of primitives to the element can include subtracting the EM contribution of each portion of the particular primitive to the element from the sum of the EM field contributions from the plurality of primitives to the element without the effect of occluders. Generating the sum of the EM field contributions from the plurality of primitives to the element can include summing the EM field contributions to the element from one or more other portions of the particular primitive, each portion and the one or more other portions forming the particular primitive.

[0058] In some implementations, determining each portion of the particular primitive that does not contribute an EM field to the element due to the influence of the occluder includes connecting the element to an end point of the occluder, determining an intersection between this connection and the particular primitive, and determining that the particular portion of the particular primitive enclosed by the intersection is the each portion of the particular primitive that does not contribute an EM field to the element due to the influence of the occluder.

[0059] Another aspect of the present disclosure features a method that includes obtaining respective primitive data for each of a plurality of primitives corresponding to an object in three-dimensional (3D) space, obtaining respective geometric specular information for each of the plurality of primitives, and storing the respective geometric specular information together with the respective primitive data for each of the plurality of primitives.

[0060] In some implementations, each geometric specular information for each of the plurality of primitives includes a reflectance of the surface of the primitive at a viewing angle.

[0061] In some implementations, the method further includes determining a respective EM field contribution from each of the plurality of primitives to each of the plurality of elements of the display by considering respective geometric specular reflection information of the primitives.

[0062] Another aspect of the disclosure features a method that includes obtaining graphics data including primitive data for each of a plurality of primitives corresponding to an object in three-dimensional (3D) space; determining, for each of the plurality of primitives, an electromagnetic (EM) field contribution to each of a plurality of elements of a display by calculating, in the 3D coordinate system, a propagation of the EM field from the primitive to the element; generating, for each of the plurality of elements, a sum of the EM field contributions from the plurality of primitives to the element; sending, for each of the plurality of elements, a respective control signal to the element, the control signal being present to modulate at least one characteristic of the element based on the sum of the EM field contributions to the element; and sending timing control signals to light emitters to activate the light emitters and illuminate the display such that light is produced by the modulation elements of the display to form a volumetric light field corresponding to the object.

[0063] Another aspect of the disclosure features a method that includes modifying each control signal using a predetermined calibration value for each of a plurality of elements of a display, applying each modified control signal to the plurality of elements of the display, measuring incident light output on the display, and evaluating the predetermined calibration value based on the light output measurements.

[0064] In some implementations, the predetermined calibration value is the same for each of the multiple elements.

[0065] In some implementations, the method further includes converting each control signal of the plurality of elements by a digital-to-analog converter (DAC), and modifying each control signal of the plurality of elements includes modifying a digital signal of each control signal using a predetermined calibration value.

[0066] In some implementations, the predetermined value comprises multiple bits.

[0067] In some implementations, the method further includes adjusting the predetermined calibration value based on a result of the evaluation. Adjusting the predetermined calibration value can include changing the values ​​of one or more of the plurality of bits. Adjusting the predetermined calibration value can include determining a combination of values ​​of the plurality of bits based on the predetermined calibration value and another calibration value determined from a previous evaluation.

[0068] In some implementations, the light output comprises a phase change or intensity difference of light between the light output and a background.

[0069] In some implementations, each control signal for an element is determined based on the sum of electromagnetic (EM) field contributions to the element from multiple primitives corresponding to objects in 3D space.

[0070] Another aspect of the present disclosure features a method that includes obtaining, for each of a plurality of elements of a display, a respective sum of electromagnetic (EM) field contributions from a plurality of primitives in three-dimensional (3D) space, where the plurality of primitives correspond to objects in the 3D space; applying a respective mathematical transform to the respective sum of the element's EM field contributions to obtain a respective transformed sum of the element's EM field contributions; determining a respective control signal based on the respective transformed sum of the element's EM field contributions; and modulating a characteristic of the element based on the determined respective control signal of the element.

[0071] In some implementations, the method further includes introducing incident light onto a plurality of elements of a display, measuring a first output of the light, and adjusting one or more coefficients of each mathematical transform of the plurality of elements based on a result of measuring the first output of the light. The method may further include altering a depth of a holographic pattern corresponding to an object within a field of view of the display, measuring a second output of the light, and adjusting one or more coefficients of each mathematical transform based on the first and second outputs. The method may further include changing a plurality of primitives corresponding to the first holographic pattern to a second plurality of primitives corresponding to a second holographic pattern, measuring the second output of the light, and adjusting one or more coefficients of each mathematical transform based on the first and second outputs. The first holographic pattern and the second holographic pattern may correspond to an object. The second holographic pattern may correspond to a second object different from the object associated with the first holographic pattern. The first output of the light may be measured by an image sensor. The image sensor may be configured to use a machine vision algorithm to determine what is being displayed and calculate a goodness-of-fit parameter. Each of the first and second holographic patterns can include a grid of dots, and the goodness-of-fit parameter is at least one of how close the dots are to each other, how close to the center the dots are located, and how deformed the dots are.

[0072] In some implementations, the mathematical transform is derived from Zernike polynomials.

[0073] In some implementations, the mathematical transformations of multiple elements vary from element to element.

[0074] In some implementations, the method further includes reproducing a series of samples of known colors and intensities by illuminating the display, measuring the output light using a colorimeter device calibrated to the CIE standard observer curve, and defining the output light of the display in the CIE XYZ color space. The method can further include determining deviations of the defined output light values ​​from the known standard values, and adapting the output colors on the display to adjust the output colors to the correct colors.

[0075] Another aspect of the present disclosure features a method that includes determining a cell gap of a liquid crystal (LC) display based on a pitch of display elements of the LC display, and calculating a minimum value of birefringence of the LC mixture based on the cell gap of the LC display and a predetermined retardation.

[0076] In some implementations, the method further includes improving the switching speed of the LC display while maintaining the birefringence of the LC mixture above a minimum value. Improving the switching speed can include at least one of increasing the dielectric anisotropy of the LC mixture and reducing the rotational viscosity of the LC mixture.

[0077] In some implementations, the LC display includes a liquid crystal on silicon (LCOS) device with a silicon backplane.

[0078] In some implementations, an LC display includes a liquid crystal layer, a transparent conductive layer above the liquid crystal layer as a common electrode, and a backplane having a plurality of metal electrodes below the liquid crystal layer, each of the plurality of metal electrodes being separated from each other, and the backplane configured to control a voltage on each of the plurality of metal electrodes.

[0079] Another aspect of the disclosure features a display that includes a backplane and a plurality of display elements on the backplane, at least two of the plurality of display elements having different sizes.

[0080] In some implementations, a larger of the at least two display elements includes a buffer and a smaller of the at least two display elements does not include a buffer, the larger display element may be connected to a first plurality of display elements by a conductive line, the buffer is configured to buffer a voltage applied to the conductive line such that the voltage is applied only to a second plurality of display elements in the first plurality, and the number of display elements in the second plurality is less than the number of display elements in the first plurality.

[0081] In some implementations, the buffer comprises analog circuitry in the form of transistors or digital circuitry in the form of logic gates.

[0082] In some implementations, the distribution of sizes of the plurality of display elements is substantially the same as the size of the smaller of the at least two display elements.

[0083] In some implementations, the display is configured to be a liquid crystal on silicon (LCOS) device.

[0084] Another aspect of the disclosure features a display that includes a backplane and a plurality of display elements on the backplane, at least two of the plurality of display elements having different shapes.

[0085] In some implementations, the backplane includes respective circuitry for each of the display elements, and each circuitry for at least two display elements has a shape corresponding to the different shapes of the at least two display elements.

[0086] In some implementations, the distribution of sizes of the plurality of display elements is substantially the same as a predetermined size.

[0087] In some implementations, the display is configured to be a liquid crystal on silicon (LCOS) device.

[0088] Another aspect of the disclosure features a method that includes obtaining graphics data including primitive data for each of a plurality of primitives corresponding to an object in three-dimensional (3D) space; determining, for each of the plurality of primitives, an electromagnetic (EM) field contribution to each of a plurality of elements of a display by calculating, in the 3D coordinate system, a propagation of the EM field from the primitive to the element; generating, for each of the plurality of elements, a sum of the EM field contributions from the plurality of primitives to the element; sending, for each of the plurality of elements, a respective control signal to the element, the control signal being present to modulate at least one characteristic of the element based on the sum of the EM field contributions to the element; and sending timing control signals to light emitters to activate the light emitters and illuminate the display such that light is produced by the modulation elements of the display to form a volumetric light field corresponding to the object.

[0089] Other embodiments of each aspect include corresponding computer systems, apparatus, and computer programs stored on one or more computer storage devices, each configured to perform the operations of the respective methods. When one or more computer systems are configured to perform particular operations or actions, it means that software, firmware, hardware, or a combination thereof is installed on the system that, when in operation, causes the system to perform the operations or actions. When one or more computer programs are configured to perform particular operations or actions, it means that the one or more programs contain instructions that, when executed by a data processing device, cause the device to perform the operations or actions.

[0090] Another aspect of the present disclosure features a device that includes one or more processors and a non-transitory computer-readable storage medium in communication with the one or more processors and storing instructions executable by the one or more processors that, upon execution of such instructions, cause the one or more processors to perform one or more of the methods disclosed herein.

[0091] Another aspect of the present disclosure features a non-transitory computer-readable storage medium storing instructions executable by one or more processors that, upon execution of such instructions, cause the one or more processors to perform a method in accordance with one or more of the methods disclosed herein.

[0092] Another aspect of the present disclosure features a display including a plurality of elements and a controller coupled to the display and configured to perform one or more of the methods disclosed herein. The controller can include a plurality of computing units, each configured to perform an operation on one or more primitives of a plurality of primitives corresponding to an object in three-dimensional (3D) space. In some implementations, the controller is locally coupled to the display, and each of the computing units is coupled to one or more respective elements of the display and configured to send a respective control signal to each of the one or more respective elements. The computing units can be configured to operate in parallel.

[0093] The controller may include at least one of an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a programmable gate array (PGA), a central processing unit (CPU), a graphics processing unit (GPU), or standard computing cells. The display may include a spatial light modulator (SLM), including a digital micro-mirror device (DMD) or a liquid crystal on silicon (LCOS) device. The display may be configured to perform phase modulation, amplitude modulation, or phase and amplitude modulation. The controller may be coupled to the display via a memory buffer.

[0094] In some implementations, the system includes a light emitter disposed adjacent to the display and configured to emit light on the display, The light emitter can be coupled to a controller and configured to be turned on and off based on a control signal from the controller.

[0095] In some cases, the light emitter is coupled to a controller configured to control the amplitude or brightness of one or more light emitting elements in the light emitter via a memory buffer. The memory buffer of the light emitter can have a size smaller than the memory buffer of the display. The number of light emitting elements in the light emitter can be less than the number of elements of the display. The controller can be configured to simultaneously activate one or more light emitting elements of the light emitter.

[0096] The light emitter can be a coherent light source, a semi-coherent light source, or a non-coherent light source. In some implementations, the light emitter is configured to emit white light, and the display is configured to diffract the white light into light having different colors. In some implementations, the light emitter includes two or more light-emitting elements, each configured to emit light having a different color. The controller can be configured to sequentially modulate the display using information associated with a first color during a first period and modulate the display using information associated with a second color during a second consecutive period, and the controller can be configured to control the light emitter to sequentially turn on the first light-emitting element during the first period to emit light having the first color and to sequentially turn on the second light-emitting element during the second period to emit light having the second color.

[0097] In some implementations, a light emitter is positioned in front of a display surface and configured to emit light onto the display surface at an angle of incidence in the range of 0 degrees to 90 degrees, the emitted light being reflected from the display surface. In some cases, the light emitted from the light emitter comprises collimated light. In some cases, the light emitted from the light emitter comprises diverging light. In some cases, the light emitted from the light emitter comprises semi-collimated light.

[0098] In some implementations, the light emitter is positioned behind the rear surface of the display and configured to emit divergent light onto the rear surface of the display, with the emitted light being transmitted through the display and out of the front surface of the display.

[0099] In some implementations, the light emitter includes a light source configured to emit light and a waveguide coupled to the light source and positioned adjacent to the display, the waveguide configured to receive the light emitted from the light source and direct the emitted light to the display. In some cases, the light from the light source is coupled to the waveguide from a vertical cross section of the waveguide via an optical coupler. In some cases, the light source and the waveguide are integrated in a planar configuration and positioned on the surface of the display. The waveguide can be configured to direct the light to uniformly illuminate the display.

[0100] In some cases, a waveguide is placed on the backside of the display, and light is guided to transmit through the display and diffracted out of the front side of the display. A controller may be placed on the backside of the waveguide. In some cases, a waveguide is placed on the front side of the display, and light is guided to be incident on the front side of the display and reflected by the front side.

[0101] Another aspect of the disclosure features a system including a display including an array of elements and an integrated circuit including an array of computing units, each of the computing units coupled to one or more respective elements of the display and configured to: calculate an electromagnetic (EM) field contribution from at least one primitive of the plurality of primitives to each of the array of elements; and generate, for each of the one or more respective elements, a respective sum of the EM field contributions from the plurality of primitives to the element.

[0102] Each of the computing units may be configured to receive, from other computing units of the array of computing units, calculated EM field contributions from other primitives of the plurality of primitives to each of the one or more respective elements, and to generate, for each of the one or more respective elements, a respective total of the EM field contributions by adding the received calculated EM field contributions to the element from the other primitives.

[0103] Each of the computing units may be configured to generate, for each of the one or more respective elements, a respective control signal for modulating at least one characteristic of the element based on a respective sum of the EM field contributions to the element.

[0104] In some implementations, the integrated circuit includes respective accumulators configured to store accumulated results of calculated EM field contributions from a plurality of primitives to each of the elements of the display. The integrated circuit may be configured to initialize the accumulators at the start of a calculation operation. In some examples, the integrated circuit includes respective memory buffers for each of the elements, and the integrated circuit may be configured to accumulate the calculated EM field contributions from the plurality of primitives to the elements to obtain respective sums of the EM field contributions as final accumulated results in the respective accumulators, and transfer the final accumulated results from the respective accumulators to the respective memory buffers of the elements.

[0105] In some implementations, the system further includes an illuminator disposed between the integrated circuit and the display and configured to receive control signals from the integrated circuit and illuminate the display based on the control signals, and the integrated circuit, the illuminator, and the display may be integrated as a single unit.

[0106] Yet another aspect of the present disclosure features a system disclosed herein that includes a computing device configured to generate data including each primitive data of a plurality of primitives corresponding to an object in three-dimensional (3D) space. The system is configured to receive graphics data from the computing device and process the graphics data to present the object in 3D space. The computing device includes an application programming interface (API) configured to create the primitives including each primitive data by rendering a computer-generated (CG) model of the object.

[0107] In this disclosure, the term "primitive" refers to an indivisible basic element for input or output in a computing system. The element can be a geometric or figure element. The term "hologram" refers to a pattern displayed on a display that contains amplitude or phase information, or a combination thereof, about an object. The term "holographic reconstruction" refers to a volumetric light field (e.g., a holographic light field) from a display when illuminated.

[0108] The details of one or more implementations of the subject matter herein are set forth in the accompanying drawings and related description. Other features, aspects, and advantages of the subject matter will become apparent from the description, drawings, and claims.

[0109] It should be understood that various aspects of the implementations can be combined in different ways. For example, features of certain methods can be combined with features of other methods. [Brief explanation of the drawings]

[0110] [Figure 1A] FIG. 1 is a schematic diagram illustrating an exemplary system including a holographic display. [Figure 1B] FIG. 1 is a schematic diagram illustrating an exemplary holographic display. [Figure 1C] FIG. 1 illustrates an exemplary system for a 3D display. [Figure 2] FIG. 1 illustrates an exemplary configuration for electromagnetic (EM) propagation calculations. [Figure 3-1] Figure 3A illustrates an exemplary EM propagation of a point primitive relative to an element of a display; Figure 3B illustrates an exemplary EM propagation of a line primitive relative to an element of a display; and Figure 3C illustrates an exemplary EM propagation of a triangle primitive relative to an element of a display. [Figure 3-2]Figure 3D illustrates an exemplary implementation of Maxwell holographic occlusion of a point primitive, including a line primitive as an occluder, Figure 3E illustrates an exemplary implementation of Maxwell holographic occlusion of a line primitive, including another line primitive as an occluder, and Figure 3F illustrates an exemplary implementation of Maxwell holographic occlusion of a triangle primitive, including a line primitive as an occluder. [Figure 3-3] FIG. 1 illustrates an exemplary implementation of Maxwell holographic stitching. [Figure 4] 1 is a flowchart of an exemplary process for displaying an object in 3D. [Figure 5A] FIG. 1 illustrates an exemplary system implementation for a 3D display. [Figure 5B] FIG. 1 illustrates an exemplary system implementation for a 3D display. [Figure 5C] FIG. 1 illustrates an exemplary system implementation for a 3D display. [Figure 5D] FIG. 1 illustrates an exemplary system implementation for a 3D display. [Figure 5E] FIG. 1 illustrates an exemplary system implementation for a 3D display. [Figure 5F] FIG. 1 illustrates an exemplary system implementation for a 3D display. [Figure 6A] 1A and 1B illustrate exemplary displays including display elements having non-uniform shapes. [Figure 6B] 1A and 1B illustrate exemplary displays including display elements having different sizes. DETAILED DESCRIPTION OF THE INVENTION

[0111] Implementations of the present disclosure feature techniques for enabling the 3D display of complex computer-generated scenes as true holograms. This technology provides a new deterministic solution to real-time dynamic computational holography based on Maxwell's equations for electromagnetic fields, which can be expressed as Maxwell holography. Calculations (or computations) in Maxwell holography can be expressed as Maxwell holographic calculations (or Maxwell holographic computations). In embodiments, the present disclosure utilizes tools including field theory, topology, analytic continuation, and / or symmetry groups to approach holograms as Dirichlet or Cauchy boundary condition problems for general electric fields, enabling real-time solutions for holograms without the limitations of conventional holographic systems. In embodiments, techniques can be used to create phase-only, amplitude-only, or phase and amplitude holograms using spatial light modulators (SLMs) or any other holographic device.

[0112] Implementations of the present disclosure can provide (1) a mechanism for approximating holograms as electromagnetic boundary conditions using field theory and contact geometry instead of classical optics, (2) the derivation and implementation of the electromagnetic boundary condition method for computational holography into computer code and an application programming interface (API), i.e., the implementation of hologram calculations as 2D analytic functions relative to the plane of the hologram and subsequent discretization into a parallel algorithm, and / or (3) the implementation of a set of fully 3D holographic versions of standard computer graphics primitives (e.g., points, lines, triangles, and textured triangles) that can enable full compatibility with standard existing computer graphics tools and techniques. These techniques can enable devices to display common existing content not specifically created for holography, while simultaneously enabling existing content creators to create holographic works without having to learn specialized techniques or use specialized tools.

[0113] In particular, these techniques can involve the use of a mathematical formulation (or equation) of light as an electromagnetic (EM) phenomenon instead of the mathematical formulation of classical optics commonly used in computational holography (e.g., the Gerchberg-Saxton (GS) model). The mathematical formulation disclosed herein is derived from Maxwell's equations. In embodiments, the techniques disclosed herein involve treating the displayed image as an electromagnetic field and treating the hologram as a boundary value condition (e.g., a Dirichlet problem) that generates the electromagnetic field. Furthermore, the ubiquity of the primitive paradigm in computer graphics can be used to construct desired images; for example, these techniques can be used to display any 3D image as a holographic reconstruction, e.g., a holographic light field, instead of a projected image onto a 2D screen. Compared to depth point cloud techniques that are subject to bandwidth limitations, these techniques avoid such limitations and can use any suitable type of primitive, e.g., point primitives, line primitives, or polygonal primitives such as triangle primitives. Additionally, primitives can be rendered using color, texture, and / or shading information, which facilitates storage and compression schemes for CG holographic content, including live holographic video.

[0114] In embodiments, these techniques use Maxwell's equations to calculate the generated hologram as a boundary condition problem to model the electromagnetic field, which can eliminate the reliance on and inherent limitations of the fast Fourier transform (FFT), can eliminate the reliance on collimated sources and lasers, and / or can eliminate the limitations of previous methods for computational holography and non-deterministic solutions.

[0115] In embodiments, these techniques can be optimized for computational simplicity and speed through a mathematical optimization process that constrains independent inputs to the surface of the hologram according to the parameters of the computer-generated (CG) primitives required to construct the scene. This allows the work to be performed in a highly parallel and highly optimized manner in computing architectures (e.g., application-specific integrated circuits (ASICs) and multi-core architectures). The process of computing a hologram can be viewed as a single instruction performed on input data in the form of a computer-generated imagery (CGI) scene and can theoretically be completed in one clock cycle per CGI primitive.

[0116] In embodiments, these techniques treat a holographic scene as a collection of fully 3D holographic primitive apertures, functionally compatible with standard primitives of conventional 3D graphics, such as those employed in video games, movies, television, computer displays, or any other computing display technology. These techniques can enable efficient implementation of these aperture primitives in hardware and software without the limitations inherent in standard implementations of computational holography. The amplitude and color of the primitives can be calculated automatically. Computational complexity can grow linearly with the number of phase elements, n, compared to n^2 or n*log(n) in standard computational holography. Because the images created are fully 3D, rather than a collection of planar images, these techniques do not require iterative amplitude correction in an unknown number of steps. Furthermore, the generated hologram does not contain "conjugate" images that occupy space on the holographic device.

[0117] Because holographic primitives are a special set of mathematical objects, they can be relatively simple and fast to compute, making them well suited to parallel, distributed computational methods. This computability and parallelism enable the interactive computation of large holograms to design large-area holographic devices of theoretically unlimited size, which can function as holographic computer displays, phone displays, home theaters, and holographic rooms. Furthermore, these holograms can fill large areas with light—for example, rendering large shaded areas in 3D—without the limitations associated with traditional holographic computational methods, which can cause elements to appear as contours rather than solids. Furthermore, this relatively simple and fast computation enables the display of holograms in real time at interactive speeds unconstrained by the computational load of n^2 and by repetitive amplitude modifications of n^2.

[0118] In embodiments, these technologies can achieve natural computational feasibility on modern ASIC and multi-core architectures, enabling full compatibility with modern graphics hardware, modern graphics software, and / or modern graphics tools and toolchains. For example, these technologies can implement clear and simple holographic APIs through which high-performance rendering of any CG model can be achieved using conventional standard 3D content creation tools, such as 3DS Max®, SolidWorks®, Maya®, or Unity3D. These APIs can enable developers or users to interact with holographic devices, such as light modulators or holographic systems. The holographic APIs can create computer graphics primitives as distinct holographic scene primitives, enabling the generation of rich holographic content utilizing general-purpose and specially designed holographic computing hardware. The creation of mathematical computational architectures can enable the rendering of holograms using tools and techniques used to create conventional 3D content and software applications. Optimization of the mathematical computational architecture can enable high-performance implementations of conventional graphics and rendering displayed as holographic reconstructions.

[0119] The algorithms in these technologies are relatively simple to implement in hardware. This not only enables the computational speeds necessary for the high-quality modern rendering users expect, but also allows the algorithms to be implemented in relatively simple circuits, e.g., the gate structures of an ASIC, as part of the holographic device. Thus, instead of having to compute a scene remotely and then write it to every pixel of the display for every frame of content, the scene computation can be distributed across a computing architecture (e.g., embedded computation) built into the display device, making bandwidth issues that can be a bottleneck for high-density displays less significant. This also means that the number of display elements, and therefore the size of the holographic display, can be relatively unconstrained by the constraints that limit other technologies.

[0120] These techniques can enable multiple interactive techniques using relatively simple and relatively inexpensively implemented structured light in a variety of applications, such as, for example, solid-state light detection and ranging (LiDAR) devices, 3D printing, smart lighting, smart microdisplays, or any other application requiring structured light. These techniques can also be used for optical simulations, such as lattice simulations.

[0121] FIG. 1A shows a schematic diagram of an exemplary system 100 for 3D display. The system 100 includes a computing device 102 and a holographic display device (or Maxwell holographic display device) 110. The computing device 102 is configured to prepare data for a list of primitives corresponding to an object, e.g., a 3D object, and transmit the data to the holographic display device 110 via a wired or wireless connection, e.g., a USB-C connection or any other high-speed serial connection. The holographic display device 110 is configured to calculate electromagnetic (EM) field contributions to display elements (e.g., modulators) of a display within the holographic display device 110 from the list of primitives, modulate the display elements with a pattern, e.g., a hologram, based on the calculated EM field contributions on the display, and display a light field, e.g., a holographic reconstruction, corresponding to the object in 3D when emitted. As used herein, a hologram refers to a pattern displayed on a display that includes amplitude or phase information, or a combination thereof, about the object. Holographic reconstruction refers to the volumetric light field (e.g., a holographic light field) from the display when illuminated.

[0122] Computing device 102 may be any suitable type of device, such as a desktop computer, a personal computer, a laptop computer, a tablet computing device, a personal digital assistant (PDA), a network appliance, a smartphone, a smartwatch, an enhanced general packet radio service (EGPRS) mobile phone, a media player, a navigation device, an email device, a gaming console, or any suitable combination of any two or more of these or other computing devices.

[0123] The computing device 102 includes an operating system (OS) 104 that can include multiple applications 106 as a graphics engine. The applications 106 can process or render a scene, e.g., any CG model, using standard 3D content creation tools, e.g., 3DS Max®, SolidWorks®, Maya®, or Unity3D. The scene can correspond to 3D objects. The applications 106 can run in parallel to render the scene and obtain an OS graphics abstraction 101, which can be provided to a graphics processing unit (GPU) 108 for further processing. In some implementations, the OS graphics abstraction 101 is provided to a holographic display device 110 for further processing.

[0124] The GPU 108 may include specialized electronic circuitry designed for high-speed operations in computer graphics and image processing. The GPU 108 may process the graphics abstraction 101 of a scene to obtain processed scene data 103, and may use the scene data 103 to obtain a list of primitives 105 indexed in a particular order. The primitives may include at least one of point primitives, line primitives, or polygon primitives. In some implementations, the GPU 108 includes a video driver configured to generate the processed scene data 103 and the list of primitives 105.

[0125] In some implementations, GPU 108 includes a conventional renderer 120, which can render list of primitives 105 using conventional rendering techniques, such as culling and clipping, into a list of items for drawing on a conventional monitor 124, e.g., a 2D display screen. The list of items can be sent to conventional monitor 124 via image buffer 122.

[0126] In some implementations, GPU 108 includes a holographic renderer 130 to render the list of primitives 105 into graphics data that is displayed by holographic display device 110. The graphics data may include a list of primitives and corresponding primitive data. For example, the graphics data may include a hexadecimal code for each primitive.

[0127] In some implementations, the GPU 108 includes both a conventional renderer 120 and a holographic renderer 130. In some implementations, the GPU 108 includes a conventional renderer 120 and the holographic display device 110 includes a holographic renderer 130.

[0128] The primitive data corresponding to a primitive may also include color information, e.g., texture color, gradient color, or both, texture information, and / or shading information. The shading information may be obtained by any conventional CGI surface shading method that involves modulating the color or brightness of the surface of the primitive.

[0129] The primitive data of a primitive may include coordinate information for the primitive in a 3D coordinate system, such as a Cartesian coordinate system XYZ, a polar coordinate system, a cylindrical coordinate system, and a spherical coordinate system. As described in more detail below, a display element in holographic display device 110 may also include corresponding coordinate information in a 3D coordinate system. The primitive at a coordinate location may represent a 3D object adjacent to, e.g., in front of, the display element.

[0130] As an example, a primitive is a shaded line, e.g., a straight line that smoothly changes from one color to another over its length. Rendering this primitive requires four pieces of data: two endpoints and color information (e.g., RGB color values) at each endpoint. Assume the hex code for this line is A0, the line extends from the first endpoint (0.1,0.1,0.1) to the second endpoint (0.2,0.2,0.2) in the 3D coordinate system, and has half blue RGB=(0,0,128) at the first endpoint and full red RGB=(255,0,0) at the second endpoint. The holographic renderer determines the amount and type of data it expects for each primitive. For a line, the shaded line primitive data in the primitive stream can be a series of instructions such as: 0xA0 / / hex code for shaded lines 0x3dcccccd / / (0.1,0.1,0.1) floating point (single precision) First Vertex 0x3dcccccd 0x3dcccccd 0x000080 / / The color of the first vertex is (0,0,128) 0x3e4ccccd / / (0.2,0.2,0.2) floating point (single precision) Second Vertex 0x3e4ccccd 0x3e4ccccd 0xff0000 / / The color of the second vertex is (255,0,0)

[0131] For the shaded line primitive, there are a total of 31 hexadecimal words in the primitive data. Therefore, this can be a very efficient way to transmit complex scenes, and the primitive data can be further compressed. Because each primitive is a deterministic Turing step, no terminator is required. Unlike the traditional model in which this line primitive is simply drawn on a 2D display screen, this line primitive data is transmitted to the holographic display device 110, which can compute a hologram and display a corresponding holographic reconstruction that presents the line floating in space.

[0132] In some implementations, computing device 102 transmits non-primitive-based data, such as a video of a recorded light field, to holographic display device 110. Holographic display device 110 can compute successive holograms and display the video as successive holographic reconstructions in space. In some implementations, computing device 102 transmits computer-generated holographic content to holographic display device 110 simultaneously with live holographic content. Holographic display device 110 can also compute corresponding holograms and display the content as corresponding holographic reconstructions.

[0133] As shown in FIG. 1A, the holographic display device 110 includes a controller 112 and a display 114. The controller 112 can include multiple computing or processing units. In some implementations, the controller 112 includes an ASIC, a field programmable gate array (FPGA), or a GPU, or any combination thereof. In some implementations, the controller 112 includes a holographic renderer 130 to render the list of primitives 105 into graphics data that is calculated by the computing unit. In some implementations, the controller 112 receives the OS graphics abstraction 101 from the computing device 102 for further processing. The display 114 can include multiple display elements. In some implementations, the display 114 includes a spatial light modulator (SLM). The SLM can be a phase SLM, an amplitude SLM, or a phase and amplitude SLM. In some examples, the display 114 can be a digital micromirror device (DMD) or a liquid crystal on silicon (LCOS) device. In some implementations, the holographic display device 110 includes an emitter 116 adjacent to the display 114 and configured to emit light toward the display 114. The emitter 116 can be a coherent light source, such as a laser, a semi-coherent light source, such as an LED (light emitting diode), or a non-coherent light source.

[0134] Unlike conventional 3D graphics systems that receive and project a 3D scene onto a 2D display device, holographic display device 110 is configured to generate 3D output, such as holographic reconstruction 117, in the form of a light field, e.g., a 3D volume of color. In a hologram, each display element contributes to a different portion of the scene. That is, for holographic display device 110, a complete reproduction of the scene requires modulating each display element for every portion of the scene, e.g., for each primitive in the list of primitives generated by GPU 108. In some implementations, modulation of certain elements may be omitted, e.g., based on an acceptable level of accuracy in the reproduced scene.

[0135] In some implementations, controller 112 is configured to calculate the EM field contribution, e.g., phase, amplitude, or both, from each primitive to each display element, and generate, for each display element, a sum of the EM field contributions to the display element from the list of primitives. This may be done by going through all primitives and accumulating the contributions to a particular display element, or by going through each display element for each primitive.

[0136] The controller 112 can calculate the EM field contribution from each primitive to each display element based on a predetermined equation for the primitive. Different primitives can have corresponding equations. In some cases, the predetermined equation is an analytical equation, as described in further detail below with respect to FIGS. 3A-3C. In some cases, the predetermined equation is determined by solving Maxwell's equations with boundary conditions defined on the display 114. The boundary conditions can include Dirichlet boundary conditions or Cauchy boundary conditions. The display elements can then be modulated based on the sum of the EM field contributions, for example, by modulating at least one of the refractive index, amplitude index, birefringence, or phase difference of the display elements.

[0137] If the EM field values ​​at each point on the surface bounding the field are known, e.g., the solution to Maxwell's equations, then the exact unique configuration of the EM field within the volume bounded by the boundary surface can be determined. The list of primitives (or a holographic reconstruction of the corresponding hologram) and the display 114 define a 3D space, with the surface of the display 114 forming part of the boundary surface of the 3D space. By setting the EM field state (e.g., phase or phase and amplitude state) on the surface of the display 114, e.g., by shining light onto the surface of the display, the boundary conditions of the EM field can be determined. Due to the time symmetry of Maxwell's equations, the display elements are modulated based on the EM field contributions from the primitives corresponding to the hologram, so that a volumetric light field corresponding to the hologram can be obtained as a holographic reconstruction.

[0138] For example, a line primitive of a particular color of light may be set in front of the display 114. As described in more detail below with respect to FIG. 3B, an analytical expression for the linear aperture can be written as a function in space. The EM field contribution from the line primitive on a boundary surface including the display 114 can then be determined. When EM field values ​​corresponding to the calculated EM field contribution are set on the display 114, the time symmetry of Maxwell's equations allows the same linear aperture used in the calculation to appear at a corresponding location, e.g., the coordinate location of the linear primitive in a 3D coordinate system.

[0139] In some examples, as described in more detail below with respect to FIG. 3B , assume there is a line of light between two points A and B in 3D space. This light is uniformly illuminated and has intensity I for line distance l. At each infinitesimal dl along the line from A to B, an amount of light proportional to I*dl is emitted. The infinitesimal dl can act as a delta (point) source, and from the infinitesimal dl, the EM field contribution to every point on the boundary surface around the scene corresponding to the list of primitives can be determined. Thus, for every display element on display 114, an analytical equation can be determined that represents the EM field contribution at the display element from the infinitesimal line segment. A special sum / integral formula can be determined that steps along the line and accumulates the EM field contribution of the entire line to the EM field at the display element on the display. A value corresponding to this formula can be set at the display element, for example, by modulating the display element and causing it to illuminate. Then, by time reversal and a correction constant, a line can be created in 3D space at the same location as defined by points A and B.

[0140] In some implementations, the controller 112 is coupled to the display 114 via a memory buffer. The control signal 112 can generate each control signal based on the sum of the EM field contributions to each of the display elements. The control signals exist to modulate the display elements based on the sum of the EM field contributions. Each control signal is sent to a corresponding display element via the memory buffer.

[0141] In some implementations, the controller 112 is integrated with and locally coupled to the display 114. As described in further detail with respect to FIG. 1B , the controller 112 may include multiple computing units each coupled to one or more respective display elements and configured to send a respective control signal to each of the one or more respective display elements. Each computing unit may be configured to perform calculations on one or more primitives of the list of primitives. The computing units may operate in parallel.

[0142] In some implementations, the light emitters 116 are coupled to the controller 112 and configured to be turned on / off based on control signals from the controller 112. For example, the controller 112 can activate the light emitters 116 to turn on in response to the controller 112 completing a calculation, e.g., obtaining all of the sums of the EM field contributions of the display elements. As described above, when the light emitters 116 emit light on the display 114, the display's modulation elements propagate the light in different directions to form a volumetric light field corresponding to a list of primitives corresponding to the 3D object. The resulting volumetric light field corresponds to a solution to Maxwell's equations with boundary conditions defined by the modulation elements of the display 114.

[0143] In some implementations, the controller 112 is coupled to the light emitter 116 through a memory buffer. The memory buffer may be configured to control the amplitude or brightness of light emitting elements in the light emitter. The memory buffer of the light emitter 116 may have a smaller size than the memory buffer of the display 114. The number of light emitting elements in the light emitter 116 may be less than the number of display elements of the display 114, as long as the light from the light emitting elements can illuminate the entire surface of the display 114. For example, a light emitter including 64×64 organic light emitting diodes (OLEDs) may be used in a display including 1024×1024 elements. The controller 112 may be configured to simultaneously activate multiple light emitting elements of the light emitter 116.

[0144] In some implementations, the light emitter 116 is a monochromatic light source configured to emit monochromatic light, for example, red light, green light, or blue light. In some implementations, the light emitter 116 includes two or more light-emitting elements each configured to emit light having a different color. For example, the light emitter 116 may include a red light-emitting element, a green light-emitting element, and a blue light-emitting element. To display a full-color 3D object, three separate holograms for red, green, and blue colors can be calculated. That is, three EM field contributions from corresponding primitives to the display elements can be obtained. Based on the EM field contributions, the display elements can be sequentially modulated, and the light emitter 116 can be controlled to sequentially turn on the red, green, and blue light-emitting elements. Depending on the effect of temporal coherence of vision in the viewer's eye, the three colors can be combined in the eye to provide a full-color appearance. In some cases, the light emitters 116 are turned off during changes in state of the displayed image (or holographic reconstruction) and turned on for a period of time when a valid image (or holographic reconstruction) is presented, which can again rely on temporal visual coherence to make the image (or holographic reconstruction) appear stable.

[0145] In some implementations, the display 114 has a resolution fine enough to diffract visible light, for example, about 0.5 μm or less. The light emitter 116 can include a single white light source, and the emitted white light can be diffracted by the display 114 into the different colors of the holographic reconstruction.

[0146] As described in more detail below with respect to Figures 5A-5F, different configurations of the system 100 can exist. The display 114 can be reflective or transmissive. The display 114 can have a variety of sizes, ranging from small (e.g., 1-10 cm on a side) to large (e.g., 100-1000 cm on a side). Light emitted from the light emitter 116 can be from the front of the display 114 (e.g., in the case of a reflective display) or from the back of the display 114 (e.g., in the case of a transmissive display). A planar waveguide can be used to uniformly illuminate the surface of the display 114. In some implementations, the controller 112, the light emitter 116, and the display 114 can be integrated as a single unit. The integrated single unit can include the holographic renderer 130, for example, within the controller 112.

[0147] FIG. 1B shows a schematic diagram of an exemplary holographic display device 150. The holographic display device 150 can be similar to the holographic display device 110 of FIG. 1A. The holographic display device 150 includes a computing architecture 152 and a display 156. The computing architecture 152 can be similar to the controller 112 of FIG. 1A. The computing architecture 152 can include an array of parallel computing cores 154. The computing cores can be connected to adjacent computing cores via communication connections 159, such as USB-C connections or any other high-speed serial connections. The connections 159 can be included in a data distribution network through which scene data 151 (e.g., scene primitives) can be distributed among the computing cores 154.

[0148] 1A and may include an array of display elements 160 disposed on a backplane 158. The display elements 160 may be disposed on the front side of the backplane 158, and the compute core 154 may be disposed on the back side of the backplane 158. The backplane 158 may be a substrate, for example, a wafer. The compute core 154 may be on the same substrate as the display 156 or may be attached to the back side of the display 156.

[0149] Each computation core 154 may be connected to each tile (or array) of display elements 160. Each computation core 154 is configured to perform computations in parallel with one another on each primitive of the plurality of primitives in scene data 151. In some examples, computation core 154 is configured to calculate an EM field contribution from each of the primitives to each of the array of display elements 160 and generate a sum of the EM field contributions from the plurality of primitives to each of the tiles of display elements 160. Computation core 154 may receive calculated EM field contributions from other primitives of the plurality of primitives to each of the tiles of display elements 160 from other computation cores in the array of computation cores 154 and generate a sum of the EM field contributions based on the received calculated EM field contributions. Computation core 154 may generate a control signal for each of the tiles of display elements and modulate at least one characteristic of each of the tiles of display elements 160 based on the sum of the EM field contributions to the display element.

[0150] As previously mentioned, computing architecture 152 may also generate a control signal to light emitter 162, for example, in response to determining that calculation of the sum of the EM field contributions from the multiple primitives to each of the display elements is complete. Light emitter 162 emits input light 153 to illuminate modulating display element 160, which is reflected by modulating display element 160 to form a volumetric light field, e.g., holographic light field 155, corresponding to scene data 151.

[0151] As shown in FIG. 1B, tiles of display elements 160 may be interconnected into a larger display. Computational cores 154 may be interconnected accordingly for data communication and distribution. Note that in holographic computations, the parameter that varies between any two particular display elements is their physical locations. Thus, the work of computing a hologram can be shared equally between corresponding computational cores 154, allowing the entire display 150, regardless of the number of tiles, to operate at the same speed as a single tile.

[0152] FIG. 1C illustrates an exemplary system 170 for displaying objects in 3D space. System 170 may include a computing device, e.g., computing device 102 of FIG. 1A , and a holographic display device 172, e.g., holographic display 110 of FIG. 1A or 150 of FIG. 1B . A user may operate system 170 using an input device, e.g., keyboard 174 and / or mouse 176. For example, a user may create CG models of 2D object 178 and 3D object 180 via the computing device. The computing device or holographic display device 172 may include a holographic renderer, e.g., holographic renderer 130 of FIG. 1A , to render the CG models and generate corresponding graphics data for 2D object 178 and 3D object 180. The graphics data may include primitive data for each of a list of primitives corresponding to objects 178 and 180.

[0153] Holographic display device 172 may include a controller, e.g., controller 112 of FIG. 1A or 152 of FIG. 1B, and a display 173, e.g., display 114 of FIG. 1A or 156 of FIG. 1B. The controller may calculate respective sums of EM field contributions from the primitives to each display element of display 173 and generate control signals for modulating each display element based on the respective sums of EM field contributions. Holographic display device 172 may further include light emitters, e.g., light emitter 116 of FIG. 1A or light emitter 162 of FIG. 1B. The controller may generate timing control signals for activating the light emitters. When light from the light emitters illuminates the surface of display 173, the modulating display elements may propagate light in 3D space to form a volumetric light field corresponding to a holographic reconstruction of 2D object 178 and a holographic reconstruction of 3D object 180. Thus, 2D object 178 and 3D object 180 are displayed as respective holographic reconstructions floating in 3D space in front of display 173 .

[0154] In some implementations, the computing device transmits non-primitive-based data, e.g., a video of a recorded light field, to the holographic display device 172. The holographic display device 172 can compute and generate corresponding holograms, e.g., a series of consecutive holograms, and display them as corresponding holographic reconstructions in 3D space. In some implementations, the computing device transmits computer-generated holographic content to the holographic display device 172 simultaneously with live holographic content. The holographic display device 172 can also compute and generate corresponding holograms and display the content as corresponding holographic reconstructions in 3D space.

[0155] FIG. 2 illustrates an exemplary configuration 200 for electromagnetic (EM) field calculations. A display 202, e.g., an LCOS device, includes an array 204 of elements, and a list of primitives, including point primitives 206, in a 3D space 208. The 3D space 208 includes a boundary surface 210. Within a 3D coordinate system XYZ, the point primitives 206 have coordinate information (x, y, z). Each display element 204 lies in a flat plane relative to the other display elements 204 and has a 2D position (u, v). The display elements 204 also have a location in the 3D space. Through mathematical point transformation, the 2D positions (u, v) can be translated to six coordinates 250 within the 3D coordinate system. That is, the surface of the display 202 forms part of the boundary surface 210. Therefore, the EM field contributions from the list of primitives to the display elements, calculated by defining boundary conditions at the surface of the display 202, represent a portion of the total EM field contributions from the primitives to the display elements. A scale factor, for example 6, can be multiplied by the sum of the EM field contributions of each of the display elements to obtain a scaled sum of the field contributions, and the display elements can be modulated based on the scaled sum of the field contributions.

[0156] Example of EM Field Contribution of a Primitive Primitives can be used in standard computer graphics rendering. Each type of primitive in standard computer graphics corresponds, in this formulation, to a discrete mathematical function that defines a single holographic primitive, a geometric element added to a hologram. Each type of primitive can correspond to a formula for calculating the EM field contribution to a display element. A primitive can be a point primitive, a line primitive, or a polygon (e.g., triangle) primitive. As shown below, an analytical formula can be derived by calculating the EM field propagation from the corresponding primitive to the display element of a display.

[0157] 3A illustrates exemplary EM propagation from a point primitive 304 to an element 302 of a display 300. In a 3D coordinate system XYZ, it is assumed that the z coordinate is 0 across the display 300, i.e., negative z values ​​are behind the display 300 and positive z values ​​are in front of the display 300. The point primitive 304 has coordinates (x, y, z) and the display element 302 has coordinates (u, v, 0). A distance duv between the point primitive 304 and the display element 302 can be determined based on these coordinates.

[0158] The point primitive 304 can be viewed as a point charge with a time-varying amplitude. According to electromagnetic theory, the electromagnetic field E generated by such a point charge is

[0159]

number

[0160] Therefore, the electric field Eu,v at the display element (u,v) is

[0161]

number

[0162] 2, the surface of the display 300 forms only a portion of the EM field boundary. A scale factor δ can be applied to the electric field Eu,v to obtain a scaled electric field Eφ(u,v) at the display element, which scales the partial boundary as follows:

[0163]

number

[0164]

number

[0165] 3B illustrates an example of EM propagation from a line primitive 306 to a display element 302 of a display 300 in a 3D coordinate system XYZ. As previously described, the display element 302 may have coordinates (u, v, 0) when z=0. The line primitive 306 includes two endpoints: P0 having coordinates (x0, y0, z0) and P1 having coordinates (x1, y1, z1). A distance d0 between the endpoint P0 and the display element may be determined based on the coordinates. Similarly, a distance d1 between the endpoint P1 and the display element may be determined based on the coordinates. A distance d01 between the two endpoints P0 and P1, e.g., d01=d1−d0, may also be determined.

[0166] As mentioned above, the line primitives can be treated as superpositions or linear deformations, and the corresponding analytical expression of the line primitive as a linear aperture can be obtained as a delta function distributed in space, which can be a closed-form representation of a continuous 3D line segment as a hologram.

[0167] 3C illustrates exemplary EM propagation from a triangle primitive 308 to a display element 302 of a display 300 in a 3D coordinate system XYZ. As previously described, the display element 302 may have coordinates (u, v, 0) where z=0. The triangle primitive 308 includes three endpoints: P0 (x0, y0, z0), P1 (x1, y1, z1), and P2 (x2, y2, z2). Distances d0, d1, and d2 between the display element and endpoints P0, P1, and P2, respectively, may be determined based on their coordinates.

[0168] Similar to the line primitives of Figure 3B, the triangle primitives can be treated as continuous apertures in space, and an analytical expression for the EM field contribution of the triangle primitive to the display element can be obtained by integration, which can be simplified to obtain an expression for efficient calculation.

[0169] Primitive Calculation Example As previously mentioned, a controller, for example, controller 112 of Figure 1A, can calculate the EM field contribution from a primitive to a display element based on analytical expressions that may be determined as shown above. As an example, the EM field contribution of a line primitive is calculated as follows:

[0170] Each display element in the display has a physical location in space, with each display element lying in a flat plane relative to the other display elements. Assuming the display elements and their controllers are laid out as is customary in displays and memory devices, a simple mathematical point transformation can be used to convert the logical location of a particular display element, based on the display element's logical memory address in the processor, to the display element's actual physical location in space. Thus, as the display element's logical memory address is looped through the logical memory space in the processor, the corresponding actual physical location in space across the surface of the display can be identified.

[0171] As an example, if the display has a 5 μm pitch, each logical address increment will move 5 μm in the x direction, and when the display's x-axis resolution limit is reached, the next increment will return to the original physical position on the x axis and increase the physical position on the y axis by 5 μm. The third spatial coordinate, z, can be assumed to be 0 across the surface of the display, i.e., negative z values ​​are behind the display and positive z values ​​are in front of the display.

[0172] To begin the line calculation, it may be determined that the scaled physical distances between the current display element and each of the two points of the line primitive are d0 and d1. In practice, d0 and d1 may be calculated once per primitive, since all subsequent calculations of distances across display elements are small perturbations of the initial values. In this way, the calculation is performed in one dimension.

[0173] An exemplary computation process for each primitive may include the following computation code: DD=f(d1,d0), iscale=SS*COLOR*Alpha1, C1=-2*iscale*sin(DD / 2)*sin((Alpha2)*cos(Alpha3), C2=-2*iscale*sin(DD / 2)*sin(Alpha2)*sin(Alpha4), where SS, Alpha1, Alpha2, Alpha3, and Alpha4 are pre-calculated constants, COLOR is the RGB color value passed with the primitive, and all values ​​are scalar, single-precision floating-point numbers. To improve computational efficiency, both the sine and cosine functions can be looked up in tables stored in the controller.

[0174] The results of C1 and C2 may then be accumulated at each display element, for each primitive, e.g., in the display element's accumulator, and normalized once at the end of the calculation for the display element. At this point, as previously described, the controller may send a first control signal to the display element to modulate it based on the calculated result and a second control signal to the light emitter to turn it on and emit light. In response, a holographic reconstruction (or holographic light field) becomes visible to the viewer. When illuminated, the modulated display element may produce a distinct, continuous line of color in three-dimensional space through light.

[0175] In some implementations, the calculation code includes hexadecimal code, e.g., at the beginning of the code, for clearing the previous accumulation result in the accumulator. The calculation code can also include hexadecimal code, e.g., at the end of the code, for storing the accumulator results in respective memory buffers for each display element. In some implementations, a computing device, e.g., computing device 102 of FIG. 1A, sends hexadecimal codes of multiple background or static primitives to a controller upon application launch or at intervals between displaying frames that do not affect the basic display frame rate. The computing device can then send one or more combinations of the hexadecimal codes, possibly along with other foreground or dynamic primitives, to the controller at a very high rate, and the controller can form corresponding control signals to modulate the display elements of the display.

[0176] These computational processes can be two orders of magnitude simpler and faster than the most efficient line-drawing routines in conventional 2D display technologies. Furthermore, the computational algorithms scale linearly with the number of display elements. Therefore, by scaling the computing units of the controller as a 2D network processing system, the computational needs of the display's increasing surface area can be kept up with.

[0177] Example of calculation implementation A Maxwell holographic controller, such as controller 112 of FIG. 1A, can calculate the EM field contributions from the primitives to the display elements based on analytical expressions that may be determined as shown above. The controller may be implemented, for example, in an ASIC, FPGA, or GPU, or any combination thereof.

[0178] In a modern GPU pipeline, the GPU takes a description of a geometry and vertex and fragment shader programs and generates pixel output of color and depth onto one or more output image surfaces (called render targets). This process involves an explosion of information: the shape is expanded into fragment shadows, and then a visibility test is performed to select whether work needs to be performed on each of those fragments. A fragment is a record containing all the information contained in shading its sample points, such as barycentric coordinates on triangles, interpolated values ​​like color or texture coordinates, and surface derivatives. After creating these records, the visibility test is the process of rejecting records that do not contribute to the final image. Fragments that pass the visibility test can be organized into work groups called wavefronts or warps, which are executed in parallel by shader engines. The output values ​​produced by these shader engines are written back to memory as pixel values ​​for display or for later use as input textures in rendering passes.

[0179] Maxwell holography can significantly simplify the rendering process. In Maxwell holographic computations, every primitive contributes to every display element. There is no need to scale shapes to pixels, and there is no need to apply visibility tests before assembling wavefronts. This also eliminates the need for decisions or communication between Maxwell holographic pipelines, allowing the computation to become a parallel problem with multiple possible solutions, each tailored for speed, cost, size, or energy optimization. This graphics pipeline is significantly shorter, with fewer intermediate steps, no copying or movement of data, and fewer decisions, leading to reduced latency between the start of drawing and the results being ready to display. Maxwell holographic rendering can therefore create displays with extremely low latency. As described below, the accuracy of Maxwell holographic computations can be improved, for example, by using fixed-point numbers in the Maxwell holographic pipeline, and the computation speed can be optimized, for example, by optimizing mathematical functions.

[0180] Using Fixed-Point Numbers When calculating the EM contribution from each primitive at each display element (e.g., phasel), intermediate calculations are involved that generate very large numbers. These large numbers involve special handling because their fractional parts must also be preserved during the calculation.

[0181] Floating-point values ​​have the disadvantage that they are most accurate near the origin (0 on the number line) and lose one bit of precision for every power of two as they move away from the origin. For numbers close to within the range [-1,1], floating-point values ​​can be highly accurate, but as numbers reach tens of millions—for example, the point where a single-precision 32-bit IEEE-745 floating-point value has no remaining fractional digits—the entire significand (also called mantissa) is used to represent the integer portion of the value. However, it is the fractional part of large numbers that is of particular interest to preserve in Maxwell holography.

[0182] Fixed-point numbers are sometimes used in Maxwell holographic calculations. A fixed-point representation is a number whose decimal point does not change from case to case. By selecting the correct number of bits for the integer and decimal parts of a number, the same number of decimal points can be obtained regardless of the magnitude of the number. Fixed-point numbers are represented as integers using an implicit scale factor. For example, in a 16-bit fixed-point value containing 8 decimal points, 14.375 can be represented as the number 3680 (binary 0000111001100000). This fixed-point number can also be represented as an "unsigned 16.8" fixed-point number, or u16.8 for short. Negative numbers can include an additional sign bit and are stored in "two's complement" format. In this way, the accuracy of calculations can be significantly improved.

[0183] Optimizing mathematical functions As shown above, Maxwell holographic computation involves the use of transcendental mathematical functions, such as sine, cosine, and arctangent. On a CPU, these functions are implemented as floating-point library functions that may use special CPU instructions, or on a GPU, they are implemented as floating-point units within the GPU. These functions are written to accept arguments as floating-point numbers and return results in the same floating-point representation. These functions are constructed for the general case, to be accurate when floating-point numbers are exact, to round correctly, and to handle all edge cases in floating-point representations (+ / - infinity, NaN, signed zero, and denormalized floats).

[0184] In Maxwell holographic computations using fixed-point representations, there is no need to use denormalized floating-point numbers due to gradual underflow, no need to handle NaN results from operations like division by zero, no need to change floating-point rounding modes, and no need to raise floating-point exceptions to the operating system, all of which allows transcendental mathematical functions to be simplified (and / or optimized), for example, as described below.

[0185] In some cases, optimizations can be made to accept arguments in one fixed-point format and return values ​​with different levels of precision, e.g., input s28.12, output s15.14. This can be particularly desirable when computing the sine of tens of millions of large values, where the input argument can be large but the output only needs to represent the range of values ​​[-1,1], or when computing the arctangent, which accepts arbitrary values ​​but returns values ​​in the range [-π / 2,π / 2].

[0186] In some cases, optimizations can be made to implement transcendental functions as fully enumerated lookup tables, interpolated tables, semi-tables based on polynomial functions, or semi-tables based on full minimax polynomials, depending on the input range involved. Special range reduction methods can also be applied to deal with large inputs that general-purpose GPU pipelined computations sometimes omit for speed.

[0187] In some cases, another optimization can be to convert trigonometric calculations from the range [-π,π] to a signed two's complement representation in the range [-1,1], which has the advantage of not requiring expensive modulo 2π division operations.

[0188] Example of implementation of occlusion Occlusion is often considered a difficult and important subject in computer graphics, and particularly so in computational holography. This is because, at least in some cases, the occlusion problem in projected CGI is static, whereas in holographic systems, what is hidden and what is visible depends on the viewer's position and orientation. To address holographic occlusion, the wave method of GS holography, or its derivatives, has been developed. However, masking or blocking contributions from parts of a scene that are behind other parts of the scene, can be very complex and computationally expensive with GS methods.

[0189] In Maxwell holography, occlusion issues can be addressed relatively easily because which display elements (e.g., facets) correspond to which primitives is completely deterministic and trivial. For example, whether a particular display element contributes to the reconstruction of a particular primitive can be determined when the computation for that primitive is performed. After determining that multiple display elements do not contribute to a particular primitive due to occlusion, the EM contribution from the particular primitive is omitted from the computation of the sum of the EM contributions to one of the multiple display elements.

[0190] For illustrative purposes only, Figures 3D-3F illustrate the determination of display elements that do not contribute to a particular primitive (a point in Figure 3D, a line in Figure 3E, and a triangle in Figure 3F) using a line primitive as an occluder. The line primitive includes a start point O1 and an end point O2.

[0191] As shown in Figure 3D, a point primitive P0 is located near the display, behind an occluder. By extending the lines connecting O1-P0 and O2-P0, the range of display elements in the display from D1 to D2 that do not contribute to the reconstruction of the point primitive P0 is determined.

[0192] In some examples, the coordinate information of O1, O2, and P0 is calculated and known by a GPU (e.g., GPU 108 in FIG. 1A) and stored, for example, in a "Z" buffer before the scene is sent to a Maxwell holographic controller (e.g., controller 112 in FIG. 1A). For example, the coordinate information in the XZ plane with y=0 can be O1(Ox1, Oz1), O2(Ox2, Oz2), and P0(Px, Pz), where Oz1=Oz2=Oz. Based on this coordinate information, the coordinate information of D1 and D2 can be determined as follows: Dx1=Px+ρ(Px-Ox2), Dx2=Dx1+ρ(Ox2-Ox1) (4) where ρ=Pz / (Oz-Pz), and Dz1=Dz2=0.

[0193] The information for D1 and D2 may be stored as additional information in the "S" buffer of the Maxwell holographic controller in addition to the information in the Z buffer of point primitive P0. In this way, the additional information can be used to trivially mask the contribution of a particular display element (within the range of D1-D2) to a particular primitive P0 in the indexed primitive list.

[0194] 3E illustrates the determination of how a particular display element contributes to a line primitive when an occluder is present in front of the line primitive. By connecting a particular display element D0 to the start point O1 and end point O2 of the occluder, two point primitives P1 and P2 on the line primitive are determined as intersection points. Therefore, the particular display element D0 does not contribute to the reconstruction of the line primitive portion P1-P2 on the line primitive. Therefore, when calculating the total EM contribution to the particular display element D0, the EM contribution from the line primitive portion P1-P2 is not calculated.

[0195] This can be implemented in two ways. In the first method, by taking into account occlusions from occluders, the EM contributions from portions P0-P1 and P2-Pn to a particular display element D0 are summed as the EM contribution of the line primitive to the particular display element D0. In the second method, by taking into account occlusions from occluders, the EM contribution from the entire line primitive P0-Pn is calculated along with the EM contribution from portion P1-P2, and the difference between the two calculated EM contributions can be considered as the EM contribution of the line primitive to the particular display element D0. The coordinate information of P1 and P2 or portions P1-P2 as part of the line primitive that does not contribute to the particular display element D0 can be stored in the "S" buffer of the Maxwell holographic controller, along with occluder information and other information, in the "Z" buffer of the GPU.

[0196] FIG. 3F illustrates how a particular display element contributes to a triangle primitive when an occluding body is present in front of the triangle primitive. By connecting a particular display element D0 to the start point O1 and end point O2 of the occluding body, four points on the sides of the triangle primitive—P1, P2, P3, and P4—are determined as intersection points. Therefore, the particular display element D0 does not contribute to the reconstruction of the portion of the triangle primitive enclosed by points P1, P2, P3, P4, and PC. Therefore, when calculating the sum of the EM contributions to the particular display element D0, the EM contribution from the portion P1-P2-P3-P4-PC of the triangle primitive is not calculated. That is, by considering the occlusion from the occluding body, only the EM contributions from the first triangle formed by points PA, P1, and P2 and the second triangle formed by points PB, P3, and P4 are summed as the EM contribution of the triangle primitive PA-PB-PC. The coordinate information of P1, P2, P3, and P4 or the triangle primitives PA-P1-P2 and PB-P3-P4 as part of the triangle primitive PA-PB-PC contributing to a particular display element D0 is stored in the "S" buffer of the Maxwell holographic controller, along with occluder information and other information may be stored in the "Z" buffer of the GPU.

[0197] The implementation of occlusion in Maxwell holography can mask the contribution of certain primitives (or certain parts of primitives) in an indexed primitive list to certain display elements, such that the "Z" buffer in the GPU can be converted to an "S" buffer in the Maxwell holographic controller. This not only provides accurate, physically correct occlusion, but also saves computation time by allowing primitives that do not contribute to a particular display element to proceed to the calculation for the next display element. The "S" buffer can contain additional information related to the diffraction efficiency of the display.

[0198] The "S" buffer can also include rendering features such as holographic specular highlights, where the reflectivity of a surface depends on the viewing angle. In traditional CGI, specular highlights depend only on the orientation of the object being rendered, but in Maxwell holography, the direction from which the object is viewed also plays a role. Thus, geometric specular information can be encoded within the "S" buffer as additive (specular) rather than subtractive (occlusion) contributions. In Maxwell holography, the calculations for holographic specular highlights can be substantially the same as those for holographic occlusion.

[0199] Example of a stitched implementation When light falls on a display modulated using EM contributions from a list of primitives in a 3D object, the modulated display causes the light to propagate in different directions, forming a volumetric light field corresponding to the primitives. This volumetric light field is a Maxwellian holographic reconstruction. Two adjacent primitives in a 3D object, such as triangle primitives, have a shared edge. During reconstruction, a stitching problem can arise in which the light intensity of the shared edge can be doubled due to the separate reconstruction of the two adjacent primitives. This problem can affect the appearance of the reconstructed 3D object.

[0200] To address the stitching problem in Maxwell holography, adjacent primitives can be scaled down by a predetermined factor to form a gap between them, as shown in FIG. 3G. In some cases, instead of scaling two adjacent primitives, only one primitive or a portion of a primitive is scaled down. For example, a line of a triangle primitive can be scaled down to separate it from another triangle primitive. In some cases, this scaling can include scaling different portions of the primitive using different predetermined factors. This scaling can be designed so that the gap is large enough to separate adjacent primitives and minimize stitching problems, yet small enough to appear seamless in the reconstructed 3D object. The predetermined factor can be determined based on display information, such as the maximum spatial resolution of the display.

[0201] In some cases, a resizing operation can be applied to primitive data for a primitive obtained from a holographic renderer, e.g., holographic renderer 130 of Figure 1A, and the resized primitive data for the primitive is sent to a Maxwell holographic controller, e.g., the controller of Figure 1A. In some cases, the controller can perform the resizing operation on the primitive data obtained from the holographic renderer before calculating the EM contributions of the primitive to the display elements of the display.

[0202] Texture mapping implementation example Texture mapping is a technique developed in computer graphics. The basic idea is to take a source image and apply it as a decal to a surface in a CGI system, allowing detail to be rendered in a scene without the need for additional complex geometry. Texture mapping can include techniques for creating realistic lighting and surface effects in CGI systems and can generally refer to the application of surface data to a triangular mesh.

[0203] In Maxwell holography, flat-shaded, even interpolated, triangular meshes can be rendered in true 3D using an analytical relationship between any triangle in space and a phase map on a holographic device. However, to be compatible with modern rendering engines, the ability to map information onto the surface of these triangles is included. This ability can pose practical problems, as the speed of the method derives from the existence of an analytical mapping and does not allow for data-driven amplitude changes.

[0204] The Discrete Cosine Transform (DCT) is an image compression technique that can be thought of as a real-valued version of the Fast Fourier Transform (FFT). The DCT relies on an encoding / decoding process that assigns weights to cosine harmonics within a particular image. The result of the encoding is a set of weights equal to the number of pixels in the original image, and if the image is reconstructed using all weights, there is no loss in information. However, for many images, an acceptable reconstruction can be made from a small subset of the weights, allowing for large compression ratios.

[0205] The decoding (rendering) process of the DCT in two dimensions involves a weighted double sum over all DCT weights and all target pixels. This weighted double sum can be applied to Maxwell holography for texture mapping. In Maxwell holography, rendering a triangle involves a "spiked" double integral in phase space to determine the phase contribution of any individual facet to the triangle. This integral is collapsed into a double sum that mirrors the double sum in the DCT reconstruction, and the analytical expression for the triangle can then be re-derived in terms of the DCT weights. This implementation of the DCT approach in Maxwell holographic computation makes it possible to render fully texture-mapped triangles and employ image compression of the rendered texture into triangle data, leveraging existing toolsets that automatically compress texture and image data using DCT / JPEG.

[0206] In some implementations, to render a textured Maxwell holographic triangle, the desired spatial resolution for mapping on a specified surface is first calculated. A texture with this resolution is then provided and DCT-compressed with angles to obtain origin information for placing the texture at the correct position on the triangle. A list of the triangle's corners and DCT weights is then included in an indexed primitive and sent to the Maxwell holographic controller. The DCT weights can be included in the triangle primitive's EM contribution to each display element. Textured triangles can be n times slower than flat triangles, where n is the number of nonzero DCT weights sent with the primitive. Modern techniques for "fragment shading" can be implemented in Maxwell holographic systems using a DCT encoding step that replaces the filtering step of traditional projection rendering.

[0207] As an example, the following equation shows the DCT weights Bpq of an image:

[0208]

number

[0209]

number

[0210]

number

[0211] By decoding, the amplitude value Amn can be obtained as follows:

[0212]

number

[0213]

number

[0214] When calculating the EM contribution of a textured triangle primitive to a display element (eg, a facet), a DCT term with corresponding DCT weighting A*mn may be included in the calculation as follows:

[0215]

number

[0216] Process Example 4 is a flowchart of an example process 400 for displaying an object in 3D. Process 400 may be performed by a controller for a display. The controller may be controller 112 of FIG. 1A or 152 of FIG. 1B. The display may be display 114 of FIG. 1A or 156 of FIG. 1B.

[0217] Data including primitive data for each of the primitives corresponding to the objects in 3D space is obtained (402). This data may be obtained from a computing device, for example, computing device 102 of FIG. 1A. The computing device may process the scene to generate the primitives corresponding to the objects. The computing device may include a renderer to generate the primitive data for the primitives. In some implementations, the controller generates the data itself, for example, by rendering the scene.

[0218] The primitives may include at least one of a point primitive, a line primitive, or a polygon primitive. The list of primitives may be indexed in a particular order, for example, in which an object may be reconstructed. The primitive data may include color information including at least one of a texture color or a gradient color. For example, a line primitive may have at least one of a gradient color or a texture color. A polygon primitive may also have at least one of a gradient color or a texture color. The primitive data may also include texture information and / or shading information of the primitive on one or more surfaces of the primitive (e.g., a triangle). The shading information may include modulation for at least one of the color or brightness on one or more surfaces of the primitive. The primitive data may also include coordinate information for each of the primitives within a 3D coordinate system.

[0219] The display may include multiple display elements, and the controller may include multiple computing units. Coordinate information for each of the display elements within the 3D coordinate system may be determined based on coordinate information for the list of primitives within the 3D coordinate system. For example, a distance between the display and an object corresponding to the primitive may be predetermined. The coordinate information for the display element may be determined based on the predetermined distance and the coordinate information for the primitive. The coordinate information for each of the display elements may correspond to a logical memory address of the element stored in memory. In this manner, when the controller loops through the logical memory addresses of the display elements within the controller's logical memory space, the corresponding actual physical location of the display element within the space may be identified.

[0220] An EM field contribution from each of the primitives to each of the display elements is determined by calculating the propagation of the EM field from the primitive to the element in a 3D coordinate system 404. The EM field contribution can include at least one of a phase contribution or an amplitude contribution.

[0221] As described above with respect to Figures 3A-3C, at least one distance between the primitive and the display element may be determined based on the coordinate information of the display element and the coordinate information of the primitive. In some cases, at least one distance may be calculated only once for each primitive. For example, the controller may determine a first distance between a first primitive of the primitives and a first element of the display elements based on the coordinate information of the first primitive and the coordinate information of the first element, and determine a second distance between the first primitive and a second element of the elements based on the first distance and the distance between the first element and the second element. The distance between the first element and the second element may be predetermined based on the pitch of the multiple elements of the display.

[0222] The controller can determine the EM field contribution from the primitive to the display element based on a predetermined equation of the primitive and at least one distance. In some cases, the predetermined equation can be determined by analytically calculating the propagation of the EM field from the primitive to the element, as described above with respect to FIGS. 3A-3C . In some cases, the predetermined equation can be determined by solving Maxwell's equations. In particular, the Maxwell's equations can be solved by providing boundary conditions defined at the surface of the display. The boundary conditions can include Dirichlet boundary conditions or Cauchy boundary conditions. The primitive and the display element exist in a 3D space, and the surface of the display forms part of a boundary surface of the 3D space. The predetermined equation can include at least one of functions including a sine function, a cosine function, and an exponential function. During the calculation, the controller can identify values ​​of at least one of the functions in a table stored in memory, thereby improving the calculation speed. The controller can determine, for each of the primitives, the EM field contribution to each of the display elements by determining the first EM field contribution from the first primitive to the display element in parallel with determining the second EM field contribution from the second primitive to the display element.

[0223] For each display element, a sum of the EM field contributions to the display element from the list of primitives is generated (406).

[0224] In some implementations, the controller determines first EM field contributions from the primitives to the first display elements and sums the first EM field contributions of the first elements, and determines second EM field contributions from the primitives to the second display elements and sums the second EM field contributions of the second elements. The controller can include multiple computing units. The controller can determine the EM field contributions from the first primitives to the first elements with a first computing unit in parallel with determining the EM field contributions from the second primitives to the first elements with a second computing unit.

[0225] In some implementations, the controller determines a first respective EM field contribution from the first primitive to each of the display elements and determines a second respective EM field contribution from the second primitive to each of the display elements. The controller then accumulates the EM field contributions of the display elements by adding the second respective EM field contribution to the first respective EM field contribution for the display element. In particular, the controller can determine the first respective EM field contribution from the first primitive to each of the display elements using the first computing unit in parallel with determining the second respective EM field contribution from the second primitive to each of the display elements using the second computing unit.

[0226] A first control signal is sent to the display, the first control signal being present to modulate at least one characteristic of each display element based on the sum of the electromagnetic field distributions to the display element (408), wherein the at least one characteristic of the element includes at least one of a refractive index, an amplitude index, a birefringence, or a retardation.

[0227] The controller can generate a respective control signal for each of the display elements based on the sum of the EM field contributions from the primitives to the element, where each control signal is present to modulate at least one characteristic of the element based on the sum of the EM field contributions from the primitives to the element, i.e., the first control signal includes each control signal for the display element.

[0228] In some cases, the display is controlled by an electrical signal. In that case, each control signal can be an electrical signal. For example, an LCOS display includes an array of microelectrodes with voltages that individually control the intensity of the elements. An LCOS display can be filled with a birefringent liquid crystal (LC) formulation that changes the refractive index. In this way, each control signal from the controller can control the relative refractive index of the entire display element and, therefore, the relative phase of light passing through the display.

[0229] As previously mentioned, the surface of the display forms a portion of the boundary surface. The controller may multiply the sum of the field contributions for each of the elements by a scale factor to obtain a scaled sum of the field contributions and generate each control signal based on the scaled sum of the field contributions for the elements. In some cases, the controller may normalize the sum of the field contributions for each of the elements, e.g., across all elements, and generate each control signal based on the normalized sum of the field contributions for the elements.

[0230] A second control signal is sent to the light emitters, the second control signal being present to turn on the light emitters and illuminate the modulated display (410). The controller can generate and send the second control signal in response to determining completion of obtaining the sum of the field contributions for each of the display elements. Due to time symmetry (or conservation of energy), the modulation elements of the display can propagate light in different directions to form a volumetric light field corresponding to an object in 3D space. The volumetric light field can correspond to a solution of Maxwell's equations with boundary conditions defined by the modulation elements of the display.

[0231] In some implementations, the light emitter is coupled to a controller configured to control the amplitude or brightness of one or more light emitting elements in the light emitter via a memory buffer. The memory buffer of the light emitter can have a size smaller than the memory buffer of the display. The number of light emitting elements in the light emitter can be less than the number of elements of the display. The controller can be configured to simultaneously activate one or more light emitting elements of the light emitter.

[0232] In some examples, the light emitter includes two or more light-emitting elements each configured to emit light having a different color. The controller may be configured to sequentially modulate the display using information associated with a first color during a first period and to modulate the display using information associated with a second color during a second period, and to control the light emitter to sequentially turn on the first light-emitting element during the first period to emit light having the first color and to sequentially turn on the second light-emitting element during the second period to emit light having the second color. In this manner, multi-colored objects can be displayed in 3D space.

[0233] In some cases, the display has a resolution fine enough to diffract light: an illuminator can shine white light onto the display, which can diffract this white light into light having different colors, thereby displaying multicolored objects.

[0234] System Example 5A-5F show exemplary system implementations for 3D displays, any one of which can correspond, for example, to system 100 of FIG. 1A.

[0235] 5A shows a system 500 including a reflective display. System 500 includes a computer 502, a controller 510 (e.g., an ASIC), a display 512 (e.g., an LCOS device), and an emitter 514. Computer 502 can be computing device 102 of FIG. 1A, controller 510 can be controller 112 of FIG. 1A, display 512 can be display 114 of FIG. 1A, and emitter 514 can be emitter 116 of FIG. 1A.

[0236] As shown in FIG. 5A, computer 502 includes application 504, which includes renderer 503, for rendering a scene of objects. The scene data to be rendered is continuously processed by video driver 505 and GPU 506. GPU 506 can be GPU 108 of FIG. 1A and can be configured to generate a list of primitives corresponding to the scene and each primitive data. For example, video driver 505 can be configured to process the scene data to be rendered and generate a list of primitives. As previously mentioned, GPU 506 can include a conventional 2D renderer, such as conventional 2D renderer 120 of FIG. 1A, to render the primitives into a list of items for drawing on 2D display screen 508. GPU 506 or controller 510 can include a holographic renderer, such as holographic renderer 130 of FIG. 1A, to render the list of primitives into graphics data for display by display 512.

[0237] The controller 510 is configured to receive the graphics data from the computer 502, calculate the EM field contribution to each of the elements of the display 512 from the list of primitives, and generate a respective sum of the EM field contributions from the primitives to each of the elements. The controller 510 can generate a respective control signal for each of the display elements to modulate at least one property of the display element. The controller can transmit the respective control signals to the display elements of the display 512 via a memory buffer 511 of the display 512.

[0238] The controller 510 can also generate and transmit control signals, such as light emission timing signals, to activate the light emitters 514. For example, the controller 510 can generate and transmit a control signal in response to determining that the calculation of the sum of the EM field contributions from the primitives to the display elements is complete. As previously described, the controller 510 can transmit the control signals to the light emitters 514 via a memory buffer. The memory buffer can be configured to control the amplitude or brightness of the light emitting elements in the light emitters 514 and to simultaneously activate the light emitting elements.

[0239] 5A, light emitter 514 can emit collimated light beams 516 that are incident on the front surface of display 512 at an angle of incidence ranging from 0 degrees to 90 degrees. The emitted light beams are reflected from the front surface of display 512 to form a holographic light field 518 that corresponds to an object that can be seen by a viewer.

[0240] FIG. 5B illustrates another system 520 that includes another reflective display 524. Compared to system 500 of FIG. 5A, system 520 includes a larger reflective display 524. To accommodate this, display controller 522 is included in a wedge-shaped housing that can support light emitters 526. Controller 522 is similar to controller 510 of FIG. 5A and can be configured to receive graphics data from computer 521, calculate EM field contributions from primitives to each of the display elements of display 524, and generate respective sums of the EM field contributions from the primitives to each of the display elements. Controller 522 then generates respective control signals for each of the display elements to modulate at least one characteristic of the display elements and transmits the respective control signals to the display elements of display 524 via memory buffer 523 of display 524.

[0241] Controller 522 also sends control signals to light emitter 526, activating light emitter 526. Light emitter 526 emits divergent or semi-collimated light beam 527 that covers the entire surface of display 524. Light beam 524 is reflected by display 524 where it is modulated, forming holographic light field 528.

[0242] FIG. 5C shows a system 530 including a transmissive display 534. The transmissive display 534 can be, for example, a large display. The system 530 includes a controller 532, which can be similar to the controller 510 of FIG. 5A. The controller 532 can be configured to receive graphics data from the computer 531, calculate EM field contributions from the primitives to each of the display elements of the display 534, and generate respective sums of the EM field contributions from the primitives to each of the display elements. The controller 532 then generates respective control signals for each of the display elements to modulate at least one characteristic of the display elements and transmits the respective control signals to the display elements of the display 534 via a memory buffer 533 of the display 534.

[0243] 5A and 520 of FIG. 5B, the light emitter 536 in system 530 is positioned behind the rear surface of the display 534. To cover a large surface area of ​​the display 534, the light emitter 536 directs a diverging or semi-collimated light beam 535 at the rear surface of the display 534. The light beam 524 is transmitted through the display 534 to be modulated, forming a holographic light field 538.

[0244] 5D shows another system 540 that includes a transmissive display 544. System 540 also includes a controller 542 and light emitters 546. Controller 542 may be similar to controller 510 of FIG. 5A and may be configured to receive graphical data from computer 541, perform calculations on the graphical data, generate and send control signals for modulation to display 544, and generate and send timing signals for actuating light emitters 546.

[0245] The light emitter 546 can include a light source 545 and a waveguide 547. Light emitted from the light source 545 can be coupled into the waveguide 547, for example, from a vertical cross section of the waveguide. The waveguide 547 is configured to guide the light to uniformly illuminate the surface of the display 544. The light guided by the waveguide 547 is incident on the back surface of the display 544 and transmitted through the display 544 to form a holographic light field 548.

[0246] Unlike systems 500 of Figure 5A, 520 of Figure 5B, and 530 of Figure 5C, system 540 integrates controller 542, display 544, and waveguide 547 into a single unit 550. In some cases, waveguide 547 and light source 545 can be integrated in a planar form as an active light-emitting waveguide, thereby further increasing the degree of integration in single unit 550. As previously mentioned, single unit 500 can be connected to other similar units 550 to form a larger holographic display device.

[0247] 5E shows another system 560 that includes a transmissive display 564. Compared to system 540, transmissive display 564 may implement a larger display than transmissive display 544. For example, transmissive display 564 may have a larger area than controller 562, and to accommodate this, controller 562 may be located further away from display 564. System 560 includes light emitter 566 that includes light source 565 and waveguide 567. Waveguide 567 is integrated with display 546, for example, on the backside of display 564. In some implementations, display 564 may be assembled on the front side of a substrate, and waveguide 567 may be assembled on the backside of the substrate.

[0248] 1A, and may be configured to receive graphics data from computer 561, perform calculations on the graphics data, generate and transmit control signals for modulation to display 564 via memory buffer 563, and generate and transmit timing signals to activate light source 565. Light emitted from light source 565 is guided within waveguide 567 to illuminate the backside of display 564 and transmitted through display 564, forming holographic light field 568.

[0249] FIG. 5F shows another system 570 including a reflective display 574. The reflective display 574 can be, for example, a large display. A waveguide 577 of an illuminator 576 is disposed in front of the reflective display 574. A controller 572, similar to the controller 562 of FIG. 5E, can be configured to receive graphical data from a computer 571, perform calculations on the graphical data, generate and transmit control signals for modulation on the display 574 via a memory buffer 573, and generate and transmit timing signals for activating a light source 575 of the illuminator 576. Light coupled from the light source 575 of the illuminator 576 is guided onto the front surface of the display 574 and reflected by the front surface to form a holographic light field 578.

[0250] Display implementation example As mentioned above, the display in Maxwell holography can be a phase modulation device. The phase elements (or display elements) of the display can be represented as facets. For illustrative purposes only, the following describes a liquid crystal on silicon (LCOS) device functioning as a phase modulation device. An LCOS device is a display that uses a liquid crystal (LC) layer on a silicon backplane. The LCOS device can be optimized to achieve a minimum pitch of facets, a minimum crosstalk between facets, and / or a large usable phase modulation or phase difference (e.g., at least 2π).

[0251] A list of parameters can be controlled to optimize the performance of the LCOS device, including the birefringence of the LC mixture (Δn), the cell gap (d), the dielectric anisotropy of the LC mixture (Δε), the rotational viscosity of the LC mixture (η), and the maximum voltage (V) applied between the silicon backplane above the LC layer and the common electrode.

[0252] There can be fundamental trade-offs between the parameters of liquid crystal materials. For example, a fundamental boundary parameter is the available phase modulation or retardation (Re), which can be expressed as: Re=4π·Δn·d / λ (8) where λ is the wavelength of the input light. If the phase difference Re of red light, which has a wavelength of about 0.633 μm, needs to be at least 2π, then: Δn·d≧0.317μm (9) The above equation implies that there is a direct trade-off between the cell gap (d) and the birefringence (Δn) of the LC mixture.

[0253] Another boundary parameter is the switching speed, or the switching time (T) it takes for the liquid crystal (LC) molecules in the LC layer to reach the desired orientation after a voltage is applied. For example, real-time video (approximately 60 Hz) using a three-color field-sequential color system involves an LC layer modulation of 180 Hz or higher, which imposes an upper limit of 5.6 milliseconds (ms) on the LC switching speed. The switching time (T) is related to several parameters, including the liquid crystal, cell gap, and applied voltage. First, T is proportional to d². As the cell gap d decreases, the switching time decreases proportionally to the square of d. Second, the switching time is also related to the dielectric anisotropy (Δε) of the liquid crystal (LC) mixture; higher dielectric anisotropy shortens the switching time, and lower viscosity shortens the switching time.

[0254] A third boundary parameter can be the fringing field. Due to the high electron mobility of crystalline silicon, LCOS devices can be fabricated with extremely small facet sizes (e.g., less than 10 μm) and gaps between facets of less than 1 micron. When adjacent facets are operated at different voltages, the lateral component of the fringing field deforms the LC directors near the facet edges, which significantly degrades the electro-optical performance of the device. In addition, when the facet gap becomes comparable to the wavelength of the incident light, diffraction effects can cause significant light loss. To keep noise within acceptable levels, the facet gap must be kept equal to or smaller than the facet pitch.

[0255] In some cases, LCOS devices are designed with a 2 μm facet pitch and a cell gap of approximately 2 μm when fringe field boundary conditions are observed. According to the above formula, Δn·d≧0.317 μm, Δn must be 0.1585 or greater, which is achievable using current liquid crystal technology. After the minimum birefringence for a particular facet pitch is determined, the LC can be optimized for switching speed, for example, by increasing the dielectric anisotropy and / or decreasing the rotational viscosity.

[0256] Mounting uneven facets of displays In an LCOS device, a circuit chip, such as a complementary metal-oxide-semiconductor (CMOS) chip or equivalent, controls one facet by controlling the voltage of a reflective metal electrode buried beneath the chip's surface. The common electrode for all facets is provided by a transparent conductive layer made of indium tin oxide on a cover glass. The facets can have the same size and shape (e.g., square). For example, a chip can contain 1024 x 768 plates, each with an independently addressable voltage. As mentioned above, when the facet gaps become comparable to the wavelength of the incident light, diffraction effects appear in the periodic structure of the LCOS device, causing significant light loss.

[0257] In Maxwell holographic computation, each facet receives the sum of the EM contributions from each primitive and is relatively independent of one another. Therefore, the facets of an LCOS device in Maxwell holography can be designed to be different from one another. For example, as shown in FIG. 6A, an LCOS device 600 can be made of multiple non-uniform (or irregular) facets 602, with at least two facets 602 having different shapes. The non-uniform shape of the facets 602 can significantly reduce or eliminate diffraction aberrations, among other effects, and thus improve image quality. Although the facets can have non-uniform shapes, they can be designed to have a size distribution (e.g., approximately 3 μm) that meets the desired spatial resolution. A silicon backplane can be configured to provide each circuit (e.g., including metal electrodes) for each facet according to the facet's shape.

[0258] To select a particular phasel in an array of phasels in an LCOS device, a first voltage is applied to a word line connecting the row of phasels containing the particular phasel, and a second voltage is applied to a bit line connecting the column of phasels containing the particular phasel. Each phasel has a resistance that can limit the speed at which an LCOS device can operate.

[0259] As previously mentioned, in Maxwell holography, facets can have different sizes. As shown in FIG. 6B, an LCOS device 650 is designed to include one or more facets 654 that are larger than the other facets 652. All of the facets can still have a size distribution that meets the desired resolution. For example, 99% of the facets have a size of 3 μm, and 1% of the facets have a size of 6 μm. The larger size of facets 654 allows for at least one buffer 660 to be placed within facets 654 in addition to other circuitry identical to that within facets 652. Buffer 660 is configured to buffer the applied voltage so that the voltage is applied to only a smaller number of facets within a row or column of facets. Buffer 660 can be an analog circuit, e.g., made of transistors, or a digital circuit, e.g., made of multiple logic gates, or any combination thereof.

[0260] For example, as shown in FIG. 6B, to select a particular facel 652*, a voltage is applied to word line 651 and another voltage is applied to bit line 653. Facel 652* is in the same row as a larger facel 654, which includes a buffer 660. The voltage is primarily applied to the first few facels in the row, before the larger facel 654, and is blocked by the buffer 660 within the larger facel 654. In this manner, the operational speed of LCOS device 650 can be improved. To further improve the performance of LCOS device 650, other circuitry can also be placed within LCOS device 650 along with the larger size of facel 654. While facel 654 and facel 652 in FIG. 6B have square shapes, the facels can have shapes different from those shown in FIG. 6A, as long as one or more facels 654 have a larger size than the other facels 652.

[0261] Calibration Example The unique properties of Maxwell holography in this disclosure allow for protection of the calibration techniques, which can provide a significant competitive advantage in the practical manufacture of high quality displays. In combination with the Maxwell holographic computational techniques, several calibration techniques can be implemented, including: (i) Using an image sensor with a Dirichlet boundary condition modulator and / or with mechanical and software diffractive and non-diffractive calibration techniques. (ii) Software tuning and calibration, including individual color calibration and tuning, using Dirichlet boundary condition modulators; and (iii) Embedding silicon functionality into boundary condition modulators, allowing photon detection to be directly integrated into the modulator, which when combined with Maxwell holography creates a powerful and unique method for simplifying the manufacturing calibration process.

[0262] In the following, for purposes of explanation only, three types of calibration are performed for a phase-based display, for example an LCOS display: Each phase element can be represented as a facet.

[0263] Phase Calibration The amount of phase added to light impinging on an LCOS phase element (or phasel) can be directly determined by the voltage applied to the LCOS phasel. This is due to the rotation of birefringent liquid crystals (LC) in the presence of an electric field, thus changing the index of refraction and slowing the speed of light, which alters the phase. The altered phase may depend on the electrical properties of the LC and the silicon device in which the LC resides. To achieve high-quality holographic images, the digital signal sent to the LCOS must be converted to an accurate analog voltage. Phase calibration is included to ensure that the LCOS device can properly convert the digital signal to the analog signal applied to the LC to produce the largest phase range. This conversion is expected to result in linear behavior; that is, regardless of the starting voltage value, when the voltage is changed by a fixed increment, the phase also changes by a fixed increment.

[0264] In some cases, a user can modify the digital-to-analog converter (DAC) using the LCOS device to control the amount of analog voltage output given a digital input signal. A digital potentiometer can be applied to each input bit. For example, if there are eight input bits, there can be eight digital potentiometers corresponding to each input bit. The same digital input from the digital potentiometer can be applied to all facels of the LCOS device. A bit set to "1" activates a voltage, and a bit set to "0" does not activate a voltage. All voltages from such "1" bits are summed to obtain the final voltage sent to each facel. There may be a DC voltage applied in all cases so that all "0" bits result in a non-zero reference voltage. Thus, by setting the value of the digital potentiometer of the LCOS device, phase calibration of the LCOS device can be performed. For example, as described above, the controller can calculate the EM field contribution to each of the display's facels from a list of primitives, generate respective sums of the EM field contributions from the primitives to each of the facels, and generate respective control signals for each of the facels to modulate the phase of the facels. The same digital input from a digital potentiometer, which is the difference between the phase calibrations for each facet, can be applied to all of the facets of the LCOS device to adjust each control signal. The digital input can be set once during operation of the LCOS device, for example to display a hologram.

[0265] To determine the optimal set of calibration values ​​for the phase of a digital input, a genetic algorithm can be applied, where many input values, such as phase range or holographic image contrast, lead to a single output value. This output value can be reduced to a single numerical value called fitness. The genetic algorithm can be configured to explore different combinations of input values ​​until it achieves the output with the highest fitness. In some cases, the algorithm can receive two or more of the most fit inputs and combine their component values ​​together to create a new input that has the characteristics of the received inputs but is different from each of the received inputs. In some cases, the algorithm can change one of these component values ​​to something other than any of the received fit inputs, referred to as a "mutation," to add diversity to the available fit inputs. In some cases, one or more optimal values ​​can be found by leveraging knowledge gained from previous measurements with good results while trying new values ​​so that the optimum is not limited to a local maximum.

[0266] There are several ways to calculate the goodness-of-fit output value. One method is to calculate the phase change of light given a set of digital inputs applied to all facets on the LCOS. In this scheme, the incident light can be polarized. As it acts on the LCOS, the polarization of the incident light can change depending on the rotation of the LC. The incident light can be reflected through another polarizer set to either the same polarization or a polarization 90 degrees different from the original polarization, and then enter the photodetector. Therefore, as the rotation of the LC changes, the intensity as seen by the photodetector can change. Therefore, the phase change of the light can be indirectly perceived through the intensity fluctuation. Another method for calculating the phase change is to measure the intensity difference of the Maxwell holographic reconstruction from the background. This is most effective in projection displays. Measuring intensity in such cases may require using computer vision algorithms to identify the Maxwell holographic reconstruction and measure its intensity.

[0267] Calibrating the adjustment Light sources are not guaranteed to be tuned within a holographic device and therefore require adjustment. Different liquid crystals (LCs) may behave differently given the wavelength of the light source. Furthermore, both LCs and light sources may vary from device to device, potentially providing different characteristics, e.g., changes in object size, for the same input hologram when displayed with different base colors. Furthermore, certain hardware features may apply different optical effects, e.g., lensing, to the output light, which also require correction.

[0268] In some implementations, the problem described above may be addressed by applying a mathematical transform to the phase calculated for the facets of the display. The phase is the respective sum of the EM field contributions from a list of primitives to the facets. This mathematical transform may be derived from a mathematical formula, e.g., a Zernike polynomial, and may be varied by changing the polynomial coefficients or other variable input values. This mathematical transform may vary from facet to facet and from color to color. For example, there are coefficients in the Zernike polynomial that correspond to the amount of tilt applied to the light after it reflects off the display.

[0269] To determine these coefficients / input values, a hardware arrangement can be created where a camera is pointed at a reflective surface in the case of a projection display, or directly within the LCOS in the case of a direct-view display. A series of holographic test patterns and objects can be sent to the display and viewed by the camera. The camera can use machine vision algorithms to determine what is being displayed and then calculate its goodness of fit. For example, if a grid of dots is the test pattern, goodness of fit can be measured by how close the dots are to each other, how centered the dots are, and how distorted the dots are (e.g., size or pincushion). Different features can have different goodness-of-fit values. Depending on these values, corrections can be applied, for example, in the form of changing the coefficients of Zernike polynomials, until a predetermined satisfactory level of goodness of fit is achieved. These test patterns can be rendered at different distances to ensure that the adjustment of the object is consistent at all distances, not just at one point. Such depth-based calibration may involve an iterative process that involves changing the depth of the holographic test pattern, and in the case of a projection display, the depth of the reflective surface, and previous calibrations may be repeated until converging on a solution that works correctly at both depths. Finally, a white dot may be displayed to indicate the validity of the calibration.

[0270] Color calibration: In holographic or other displays, it is important that colors match between displays when any two units render the same image, and also that colors defined by television and computer display standards, such as the Rec709 standard for the sRGB color space for high-definition television (HDTV) or computer monitors, match. Different batches of hardware components, such as LEDs and laser diodes, can behave differently for the same input and output different colors as perceived by the human eye. Therefore, it is important that a color standard exists to which all display units can be calibrated.

[0271] In some implementations, objective measurements of colors specified by intensity and chromaticity measurements can be obtained by measuring color intensities against the CIE (Commission Internationale de l'Eclairage) standard observer curve. By requiring each display to reproduce a series of known color and intensity samples and then measuring the output light using a colorimeter device calibrated against the CIE standard observer curve, the color output of the device in the CIE XYZ color space can be objectively defined. Deviations of the measurements from any known correct value can be used to adapt the output color on the display to adjust the output color to the correct color, which can be done by using an iterative measure-adapt-measure feedback loop. After Maxwell holography produces accurate output for a particular series of inputs, the final adaptation can be encoded as an illuminant lookup table that maps input values ​​to output intensities and a color matrix transform that converts input colors to output color space values. These calibration tables can be embedded in the device itself to produce reliable, objective output colors.

[0272] Furthermore, given an LCOS device with features fine enough to control diffraction with sub-wavelength precision, there may be no need for tristimulus emission (e.g., a linear mix of red, green, and blue); the LCOS device may be illuminated using a single-spectrum light source, and the output of the facets may be selectively adjusted to produce tristimulus, tetrastimulus, or even N-stimulus output light, which, in combination with spatial dithering patterns, can reproduce the full spectral output of a color rather than the typical tristimulus approximation. Given a sufficiently broad-spectrum light emitter, Maxwell holography can produce any reflected color within the spectral locus of the human visual system.

[0273] Implementations of the objects and functional operations described herein may be implemented in digital electronic circuitry, in computer software or firmware tangibly embodied in computer hardware containing the structures disclosed herein and equivalents thereof, or in one or more combinations thereof. Implementations of the subject matter described herein may be implemented as one or more computer programs, such as one or more modules of computer program instructions encoded on a tangible, non-transitory computer storage medium for execution by or to control the operation of a data processing apparatus. Alternatively or additionally, the program instructions may be encoded on an artificially generated propagated signal, such as a machine-generated electrical, optical, or electromagnetic signal, generated to encode information for transmission to an appropriate receiver apparatus for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random-access or sequential-access memory device, or a combination of one or more of these.

[0274] The terms “data processing apparatus,” “computer,” or “electronic computing device” (or equivalents as understood by those skilled in the art) refer to data processing hardware and encompass all kinds of apparatus, devices, and machines for processing data, including, for example, a programmable processor, computer, or multiple processors or computers. An apparatus may be or further include special-purpose logic circuitry, such as a central processing unit (CPU), FPGA (field programmable gate array), or ASIC (application-specific integrated circuit). In some implementations, data processing apparatuses and special-purpose logic circuitry may be hardware- and software-based. An apparatus may optionally include code that creates an execution environment for a computer program, such as code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or one or more combinations of these. This specification contemplates using a data processing apparatus with or without a conventional operating system.

[0275] A computer program, which may be referred to or described as a program, software, software application, module, software module, script, or code, can be written in any form of programming language, including compiled or interpreted, or declarative or procedural, and can be deployed in any form, including as a stand-alone program suitable for use in a computing environment, or as a module, component, subroutine, or other unit. A computer program may, but need not, correspond to a file in a file system. A program may be stored as part of a file holding other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program, or in multiple integrated files, e.g., files storing one or more modules, subprograms, or portions of code. A computer program may be deployed to run on one computer or multiple computers, located at one site or distributed across multiple sites, interconnected by a communications network. While portions of the program depicted in the various figures are shown as individual modules implementing various features and functionality through various objects, methods, or other processes, the program may instead include multiple sub-modules, third-party services, components, libraries, etc., as needed. Conversely, features and functionality of various components may be combined into a single component if desired.

[0276] The processes and logic flows described herein may be performed by one or more programmable computers executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and apparatus may be implemented as, special purpose logic circuitry, such as a CPU, GPU, FPGA, or ASIC.

[0277] A computer suitable for running a computer program can be based on a general-purpose microprocessor, a special-purpose microprocessor, both, or any other type of CPU. Typically, a CPU receives instructions and data from read-only memory (ROM) and / or random access memory (RAM). The key elements of a computer are the CPU for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer also includes one or more mass storage devices, e.g., magnetic, magneto-optical, or optical disks, for storing data, or is operatively coupled to such mass storage devices, or receives and / or transmits data to and from such mass storage devices. However, a computer need not include such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a portable audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, e.g., a universal serial bus (USB) flash drive, to name a few.

[0278] Computer-readable media suitable for storing computer program instructions and data (either transient or non-transient, as appropriate) include all forms of non-volatile memory, media, and memory devices, including semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices, magnetic disks such as internal hard disks or removable disks, magneto-optical disks, and CD-ROM, DVD-R, DVD-RAM, and DVD-ROM disks. Memory may store various objects or data, including caches, classes, frameworks, applications, backup data, jobs, web pages, web page templates, database tables, repositories for storing business and dynamic information, and any other suitable information, including any parameters, variables, algorithms, instructions, rules, constraints, or references thereto. Additionally, memory may include any other suitable data, such as logs, policies, security or access data, report files, and the like. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.

[0279] To provide for user interaction, implementations of the subject matter described herein may be implemented on a computer that includes a display device for displaying information to a user, e.g., a cathode ray tube (CRT), liquid crystal display (LCD), light-emitting diode (LED), or plasma monitor, as well as a keyboard and pointing device, e.g., a mouse, trackball, or trackpad, that the user can use to provide input to the computer. Input may be provided to the computer using a touchscreen, such as the surface of a tablet computer with pressure sensitivity, a multi-touch screen using capacitive or electrical detection, or other types of touchscreen. Other types of devices may also be used to provide for user interaction. For example, feedback provided to the user can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback, and input from the user may be received in any form, including acoustic, voice, or tactile input. Additionally, the computer may interact with a user by sending documents to and receiving documents from a device being used by the user, for example, by sending a web page to a web browser on the user's client device in response to a request received from the web browser.

[0280] The terms "graphical user interface" or "GUI" may be used in the singular or plural to describe one or more graphical user interfaces and each of the displays of a particular graphical user interface. Thus, a GUI may refer to any graphical user interface that processes information and efficiently presents information results to a user, including, but not limited to, a web browser, a touch screen, or a command line interface (CLI). Generally, a GUI may include multiple user interface (UI) elements, some or all of which are associated with a web browser, such as interactive fields, pull-down lists, and buttons that can be operated by a business-suited user. These and other UI elements may relate to or represent the functionality of a web browser.

[0281] Implementations of the subject matter described herein may be implemented in a computing system that includes a back-end component, e.g., a data server, or a middleware component, e.g., an application server, or a front-end component, e.g., a client computer equipped with a graphical user interface or web browser through which a user can interact with an implementation of the subject matter described herein, or any combination of one or more such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of wired or wireless digital data communication, e.g., a communications network. Examples of communication networks include a local area network (LAN), a radio access network (RAN), a metropolitan area network (MAN), a wide area network (WAN), a worldwide interoperability for microwave access (WIMAX), a wireless local area network (WLAN) using, for example, 902.11 a / b / g / n and 902.20, all or part of the Internet, and any other communication system or systems in one or more locations. Networks may communicate using, for example, internet protocol (IP) packets, frame relay frames, asynchronous transfer mode (ATM) cells, voice, video, data, or other suitable information between network addresses.

[0282] A computing system may include clients and servers. Clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0283] In some implementations, any or all of the components of a computing system, both hardware and software, may interface with each other or with each other using application programming interfaces (APIs) or a service layer. APIs may include specifications for routines, data structures, and object classes. APIs may be computer language-independent or computer language-dependent and may refer to a complete interface, a single function, or a set of APIs. A service layer provides software services to a computing system. Through this service layer, all service consumers may be able to access the functionality of various components of the computing system. Software services provide reusable, defined business functionality through defined interfaces. For example, an interface may be software written in any suitable language that provides data in any suitable format. The APIs and service layer may be integral or stand-alone components in relation to other components in the computing system. Furthermore, without departing from the scope of this specification, any or all portions of the service layer may be implemented as a child module or sub-module of another software module, enterprise application, or hardware module.

[0284] While this specification contains many specific implementation details, these should not be construed as limitations on the scope of all inventions or claimed subject matter, but rather as descriptions of features that may be specific to particular implementations of particular inventions. Certain features described herein in the context of separate implementations may also be implemented in combination in a single implementation. Conversely, individual features described in the context of a single implementation may also be implemented separately in multiple implementations or in any suitable subcombination. Furthermore, while features may be described as functioning in a particular combination and even initially claimed as such, one or more features from a claimed combination may, in some cases, be deleted from that combination, and the claimed combination may be directed to a subcombination or a variation of a subcombination.

[0285] Specific implementations of the subject matter have been described. As will be apparent to those skilled in the art, other implementations, modifications, and permutations of the described implementations are within the scope of the following claims. Although acts are depicted in a particular order in the figures or claims, it should not be understood that such acts must be performed in the particular order depicted, or in any sequential order, to achieve desirable results, nor should it be understood that all depicted acts must be performed (some acts may be considered optional). In particular circumstances, multitasking or parallel processing may be advantageous and may be performed where deemed appropriate. It should be noted that the present invention includes the following aspects. [Aspect 1] determining, for each of a plurality of primitives corresponding to objects in three-dimensional (3D) space, an electromagnetic (EM) field contribution to each of a plurality of elements of a display by calculating, in a 3D coordinate system, a propagation of the EM field from the primitive to the element; and generating, for each of the plurality of elements, a sum of the EM field contributions to the element from the plurality of primitives. [Aspect 2] 2. The method of embodiment 1, wherein the EM field contribution comprises at least one element selected from the group consisting of a phase contribution and an amplitude contribution. Aspect 3 3. The method of embodiment 1 or 2, wherein the primitives comprise at least one element selected from the group consisting of point primitives, line primitives, and polygon primitives. Aspect 4 4. The method of any one of aspects 1 to 3, wherein the primitives have line primitives that include at least one element selected from the group consisting of a gradient color, a textured color, and a shading effect. Aspect 5 5. The method of any one of aspects 1-4, wherein the primitives have polygon primitives that include at least one element selected from the group consisting of gradient colors, textured colors, and shading effects. Aspect 6 6. The method of any one of aspects 1-5, wherein the plurality of primitives are indexed in a particular order. Aspect 7 7. The method of any one of aspects 1-6, further comprising obtaining respective primitive data for each of the plurality of primitives. Aspect 8 The method of aspect 7, wherein the respective primitive data for each of the plurality of primitives includes respective color information for the primitive, and the determined EM field contribution for each of the elements includes information corresponding to the respective color information for the primitive. Aspect 9 9. The method of embodiment 8, wherein the color information includes at least one element selected from the group consisting of a texture color and a gradient color. Aspect 10 10. The method of any one of aspects 7-9, wherein the respective primitive data for each of the plurality of primitives includes texture information for the primitive. Aspect 11 Aspect 11. The method of any one of aspects 7-10, wherein the respective primitive data for each of the plurality of primitives includes shading information on one or more surfaces of the primitive. Aspect 12 The method of embodiment 11, wherein the shading information includes modulation for at least one element selected from the group consisting of at least one of color on the one or more surfaces of the primitive and brightness on the one or more surfaces of the primitive. Aspect 13 13. The method of any one of aspects 7 to 12, wherein the respective primitive data for each of the plurality of primitives includes respective coordinate information for the primitive within the 3D coordinate system. Aspect 14 14. The method of claim 13, wherein respective coordinate information for each of the plurality of elements in the 3D coordinate system is determined based on the respective coordinate information of the plurality of primitives in the 3D coordinate system. Aspect 15 15. The method of embodiment 14, wherein the respective coordinate information for each of the elements corresponds to a logical memory address of the element stored in a memory. Aspect 16 The method of aspect 14 or 15, wherein for each of the plurality of primitives, determining the contribution of the EM field to each of the plurality of elements includes determining at least one distance between the element and the primitive based on the respective coordinate information of the element and the respective coordinate information of the primitive within the 3D coordinate system. Aspect 17 determining, for each of the plurality of primitives, the contribution of the EM field to each of the plurality of elements; determining a first distance between the first primitive of the plurality of primitives and the first element of the plurality of elements based on the coordinate information of each first primitive and the coordinate information of each first element; determining a second distance between the first primitive and a second element of the plurality of elements based on the first distance and a distance between the first element and the second element. Aspect 18 20. The method of embodiment 17, wherein the distance between the first element and the second element is predetermined based on a pitch of the plurality of elements of the display. Aspect 19 At least one of the plurality of primitives is a line primitive including first and second endpoints, and determining at least one distance between the element and the primitive includes: determining a first distance between the element and the first endpoint of the line primitive;

[0033] Aspect 19. The method of any one of aspects 16-18, comprising determining a second distance between the element and the second point of the line primitive. Aspect 20 At least one of the plurality of primitives is a triangle primitive including first, second, and third endpoints, and determining at least one distance between the element and the primitive includes: determining a first distance between the element and the first endpoint of the triangle primitive; determining a second distance between the element and the second point of the triangle primitive;

[0033] Aspect 19. The method of any one of aspects 16-18, comprising determining a third distance between the element and the third point of the triangle primitive. Aspect 21 21. The method of any one of aspects 16 to 20, wherein, for each of the plurality of primitives, determining the EM field contribution to each of the plurality of elements comprises determining the EM field contribution from the primitive to the element based on a predetermined equation for the primitive and the at least one distance. Aspect 22 22. The method of embodiment 21, wherein the predetermined equation is determined by analytically calculating the propagation of the EM field from the primitive to the element. Aspect 23 22. The method of embodiment 21, wherein the predetermined equation is determined by solving Maxwell's equations. Aspect 24 24. The method of embodiment 23, wherein the Maxwell's equations are solved by providing boundary conditions defined at a surface of the display. Aspect 25 25. The method of embodiment 24, wherein the boundary conditions comprise Dirichlet boundary conditions or Cauchy boundary conditions. Aspect 26 26. The method of any one of aspects 21 to 25, wherein the plurality of primitives and the plurality of elements exist within the 3D space, and a surface of the display forms part of a boundary surface of the 3D space. Aspect 27 the predetermined equation includes at least one element selected from the group consisting of a function including a sine function, a function including a cosine function, and a function including an exponential function; 27. The method of any one of aspects 21-26, wherein determining the EM field contribution comprises identifying a value of the at least one of the functions in a table stored in a memory. Aspect 28 determining, for each of the plurality of primitives, the EM field contribution to each of the plurality of elements and generating the sum of the field contributions for each of the plurality of elements; determining a first EM field contribution from the plurality of primitives to a first element of the plurality of elements and summing the first EM field contribution to the first element; 28. The method of any one of aspects 1-27, comprising determining a second EM field contribution from the plurality of primitives to a second element of the plurality of elements and summing the second EM field contribution to the second element. Aspect 29 determining the first EM field contribution from the plurality of primitives to the first element; 29. The method of embodiment 28, comprising determining an EM field contribution from a first primitive of the plurality of primitives to the first element in parallel with determining an EM field contribution from a second primitive of the plurality of primitives to the first element. Aspect 30 determining, for each of the plurality of primitives, the contribution of the EM field to each of the plurality of elements; determining a first respective EM field contribution from a first one of the plurality of primitives to each of the plurality of elements; determining a second respective EM field contribution from a second one of the plurality of primitives to each of the plurality of elements; Including, generating the sum of the field contributions for each of the plurality of elements; accumulating the EM field contributions of the elements by adding the second respective EM field contributions for the elements to the first respective EM field contributions for the elements. 28. The method of any one of embodiments 1 to 27, comprising: Aspect 31 31. The method of aspect 30, wherein determining the first EM field contribution from the first primitive to each of the plurality of elements is performed in parallel with determining the second EM field contribution from the second primitive to each of the plurality of elements. Aspect 32 determining, for each of the plurality of primitives, the contribution of the EM field to each of the plurality of elements; Aspect 32. The method of any one of aspects 1-31, comprising determining a first EM field contribution from a first primitive of the plurality of primitives to the first element of the plurality of elements in parallel with determining a second EM field contribution from a second primitive of the plurality of primitives to a first element. Aspect 33 Aspect 33. The method of any one of aspects 1-32, further comprising: generating, for each of the plurality of elements, a respective control signal based on the sum of the EM field contributions from the plurality of primitives to the element, wherein each control signal is present to modulate at least one characteristic of the element based on the sum of the EM field contributions from the plurality of primitives to the element. Aspect 34 34. The method of embodiment 33, wherein the at least one characteristic of the element comprises at least one member selected from the group consisting of refractive index, amplitude index, birefringence, and retardation. Aspect 35 35. The method of claim 33 or 34, wherein each control signal comprises an electrical signal, an optical signal, a magnetic signal, or an acoustic signal. Aspect 36 36. The method of any one of aspects 33 to 35, further comprising multiplying the sum of the field contributions for each of the elements by a scale factor to obtain a scaled sum of the field contributions; and wherein each of the control signals is generated based on the scaled sum of the field contributions for the elements. Aspect 37 normalizing the sum of the field contributions for each of the elements; Aspect 37. The method of any one of aspects 33 to 36, wherein each control signal is based on the normalized sum of the field contributions for the elements. Aspect 38

[0039] Aspect 38. The method of any one of aspects 33-37, further comprising transmitting the respective control signals to the elements. Aspect 39 further comprising transmitting a control signal to the light emitter; Aspect 39. The method of any one of aspects 1-38, wherein the control signal indicates turning on the light emitter such that the light emitter emits light on the display. Aspect 40 40. The method of embodiment 39, wherein the control signal is transmitted in response to determining completion of obtaining the sum of the field contributions for each of the plurality of elements. Aspect 41 41. The method of claim 39 or 40, wherein the modulation elements of the display propagate the light in different directions to form a volumetric light field corresponding to the object in the 3D space. Aspect 42 42. The method of embodiment 41, wherein the volumetric light field corresponds to a solution of Maxwell's equations with boundary conditions defined by the modulation elements of the display. Aspect 43 43. The method of any one of aspects 39 to 42, wherein the light comprises white light and the display is configured to diffract the white light into light having different colors. Aspect 44 44. The method of any one of aspects 1-43, further comprising representing values ​​using a fixed-point representation during calculations. Aspect 45 45. The method of embodiment 44, wherein each of the values ​​is expressed as an integer with an implicit scale factor. Aspect 46 46. ​​The method of any one of aspects 1-45, further comprising performing the mathematical function using a fixed-point representation. Aspect 47 47. The method of embodiment 46, wherein the mathematical function includes at least one member selected from the group consisting of sine, cosine, and arctangent. Aspect 48 performing the mathematical function, receiving an expression in a first fixed-point format; 48. The method of claim 46 or 47, comprising outputting the value in a second fixed-point format having a different level of precision than the precision of the first fixed-point format. Aspect 49 performing the mathematical function includes looking up a table for calculation of the mathematical function; 49. The method of any one of aspects 46 to 48, wherein the table comprises at least one element selected from the group consisting of a fully enumerated lookup table, an interpolated table, a semi-table based on a polynomial function, and a semi-table based on a full minimax polynomial. Aspect 50 performing the mathematical function, 50. The method of any one of embodiments 46-49, comprising applying a special range reduction to the input. Aspect 51 performing the mathematical function, 51. The method of any one of aspects 46-50, comprising converting trigonometric calculations in the range [-π,π] to signed two's complement representation in the range [-1,1]. Aspect 52 52. The method of any one of aspects 1-51, further comprising resizing a first primitive adjacent to a second primitive by a predetermined factor such that reconstruction of the first primitive does not overlap with reconstruction of the second primitive. Aspect 53 53. The method of embodiment 52, wherein the pre-determined coefficients are determined based at least in part on a resolution of the display. Aspect 54 acquiring respective primitive data for each of the plurality of primitives, wherein the respective primitive data for each of the plurality of primitives includes respective coordinate information of the primitive within the 3D coordinate system; determining new coordinate information for each of the first primitives based on the coordinate information for each of the first primitives and the predetermined coefficients; 54. The method of embodiment 52 or 53, further comprising: Aspect 55 55. The method of embodiment 54, further comprising determining an EM field contribution from the first primitive to each of the plurality of elements based on each of the new coordinate information of the first primitive. Aspect 56 56. The method of any one of embodiments 52 to 55, further comprising resizing the second primitive by the predetermined factor. Aspect 57 the first primitive and the second primitive share a common portion; 57. The method of any one of aspects 52 to 56, wherein resizing the first primitives comprises resizing the intersection of the first primitives. Aspect 58 Resizing the first primitive includes: 58. The method of any one of embodiments 52-57, comprising resizing the first primitive in a predetermined direction. Aspect 59 acquiring primitive data for each of a plurality of primitives corresponding to an object in three-dimensional (3D) space; calculating a first respective electromagnetic (EM) field contribution from a first of the plurality of primitives to each of a plurality of elements of a display; calculating a second respective EM field contribution from a second one of the plurality of primitives to each of the plurality of elements of the display; The method, wherein calculating the first EM field contribution from the first primitive is at least partially parallel with calculating the second EM field contribution from the second primitive. Aspect 60 60. The method of embodiment 59, wherein calculating a first EM field contribution from the first primitive to a first element of the plurality of elements is in parallel with calculating a second EM field contribution from a second primitive of the plurality of primitives to the first element. Aspect 61 61. The method of embodiment 59 or 60, comprising calculating a respective EM field contribution from each of the plurality of primitives to each of the plurality of elements. Aspect 62 said calculation of each EM field contribution comprising: extending the shape of the object onto the plurality of elements; Applying a visibility test before assembling the wavefronts; 62. The method of embodiment 61, wherein the method does not include at least one element selected from the group consisting of decision-making or communication between parallel computations on different primitives. Aspect 63 said calculation of each EM field contribution comprising: Parallel computation of different primitives can be tailored for speed, cost, size, or energy optimization; Reduce the wait time between starting a drawing and being ready to see the results, and Using fixed-point representation to improve precision; Optimizing mathematical functions to optimize calculation speed 63. The method of embodiment 61 or 62, wherein the method is configured to cause at least one element selected from the group consisting of: Aspect 64 64. The method of any one of embodiments 59-63, further comprising representing values ​​using a fixed-point representation during calculations. Aspect 65 Representing values ​​using the fixed-point representation Subnormal floating point for gradual underflow, and Handling NaN results from operations involving division by zero; Changing the floating-point rounding mode; Raising floating-point exceptions to the operating system 65. The method of embodiment 64, wherein the method does not include at least one element selected from the group consisting of: Aspect 66 66. The method of any one of aspects 59 to 65, further comprising, for each of the plurality of elements, accumulating EM field contributions to the element by adding the second EM field contribution to the element to the first EM field contribution to the element. Aspect 67 67. The method of any one of aspects 59 to 66, further comprising generating, for each of the plurality of elements, a respective control signal based on a sum of the EM field contributions from the plurality of primitives to the element, wherein each control signal is present to modulate at least one characteristic of the element based on the sum of the EM field contributions from the plurality of primitives to the element. Aspect 68 acquiring primitive data for each of a plurality of primitives corresponding to an object in three-dimensional (3D) space; using the primitive data for each first primitive and a second primitive to resize the first primitive adjacent to the second primitive by a predetermined factor; updating the primitive data of the first primitive based on the result of the resizing. Aspect 69 the respective primitive data for each of the plurality of primitives includes respective coordinate information of the primitive within a 3D coordinate system; updating each of the primitive data 69. The method of embodiment 68, comprising determining new coordinate information for each of the first primitives based on the coordinate information for the first primitives and the predetermined coefficients. Aspect 70 70. The method of claim 68 or 69, wherein the pre-determined coefficients are determined such that the reconstruction of the first primitive does not overlap the reconstruction of the second primitive in the 3D space. Aspect 71 A method according to any one of aspects 68 to 70, wherein the resizing is performed so that a gap between the reconstruction of the first primitive and the reconstruction of the second primitive in the 3D space is large enough to separate the first and second primitives and minimize the effects of overlap, and small enough to make the reconstructions appear seamless. Aspect 72 72. The method of any one of embodiments 68 to 71, wherein the pre-determined coefficients are determined based at least in part on a resolution of the display. Aspect 73 73. The method of any one of aspects 68-72, further comprising storing the updated primitive data of the first primitive in a buffer. Aspect 74 A method according to any one of aspects 68 to 73, wherein the resizing is performed during a rendering process of the object to obtain the primitive data for each of the plurality of primitives. Aspect 75 sending updated primitive data for the plurality of primitives to a controller; Aspect 75. The method of any one of aspects 68 to 74, wherein the controller is configured to determine a respective electromagnetic (EM) field contribution from each of the plurality of primitives to each of a plurality of elements of a display based on the updated primitive data for the plurality of primitives. Aspect 76 Aspect 75. The method of any one of aspects 68 to 74, further comprising determining an EM field contribution from the first primitive to each of a plurality of elements of a display based on the updated primitive data for the first primitive. Aspect 77 77. The method of any one of embodiments 68 to 76, further comprising resizing the second primitive by the predetermined factor. Aspect 78 the first primitive and the second primitive share a common portion; 78. The method of any one of aspects 68 to 77, wherein resizing the first primitives comprises resizing the intersection of the first primitives. Aspect 79 Resizing the first primitive includes: 79. The method of any one of embodiments 68-78, comprising resizing the first primitive in a predetermined direction. Aspect 80 Resizing the first primitive includes: Resizing a first portion of the first primitive by a first predetermined factor; and scaling a second portion of the second primitive by a second predetermined factor; Aspect 80. The method of any one of aspects 68-79, wherein the first predetermined coefficient is different from the second predetermined coefficient. Aspect 81 obtaining a plurality of discrete cosine transform (DCT) weights of an image that are mapped to a designated surface of a particular primitive among a plurality of primitives corresponding to an object in three-dimensional (3D) space; determining each EM field contribution from the particular primitive to each of a plurality of elements of a display by considering the effect of the plurality of DCT weights of the image. Aspect 82 determining a resolution of the image to be mapped to the specified surface of the particular primitive; 82. The method of embodiment 81, further comprising: determining the plurality of DCT weights of the image based on the resolution. Aspect 83 83. The method of embodiment 81 or 82, further comprising decoding the DCT weights of the image to obtain respective DCT amplitudes for each pixel of the image. Aspect 84 84. The method of embodiment 83, further comprising storing a value associated with each DCT amplitude of the pixel of the image together with primitive data for the particular primitive. Aspect 85 determining the contribution of each EM field A method according to aspect 83 or 84, comprising calculating the contribution of each EM field from the particular primitive to each of the plurality of elements using the value associated with each DCT amplitude of the pixel of the image. Aspect 86 A method according to any one of aspects 81 to 85, further comprising selecting specific DCT terms to be included in the determination of the contribution of each EM field, each of the specific DCT terms comprising a respective DCT weighting greater than a predetermined threshold. Aspect 87 obtaining information about a particular primitive and an occluder of the particular primitive, wherein the particular primitive is within a plurality of primitives corresponding to an object in three-dimensional (3D) space; and determining one or more particular elements of a plurality of elements of the display that do not contribute to the reconstruction of the particular primitive due to the effect of the occluder. Aspect 88 88. The method of embodiment 87, further comprising storing the information about the particular element together with the information about the particular primitive and the occluder. Aspect 89 89. The method of claim 87 or 88, wherein the determination is performed during a rendering process of the object to obtain primitive data for the plurality of primitives. Aspect 90 A method according to any one of aspects 87 to 89, further comprising transmitting the stored information of the particular element, together with the information of the particular primitive and the occluder, to a controller configured to calculate the electromagnetic (EM) contributions of the plurality of primitives to the plurality of elements of the display. Aspect 91 A method according to any one of aspects 87 to 89, further comprising, for each of the particular elements, generating a sum of the electromagnetic (EM) field contributions from the plurality of primitives to the one of the particular elements by excluding the EM field contribution from the particular primitive to the one of the particular elements. Aspect 92 92. The method of any one of aspects 87 to 91, further comprising, for each of the plurality of elements other than the particular element, generating a respective sum of EM field contributions from the plurality of primitives to the element. Aspect 93 93. The method of any one of aspects 87 to 92, further comprising masking the EM field contribution of the particular element to the particular primitive. Aspect 94 determining the one or more particular elements connecting the particular primitive to an end point of the occlusion; extending the connection to the display and determining an intersection between the connection and the display; A method described in any one of aspects 87 to 93, comprising determining that a specific range defined by the intersection is the specific element that does not contribute to the reconstruction of the specific primitive due to the influence of the occluder. Aspect 95 obtaining information about a particular primitive and an occluder of the particular primitive, wherein the particular primitive is within a plurality of primitives corresponding to an object in three-dimensional (3D) space; determining, for each of a plurality of elements of a display, each portion of the particular primitive that does not contribute an electromagnetic (EM) field to the element due to the influence of the occlusion; A method comprising: Aspect 96 89. The method of embodiment 88, further comprising storing the information for each portion of the particular primitive together with the information for the particular primitive and the occluder. Aspect 97 97. The method of claim 95 or 96, wherein the determination is performed during a rendering process of the object to obtain primitive data for the plurality of primitives. Aspect 98 A method described in any one of aspects 95 to 97, further comprising transmitting the stored information for each portion of the specific information, together with the information for the specific primitive and the occluder, to a controller configured to calculate the electromagnetic (EM) contributions of the multiple primitives to the multiple elements of the display. Aspect 99 A method according to any one of aspects 95 to 98, further comprising masking the EM field contribution of each of the plurality of elements to the respective portion of the particular primitive. Aspect 100 A method according to any one of aspects 95 to 99, further comprising, for each of the plurality of elements, generating a sum of the EM field contributions from the plurality of primitives to the element by excluding the EM field contribution to the element from each portion of the particular primitive. Aspect 101 101. The method of claim 100, wherein generating the sum of EM field contributions from the plurality of primitives to the element comprises: subtracting the EM contribution of each portion of the particular primitive to the element from the sum of EM field contributions from the plurality of primitives to the element without the influence of the occluding body. Aspect 102 The method of aspect 100, wherein generating the sum of EM field contributions from the multiple primitives to the element includes summing EM field contributions to the element from one or more other portions of the particular primitive, wherein each of the portions and the one or more other portions form the particular primitive. Aspect 103 determining each portion of the particular primitive that does not contribute an EM field to the element due to the occlusion; connecting the element to an end point of the closure; determining an intersection between the connection and the particular primitive; A method described in any one of aspects 95 to 102, comprising determining that the specific portions of the specific primitive surrounded by the intersection are the respective portions of the specific primitive that do not contribute the EM field to the element under the influence of the occluding body. Aspect 104 obtaining respective primitive data for each of a plurality of primitives corresponding to an object in three-dimensional (3D) space; obtaining respective geometric specular information for each of the plurality of primitives; storing said respective geometric specular information together with respective primitive data for each of said plurality of primitives. Aspect 105 The respective geometric specular reflection information of each of the plurality of primitives is 105. The method of claim 104, comprising reflectivity of a surface of the primitive at a viewing angle. Aspect 106 The method of aspect 104 or 105, further comprising determining the contribution of each EM field from each of the plurality of primitives to each of a plurality of elements of the display by taking into account the geometric specular reflection information of each of the primitives. Aspect 107 obtaining graphics data including primitive data for each of a plurality of primitives corresponding to an object in three-dimensional (3D) space; determining, for each of the plurality of primitives, an electromagnetic (EM) field contribution to each of a plurality of elements of a display by calculating, in a 3D coordinate system, a propagation of the EM field from the primitive to the element; generating, for each of the plurality of elements, a sum of the EM field contributions to the element from the plurality of primitives; for each of the plurality of elements, transmitting a respective control signal to the element, wherein the control signal is present to modulate at least one characteristic of the element based on the sum of the EM field contributions to the element; and transmitting timing control signals to light emitters to activate the light emitters and illuminate the display such that the light is produced by the modulation elements of the display to form a volumetric light field corresponding to the object. Aspect 108 modifying each control signal using a predetermined calibration value for each of a plurality of elements of the display; applying each of the modified control signals to the plurality of elements of the display; measuring the incident light output on said display; evaluating the predetermined calibration value based on the measurement of the output of the light; and A method comprising: Aspect 109 109. The method of embodiment 108, wherein the pre-determined calibration value is the same for each of the plurality of elements. Aspect 110 converting the control signals of the plurality of elements by a digital-to-analog converter (DAC); modifying the respective control signals of the plurality of elements, 110. The method of embodiment 108 or 109, comprising modifying a digital signal of each control signal using the predetermined calibration value. Aspect 111 111. The method of any one of embodiments 108-110, wherein the predetermined value comprises a plurality of bits. Aspect 112 112. The method of any one of aspects 108 to 111, further comprising adjusting the pre-determined calibration value based on a result of the evaluation. Aspect 113 adjusting the predetermined calibration value; 113. The method of embodiment 112, comprising modifying values ​​of one or more of the plurality of bits. Aspect 114 adjusting the predetermined calibration value; 113. The method of embodiment 112, comprising determining a combination of values ​​for the plurality of bits based on the pre-determined calibration value and another calibration value determined from a previous evaluation. Aspect 115 Aspects 115. The method of any one of aspects 108 to 114, wherein the output of the light comprises a phase change or intensity difference of the light between the output of the light and a background. Aspect 116 A method described in any one of aspects 108 to 115, wherein each control signal for the element is determined based on the sum of electromagnetic (EM) field contributions to the element from multiple primitives corresponding to objects in 3D space. Aspect 117 For each of the plurality of elements of the display: obtaining respective sums of electromagnetic (EM) field contributions from a plurality of primitives in three-dimensional (3D) space, wherein the plurality of primitives correspond to objects in the 3D space; applying a respective mathematical transform to the respective sums of the EM field contributions of the elements to obtain respective transformed sums of the EM field contributions of the elements; determining a respective control signal based on the respective transformed sums of the EM field contributions of the elements; and modulating a characteristic of the element based on the determined respective control signals of the element. Aspect 118 directing incident light onto the plurality of elements of the display; measuring a first output of the light; A method as described in embodiment 117, further comprising adjusting one or more coefficients of the mathematical transforms of each of the plurality of elements based on the result of the measurement of the first output of the light. Aspect 119 Varying the depth of a holographic pattern corresponding to the object within the field of view of the display; measuring a second output of said light; 19. The method of claim 118, further comprising: adjusting the one or more coefficients of each mathematical transform based on the first and second outputs. Aspect 120 changing the plurality of primitives corresponding to a first holographic pattern to a second plurality of primitives corresponding to a second holographic pattern; measuring a second output of said light; 19. The method of claim 118, further comprising: adjusting the one or more coefficients of each mathematical transform based on the first and second outputs. Aspect 121 The method of embodiment 120, wherein the first holographic pattern and the second holographic pattern correspond to the object. Aspect 122 The method of embodiment 120, wherein the second holographic pattern corresponds to a second object different from the object associated with the first holographic pattern. Aspect 123 Aspects 118 to 120, wherein the first output of the light is measured by an image sensor. Aspect 124 The method of embodiment 123, wherein the image sensor is configured to use a machine vision algorithm to determine what is being displayed and calculate a goodness-of-fit parameter. Aspect 125 each of the first and second holographic patterns comprising a grid of dots; The fitness parameter is how close the points are to each other; How close to the center the point is located; and how deformed the point is. Aspect 126 126. The method of any one of aspects 117 to 125, wherein the mathematical transform is derived from Zernike polynomials. Aspect 127 127. The method of any one of embodiments 117 to 126, wherein the mathematical transform of the plurality of elements varies from element to element. Aspect 128 Reproducing a series of samples of known colors and intensities by illuminating the display; measuring the output light using a colorimeter device calibrated to the CIE standard observer curve; defining the output light of the display in a CIE XYZ color space; 128. The method of any one of embodiments 108 to 127, further comprising: Aspect 129 determining deviations of said defined light output values ​​from known standard values; Adapting the output color on the display to adjust the output color to the correct color; 129. The method of embodiment 128, further comprising: Aspect 130 determining a cell gap of a liquid crystal (LC) display based on a pitch of display elements of the LC display; and calculating a minimum value of birefringence of the LC mixture based on the cell gap of the LC display and a predetermined retardation. Aspect 131 131. The method of embodiment 130, further comprising improving the switching speed of the LC display while maintaining the birefringence of the LC mixture above the minimum value. Aspect 132 Improving the switching speed Increasing the dielectric anisotropy of the LC mixture. and reducing the rotational viscosity of the LC mixture. Aspect 133 133. The method of any one of embodiments 130 to 132, wherein the LC display comprises a liquid crystal on silicon (LCOS) device having a silicon backplane. Aspect 134 The LC display A liquid crystal layer; a transparent conductive layer on the liquid crystal layer as a common electrode; a backplane comprising a plurality of metal electrodes underlying the liquid crystal layer; Aspects 134. The method of any one of aspects 130 to 133, wherein each of the plurality of metal electrodes is isolated from one another, and the backplane is configured to control a voltage of each of the plurality of metal electrodes. Aspect 135 a backplane; a plurality of display elements on the backplane, A display wherein at least two of the plurality of display elements have different sizes. Aspect 136 136. The display of embodiment 135, wherein a larger of the at least two display elements includes a buffer, and a smaller of the at least two display elements does not include a buffer. Aspect 137 A display as described in embodiment 136, wherein the larger display element is connected to a first plurality of display elements by a conductive line, the buffer is configured to buffer the voltage applied to the conductive line so that the voltage is applied only to a second plurality of display elements within the first plurality of display elements, and the number of display elements in the second plurality is less than the number of display elements in the first plurality. Aspect 138 A display described in any one of embodiments 135 to 137, wherein the buffer comprises analog circuitry in the form of transistors or digital circuitry in the form of logic gates. Aspect 139 A display described in any one of aspects 135 to 138, wherein the distribution of sizes of the multiple display elements is substantially the same as the size of the smaller of the at least two display elements. Aspect 140 140. The display of any one of aspects 135 to 139, wherein the display is configured to be a liquid crystal on silicon (LCOS) device. Aspect 141 a backplane; a plurality of display elements on the backplane, A display wherein at least two of the plurality of display elements have different shapes. Aspect 142 the backplane includes respective circuitry for each of the display elements; A display as described in embodiment 141, wherein each of the circuits of the at least two display elements has a shape corresponding to the different shapes of the at least two display elements. Aspect 143 143. The display of embodiment 141 or 142, wherein the distribution of sizes of the plurality of display elements is substantially the same as a predetermined size. Aspect 144 143. The display of any one of aspects 141 to 142, wherein the display is configured to be a liquid crystal on silicon (LCOS) device. Aspect 145 one or more processors; a non-transitory computer-readable storage medium in communication with the one or more processors and storing instructions executable by the one or more processors that, upon execution of such instructions, cause the one or more processors to perform the method of any one of aspects 1-134. Aspect 146 135. A non-transitory computer-readable storage medium storing instructions executable by one or more processors that, when executed, cause the one or more processors to perform the method of any one of aspects 1-134. Aspect 147 a display comprising a plurality of elements; A controller coupled to the display and configured to perform the method of any one of aspects 1 to 134. Aspect 148 The system of aspect 147, wherein the controller comprises a plurality of computing units, each of the computing units configured to perform an operation on one or more primitives among a plurality of primitives corresponding to an object in three-dimensional (3D) space. Aspect 149 The system of aspect 148, wherein the controller is locally coupled to the display, and each of the computing units is coupled to one or more respective elements of the display and configured to transmit respective control signals to each of the one or more respective elements. Aspect 150 150. The system of aspect 148 or 149, wherein the computing units are configured to operate in parallel. Aspect 151 151. The system of any one of aspects 147 to 150, wherein the controller comprises at least one element selected from the group consisting of an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable gate array (PGA), a central processing unit (CPU), a graphics processing unit (GPU), and a standard computational cell. Aspect 152 A system described in any one of aspects 147 to 151, wherein the display comprises a spatial light modulator (SLM) including a digital micromirror device (DMD) or an LCOS (liquid crystal on silicon) device. Aspect 153 A system described in any one of aspects 147 to 152, wherein the display is configured to perform phase modulation, amplitude modulation, or phase modulation and amplitude modulation. Aspect 154 Aspects 147-153, wherein the controller is coupled to the display via a memory buffer. Aspect 155 Aspects 147 to 154, the system further comprising an illuminant positioned adjacent to the display and configured to emit light on the display. Aspect 156 The system of embodiment 155, wherein the light emitter is coupled to the controller and configured to be turned on / off based on a control signal from the controller. Aspect 157 The system of aspect 155 or 156, wherein the light emitter is coupled to the controller configured to control the amplitude or brightness of one or more light-emitting elements within the light emitter via a memory buffer. Aspect 158 The system of embodiment 157, wherein the memory buffer of the light emitter has a size smaller than the memory buffer of the display. Aspect 159 The system of embodiment 157 or 158, wherein the number of light-emitting elements in the light emitter is less than the number of elements in the display. Aspect 160 Aspects 157 to 159, a system described in any one of aspects 157 to 159, wherein the controller is configured to simultaneously activate the one or more light-emitting elements of the light emitter. Aspect 161 161. The system of any one of aspects 155 to 160, wherein the light emitter is a coherent light source, a semi-coherent light source, or a non-coherent light source. Aspect 162 Aspects 162. The system of any one of aspects 155 to 161, wherein the light emitter comprises two or more light-emitting elements each configured to emit light having a different color. Aspect 163 the controller is configured to successively modulate the display using information associated with a first color during a first period of time and to modulate the display using information associated with a second color during a second successive period of time; Aspects 155 to 162. The system of any one of aspects 155 to 162, wherein the controller is configured to control the light emitters to continuously turn on a first light-emitting element during the first period to emit light having the first color and to continuously turn on a second light-emitting element during the second period to emit light having the second color. Aspect 164 Aspects 155 to 161, a system described in any one of aspects 155 to 161, wherein the light emitter is configured to emit white light and the display is configured to diffract the white light into light having different colors. Aspect 165 A system described in any one of aspects 155 to 164, wherein the light emitter is positioned in front of the surface of the display and configured to emit the light onto the surface of the display at an incident angle in the range of 0 degrees to 90 degrees, and the emitted light is reflected from the surface of the display. Aspect 166 The system of embodiment 165, wherein the light emitted from the light emitter comprises collimated light. Aspect 167 The system of embodiment 165, wherein the light emitted from the light emitter comprises divergent light. Aspect 168 The system of embodiment 165, wherein the light emitted from the light emitter comprises semi-collimated light. Aspect 169 the light emitter is positioned behind a rear surface of the display and configured to emit divergent light toward the rear surface of the display; Aspects 155 to 164, a system, wherein the emitted light is transmitted through the display and out of the display from the front of the display. Aspect 170 The light emitter is a light source configured to emit the light; Aspects 155 to 164, a system comprising: a waveguide coupled to the light source and positioned adjacent to the display, the waveguide configured to receive the emitted light from the light source and guide the emitted light to the display. Aspect 171 The system of embodiment 170, wherein the light from the light source is coupled into the waveguide from a vertical cross section of the waveguide via an optical coupler. Aspect 172 The system of embodiment 170, wherein the light source and the waveguide are integrated in a planar form and positioned on the surface of the display. Aspect 173 Aspect 173. The system of any one of aspects 170 to 172, wherein the waveguide is configured to guide the light to uniformly illuminate the display. Aspect 174 A system described in any one of aspects 170 to 173, wherein the waveguide is positioned on the back surface of the display, and the light is guided to transmit through the display and diffracted out of the display from the front surface of the display. Aspect 175 The system of embodiment 174, wherein the controller is located on the back surface of the waveguide. Aspect 176 Aspects 170 to 173, a system, wherein the waveguide is positioned in front of the display, and the light is guided to be incident on the front of the display and reflected by the front. Aspect 177 a display comprising an array of elements; an integrated circuit comprising an array of computing units, each of the computing units coupled to a respective one or more elements of the display; calculating an electromagnetic (EM) field contribution from at least one primitive of a plurality of primitives to each of the arrays of elements; and generating, for each of the one or more respective elements, a respective sum of the EM field contributions from the plurality of primitives to the element. Aspect 178 each of the computing units receiving, from other computing units of the array of computing units, calculated EM field contributions from other primitives of the plurality of primitives to each of the one or more respective elements; The system of aspect 177 is configured to: for each of the one or more elements, generate the respective sums of the EM field contributions by adding the received calculated EM field contributions to the element from the other primitives. Aspect 179 The system of aspect 177 or 178, wherein each of the computing units is configured to generate, for each of the one or more elements, a respective control signal for modulating at least one characteristic of the element based on the respective sum of the EM field contribution to the element. Aspect 180 A system described in any one of aspects 177 to 179, further comprising an illuminator disposed between the integrated circuit and the display and configured to receive a control signal from the integrated circuit and illuminate the display based on the control signal, wherein the integrated circuit, the illuminator, and the display are integrated as a single unit. Aspect 181 A system described in any one of aspects 177 to 180, wherein the integrated circuit includes an accumulator configured to store an accumulation result of the calculated EM field contributions from the plurality of primitives to each of the elements of the display. Aspect 182 182. The system of embodiment 181, wherein the integrated circuit is configured to initialize the accumulator at the start of a calculation operation. Aspect 183 the integrated circuit includes a respective memory buffer for each of the elements; The system of aspect 181 or 182, wherein the integrated circuit is configured to accumulate the calculated EM field contributions from the multiple primitives to the element to obtain the respective sums of the EM field contributions as final accumulation results in each accumulator, and transfer the final accumulation results from each accumulator to the respective memory buffers of the element. Aspect 184 a computing device configured to generate data including primitive data for each of a plurality of primitives corresponding to an object in three-dimensional (3D) space; and 184. The system of any one of embodiments 147 to 183, comprising: A system configured to receive the graphics data from the computing device and process the graphics data to present the object in the 3D space. Aspect 185 The system of aspect 184, wherein the computing device comprises an application programming interface (API) configured to create the primitives including each primitive data by rendering a computer-generated (CG) model of the object.

[0286] Therefore, the description of exemplary implementations provided above does not define or constrain this specification. Other changes, substitutions, and alterations are also possible without departing from the spirit and scope of this specification. [Explanation of symbols]

[0287] 100 systems 101 OS Graphics Abstraction 102 Computer 103 Processed Scenes 104 OS 105 Primitive List 106 Applications 108 GPU 110 Holographic Display 112 Controller 114 Display 116 Luminous Object 117 Holographic Reconstruction 120 Traditional Renderer 122 Image Buffer 124 Conventional Monitor 130 Holographic Renderer 150 Holographic Display Device 151 Scene Data 152 Computing Architecture 153 Input light 154 computing cores 155 Holographic Light Field 156 displays 158 Backplane 159 Communication Connections 160 Display element 162 Luminous Object 170 Systems 172 Holographic Display Device 173 Display 174 keyboards 176 Mouse 178 2D objects 180 3D objects 200 Example Configurations for Electromagnetic (EM) Field Calculations 202 Display 204 Display element 206-point primitives 208 3D space 210 Boundary 250 coordinates 300 displays 302 Display element 304-point primitives 306 Line Primitives 308 Triangle Primitives 400 processes 402 Step 404 Step 406 Steps 408 Steps 410 steps 500 Systems 502 Computer 503 Renderer 504 Application 506 GPU 505 Video Driver 508 2D display screen 510 Controller 511 memory buffers 512 display 514 Luminous Object 516 Collimated beam 518 Holographic Light Field 520 System 521 Computer 522 Controller 523 memory buffer 524 display 526 Luminous Object 527 Semi-collimated beam 528 Holographic Light Field 530 System 531 Computer 532 Controller 533 memory buffer 534 Display 535 semi-collimated beam 536 Luminous Object 538 Holographic Light Field 540 System 541 Computer 542 Controller 544 Display 545 Light source 546 Luminous Object 547 Waveguide 548 Holographic Light Field 550 Single Unit 560 System 561 Computer 562 Controller 563 memory buffers 564 Transmissive Display 565 light source 566 Luminous Object 567 Waveguide 568 Holographic Light Field 570 System 571 Computer 572 Controller 573 Memory Buffer 574 Reflective Display 575 Light source 576 Luminous Object 577 Light emitting waveguide 578 Holographic Light Field 600 LCOS devices 602 Fasel 650 LCOS devices 651 Word Line 652 Fasel 652* Facel 653 bit lines 654 Fasel 660 buffers

Claims

1. A backplane having a plurality of circuits; a plurality of display elements on the backplane; A display comprising: the display is a phase modulation device for holographic reconstruction of objects in three-dimensional (3D) space; each of the plurality of display elements is configured to be phase modulated with a respective phase corresponding to a sum of electromagnetic (EM) field contributions to the display element from a plurality of primitives corresponding to the object, a primitive referring to a basic geometric or graphic element for input or output in a computing system, the EM field contribution from each primitive to the display element being determined based on a calculation of an EM field propagation from the primitive to the display element in a 3D coordinate system; the plurality of display elements are configured to have non-uniform or irregular sizes or shapes to form a non-periodic structure for reducing or eliminating at least one diffraction effect; A display, wherein each of the plurality of display elements of the display is coupled to a respective circuit of the plurality of circuits having a corresponding metal electrode of a plurality of metal electrodes, each of the plurality of metal electrodes being isolated from one another, and each of the plurality of circuits being provided on the display element according to a shape of the display element of the display.

2. 10. The display of claim 1, wherein at least two display elements of the plurality of display elements have an irregular polygonal shape.

3. 10. The display of claim 1, wherein at least two display elements of the plurality of display elements have at least two different polygonal shapes.

4. 10. The display of claim 1, wherein adjacent ones of the plurality of display elements have different shapes.

5. 10. The display of claim 1, wherein at least one display element of the plurality of display elements has a different shape than one or more display elements adjacent to the at least one display element.

6. 10. The display of claim 1, wherein the size distribution of the plurality of display elements is centered around a single value.

7. 10. The display of claim 1, wherein the size distribution of the plurality of display elements is related to the spatial resolution of the display.

8. 10. The display of claim 1, The display A liquid crystal layer; a transparent conductive layer on the liquid crystal layer as a common electrode; Furthermore, A display wherein the backplane includes the plurality of metal electrodes underlying the liquid crystal layer, the backplane being configured to control a voltage on each of the plurality of metal electrodes.

9. 10. The display of claim 1, A display wherein the display elements are configured to be phase and amplitude modulated.

10. 10. The display of claim 1, The display, wherein the modulation device comprises a spatial light modulator (SLM) including a digital micromirror device (DMD) or a liquid crystal on silicon (LCOS) device.

11. 10. The display of claim 1, the modulation device is a liquid crystal (LC) on silicon (LCOS) device; The LCOS device has the following characteristics: a cell gap of the LCOS device is determined based on the pitch of the display elements of the LCOS device; the minimum birefringence of the LC mixture is based on the cell gap of the LCOS device and a predetermined retardation; or the switching speed of the LCOS device is related to the birefringence of the LC mixture being above the minimum value, the dielectric anisotropy of the LC mixture, and the rotational viscosity of the LC mixture; a display having at least one of:

12. A display according to any one of claims 1 to 11; a controller coupled to the display and configured to send at least one control signal to at least one display element of the plurality of display elements to modulate at least one characteristic of the at least one display element; A system comprising:

Images