Three-dimensional display methods, displays, and systems using electromagnetic field calculations

Abstract

Methods, apparatus, devices, and systems for displaying objects in three dimensions (3D) are provided. In one aspect, a method includes: obtaining data including corresponding primitive data of primitives corresponding to the object; determining the electromagnetic field contribution of the primitives to the display elements by calculating the electromagnetic field propagation from each primitive to each display element of the display screen; generating a sum of electromagnetic field contributions of a plurality of the primitives to each display element; sending a corresponding control signal to each display element to modulate at least one characteristic of the display element based on the sum of electromagnetic field contributions; and sending a timing control signal to an illuminator to activate the illuminator thereby illuminating light onto the display screen, such that the modulated display elements of the display screen cause the light to form a volumetric light field corresponding to the object.

Description

[0001] This patent application is a divisional application of the patent application filed on January 16, 2019, with application number 2019800085091 and invention title "Three-dimensional display method using electromagnetic field calculation". Technical Field

[0002] This disclosure relates to three-dimensional (3D) displays, and more specifically to 3D displays using computational techniques. Background Technology

[0003] The development of traditional two-dimensional (2D) projection and 3D rendering has led to new approaches for 3D displays, including numerous hybrid technologies that combine head and eye tracking with traditional display devices for use in virtual reality (VR), augmented reality (AR), and mixed reality (MR). These technologies attempt to replicate the experience of holography by combining tracking and measurement-based computation to simulate stereoscopic or intraocular light fields that can be represented by actual holograms. Summary of the Invention

[0004] This disclosure describes methods, apparatus, devices, and systems for calculating electromagnetic (EM) fields for three-dimensional (3D) displays.

[0005] This disclosure provides techniques that overcome limitations present in known technologies. As an example, the techniques disclosed herein can be implemented without using bulky wearable devices such as "3D glasses." As another example, the techniques disclosed herein can be selectively implemented without being limited by: tracking mechanism accuracy, display device quality, relatively long processing times and / or relatively high computational requirements, and / or the inability to display objects to multiple viewers simultaneously. As yet another example, the techniques can be implemented without specialized tools and software, thereby enabling the development of content that extends beyond and builds upon the tools and software used in conventional 3D content creation. Various different embodiments can exhibit one or more of the foregoing advantages. For example, certain embodiments of this disclosure can produce real-time, full-color, realistic 3D images that appear to be real 3D objects in the real world and can be viewed simultaneously without obstruction by multiple viewers from different points.

[0006] One aspect of this disclosure is characterized by a method comprising: determining an electromagnetic field contribution to an element by calculating electromagnetic field propagation from the element to each of a plurality of elements of a display screen in a 3D coordinate system for each of a plurality of primitives corresponding to an object in a three-dimensional (3D) space; and generating a sum of electromagnetic field contributions from the plurality of primitives to the element for each of the plurality of elements.

[0007] The electromagnetic field contribution may include at least one selected from the group consisting of phase contribution and amplitude contribution. The primitives may include at least one selected from the group consisting of point primitives, line primitives, and polygon primitives. The primitives may include line primitives having information including at least one selected from the group consisting of gradient color, textured color, and shading effect. The primitives may also include polygon primitives having information including at least one selected from the group consisting of gradient color, textured color, and shading effect. The plurality of primitives may be indexed in a specific order.

[0008] In some embodiments, the method further includes obtaining corresponding primitive data for each of the plurality of primitives. The corresponding primitive data for each of the plurality of primitives may include corresponding color information of the primitive, and the determined electromagnetic field contribution to each of the elements includes information corresponding to the corresponding color information of the primitive. The color information may include at least one selected from the group consisting of textured color and gradient color. The corresponding primitive data for each of the plurality of primitives may include texture information of the primitive. The corresponding primitive data for each of the plurality of primitives may include shading information on one or more surfaces of the primitive. The shading information may include modulation of at least one selected from the group consisting of color on the one or more surfaces of the primitive and brightness on the one or more surfaces of the primitive.

[0009] In some implementations, the corresponding primitive data for each of the plurality of primitives includes the corresponding coordinate information of the primitive in the 3D coordinate system. The corresponding coordinate information for each of the plurality of elements in the 3D coordinate system can be determined based on the corresponding coordinate information of the plurality of primitives in the 3D coordinate system. The corresponding coordinate information for each element can correspond to a logical storage address stored in memory for that element.

[0010] Determining the electromagnetic field contribution of each of the plurality of primitives to each of the plurality of elements may include: determining at least one distance between the element and the primitive in the 3D coordinate system based on the corresponding coordinate information of the element and the corresponding coordinate information of the primitive. Determining the electromagnetic field contribution of each of the plurality of primitives to each of the plurality of elements includes: determining a first distance between the first primitive and the first element based on the corresponding coordinate information of the first primitive and the first element; and determining a second distance between the first primitive and the second element based on the first distance and the distance between the first element and the second element. The distance between the first element and the second element may be predetermined based on the pitch of the plurality of elements of the display screen.

[0011] In some examples, at least one of the plurality of primitives is a line primitive including a first endpoint and a second endpoint, and determining at least one distance between the element and the primitive includes: determining a first distance between the element and the first endpoint of the line primitive; and determining a second distance between the element and the second endpoint of the line primitive. At least one of the plurality of primitives is a triangular primitive including a first endpoint, a second endpoint, and a third endpoint, and determining at least one distance between the element and the primitive includes: determining a first distance between the element and the first endpoint of the triangular primitive; determining a second distance between the element and the second endpoint of the triangular primitive; and determining a third distance between the element and the third endpoint of the triangular primitive.

[0012] In some implementations, determining the electromagnetic field contribution of each of the plurality of primitives to each of the plurality of elements includes: determining the electromagnetic field contribution of the primitive to the element based on a predetermined expression for the primitive and the at least one distance. The predetermined expression is determined by analytically calculating the electromagnetic field propagation from the primitive to the element. The predetermined expression is determined by solving Maxwell's equations. Maxwell's equations can be solved by providing boundary conditions defined at the surface of the display screen. The boundary conditions may include Dirichlet boundary conditions or Cauchy boundary conditions. The plurality of primitives and the plurality of elements may be in the 3D space, and the surface of the display screen may form part of the boundary surface of the 3D space. The predetermined expression includes at least one selected from the group consisting of functions including sine functions, functions including cosine functions, and functions including exponential functions, and determining the electromagnetic field contribution includes: identifying the value of at least one of the functions in a table stored in memory.

[0013] In some implementations, determining the electromagnetic field contribution of each of the plurality of primitives to each of the plurality of elements, and generating a sum of the electromagnetic field contributions to each of the plurality of elements, includes: determining a first electromagnetic field contribution of the plurality of primitives to a first element among the plurality of elements, and summing the first electromagnetic field contributions for the first element; and determining a second electromagnetic field contribution of the plurality of primitives to a second element among the plurality of elements, and summing the second electromagnetic field contributions for the second element. Determining the first electromagnetic field contribution of the plurality of primitives to the first element may include performing the determination of the electromagnetic field contribution of the first primitive to the first element and the determination of the electromagnetic field contribution of the second primitive to the first element in parallel.

[0014] In some implementations, determining the electromagnetic field contribution of each of the plurality of primitives to each of the plurality of elements includes: determining a corresponding first electromagnetic field contribution of a first primitive to each of the plurality of elements; and determining a corresponding second electromagnetic field contribution of a second primitive to each of the plurality of elements. Generating the sum of the electromagnetic field contributions to each of the plurality of elements may include: accumulating the electromagnetic field contribution to the element by adding the corresponding second electromagnetic field contribution to the element to the corresponding first electromagnetic field contribution. Determining the corresponding first electromagnetic field contribution of the first primitive to each of the plurality of elements may be performed in parallel with determining the corresponding second electromagnetic field contribution of the second primitive to each of the plurality of elements.

[0015] Determining the electromagnetic field contribution of each of the plurality of primitives to each of the plurality of elements may include: determining the first electromagnetic field contribution of a first primitive to a first element and determining the second electromagnetic field contribution of a second primitive to the first element in parallel.

[0016] In some embodiments, the method further includes: generating a corresponding control signal for each of the plurality of elements based on the sum of the electromagnetic field contributions of the plurality of primitives to the element, the corresponding control signal being used to modulate at least one characteristic of the element based on the sum of the electromagnetic field contributions of the plurality of primitives to the element. The at least one characteristic of the element may include at least one selected from the group consisting of refractive index, amplitude index, birefringence, and hysteresis. The corresponding control signal may include an electrical signal, an optical signal, a magnetic signal, or an acoustic signal. In some cases, the method further includes: multiplying a scaling factor by the sum of the electromagnetic field contributions for each of the elements to obtain a scaled sum of electromagnetic field contributions, wherein the corresponding control signal is generated based on the scaled sum of electromagnetic field contributions for the element. The method may further include: normalizing the sum of electromagnetic field contributions for each of the elements, wherein the corresponding control signal is based on the normalized sum of electromagnetic field contributions for the element. The method may further include: sending the corresponding control signal to the element.

[0017] In some embodiments, the method further includes sending a control signal to an illuminator, the control signal instructing the illuminator to be turned on such that the illuminator emits light onto the display screen. The control signal may be sent in response to determining that a sum of electromagnetic field contributions for each of the plurality of elements has been obtained. The modulated elements of the display screen can cause the light to propagate in different directions to form a volumetric light field corresponding to the object in the 3D space. The volumetric light field may correspond to a solution of Maxwell's equations having boundary conditions defined by the modulated elements of the display screen. The light may include white light, and the display screen may be configured to diffract the white light into light of different colors.

[0018] In some embodiments, the method further includes representing values ​​using fixed-point representation during computation. Each value may be represented as an integer with an implicit scaling factor.

[0019] In some implementations, the method further includes performing a mathematical function using a fixed-point representation. The mathematical function may include at least one selected from a group consisting of sine, cosine, and arctangent. Performing the mathematical function may include receiving an expression in a first fixed-point format and outputting a value in a second fixed-point format with a precision level different from the first fixed-point format. Performing the mathematical function may include looking up a table for the calculation of the mathematical function, wherein the table includes at least one selected from a group consisting of a fully enumerated lookup table, an interpolation table, a polynomial function based on a half-table, and a fully min-max polynomial based on a half-table. Performing the mathematical function may include applying a dedicated range reduction to the input. Performing the mathematical function includes transforming trigonometric calculations from the range [-π, π] to a signed 2's complementary representation in the range [-1, 1].

[0020] Another aspect of this disclosure is a method comprising: obtaining primitive data of a plurality of primitives corresponding to objects in a three-dimensional (3D) space; calculating a first electromagnetic field contribution of a first primitive of the plurality of primitives to each of a plurality of elements of a display screen; and calculating a second electromagnetic field contribution of a second primitive of the plurality of primitives to each of the plurality of elements of the display screen, wherein the calculation of the first electromagnetic field contribution from the first primitive and the calculation of the second electromagnetic field contribution from the second primitive are performed at least partially in parallel.

[0021] In some implementations, calculating the first electromagnetic field contribution of the first primitive to a first element among the plurality of elements is performed in parallel with calculating the second electromagnetic field contribution of a second primitive among the plurality of primitives to the first element. The method may include calculating the respective electromagnetic field contribution of each of the plurality of primitives to each of the plurality of elements. The calculation of the respective electromagnetic field contribution may be performed without having at least one of the following: extending the geometry of the object to the plurality of elements, applying visibility testing before packaging the wavefront, and decision-making or communication between parallel computations for different primitives. Calculating the respective electromagnetic field contribution may be configured to facilitate at least one of the following: adjusting parallel computations for different primitives to achieve speed, cost, size, or energy optimization, reducing the latency between initiating rendering and preparing the results for display, using fixed-point representation to increase accuracy, and optimizing computation speed by optimizing mathematical functions.

[0022] In some implementations, the method further includes representing the value using a fixed-point representation during computation. Representing the value using the fixed-point representation can be done without having at least one of the following: anomalous floating-point operations for incremental underflow, handling non-numerical values ​​resulting from operations including division by zero, changing the floating-point rounding mode, and causing a floating-point exception in the operating system.

[0023] In some embodiments, the method further includes: for each of the plurality of elements, accumulating the electromagnetic field contribution to the element by adding the corresponding second electromagnetic field contribution to the element to the corresponding first electromagnetic field contribution to the element.

[0024] In some embodiments, the method further includes: for each of the plurality of elements, generating a corresponding control signal based on the sum of the electromagnetic field contributions of the plurality of primitives to the element, the corresponding control signal being used to modulate at least one characteristic of the element based on the sum of the electromagnetic field contributions of the plurality of primitives to the element.

[0025] In some embodiments, the method further includes scaling a first primitive adjacent to a second primitive by a predetermined factor such that the reconstruction of the first primitive does not overlap with the reconstruction of the second primitive. The predetermined factor may be determined at least in part based on the resolution of the display screen. The method may further include obtaining corresponding primitive data for each of the plurality of primitives, wherein the corresponding primitive data for each of the plurality of primitives includes corresponding coordinate information of the primitive in the 3D coordinate system; and determining new corresponding coordinate information for the first primitive based on the corresponding coordinate information of the first primitive and the predetermined factor. The method may further include determining the electromagnetic field contribution of the first primitive to each of the plurality of elements based on the new corresponding coordinate information of the first primitive. The method may further include scaling the second primitive by the predetermined factor. The first primitive and the second primitive may share a common portion, and scaling the first primitive includes scaling the common portion of the first primitive. Scaling the first primitive may include scaling the first primitive in a predetermined direction.

[0026] Another aspect of this disclosure is a method comprising: obtaining primitive data of a plurality of primitives corresponding to objects in a three-dimensional (3D) space; scaling the first primitive by a predetermined factor using corresponding primitive data of a first primitive adjacent to a second primitive and corresponding primitive data of the second primitive; and updating the corresponding primitive data of the first primitive based on the scaling result.

[0027] In some implementations, the corresponding primitive data of each of the plurality of primitives includes the corresponding coordinate information of the primitive in a 3D coordinate system, and updating the corresponding primitive data includes: determining new corresponding coordinate information of the first primitive based on the corresponding coordinate information of the first primitive and the predetermined factor.

[0028] In some implementations, the predetermined factor is determined such that the reconstruction of the first primitive in the 3D space does not overlap with the reconstruction of the second primitive.

[0029] In some implementations, the scaling is performed such that: the gap between the reconstructions of the first primitive and the second primitive in the 3D space is large enough to separate the first primitive and the second primitive to minimize the overlap effect, and the gap is small enough to make the reconstruction appear seamless.

[0030] In some implementations, the predetermined factor is determined at least in part based on the resolution of the display screen.

[0031] In some implementations, the method further includes storing updated primitive data of the first primitive in a buffer.

[0032] In some implementations, the scaling is performed during the rendering process of the object to obtain the primitive data of each of the plurality of primitives.

[0033] In some embodiments, the method further includes sending updated primitive data of the plurality of primitives to a controller, wherein the controller is configured to determine, based on the updated primitive data of the plurality of primitives, the respective electromagnetic field contribution of each of the plurality of primitives to each of the plurality of elements of the display screen.

[0034] In some implementations, the method further includes determining the electromagnetic field contribution of the first primitive to each of a plurality of elements of the display screen based on updated primitive data of the first primitive.

[0035] In some embodiments, the method further includes scaling the second primitive by the predetermined factor.

[0036] In some implementations, the first primitive and the second primitive share a common portion, and scaling the first primitive includes scaling the common portion of the first primitive.

[0037] In some implementations, scaling the first primitive includes scaling the first primitive in a predetermined direction.

[0038] In some implementations, scaling the first primitive includes scaling a first portion of the first primitive by a first predetermined factor and scaling a second portion of the second primitive by a second predetermined factor, wherein the first predetermined factor is different from the second predetermined factor.

[0039] Another aspect of this disclosure is a method comprising: obtaining a plurality of discrete cosine transform (DCT) weights of an image to be mapped onto a specified surface of a particular primitive among a plurality of primitives corresponding to an object in a three-dimensional (3D) space; and determining, by taking into account the influence of the plurality of DCT weights of the image, the respective electromagnetic field contribution of the particular primitive to each of a plurality of elements of a display screen.

[0040] In some implementations, the method further includes: determining the resolution of the image to be mapped onto a specified surface of the particular primitive; and determining the plurality of DCT weights of the image based on the resolution.

[0041] In some embodiments, the method further includes: decoding the DCT weights of the image to obtain the corresponding DCT magnitude for each pixel of the image.

[0042] In some implementations, the method further includes storing values ​​associated with the corresponding DCT amplitudes of pixels in the image, together with the primitive data of the particular primitive. Determining the corresponding electromagnetic field contribution may include calculating the corresponding electromagnetic field contribution of the particular primitive to each of the plurality of elements using values ​​associated with the corresponding DCT amplitudes of pixels in the image.

[0043] In some embodiments, the method further includes selecting specific DCT terms to be included when determining the corresponding electromagnetic field contribution, each of the specific DCT terms having a corresponding DCT weight higher than a predetermined threshold.

[0044] Another aspect of this disclosure is a method comprising: obtaining information about a given primitive and an occlusion of the given primitive, wherein the given primitive belongs to one of a plurality of primitives corresponding to an object in a three-dimensional (3D) space; and determining one or more specific elements of a plurality of elements of a display screen that do not contribute to the reconstruction of the given primitive due to the influence of the occlusion.

[0045] In some implementations, the method further includes storing information about the specific element, as well as information about the given primitive and the occlusion.

[0046] In some implementations, the determination is performed during the rendering process of the object to obtain the primitive data of the plurality of primitives.

[0047] In some embodiments, the method further includes sending stored information about the specific element, as well as information about the given primitive and the obstruction, to a controller configured to calculate the electromagnetic field contribution of the plurality of primitives to the plurality of elements of the display screen.

[0048] In some embodiments, the method further includes: for each of the specific elements, generating a sum of the electromagnetic field contributions of the plurality of primitives to the specific element by excluding the electromagnetic field contribution of the given primitive to the specific element.

[0049] In some embodiments, the method further includes: for each of the plurality of elements other than the specific element, generating the sum of the respective electromagnetic field contributions of the plurality of primitives to the element.

[0050] In some implementations, the method further includes masking the contribution of the specific element to the reconstruction of the given primitive.

[0051] In some implementations, determining the one or more specific elements includes: connecting the given primitive to an endpoint of the occlusion; extending the connection to the display screen to determine an intersection between the connection and the display screen; and identifying elements within a specific range defined by the intersection as the specific elements that do not contribute to the reconstruction of the given primitive due to the influence of the occlusion.

[0052] Another aspect of this disclosure is a method comprising: obtaining information about a given primitive and an occluder of the given primitive, wherein the given primitive belongs to one of a plurality of primitives corresponding to an object in a three-dimensional (3D) space; and for each of a plurality of elements of a display screen, determining a corresponding portion of the given primitive that does not contribute an electromagnetic field to the element due to the influence of the occluder.

[0053] In some embodiments, the method further includes storing information about the corresponding portion of the given primitive and information about the given primitive and the obstruction.

[0054] In some implementations, the determination is performed during the rendering process of the object to obtain the primitive data of the plurality of primitives.

[0055] In some embodiments, the method further includes sending to a controller information stored of the corresponding portion of the given primitive and information of the given primitive and the obstruction, the controller being configured to calculate the electromagnetic field contribution of the plurality of primitives to the plurality of elements of the display screen.

[0056] In some embodiments, the method further includes: masking the electromagnetic field contribution of the corresponding portion of the given primitive to each of the plurality of elements.

[0057] In some embodiments, the method further includes: for each of the plurality of elements, generating a sum of electromagnetic field contributions of the plurality of primitives to the element by excluding the electromagnetic field contribution of the corresponding portion of the given primitive to the element. Generating the sum of electromagnetic field contributions of the plurality of primitives to the element may include: subtracting the electromagnetic field contribution of the corresponding portion of the given primitive to the element from the sum of electromagnetic field contributions of the plurality of primitives to the element without the influence of the obstruction. Generating the sum of electromagnetic field contributions of the plurality of primitives to the element may include: summing the electromagnetic field contributions of one or more other portions of the given primitive to the element, the corresponding portion and the one or more other portions forming the given primitive.

[0058] In some implementations, determining the corresponding portion of the given primitive that does not contribute to the electromagnetic field of the element due to the influence of the obstruction includes: connecting the element to an endpoint of the obstruction; determining the intersection point between the connection and the primitive; and identifying a specific portion of the given primitive surrounded by the intersection point as the corresponding portion of the given primitive that does not contribute to the electromagnetic field of the element due to the influence of the obstruction.

[0059] Another aspect of this disclosure is a method comprising: obtaining corresponding primitive data of each of a plurality of primitives corresponding to an object in a three-dimensional (3D) space; obtaining corresponding geometric specular highlight information of each of the plurality of primitives; and storing the corresponding geometric specular highlight information and the corresponding primitive data of each of the plurality of primitives.

[0060] In some implementations, the corresponding geometric specular specular information for each of the plurality of primitives includes: the reflectivity of the surface on which the primitive is located according to the viewing angle.

[0061] In some embodiments, the method further includes determining the electromagnetic field contribution of each of the plurality of primitives to each of the plurality of elements of the display screen by taking into account the corresponding geometric specular specular information of the primitives.

[0062] Another aspect of this disclosure is a method comprising: obtaining graphic data including corresponding primitive data of a plurality of primitives corresponding to an object in three-dimensional (3D) space; determining an electromagnetic field contribution to an element for each of the plurality of primitives by calculating electromagnetic field propagation from the primitive to each of a plurality of elements of a display screen in a 3D coordinate system; generating a sum of electromagnetic field contributions of the plurality of primitives to the element for each of the plurality of elements; sending a corresponding control signal to the element for each of the plurality of elements, the control signal being used to modulate at least one characteristic of the element based on the sum of electromagnetic field contributions to the element; and sending a timing control signal to an illuminator to activate the illuminator thereby illuminating light onto the display screen such that the modulated elements of the display screen cause the light to form a volumetric light field corresponding to the object.

[0063] Another aspect of this disclosure is a method comprising: changing a corresponding control signal for each of a plurality of elements of a display screen using a predetermined calibration value; applying the changed corresponding control signal to each of the plurality of elements of the display screen; measuring the output of light incident on the display screen; and evaluating the predetermined calibration value based on the measurement of the light output.

[0064] In some implementations, the predetermined calibration value is the same for each of the plurality of elements.

[0065] In some embodiments, the method further includes: converting the respective control signals of the plurality of elements by a digital-to-analog converter (DAC), wherein changing the respective control signals of the plurality of elements includes: changing the digital signal of the respective control signal using the predetermined calibration value.

[0066] In some implementations, the predetermined calibration value includes multiple bits.

[0067] In some embodiments, the method further includes adjusting the predetermined calibration value based on the result of the evaluation. Adjusting the predetermined calibration value may include modifying one or more values ​​of the plurality of bits. Adjusting the predetermined calibration value may include determining a combination of values ​​of the plurality of bits based on the predetermined calibration value and another calibration value determined according to a previous evaluation.

[0068] In some implementations, the light output includes a phase transition of the light or an intensity difference between the light output and the background.

[0069] In some implementations, the corresponding control signal for the element is determined based on the sum of the electromagnetic field contributions of multiple primitives corresponding to objects in 3D space to the element.

[0070] Another aspect of this disclosure is a method comprising: for each of a plurality of elements of a display screen, obtaining a sum of corresponding electromagnetic field contributions from a plurality of primitives in a three-dimensional (3D) space, the plurality of primitives corresponding to objects in the 3D space; applying a corresponding mathematical transformation to the sum of the corresponding electromagnetic field contributions to the element to obtain a transformed sum of corresponding electromagnetic field contributions to the element; determining a corresponding control signal based on the transformed sum of corresponding electromagnetic field contributions to the element; and modulating characteristics of the element based on the determined corresponding control signal for the element.

[0071] In some embodiments, the method further includes: introducing light incident on the plurality of elements of the display screen; measuring a first output of the light; and adjusting one or more coefficients of a mathematical transformation of each of the plurality of elements based on the measurement of the first output of the light. The method may further include: changing the depth of a holographic pattern corresponding to the object according to the viewing angle of the display screen; measuring a second output of the light; and adjusting one or more coefficients of the corresponding mathematical transformation based on the first output and the second output. The method may further include: changing the plurality of primitives corresponding to the first holographic pattern to a plurality of second primitives corresponding to the second holographic pattern; measuring a second output of the light; and adjusting one or more coefficients of the corresponding mathematical transformation based on the first output and the second output. The first holographic pattern and the second holographic pattern may correspond to the object. The second holographic pattern may correspond to a second object, which is different from the object associated with the first holographic pattern. The first output of the light may be measured by an imaging sensor. The imaging sensor may be configured to use a machine vision algorithm to determine the content being displayed and calculate fitness parameters. Both the first holographic pattern and the second holographic pattern may include a dot grid, and wherein the fitness parameter is selected from at least one of the following: the degree of proximity of the dots, the degree of centering of the dot location, and the degree of distortion of the dots.

[0072] In some implementations, the mathematical transformation is derived from Zernike polynomials.

[0073] In some implementations, the mathematical transformations of the plurality of elements are changed element by element.

[0074] In some embodiments, the method further includes: reproducing a sample set of known colors and intensities by illuminating the display screen; measuring the output light using a colorimeter device calibrated to the International Committee for International Education (CIE) standard observation curve; and defining the output light of the display screen in the International Committee for International Education (CIE) XYZ color space. The method may also include: determining the deviation of the defined output light value from a known standard value; and adjusting the output color on the display screen to bring it back into alignment.

[0075] Another aspect of this disclosure is a method comprising: determining the cell gap of a liquid crystal display (LC) display based on the pitch of the display elements of the LC display; and calculating a minimum value of birefringence of a liquid crystal mixture based on the cell gap and a predetermined delay of the LC display.

[0076] In some embodiments, the method further includes increasing the switching speed of the liquid crystal display screen while keeping the birefringence of the liquid crystal mixture above a minimum. Increasing the switching speed may include at least one selected from the group consisting of: increasing the dielectric anisotropy of the liquid crystal mixture; and decreasing the rotational viscosity of the liquid crystal mixture.

[0077] In some embodiments, the liquid crystal display includes a liquid crystal on silicon (LCOS) device having a silicon backplane.

[0078] In some embodiments, the liquid crystal display includes: a liquid crystal layer; a transparent conductive layer located on top of the liquid crystal layer as a common electrode; and a backplate including a plurality of metal electrodes located at the bottom of the liquid crystal layer, wherein each of the plurality of metal electrodes is isolated from each other, and the backplate is configured to control the voltage of each of the plurality of metal electrodes.

[0079] Another aspect of this disclosure is a display screen comprising: a back panel; and a plurality of display elements on the back panel, wherein at least two of the plurality of display elements have different sizes.

[0080] In some embodiments, the larger of the at least two display elements includes a buffer, and the smaller of the at least two display elements does not include a buffer. The larger display element can be connected to a first number of display elements via wires, and the buffer is configured to buffer voltage applied to the wires such that the voltage is applied only to a second number of display elements within the first number of display elements, the number of which is less than the number of the first number of display elements.

[0081] In some implementations, the buffer comprises analog circuitry in the form of transistors or digital circuitry in the form of logic gates.

[0082] In some implementations, the size distribution of the plurality of display elements is substantially equal to the size of the smaller of the at least two display elements.

[0083] In some embodiments, the display screen is configured as a liquid crystal on silicon device.

[0084] Another aspect of this disclosure is a display screen comprising: a back panel; and a plurality of display elements on the back panel, wherein at least two of the plurality of display elements have different shapes.

[0085] In some embodiments, the backplate includes corresponding circuitry for each display element, and the shape of the corresponding circuitry for each of the at least two display elements corresponds to the different shapes of the at least two display elements.

[0086] In some implementations, the size distribution of the plurality of display elements is substantially equal to a predetermined size.

[0087] In some embodiments, the display screen is configured as a liquid crystal on silicon device.

[0088] Another aspect of this disclosure is a method comprising: obtaining graphic data including corresponding primitive data of a plurality of primitives corresponding to an object in three-dimensional (3D) space; determining an electromagnetic field contribution to an element for each of the plurality of primitives by calculating electromagnetic field propagation from the primitive to each of a plurality of elements of a display screen in a 3D coordinate system; generating a sum of electromagnetic field contributions of the plurality of primitives to the element for each of the plurality of elements; sending a corresponding control signal to the element for each of the plurality of elements, the control signal being used to modulate at least one characteristic of the element based on the sum of electromagnetic field contributions to the element; and sending a timing control signal to an illuminator to activate the illuminator thereby illuminating light onto the display screen such that the modulated elements of the display screen cause the light to form a volumetric light field corresponding to the object.

[0089] Other embodiments of the aforementioned aspects include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, all configured to perform the actions of the method. The configuration of one or more computer systems to perform a specific operation or action means that software, firmware, hardware, or a combination thereof are installed on the system that, when executed, causes the system to perform the operation or action. The configuration of one or more computer programs to perform a specific operation or action means that the one or more programs include instructions that, when executed by a data processing apparatus, cause the apparatus to perform the operation or action.

[0090] Another aspect of this disclosure is a device comprising: one or more processors; and a non-transitory computer-readable storage medium communicating with the one or more processors and storing instructions executable by the one or more processors, wherein the instructions, when executed, cause the one or more processors to perform one or more methods disclosed herein.

[0091] Another aspect of this disclosure is a non-transitory computer-readable storage medium for storing instructions that can be executed by one or more processors and, when executed, cause the one or more processors to perform one or more methods disclosed herein.

[0092] Another aspect of this disclosure features a display screen comprising multiple elements; and a controller coupled to the display screen and configured to perform one or more methods disclosed herein. The controller may include multiple computing units, each configured to operate on one or more primitives of a plurality of primitives corresponding to objects in three-dimensional (3D) space. In some embodiments, the controller is locally coupled to the display screen, and each computing unit is coupled to one or more corresponding elements of the display screen and configured to send a corresponding control signal to each of the one or more corresponding elements. The computing units may be configured to operate in parallel.

[0093] The controller may include at least one selected from the group consisting of an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a programmable gate array (PGA), a central processing unit (CPU), a graphics processing unit (GPU), and a standard computing unit. The display screen may include a spatial light modulator (SLM), which includes a digital micromirror device (DMD) or a liquid crystal on silicon (LCOS) device. The display screen may be configured for phase modulation, amplitude modulation, or both phase and amplitude modulation. The controller may be coupled to the display screen via a storage buffer.

[0094] In some embodiments, the system includes an illuminator arranged adjacent to the display screen and configured to emit light onto the display screen. The illuminator is coupled to the controller and configured to be turned on / off based on a control signal from the controller.

[0095] In some cases, the illuminator is coupled to the controller via a storage buffer configured to control the amplitude or brightness of one or more light-emitting elements in the illuminator. The size of the storage buffer for the illuminator may be smaller than the size of the storage buffer for the display screen. The number of light-emitting elements in the illuminator may be less than the number of elements in the display screen. The controller may be configured to simultaneously activate the one or more light-emitting elements of the illuminator.

[0096] The illuminator can be a coherent light source, a semi-coherent light source, or an incoherent light source. In some embodiments, the illuminator is configured to emit white light, and the display screen is configured to diffract the white light into light of different colors. In some embodiments, the illuminator includes two or more light-emitting elements, each configured to emit light of different colors. The controller can be configured to sequentially modulate the display screen using information associated with a first color during a first time period and information associated with a second color during a sequential second time period, and the controller can be configured to control the illuminator to sequentially activate the first light-emitting element to emit light of the first color during the first time period and activate the second light-emitting element to emit light of the second color during the second time period.

[0097] In some embodiments, the illuminator is disposed in front of the surface of the display screen and configured to emit light onto the surface of the display screen at an angle of incidence ranging from 0 degrees to 90 degrees, and the emitted light is reflected from the surface of the display screen. In some cases, the light emitted from the illuminator comprises collimated light. In some cases, the light emitted from the illuminator comprises divergent light. The light emitted by the illuminator comprises semi-collimated light.

[0098] In some embodiments, the illuminator is disposed behind the rear surface of the display screen and configured to emit divergent light onto the rear surface of the display screen, and the emitted light is transmitted through the display screen and exits the display screen from the front surface of the display screen.

[0099] In some embodiments, the illuminator includes: a light source configured to emit the light; and a waveguide coupled to the light source and disposed adjacent to the display screen, the waveguide being configured to receive light emitted from the light source and guide the emitted light to the display screen. In some cases, light from the light source is coupled to the waveguide from a side cross-section via an optical coupler. In some cases, the light source and the waveguide are integrated in a planar form and positioned on the surface of the display screen. The waveguide may be configured to guide the light to uniformly illuminate the display screen.

[0100] In some cases, the waveguide is positioned on the rear surface of the display screen, and the light is guided to transmit through the display screen and diffract out of the display screen from the front surface. The controller may be positioned on the rear surface of the waveguide. In some cases, the waveguide is positioned on the front surface of the display screen, and the light is guided to be incident on the front surface of the display screen and reflected by the front surface.

[0101] Another aspect of this disclosure features a system comprising: a display screen including an array of elements; and an integrated circuit including an array of computing units, each computing unit being coupled to one or more corresponding elements of the display screen and configured to: calculate the electromagnetic field contribution of at least one of a plurality of primitives to each element in the array of elements; and, for each of the one or more corresponding elements, generate a sum of the respective electromagnetic field contributions of the plurality of primitives to the element.

[0102] Each computing unit can be configured to: receive, from other computing units in the computing unit array, the electromagnetic field contribution of the other primitives among the plurality of primitives to each of the one or more corresponding elements; and, for each of the one or more corresponding elements, generate a sum of the corresponding electromagnetic field contributions by adding the received electromagnetic field contributions of the other primitives to the element.

[0103] Each of the computing units can be configured to generate a corresponding control signal for each of the one or more corresponding elements to modulate at least one characteristic of the element based on the sum of the corresponding electromagnetic field contributions to the element.

[0104] In some embodiments, the integrated circuit includes a corresponding accumulator configured to store the accumulated electromagnetic field contributions of the plurality of primitives to each element of the display screen. The integrated circuit may be configured to clear the accumulator at the start of a calculation operation. In some examples, the integrated circuit includes a corresponding storage buffer for each element, and the integrated circuit may be configured to accumulate the calculated electromagnetic field contributions of the plurality of primitives to the element to obtain a sum of the corresponding electromagnetic field contributions as a final accumulation result in the corresponding accumulator, and to pass the final accumulation result from the corresponding accumulator to the corresponding storage buffer for the element.

[0105] In some embodiments, the system further includes an illuminator positioned between the integrated circuit and the display screen and configured to receive control signals from the integrated circuit and illuminate the display screen based on the control signals, wherein the integrated circuit, the illuminator, and the display screen may be integrated into a single unit.

[0106] Another aspect of this disclosure features a system comprising: a computing device configured to generate data including corresponding primitive data of a plurality of primitives corresponding to an object in a three-dimensional (3D) space; and a system as disclosed herein. The system is configured to receive the graphics data from the computing device and process the graphics data to render the object in the 3D space. The computing device may include an application programming interface (API) configured to create the primitives using the corresponding primitive data by rendering a computer-generated (CG) model of the object.

[0107] In this disclosure, the term "primitive" refers to a fundamental, indivisible element within a computing system used for input and output. This element can be a geometric or graphical element. The term "hologram" refers to a pattern displayed on a screen that contains amplitude or phase information about an object, or a combination thereof. The term "holographic reconstruction" refers to a volumetric light field (e.g., a holographic light field) from an illuminated screen.

[0108] Details of one or more embodiments of the subject matter herein are set forth in the accompanying drawings and related description. Other features, aspects, and advantages of the subject matter will become apparent from the specification, drawings, and claims.

[0109] It should be understood that various aspects of the implementation can be combined in different ways. As an example, features of some methods can be combined with features of other methods. Attached Figure Description

[0110] Figure 1A A schematic diagram of an exemplary system including a holographic display is shown.

[0111] Figure 1B A schematic diagram of an exemplary holographic display is shown.

[0112] Figure 1C An exemplary system for 3D display is shown.

[0113] Figure 2 An exemplary configuration for electromagnetic (EM) propagation calculations is shown.

[0114] Figure 3A An exemplary electromagnetic propagation of a point primitive relative to a display element is shown.

[0115] Figure 3B An exemplary electromagnetic propagation of a line element relative to a display element is shown.

[0116] Figure 3C An exemplary electromagnetic propagation of a triangular primitive relative to a display element is shown.

[0117] Figure 3D An exemplary implementation of Maxwellian holographic occlusion, in which point primitives are used as occluders, is shown.

[0118] Figure 3E An exemplary implementation of Maxwellian holographic occlusion, in which a line primitive is occluded by another line primitive, is shown.

[0119] Figure 3F An exemplary implementation of Maxwellian holographic occlusion, in which triangular primitives are used as occluders, is shown.

[0120] Figure 3G An exemplary implementation of Maxwell holographic bonding is shown.

[0121] Figure 4 This is a flowchart illustrating an exemplary process for processing 3D objects.

[0122] Figures 5A to 5F An implementation of an exemplary system for 3D display is shown.

[0123] Figure 6A An exemplary display screen with display elements having a non-uniform shape is shown.

[0124] Figure 6B An exemplary display screen with display elements of different sizes is shown. Detailed Implementation

[0125] The embodiments of this disclosure are characterized by a technique for realizing the 3D display of complex computer-generated scenes as realistic holograms. The technique provides a novel and deterministic solution to a real-time dynamic computational holography based on Maxwell's equations for electromagnetic fields, which can be represented as Maxwell's holography. The computation in Maxwell's holography can be represented as Maxwell's holographic computation. In embodiments, this disclosure utilizes tools including field theory, topology, analytical continuation, and / or symmetry groups to treat holograms as Dirichlet or Cauchy boundary condition problems for general electric fields, enabling real-time solving of holograms without the limitations of conventional holographic systems. In embodiments, the technique can be used with spatial light modulators (SLMs) or any other holographic device to create phase-only, amplitude-only, or phase and amplitude holograms.

[0126] Implementations of this disclosure provide: 1) a mechanism for approximating holograms as electromagnetic boundary conditions using field theory and contact geometry instead of classical optics; 2) computer code and application programming interfaces (APIs) for deriving and implementing methods for computational holography of electromagnetic boundary conditions, i.e., implementing hologram computation as a 2D analytic function of the hologram plane and subsequently discretizing it into a parallel algorithm; and / or 3) a complete set of full 3D holographic versions of standard computer graphics primitives (e.g., points, lines, triangles, and textured triangles), enabling full compatibility with standard existing computer graphics tools and techniques. These techniques enable devices to display existing, general-purpose content not specifically created for holography, while simultaneously allowing existing content creators to create holographic works without having to learn special techniques or use special tools.

[0127] Specifically, the technique may involve using mathematical formulas (or expressions) for light as electromagnetic (EM) phenomena instead of the mathematical formulas for classical optics commonly used in computational holography, such as the Gerchberg-Saxton (GS) model. The mathematical formulas disclosed herein are derived from Maxwell's equations. In embodiments, the technique disclosed herein involves treating the displayed image as an electromagnetic field and the hologram as boundary value conditions (e.g., the Dirichlet problem) that generate the electromagnetic field. Additionally, the desired image can be constructed using primitive paradigms prevalent in computer graphics, allowing, for example, the technique to be used to display any 3D image as a holographic reconstruction (e.g., a holographic light field) rather than a projected image on a 2D screen. Compared to bandwidth-limited depth point cloud techniques, the technique avoids these limitations and uses any suitable type of primitive, such as point primitives, line primitives, or polygon primitives (e.g., triangle primitives). Furthermore, color information, texture information, and / or shading information can be utilized to render the primitives. This facilitates the recording and compression schemes for CG holographic content, including live holographic video.

[0128] In an embodiment, the technique uses Maxwell's equations to compute the generated hologram as a boundary condition problem modeled by electromagnetic fields. This eliminates the dependence on Fast Fourier Transform (FFT) and its inherent limitations, eliminates the dependence on aligned light sources and lasers, and / or eliminates the limitations of previous methods and nondeterministic solutions for computational holography.

[0129] In this embodiment, the technique can be optimized for computational simplicity and speed through mathematical optimization processes based on the parameters of the computer-generated (CG) primitives required to construct the scene. These mathematical optimization processes constrain independent inputs to the surface of the hologram. This allows the work to be performed in a highly parallel and optimal manner within computational architectures such as application-specific integrated circuits (ASICs) and multi-core architectures. The processing of the computational hologram can be considered as a single instruction executed on the input data in the form of a computer-generated image (CGI) scene, and theoretically can be completed within a single clock cycle of each CGI primitive.

[0130] In this embodiment, the technique treats the holographic scene as a component of fully 3D holographic primitive apertures, which are functionally compatible with standard primitives of conventional 3D graphics, such as those used in video games, movies, television, computer displays, or any other computational display technology. The technique enables the efficient implementation of these aperture primitives in both hardware and software without the limitations inherent in standard implementations of computational holography. The amplitude and color of the primitives can be automatically calculated. The computational complexity increases linearly with the number of phase elements n, compared to n^2 or n*log(n) in standard computational holography. The created image is fully 3D and not a collection of planar images, and the technique does not require iterative amplitude correction with an unknown number of steps. Furthermore, the generated hologram does not have a “conjugate” image that occupies space on a holographic device.

[0131] Because holographic primitives are part of a special set of mathematical objects, they can be computed relatively simply and quickly, and they are uniquely suited to parallel, distributed computing methods. Computability and parallelism allow for interactive computation of large holograms, enabling the design of theoretically infinitely large-area holographic devices that can function as holographic computer displays, telephone displays, home theaters, and even holographic rooms. Furthermore, holograms can be filled with light to fill large areas, for example, rendering large shading areas in 3D, without the limitations associated with conventional holographic computation methods, which cause elements to be rendered as outlines rather than solids. Moreover, the relatively simple and fast computation allows for the display of real-time holograms at interactive speeds unconstrained by n^2 computational loads and iteration amplitude corrections.

[0132] In embodiments, the technology enables natural computability on modern ASICs and multi-core architectures, and achieves full compatibility with modern graphics hardware, modern graphics software, and / or modern graphics tools and toolchains. For example, the technology enables a clear and simple holographic API, and allows the use of common standard 3D content creation tools (e.g., ...) via the API. The API (or Unity3D) enables high-performance rendering of arbitrary CG models. It allows developers or users to interact with holographic devices (e.g., light modulators or holographic systems). The Holographic API can create computer graphics primitives as discrete holographic scene primitives, allowing the generation of rich holographic content using general-purpose and specially designed holographic computing hardware. The creation of mathematical and computational architectures allows the rendering of holograms using tools and techniques used for creating regular 3D content and software applications. Optimization of the mathematical and computational architectures allows executable implementations of regular graphics and rendering to be displayed as holographic reconstructions.

[0133] The algorithms in this technology are relatively easy to implement in hardware. This not only allows users to achieve the computational speed required for high-quality, modern rendering, but also allows the algorithms to be implemented in relatively simple circuits (e.g., ASIC gate structures as part of a holographic device). Consequently, the bandwidth issues that plague high-density displays may become irrelevant because scene computation can be distributed across a computational architecture built into the display device (e.g., built-in computing), rather than having to be computed remotely and then written to every pixel of the display for each frame of content. This also means that the number of display elements and therefore the size of the holographic display can be relatively unconstrained by other technologies.

[0134] The technology enables a variety of interactive technologies using structured light to be implemented relatively simply and inexpensively in different applications, including, for example, solid-state light detection and ranging (LiDAR) devices, 3D printing, smart illuminators, smart microdisplays, or any other application requiring structured light. The technology can also be used for optical simulations, such as for grating simulations.

[0135] Figure 1AA schematic diagram of an exemplary system 100 for 3D display is shown. System 100 includes a computing device 102 and a holographic display device (or Maxwell holographic display device) 110. The computing device 102 is configured to prepare data for a list of primitives corresponding to an object, such as a 3D object, and transmit the data to the holographic display device 110 via a wired or wireless connection (e.g., a USB-C connection or any other high-speed serial connection). The holographic display device 110 is configured to calculate the electromagnetic field contribution of the list of primitives to display elements (e.g., modulators) of the display screen in the holographic display device 110, modulate the display elements using a pattern, such as a hologram, based on the calculated electromagnetic field contribution to the display screen, and display the light field corresponding to the object in 3D form under illumination, such as holographic reconstruction. Here, a hologram refers to a pattern displayed on the display screen that contains amplitude information or phase information about the object, or a combination thereof. Holographic reconstruction refers to the volumetric light field (e.g., a holographic light field) from the display screen when the holographic screen is illuminated.

[0136] Computing device 102 may be any suitable type of device, such as a desktop computer, personal computer, notebook computer, tablet computing device, personal digital assistant (PDA), network device, smart mobile phone, smartwatch, enhanced general packet radio service (EGPRS) mobile phone, media player, navigation device, email device, game console, or any suitable combination of any two or more of these computing devices or other computing devices.

[0137] Computing device 102 includes an operating system (OS) 104, which may include multiple applications 106 as a graphics engine. Applications 106 may use standard 3D content creation tools (e.g., The scene can be processed or rendered using a device such as Unity3D, for example, any arbitrary CG model. The scene may correspond to a 3D object. Application 106 can operate in parallel to render the scene to obtain OS graphics abstraction 101, which can be provided to graphics processing unit (GPU) 108 for further processing. In some embodiments, OS graphics abstraction 101 is provided to holographic display device 110 for further processing.

[0138] GPU 108 may include dedicated electronic circuitry designed for rapid manipulation of computer graphics and image processing. GPU 108 may process an OS graphics abstraction 101 of a scene to obtain processed scene data 103, which can be used to obtain a primitive list 105 indexed in a specific order. Primitives may include at least one of point primitives, line primitives, or polygon primitives. In some embodiments, GPU 108 includes a video driver configured to generate the processed scene data 103 and the primitive list 105.

[0139] In some implementations, GPU 108 includes a conventional renderer 120 that renders the primitive list 105 into a list of items for drawing on a conventional monitor 124, such as a 2D display screen, using conventional rendering techniques such as culling and pruning. The list of items can be sent to the conventional monitor 124 via a screen buffer 122.

[0140] In some implementations, GPU 108 includes a holographic renderer 130 for rendering the primitive list 105 into graphical data to be displayed by the holographic display device 110. The graphical data may include the primitive list and corresponding primitive data. For example, the graphical data may include the hexadecimal code for each primitive.

[0141] In some implementations, GPU 108 includes both conventional renderer 120 and holographic renderer 130. In some implementations, GPU 108 includes conventional renderer 120, and holographic display device 110 includes holographic renderer 130.

[0142] The corresponding primitive data may also include color information (e.g., textured color, gradient color, or both), texture information, and / or shading information. Shading information can be obtained through any conventional CGI surface shading method that involves modulating the color or brightness of the primitive surface.

[0143] The primitive data of a primitive may include the coordinate information of the primitive in a 3D coordinate system, such as a Cartesian coordinate system (XYZ), a polar coordinate system, a cylindrical coordinate system, and a spherical coordinate system. As further detailed below, the display elements in the holographic display device 110 may also have corresponding coordinate information in a 3D coordinate system. A primitive at a coordinate position may represent a 3D object adjacent to a display element (e.g., in front of the display element).

[0144] As an example, a primitive is a shading line, for instance, a straight line that smoothly changes from one color to another across its span. A primitive requires four data elements for rendering: two endpoints and color information (e.g., RGB color values) at each endpoint. Assuming the line's hexadecimal code is A0, in a 3D coordinate system, the line extends from a first endpoint (0.1, 0.1, 0.1) to a second endpoint (0.2, 0.2, 0.2), with the first endpoint being half blue: RGB = (0, 0, 128), and the second endpoint being full red: RGB = (255, 0, 0). The holographic renderer determines how much data and what type of data is expected for each primitive. For the line, the primitive data for the shading line in the primitive stream can be the following set of instructions:

[0145] 0xA0 / / hex code for the shaded line

[0146] 0x3dcccccd / / first vertex at(0.1,0.1,0.1)float(single)

[0147] 0x3dcccccd

[0148] 0x3dcccccd

[0149] 0x000080 / / first vertex color is(0,0,128)

[0150] 0x3e4ccccd / / second vertex at(0.2,0.2,0.2)float(single)

[0151] 0x3e4ccccd

[0152] 0x3e4ccccd

[0153] 0xff0000 / / second vertex color is(255,0,0)

[0154] The primitive data of the colored line primitives contains a total of 31 hexadecimal words. Therefore, this is an extremely efficient way to transmit complex scenes and allows for further compression of the primitive data. Because each primitive is a deterministic Turing step, no terminator is required. Unlike traditional models that simply draw line primitives on a 2D display, the line primitive data is sent to a holographic display device 110, which can calculate a hologram and display the corresponding holographic reconstruction of the line as if it were floating in space.

[0155] In some embodiments, computing device 102 sends non-primitive data, such as recorded light field video, to holographic display device 110. Holographic display device 110 may compute sequential holograms to display the video as a spatially sequential holographic reconstruction. In some embodiments, computing device 102 simultaneously sends CG holographic content and live holographic content to holographic display device 110. Holographic display device 110 may also compute corresponding holograms to display the content as the corresponding holographic reconstruction.

[0156] like Figure 1AAs shown, the holographic display device 110 includes a controller 112 and a display screen 114. The controller 112 may include multiple computing units or processing units. In some embodiments, the controller 112 includes an ASIC, a field-programmable gate array (FPGA), or a GPU, or any combination thereof. In some embodiments, the controller 112 includes a holographic renderer 130 for rendering the primitive list 105 into graphical data to be computed by the computing units. In some embodiments, the controller 112 receives an OS graphics abstraction 101 from the computing device 102 for further processing. The display screen 114 may include multiple display elements. In some embodiments, the display screen 114 includes a spatial light modulator (SLM). The SLM may be a phase SLM, an amplitude SLM, or a phase and amplitude SLM. In some examples, the display screen 114 is a digital micromirror device (DMD) or a liquid crystal on silicon (LCOS) device. In some embodiments, the holographic display device 110 includes an illuminator 116 adjacent to the display screen 114 and configured to emit light toward the display screen 114. The illuminator 116 can be a coherent light source, such as a laser, a semi-coherent light source, such as an LED (light-emitting diode), or an incoherent light source.

[0157] Unlike conventional 3D graphics systems that employ a 3D scene and project it onto a 2D display device, the holographic display device 110 is configured to produce 3D output, such as a holographic reconstruction 117 in the form of a light field, for example, a color 3D volume. In the hologram, each display element contributes to each part of the scene. That is, for the holographic display device 110, each display element needs to be modulated for each part of the scene (e.g., each primitive in the list of primitives generated by the GPU 108) for a complete reproduction of the scene. In some embodiments, modulation of certain display elements may be omitted based on, for example, an acceptable level of accuracy in reproducing the scene.

[0158] In some implementations, controller 112 is configured to calculate the electromagnetic field contribution (e.g., phase, amplitude, or both) of each primitive to each display element, and to generate a list of primitives for each display element summing the electromagnetic field contributions to that display element. This can be done either by iterating through each primitive for a given display element and accruing its contribution to that given display element, or by iterating through each display element for each primitive.

[0159] The controller 112 can calculate the electromagnetic field contribution of each primitive to each display element based on a predetermined expression of the primitives. Different primitives can have corresponding expressions. In some cases, the predetermined expression is analytical, as shown below regarding... Figures 3A to 3CFurther details are provided. In some cases, a predetermined expression is determined by solving Maxwell's equations using boundary conditions defined at display screen 114. Boundary conditions may include Dirichlet boundary conditions or Cauchy boundary conditions. The display element can then be modulated based on the sum of electromagnetic field contributions, for example, by modulating at least one of the following: the refractive index, amplitude index, birefringence, or hysteresis of the display element.

[0160] If the value of the electromagnetic field at every point on the surface defining the field (e.g., a solution to Maxwell's equations) is known, then the exact, unique configuration of the electromagnetic field within the volume defined by the boundary surface can be determined. The list of primitives (or a holographic reconstruction corresponding to the hologram) and display screen 114 define a 3D space, and the surface of display screen 114 forms part of the boundary surface of the 3D space. The boundary conditions of the electromagnetic field can be determined by setting the electromagnetic field states (e.g., phase or phase and amplitude states) on the surface of display screen 114, for example, by illuminating the display surface with light. Due to the time symmetry of Maxwell's equations, when the display elements are modulated based on the electromagnetic field contributions from the primitives corresponding to the hologram, a volumetric light field corresponding to the hologram as a holographic reconstruction can be obtained.

[0161] For example, line elements illuminated with a specific color can be placed in front of the display screen 114. The following is for... Figure 3B To elaborate further, the analytical expression for the linear aperture can be written as a function in space. The electromagnetic field contribution from the line primitives on the boundary surface including display screen 114 can then be determined. If the electromagnetic field value corresponding to the calculated electromagnetic field contribution is set on display screen 114, the same linear aperture used in the calculation can appear at the corresponding location, for example, at the coordinate position of the line primitive in the 3D coordinate system, due to the time symmetry of Maxwell's equations.

[0162] In some examples, the following applies to Figure 3BTo elaborate further, suppose there exists a line of light between two points A and B in 3D space. The light is emitted uniformly and has an intensity of I per line length l. At each differential dl along the line from A to B, an amount of light proportional to I*dl is emitted. Infinitesimal dl can act as delta (point) sources, and the electromagnetic field contribution of the differential dl to any point on the boundary surface surrounding the scene corresponding to the primitive list can be determined. Therefore, for any display element on display screen 114, an analytical equation representing the electromagnetic field contribution of the differential segment of the line to the display element can be determined. A specific type of summation / integration can be determined as an expression for the electromagnetic field contribution of traveling along the line and accumulating the entire line to the electromagnetic field at the display element on the display screen. The value corresponding to this expression at the display element can be set, for example, by modulating the display element and illuminating it. Then, by time reversal and correction constants, the line can be created at the same location defined by points A and B in 3D space.

[0163] In some implementations, controller 112 is coupled to display screen 114 via a storage buffer. Control signal 112 may generate a corresponding control signal based on the sum of the electromagnetic field contributions to each display element. The control signal is used to modulate the display element based on the sum of the electromagnetic field contributions. The corresponding control signal is sent to the corresponding display element via the storage buffer.

[0164] In some embodiments, the controller 112 is integrated with and locally coupled to the display 114. (See also: Regarding...) Figure 1B As further detailed, the controller 112 may include a plurality of computing units, each coupled to one or more corresponding display elements and configured to send corresponding control signals to each of the one or more corresponding display elements. Each computing unit may be configured to perform calculations on one or more primitives in a primitive list. The computing units may operate in parallel.

[0165] In some implementations, the illuminator 116 is coupled to the controller 112 and configured to turn on / off based on a control signal from the controller 112. For example, the controller 112 may activate the illuminator 116 to turn on in response to the controller 112 completing a calculation (e.g., obtaining the sum of all electromagnetic field contributions to the display elements). As described above, when the illuminator 116 emits light onto the display screen 114, the modulated display elements cause the light to propagate in different directions to form a volumetric light field corresponding to a list of primitives, where the list of primitives corresponds to 3D objects. The resulting volumetric light field corresponds to a solution to Maxwell's equations with boundary conditions defined by the modulated display elements of the display screen 114.

[0166] In some embodiments, controller 112 is coupled to illuminator 116 via a storage buffer. The storage buffer can be configured to control the amplitude or brightness of the light-emitting elements in the illuminator. The storage buffer for illuminator 116 can be smaller in size than the storage buffer for display screen 114. The number of light-emitting elements in illuminator 116 can be less than the number of display elements in display screen 114, as long as light from the light-emitting elements can illuminate the entire surface of display screen 114. For example, an illuminator with 64x64 OLEDs (organic light-emitting diodes) can be used in a display screen with 1024x1024 elements. Controller 112 can be configured to simultaneously activate multiple light-emitting elements of illuminator 116.

[0167] In some embodiments, illuminator 116 is a monochromatic light source configured to emit monochromatic light, such as red, green, or blue light. In some embodiments, illuminator 116 includes two or more light-emitting elements, each configured to emit light of a different color. For example, illuminator 116 may include red, green, and blue light-emitting elements. To display a full-color 3D object, three separate red, green, and blue holograms can be calculated. That is, the three electromagnetic field contributions of the respective primitives to the display elements can be obtained. The display elements can be modulated sequentially based on these three electromagnetic field contributions, and illuminator 116 can be controlled to sequentially turn on the red, green, and blue light-emitting elements. Relying on the temporal coherence of visual effects in the viewer's eye, the three colors can be combined in the eye to present full color. In some cases, illuminator 116 is turned off during state changes of the displayed image (or holographic reconstruction) and turned on while the effective image (or holographic reconstruction) is presented for a period of time. This can also rely on the temporal coherence of vision to make the image (or holographic reconstruction) appear stable.

[0168] In some embodiments, the display screen 114 has a resolution small enough to diffract visible light, for example, a resolution on the order of 0.5 μm or smaller. The illuminator 116 may include a single white light source, and the emitted white light may be diffracted by the display screen 114 into different colors for use in holographic reconstruction.

[0169] The following is for Figures 5A to 5FAs further detailed, different configurations for system 100 may exist. Display screen 114 may be reflective or transmissive. Display screen 114 may have various sizes, ranging from small scales (e.g., 1-10 cm per side) to large scales (e.g., 100-1000 cm per side). Illumination from illuminator 116 may originate from the front surface of display screen 114 (e.g., for a reflective display screen) or from the rear surface of display screen 114 (e.g., for a transmissive display screen). Planar waveguides may be used to uniformly illuminate the surface of display screen 114. In some embodiments, controller 112, illuminator 116, and display screen 114 may be integrated together into a single unit. The integrated single unit may include, for example, a holographic renderer 130 in controller 112.

[0170] Figure 1B A schematic diagram of an exemplary holographic display device 150 is shown. The holographic display device 150 may be similar to... Figure 1A The holographic display device 110. The holographic display device 150 includes a computing architecture 152 and a display screen 156. The computing architecture 152 may be similar to... Figure 1A The controller 112. The computing architecture 152 may include an array of parallel computing cores 154. The computing cores may be connected to adjacent computing cores via a communication connection 159 (e.g., a USB-C connection or any other high-speed serial connection). The communication connection 159 may be included in a data distribution network through which scene data 151 (e.g., scene primitives) can be distributed among the computing cores 154.

[0171] Display 156 can be similar to Figure 1A The display screen 114 may include an array of display elements 160 located on a backplane 158. The display elements 160 may be arranged on the front surface of the backplane 158, and the computing core 154 may be arranged on the rear surface of the backplane 158. The backplane 158 may be a substrate, such as a wafer. The computing core 154 may be on the same substrate as the display screen 156 or connected to the rear surface of the display screen 156.

[0172] Each computing core 154 may be connected to a corresponding tile (or array) of display element 160. Each computing core 154 is configured to perform calculations on various primitives among a plurality of primitives in scene data 151 in parallel with each other. In some examples, computing core 154 is configured to calculate the electromagnetic field contribution of each corresponding primitive to each display element in the array of display elements 160, and generate a sum of electromagnetic field contributions of the plurality of primitives to each display element in the corresponding tile of display element 160. Computing core 154 may receive, from other computing cores in the array of computing cores 154, the electromagnetic field contributions of other primitives among the plurality of primitives to each display element in the corresponding tile of display element 160, and generate a sum of electromagnetic field contributions based on the received calculated electromagnetic field contributions. Computing core 154 may generate control signals for each display element in the corresponding tile of display element 160 to modulate at least one characteristic of each display element in the corresponding tile of display element 160 based on the sum of electromagnetic field contributions to the display element.

[0173] As described above, the computing architecture 152 may also generate a control signal for the illuminator 162, for example, in response to the completion of a calculation determining the sum of the electromagnetic field contributions of the plurality of primitives to each display element. The illuminator 162 emits input light 153 to illuminate the modulated display element 160, and the input light 153 is reflected by the modulated display element 160 to form a volumetric light field, such as a holographic light field 155, corresponding to the scene data 151.

[0174] like Figure 1B As shown, blocks of display element 160 can be interconnected to form a larger display screen. Correspondingly, computation cores 154 can be interconnected for data communication and distribution. Note that the parameter that varies in the holographic computation between any two given display elements is their physical location. Therefore, the task of computing the hologram can be evenly distributed among the respective computation cores 154, and the entire display 150 can operate at the same speed as a single block, i.e., regardless of the number of blocks.

[0175] Figure 1C An exemplary system 170 for displaying objects in 3D space is shown. System 170 may include, for example... Figure 1A The computing device 102 and the holographic display device 172, for example, Figure 1A Holographic display 110 or Figure 1B A holographic display 150. A user can operate the system 170 using an input device (e.g., a keyboard 174 and / or a mouse 176). For example, a user can create CG models for 2D objects 178 and 3D objects 180 using a computing device. The computing device or holographic display device 172 may include a holographic renderer, for example, Figure 1AThe holographic renderer 130 renders CG models to generate corresponding graphical data for 2D objects 178 and 3D objects 180. The graphical data may include corresponding primitive data of the primitive list corresponding to objects 178 and 180.

[0176] The holographic display device 172 may include a controller, such as Figure 1A Controller 112 or Figure 1B Controller 152, and display screen 173, for example Figure 1A The display screen 114 or Figure 1B The display screen 156. The controller can calculate the sum of the corresponding electromagnetic field contributions of the primitives to each display element of the display screen 173, and generate a control signal for modulating each display element based on the sum of the corresponding electromagnetic field contributions. The holographic display device 172 may also include an illuminator, for example, Figure 1A Lighting fixture 116 or Figure 1B Illuminator 162. A controller can generate a timing control signal to activate the illuminator. When light from the illuminator illuminates the surface of display screen 173, modulated display elements can cause the light to propagate in 3D space to form volumetric light fields corresponding to the holographic reconstructions of 2D object 178 and 3D object 180. Therefore, 2D object 178 and 3D object 180 are displayed as corresponding holographic reconstructions floating in 3D space in front of display screen 173.

[0177] In some embodiments, the computing device sends non-primitive-based data (e.g., recorded light field video) to the holographic display device 172. The holographic display device 172 can calculate and generate corresponding holograms (e.g., a series of sequential holograms) to display as corresponding holographic reconstructions in 3D space. In some embodiments, the computing device sends CG holographic content and live holographic content simultaneously to the holographic display device 172. The holographic display device 172 can also calculate and generate corresponding holograms to display the content as corresponding holographic reconstructions in 3D space.

[0178] Figure 2An exemplary configuration 200 for electromagnetic (EM) field calculation is shown. A display screen 202 (e.g., an LCOS device) comprising an array of display elements 204 and a list of primitives comprising point primitives 206 are shown in a 3D space 208. The 3D space 208 includes a boundary surface 210. In a 3D coordinate system XYZ, the point primitives 206 have coordinate information (x, y, z). Each display element 204 lies in a plane relative to other display elements 204 and has a 2D position (u, v). The display elements 204 also have a position in 3D space. Through mathematical point transformation, the 2D position (u, v) can be transformed into six coordinates 250 in the 3D coordinate system. That is, the surface of the display screen 202 forms part of the boundary surface 210. Therefore, the contribution of the list of primitives to the electromagnetic field of the display elements, calculated by defining the boundary conditions at the surface of the display screen 202, represents a portion of the total electromagnetic field contribution of the primitives to the display elements. A scaling factor (e.g., six) can be multiplied by the sum of electromagnetic field contributions to each display element to obtain a scaled sum of electromagnetic field contributions, and the display elements can be modulated based on the scaled sum of electromagnetic field contributions.

[0179] Exemplary electromagnetic field contribution of the primitive

[0180] Primitives can be used in standard computer graphics rendering. Each type of primitive in standard computer graphics corresponds to a discrete mathematical function in this process, which defines a single holographic primitive added to the graphic elements of the hologram. Each type of primitive can correspond to an expression used to calculate the contribution of the electromagnetic field to the display elements. Primitives can be point primitives, line primitives, or polygonal (e.g., triangle) primitives. As shown below, the analytical expression can be derived by calculating the electromagnetic field propagation from the corresponding primitive to the display elements on the screen.

[0181] Figure 3A An exemplary electromagnetic propagation from point primitive 304 to display element 302 on display screen 300 is illustrated. In the 3D coordinate system XYZ, it is assumed that the z-coordinate is 0 on display screen 300, meaning that the z-value is negative behind display screen 300 and positive in front of display screen 300. The coordinates of point primitive 304 are (x, y, z), and the coordinates of display element 302 are (u, v, 0). The distance d between point primitive 304 and display element 302 is... uv They can be determined based on their coordinates.

[0182] Point element 304 can be considered as a point charge with a time-varying amplitude. According to electromagnetic theory, the electric field E generated by this point charge can be expressed as:

[0183]

[0184] Where λ represents the wavelength of the electromagnetic wave, and d represents the distance from the point charge.

[0185] Therefore, the electric field E at element (u,v) is displayed. u,v It can be represented as:

[0186]

[0187] Where I represents the relative intensity of the holographic primitive electric field contributed by the dot primitive 304 at the display element.

[0188] As described above with reference to Figure 1, the surface of the display screen 300 forms only a portion of the boundary surface of the electromagnetic field. A scaling factor can be applied to the electric field E. u,v To obtain the scaled electric field at the display element, adjusted for partial boundaries.

[0189]

[0190] in

[0191] Figure 3B An example of electromagnetic propagation from line element 306 to display element 302 on display screen 300 is shown in a 3D coordinate system XYZ. As described above, the coordinates of display element 302 can be (u, v, 0), where z = 0. The coordinates of the two endpoints P0 and P1 of line element 306 are (x0, y0, z0) and (x1, y1, z1), respectively. The distance d0 between endpoint P0 and the display element can be determined based on their coordinates. Similarly, the distance d1 between endpoint P1 and the display element can be determined based on their coordinates. The distance d between the two endpoints P0 and P1... 01 It can also be determined, for example, d 01 =d1-d0.

[0192] As described above, line primitives can be viewed as superpositions or linear deformations, and the corresponding analytical expression for a line primitive as a linear aperture can be obtained as a distributed incremental function in space. This analytical expression can be a closed expression of continuous 3D line segments of a hologram.

[0193] Figure 3C An exemplary electromagnetic propagation from a triangular primitive 308 to a display element 302 on a display screen 300 is illustrated in a 3D coordinate system XYZ. As described above, the coordinates of the display element 302 can be (u, v, 0), where z = 0. The triangular primitive 308 has three endpoints: P0(x0, y0, z0), P1(x1, y1, z1), and P2(x2, y2, z2). The distances d0, d1, and d2 between the display element and the endpoints P0, P1, and P2 can be determined based on their respective coordinates.

[0194] Similar to Figure 3BIn the diagram, the line primitives and triangular primitives can be considered as continuous apertures in space, and the analytical expression for the electromagnetic field contribution of the triangular primitives to the display elements can be obtained through integration. This can be simplified to obtain an expression for efficient computation.

[0195] Exemplary computation of primitives

[0196] As described above, for example, the controller 112 of Figure 1 can calculate the electromagnetic field contribution of the primitives to the display elements based on an analytical expression that can be determined as shown above. For example, the electromagnetic field contribution of the line primitives is calculated as follows.

[0197] Each display element in a display screen has a physical location in space, and each display element lies in a plane relative to other display elements. Assuming the display elements and their controllers are arranged as is customary in display and storage devices, a simple mathematical point transformation can be used to transform the logical location of a given display element into its actual physical location in space, based on its logical memory address in the processor. Therefore, as the logical memory address of a display element cycles through the processor's logical memory space, the corresponding actual physical location in the surface space of the display screen can be identified.

[0198] As an example, if the display's pitch is 5 μm, each logical address increment can move 5 μm along the x-direction, and when the display's x-resolution limit is reached, the next increment will move back to the initial x-physical position and increase the y-physical position by 5 μm. It can be assumed that the third spatial coordinate z is zero on the display surface, meaning negative z values ​​are behind the display and positive z values ​​are in front of the display.

[0199] To begin the line calculation, a scaled physical distance between the currently displayed element and each of the two points of the line primitive can be determined as d0 and d1. In fact, d0 and d1 can be calculated once for each primitive, since each subsequent calculation of the distance for all displayed elements is a small variation of the initial values. In this way, the calculation is performed in one dimension.

[0200] An example computational process for each primitive may include the following computational code:

[0201] DD = f(d1, d0),

[0202] iscale = SS * COLOR * Alpha1,

[0203] C1=-2*iscale*sin(DD / 2)*sin((Alpha2)*cos(Alpha3),

[0204] C2=-2*iscale*sin(DD / 2)*sin(Alpha2)*sin(Alpha4),

[0205] Here, SS, Alpha1, Alpha2, Alpha3, and Alpha4 are pre-calculated constants, COLOR is the RGB color value passed along with the primitives, and all values ​​are scalar, single-precision floating-point numbers. Both sine and cosine functions can be looked up in a table stored in the controller to improve computational efficiency.

[0206] Then, at each display element, for example in the accumulator used for the display element, the results of C1 and C2 are accumulated for each primitive, and can be normalized once at the end of the calculation for the display element. At this time, as described above, the controller can send a first control signal to the display element to modulate the display element based on the calculated results, and send a second control signal to the illuminator to turn on the illuminator to emit light. Therefore, the holographic reconstruction (or holographic light field) is visible to the viewer. When the modulated display element is illuminated, light can produce clear, continuous lines of color in three-dimensional space.

[0207] In some implementations, the calculation code includes hexadecimal codes for clearing previously accumulated values ​​from the accumulator, for example, at the beginning of the code. The calculation code may also include, for example, hexadecimal codes at the end of the code for storing the accumulator result into a corresponding storage buffer for each display element. In some implementations, for example... Figure 1A The computing device 102 sends multiple background or static primitive hexadecimal codes to a controller when the application starts or during intervals between displays of frames without affecting the primary display frame rate. Subsequently, the computing device may potentially send one or more combinations of the hexadecimal codes, along with other foreground or dynamic primitives, to the controller at a higher rate, which can then generate corresponding control signals to modulate the display elements of the screen.

[0208] The computational processing is orders of magnitude simpler and faster than the most efficient line drawing routines in traditional 2D display technologies. Furthermore, the computational algorithm scales linearly with the number of display elements. Therefore, scaling the controller's computing unit to a 2D networked processing system can keep pace with the computational demands of an increasing display surface area.

[0209] Exemplary Computational Implementation

[0210] For example, Figure 1A The Maxwell holographic controller of controller 112 can calculate the electromagnetic field contribution of the primitives to the display elements based on an analytical expression that can be determined as shown above. The controller can be implemented in, for example, an ASIC, FPGA, or GPU, or any combination thereof.

[0211] In modern GPU pipelines, the GPU describes the geometry and vertex and fragment shading procedures to produce color and depth pixel outputs to one or more output image surfaces (called render targets). This process involves an explosive fan-out of information, where the geometry is expanded into shading fragments, followed by a visibility test to select whether work needs to be done on each of these fragments. A fragment is a record containing all the information involved in shading that sample point (e.g., the centroid coordinates of a triangle, interpolations such as color or texture coordinates, surface derivatives, etc.). The visibility test creates these records and then rejects those that do not contribute to the final image. Fragments that pass the visibility test are encapsulated into workgroups called wavefronts or warps, which are executed in parallel by the shader engine. These produce output values ​​that are written back to memory as pixel values, ready for display, or used as input textures for later rendering passes.

[0212] In Maxwell's holography, rendering processes can be significantly simplified. In Maxwell's holographic computation, each primitive contributes to each display element. There is no need to extend geometry to pixels, and no need to apply visibility tests before packing wavefronts. This also eliminates the need for decision-making or communication between Maxwell's holographic pipelines and allows computation to become a parallel problem with multiple possible solutions, each tailored for speed, cost, size, or energy optimization. The graphics pipeline is significantly shorter, with fewer intermediate steps, no data copying or movement, and fewer decisions, resulting in lower latency between initiating drawing and the result being ready for display. This allows Maxwell's holographic rendering to create displays with extremely low latency. As described below, this allows Maxwell's holographic computation to improve accuracy, for example, by using the number of fixed points in the Maxwell's holographic pipeline, and to optimize computational speed, for example, by optimizing mathematical functions.

[0213] Use fixed point numbers

[0214] When calculating the electromagnetic field contribution of each primitive at each display element (e.g., phase unit), intermediate calculations involve generating very large values. These large values ​​require special handling because they also need to retain decimal parts during calculation.

[0215] Floating-point values ​​have the following drawbacks: they are most accurate close to the origin (zero on the number line) and lose one bit of precision for every two powers as they move away from the origin. For values ​​close to the range [-1, 1], floating-point values ​​can be highly precise, but once values ​​reach tens of millions, such as a single-precision 32-bit IEEE-754 floating-point value where no decimal places remain, the entire significant digit (also called the mantissa) is used to represent the integer part of the value. However, Maxwell's holography is particularly interested in preserving the fractional part of large values.

[0216] In some cases, fixed-point numbers are used in Maxwell's holographic calculations. A fixed-point number representation is a numerical value where the decimal point does not change regardless of the value. By choosing the correct number of bits for the integer and fractional parts of the value, the same number of fractional bits can be obtained regardless of the value's magnitude. A fixed-point number representation is an integer with an implicit scaling factor; for example, in a 16-bit fixed-point value with 8 fractional bits, 14.375 can be represented as the value 3680 (negative binary is 000011100100000). This can also be represented as an "unsigned 16.8" fixed-point number, or simply u16.8. Negative numbers can have an additional sign bit and be stored in a "2s compliment" format. This significantly improves the accuracy of the calculation.

[0217] Optimization of mathematical functions

[0218] As shown above, Maxwell's holographic computation involves the use of a priori mathematical functions, such as sine, cosine, arctangent, etc. In a CPU, these functions are implemented as floating-point library functions that can be used with dedicated CPU instructions, or as floating-point units in a GPU. These functions are written to take an argument as a floating-point number and return the result in the same floating-point representation. These functions are constructed such that, in general, they are accurate, correctly rounded, and handle every edge case in the floating-point representation (+ / - infinity, non-numeric, signed zero, and anomalous floating-point numbers).

[0219] In Maxwell's holographic computation, fixed-point representation eliminates the need for anomalous floating-point numbers for asymptotic underflow, handles non-numerical operations such as division by zero, alters floating-point rounding modes, and prevents floating-point exceptions in the operating system. All of this allows for simplification (and / or optimization) of prior mathematical functions, for example, as described below.

[0220] In some cases, optimizations can be made to obtain the independent variable in a fixed-point format and return that value to different levels of precision; for example, an input of 28.12 and an output of 15.14. This may be particularly desirable when calculating sines of tens of millions of values, where the input independent variable can be large, but the output may only need to represent the range [-1, 1], or take any value but return the arctangent of the value in the range [-π / 2, π / 2].

[0221] In some cases, depending on the range of inputs involved, optimizations can be made to freely implement the prior function as a fully enumerated lookup table, interpolation table, or a semi-table-based polynomial function or a semi-table-based fully minimax polynomial. This optimization also allows for the application of range-specific scaling methods for larger inputs, methods that general-purpose GPU pipeline computations might skip for speed reasons.

[0222] In some cases, another optimization could be to transform the trigonometric calculations from the range [-π,π] to a signed 2's expression in the range [-1,1], which has the advantage of not requiring the costly modulo division by 2π.

[0223] Exemplary implementation of occlusion

[0224] Occlusion is generally considered a significant challenge in computer graphics, and even more so in computational holography. This is because, while the occlusion problem is static in at least some cases in projected CGI, what is hidden and what is visible in a holographic system depends on the viewer's position and orientation. Wave methods, such as GS holography or its derivatives, have been developed to address holographic occlusion. However, in GS methods, masking or blocking contributions from parts located behind other parts of the scene can be extremely complex and computationally expensive.

[0225] In Maxwell's holography, the occlusion problem can be solved relatively easily because which display elements (e.g., phase units) correspond to which primitives is completely deterministic and subtle. For example, when performing calculations for a given primitive, it can be determined whether a given display element contributes to the reconstruction of that primitive. After determining that several display elements do not contribute to a given primitive due to occlusion, the electromagnetic field contribution from that given primitive is omitted from the calculation of the sum of electromagnetic field contributions to one of the several display elements when calculating the sum of electromagnetic field contributions to one of the several display elements.

[0226] For illustrative purposes only. Figures 3D to 3F This illustrates the case where a given primitive is not targeted when it is determined to be an online primitive as an occlusion. Figure 3D The point in Figure 3E The lines in Figure 3FThe display element (the triangle in the diagram) contributes to the display. The starting point of the line element is O1, and the ending point is O2.

[0227] like Figure 3D As shown, point element P0 is behind the obstruction and closer to the display screen. By extending the line connecting O1-P0 and O2-P0, the range D1 to D2 of display elements in the display screen that do not contribute to the reconstruction of point element P0 is determined.

[0228] In some examples, the coordinate information of O1, O2, and P0 is known, for example, stored in the GPU (e.g., Figure 1A The GPU 108) sends the scene to the Maxwell holographic controller (e.g., Figure 1A The coordinates are calculated in the "Z" buffer of the controller 112. For example, in the XZ plane where y = 0, the coordinate information could be O1(Ox1, Oz1), O2(Ox2, Oz2), and PO(Px, Pz), where Oz1 = Oz2 = Oz. Based on this coordinate information, the coordinate information of D1 and D2 can be determined as follows:

[0229] Dx1= Px + ρ (Px - Ox2), Dx2= Dx1 + ρ (Ox2 - Ox1) (4),

[0230] Where ρ=Pz / (Oz-Pz), and Dz1=Dz2=0.

[0231] In addition to the information in the Z buffer used for point primitive P0, the information of D1 and D2 can also be stored as additional information in the "S" buffer used for the Maxwell holographic controller. In this way, the additional information can be used to simply mask the contribution of a particular display element (within the range from D1 to D2) to a particular primitive P0 in the indexed list of primitives.

[0232] Figure 3E This illustrates how a specific display element contributes to a line element when an obstruction is in front of it. By connecting the specific display element D0 to the start point O1 and end point O2 of the obstruction, two point elements P1 and P2 on the line element are identified as intersection points. Therefore, the specific display element D0 contributes nothing to the reconstruction of the line element portion from P1 to P2. Consequently, when calculating the sum of the electromagnetic field contributions to the specific display element D0, the electromagnetic field contribution from the P1-P2 portion of the line element is not calculated.

[0233] This can be achieved in two ways. In the first way, by considering occlusion from obstructions, the electromagnetic field contributions of the P0-P1 and P2-Pn portions to a specific display element D0 are added together as the electromagnetic field contribution of the line primitive to the specific display element D0. In the second way, the electromagnetic field contribution from the entire line primitive P0-Pn and the electromagnetic field contribution from the P1-P2 portion are calculated. By considering occlusion from obstructions, the difference between the two calculated electromagnetic field contributions can be regarded as the electromagnetic field contribution of the line primitive to the specific display element D0. The coordinate information of the P1 and P2 or P1-P2 portions can be used as line primitive portions that do not contribute to the specific display element D0, and are stored in the "S" buffer of the Maxwell holographic controller along with the information of the obstructions and other information in the GPU's "Z" buffer.

[0234] Figure 3F This illustrates how a specific display element contributes to a triangle primitive when an occlusion is in front of it. By connecting the specific display element D0 to the start point O1 and end point O2 of the occlusion, the four point primitives P1, P2, P3, and P4 on the edge of the triangle primitive are determined as intersection points. Therefore, the specific display element D0 contributes to the points P1, P2, P3, P4, and P5 on the edge of the triangle primitive. C The reconstruction of the enclosed portion makes no contribution. Therefore, when calculating the sum of the electromagnetic field contributions to a specific display element D0, P1-P2-P3-P4-P from the triangular primitives are not calculated. C Partial electromagnetic field contribution. That is, by considering the obstruction of the obstruction, only the future free point P... A The first triangle formed by P1 and P2 and the triangle formed by point P B The electromagnetic field contributions of the second triangle formed by P3 and P4 are added together and used as the basic element P of the triangle. A -P B -P C The electromagnetic field contribution. P1, P2, P3, and P4, or triangular primitive P A -P1-P2 and P B The coordinate information of -P3-P4 can be used as the basic element P of a triangle. A -P B -P C The portion that contributes to a specific display element D0, along with information about the occlusion and other information in the GPU's "Z" buffer, is stored in the Maxwell holographic controller's "S" buffer.

[0235] The occlusion implementation in Maxwell's holography allows for the transformation of the "Z" buffer in the GPU into an "S" buffer in the Maxwell holographic controller, masking the contribution of specific primitives (or specific portions of primitives) in an indexed list of primitives to a particular display element. This not only provides accurate, physically correct occlusion but also saves computation time, as primitives that do not contribute to a given display element continue to be used for calculations for the next display element. The "S" buffer may contain additional information related to the diffraction efficiency of the display.

[0236] The “S” buffer can also include rendering features such as holographic specular highlights, where the surface reflectivity depends on the viewing angle. In traditional CGI, specular highlights depend only on the orientation of the rendered object, but in a Maxwell holographic background, the viewing direction also plays a role. Therefore, geometric specular information can be encoded in the “S” buffer as an additive (specular) contribution rather than a subtractive (occlusion) contribution. In Maxwell holography, the mathematical operations for holographic specular highlights are essentially the same as those for holographic occlusion.

[0237] Exemplary implementation of the connection

[0238] When light shines on a display screen modulated with electromagnetic field contributions from a list of primitives of a 3D object, the modulation causes the light to propagate in different directions to form a volumetric light field corresponding to each primitive. This volumetric light field is a Maxwell holographic reconstruction. Two adjacent primitives in a 3D object (e.g., a triangle primitive) share an edge. During reconstruction, a joining problem may arise where the light intensity of the shared edge may double due to the individual reconstruction of the two adjacent primitives. This can affect the appearance of the reconstructed 3D object.

[0239] To solve the joining problem in Maxwell's holography, such as Figure 3G As shown, adjacent primitives can be scaled down by a predetermined factor to create gaps between them. In some cases, instead of scaling both adjacent primitives, only one primitive or a portion of a primitive is scaled down. For example, lines in a triangle primitive can be scaled down to separate them from another triangle primitive. In some cases, scaling may involve scaling different portions of a primitive using different predetermined factors. Scaling can be designed such that the gaps are large enough to minimize the stitching problem between adjacent primitives and small enough to allow the reconstructed 3D object to appear seamlessly. The predetermined factor can be determined based on information about the display (e.g., the display's maximum spatial resolution).

[0240] In some cases, scaling operations can be applied from a holographic renderer (e.g., Figure 1A The holographic renderer 130) obtains the primitive data of the primitives and sends the scaled primitive data of the primitives to the Maxwell holographic controller, for example, Figure 1AThe controller 112. In some cases, the controller may perform scaling operations on the primitive data obtained from the holographic renderer before calculating the electromagnetic field contribution of the primitives to the display elements of the display screen.

[0241] Exemplary implementation of texture mapping

[0242] Texture mapping is a technique developed in computer graphics. The basic idea is to capture a source image and apply it as a decal to a surface in a CGI system, allowing details to be rendered into the scene without adding complex geometry. Texture mapping can include techniques for creating realistic lighting and surface effects in CGI systems, and can broadly refer to applying surface data to a triangular mesh.

[0243] In Maxwell's holography, an analytical relationship between arbitrary triangles in space and a phase map on a holographic device can be used to render a planar shaded and interpolated triangular mesh in real 3D. However, compatibility with modern rendering engines involves the ability to map information onto the surfaces of these triangles. This can present practical problems because the speed of the method stems from the existence of analytical mapping, which does not allow for data-driven amplitude variations.

[0244] Discrete Cosine Transform (DCT) is an image compression technique and can be considered a real-valued version of the Fast Fourier Transform (FFT). DCT relies on an encoder-decoder process that assigns weights to the cosine harmonics in a given image. The encoded result is a set of weights, where the number of weights equals the number of pixels in the original image, and no information is lost if each weight is used to reconstruct the image. However, in many images, acceptable reconstruction can be achieved based on a small subset of the weights, resulting in a large compression ratio.

[0245] Decoding (rendering) DCT in two dimensions involves a weighted floating-point double sum for each DCT weight and each target pixel. This can be applied to Maxwell's holography for texture mapping. In Maxwell's holography, triangle rendering involves a "spiky" double integral in phase space to determine the phase contribution of any individual phase to the triangle in question. The integral is converted into a floating-point sum that reflects the integral in the DCT reconstruction, and then the analytic triangle expression is re-derived based on the DCT weights. This implementation of DCT techniques in Maxwell's holographic computation enables the rendering of complete texture-mapped triangles, thereby applying image compression to the rendered texture triangle data and leveraging the advantages of existing toolsets for automatically compressing texture and image data using DCT / JPEG.

[0246] In some implementations, to render a Maxwell holographic textured triangle, the required spatial resolution for mapping on a specified surface is first calculated. A texture with that resolution is then provided, and a DCT compressed using angle and origin information is obtained to correctly position the triangle on it. A list of the triangle's angles and DCT weights is then included in an indexed primitive list and sent to the Maxwell holographic controller. The DCT weights may be included in the electromagnetic field contribution of the triangle primitives to each display element. Textured triangles can be rendered n times slower than planar triangles, where n is the number of (non-zero) DCT weights sent with the primitives. Modern techniques for "fragment shading" can be implemented in Maxwell holographic systems, where the DCT encoding step replaces the filtering step used for conventional projective rendering.

[0247] As an example, the following expression shows the DCT weights B of an image. pq :

[0248]

[0249] in, M and N are the corners of the rectangular image, and (p,q) is the DCT term.

[0250] The amplitude value Amn can be obtained through decoding as follows:

[0251]

[0252] in

[0253] When calculating the electromagnetic field contribution of textured triangle primitives to display elements (e.g., phase cells), with corresponding DCT weights... The DCT term may be included in the following calculations:

[0254]

[0255] Where X and Y are the angles of the triangle in the coordinate system, and T corresponds to the electromagnetic field contribution of the triangle primitive to the displayed element. For non-zero terms B in DCT pq The contribution of this component is considered. The number of (p,q)DCT terms can be selected by taking into account both information loss and information compression during reconstruction.

[0256] Exemplary processing

[0257] Figure 4 This is a flowchart of an exemplary process 400 for displaying a 3D object. Process 400 can be executed by a controller for the display screen. The controller may be... Figure 1A Controller 112 or Figure 1BThe controller 152. The display screen may be the display screen 114 of Figure 1A or... Figure 1B The display screen is 156.

[0258] Obtain data (402) including the primitive data of multiple primitives corresponding to objects in 3D space. This data can be obtained from, for example... Figure 1A The computing device 102 obtains the data. The computing device can process the scene to generate primitives corresponding to objects. The computing device may include a renderer to generate primitive data of the primitives. In some implementations, the controller itself generates the data, for example, by rendering the scene.

[0259] Primitives may include at least one of point primitives, line primitives, or polygon primitives. The list of primitives is indexed in a specific order, for example, to reconstruct objects. Primitive data may include color information with textured and / or gradient colors. For example, line primitives may have gradient and / or textured colors. Polygon primitives may also have gradient and / or textured colors. Primitive data may also include texture information of the primitives and / or shading information on one or more surfaces of the primitives (e.g., triangles). Shading information may include modulation of color and / or brightness on one or more surfaces of the primitives. Primitive data may also include the corresponding coordinate information of the primitives in a 3D coordinate system.

[0260] The display screen may include multiple display elements, and the controller may include multiple computing units. The coordinates of each display element in the 3D coordinate system can be determined based on the corresponding coordinates of the primitive list in the 3D coordinate system. For example, the distance between the display screen and the object corresponding to the primitive can be predetermined. Based on the predetermined distance and the coordinates of the primitives, the coordinates of the display elements can be determined. The corresponding coordinates of each display element can correspond to a logical memory address stored in memory for that display element. In this way, when the controller cycles through the logical memory addresses for display elements in the controller's logical memory space, the corresponding actual physical location of the display element in that space can be identified.

[0261] The electromagnetic field contribution of the plurality of primitives to the display element is determined by calculating the electromagnetic field propagation from each of the plurality of primitives to each display element in a 3D coordinate system (404). The electromagnetic field contribution may include phase contribution and / or amplitude contribution.

[0262] As mentioned above Figures 3A to 3CAs shown, at least one distance between a primitive and a display element can be determined based on the corresponding coordinate information of the display element and the corresponding coordinate information of the primitive. In some cases, for each primitive, the at least one distance may be calculated or calculated only once. For example, the controller may determine a first distance between a first primitive and a first element in the display elements based on the corresponding coordinate information of the first primitive and the corresponding coordinate information of the first element, and determine a second distance between the first primitive and a second element in the display elements based on the first distance and the distance between the first element and the second element. The distance between the first element and the second element may be predetermined based on the pitch of multiple elements of the display screen.

[0263] The controller can determine the electromagnetic field contribution of the primitive to the display element based on a predetermined expression of the primitive and the at least one distance. In some cases, as described above, for... Figures 3A to 3C As shown, a predetermined expression can be determined analytically by calculating the electromagnetic field propagation from the primitives to the display elements. In some cases, the predetermined expression is determined by solving Maxwell's equations. Specifically, Maxwell's equations can be solved by providing boundary conditions defined at the surface of the display screen. Boundary conditions may include Dirichlet boundary conditions or Cauchy boundary conditions. The primitives and display elements are in 3D space, and the surface of the display screen forms part of the boundary surface of the 3D space. The predetermined expression may include at least one of a sine function, a cosine function, and an exponential function. During calculation, the controller can identify the value of at least one function in a table stored in memory, which can improve calculation speed. The controller can determine the electromagnetic field contribution to each display element for each primitive in a way that the determination of the first electromagnetic field contribution of the first primitive to the display element is performed in parallel with the determination of the second electromagnetic field contribution of the second primitive to the display element.

[0264] For each display element, generate the sum of the electromagnetic field contributions of the primitive list to that display element (406).

[0265] In some implementations, the controller determines a first electromagnetic field contribution of multiple primitives to a first display element and sums the first electromagnetic field contributions for the first display element; and determines a second electromagnetic field contribution of multiple primitives to a second display element and sums the second electromagnetic field contributions for the second display element. The controller may include multiple computing units. The controller may perform the determination of the electromagnetic field contribution of the first primitives to the first element by a first computing unit and the determination of the electromagnetic field contribution of the second primitives to the first element by a second computing unit in parallel.

[0266] In some implementations, the controller determines a first electromagnetic field contribution of a first primitive to each display element and a second electromagnetic field contribution of a second primitive to each display element. The controller then accumulates the electromagnetic field contribution to the display element by adding the corresponding second electromagnetic field contribution to the corresponding first electromagnetic field contribution. Specifically, the controller can perform the determination of the first electromagnetic field contribution of the first primitive to each display element using a first computing unit and the determination of the second electromagnetic field contribution of the second primitive to each display element using a second computing unit in parallel.

[0267] A first control signal is sent to the display screen for modulating at least one characteristic (408) of the display element based on the sum of the electromagnetic field contributions to each display element. The at least one characteristic of the display element includes at least one of refractive index, amplitude index, birefringence, or hysteresis.

[0268] The controller can generate a corresponding control signal for each display element based on the sum of the electromagnetic field contributions of multiple primitives to that display element. The corresponding control signal is used to modulate at least one characteristic of the element based on the sum of the electromagnetic field contributions of the multiple primitives to that element. That is, the first control signal includes a corresponding control signal for the display element.

[0269] In some examples, the display is controlled by electrical signals. Therefore, the corresponding control signals can be electrical signals. For example, an LCOS display includes an array of tiny electrodes where voltage is individually controlled as element intensity. An LCOS display may be filled with a birefringent liquid crystal (LC) formulation whose refractive index is varied. Thus, the corresponding control signals from the controller can control the relative refractive index of each display element, thereby controlling the relative phase of the light passing through the display.

[0270] As described above, the display surface forms part of the boundary surface. The controller can multiply a scaling factor by the sum of the electromagnetic field contributions for each display element to obtain a scaled sum of electromagnetic field contributions, and generate a corresponding control signal based on the scaled sum of electromagnetic field contributions for that display element. In some cases, the controller can normalize the sum of electromagnetic field contributions for, for example, each of all display elements, and generate a corresponding control signal based on the normalized sum of electromagnetic field contributions for that display element.

[0271] A second control signal is sent to the illuminator to turn it on so that light shines onto the modulated display screen (410). In response to determining that the sum of electromagnetic field contributions for each display element has been obtained, the controller can generate and send the second control signal. Due to time symmetry (or energy conservation), the modulated display elements of the display screen can cause light to propagate in different directions to form a volumetric light field corresponding to an object in 3D space. The volumetric light field can correspond to a solution to Maxwell's equations with boundary conditions defined by the modulated display elements of the display screen.

[0272] In some implementations, the illuminator is coupled to a controller via a storage buffer configured to control the amplitude or brightness of one or more light-emitting elements in the illuminator. The storage buffer used for the illuminator can be smaller than that used for the display screen. The number of light-emitting elements in the illuminator can be less than the number of display elements on the display screen. The controller can be configured to simultaneously activate one or more light-emitting elements of the illuminator.

[0273] In some examples, the illuminator includes two or more light-emitting elements, each configured to emit light of a different color. The controller can be configured to sequentially modulate the display screen using information associated with a first color during a first time period and information associated with a second color during a sequential second time period, and to control the illuminator to sequentially activate the first light-emitting element to emit light of the first color during the first time period and the second light-emitting element to emit light of the second color during the second time period. In this way, multi-colored objects can be displayed in 3D space.

[0274] In some examples, the display screen is small enough to diffract the resolution of the light. An illuminator emits white light onto the display screen, which diffracts the white light into different colors, thus displaying multicolored objects.

[0275] Exemplary System

[0276] Figures 5A to 5F An implementation of an exemplary system for 3D display is shown. Any system may correspond to, for example... Figure 1A System 100.

[0277] Figure 5A A system 500 with a reflective display screen is shown. System 500 includes a computer 502, a controller 510 (e.g., an ASIC), a display screen 512 (e.g., an LCOS device), and a illuminator 514. The computer 502 may be... Figure 1A The computing device 102, the controller 510 may be Figure 1A The controller 112. The display screen 512 can be... Figure 1AThe display screen 114 and the lighting device 514 can be Figure 1A Lighting fixture 116.

[0278] like Figure 5A As shown, computer 502 includes application 504 with renderer 503 for rendering objects in a scene. The rendered scene data is processed sequentially by video driver 505 and GPU 506. GPU 506 is... Figure 1A The GPU 108 is configured to generate a list of primitives and corresponding primitive data corresponding to the scene. For example, the video driver 505 can be configured to process the rendered scene data and generate the list of primitives. As described above, the GPU 506 may include a conventional 2D renderer, such as... Figure 1A A conventional 2D renderer 120 renders primitives as a list of items to be drawn on a 2D display 508. A GPU 506 or controller 510 may include a holographic renderer, for example... Figure 1A The holographic renderer 130 renders the list of primitives into graphical data that will be displayed on the display screen 512.

[0279] The controller 510 is configured to receive graphic data from the computer 502, calculate the electromagnetic field contribution of the primitive list to each display element of the display screen 512, and generate the sum of the corresponding electromagnetic field contributions of the primitives to each display element. The controller 510 can generate a corresponding control signal for each display element to modulate at least one characteristic of the display element. The controller can send the corresponding control signals to the display elements of the display screen 512 via a storage buffer 511 for the display screen 512.

[0280] The controller 510 can also generate and send control signals, such as illumination timing signals, to activate the illuminator 514. For example, in response to determining that the sum of the electromagnetic field contributions of the computational primitives to the display elements has been completed, the controller 510 can generate and send control signals. As described above, the controller 510 can send control signals to the illuminator 514 via a storage buffer. The storage buffer can be configured to control the amplitude or brightness of the light-emitting elements in the illuminator 514 and simultaneously activate the light-emitting elements.

[0281] like Figure 5A As shown, the illuminator 514 emits a collimated beam 516 that is incident on the front surface of the display screen 512 at an angle of incidence between 0 and 90 degrees. The emitted beam is reflected from the front surface of the display screen 512 to form a holographic light field 518, which corresponds to an object visible to the viewer.

[0282] Figure 5B Another system 520 with another reflective display screen 524 is shown. (Compared to...) Figure 5ACompared to system 500, system 520 has a larger reflective display screen 524. To accommodate this, the display controller 522 is included in a wedge-shaped housing, which can serve as a support for the illuminator 526. The controller 522 is similar to... Figure 5A The controller 510 is configured to receive graphic data from the computer 521, calculate the electromagnetic field contribution of multiple primitives to each display element of the display screen 524, and generate a sum of the corresponding electromagnetic field contributions of the multiple primitives to each display element. The controller 522 then generates a corresponding control signal for each display element to modulate at least one characteristic of the display element, and sends the corresponding control signal to the display element of the display screen 524 through a storage buffer 523 for the display screen 524.

[0283] The controller 522 also sends a control signal to the illuminator 526 to activate the illuminator 526. The illuminator 526 emits a divergent or semi-collimated beam 527 to cover the entire surface of the display screen 524. The beam 527 is reflected by the modulated display screen 524 to form a holographic light field 528.

[0284] Figure 5C A system 530 with a transmissive display screen 534 is shown. The transmissive display screen 534 may be, for example, a large display screen. The system 530 includes a controller 532, which may be similar to... Figure 5A The controller 510. The controller 532 can be configured to receive graphics data from the computer 531, calculate the electromagnetic field contribution of a plurality of primitives to each display element of the display screen 534, and generate a sum of the corresponding electromagnetic field contributions of the plurality of primitives to each display element. The controller 532 then generates a corresponding control signal for each display element to modulate at least one characteristic of the display element, and sends the corresponding control signal to the display element of the display screen 534 through a storage buffer 533 for the display screen 534.

[0285] Controller 532 also sends a control signal to illuminator 536 to activate illuminator 536. Unlike... Figure 5A System 500 and Figure 5B In system 520 and system 530, illuminator 536 is positioned behind the rear surface of display screen 534. To cover the large surface of display screen 534, illuminator 536 emits a divergent or semi-collimated beam 535 onto the rear surface of display screen 534. The beam 535 is transmitted through and modulates display screen 534 to form a holographic light field 538.

[0286] Figure 5D Another system 540 with a transmissive display screen 544 is shown. System 540 also includes a controller 542 and an illuminator 546. The controller 542 may be similar to... Figure 5AThe controller 510 is configured to receive graphic data from computer 541, perform calculations on the graphic data, generate control signals for modulation and send them to display screen 544, and generate and send timing signals to activate illuminator 546.

[0287] Illuminator 546 may include a light source 545 and a waveguide 547. Light emitted from the light source 545 may be coupled to the waveguide 547, for example, from a side cross-section of the waveguide. The waveguide 547 is configured to guide light to uniformly illuminate the surface of the display screen 544. The light guided by the waveguide 547 is incident on the rear surface of the display screen 544 and transmitted through the display screen 544 to form a holographic light field 548.

[0288] Unlike Figure 5A System 500 Figure 5B System 520 and Figure 5C In system 530, and system 540, controller 542, display screen 544, and waveguide 547 are integrated together into a single unit 550. In some cases, waveguide 547 and light source 545 can be integrated in a planar form as an active waveguide illuminator, which can further improve the integration of the single unit 550. As described above, the single unit 500 can be connected to other similar units 550 to form a larger holographic display device.

[0289] Figure 5E Another system 560 is shown, featuring another transmissive display 564. Compared to system 540, the transmissive display 564 can potentially realize a larger display than the transmissive display 544. For example, the transmissive display 564 may have a larger area than the controller 562, and to accommodate this, the controller 562 may be positioned remotely from the display 564. System 560 includes an illuminator 566 having a light source 565 and a waveguide 567. The waveguide 567 is integrated with the display 564, for example, into the rear surface of the display 564. In some embodiments, the display 564 is constructed on the front surface of a substrate, and the waveguide 567 may be constructed on the rear surface of the substrate.

[0290] Controller 562 can be similar to Figure 1A The controller 510 is configured to receive graphic data from the computer 561, perform calculations on the graphic data, generate control signals and send them to the display screen 564 via a storage buffer 563, and generate and send timing signals to activate the light source 565. Light emitted from the light source 565 is guided in a waveguide 567 to illuminate the rear surface of the display screen 564 and is transmitted through the display screen 564 to form a holographic light field 568.

[0291] Figure 5FAnother system 570 with a reflective display screen 574 is shown. The reflective display screen 574 can be, for example, a large display screen. The waveguide 577 of the illuminator 576 is positioned on the front surface of the reflective display screen 574. Similar to... Figure 5E The controller 572 of the controller 562 can be configured to receive graphic data from the computer 571, perform calculations on the graphic data, generate control signals and send them to the display screen 574 through the storage buffer 573, and generate and send timing signals to activate the light source 575 of the illuminator 576. The light coupled from the light source 575 of the illuminator 576 is guided to be incident on the front surface of the display screen 574 and reflected by the front surface to form a holographic light field 578.

[0292] Exemplary display screen implementation

[0293] As mentioned above, the display screen in Maxwell's holography can be a phase modulation device. The phase element (or display element) of the display screen can be represented as a phase unit. For illustrative purposes only, liquid crystal on silicon (LCOS) devices are discussed below as phase modulation devices. An LCOS device is a display screen that uses a liquid crystal (LC) layer on top of a silicon backplane. LCOS devices can be optimized to achieve the minimum possible phase pitch, minimal crosstalk between phases, and / or a large available phase modulation or hysteresis (e.g., at least 2π).

[0294] A list of parameters can be controlled to optimize the performance of LCOS devices, including birefringence (Δn) of the LC mixture, cell gap (d), dielectric anisotropy of the LC mixture (Δε), rotational viscosity of the LC mixture (η), and maximum applied voltage (V) between the silicon backplane and the common electrode on top of the LC layer.

[0295] There can be fundamental trade-offs between the parameters of liquid crystal materials. For example, a fundamental boundary parameter is the available phase modulation or hysteresis (Re), which can be expressed as:

[0296] Re=4π·Δn·d / λ (8),

[0297] Where λ is the wavelength of the input light. If, for red light with a wavelength of approximately 0.633 μm, the hysteresis Re needs to be at least 2π, then...

[0298] Δn·d≥0.317μm (9).

[0299] The above expression implies a direct trade-off between the unit gap (d) and birefringence (Δn) in the LC mixture.

[0300] Another boundary parameter is the switching speed or switching time (T) that it takes for the liquid crystal molecules in the liquid crystal (LC) layer to reach the desired orientation after a voltage is applied. For example, for real-time video (approximately 60 Hz) using a 3-color field sequential color system, involving modulation of the LC layer to a minimum of 180 Hz, this results in an upper boundary of 5.6 milliseconds (ms) for the LC switching speed. The switching time (T) is related to several parameters, including the liquid crystal, the cell gap, and the applied voltage. First, T is related to d 2 The switching time is proportional to the square of the cell gap d. Furthermore, the switching time is also related to the dielectric anisotropy (Δε) of the liquid crystal mixture, where higher dielectric anisotropy results in a shorter switching time, and lower viscosity also results in a shorter switching time.

[0301] The third boundary parameter can be the edge field. Due to the high electron mobility of crystalline silicon, LCOS devices can be constructed with very small phase cell sizes (e.g., less than 10 μm) and submicron phase cell gaps. When adjacent phase cells operate at different voltages, the liquid crystal director near the phase cell edges is distorted by the lateral component of the edge field, which significantly degrades the electro-optic performance of the device. Furthermore, as the phase cell gap becomes comparable to the incident light wavelength, diffraction effects can cause severe light loss. The phase cell gap needs to be kept less than or equal to the phase distance to keep noise within acceptable levels.

[0302] In some examples, LCOS devices are designed with a phase spacing of 2 μm and a cell gap of approximately 2 μm if the edge field boundary conditions are satisfied. According to the expression Δn·d ≥ 0.317 μm, Δn needs to be equal to 0.1585 or greater, which can be achieved using existing liquid crystal techniques. Once the minimum birefringence for a given phase spacing is determined, the LC can be optimized for switching speed, for example, by increasing dielectric anisotropy and / or decreasing rotational viscosity.

[0303] Non-uniform phase unit implementation of the display screen

[0304] In LCOS devices, a circuit chip (e.g., a complementary metal-oxide-semiconductor (CMOS) chip or equivalent) controls the voltage on reflective metal electrodes buried beneath the chip surface, with each reflective metal electrode controlling one phase cell. A common electrode for all phase cells is provided by a transparent conductive layer made of indium tin oxide on a cover glass. The phase cells can have the same size and shape (e.g., square). For example, the chip can have 1024x768 plates, each with an independently addressable voltage. As mentioned above, when the phase cell gaps become comparable to the incident light wavelength, diffraction effects can occur in the periodic structure of the LCOS device, potentially leading to significant light loss.

[0305] In Maxwell's holographic calculations, each phase unit receives the sum of electromagnetic field contributions from each elementary element and is relatively independent of each other. Therefore, the phase units of an LCOS device in Maxwell's holography can be designed to be distinct from one another. For example, as... Figure 6A As shown, the LCOS device 600 may consist of a plurality of non-uniform (or irregular) phase cells 602. At least two phase cells 602 have different shapes. The non-uniform shape of the phase cells 602 can greatly reduce or eliminate diffraction aberrations and other effects, and thus improve image quality. Although the phase cells may have non-uniform shapes, the phase cells may be designed to have a size distribution that meets the desired spatial resolution (e.g., about 3 μm). The silicon backplane may be configured to provide corresponding circuitry (e.g., including metal electrodes) for each phase cell according to the shape of the phase cells.

[0306] In the phase cell array of an LCOS device, to select a specific phase cell, a first voltage is applied to the word line connecting the row of phase cells containing the specific phase cell, and a second voltage is applied to the bit line connecting the column of phase cells containing the specific phase cell. Because each phase cell has a resistor, the operating speed of the LCOS device is limited.

[0307] As mentioned above, in Maxwell's holography, phase units can have different sizes. For example... Figure 6B As shown, the LCOS device 650 is designed to have one or more phase cells 654, the size of which is larger than the size of the other phase cells 652. All phase cells can still have a size distribution that satisfies the desired resolution. For example, 99% of the phase cells have a size of 3 μm, and only 1% have a size of 6 μm. In addition to other circuitry identical to that in the phase cells 652, the larger size of the phase cells 654 allows for the arrangement of at least one buffer 660 within the phase cells 654. The buffer 660 is configured to buffer applied voltages such that voltages are applied only to a small number of phase cells within a row or column of phase cells. The buffer 660 can be analog circuitry (e.g., composed of transistors) or digital circuitry (e.g., composed of multiple logic gates) or any combination thereof.

[0308] For example, such as Figure 6BAs shown, a voltage is applied to word line 651 and another voltage is applied to bit line 653 to select a specific phase cell 652*. Phase cell 652* is in the same row as a larger phase cell 654, which includes a buffer 660. The voltage is primarily applied to a plurality of first phase cells preceding the larger phase cell 654 in the row and is blocked by the buffer 660 in the larger phase cell 654. In this way, the operating speed of the LCOS device 650 can be increased. Utilizing the larger size of the phase cell 654, other circuitry can also be arranged in the LCOS device 650 to further improve its performance. Although... Figure 6B The diagram shows phase unit 654 and phase unit 652 having a square shape; the phase unit may also have, for example, a square shape. Figure 6A The different shapes shown are provided as long as one or more phase units 654 are larger than the other phase units 652.

[0309] Exemplary calibration

[0310] The unique properties of Maxwell's holography in this disclosure allow for the protection of calibration techniques that can provide a significant competitive advantage in the actual production of high-quality displays. Multiple calibration techniques can be implemented in combination with Maxwell's holographic computing techniques, including:

[0311] (i) Using an image sensor in combination with a Dirichlet boundary condition modulator and / or in combination with mechanical and software diffraction and non-diffraction calibration techniques;

[0312] (ii) Software alignment and software calibration, including separate color calibration and alignment utilizing Dirichlet boundary conditions; and

[0313] (iii) Embedding silicon features in a boundary condition modulator, which allows optical detection to be built directly into the modulator, creating a powerful and unique approach to simplifying manufacturing calibration processes when combined with Maxwell's holography.

[0314] In the following text, for illustrative purposes only, three types of calibration are implemented for phase-based displays (e.g., LCOS displays). Each phase element can be represented as a phase unit.

[0315] Phase calibration

[0316] The amount of phase added to the light illuminating the phase element (or phase cell) of an LCOS can be directly known from the voltage applied to the LCOS phase cell. This is because the birefringent liquid crystal (LC) rotates in the presence of an electric field and thus changes its refractive index, thereby slowing down the light and altering its phase. The changed phase can depend on the liquid crystal (LC) and the electrical properties of the silicon device in which the LC resides. The digital signal sent to the LCOS needs to be converted into the correct analog voltage to achieve a high-quality holographic image. LCOS devices involve phase calibration to ensure that the digital signal is properly converted into the analog signal applied to the LC, resulting in the maximum phase range. This conversion is expected to result in linear behavior; that is, when the voltage changes in a fixed increment, the phase also changes in a fixed increment, regardless of the initial voltage value.

[0317] In some cases, LCOS devices allow users to change the digital-to-analog converter (DAC), enabling users to control the amount of analog voltage output given a digital input signal. Digital potentiometers can be applied to each input bit. For example, if there are 8 input bits, then there can be 8 digital potentiometers corresponding to each input bit. The same digital input from the digital potentiometers can be applied to all phase cells of the LCOS device. Bits set to "1" activate the voltage, and bits set to "0" do not activate the voltage. All voltages from such "1" bits are summed together to obtain the final voltage sent to each phase cell. It is also possible to apply a DC voltage in all cases, such that all "0" bits result in a non-zero baseline voltage. Therefore, phase calibration of the LCOS device can be achieved by setting the values ​​of the digital potentiometers used for the LCOS device. For example, as described above, the controller can calculate the electromagnetic field contribution of the primitive list to each phase cell of the display, generate a sum of the corresponding electromagnetic field contributions of multiple primitives to each phase cell, and generate a corresponding control signal for each phase cell to modulate the phase of the phase cell. The same digital input from a digital potentiometer can be applied to adjust the control signals for each phase unit of the LCOS device, unlike phase calibration which is performed unit by unit. The digital input can be set once during operation of the LCOS device, for example, for displaying a hologram.

[0318] To determine an optimal set of phase calibration values ​​for digital inputs, a genetic algorithm can be applied, where multiple input values ​​result in a single output value, such as phase range or holographic image contrast. This output value can be simplified to a numerical value called fitness. The genetic algorithm can be configured to explore different combinations of input values ​​until an output with the highest fitness is achieved. In some cases, the algorithm may take two or more of the best-fitting inputs and combine multiple component values ​​of them to create a new input that has the characteristics of the adopted inputs but is different from each adopted input. In some cases, the algorithm may change one of these component values ​​to something not derived from any of the adopted suitable inputs; this is referred to as a "mutation," and the variant can be added to the available suitable inputs. In some cases, while trying new values, one or more optimal values ​​can be found by leveraging knowledge gained from previous measurements with good results, thus the optimal value is not limited to a local maximum.

[0319] There are several ways to calculate the fitness output value. One way is to calculate the phase change of light given a set of digital inputs applied to all phase units on the LCOS. In this scheme, the incident light is polarized. When incident on the LCOS, the polarization of the incident light can change according to the rotation of the LC. The incident light can enter the photodetector by being set to the same polarization as the original polarization or to a polarization 90 degrees different from the original polarization. Therefore, the intensity observed from the photodetector can change when the rotation of the LC changes. Thus, the phase change of light can be indirectly perceived through the change in light intensity. Another way to calculate the phase change is to measure the intensity difference between the Maxwell holographic reconstruction and the background. This is most effective in projection displays. In this case, measuring the intensity may require using computer vision algorithms to identify the Maxwell holographic reconstruction and measure its intensity.

[0320] Alignment Calibration

[0321] The alignment of the light source within the holographic device cannot be guaranteed and therefore requires alignment. Different liquid crystals (LCs) can behave differently for a given light source wavelength. Furthermore, both the LC and the light source can be varied device-by-device, thus giving the same input hologram different characteristics when displayed with different base colors, such as object scaling. Additionally, certain hardware features can apply different optical effects, such as lensing, to the same output light requiring correction.

[0322] In some implementations, the aforementioned problem can be addressed by applying a mathematical transformation to the phase calculated for the phase cells of the display screen. The phase is the sum of the corresponding electromagnetic field contributions of the primitive list to the phase cells. The mathematical transformation can be derived from mathematical expressions (e.g., Zernike polynomials) and can be varied by changing the polynomial coefficients or other varying input values. The mathematical transformation can be applied per phase cell and according to color. For example, there exist Zernike polynomial coefficients that correspond to the amount of tilt applied to light after reflection from the display screen.

[0323] To determine these coefficients / input values, a hardware setup can be created where the camera is pointed at the reflective surface in the case of a projection display and directly at the LCOS in the case of a direct-view display. A series of holographic test patterns and objects can be sent to the display and viewed by the camera. The camera can use machine vision algorithms to determine what is being displayed and subsequently calculate its fitness. For example, if the test pattern is a dot grid, the fitness is the proximity of the points, the centering of the point locations, the amount of distortion the points have (e.g., scaling or pincushion distortion), etc. Different fitness values ​​can exist for different characteristics. Depending on these values, corrections can be applied, for example, by changing the coefficients of the Zernike polynomial until a predetermined satisfactory fitness level is reached. These test patterns can be rendered at different distances to ensure that the alignment of the object is consistent at all distances, not just at a single point. Such depth-based calibration involves an iterative process that involves changing the depth of the reflective surface and the holographic test pattern in the case of a projection display, where previous calibrations can be repeated until convergence to a solution feasible at both depths. Finally, white dots can be displayed to show the effect of the calibration.

[0324] Color calibration

[0325] In displays, whether holographic or otherwise, when any two units render the same image, it is crucial that the displays match color with each other and, additionally, with the colors defined by television (TV) and computer display standards, such as the Rec.709 standard for high-definition television (HDTV) or the sRGB color space for computer monitors. Different batches of hardware components (e.g., LEDs and laser diodes) can exhibit different behaviors for the same input and can output different colors when perceived by the human eye. Therefore, it is important to have a color standard capable of calibrating all display units.

[0326] In some implementations, objective measurements of color specified by measuring intensity and chromaticity can be obtained by comparing color intensity to the CIE Standard Observer curve. By requesting a sample set of known colors and intensities to be reproduced on each display, and then measuring the output light using a colorimeter device calibrated according to the CIE Standard Observer curve, the device's color output in the CIE XYZ color space can be objectively defined. Any deviations from known good values ​​can be used to adjust the output color on the display to bring it back to alignment, which can be implemented using an iterative measurement-adjustment-measurement feedback loop. Once the Maxwell holographic device produces accurate output for a given set of inputs, the final adjustment can be encoded as a lookup table for mapping input values ​​to output intensities in the illuminators, and as a color matrix transformation encoded to transform input colors into output color space values. These calibration tables can be embedded in the device itself to produce reliable, objective output colors.

[0327] Furthermore, given an LCOS device with sufficiently fine features to control diffraction with subwavelength precision, tristimulus illumination (e.g., a linear mixture of red, green, and blue) may not be necessary, and the LCOS device can be illuminated by a single broadband light source and the phase unit output can be selectively tuned to produce tristimulus, tetrastimulus, or even N-stimulus output colors. These colors, combined with spatial jitter patterns, can reproduce the full spectral output of color rather than the common tristimulus approximation. Given a sufficiently broad spectral illuminator, this allows Maxwell's holography to produce any reflected color within the spectral locus of the human visual system.

[0328] The embodiments of the subject matter and functional operation described herein may be implemented in digital electronic circuits, in tangibly implemented computer software or firmware, in computer hardware including the structures disclosed herein and their structural equivalents, or in combinations thereof. Embodiments of the subject matter described herein may be implemented as one or more computer programs, for example, one or more computer program instruction modules encoded on a tangible, non-transitory computer storage medium, for performing or controlling the operation of a data processing device by data processing. Optionally or additionally, program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device for execution by the data processing device. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination thereof.

[0329] The terms "data processing apparatus," "computer," or "electronic computer equipment" (or equivalents as understood by one of ordinary skill in the art) refer to data processing hardware and include means, devices, and machines for processing all kinds of data, such as programmable processors, computers, or multiple processors or computers. The apparatus may also be or further include special-purpose logic circuitry, such as a central processing unit (CPU), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). In some embodiments, the data processing apparatus and special-purpose logic circuitry may be hardware-based and software-based. The apparatus may optionally include code that creates an execution environment for computer programs, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these. This document contemplates the use of the data processing apparatus with or without a conventional operating system.

[0330] A computer program, also referred to or described as a program, software, software application, module, software module, script, or code, can be written in any form of programming language, including compiled or deductive languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may, but does not necessarily, correspond to a file in a file system. A program may be stored as a portion of a file containing other programs or data, for example, one or more scripts stored in a markup language document; a single file or multiple co-located files dedicated to the program in question, for example, a file storing one or more modules, subroutines, or code portions. A computer program may be deployed to execute on a single computer, or on multiple computers located in one location or distributed across multiple locations and interconnected by a communication network. Although the parts of the program shown in the figures are individual modules implementing various features and functions through various objects, methods, or other processes, the program may suitably include multiple submodules, third-party services, components, libraries, etc. Conversely, the features and functions of various components may be suitably combined into a single component.

[0331] The processes and logic flows described herein can be executed by one or more programmable computers, which execute one or more computer programs to perform functions by manipulating input data and producing outputs. The processes and logic flows can also be executed by dedicated logic circuitry, and the devices can be implemented as dedicated logic circuitry, such as CPUs, GPUs, FPGAs, or ASICs.

[0332] A computer suitable for executing computer programs may be based on a general-purpose microprocessor and / or a special-purpose microprocessor, or any other type of CPU. Typically, the CPU receives instructions and data from read-only memory (ROM) and / or random access memory (RAM). The main components of a computer are the CPU for making or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer also includes, or is operatively coupled to, one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, to receive data from and / or transfer data to these mass storage devices. However, a computer does not necessarily need to have such devices. Furthermore, a computer may be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name just a few.

[0333] Computer-readable media suitable for storing computer program instructions and data (whether transient or non-transitory, as the case may be) include all forms of non-volatile memory, media, and storage devices, including: semiconductor storage devices, such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices; disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM, DVD-R, DVD-RAM, and DVD-ROM discs. Memory can store a variety of objects or data, including caches, classes, frames, applications, backup data, jobs, web pages, web page templates, database tables, repositories storing business and dynamic information, and any other suitable information including any parameters, variables, algorithms, instructions, rules, constraints, or references. Additionally, memory may include any other suitable data, such as logs, policies, security or access data, report files, and other data. Processors and memory may be supplemented by dedicated logic circuitry or integrated into dedicated logic circuitry.

[0334] To provide user interaction, the embodiments of the subject matter described herein can be implemented on a computer having: a display device for displaying information to the user, such as a CRT (cathode ray tube), LCD (liquid crystal display), LED (light-emitting diode), or plasma monitor; and a keyboard and a pointing device that the user can use to provide input to the computer, such as a mouse, trackball, or trackpad. Input to the computer can also be provided using a touchscreen, such as a pressure-sensitive tablet computer surface, a multi-touch screen using capacitive or inductive touchscreens, or other types of touchscreens. Other types of devices can also be used to provide user interaction; for example, feedback provided to the user can be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and any form of input from the user can be received, including sound, voice, or tactile input. Furthermore, the computer can interact with the user by sending and receiving documents to and from the device used by the user; for example, by sending a webpage to a web browser on the user's client device in response to a request received from a web browser.

[0335] The term "graphical user interface" or "GUI" can be used in the singular or plural to describe one or more graphical user interfaces and each display of a particular graphical user interface. Therefore, a GUI can refer to any graphical user interface, including but not limited to a web browser, a touchscreen, or a command-line interface (CLI) that processes information and effectively presents the results to a user. Generally, a GUI may include multiple user interface (UI) elements, some or all of which are associated with a web browser, such as interactive fields, dropdown lists, and buttons that can be operated by a business suite user. These and other UI elements may be related to or represent the functionality of a web browser.

[0336] The embodiments of the subject matter described herein can be implemented in computing systems including back-end components such as data servers, or middleware components such as application servers, or front-end components such as client computers with graphical user interfaces or web browsers, or any combination of one or more such back-end, middleware, or front-end components, wherein a user can interact with the embodiments of the subject matter described herein through a graphical user interface or web browser. The components of the system can be interconnected through any form or medium of wired or wireless digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs), radio access networks (RANs), metropolitan area networks (MANs), wide area networks (WANs), WiMAX, wireless local area networks (WLANs) using, for example, 902.11a / b / g / n and 902.20, all or part of the Internet, and any other communication system or systems in one or more locations. For example, the network can communicate with network addresses via Internet Protocol (IP) packets, Frame Relay frames, Asynchronous Transfer Mode (ATM) units, voice, video, data, or other suitable information.

[0337] A computing system may include clients and servers. Clients and servers are typically geographically separated and usually interact through a communication network. The client-server relationship is established by computer programs running on their respective computers and having a client-server relationship with each other.

[0338] In some implementations, any or all components of a computing system, including hardware and software, may interface or interface with each other using application programming interfaces (APIs) or service layers. APIs may include specifications for routines, data structures, and object classes. APIs may be language-independent or language-dependent and refer to a complete interface, a single function, or even a set of APIs. Service layers provide software services to the computing system. The functionality of the various components of the computing system is accessible to all service consumers using the service layer. Software services provide reusable, defined business functions through defined interfaces. For example, an interface may be software written in any suitable language and providing data in any suitable format. APIs and service layers may be components integrated with other components of the computing system or independent components. Furthermore, without departing from the scope of this document, any or all portions of the service layer may be implemented as child or sub-modules of another software module, enterprise application, or hardware module.

[0339] While this document contains details of numerous specific embodiments, these details should not be construed as limiting the scope of any invention or the scope of the claims, but rather as descriptions of features characteristic of particular embodiments of a particular invention. Certain features described herein in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although features may be described as functioning in certain combinations, or even initially claimed in this way, in some cases one or more features from the claimed combination may be removed from that combination, and the claimed combination may be for sub-combinations or variations thereof.

[0340] Specific embodiments of the subject matter have been described. It will be apparent to those skilled in the art that other embodiments, modifications, and substitutions of the described embodiments are within the scope of the appended claims. Although the operations are described in a specific order in the drawings or in the claims, this should not be construed as requiring such operations to be performed in the specific order or sequence shown, or to perform all the operations shown (some operations may be considered optional) to obtain the desired result. In some cases, multitasking or parallel processing may be advantageous and is appropriately implemented.

[0341] Therefore, the description of the exemplary embodiments provided above is not limiting or restrictive of this document. Other changes, substitutions, and modifications are possible without departing from the spirit and scope of this document.

Claims

1. A display screen, comprising: Multiple display elements, wherein at least two of the multiple display elements have different shapes; The display screen includes: Liquid crystal layer; A transparent conductive layer serving as a common electrode is located on top of the liquid crystal layer; and A plurality of metal electrodes are located at the bottom of the liquid crystal layer, wherein each of the plurality of metal electrodes is isolated from the others; In this embodiment, for each of the plurality of display elements of the display screen, a corresponding metal electrode is configured according to the shape of the display element so that the corresponding metal electrode has a shape corresponding to the shape of the display element, wherein each metal electrode for at least two display elements has a shape corresponding to a different shape of the at least two display elements.

2. The display screen as claimed in claim 1, wherein, The larger of the at least two display elements includes a buffer, and The smaller of the at least two display elements does not include a buffer.

3. The display screen as claimed in claim 2, wherein, The larger display element is connected to a first number of display elements via wires, and The buffer is configured to buffer the voltage applied to the conductor, such that the voltage is applied only to a second number of display elements within the first number of display elements, the number of the second number of display elements being less than the number of the first number of display elements.

4. The display screen as claimed in claim 2 or 3, wherein, The buffer includes analog circuits in the form of transistors or digital circuits in the form of logic gates.

5. The display screen as claimed in any one of claims 1 to 3, wherein, The size distribution of the plurality of display elements is substantially equal to the size of the smaller display element among the at least two display elements.

6. The display screen as claimed in any one of claims 1 to 3, wherein, The display screen includes a spatial light modulator (SLM), which includes a digital micromirror device (DMD) or a liquid crystal on silicon (LCOS) device.

7. The display screen as claimed in any one of claims 1 to 3, wherein, The size distribution of the multiple display elements is basically equal to the predetermined size.

8. The display screen as claimed in any one of claims 1 to 3, wherein, The display screen includes a liquid crystal display (LC) screen, and The cell gap of the liquid crystal display screen is determined based on the pitch of the display elements of the liquid crystal display screen; Alternatively, the minimum value of birefringence of the liquid crystal mixture can be calculated based on the cell gap and the predetermined delay of the liquid crystal display screen; Alternatively, while keeping the birefringence of the liquid crystal mixture above a minimum, increasing the switching speed of the liquid crystal display includes selecting at least one from the group consisting of: increasing the dielectric anisotropy of the liquid crystal mixture; and decreasing the rotational viscosity of the liquid crystal mixture.

9. The display screen as claimed in any one of claims 1 to 3, wherein, The display screen includes: Back panel, It is configured to control the voltage of each of the plurality of metal electrodes.

10. The display screen as claimed in any one of claims 1 to 3, wherein, The display screen is configured as one of the following: phase modulation, amplitude modulation, or both phase and amplitude modulation.

11. A display system, comprising: The display screen as described in any one of claims 1 to 10; as well as A controller, coupled to the display screen and configured to send at least one control signal to at least one display element of the display screen to modulate at least one characteristic of the at least one display element.

12. The system of claim 11, wherein, The controller includes at least one of the following: Application-Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA), Programmable Gate Array (PGA), Central Processing Unit (CPU), Graphics Processing Unit (GPU), or Standard Computing Unit.

13. The system as claimed in claim 11 or 12, wherein, The at least one characteristic of the at least one display element includes at least one of the following: refractive index, amplitude index, birefringence, or hysteresis.

14. The system as claimed in claim 11 or 12, wherein, The at least one control signal includes at least one of the following: an electrical signal, an optical signal, a magnetic signal, or an acoustic signal.

Images