Display device

Abstract

An apparatus for displaying a synthetic scene image for augmented reality, virtual reality, and other display types. One or more input beams (70, 76) are incident on a display panel incorporating a layer (100) of a phase-matched nonlinear optical (NLO) material. Each pair of input beams is crossed with each other within the NLO material in a non-collinear geometry to generate a generated beam (84) directed towards the observer's eye (160) by sum frequency generation. Synthetic scene image information is applied to each pair of input beams by amplitude and phase modulation to set the amplitude (i.e., luminance), wave vector (i.e., direction), and radius of curvature of the wavefront (i.e., depth) of the synthetic scene image. In the case of color display, three pairs of input beams having sum frequencies of the three primary colors are used.

Description

【Background Art】 【0001】 Augmented reality (AR) and virtual reality (VR) displays are incorporated into various wearable forms, including headsets. In the case of AR headsets, the glasses (i.e., spectacles) form is common. In the case of VR headsets, the ski goggle form is common. A recent review article on AR and VR display technologies is Non-Patent Document 1, which is at https: / / doi.org / 10.1016 / j.isci.2020.101397. 【0002】 In AR, the image of the synthetic scene is superimposed on the natural scene that is simultaneously observed by the observer. The AR eyeglass lenses are at a vertex distance of about 12 - 15 mm, which is much closer than the near point of the eye, which gradually increases with age from about 7 cm for young people to over 50 cm for the elderly. (The near point of the eye is the closest distance at which the eye can focus.) Thus, in AR, the synthetic scene is represented by a virtual image that is placed farther from the eye than the AR eyeglass lenses. VR image formation is simpler than AR because there is no natural scene and only the synthetic scene is generated. 【0003】 In the case of glasses - form AR headsets, the currently most common technical approach is to use a micro - projector system. The virtual image that reproduces the synthetic scene is sent to one or both of the glasses lenses. (We use the term "lens" for the sake of convenience of language, but it should be understood that the lens may be blank, i.e., it may not have a focusing effect in the sense of normal vision correction for nearsightedness or farsightedness, etc.). The lenses of AR glasses are used as a medium for sending the light representing the synthetic scene into the path of the natural scene light through a suitable beam combiner. AR beam combiners can be sub - classified into reflective (e.g., mirrors or prisms) or diffractive (i.e., certain types of gratings). The virtual image of the synthetic scene should give the impression of depicting one or more objects in the correct place within the natural scene, which means not only the correct position (essentially the angle) within the field of view but also preferably the correct distance from the eye. 【0004】 One or more of the existing AR and VR display technologies have some problems. One problem is how to avoid unwanted light scattering on the glasses lenses caused by the projected beam, which degrades the appearance of the scene and fails to give the impression that the AR glasses are completely transparent. For example, the glasses lenses are placed approximately 12 - 15 mm from the eyeball, and since this distance is similar to the focal length of the eye, any unwanted light scattering from the glasses is collimated by the eye and spreads widely across the retina. In bright ambient light conditions, such scattering is perceived as glare. In low - ambient - light conditions, such scattering can be much brighter than the light from the natural scene and thus can significantly impair vision. Light - scattering artifacts such as the rainbow effect are particularly severe problems when grating - based beam combiners are used. 【0005】 Another problem related to both AR and VR is how to avoid the vergence - accommodation conflict, which is a characteristic of binocular vision. The normal relationship between accommodation and vergence that exists when viewing a natural scene is disrupted by the rendering of a synthetic scene, resulting in headaches and nausea. The vergence - accommodation conflict occurs when an object within a synthetic scene is perceived to be located at a certain distance (and vergence locks to that distance), while the light representing that object originates from a different distance (and accommodation locks to that distance). Even in monocular vision, the brain has accommodation expectations based on the brain's understanding of the juxtaposition of objects in a familiar scene. Thus, headaches and nausea can also occur from monocular viewing of a synthetic scene that causes the eyes to accommodate at a distance while looking at an object, where the distance is different from the brain's expectation regarding the object's distance. 【Prior Art Documents】 【Non - Patent Documents】 【0006】 【Non - Patent Document 1】 Zhan, Yin, Xiong, He, and Wu, "Augmented Reality and Virtual Reality Displays: Perspectives and Challenges", Perspective, Vol.23, Issue8, 101397(2020) 【Summary of the Invention】 【0007】 The present invention proposes a display device based on a display panel that hosts phase - matching non - linear mixing of a pair of input beams intersecting within the display panel in a non - collinear geometry. The intersection occurs in a region of a nonlinear optical (NLO) material within the display panel. Each pair of intersecting input beams generates a generated beam by sum frequency generation (SFG) or other non - linear mixing processes within the NLO material. 【0008】 The display device according to this design can be provided for image formation close to the eyes for both AR and VR. The display device according to this design can also be provided for large displays such as large screens for video conferencing or televisions. 【0009】 According to one aspect of the present invention, there is provided a display device that displays a synthetic scene image in response to an input of image information, a first beam source for providing a first input beam of a first frequency and bandwidth, a second beam source for providing a second input beam of a second frequency and bandwidth, a display panel including an NLO material that is phase-matched to the first and second input beams and is phase-matched to a generated beam of a sum frequency equal to the sum of the first and second frequencies, an input beam routing component arranged to introduce the first and second input beams into the display panel, wherein the first and second input beams cross each other in a non-collinear geometry within the NLO material and cross the display panel along first and second paths to define an intersection volume where the generated beam is generated by sum frequency generation, the input beam routing component; a controller operable to form a synthetic scene image by amplitude-modulating at least one of the first and second input beams and phase-modulating at least one of the first and second input beams in response to an input of image information, thereby setting each of the amplitude, wave vector, and wavefront curvature radius of the generated beam A display device comprising is provided. 【0010】 In some embodiments, the first input beam encodes image intensity information that defines the intensity of the generated beam generated in each intersection volume of a given synthetic scene image, and the second input beam encodes image depth information that defines the radius of curvature of the wavefront in the generated beam generated in all intersection volumes of a given synthetic scene image. This is a convenient configuration to enable each synthetic scene image formed to be associated with a different perceived depth by setting its radius of curvature of the wavefront. Specifically, when the second input beam has a substantially flat wavefront meaning that the radius of curvature of the wavefront is greater than 5 meters, the synthetic scene image is formed at a perceived distance of infinity, and when the second input beam has a radius of curvature of the wavefront less than 5 meters, the synthetic scene image is formed at a perceived distance that is finite. Thus, it is possible to rapidly and continuously form several synthetic scene images to construct different layers of the landscape within the image frame, for example, a background at infinity and one or more closer layers representing particular image elements. 【0011】 In some embodiments, the controller is operable to group synthetic scene images into image frames, and different synthetic scene images of a given image frame have different radii of curvature of the wavefront to represent image elements located at different respective perceived distances. 【0012】 In other embodiments, the display device is for displaying a synthetic scene image in color and includes a plurality of beam sources including first and second input beam sources, the first and second input beams forming one of three pairs of input beams generated by the plurality of beam sources, and the first through third pairs of input beams combining to generate first through third generated beams having first through third sum frequencies that provide the first through third primary colors. In such embodiments, the plurality of beam sources for forming the three pairs of input beams may comprise an emitter array and first through third lasers, the emitter array providing one of the input beams for all three pairs of input beams, the other input beam of each pair being provided by one of the first through third lasers, and the first through third primary colors being provided by time slicing such that the first through third generated beams are sequentially generated. In particular, the emitter array may be driven to amplitude modulate the input beam it generates. For example, the NLO material may include first through third spatially modulated regions that may provide quasi-phase matching for the first through third pairs of input beams, respectively. One option here is where the first through third spatially modulated regions are formed within the NLO material at first through third depth portions within the display panel. Another option here is where the first through third spatially modulated regions are formed in the NLO material as an array of spot clusters, each spot cluster including adjacent spots of each of the first through third spatially modulated regions. 【0013】 For a color display in some embodiments, the controller may be operable to group synthetic scene images within an image frame, with different synthetic scene images of a given image frame being respective images of the first through third primary colors. 【0014】 For a color display in some embodiments, the controller may be operable to group synthetic scene images within an image frame, where different synthetic scene images of a given image frame are images of the first through third primary colors and have different wavefront curvature radii, thereby representing color image elements located at different respective perceived distances. 【0015】 In some embodiments, the input beam routing component is adjustable to provide at least one change in at least one of the first and second paths to vary the angle at which the first and second input beams intersect within the NLO material, thereby varying the direction of the wavevector of the generated beam. 【0016】 In some embodiments, the input beam routing component is adjustable to provide at least one change in at least one of the first and second paths to vary the location where the first and second paths intersect within the NLO material. 【0017】 In some embodiments, the display device further comprises an amplitude modulator arranged to amplitude modulate the first input beam and a phase modulator arranged to phase modulate the second input beam. In other embodiments, the display device further comprises a combined amplitude and phase modulator for amplitude and phase modulating one of the first and second input beams. 【0018】 In some embodiments, the NLO material is spatially modulated with respect to its second-order nonlinearity to provide phase matching through quasi-phase matching. In other embodiments, the NLO material is homogeneous with respect to its second-order nonlinearity and provides phase matching through birefringent phase matching. 【0019】 Regarding the first and second frequencies and their sum frequency: The sum frequency is preferably within the visible range, i.e., about 790 - 405 THz / 380 - 740 nm. The first and second frequencies are within the infrared range, i.e., about 400 - 150 THz / 750 nm - 2 μm. The ratio of the first frequency to the second frequency is 0.5 - 2.0. 【0020】 In the case of AR, the display panel including the NLO material is transparent over the visible range, i.e., about 790 - 405 THz / 380 - 740 nm, so that the natural scene light can pass through to the observer's eyes. 【0021】 In some embodiments, the display panel has a peripheral region including a material that is absorptive to light of the first and second frequencies, whereby the first and second input beams are absorbed after intersecting each other within the NLO material when the first and second paths reach the peripheral region. 【0022】 In some embodiments, the display device further comprises one or more optical sensors disposed outside the display panel to detect abnormal leakage of the input beam light from the display panel indicating structural damage to the display panel. 【0023】 The NLO material can be included within the display panel within the NLO material layer. In one particular implementation, the display panel comprises a front filter layer disposed in front of the NLO material layer, the front filter layer being opaque to the first and second frequencies and transparent to the sum frequency, and blocking outward emission of light from the display panel from the first and second input beams. In another particular implementation, the display panel comprises a back filter layer disposed behind the NLO material layer, the back filter layer being opaque to the first and second frequencies and blocking inward emission of light from the display panel from the first and second input beams. The back filter layer may be transparent to visible frequencies. One option is to distribute the NLO material continuously across the NLO material layer such that phase matching, and thus generated beam generation, can occur anywhere within the NLO material layer. Another option is to distribute the NLO material as an array of spots across the NLO material layer such that phase matching, and thus generated beam generation, is limited to the locations of those spots. 【0024】 In some embodiments, the display panel comprises a light-shielding layer having an array of pixels that can be individually addressed by electrical control lines to switch the pixels between a first state that is opaque to visible frequencies and a second state that is transmissive to visible frequencies, whereby natural scene light can be mixed from selected regions of the light-shielding layer. 【0025】 In some embodiments, the first and second paths traverse the display panel by successive reflections. The successive reflections may be from the front and back surfaces of the display panel by total internal reflection. Alternatively, the successive reflections may be from front and back mirror layers disposed within the display panel and disposed on the front and back of the NLO material, respectively. Specifically, the front and back mirror layers may be reflective at the frequencies of the input beams and transmissive across visible frequencies. 【0026】 In some embodiments, the NLO material is transparent across visible light. In some embodiments, the first and second frequencies are different. In other embodiments, the first and second frequencies are equal such that the sum frequency generation is second harmonic generation. When the first and second frequencies are equal, the first and second input beam sources can be the same single beam source, and both the first and second input beams are derived from that beam source. 【0027】 The display device described above may be incorporated into a wearable headset such that the display panel is disposed in front of the wearer's eyes with a vertex-to-vertex distance of less than 30 mm for image formation close to the eyes. 【0028】 Assuming that SFG is a non-linear process utilized, and also assuming that the generated beam forms a synthetic scene image visible to the human eye at visible wavelengths / frequencies (approximately 380 - 740 nm / 790 - 405 THz), each pair of input beams will both be in the infrared (e.g., 750 nm - 2 μm / 400 - 150 THz) provided that they are not too different in wavelength / frequency (e.g., having a ratio of less than 2:1). 【0029】 As an alternative non-linear mixing process for SFG, three-photon addition mixing can be provided by a combination of collinear mixing of second harmonic generation (SHG) in one of the input beams and non-collinear mixing of the second harmonic with the other input beam of the pair. 【0030】 Phase matching in the NLO material can occur without structuring the material, for example, by using a sufficiently thin layer of an NLO material having a high birefringence. However, the use of quasi-phase matching (QPM) is advantageous in many embodiments. To achieve QPM, the NLO material is structured to spatially modulate its non-linear properties. Most commonly, periodic poling is used to create a grating structure characterized by a single periodicity and alternating signs of the second-order non-linearity χ(2). 【0031】 The pair of two input beams are directed across the NLO material at an angle that ensures that the wavevector of the generated beam has a direction that propagates into the observer's eye, i.e., through the pupil. 【0032】 Noncollinear mixing of the first and second input beams in the NLO material enables the generated beam to be generated, and its properties are controlled in terms of amplitude (i.e., image luminance), wavevector (i.e., directing the generated light towards the pupil), wavefront curvature (i.e., controlling the perceived distance of the image), and the location of the image element within the field of view (i.e., the position on the NLO material). 【0033】 To produce a realistic impression of binocular vision, the ability to control the wavevector direction and the location of the image element is utilized to present slightly different images of the same synthetic object to the observer's left and right eyes, e.g., on each lens of a pair of AR glasses, resulting in a natural convergence and associated perception of depth (stereopsis) for the synthetic object. For synthetic objects that are close enough (i.e., less than about 4 - 5 meters) for monocular depth perception, the convergence simulation can be combined with setting the wavefront curvature of the synthetic object such that the focus (accommodation) of the eye is accurately or at least approximately matched to the same distance as that associated with the convergence, thereby avoiding convergence - accommodation conflict. For synthetic objects at greater distances, a plane wavefront can be used. 【0034】 The non-linear process generates an oscillating polarization at the frequency of the generated beam, i.e., in visible light. This process is essentially a wave mixing phenomenon, which means that the generated beam has a transverse (i.e., spatial) coherence with a defined phase relationship between different parts of the emitted wave. Thus, the generated beam generates a wavefront with a specific curvature. This is a fundamental property not possessed by the light generated by pixel emitters in conventional microdisplays as used in current VR headsets, or point emitters as used in current AR glasses. Rather, the generated beam produced by non-collinear mixing of a pair of input beams in an NLO material has properties more related to conventional diffractive optical techniques that also emit wavefronts by diffraction. The transverse coherent technique for generating light for a synthetic scene image allows the radius of curvature of the wavefront to be set, thereby allowing the synthetic scene image to be placed at a certain distance that can be varied by changing the radius of curvature of one of the wavefronts of the input beams. Thus, a natural depth perception can be given to the synthetic scene image. Different synthetic objects can be given different depth appearances, so that they appear at the appropriate depth relative to each other and, in the case of AR, relative to the objects in the natural scene. This not only contributes to the perceived realism of the synthetic scene image but also makes it possible to avoid convergence accommodation conflicts. 【0035】 A display device can be provided that can direct light for a synthetic scene image towards the eye from a wide range of angles (i.e., from a wide range of positions across the field of view). This property is beneficial as it takes into account the large angular turning range of the eye. This property is also beneficial as it enables the generation of a synthetic scene image that appears to be emitted from an appropriate position within the field of view when the eye is directed in a particular direction. Further, as described above, the generated beam light forming the synthetic scene image can be emitted from the display panel with a defined radius of curvature such that different parts of the synthetic scene image can be placed within the synthetic scene at appropriate perceived distances. 【0036】 In the case of an AR headset, the display panel can be realized in a lens form. In the conventional glasses form where a pair of lenses are set in a frame for the left and right eyes, there would be physically separated display panels for each lens. In the ski goggles form with a single lens sheet, a single display panel can be used. In the case of AR, the NLO material in the display panel is preferably transparent (in visible light), and as a result, it can be incorporated into the display panel without reducing the transmission of light moving from the natural scene entering the eye through the display panel. Thus, not only is the NLO material itself transparent, but since the wavelength of the input beam is outside the visible range (or can be easily selected to be so at least), an AR display panel that truly appears transparent to the observer can be provided. The light from the input beam is not only invisible to the human eye, but more importantly, by arranging a layer of a suitable filter material (e.g., as a band-pass filter or an edge filter) between the NLO material layer and the observer's eye, it is easy to prevent the input beam light of non-visible frequencies / wavelengths from entering the observer's eye. The filter material is selected to be opaque to the input beam wavelength and transparent over the visible range of wavelengths. In the case of an AR headset, it is also possible to adjust the brightness of the objects in the synthetic scene image. This can be easily done by adjusting the intensity of one or both of the input beams of the beam pair. In the case of a VR headset, the NLO material does not need to be transparent (in visible light), i.e., it can be opaque (in visible light). 【0037】 The display panel according to the present invention can be considered a transparent microdisplay in the sense that visible light is generated in a local region (cross-input beam intersection volume) where a wavefront can be generated across the display, like a microdisplay. Due to the feature of the present invention where only visible light is generated for the image, problems associated with stray light and scattering in conventional microprojection systems for AR (or VR) do not occur. Instead of having visible beams crossing a lens to form an image in the eye's field of view, we have input beams that are infrared beams, i.e., wavelengths outside the visible range, and by placing a suitable filter layer between the eye and the NLO material layer, we can prevent them from entering the eye in any case. As a result, there are no glares and bright scattering artifacts that would interfere with night vision. Thus, with respect to its light generation, it is similar to a microdisplay VR display device, but different from a microdisplay, a display device that can be used for AR (in visible light) and is transparent can be realized. 【0038】 Color display The above description of the present invention is limited to considering generating a single generated beam from a single pair of input beams. This can be considered from the perspective of explaining a monochrome display or one of the three color components of a color display. In the case of a color display, three generated beams are required, one for each of the three primary colors, and thus there will be three pairs of input beams. 【0039】 According to another aspect of the present invention for a color display, a display device for displaying a synthetic scene image in response to an input of image information, a plurality of beam sources each having a respective frequency and bandwidth for providing a first to a third input beam pair of two input beams, each input beam pair having a pair of frequencies that together become the first to third primary color frequencies, a plurality of beam sources; A display panel including an NLO material phase-matched to the first to third input beam pairs and the first to third generated beams at the first to third primary color frequencies equal to the sum of the frequencies of the first to third pairs of input beams, An input beam routing component arranged to introduce two input beams of each input beam pair into the display panel, wherein each pair of input beams traverses the display panel in first and second paths intersecting each other in a non-collinear geometry within the NLO material, defining an intersection volume in which the generated beam for the input beam pair is generated by sum frequency generation, the input beam routing component; An amplitude modulator operable to amplitude modulate at least one beam of each input beam pair; A phase modulator operable to phase modulate at least one beam of each input beam pair; A controller operable to form a color image frame, each image frame including first to third composite scene images respectively generated by the first to third input beam pairs, the controller being responsive to input of image information to control the amplitude modulator and the phase modulator to form each of the first to third composite scene images by setting the generated beam amplitude, the generated beam wave number vector, and the generated beam wavefront radius of curvature. A display device comprising the above is provided. 【0040】 For a color display, the plurality of beam sources may comprise an emitter array and first to third lasers, the emitter array providing one of the input beams for all three input beam pairs, the other input beam of each pair being provided by one of the first to third lasers, and the first to third primary colors being provided by time slicing such that the first to third generated beams are sequentially generated. 【0041】 Having three pairs of input beams means that there are a total of six input beams, but this is not equivalent to having six independently generated input beams. First, the same wavelength can be shared between different pairs of input beams, and thus the total number of wavelengths required to form three pairs need not be as many as six, and can be reduced to five, four, or three, with three being the minimum possible number. Second, time slicing (i.e., multiplexing) can be used, so that all three colors can be generated by using a common first input beam in combination with a dedicated second input beam for each color, thereby reducing the total number of independently generated input beams required to four. 【0042】 When QPM is used for a color display, the NLO material can be structured with a single lattice period that is used for all three colors. The lack of exact phase matching that will exist for at least two of the three colors can be tolerated and, if necessary or desired, compensated for by varying the overall intensity of one or both of the input beams for each beam pair. The angle at which the input beam traverses the QPM NLO material can also be selected to be different for different colors such that exact phase matching (or at least phase matching that is closer to exact phase matching) is provided for all three beam pairs (i.e., all three colors). Alternatively, to provide quasi-phase matching, the NLO material can be periodically poled (or equivalently, orientation patterned in the case of a non-birefringent material) with three different periods, such that there are three regions of QPM NLO material with different lattice periods, each period being optimized for one of the three colors. Another alternative is to create a single synthetic QPM structure with three phase-matching peaks optimized for the three desired colors of the display. The color-specific QPM regions can be distributed in a regular or irregular 2D array of spot clusters within the image generation plane, e.g., across a display panel, with each spot cluster containing three laterally adjacent color-specific QPM regions, i.e., spots. Since the spots within each cluster are very close together, the fact that they are not at the same location within the field of view will not be perceivable. Alternatively, the three color-specific QPM regions can be distributed along an axis perpendicular to the image generation plane, e.g., across three different depth portions of a display panel. Here, the color-specific QPM regions can be arranged in a 2D array, as in the clusters described above, or can be a continuous region across the display panel as needed to cover the field of view required for synthetic scene image formation. 【0043】 In the following, the present invention will be described mainly with respect to generating a single generated beam, i.e., generating one color component of a monochrome image or a color image. Nevertheless, it will be understood that a color image can be generated from three such generated beams from three pairs of input beams. 【0044】 Image information for forming a synthetic scene image close to the eye In particular, in order to generate a convincing virtual image on the image generation plane, as required for AR glasses, it is necessary to accurately reproduce the image location information encoded in the light from an equivalent natural scene. The image position information is a subset of the entire image information and defines the perception of the 3D arrangement of objects in the scene. As will be described in more detail below, in the case of monocular vision, the image location information in a natural scene is essentially the combination of the location where light from different parts of the natural scene appears in the field of view (i.e., the angle of incidence of light on the eye) and the radius of curvature that the light wave has when it reaches the eye from each point in the natural scene. For nearby objects, the wavefront curvature is perceivable, but for distant objects, there is no perceivable curvature, i.e., the wavefront is effectively a plane wave. For synthetic scene image formation in a display panel close to the eye (e.g., at a vertex-to-vertex distance of about 12 - 15 mm where spectacle lenses are typically placed), the angle of incidence and radius of curvature information from the natural scene can be mimicked by generating light at a specific location on the image generation plane (thus appearing at the correct point in the field of view) and having a specific direction of light propagation from that location (thus allowing the light generated at that location in the image generation plane to enter the eye). The light generated at the image generation plane must not only carry the correct image location information but also have the correct intensity and, for a color image, the correct color. 【0045】 In the context of the present invention, accurately encoding the image location information for the synthetic virtual image involves generating a generated beam light in the NLO material with an accurate location (field angle), an accurate wave vector direction, and an accurate wavefront radius of curvature. To generate an image, of course, the image intensity information must also be reproduced, which can be done in a straightforward manner through appropriate amplitude modulation of one or both of the first and second input beams. In the case of a color image, the image color information must also be reproduced, which is done by generating three generated beams of the three primary colors (e.g., RGB) in the QPM NLO material at each location within the field of view, i.e., providing an appropriate color gamut in the same way as any color projection system. 【0046】 To provide a convincing binocular perception of the synthetic scene, the respective images formed for each of the left and right eyes will match but not be identical. In particular, for the left and right eyes, light from the same point within the synthetic scene is formed at different locations within the monocular fields of view of each eye with different wave vector directions, thereby mimicking natural convergence and the associated depth perception (stereopsis). For any part of the synthetic scene that can be returned to infinity, in an equivalent natural scene, light from distant objects will reach both the left and right eyes as a plane wave at the same location within the respective monocular fields of view of the left and right eyes, so it will be understood that the images formed for each of the left and right eyes can be made the same. However, when generating a realistic visual scene, any intermediate-distance object can be blocked or occluded differently between the left-eye scene and the right-eye scene, and thus, although both are plane waves, they should be understood to emanate from different directions for the two eyes. 【0047】 Image Formation of the Synthetic Scene We disclose two basic techniques for forming a synthetic virtual image on an image generation surface close to the eye. The first is a scanning technique that constructs each image frame pixel by pixel, for example, by rastering. The second is to form the entire image frame simultaneously. 【0048】 Image formation in angular space (input beam scanning / retinal scanning): The first and second input beams are moved both in angle and position with respect to the transverse of their NLO regions to generate a generated beam that is scanned across the monocular visual field of the observer to form an image encoded in angular space using a retinal scanning technique. Thus, the image frame is constructed by rapidly scanning visible light across the retina. More specifically, when the first and second input beams intersect in the NLO region, their intersection volume is small, not limited by diffraction, and is provided with respective beam cross-sections dimensioned to provide a plane wave (through focusing of the eye) that is mapped to a point on the retina. Thus, the intersection volume at any location within the image generation plane corresponds to a single angular field emission (as a wavefront) for constructing a synthetic virtual image on the eye. For each intersection volume, the first and second angles at which the first and second input beams cross the NLO region define the direction of the wavevector of the generated beam. The wavevector direction is set to ensure that the generated beam is directed towards the pupil of the eye. To construct the entire (monocular) image, the angle of one or both of the first and second input beams is changed, so that the input beam intersection volume, i.e., the location where the generated beam is generated, remains essentially constant, but the angle of the non-linearly generated light changes to different angular positions within the (monocular) visual field of the observer's eye. At the same time, the intensity of the generated beam is changed to reproduce the image intensity information, which can be done by amplitude modulation of the first and / or second input beams, and preferably, taking into account any change in the conversion efficiency of the non-linear mixing as a result of the change in the input beam transverse angle as the crossing angle changes across the image generation plane, a complete image frame is constructed over time. When the angle of incidence on the intersection volume is changed, the wavevector direction is also changed to ensure that the generated beam always remains directed towards the pupil of the eye. This can be done by appropriate adjustment of the angles at which the first and second input beams cross the NLO material. The scanning of the intersection volume across the monocular visual field for constructing each image frame may follow, for example, a raster pattern for each line. The raster may include a back-and-forth movement that meanders, for example, from left to right and then from right to left and so on. 【0049】 Full image formation (up - conversion analogy): This approach to image formation can be partially understood by analogy to conventional up - conversion imaging systems. A first input beam (e.g., 1064 nm) contains image intensity information, and a second input beam (e.g., 1550 nm) intersects the first input beam within an NLO material to reproduce an up - converted version of the first input beam (as a generated beam). This analogy is useful but not complete as it does not account for the fact that non - collinear phase matching according to the present invention also enables control of the wavefront curvature of the generated beam such that image depth information is incorporated into the synthetic virtual image. Thus, the image depth information is "added" to the "up - converted version" of the intensity - modulated first input beam (i.e., the generated beam) by phase modulation of the second input beam. This wavefront engineering feature of the present invention has no analogue within conventional up - conversion imaging systems. 【0050】 In one embodiment of the present invention, there is an implementation example that is closest to the up - conversion imaging analogy. The first input beam is amplitude - modulated across its beam cross - section to carry image intensity information that defines the luminance specific to the intersection volume in the generated beam. The second input beam is phase - modulated across its beam cross - section to carry image depth information that defines the wavefront curvature specific to the intersection volume in the generated beam. In other implementation examples that are further away from the up - conversion imaging analogy, both amplitude and phase are modulated across the beam cross - section of one or both of the first and second input beams. For example, the first beam may be amplitude - modulated and the second beam may be amplitude - and phase - modulated. 【0051】 In a first specific example, both the first and second input beams are highly coherent laterally (i.e., in the cross-section of the beam). This is the case when the beam sources for both the first and second input beams are lasers and the beam modulation is performed using either a spatial light modulator (SLM) in transmission or a liquid crystal on silicon (LCOS) device in reflection. Here, we note that both the SLM and LCOS devices can be operated to modulate only the amplitude, only the phase, or both the amplitude and the phase. 【0052】 In a second specific example, the first input beam is incoherent and is generated by a 2D array of incoherent emitters (e.g., an LED or OLED array) that provide amplitude modulation through the ability to drive each emitter independently. As required for phase modulation, the second input beam is coherent and is based on a laser source where the laser beam is phase-modulated (e.g., by an SLM or LCOS device) and optionally also amplitude-modulated as already described for the first specific example. 【0053】 In the context of an RGB display that requires three input beam pairs to generate three colors, the same incoherent emitter array can be used to generate a first input beam that is used for all three colors, and each color has its own dedicated second input beam. The three second input beams are generated by three different laser sources having three different wavelengths that, in total with the wavelength of the incoherent emitter array, generate red, green, and blue light, respectively. Then, time-division multiplexing (i.e., slicing) can be used to order through appropriate drive electronics that synchronize the outputs of the IR emitter array and the three laser sources, through red, green, and blue composite scene image generation. This kind of sequential drive (or on / off modulation) of the three laser sources provides a kind of temporal RGB image formation similar to the color wheel approach in desktop projectors. If the refresh rates of the three colors are high enough, the observer perceives a full-color palette RGB image. 【0054】 Since it is possible to apply both phase modulation and amplitude modulation to a single beam to generate a combined phase- and amplitude-modulated single beam (e.g., using an SLM or LCOS device), further embodiments are possible where the other input beams are not spatially modulated at all. 【0055】 Other features Here, some of the other features of a particular embodiment are briefly summarized. For the delivery of input beams to the display panel, a convenient approach is to introduce the first input beam at one end of the display panel, e.g., the left end, or near it, and the second input beam at the other end of the display panel, e.g., the right end, or near it. However, it would also be possible to introduce both the first and second input beams at the same end of the display panel. This can be enabled by a more complex phase matching scheme that uses an appropriate reciprocal lattice vector for non-collinear phase matching. This can also be enabled by reflecting one of the first and second input beams from the other end of the display panel so that the first and second input beams intersect in the same non-collinear geometry as in the case of end introductions on the opposite sides of the first and second input beams. 【0056】 There are various options for the distribution of the NLO material within the NLO material layer. In some configurations of the device, the NLO material is continuously distributed over the NLO material layer, while in other configurations, it may be attractive to have spots of NLO material arranged in a grid pattern across the NLO material layer. In the context of the grid-of-spots approach, the NLO material means an NLO material that is phase-matched to facilitate non-linear mixing, and thus, when QPM is used, the spots can be local regions where the space is structured for QPM within a layer of the same unstructured NLO material. Note that with the grid-of-spots approach having a limited localization region of the NLO material that generates the generated beam, it is still possible to generate a compelling image as the eye continuously rotates. As long as the localized regions are close enough to each other such that light from at least one such region can be directed towards the pupil of the eye, a compelling image can be formed. 【0057】 In certain embodiments of the present invention, the first and second input beams are each introduced at an angle to one end or near one end of the display panel, with respect to the front and back of the display panel. Each input beam then propagates across the display panel, where non-collinear mixing occurs by one or more reflections. In some embodiments, total internal reflection (TIR) from the interface from the display panel to air is used to move each input beam across the display panel, from where each input beam is introduced to where each input beam intersects the other input beam of its pair, to generate a generated beam. In other embodiments, reflective mirror layers are provided at the front and back of the NLO material layer to reflect at the wavelength of the input beams, such that the input beams cross the display panel by successive mirror reflections. One or both of the front mirror layer and the back mirror layer may be at the front or back of the display panel, respectively, or may be an embedded layer within the display panel. Mirror reflection allows for reflection at a larger angle (i.e., closer to the normal) compared to TIR, which can be beneficial as it can result in a larger poling period and easier manufacturing when quasi-phase matching is used. 【0058】 Additional functional layers can be added to the display panel. For example, it may be attractive to add a front filter layer to the front of the NLO material layer, which absorbs at the wavelength of the input beams (and transmits at the wavelength of the generated beam), such that any scattered input beam light is prevented from reaching the eye. This also helps to ensure that laser safety requirements are met if either or both of the input beams are laser beams. A similar back filter layer can be added to the back of the NLO material layer to prevent input beam light from leaking into the environment. If the display panel is for AR, both the front filter layer and the back filter layer should also be transparent over the visible frequencies, such that natural scene light propagates through the display panel to the eye without attenuation. 【0059】 Further safety measures include providing one or more light sensors, such as photodiodes, outside the display panel to detect abnormal leakage of the input beam light from the display panel indicating that the display panel is structurally damaged. The light sensors may be arranged around the rim or outer edge of the display panel. The operation of the display device can be stopped as necessary, for example, when the display panel is damaged by a deep scratch. Further possibilities for AR are to include a photochromic or electrochromic layer so that the synthetic scene light can be appropriately mixed with the natural scene light and, for example, the transmission of the natural scene light can be reduced under bright ambient light conditions. The darkening operation may be automatic (as in a photochromic lens) or electrically operated. Further, the darkening may be uniform across the entire lens area or may be local, for example, to attenuate light from bright spots in the natural scene. The vision correction lens layer may also be provided either in front of or behind the NLO material layer (or both if two correction lenses are desired). When the correction lens layer is provided, an input beam mirror layer is preferably inserted between the NLO material layer and the vision correction lens layer so that the input beam light does not enter the vision correction lens layer and thus the input beam path is not affected. In other words, the TIR solution is not preferred when vision correction lenses are included. 【0060】 Here, the present invention will be further described by way of example only with reference to the accompanying drawings. 【Brief Description of the Drawings】 【0061】 【Figure 1】 An explanation of how the imaging of a distant object requires wavefront information. 【Figure 2】 A diagram showing how the imaging into the eye through a lens element depends on both the wavefront direction and the wavefront curvature. 【Figure 3】 A diagram explaining why a point light emitter cannot be used for a near-eye display. 【Figure 4】Disclosed is a non-linear optical method for generating a near-eye artificial image that can be superimposed on natural scene light. 【Figure 5】 Illustrates the concept of quasi-phase matching. 【Figure 6】 Illustrates two-dimensional phase matching and can construct different lattice vectors. 【Figure 7】 Illustrates an implementation form of NLO mixing for generating synthetic scene light with a bounce path in a lens. 【Figure 8】 Illustrates non-collinear phase matching. 【Figure 9】 Shows a more detailed diagram illustrating the overlapping non-linear wavefronts in a layered NLO QPM material to generate image information. 【Figure 10】 Shows a path that enables beams with different pairs of crossing angles with respect to the QPM structure to overlap at a common location. 【Figure 11】 Shows a scheme for steering a beam into a device. 【Figure 12a】 Shows a scheme for deflecting a beam without changing its angle. 【Figure 12b】 Shows components for directing light to a desired point within an NLO material layer while changing the angle at which the light approaches that point. 【Figure 13】 Shows a lens structure having an input beam and an emitter that can generate an image with a desired apparent depth of focus. 【Figure 14】 Shows the imprint of a desired wavefront by non-linear mixing of amplitude and phase structured beams. 【Figure 15】 Shows a system for generating a far-field pattern from an image to enable conversion through an NLO material. 【Figure 16】 Shows a multilayer optical composite for fabricating a display system. 【Figure 17】 Illustrates the change of the local Kg position across a glasses-type display system. 【Figure 18】 Shows different lens designs with respect to cross-sections having appropriate advantages. 【Figure 19】An eyeglass lens AR system incorporating a plurality of sensors and a communication unit to an external computing device is shown. 【Figure 20】 An eyeglass system incorporating a series of diode elements around the outside to detect damage to the system is shown. 【Figure 21】 An electrically controlled dimming system incorporated into the lens is shown. 【Figure 22】 A display device incorporating a sub-region of NLO material and having separate red, green, and blue generation regions is shown. 【Figure 23】 The QPM period as a function of the crossing angle for a simple second harmonic generation process is shown, indicating the reason why it is advantageous to use a larger crossing angle. 【Figure 24】 A technique for creating a lens where the beam is incident from one side but maintains a crossing in the opposite direction is shown. 【Figure 25】 A time series display for providing RGB and images with different ROCs to create a convincing full color and a realistic display of depth of field is shown. 【Figure 26】 An inward-facing camera for tracking the pupil position is shown. 【Figure 27】 How to use blanking based on an inward-facing camera to reduce unwanted visual effects is shown. 【Figure 28】 A lens having an absorption layer dispersed around the edge to prevent unwanted reflection of light is shown. 【Figure 29】 A lens having a sub-region of NLO material is shown, where each sub-region includes elements capable of generating red, green, and blue. 【Figure 30】 It is shown that the beam crossing angle across the lens requires local control of the QPM grating direction. 【Figure 31】 A virtual reality (VR) configuration according to the present invention is shown, where a near-eye image is generated without the complexity of using a thin lens. 【Figure 32】Schematic diagram of image generation in an NLO material layer using an incoherent emitter array to generate one of the input beams. 【Figure 33a】 Shows various options for modulating two beams interacting in the NLO material. 【Figure 33b】 Shows various options for phase control across the input beam to provide radius of curvature information. 【Figure 34】 Shows a schematic diagram of a control system showing the input and output. 【Mode for Carrying Out the Invention】 【0062】 Optical Characteristics of the Human Eye and Vision When designing a display device, it is important to consider the optical characteristics of the eye, how the eye is controlled individually and jointly by the brain, and how the brain processes the image information from the eye. 【0063】 To enter the eyeball, light passes through the pupil of the eye, through the cornea, through the lens, and from there to the back of the eyeball where the retina is located. The effective radius of the pupil is controlled by the iris (pupil muscle). The pupil expands and contracts according to the ambient luminance, increasing and decreasing the amount of light entering the eye. The pupil diameter varies over a range of several millimeters, and the range of 2 - 4 mm is the most common. The eyeball diameter is about 25 mm. 【0064】 When glasses are worn, light from distant objects entering the eye passes through the area of the spectacle lens, whose diameter is essentially the same as that of the iris. The distance from the spectacle lens to the front of the eyeball is about 12 - 15 mm and is called the posterior vertex distance or vertex distance. 【0065】 The visual field of a single eye is called the monocular visual field and is typically defined in terms of the angular range of what can be seen by the eye when it is fixed on a point with its vision while the eye is oriented in a particular direction. The monocular visual field extends approximately 60 degrees laterally inward (towards the nose) and approximately 107 degrees laterally outward with respect to the vertical meridian of the eye. With respect to the horizontal meridian of the eye, it extends approximately 70 degrees vertically upward and approximately 80 degrees vertically downward. The binocular visual field is the overlap of the monocular visual fields of the left and right eyes and is thus considerably larger laterally. In this specification, references to "visual field" refer to the monocular visual field. When referring to the binocular visual field, this is stated explicitly. 【0066】 Normal human vision involves a series of movements of the head from the spine and rotations of the eyes within the sockets. The head movements can be thought of as corresponding to roll, pitch, and yaw angles relative to straight-ahead, where yaw is generated by the normal left and right head rotations, pitch is generated by the up and down nodding movements of the head, and roll is generated by the left and right movements of the head. The range of eye rotation in the human eye is approximately +25 degrees up, approximately -30 degrees down, and approximately ±45 degrees left and right. Eye rotation is used to place the retinal image of the most interesting natural scene locations onto the fovea, which is the sub-region of the retina with the highest resolution, and the fovea provides an angular range of approximately 1 to 2 degrees within the visual field (compared to the approximately 18 degrees of the visual field of the entire retina, i.e., the macula). Further, the eyes naturally move around (saccadic eye movements), which serves to keep the eyes moving to avoid the effects of saturation and unresponsiveness. Thus, the eyes are constantly moving around the scene, and the brain performs image processing to cancel out the blurring effects of the eye movements. 【0067】 In the case of monocular vision, considering the light emitted from a given point within a natural scene, the light is effectively a spherical wave having its origin at that point. The spherical wave impinges on the front of the eye and passes through the cornea, pupil, and lens to reach the retina. The refractive power of the lens is adjusted by the ciliary muscle to form a focused image of the scene light on the retina. The lens receives the wavefront, which has a specific central angle and radius of curvature. In the case of light from a distant point (effectively at infinity) in the natural scene, the wave can be treated as entering the eye as a plane wave, i.e., the wavefront is a plane wavefront. For most individuals' monocular vision perception, infinity is any distance greater than about 4 or 5 meters. The angle that the incident light from such a distant point makes with the axis of the eye determines the position on the retina where the light strikes. In the case of light from a nearer point in the natural scene, the wave enters the eye with a finite radius of curvature, which is perceived in monocular vision by the accommodation of the eye. The eye accommodates, i.e., adjusts its focus, to bring the light from the nearer point into a sharp focus on the retina. The human brain associates the amount of accommodation with the object distance, and thus distance perception occurs. (Distance perception also takes into account the convergence effect from binocular vision.) Therefore, in monocular vision, the natural scene can be considered as a collection of point light sources at different positions within the monocular visual field, and the 3D location information of each point is encoded by the input angle of the light (i.e., the 2D position within the monocular visual field) and the radius of curvature of the light in the eye (i.e., the depth). Similar to the 3D location information, of course, the light from each point may further carry intensity information and color information. 【0068】 In near-eye display systems such as AR and VR, the term "eyebox" (or eye motion box) is used to refer to the volume within which the eye receives an acceptable view of the image, whether it is a real image or a virtual image. A simple definition of the eyebox is a 3D volume within which the entire field of view (FOV) is visible for a standard pupil size, taking into account the fact that in human vision, the eye is constantly moving to focus on different regions within the FOV. The specified dimensions of the eyebox in an AR or VR headset are typically larger than the theoretical range of movement of the eye's pupil to cover alignment tolerances and individual differences in pupil distance. 【0069】 Nonlinear optical (NLO) materials One type of nonlinear mixing in NLO materials utilizes second-order nonlinearity χ(2). In nonlinear mixing, there is conservation of energy, which means that the energy of the photons generated by nonlinear mixing depends on the sum of the energies of the photons of the input beams, and considering the Planck relation, it means that the output frequency is given by the sum or difference of the frequencies of the input beam frequencies. 【0070】 Phase matching occurs due to the wavelength dependence of the refractive index of the NLO material. The type of phase matching used in embodiments of the present invention is known as noncollinear phase matching. Noncollinear phase matching occurs when two (or more) optical beams having different propagation directions intersect to form a beam intersection region within a phase-matched NLO material. Phase matching ensures that the wavevector of the generated beam produced by non-linear mixing is the vector sum of the wavevectors of the input optical beams (and the compensating grating k-vector). The term noncollinear phase matching is used in contrast to the term collinear phase matching, which refers to the situation where two input beams overlap spatially and co-propagate such that the wavevectors of the input beams and the generated beam are all in the same direction. The noncollinear mixing of two input beams can be characterized by two angles inherent between the generated beam and each of the two input beams, which together amount to the intersection angle between the two input beams, i.e., the beam intersection angle. In the case of two input beams having wavevectors of equal magnitude, i.e., equal wavelength, the two angles are equal and this angle can be referred to as the beam intersection half-angle. 【0071】 In practice, phase matching is generally implemented using a technique called quasi-phase matching (QPM), which overcomes the fact that the phase mismatch of the refractive index dispersion in a birefringent material does not allow for phase matching unless the birefringent material is very thin. The NLO material is structured to spatially modulate its non-linear properties, typically using periodic poling to create a linear grating structure in which the sign of the second-order non-linearity χ(2) alternates. In the case of noncollinear mixing, the phase matching requirements for the QPM NLO material need to take into account the angles at which each of the input beams cross the QPM NLO material. It should be noted here that the poling period required for noncollinear mixing will be shorter than in the case of collinear mixing since only the components of the wavevectors of each input beam are in the poling direction. 【0072】 Conventional birefringent NLO materials can be used. When the NLO material is quasi-phase-matched, conventional inorganic NLO materials, especially ferroelectrics, and oxides that can be patterned by periodic poling can be used. Examples are lithium niobate (LN) that produces periodically poled (PP) lithium niobate (PPLN), magnesium oxide-doped periodically poled lithium niobate (PPMgOLN), lithium tantalate (LT) that produces PPLT, and potassium titanyl phosphate (KTP) that produces PPKTP. As is well known, the microstructuring of ferroelectric NLO materials with a QPM structure, such as a linear grating, is achieved by electric field poling. Some non-ferroelectric materials can also be processed to spatially modulate their non-linear properties using a recently developed technique called orientation patterning of III-V zinc blende semiconductor crystals, such as GaAs. Oriented-patterned GaAs is not transparent in visible light, but other transparent materials (e.g., oriented-patterned GaN) can be used. Further, similar to using inorganic materials as described above, the polarized NLO material may also be an organic material (or a layer of appropriately non-centrosymmetric organic materials). It is also worth noting that electric field poling of the layer can be performed. 【0073】 Description of the Drawings Figure 1 shows how an image is received by a human eye that shows an eyeball 160 having a pupil 170. The eye receives an image of an object 180 shown as a "stickman". The light scattered from the object 180 is received at the pupil 170 and then focused by the lens onto the retina at the back of the eyeball, where it is detected by rod cells and cone cells. The light can be regarded as light rays 184. Typically, the distance between the eyeball 180 and the object is at least 15 cm (or thereabouts), representing the near point of the eye. The distance can be much larger. For example, in the case of a star, this is millions of kilometers or more. The fact that this distance can be "large" is indicated by line 186 to represent the discontinuity in the scale in the figure. The light scattered from the object can be represented by a wavefront 188, and its inset is shown as 190. Each wavefront is separated by one wavelength λ of light. There is a defined phase relationship between any two points on a given wavefront (shown here as φ_1 and φ_2). In this figure, the wavefronts are shown as being curved with a wavefront curvature corresponding to the distance 192 from the object 180. The lens of the human eye changes shape under muscle control (ciliary muscle), and the shape taken generates a focusing of the incident (generally, curved wavefronts) such that the desired part of the visual field is sharply focused on the retina. 【0074】 Figure 2 shows the eye 160 and an object 180 placed at a distance 192 from the eye 160, and this distance is large. An eyeglass lens 20 is placed in front of the eye. The light scattered from the object 180 that passes through the eyeglass lens is associated with the wavefront 188. Two sets of light rays emitted from points A and B are shown. With respect to the principal optical axis of the eye 160, point A is on the axis and point B is off the axis. The light incident on the eye enters the eye through the pupil, which is controlled by the iris 172. The light rays emitted from points A and B are associated with their respective wavefronts 188A and 188B and are focused by the lens 166 onto two different points 190A and 190B on the retina 168, respectively. The angle that the light rays make with respect to the optical axis of the eye determines where the image is formed on the retina, and the wavefront curvature determines the position of the focus. 【0075】Figure 3 shows that the light radiated from a point source close to the eyeball will have a wavefront curvature that cannot match the wavefront curvature of the light from a natural scene. The eyeball 160 and a distant object 180 are shown. The wavefront curvature of the light from the object 180 corresponds to the large distance from the object. A single piece of optical material (eyeglass lens) 20 is shown as being located near the eyeball and accommodating a point source 192. This point source 192 may be fluorescence from an optical material (such as a dye or rare earth element) or a scattering point (such as dust particles). The point source will generate a wavefront 188 centered on the point source 192 with an associated radius of curvature. Since the point source is located very close to the eyeball (e.g., about 1 cm), the radius of curvature of the light when it enters the eye will be small. This causes two problems to be solved. First, the wavefront emitted from the point source 192 diverges too much to be focused by the lens (i.e., the eye cannot generate a sharp image on the retina). Second, the radius of curvature of the wavefront emitted from the point source 188 will be very different from, for example, the radius of curvature of the natural scene light from the object 180. To solve the first problem, the radius of curvature must have a value that exceeds the threshold necessary for the eye to be able to focus the image on the retina. To solve the second problem, when the AR light source generates an object on the optical material, in order to obtain a convincing visual effect, the radius of curvature of the light representing the virtual object must be the same as, or approximately the same as, the radius of curvature that would be the case if the virtual object were a real object. For example, if the real scene is the green of a golf course with a hole and the virtual object is a golf ball placed in the hole, the radius of curvature of the wavefront representing the golf ball must be made to match the radius of curvature from the real scene near the location where the golf ball is presented. Such matching makes the distance perception realistic for the observer. Since the natural coincidence between the point where the eye focuses (accommodation) and the point where the optical axes of the two eyes intersect for binocular vision (vergence) is broken, a large mismatch between the wavefront curvatures presented to the observer in a single scene is known to be uncomfortable, so such a coincidence is also important for a comfortable viewing experience. This is called vergence-accommodation conflict and induces headache and nausea sensations similar to motion sickness.In this regard, it should be noted that in an AR (or MR) system, an outward-facing camera combined with a distance measurement device such as LiDAR can be used to survey a scene and obtain relevant information regarding the appropriate placement of virtual objects within the scene. The AR headset may desire to overlay an image of a new object onto the natural scene. This may also be to overlay an object that does not exist in the natural scene (for example, to show two tennis players playing a match overlaid on the natural scene of an empty tennis court). The AR headset may also desire to replace the image of a real object within the scene with a virtual object (for example, replacing the image of livestock within a natural scene with the image of a dinosaur). 【0076】 Figure 4 is a schematic diagram showing the operating principle used according to an embodiment of the present invention. An eye 160 having a retina 168 is shown. A transparent (in visible light) NLO material 100 in the form of an eyeglass lens with a front (proximal) surface and a back (distal) surface is placed in front of the eye at the vertex-to-vertex distance. Light exiting the spectral lens from the front surface and traveling towards the wearer's eye travels in the proximal or inward direction. Light exiting the spectral lens from the back surface and entering the environment travels in the distal or outward direction. The inventors refer to this lens as an eyeglass lens by its form, but this does not mean that the lens has any visual correction function. 【0077】 The first and second input beams 70 and 76 are input into the NLO material 100 and intersect such that an intersection region is formed within the NLO material 100. This is a non-collinear geometry. The first and second input beams 70, 76 each have a first and a second frequency. The first and second frequencies may be the same as each other or different from each other. In the intersection region, non-linear mixing occurs by sum frequency generation (SFG), and a generated beam is generated. The generated beam has a sum frequency equal to the sum of the first and second frequencies. 【0078】 The generated beam generated in the intersection region by non-linear mixing is virtually a point light source, and thus generates its own wavefront 84 having a wave vector direction that will send the generated beam light to the pupil. The generated beam is generated only where the beams overlap, i.e., in the intersection region. The NLO material can be selected to be transparent to visible light, and thus an observer can "look through" the material as would be required to enable AR to overlay the synthetic scene image on the natural scene, as schematically shown by the distant object 180 having a wavefront 188 (represented by the discontinuity 186). 【0079】 As proposed, the physical properties of the extended light source generated by non-linear mixing differ from those of the extended light source generated by projection or emission from a fluorescent material in that non-linear mixing by the intersecting beams enables wavefront generation and phase matching, which can be achieved, for example, by quasi-phase matching, and as a result, the generated beam can in principle be generated in an intersection region having an arbitrary desired radius of curvature. 【0080】 Synthetic scene images can be generated in NLO materials in many different ways. One option is to move the first and second input beams, as indicated by reference numerals 71 and 77, such that the angle of beam intersection is a function of position across an image generation region for "writing" the synthetic scene image into the NLO material. To achieve this, it may be necessary to vary the point of incidence of the input beams onto the NLO material. If the first and second input beams are appropriately modulated in terms of intersection angle and amplitude, the synthetic scene image can be constructed, for example, using a certain kind of rastering. Another option is to give one or both of the input beams a patterned cross-section, e.g., 2D amplitude modulation, such that when the two input beams mix in the intersection region, a complete object is generated by the static overlay of the two input beam cross-sections and their angular spectra. Combinations of these two techniques can be used with a hybrid of scanning and beam profile modulation. Within the scope of the general concept of generating light for synthetic scene images by phase-matched nonlinear processes, the wavefront 84 generated by such nonlinear mixing can be given a desired direction and a desired radius of curvature, and it should be further noted that the desired direction is towards the eyeball as illustrated. 【0081】 In summary, an image is generated on the retina 168 that combines the wavefront 188 from the natural scene and the wavefront 84 of the synthetic scene image generated within the crystalline lens, and both wavefronts have the same or a similar radius of curvature. 【0082】 FIG. 5 illustrates an important concept of quasi-phase matching (QPM). QPM is a property that can be applied to certain non-linear crystals by a method known as periodic poling. Ferroelectric non-linear crystals are processed by periodic poling to create a periodic domain structure that modulates the sign of the non-linear susceptibility of the crystal such that the non-linear susceptibility of the crystal alternates between positive and negative as light propagates along a certain direction within the crystal. Thus, periodic poling mimics the effect of joining slices of birefringent materials together to form a stack with alternating crystal orientations, which, of course, is not practical for anything other than extremely long wavelengths, at least in the far-infrared or microwave regions. The ferroelectric material most commonly used for periodic poling is lithium niobate, which is called periodically poled lithium niobate (PPLN) after periodic poling. Another material is periodically poled potassium titanyl phosphate (PPKTP). More recently, a technique called orientation patterning has been developed to enable periodic poling of non-ferroelectric materials such as III-V zinc blende semiconductor crystals like GaAs. Similar to using inorganic materials as described above, the polarized NLO material may also be an organic material. 【0083】 As shown in the upper part of the figure, the QPM NLO material 200 includes an alternative series of layers each having "up" and "down" domains 202 and 204, respectively. The material is structured in this way because the dispersion of the material (the change in refractive index with wavelength) prevents efficient non-linear mixing due to phase mismatch. This is shown in the central part of the figure indicating how the refractive index n depends on the wavelength λ. Generally, the refractive index of a normal NLO material decreases with increasing wavelength (referred to as normal dispersion). In SFG, three wavelengths are involved. The first and second wavelengths λ_1 and λ_2 mix to generate a third wavelength λ_3 having a wavelength such that energy is conserved. The wavelengths of non-linear mixing (in this case, sum frequency mixing) are set as follows. 【0084】 1 / λ_3 = 1 / λ_1 + 1 / λ_2 This corresponds to the conservation of energy. Since the refractive index varies with wavelength, there is a phase mismatch between the three optical wave vectors. The wave vectors are given by the following equation. 【0085】 K_i = 2*πn_i / λ_i where n_i is the refractive index of the NLO material at wavelength i, λ_i is the wavelength. 【0086】 A wave vector matching diagram where K_3 is longer than the vector sum of K_1 and K_2 is shown at the bottom of the figure. This difference occurs because the refractive index n_3 at λ_3 is larger than the refractive indices n_1 and n_2 at wavelengths λ_1 and λ_2. The k_vector mismatch is K_G. In quasi-phase matching, the period of the QPM structure 200 is set as follows. 【0087】 Lg = 2*π / K_G. where Lg is the QPM period. It should be emphasized that in this explanation, it is assumed that the waves co-propagate in the same direction. 【0088】 Figure 6 shows a more complex geometry of a polarized NLO material. Instead of the quasi-phase matching structure being a simple linear grating structure, the nonlinear crystal is processed by poling local regions so that in those domains, the plane of the drawing within the nonlinear crystal has "down" domains pointing in the opposite direction and the "up" domains pointing upward (from the plane of the drawing). The shape of the poled regions can advantageously follow the crystal symmetry of the NLO material and thus, for example, can be hexagonal in the case of LiNbO3. The pattern shown is that of circularly polarized domains in a hexagonal close packing (HCP) pattern, but any other distribution, for example, square or rectangular packing of square or rectangular domains may be selected. Further, the patterning may be aperiodic. The figure shows the physical structure as having two vectors a_1 and a_2 representing the period of the two-dimensional poling. Also shown are two related KG basis vectors b_1 and b_2. Note that these can be used to phase match the interactions in non-collinear wave mixing, and as a result, other selections of angles can be used. All that is required is that there exist suitable nonlinear grating vectors that enable the phase matching diagram to generate a generated beam that is properly oriented, i.e., oriented within the pupil of the eye. 【0089】 FIG. 7 is a schematic diagram of a portion of the NLO material 100, which is spatially modulated within the region 108 with respect to its second - order non - linearity to provide quasi - phase matching to the intersecting input beams 70, 76. The QPM region 108 is schematically shown as a linear grating. The first and second input beams 70 and 76 are introduced into the lens from opposite ends and propagate through one or more reflections, e.g., internal total reflections from the inner and outer surfaces of the NLO material 100 (one reflection from the outer surface is shown for each of the first and second input beams 70, 76 until it intersects in the intersection region that overlaps the QPM region 108). The quasi - phase - matching condition is satisfied in the QPM region 108, and a generated beam 84 is generated by non - linear wave mixing to produce a wavefront that propagates towards the pupil 170 of the eyeball 160. 【0090】 FIG. 8 shows another representation of the input beam intersection region. The first and second input beams 84 and 85 have respective wave - number vectors k_1 and k_2 that generate a generated beam having a wave - number vector k_g that is the vector sum of k_1 and k_2. Ideally, the vector sum of k_1 and k_2, i.e., k_g, must be exactly equal to the wave - number vector associated with the QPM grating period K_G for maximum conversion efficiency. A phase - matching vector diagram is shown in the figure on the right. Note that the required period of the QPM grating can be calculated by simple trigonometry from the input wave - number vectors k_1 and k_2 and by taking into account the dispersion characteristics of the NLO material. 【0091】 FIG. 9 is a schematic diagram of a display panel incorporating the NLO material layer 100. The first and second input beams 70, 76 are directed across the display panel 100 by multiple reflections from mirror layers formed on the front and back of the NLO material layer, whereby the beams 70, 76 cross each other in the intersection region 110 and it can be seen that the generated beam wavefront 84 is generated by non-linear mixing. It will also be understood that the beams 70, 76 are focused at the intersection region 110. The distant object 180 is located several meters away from the eye, as indicated by the discontinuity 186. Natural light is scattered from the distant object 180 towards the eye 160. Thus, the generated beam light representing the synthetic scene is superimposed on the light from the natural scene light. The light enters the eye 160 through the pupil 170 and forms a combined / natural scene image fused on the retina. 【0092】 FIG. 10 shows a display panel having a multilayer structure including an NLO material layer 100, a front mirror layer 136, and a back mirror layer 130. The front and back mirror layers are reflective at the frequency of the input beam and transmissive over the visible frequency range. To show how the angle and position (θ,z) at which the input beam is introduced into the display panel can be varied to change the angle at which the input beam crosses the intersection region, two different input beam paths 70A, 70B for the first input beam are shown. The back mirror layer 138 covers the central portion of the display panel rather than the left end portion of the front surface of the display panel. The first input beam is introduced into this left end portion at an angle and position (θ1,z1) or alternatively (θ2,z2). The first input beam is then scattered from the front mirror layer 136 to the back mirror layer 138 and the like, thereby traversing the display panel to the intersection region 108 including the QPM grating structure 108. The second input beam is similarly introduced at the right end portion (not shown) of the display panel and reaches the intersection region 108 along beam path 76A or 76B. The first and second input beams then cross each other in a non-collinear geometry in the QPM region 108 to generate a generated beam by sum frequency generation. The desired position and angle for introducing the input beam can be easily calculated by backtracking the light rays from the desired point of the intersection region 108 and the angle at which the input beam should cross the QPM region 108. Since the path length of each beam changes as a result of varying the angle and position of injection of each beam into the display panel, a lens or other focusing element (not shown) external to the display panel can be adjusted to ensure that both beams 70, 76 remain focused at the intersection region 108. 【0093】 By providing the mirror layers 136, 138, reflection over a wider angular range than would be possible when relying on internal total reflection from the interface from the panel to the air becomes possible. Here, assuming that the non-linear mixing process is by frequency addition, and further assuming that the normal case where the image is formed in the visible region and the frequencies of the optical beams to be mixed are the same or not very different, the optical beams will be in the infrared (IR) wavelength region, and thus it should be noted that the reflective coating will be designed to efficiently reflect light at the infrared wavelength (or wavelengths) of the optical beams. 【0094】 Alternatively, both the first and second input beams can be inserted from the same end of the lens, and the lens structure can be designed such that the second beam is reflected from the reflection element on the right side of the figure and thus bounces back along path 76A or 76B as shown. Thus, by appropriate routing and / or scanning of the first and second input beams, it will be understood that they can be introduced to propagate through the lens at different angles, which require different introduction points to ensure that the angle change does not move the intersection region. 【0095】 It will be understood that the first and second input beams can be exactly phase-matched to the QPM NLO material only at one specific combination of their respective angles. Thus, scanning the input beams to change their angles will gradually make the phase matching between the intersecting input beams and the QPM NLO material less accurate as the angular divergence from the specified perfect phase-matching angle increases. Thus, the conversion efficiency, and thus the intensity of the mixed generated beam light, will gradually decrease as these angles deviate from the specified angles. The conversion efficiency is a function of the beam angle and will thus change, for example, during rastering while scanning each beam, and this can be compensated for by the temporal variation of the beam intensity during scanning to ensure a non-changing intensity scale for the generation of the generated beam light over the synthetic scene image. 【0096】 It is also generally preferred to make the distance between the front mirror layer and the back mirror layer large enough so that the number of reflections (bounces) is maintained at a relatively low number while allowing a relatively large bounce angle θ. This reduces the overlap between spots (i.e., the beam intersection regions) and thus reduces unwanted guided wave and mode excitation effects. 【0097】 FIG. 11 shows an exemplary optical component for routing an input beam from a beam source (not shown) to a display panel (not shown). The beam 220 from the beam source strikes a first mirror 222A and passes onto a second mirror 222B that is rotatable by an angle φ_1 and then onto a third mirror 222C that is rotatable by an angle φ_2. Thus, different beam paths 224A and 224B can be generated that reach different angles and positions on the display panel. This is merely an example of an optical component that can perform routing, and it will be understood that other techniques can perform the same function. 【0098】 FIG. 12a shows a further exemplary optical component 230 that may be included in an optical component for routing an input beam from a beam source. In the figure, the component is a simple optical glass block. Optical component 230 enables the beam to be translated, i.e., the position of the beam to be changed, without changing the propagation angle of the beam. Optical component 230 is rotatably mounted by an angle θ and is a block of glass or other optical material 230. Optical component 230 is transparent to beam 220. Beam 220 enters optical component 230, refracts at the input surface, and follows the path shown through the block. Upon exit, the beam path is translated downward to beam path 224A, which is parallel to beam path 220 but displaced according to angle θ. Thus, displacement due to angle change is achieved. In terms of polarization, the optical element (glass block) 230 may be arranged to utilize the Brewster angle to minimize unwanted reflections, or may be anti-reflection coated. In miniaturized systems, there are various suitable mounting techniques, such as microelectromechanical system (MEMS) mirror technology, electro-optic systems, and other known beam scanning techniques. The desire to miniaturize and limit the number of moving parts also makes solid-state devices or MEMS devices attractive. 【0099】 FIG. 12b shows components for directing light at a desired point within the NLO material layer while varying the angle at which the light approaches that point. This figure shows an input beam (shown here as coming from the right side of the figure). This input beam is labeled 76 in this case for consistency with other figures and enters a first lens 240A. This first lens 240A is placed at a focal distance (f’) from a rotatable mirror 222. Assuming the input beam 76 is approximately collimated, lens 240A will produce a small focused spot on mirror 222. The light reflected from mirror 222 is directed towards a second lens 240B. The second lens 240B is placed at a distance 2f from mirror 222 and images the small spot from mirror 222 onto a desired re - focused spot labeled 110, which is where this light beam will be delivered within the NLO material where it intersects with other beams. In this example, the distance from lens 240B to the re - focused spot is also 2f. These focal distance values satisfy conventional imaging requirements. When mirror 222 rotates about either axis (refer to rotation angle θ), the resulting beam path will be deflected, but due to the imaging operation of lens 240B, the spot of light will be sent to the same location 110 but approach at an angle set by the rotation of mirror 222. Advantageously, mirror 222 will be rotated such that its center of rotation coincides with the reflective surface of the mirror. It will be understood that various values of focal distance can be used as long as the mirror is imaged within the beam - intersection interaction region. Further, additional optical elements can be inserted into the system to steer the intersection to a desired region of a display element (AR eyewear system). Further, additional optical elements can be used to ensure that wherever the desired interaction region is placed within the display system, it will still have an appropriate size. 【0100】 FIG. 13 shows an AR display device according to an embodiment. Since the first of the input beams has a phase-modulated beam profile and the other of the input beams has an amplitude-modulated beam profile, both amplitude and phase modulation are encoded in the generated beam generated by the non-linear mixing of the first and second input beams in the phase-matching NLO material layer in the intersection region 110 where the two input beams cross. The display panel generates an image of the synthetic object. The information necessary for the synthetic scene image is effectively encoded within the angular spectrum of the generated beam light hitting the eye. The focusing requirement of the lens 166 is such that it acquires the curved wavefront 181 from the natural scene object 180 and brings it to a single focus on the retina 168. The eyeball 162 is shown together with the pupil 170, the lens 166, and the retina 168. The pupil 170 of the eye receives light from the natural scene object 180. The natural scene object 180 scatters the incident light and generates a wavefront 181 having spherical waves. The light of the wavefront 181 passes through the display panel, hits the eye, enters the pupil 170, is focused by the lens 166, and forms a clearly focused image on the retina 168. When light is emitted and the light rays follow from other points on the natural scene object 180, it will exist at different angles with respect to the optical axis of the eye and will be focused at different positions on the retina 168. 【0101】 When the AR headset is worn, the display panel 88 is disposed in front of the eye 160. The display panel 88 is transparent to light scattered from the natural scene as included in the wavefront 188 and further generates an artificial image in response to an intersecting pair of image generation input beams, one pair of which is shown. The image information is delivered to the display panel 88 through a suitable delivery setup. The display panel 88 is shown curved and has an outer surface 92 and an inner surface 90 that forms an interface from the display panel 88 to the air. 【0102】 The illustrated display panel 88 has a composite layered structure having an NLO material layer and front and back mirror layers 136, 138. The mirror layers 136, 138 are reflective at the infrared wavelengths of the input beam. 【0103】 The NLO material layer of the display panel 88 has a QPM region 108 disposed to cover a portion of the display panel 88 shown as a very small portion, although it will be understood that the QPM region 108 can extend across the entire field of view of the eye when the eye is looking straight ahead and / or across the entire field of view of the eye at all possible swivel angles and / or across the entire area of the NLO material layer. 【0104】 Referring to the left side of the figure, an amplitude-modulated first input beam 70 is provided. The first input beam 70 is generated by an emitter array 16 that encodes image information within the beam cross-section by amplitude modulation, i.e., intensity modulation, of the individual emitters of the array by suitable driving. The first input beam 70 is radiated from the emitter array and then reflected by mirror 222A and transmitted through relay lens 240A onto the left end face of display panel 88. Mirror 222A may be deflectable, for example, by rotation about one or two axes to change the path of the first input beam 70. Similarly, lens 240A may be mounted translatably and / or rotatably to change the ray path of the first input beam 70. Further, other mirrors, lenses, or other optical components may be included for beam manipulation of the direction, position, and divergence of the first input beam 70. Thus, the position and angle at which the first input beam 70 is introduced onto the end face of the display panel can be adjusted. The left end face of the display panel is shown as being set at an angle with respect to the normal, i.e., a 90-degree cutoff of the display panel, to facilitate internal coupling of the first input beam 70. Alternatively, the first input beam 70 may be introduced onto the front face of the display panel. Another alternative is to use a grid internal coupling of the first input beam 70, where the grid can cover a portion of an area on either the front or end face of the display panel. After introduction into the display panel, the first input beam 70 traverses the display panel from left to right by successive reflections from mirror layers 136, 138. Referring to the right side of the figure, a phase-modulated second input beam 76 is provided. The second input beam 76 is generated by laser source 18. The second input beam 76 travels from laser source 18 through beam expansion lenses 240A and 240B to first flat mirror 222B, where it is reflected onto second mirror 222C. The second mirror within this beam path 222C is a phase retardation mirror, such as a liquid crystal phase modulator (which enables a spatially varying phase to be imparted to the input), and is used to apply a spatially structured phase modulation to the laser beam.Regarding the introduction into the display panel 88 and the propagation across the display panel 88, the same description as above for the first input beam 70 also applies to the second input beam 76. The second input beam 76 is shown as being introduced at the right end face of the display panel. Then, the first and second input beams 70, 76 intersect in the intersection region 110 where the QPM NLO material is located, resulting in the generation of a generated beam having the wavefront 84 and the frequency at the sum frequency of the first and second input beams 76. The wavefront 84 enters the pupil 170 and then is focused onto the retina 168. The generated beam light is superimposed on the natural scene light 181 from the object 180. By setting the radius of curvature of the wavefront of the second input beam 76 (by imposing a spatially structured phase delay across the beam), the composite scene image can also be given an arbitrary desired radius of curvature such that the composite scene image is a virtual image formed at an arbitrary desired distance from the eye. The radius of curvature of the second input beam 76 can be changed by using Gaussian beam optics, thus reducing the size of the laser beam. Reducing the cross-sectional area of the laser beam will result in an increase in the beam divergence of the resulting non-linear wavefront. 【0105】 Therefore, it is possible to change the virtual image distance of the composite scene image to provide the eye with a fused composite scene image and natural scene image in which the composite scene image realistically fits the natural scene. 【0106】 In the above embodiment, it was implicitly assumed that the QPM NLO material is a simple linear grating. However, in further embodiments, the QPM NLO material may have a more complex spatial profile that incorporates some curvature so as to generate a diverging (or converging) generated beam. In yet another embodiment, it is possible to apply both amplitude information and phase information to either or both of the first and second input beams 70, 76. 【0107】 The emitter array 16 may be an OLED array such as a micro OLED array. There is a controller device 324 that controls the output of the OLED array to generate intensity information within an image. Other options for the emitter array include an inorganic LED array, an array of quantum dot emitters, a liquid crystal device with appropriate backlighting, a liquid crystal on silicon (LCOS) with appropriate illumination, a VCSEL array, and a digital micromirror device (DMD) type of tilted MEMS display device such as a DLP device from Texas Instruments. Note that the system does not require coherence in the amplitude structured beam, which is advantageous not only in enabling an incoherent emitter light source (such as a micro OLED array) but also in reducing speckle in the system. 【0108】 Accordingly, the first input beam 70 is formed by imposing a variable phase modulation on a beam generated by a single emitter, such as a laser having, for example, a Gaussian intensity profile. The second input beam 76 is generated by the emitter array to have a beam cross-section that carries a spatially varying amplitude modulation. In other embodiments, the first input beam 70 can have a cross-section that is spatially modulated in both phase and amplitude, and the second input beam 76 may be modulated in either phase or amplitude, or both. Suitable sources for combined phase and amplitude modulation would be a phase-controlled emitter array such as a VCSEL array with external phase modulation, or an array of independent emitters with appropriate phase control. 【0109】 The advantage of embodiments that enable both the first and second input beams to have a spatial modulation of their intensities is that this can be used to compensate for the fact that different portions of each beam cross-section are approximately accurately phase-matched to the QPM NLO material. The display device may be designed such that the light along the main optical axis of each beam is accurately phase-matched to the QPM NLO material within the QPM region where they intersect, while the phase matching of the cross-beam components with the QPM NLO material gradually becomes less accurate as the distance from the optical axis of each beam increases. Thus, the conversion efficiency will gradually decrease as it moves away from the optical axis of each beam. Therefore, the intensity profile of the beam can be modified to increase the intensity at the ends of each beam such that the intensity scale of the generated beam light is kept constant across the entire intersection region of the beams. 【0110】 FIG. 14 is a schematic view of a beam intersection region 110 where the QPM NLO material 108 is located. The first input beam 70 has a beam cross-section with a substantially Gaussian profile 81 to provide a simple planar phase surface, i.e., there is no spatial phase change across the beam or only the spatial phase change inherent to a Gaussian beam. The second input beam 76 generally has a beam cross-section amplitude- and phase-modulated according to amplitude and phase modulation functions 75, respectively. The first and second input beams 70 and 76, which are in the infrared, interact via the second-order non-linearity of the QPM NLO material 108 to utilize a phase-matching scheme as described above with reference to FIG. 8 using an appropriate non-linear lattice K-vector. The non-linear interaction is, in this case, the sum frequency of the first and second input beams, and the sum frequency generates an output beam 82 having a wavefront 84 that varies in both amplitude and phase as determined by the amplitudes and phases of the input waves to non-linearly polarized light at a frequency that is visible. Thus, the information encoded in the second input beam by the amplitude and phase modulation functions 75 is transferred to the generated beam wavefront 84. 【0111】 Figure 15 is a schematic view applicable to the setup of Figure 14. That is, there is a first input beam 70 having a beam cross-section of approximately Gaussian profile 81 to provide a plane wavefront combined with a second input beam having an amplitude and / or phase modulation wavefront. The second input beam is arranged at a focal length f from the relay lens 242, and is thereby generated by a suitable emitter array 16 arranged at a focal length f from the intersection region 110. This arrangement places the image information contained in the amplitude and phase modulated wavefront from the emitter array 16 on the Fourier plane of the imaging system. Thus, the visible beam resulting from the non-linear mixing of the input beams will contain the same spatial image information as that contained in the second input beam, and this image information is then encoded essentially as a plane wave and proceeds to the eye. The visible generated beam light generated by non-linear mixing will have a wavefront with a radius of curvature following that of the first input beam 70 such that the generated beam light will have an equivalent imaging position determined by the radius of curvature of the first input beam 70. The shortening caused by the finite intersection angle of the two input beams can be compensated using simple trigonometry and can be taken into account in either or both of the beam profile and image configuration. 【0112】 Figure 16 shows a more complex exemplary layer structure for the AR display panel 88. Air 128 surrounds the display, which has a refractive index very close to 1. The figure shows the distal outer side (also called "front" to conform to the colloquial description of an eyeglass lens) of the display panel 88 that has a surface between the air and the display panel, which is the farthest from the eye. The figure also shows the proximal surface (also called "back" to conform to the colloquial description of an eyeglass lens) inside the display panel closest to the eyeball. 【0113】 Each layer is referred to in order from distal to proximal. · Layer 146 is an anti-reflection coating (ARC) layer AR_1 that reduces the reflection of visible light propagating in air 128 from the interface between the panel and air. The outer anti-reflection coating layer 146 / AR_1 should preferably be anti-reflective over the entire visible range of wavelengths to receive light from natural scene light sources. Such ARC coatings are routinely deposited on optical elements such as camera lenses. Also, it is desirable to make this layer "hard" to resist scratches. 【0114】 · Layer 142 is a front vision correction layer (labeled as N1). This is made of an optically transparent material (in visible light) having a refractive power to correct vision impairments of the wearer such as myopia, hyperopia, astigmatism, etc. Vision correction may involve changing the curvature of this layer or incorporating a polarizing filter as is common in sunglasses. Alternatively, layer 142 may be optically neutral, i.e., may not provide an image enhancement effect or may be omitted. Another possible function of the back vision enhancement layer 142 is to provide dimming in response to specific natural light conditions. For example, the vision enhancement layer 142 can incorporate a UV-sensitive photochromic material such as silver halide as used in conventional dimming display panels. Another option for dimming would be to use an electrochromic switchable material such as is sometimes used for the auto-dimming of automotive rearview mirrors. For example, the vision enhancement layer 142 can incorporate tungsten trioxide. In the case of electrochromic materials, a power source that will be brought into contact with the vision enhancement layer 142 to supply power to the dimming function will be required. 【0115】 · Layer 130 is the front filter layer (labeled as A1). The front filter layer generally absorbs the wavelength of the input beam that is in the infrared region, i.e., it is opaque. This avoids light from the input beam being radiated from the front of the display panel into the air 128. The provision of the front filter layer 130 may be required to comply with laser safety regulations when one or both of the first and second input beams are generated by a laser source. The front filter layer 130 can be made of an organic material having absorption at an appropriate infrared wavelength. Further, it is desirable for the front filter layer to transmit visible light so as not to attenuate natural scene light. Such a filter may be known as a heat absorption filter or a "short pass" filter and can be based on glass such as KG1 from Schott. Alternatively, the front filter layer 130 can be based on thin film interference effects, which may be suitable when the light of the input beam is highly monochromatic and narrowband, as is the case when it is generated by a laser. 【0116】 · Layer 150A is a front spacer layer of an optically transparent material (in visible light) provided to separate the IP absorption layer 150 from the next layer 136. · Layer 136 is a front mirror layer for internally reflecting the input beam when the input beam traverses the display panel 88 laterally. By providing the front mirror layer 136, the use of total internal reflection at the interface 146 from the distal panel to the air is avoided, and any possible loss, scattering, or other undesirable effects from the input beam traversing the outer layers 146 - 150 are also avoided. 【0117】 · Layer 150B is another spacer layer that is optically transparent in both visible and infrared (the wavelength of the input beam). This spacer layer 150B is provided to increase the thickness of the material traversed by the input beam, thereby reducing the number of internal reflections ("bounces") required for each input beam to reach the intersection region. 【0118】 · Layer 100 is a layer of NLO material and is quasi-phase matched to the input beams considering the angles at which the input beams will cross each other in a region that will be covered by the eye's field of view when looking straight ahead at least. 【0119】 · Layer 150C is another spacer layer having the same functions and characteristics as spacer layer 150B. · Layer 138 is a back mirror layer for internally reflecting the input beams when the input beams cross the display element 88 laterally and has the same functions and characteristics as front mirror layer 136. 【0120】 · Layer 150D is a back spacer layer of an optically transparent material (at visible and infrared wavelengths of the input beams) provided to separate back mirror layer 138 from the next layer 1814. · Layer 132 is a back filter layer configured to absorb stray light from the input beams, i.e., at the wavelength of the input beams, which will generally have the same characteristics as front filter layer 130 within the infrared. Thus, back filter layer 132 prevents light at the wavelength of the input beams from entering the air 128, thereby protecting the wearer's eyes. 【0121】 · Layer 144 can be a back vision correction layer having any of the functions or characteristics described above for front vision correction layer 142. · Layer 148 is a back anti-reflection coating (ARC) layer AR_2 having the same characteristics as front anti-reflection coating (ARC) layer AR_1 / 146. Its role is to reduce the reflection of visible light propagating in the air 128 from the interface from the panel to the air. 【0122】 One particular advantage of this layer structure is that visual correction can be applied independently of the management of the input beam and the generation of the generated beam optical nonlinear mixing by confining the optical path of the input beam to a portion of the layer stack between the front mirror layer 136 and the back mirror layer 138. That is, visual correction of refractive anomalies in the crystalline lens can be achieved in one or more layers disposed outside the sub-stack 136-138, preferably outside the sub-stack in the distal (front) direction, for example in layer 142 as is done in the illustrated layer stack. In particular, the optical path of the input beam can be managed without considering the visual correction layer. 【0123】 Note that some of the above layers can be omitted. The anti-reflection coating layer is optional. Further, the proximal reflection of the input beam may be performed by total internal reflection from the interface of the front panel to air, in which case the back mirror layer is omitted. Similarly, if total internal reflection is used for the distal reflection of the input beam, the front mirror layer is omitted. Depending on the nature of the light source, e.g., whether the input beam is coherent or not, and their wavelength and maximum output power, the back filter layer may be omitted, or both the back and front filter layers may be omitted. The front and back visual correction layers are also optional. 【0124】 It is worth noting that the largest unwanted reflections are likely to occur between air and the display panel (due to the large refractive index difference with respect to air) and also between the nonlinear layer 100 which can have a very large refractive index difference depending on the choice of NLO material. However, note that additional layers can be added, for example, to provide an anti-reflection coating between the different optical materials involved. 【0125】 FIG. 17 shows an eyeball 162 connected to the brain through the optic nerve 174. The eyeball can rotate within the socket and thus can align its principal optical axis in different directions, such as A, B, and C as shown. The display panel 88 is configured such that the quasi-phase-matching lattice vector Kg varies across the lens, such that in all rotational positions of the eyeball, e.g., with the optical axis along A, B, or C, the QPM NLO material is oriented to generate a generated beam light that will scatter towards the iris. This can be achieved by orienting local sections of the NLO material at appropriate positions and orientations within the display panel. Alternatively, this can be achieved by using a display panel having appropriate curvatures on the front and back surfaces. The curvatures of the front and back surfaces will affect both the direction of the wavevector of the generated beam and the path of the input beam within the display panel. It should be noted that depending on how light is introduced into the panel (relative position within the panel and curvature of the reflective layer), it may be advantageous to arrange Kg in a direction that achieves optimal local phase matching, including the effects of the angle of incidence and the optical axis. 【0126】 FIG. 18 shows that the display panel can have different shapes. The display panel 88A has a planar shape with parallel and flat front and back surfaces. This provides an input beam path that is easy to determine. The display panel 88B has front and back surfaces with the same radius of curvature (i.e., the front and back surfaces have respective curvatures associated with their respective offset central points). The display panel 88C has front and back surfaces that are curved with different radii of curvature such that both circles have a common central point, thereby providing a display panel of a constant thickness. In all cases, the curvature may be on the surface of a cylinder or on the surface of a sphere. Alternatively, different curvatures may exist in two surfaces to provide a more complex curved surface. 【0127】 Figure 19 is a schematic perspective view of a display device in the form of AR glasses 10. The glasses 10 include conventional components such as left and right glasses arms (i.e., temples) 32, 34 and a frame having left and right rims for accommodating the left and right eye lenses. The display panels as described above form each of the eye lenses. The frame includes a bridge, nose pads, and left and right arm locating points that may or may not be hinged. To support its AR function, the glasses arms and the frame may be modified compared to conventional glasses to accommodate the necessary additional components by means of an internal housing and / or an external attachment. These components can include electronic circuit components, batteries, light sources, and optical elements. A wireless transmitter or transceiver 40 may be included to communicate according to a wireless protocol such as the Bluetooth® or WiFi protocol. The arm-mounted wireless transceiver 40 enables the AR glasses 10 to wirelessly communicate with an external control device 44 via a communication path 42. The external control device 44 has its own wireless receiver or transceiver 43, processing capabilities via a processor 46, and an associated memory 48, and an external network communication section 49 that may include WiFi, 4G / 5G, optical LAN, wired Ethernet®, etc. The external control device can also provide a positioning device, such as a global positioning sensor (GPS), and location detection via a map of communication wireless signals and / or wireless network signals. The external control device may in particular be a dedicated stand-alone unit for controlling the AR glasses 10, or it may be a mobile phone, tablet, personal computer, etc. operating with a suitable application (computer program). The external control device can then communicate data with further devices and in particular can have access to remote computing resources within an ad hoc network (e.g., as provided by cloud computing).Therefore, the processing-intensive tasks for controlling image formation in the AR glasses may be delegated to an external control device away from the AR glasses, and optionally, to computing resources more remote to which the external control device communicates. Therefore, by limiting the processing tasks executed on the processor incorporated in the glasses to those that are not relatively processing-intensive, the processing power incorporated in the AR glasses themselves, and thus the associated power consumption, can be kept low. 【0128】 The eyeglass frame houses left and right outward-facing cameras 50 and 52 that face outward for viewing natural scenes. Using a pair of cameras arranged side by side at the same height makes it possible to construct a stereoscopic image, i.e., to enable binocular vision similar to that of the wearer's eyes, thereby making it possible to determine the distance to natural scene objects, at least when the distance is relatively short. The eyeglass frame 10 further houses a rangefinder such as a LiDAR device or a point cloud imaging system 58 to detect objects in the natural scene and determine the distance of the objects from the wearer. In the LiDAR system, the LiDAR device includes at least one laser and a detector, and measures the flight time of the laser light from the LiDAR device to the scattering source and the flight time of the laser light returning from the scattering source to the LiDAR device. The laser beam can be scanned to construct a map of the objects in the natural scene, or multiple laser beams can be generated to obtain information in parallel from multiple points in the natural scene. The movement of the laser beam as a result of the natural turning of the wearer's head can also be utilized to scan the natural scene. In the point cloud imaging system, the laser beam is projected onto the natural scene and imaged by the cameras 50, 52, enabling a model to be constructed with the geometry of the natural scene. The QPM non-linear mixing process makes it possible to set the radius of curvature of the wavefront generated at the wavefront of the composite scene image to a desired value in response to the distance (and position) information measured by the rangefinder (and camera). Therefore, the object information, particularly the object distance information, collected by the camera and the rangefinder has a specific synergistic effect with the AR glasses according to the present invention. For example, when the extended part of the image superimposes (or completely replaces) features on an object in the natural scene, the extended part of the image can be generated using a wavefront having a radius of curvature equal to the radius of curvature that the extended part of the image would have if it were at the measured distance of the natural scene object to which the extended part of the image is related. Similarly, the placement of the extended composite scene image in the natural scene can be accurately performed based on the position information obtained from the camera. The position information and the distance information may, of course, be combined to assist in the segmentation process of the natural scene in order to identify the objects therein. 【0129】 One potentially useful remote computing resource is, for example, to provide mapping information related to the live view of the wearer of a natural scene as an aid in segmenting the natural scene, and also to provide realistic lighting when rendering augmented reality objects. Such mapping information can be, for example, from a mapping application, live satellite imagery, or live flight tracking. 【0130】 The levels of potentially useful computing resources are almost infinite. For example, computing resources can be used to regenerate in real time an entire simulated version of a natural scene in combination with additional mapping information obtained from remote computing resources based on what is observed by cameras and rangefinders. The augmented reality objects can then be placed within the simulated version of the natural scene and volume rendered to provide realistic lighting based on, for example, texture. 【0131】 Figure 20 shows a further view of additional elements that can be combined with a display device in the form of AR glasses. · 100 is a transparent or nearly transparent optical element that includes an NLO material and the aforementioned IR beam that generates visible light directed towards the eye. 【0132】 · 26 is the frame of the glasses and houses the necessary electronics and optical components along with the arms. · 50 and 52 are outward-facing left and right cameras. 【0133】 · 58 is a point cloud or LiDAR system ·60 is an infrared light detector disposed on the rim at the periphery of each lens. At least one light detector is required for each lens. The infrared light detector is provided to detect light leakage from the input beam, and the light should be substantially completely contained within the lens (and before being introduced into the lens within the arms and frame portions of the glasses). Measurement or increase of the light detector signal will be an indicator of damage. The damage can be a deep lens scratch or physical damage to the frame or arm that exposes the lens end face. When infrared light leakage or an excessive amount of infrared light leakage is detected, the glasses will be stopped, i.e., the light source will be turned off, thereby ensuring safety including laser safety when one or more laser sources are used. 【0134】 ·54 and 56 are additional cameras used to track the eye position, facing inward, i.e., towards the eye. This can be used to assist in determining the optical image that needs to be transmitted and to ensure that it is only directed as required for the light to be captured by the iris. This saves power and thus increases battery life. These cameras can also collect information about the observer's stereovision so that convergence and accommodation discrepancies can be addressed and the artificial image can be correctly positioned relative to the natural scene. Importantly, these two cameras can be used to compensate for the different inter-pupillary distances (or equivalently, inter-ocular distances) of different observers and ensure that the light from the display is directed towards the observer's eye pupils. This has the effect of automatically increasing the eye box and overcomes the major difficulties associated with grid-based or other AR implementations. 【0135】 ·62 indicates a magnetic compass element that can be included to help determine the compass direction in which the glasses, and thus the wearer's head, is facing. Again, this information can be processed in the electronics mounted on the glasses or by remote processing and can be fused with the mapping information. 【0136】 ·64 is a set of accelerometers that can be used to track the movement of the glasses and determine parameters such as the roll angle, pitch angle, and yaw angle of the wearer's head. The reference directions for these angles are assumed to be when the head is held horizontally and the cervical spine is not twisted. The accelerometer data can be processed within the on-board electronics or by external processing to track the movement of the glasses and thus the movement of the wearer's head. 【0137】 Although not shown, the AR glasses can be equipped with an ambient light sensor, such as a photodetector with high sensitivity across the visible region, to measure the overall (omnidirectional) brightness of the natural environment. The output from the ambient light sensor can be used to adjust the overall luminance of the synthetic scene image so that the synthetic scene image remains visible under bright light conditions while at the same time not being too bright under dark light conditions such as in a dimly lit room or at night. 【0138】 An inward-facing camera can be used to compensate for the different inter-pupillary distances of different users and the accompanying variations in the position of the eyes relative to the glasses to ensure correct 3D synthetic scene generation and to ensure that light enters the pupils correctly. 【0139】 FIG. 21 shows a display panel 88 provided with an additional layer 140 to provide dimming in response to specific natural light conditions. For example, the dimming layer 140 can incorporate a UV-sensitive photochromic material such as silver halide as used in conventional dimming display panels. Another option for dimming would be to use an electrochromic switchable material such as is sometimes used for the auto-dimming of automotive rearview mirrors, train and commercial aircraft windows. For example, the electrochromic material could be tungsten trioxide. In the case of an electrochromic material, a power source 139 (either a voltage source or a current source) would need to contact the dimming layer 140 via a suitable electrical connection line 141 to power the dimming function. Another option for the dimming layer would be a liquid crystal layer having modifiable transmission, which could be pixel-based. The dimming effect can be applied across the entire display panel (as in a photochromic lens). Another interesting approach is for the dimming effect to be selectively applied only in specific regions of an image. A simple example would be to form a dark rectangular area to provide a suitable background for a synthetic scene image component that displays written information, which otherwise would not be legible when overlaid on part of a bright natural scene. A more sophisticated example is to use data collected from a natural scene, specifically data collected by a forward-facing camera and a rangefinder of the glasses and / or other data collection sources that may be available from a vehicle system when driving a vehicle or an aircraft system when flying an aircraft, to provide selective local dimming based on objects identified by segmentation of the natural scene. The local dimming may exactly match the extent of the objects in the natural scene to attenuate the light from those natural scene objects (either wholly or in part), and the local area dimming may be performed in combination with the insertion of a synthetic scene image in the same local area. There are various examples where it is desirable to attenuate overly bright elements within a natural scene that would otherwise make it impossible or difficult to view the rest of the natural scene. These include attenuating the light from the headlights of an approaching vehicle, for example the light from the sun when looking upwards.Another example is to suppress light pollution when looking at the night sky. Another example is when the AR glasses are configured as driver assistance for night driving. Here, additional information from a camera and radar that are part of the vehicle can be additionally included in the image processing. It is possible to reveal or emphasize natural scene objects that are normally visible during the day but may not be visible at night or are too dark to be clearly seen, thereby improving road safety and generating a night vision augmented reality driving experience. 【0140】 FIG. 22 shows a schematic of the AR glasses 10, where the NLO material within the lens 100 is formed in discrete regions, i.e., spots 104, across the image area, and the spots are embedded in regions of material that are either not NLO material or are NLO material but not periodically poled (or otherwise quasi-phase matched to the input beam). Thus, visible light generation is limited to the spots of QPM NLO material, and the remaining regions between the spots are inactive, i.e., cannot generate any significant beam of visible light by non-linear mixing of the input beam. When the eye rotates within its socket, it is only necessary to ensure that a very small amount of synthetic scene image light reaches the eye. Assuming that the iris can be 4 - 8 mm wide and the lens surface is only about 12 mm from the eye, the NLO material can be confined to very small area spots without degrading image perception compared to full area coverage by QPM NLO material. In other words, considering the total area covered by QPM NLO material, the fill factor of the QPM NLO material can be very small, e.g., 5 - 20%. For example, the spots can be circular, 0.3 - 0.5 mm in diameter, and can be distributed in a hexagonal close packed (HCP) grid with a grid spacing of 1 - 2 mm. The lower part of the figure shows details of a portion 104 of the lens according to one implementation option for RGB image generation. Each spot is actually a composite group of three spots 106R, 106G, and 106B having different QPM structures, which generate red, green, and blue light respectively, thereby enabling the formation of an RGB image. Alternatively, RGB image formation can be achieved when three QPM structures for red, green, and blue are spatially overlaid, i.e., embedded at different depths z within the lens at the same xy location. Another alternative for RGB image formation is to perform poling of the NLO material according to a single, more complex poling pattern that phase matches all three colors of red, green, and blue. This approach of designing a lens with a grid of spots where mixing can occur has advantages over designs having a continuous region of QPM NLO material across the portion of the lens where the synthetic scene image is formed.That is, non-linear mixing, i.e., visible light generation for the composite scene image, can occur only at specific locations where the spots are present. This simplifies beam management, e.g., beam routing and beam modulation. This is because beams traversing outside the spots are acceptable as they do not generate visible light. 【0141】 Figure 23 is a graph plotting the dependence of the QPM grating period P on the angle θ, where the angle θ is the angle obtained by subtracting 90 from the half angle of reflection from the optical axis normal. This example is calculated for the non-linear mixing of two 1064 nm beams to generate green light at 532 nm. When the traversal of the display panel by the input light beam is due to reflection at a shallow angle (right end of the graph), the required period of the QPM grating becomes very small, which can be seen to make manufacturing more difficult and less accurate. For example, at 60 degrees (i.e., 30 degrees), the required periodicity is approximately 400 nm in the UV, and at 70 degrees it is approximately 300 nm. At an angle close to the critical angle for total internal reflection, i.e., approximately 40 degrees for a typical glass-to-air interface, the grating period is approximately 800 nm. When the lens is provided with inner and outer infrared reflective material layers, the input beam traverses the lens by mirror reflection that enables the use of reflections at larger angles. For example, with a grating period of 2 micrometers, the angle is approximately 22 degrees. On the other hand, each reflection is a cause of losses, e.g., due to scattering, and thus the best range of angles to select is where there is a good compromise between ease of manufacture (and accuracy) and having reflections shallow enough to keep the total number of reflections required for the input beams of each beam pair to cross relatively small. 【0142】 FIG. 24 is a schematic diagram of a structure for introducing the first and second input beams 70, 76 into the eyeglass display lens. The first and second input beams 70, 76 are introduced into respective distal and proximal lens layers within the bilayer lens structure at the eyeglass arm or temple side 264 of the lens. Each input beam 70, 76 traverses its lens layer. At the bridge side 266 of the lens, the first input beam 70 of the distal layer is reflected by mirror surfaces 260 (two mirrors at 90 degrees to each other) back into the proximal layer and propagates in a direction opposite to the second input beam 76 to reach the desired region 110 where the beams overlap within the NLO material layer. There is an additional mirror layer 262 within the display element, which is reflective at the infrared wavelengths of the first and second input beams 70, 76. Thus, in this design, the first and second input beams 70, 76 are brought about so as to intersect within region 110 as desired while being introduced at the same end of the lens, which may conveniently be the temple side as shown. It should be noted that other methods can be used, including routing around the lens using suitable optical pipes or waveguide structures for introducing the first and second input beams. Similarly, it is not necessary to use two separate layers (since it is also possible to retroreflect the first input beam 70 with a single layer as in the figure). 【0143】 FIG. 25 shows a time slicing sequence covering both RGB and ROC time multiplexing to generate a single exemplary frame of a synthetic scene image. To form an image frame, a series of time slices are provided over time t, each time slice being unique to one primary color, e.g., red, green, blue (RGB) and one radius of curvature (ROC), e.g., 0.25 meters. Sub-frames of three time slices generate a composite color image with one ROC. To generate a color composite scene image containing objects at multiple distances within the scene, multiple such sub-frames are required, one for each ROC. In a full frame, the composite sub-images are formed at multiple different perceived depths in the composite scene. Thus, by sequentially displaying the associated display fields, a convincing white light effect can be constructed and any color can be rendered, as in a conventional display such as an RGB-based projector. The exemplary radii of curvature shown are 0.25 m, 0.5 m, 1.0 m, 2.0 m, and 4.0 m. In this example, they are shown in a monotonically increasing sequence, and that sequence is repeated. However, it will be understood that any given frame may use any sequence of time slices. In fact, sequences other than those shown may be more efficient for generating by hardware and / or for generating images that are perceived to have higher quality by human vision. The five exemplary ROC values shown have a minimum ROC value of 0.25 meters, which is approximately equal to or just below the near point of normal vision. This value allows even very close objects, and even objects that are slightly too close, to be displayed in focus. The maximum distance in this example is 4 meters, which is virtually the same as the far distance (or infinity) for a normal eye in monocular vision. It will be understood that fewer or more numbers of distances may be selected and it is also possible to adaptively control the distances and frames.Furthermore, the number of ROCs per frame may be changed according to the synthetic scene content. For example, it may be decreased or increased as a particular object enters or exits the synthetic scene. As described above, the ROC value is set by using a non-linear mixing process and changing the crossing angle of the first and second input beams. 【0144】 FIG. 26 shows a schematic cross-sectional view of an eyewear form system comprising standard components of an eyewear frame 26 that houses left and right eyeglass lenses 22, 24 separated by a nose bridge 28 and has left and right temple arms 32, 34. In this embodiment, left and right inward-facing cameras 54, 56 are mounted on or adjacent to the nose bridge 28 to capture images of left and right eyes 162 shown in cross-section through the eyeballs. The inward-facing cameras 52, 54 are shown positioned on the nasal side of each lens. However, it will be understood that alternative arrangements can be used, for example, on the temple side of each lens. The function of the inward-facing cameras 54, 56 is to track the line of sight of a person wearing the AR glasses. By providing inward-facing cameras for line of sight tracking, many advantageous features and functions can be provided. First, the position of the pupil (and thus the line of sight) enables the system to illuminate the appropriate area of the display, so that the observer sees the image in the correct place. Appropriate artificial scene information can be displayed based on the externally-facing camera, information from LiDAR or other predetermined scene information, and the location and orientation (from a compass or accelerometer). Second, the position of the eye can be used to generate a high-resolution image that will be received by the foveal region of the retina, which is the part of the eye with the highest resolution. Thus, a high-resolution image can be generated using a limited number of spatial pixels on a modulator or emitter array, and as the line of sight moves, the input beam can be directed to different regions of the eyewear display lens to generate a high-resolution image at that new position. It should be noted that for creating a convincing synthetic scene, it may also be useful to display information (at a lower resolution but with the correct color and intensity) in the more peripheral parts of the field of view (by appropriate beam steering to other regions of the lens). A third feature of line of sight tracking is that it can be used to darken or erase the synthetic scene as the eye moves its line of sight (over a larger angle), so that unwanted ghost images are not perceived by the observer.A fourth advantage of the inward-facing cameras 54 and 56 for gaze tracking is that all of the light generated can be ensured to be directed towards the pupil of the eye, not only increasing the electrical-to-visual efficiency of the display, but also avoiding the visible light that can be perceived by another person looking at the wearer of the AR glasses display system from being directed towards other parts of the observer's eye (or other places on their eyelids or surrounding skin). A fifth advantage of gaze tracking is that the system can be adapted to compensate for different inter-pupillary distances (separation between the eyes). Since the inter-ocular distances of different people can vary, it is advantageous that the system can be adapted to steer the light towards an individual's eye. This can be achieved by changing the angle of incidence of the two beams onto the NLO material to steer the wavefront direction of the emitted light. 【0145】 FIG. 27 shows a time-slice sequence covering RGB and ROC time multiplexing to generate a single exemplary frame of a synthetic scene image that further includes a saccade motion blanking period 282. When the system is blanked, the illumination (one or both of the input beams) is switched off or to low intensity so that the synthetic scene image is not formed during the blanking period. The decision of when to blank is made by a control system 280 that performs signal processing based on gaze tracking information obtained from the inward-facing cameras 54, 56. The purpose of blanking is to prevent light from being sent to the eye when the eye is swiveling. This avoids unwanted ghost images and stripes in the visual field. This is particularly important in a pulsed system since pulsing can cause a temporary image projected across the retina. Pulsing can be present either because one or both of the beam sources are pulsed, e.g., a pulsed laser, or due to the pulsing effect caused by time slicing of color and radius of curvature RGB, ROC. 【0146】 FIG. 28 shows an absorption layer disposed in a peripheral region of a display element lens through an example of a spectacle lens 20. The spectacle lens 20 is surrounded by an absorption material region 290 disposed therein to absorb the input beam after the input beam intersects within the NLO material and reaches the peripheral region, thereby stopping further propagation of the input beam due to reflection from the lens edge after the input beam has served its purpose. The absorption material may advantageously have a refractive index similar to that of the lens material (to avoid Fresnel reflection) while incorporating an absorption function through molecular absorption, absorption by dyes, or absorption by black absorbing particles. The same design may be used for other forms of display panels, such as a large area display for meetings or a single panel for both eyes such as used in a VR headset form. These designs have in common that there is a peripheral region containing a material that is absorptive to light of the first and second frequencies, such that the first and second input beams are absorbed when the first and second paths reach the peripheral region after intersecting each other within the NLO material. Also, a photodiode 60 is disposed around the lens 20, and the photodiode 60 can be used to check the integrity of the lens structure and provide a switch-off safety function in case of damage, as described above. The lens periphery also includes a non-absorbing region 292 where the input beams 70 / 76 are injected into the lens 20, i.e., the absorption material region does not extend completely around the periphery of the lens 20. 【0147】 Figure 29 (upper) shows a schematic view of the AR glasses 10. The NLO material within the lens 100 is formed in discrete regions, i.e., spots 104, across the image region. The spots are embedded in regions of material that is either not the NLO material or is the NLO material but not periodically poled (or otherwise quasi-phase matched to the input beam). Thus, visible light generation is limited to the spots of the QPM NLO material, and the remaining regions between the spots are inactive, i.e., cannot generate any significant light beam of visible light by non-linear mixing of the input beam. When the eye rotates within its socket, it is only necessary to ensure that a very small amount of synthetic scene image light reaches the eye. Assuming that the iris can be 4 - 8 mm wide and the lens surface is only about 12 mm from the eye, the NLO material can be confined to very small area spots without degrading image perception compared to full area coverage by the QPM NLO material. In other words, considering the total area covered by the QPM NLO material, the fill factor of the QPM NLO material can be very small, e.g., 5 - 20%. For example, the spots can be circular, can have a diameter of 0.3 - 0.5 mm, and can be distributed in a hexagonal close-packed (HCP) grid with a grid spacing of 1 - 2 mm. Instead of a design having a continuous region of the QPM NLO material across the portion of the lens where the synthetic scene image is formed, this approach of designing a lens with a grid of spots where mixing can occur has advantages. That is, non-linear mixing, i.e., visible light generation for the synthetic scene image, can only occur at specific locations where the spots are present. This simplifies beam management, e.g., beam routing and beam modulation. Because the beam crossing outside the spots will not generate any visible light, it can be tolerated. 【0148】 Figure 29 (lower left) shows a first method for realizing each spot. That is, spot 103A is composed of a stack of three different QPM periods, 206R, 206G, and 206B respectively, and is designed to phase-match an appropriate non-linear mixing process for generating three different wavelengths, for example, red, green, and blue light wavelengths respectively. In the figure, the layers are shown as being in contact (as can be achieved by electric field poling), but it will be understood that gaps of inert material may be interspersed. 【0149】 Figure 29 (lower right) shows a second method for realizing each spot. That is, spot 103B is realized as an upper structure grating 208 designed to be able to phase-match simultaneously for three colors, for example, red, green, and blue. This can be manufactured by performing poling of the NLO material according to a single more complex poling pattern that phase-matches all three colors of red, green, and blue. Also, the upper structure grating is shown with its layers slightly inclined from the plane of the NLO material layer. As further described above with reference to FIG. 17, for directing the phase-matched output beam towards the pupil, the grating for each spot may be inclined to vary from spot to spot across the lens region. The same local tilting technique can be used for the multi-stack spot 103A described above. 【0150】 FIG. 30 shows two schematic views of the spectacle lens 20 for AR glasses. The upper part of the figure shows a front view of the lens 20, and the lower part of the figure shows a top view, i.e., a cross-sectional view of the lens. In the front view, the first and second input beams 70A, 70B, and 76A, 76B enter the lens 20 from the left and right sides of the lens 20 through the incident regions 292A and 292B, respectively. The first and second input beams 70A, 70B, and 76A, 76B intersect at two different intersection positions 110A and 110B. When the eye behind the lens rotates, the line of sight will be directed to many such places, and it is necessary to generate the combined scene information that will be received by the pupil of the eye. In this example, the intersection position 110A involves an upward line of sight, and the intersection position 110B involves a downward line of sight. Two different arbitrary first input beam directions 70A, 76A and two different second input beam directions 70B, 76B are shown as examples. The lower part (top view) is a slice through the lens showing the reflection path. As is apparent from considering the combination of the front view and the top view, the respective intersection angles in the non-linear overlapping regions 110A and 110B need to be considered in three dimensions, and it will be understood that the input beam injection direction shown in the front view and the traversal of the lens by reflection are relevant. From this figure, it will be understood that the vector direction intersecting at a given point on the lens 20 depends on the position on the lens where the line of sight is directed. The resulting wavefront needs to be directed towards the observer's pupil, and thus the phase matching diagram (which is a three-dimensional vector here) must be designed so that the light is properly directed. This can be achieved by setting the QPM direction to an appropriate orientation across the surface of the lens so that phase matching in the desired direction can be achieved. 【0151】 The phase matching period and the QPM grating vector will be determined when the lens is manufactured, but it should be noted that there still exists a degree of freedom that can be controlled to ensure that the light resulting from the NLO process is directed towards the eye. The first of these is that at a given crossing position 110A or 110B, the bounce angles θ (see FIG. 10) of the first and second input beams can be independently controlled. Second, the wavelength of the first and / or second input beam can be changed somewhat to change the phase matching condition, which will require that at least one of the beam sources be tunable. As already explained, by using an inward-facing camera, it is possible to ensure that the light from the system is efficiently directed towards the pupil and is adapted to different interpupillary distances. As shown, the glasses can be configured such that the beams are incident on each lens from both sides of the lens at intermediate height positions 292A and 292B. The beams are then directed to cross at different positions 110A and 110B as required. As described above, to ensure that the light is efficiently directed from the lens to the wearer's pupil, it is necessary that the QPM grating vector be locally set at the crossing positions 110A and 110B so as to ensure that the phase matching wavefront (or approximately the phase matching wavefront) moves to the pupil, i.e., so that the wavefront has an appropriate wave number vector. 【0152】 FIG. 31 shows an exemplary lens 300 suitable for a VR goggle. The lens 300 has an NLO material layer 100 embedded therein. The first and second input beams 70 and 76 enter the lens 300 from the distal side, which allows VR without a natural scene to pass through the eye and intersect within the NLO material layer 100 in the intersection region 110. The injection of the input beam from the distal side of the lens 300 is also facilitated by the fact that in VR, the lens 300 can be made much thicker than the thickness possible for an AR. The thickness of the lens 300 may be, for example, 50 mm. The illustrated form of the lens 300 is a roof shape having first and second inclined facets, and the surfaces of these facets form an angle substantially orthogonal to the first and second input beams 70, 76 at the injection points. Other features of the system remain the same as those described above with respect to the end injection of the input beam with a bounce crossing to reach the intersection region where the input beams intersect. 【0153】 FIG. 32 schematically shows image generation in an NLO material layer having a non-linear QPM grating 108 when the first input beam source is an incoherent emitter array 321 that emits a plurality of objects as incoherent infrared light (such as from a monochromatic micro-LED array), thereby acting as the first input beam. A lens 325 is disposed at an intermediate position between the emitter array 320 and the QPM grating 109, each being separated from the other by a focal length f. Three emission regions are shown as pixels 322, 323, and 324. Each of these has the same radius of curvature. An input infrared laser wave 317 that acts as the second input beam and has an intensity profile and a phase profile (corresponding to the radius of curvature) is shown as entering the QPM grating 108 (from the left). The input laser wave 317 can be given three different radii of curvature 318, 319, and 320 at different times. By changing the ROC of the input laser beam from the left and by providing different intensity patterns from the micro-LED incoherent emitter, it is possible to generate a continuous wavefront that can be continuously and rapidly cycled so that different objects appear to the observer to be at different apparent focal positions in the visible non-linear wave emitted from the QPM grating 108 as a result of non-linear mixing of the input beams. 【0154】 Overview of Options for Phase and Amplitude Modulation between Beams The following summarizes possible techniques for performing amplitude and phase modulation functions, in particular whether both are applied to one of the input beams or one is applied to each of the input beams, and also summarizes with respect to whether the beam is a large area beam (or an equivalent beam array generated by an array emitter) or a raster pencil beam. The following is intended to provide a comprehensive overview of the options, but it is not exhaustive, and further variations will be readily understood, for example, by combining elements from the following specific examples. Also, note that the labeling of the two input beams as beam 1 and beam 2 is arbitrary. 【0155】 【Table 1】 【0156】 【Table 2】 【0157】 1 More complex techniques can use an absorptive spatial modulator and can use a Fresnel zone plate that has the advantage of utilizing the coherence of a laser such as an input beam. FIG. 33a shows a series of panels showing some of the techniques summarized in Table A for controlling the amplitude and phase of two input beams. The panels are labeled with reference numbers taken from the corresponding columns of Table A. In the following, the two beams are here labeled 70 for the beam coming from the left towards the interaction region 110 and 76 for the beam coming from the right, for convenience, although it will of course be understood that the directions of the beams may be interchanged as long as the beams cross in the appropriate overlap region 110. To maintain symmetry across the entire glasses, it may also be convenient to design the system such that the system is mirror symmetric with respect to the left and right lenses. For example, in both eyeglass lenses, one of the input beams (e.g., the first input beam carrying the phase modulation) is injected from the bridge side and the other input beam (e.g., the second input beam carrying the amplitude modulation) is injected from the temple (or arm) side. 【0158】 Figure 33a in panel Ai) shows a scheme suitable for a method that is raster scanned and fully beam modulated. This shows two beams 70 and 76 interacting in the non-linear region 110. The first beam is emitted from a laser 18 driven by a laser controller unit 19. The laser controller unit can supply power to the laser and modulate the output power by changing the current when the beam is scanned at an angle to build an image. The figure shows a discontinuity 186 indicating that there are other optical elements (such as for scanning the angle etc.) placed in the optical path. In this embodiment, the power of the laser beam 70 is controlled by the laser controller unit 19. The second beam 76 is shown with a phase profile 80 suitable for controlling the radius of curvature of the resulting non-linear beam (not shown). As explained in Table A, the intensity of the entire beam 70 can also be controlled by placing a modulator in the beam path. The modulation element can be either transmissive or reflective. 【0159】 Figure 33a in panel Aii) shows a beam generated by a laser having a complex amplitude modulation imprinted across the beam. This is different from the previous example where the intensity of the entire beam is modulated, because the intensity across the beam is changed using a spatial light modulator. The figure shows two beams 70 and 76. The first of these 70 is amplitude modulated by a spatially varying reflection amplitude modulator 120 (e.g., a reflection LC-based spatial light modulator). The figure shows an amplitude modulator controller 121 electrically connected to the modulator 120. This example also shows the laser 18 and the laser controller 19. The laser controller can control the total amount of light in the beam, and the modulator controller 121 drives the modulator 120 to generate a spatially varying intensity profile across the laser beam. As mentioned above, the discontinuity 186 indicates that other optical elements will be present in the beam path of 70. All controller elements are controlled by a central controller that controls image formation (not shown). The second beam 76 has a controlled phase profile 80. 【0160】 Figure 33a in Panel Aiii) shows a system in which amplitude - modulated light is generated from a device (such as a micro - LED device) with a plurality of discrete emitters. In this approach, the first beam 70 is generated from discrete pixel - emitter sources from the emitter array 321. The emitter array is driven and controlled by a separate controller 324 that sets the intensity of each radiation element. The light emitted from two pixel positions is shown. Note that generally, the array is a two - dimensional array. In this figure, the light emitted from the emitter array 321 is shown as diverging (as in the case of a micro - LED device), and the lens 240 is arranged to collect the light and direct it in the direction of the first beam 70. It should be understood that the first beam 70 is no longer like a laser beam, but rather radiation emitted by the emitter pixels and directed by the optical system towards the interaction region 110. The emitter array may be incoherent between pixels (such as in the case of a micro - LED array) or coherent between pixels (such as in the case of having an amplitude modulator with a single large laser spot placed on its radiation surface). Alternatively, an array of emitting lasers (such as a VCSEL array) can be used, and in this case too, it may be either coherent or incoherent between each emitter. (Note that VCSEL is a vertical - cavity surface - emitting laser). The provision of a hybrid or composite emitter is described in Table A as Example Aiv), which describes placing a modulator (such as a liquid - crystal modulator) on the radiation surface of a larger emitter (such as a laser or an LED). 【0161】 Figure 33a in panel Av) shows a combined phase and amplitude modulator. This example is a variation of the example shown in panel Aii. Laser 18 and laser controller 19 are shown. The light from the laser is reflected from the combined amplitude and phase modulator 123a. This is controlled by controller 123b which gives both amplitude and phase information to the first beam 70. The combined amplitude and phase modulator 123a can either be a single device, e.g., a modified LCOS device with a phase control layer, or can be implemented using closely spaced separate elements. Again, the device can operate either transmissively or reflectively. The function of changing the radius of curvature of the emitted light is achieved here by applying phase information to the first beam 70, so the second beam 76 no longer has any controlled phase given across it. Thus, the phase 80 is flat across the beam here. However, this beam can still have some more complex phase function (e.g., it can be focused), but it is not modulated when the image is generated to produce the radius of curvature. 【0162】 Figure 33b shows an approach to controlling the phase across the beam according to Table B. It shows a series of panels demonstrating different approaches. The panels are labeled with reference signs taken from the corresponding columns of Table B. The schematic diagrams of the panels focus on showing the role of the second beam (labeled 76 in the figure), although it will be understood that the labeling as the first and second beams 70 and 76 is arbitrary. As in the description of Figure 33a, it will be recognized that it may be convenient to construct a system with mirror symmetry in the left and right lenses of a particular display device (AR glasses). 【0163】 Figure 33b in panel Bi) shows a pair of beams 70 and 76 interacting within the non-linear region 110. The first beam (coming from the left) is labeled 70 and is the beam amplitude-modulated as shown in Figure 33a and Table A. The second beam 76 is the beam coming from the right. It is shown as passing through a transmissive spatial phase modulator 122 that gives the desired spatial phase profile 80 to the beam. There is a scale discontinuity 186 indicating that other elements can be added to the optical path. The spatial phase modulator 122 is controlled by a phase modulator controller 123 that is used to generate a phase pattern corresponding to a diverging beam so as to change the radius of curvature of the light emitted from the NLO region 110, and thus to generate different out-of-focus appearances for an observer of the display device. 【0164】 Figure 33b in panel Bii) is similar to Bi) except that the second beam 76 is reflected from a phase modulator 122 that is similarly controlled by the phase modulator controller 123. The phase modulator 122 may be, for example, a deformable MEMS mirror, a liquid crystal (LCOS) type device, or a thermally deformable mirror element. Note that example Biii) from Table B is not shown in the figure for the sake of similarity to the Bii) configuration. 【0165】 Figure 33b in panel Biv) from Table B is not shown as it involves combining amplitude and phase modulation in a single hybrid element and is, for example, the same as Av. Figure 33b in panel Bv) shows a diffraction intensity modulation technique for changing the radius of curvature of beam 76. This appears superficially similar to Bi, except that the modulator is here a spatially controlled intensity modulator 120. In this figure, it is shown in transmission, but it will be understood that the same can be achieved in reflection. The intensity modulator 120 is controlled by an intensity modulator controller 121. In the simplest configuration, the radius of curvature of beam 76 (equivalent to phase profile 80) can be easily changed by transmission through an aperture of controllable width (e.g., a pinhole). By decreasing the size of the aperture, beam divergence will increase due to diffraction. Similarly, a transmissive pixel control aperture (e.g., a liquid crystal device) can create a transmission pattern (e.g., a pinhole) that will also increase diffraction as needed. A more sophisticated technique is to use a Fresnel zone plate pattern, which will be more optically efficient. 【0166】 Figure 33b in panel Bvi) shows a system based on changing the characteristics of a laser such as beam 76 so as to control the phase 80 (equal to the radius of curvature) across the beam 76 to generate the output of a non - linear interaction region 110 that generates the desired image depth for a display device. The figure shows a pair of lenses 240A and 240B used to create a beam expansion telescope. The two lenses are controllable (by a controller unit 241) and act to modify the refractive power resulting from the pair of lenses. This can be achieved in many different ways including with liquid lenses or by changing the lens spacing by moving elements (as in a zoom lens on a camera). It will be understood that different numbers of lenses can be used and the selection is based on the desired technical performance. Changing the size of beam 76 (propagating over a distance (and other optical elements represented by 186)) results in diffraction spreading as is well known in Gaussian beam optics. Thus, the radius of curvature (phase) of the beam is controlled. 【0167】 Exemplary Control System Inputs, Outputs, and Functions FIG. 34 shows a simplified schematic block diagram of a controller for a display system as described above. Controller 13 is shown to have three main inputs, namely, image data of desired artificial scene information 11, gaze monitoring data from inward cameras 54 and 56, and image data of natural scenes from outward cameras 50 and 52. Controller 13 uses external cameras 50 and 52 to obtain information from natural scenes and determine where artificial scene information 11 should be placed. The inward-facing cameras 54 and 56 are used to determine where the gaze is directed, and the gaze is used to derive where on the display element the overlapping input beams should be directed so that they overlap within the NLO material. This figure also shows the primary outputs of controller 13, which outputs are laser power 19 for the radius of curvature determination beam, OLED controller 324 that generates an intensity-structured beam, phase controller 123 (which sets the radius of curvature / phase across the laser beam), and outputs for controlling input beam steering and focusing optical elements, such as rotatable mirror 222, for example. The described system is simplified, and it will be understood that in a more complex configuration, other information such as accelerometer data, magnetic compass data, LiDAR data, etc. will be incorporated. 【0168】 As an example, a display system as described herein may have any combination or selection of the following inputs, controller functions, and outputs. Inputs: Any of the following inputs and associated controller input interfaces may be provided. 【0169】 · WiFi or 4G / 5G connection to the Internet · Bluetooth® or equivalent (wired, wireless, or optical) · Accelerometer and magnetometer · Forward-facing camera · Rearward (gaze tracking) camera · Integrity sensor photodiode · Information for determining a composite scene including locally derived and received from an external network · Coordinates and geometry of the local space (locally broadcast or from memory) · Lidar / 3D mapping information · Calibration data from the installation protocol · Light scattering from the eye as seen by the inward-facing camera System controller functions: Any of the following functions can be incorporated into the controller. 【0170】 · Determination of the composite scene from external and internal sources · Determination of local geometry and feature identity · Memory and signal processing · Current line-of-sight angle · Eye separation and other eye characteristics, eye dominance, blinking, etc. · Calculation of the overlap angle and phase control · Intensity modulation calculation · Determination of display integrity · External light level · Calculation of the need for real-world light-blocking Output: Any of the following outputs that can be considered control signals, and a related controller output interface can be provided. 【0171】 · Position data, rotation data, and acceleration data to an external user · Camera output to external processing · Controller for the first beam and controller for the second beam · Amplitude modulation · Phase modulation for setting the radius of curvature · Color modulation · Steering optics for directing the beam to a desired spot · Control of photochromic real-world light-blocking · Wavelength combinations In the case of a color display, it is necessary to supply red, green, and blue light. Assuming that the non-linear process is 3-beam SFG, the frequencies of the two input beams must add up to the frequency of the desired color of the generated beam. There are a number of available laser and non-laser sources that provide a near-infinite number of possible combinations of two frequencies that add up to red, green, and blue. 【0172】 One exemplary combination is as follows. 1064nm + 1550nm = 630nm (red) 1064nm + 1064nm = 532nm (green) 1064nm + 780nm = 450nm (blue) This example also shows a practical approach for reducing the number of source wavelengths required. In this example, only three different wavelengths (frequencies), i.e., less than two wavelengths per color, are used. 【0173】 A second exemplary combination is as follows. 1064nm + 1560nm = 632.5nm (red) 1064nm + 1064nm = 532nm (green) 1064nm + 780nm = 450nm (blue) This example shows a further approach for reducing the number of source wavelengths required. In this example, since 780nm light can be generated by second harmonic generation of a 1560nm laser, only two laser sources are required. 【0174】 It will be understood that the choice of the number of lasers used depends on the availability, efficiency, price, lifetime, and physical volume of each technology. Note that it is possible to use a single laser (e.g., 3144nm), which will become 1572nm via frequency doubling, and its third harmonic will be 1048nm. 【0175】 Thus, a third exemplary combination using a single starting laser of 3144nm is as follows. 1048 nm + 1572 nm = 628.8 nm (Red) 1048 nm + 1048 nm = 524 nm (Green) 1048 nm + 786 nm = 449.1 nm (Cyan) Another specific example is as follows. 【0176】 950 nm + 1800 nm = 621 nm (Red) 950 nm + 1210 nm = 532 nm (Green) 950 nm + 880 nm = 456 nm (Cyan) Here, the 950 nm beam can be generated by an OLED array (or other infrared light-emitting diode array), and the other three beams can be generated at 1800 nm, 1210 nm, and 880 nm by their respective laser sources. In this example, there are four different wavelengths. 【0177】 The general desirable features of the present invention are apparent from the above embodiments. That is, since the color of the display is essentially within the visible range, the input beams are within the infrared range, and thus are invisible for most, if not all, of the actual combinations of two wavelengths. Therefore, the scattering problem of conventional microprojector displays is essentially solved because the scattered light from the source beams is in the infrared region and thus invisible. 【0178】 The conversion efficiency in the non - linear process depends on the product of the input beam intensities. For this reason, a CW source can be used, but it is preferred to operate the source in pulse mode to generate high peak power and thus higher conversion efficiency. The laser source for generating the input beams may be operated in any of the following modes: CW, long pulse, nanosecond pulse, picosecond pulse, or femtosecond pulse. The choice of which source to use and in which operating mode to use will depend on various factors including the non - linearity of the NLO material, the damage threshold of the NLO material, the required drive power, form factor, weight, cost, lifetime, etc. 【0179】 Furthermore, any particular beam source used will typically generate a beam having a specific frequency response defined in terms of a single frequency (or wavelength) - the peak frequency - and the bandwidth as a measure of the spread of radiation above and below the peak frequency. In the context of the present specification, the bandwidth of the beam source is defined as the full width at half maximum (FWHM). When a laser is used as the beam source, the bandwidth becomes very narrow, while when a non - laser source is used as the beam source, the bandwidth can become quite wide. Considering the visible and near - infrared wavelengths described in the above examples, typical bandwidths are as follows. For a single - mode edge - emitting LD, the bandwidth is on the order of several nanometers in the case of a distributed feedback (DFB) or distributed Bragg reflector (DBR) edge - emitting LD, and can decrease to sub - pm. For non - laser sources, the bandwidth is generally higher. For example, an OLED can have a bandwidth of several tens of nanometers. In the present specification, generally, reference is made to the "frequency" (meaning the peak frequency) of the beam source, and the bandwidth is not referred to unless relevant. 【0180】 Other aspects and variations Sum - frequency generation (SFG) by non - linear wave mixing is typically associated with the need to use input beams having very high power (e.g., laser power in the mW to watt level), but such high - power input beams are generally not required in embodiments of the present invention due to the high sensitivity of the human eye. For the human eye, optical power at the sub - microwatt level is very bright, so the maximum output power of the beam source required by embodiments of the present invention close to the eye, such as AR glasses or VR headsets, remains very modest. 【0181】 Many of the conventional approaches to optical design known from existing AR and VR display devices can also be used in the display devices implementing the present invention. For example, it is well known to use a relay lens between a modulated light source (e.g., a micro OLED emitter array) and an AR display panel to transmit light to a light-emitting point on the display panel, where the light effectively becomes an angular spectrum. This concept is well known with respect to Fourier optics, for example, in a 4f imaging system having a simple lens arranged at one focal length from a radiation screen and set such that the position where a plane wave is reflected is also separated by one focal length. In the simplest case, assuming that the focal length of the lens matches the focal length of the eye (e.g., about 16 mm), a symmetric 4f imaging system is provided. The display panel is effectively arranged at an intermediate plane or Fourier plane and thus contains angular information. Modifications are possible where the relay lens has a different focal length compared to the eye. Furthermore, the relay lens may also be modified in a more advanced way to compensate for unwanted optical distortions such as, for example, correcting chromatic aberration and other aberrations, or to provide other effects such as providing different magnifications. When designing the relay lens, it should be noted that non-collinear geometry stretches the spatial distribution of the image information in the generated beam compared to the corresponding image information in the first and optionally second input beams that need to be considered. One simple approach is to use the principle of shine-through, in which the lens is tilted to image onto a plane at an angle. As already common in optical design theory, more complex optical lenses and combinations of lenses can be deployed to correct for distortion and aberration. There is also an effect due to the curvature of the display panel, which can be corrected by combining it with determining the local angle in the QPM grating direction and the selection of image information. 【0182】 If the AR display device is satisfactory, a person wearing AR glasses can place arbitrary images anywhere within the field of view. For example, when the wearer looks at their wrist to see a wristwatch, the wearer doesn't actually need an actual watch face to exist. Rather, the watch face can be superimposed on the watch face. Similarly, the mobile phone no longer requires a display. Instead, the AR display can have any display area that superimposes what would be on the display. To perceive the display, the wearer of the AR display no longer needs a screen, and the same approach can be taken for a TV or personal computer that only requires a specified area on a wall or device. The keyboard can also be projected. 【0183】 It will be understood that any of the techniques disclosed herein in the context of AR are also applicable to VR (simply by blocking all natural scene light). When referring to the SFG of two input beams, note that it includes frequency doubling, i.e., second harmonic generation (SHG), and SHG is a special case of SFG where both input beams have the same wavelength. 【0184】 The specific features and advantages of the proposed design are specific to the AR visual system, but the proposed design is also advantageous for the VR visual system. In particular, as a result of being able to design the wavefront curvature of the VR image, VR goggles can be fabricated with a much smaller depth. The ability to place different composite objects at different perceived depths makes the VR image more persuasive and, in the context of the virtual image formed within the VR goggles, is as advantageous as AR glasses in that it allows for the avoidance of vergence accommodation conflicts. 【0185】 Furthermore, the disclosure herein can also be deployed to provide an imaging solution for other optically visible systems where the eye is close to the location where imaging is performed, including but not limited to binoculars, monoculars, telescopes, camera viewfinders, movie camera viewfinders, microscope eyepieces, medical imaging devices, endoscopes, magnifying elements, rifle scopes, and military targeting systems. 【Explanation of Signs】 【0186】 10 Headset (e.g., AR glasses) 11 Information defining an artificial scene 12 Display device 13 Controller system 14 Beam source 16 Emitter array (as a beam source) 18 Laser (as a beam source) 19 Laser control unit 20 Eyeglass lens 22 Eyeglass lens, left eye 24 Eyeglass lens, right eye 26 Eyeglass frame 28 Eyeglass frame, bridge 30 Eyeglass frame, nose pad 32 Eyeglass arm, left (temple) 34 Eyeglass arm, right (temple) 36 Eyeglass lens rim, left 38 Eyeglass lens rim, right 40 Headset wireless transmitter / transceiver 42 Communication path to an external control device 43 External wireless transmitter / transceiver 44 External control device 46 External control device, processor 48 External control device, memory 49 Communication interfaces with an external network (WiFi, Bluetooth®, etc.). 【0187】 50 External camera, left side (for natural scene acquisition) 52 External camera, right side (for natural scene acquisition) 54 Internal camera, left side (for gaze tracking) 56 Internal camera, right side (for gaze tracking) 58 Range finder for natural scene objects (e.g., LiDAR device) 60 Photodetector (for input light leakage detection) 62 Compass mounted on the frame or arm 64 Accelerometer mounted on the frame or arm 70 (Pair of) first input beam / beam path 71 Angular scanning of the first beam 72 First input beam wave number vector 74 First input beam wavefront 75 Amplitude modulation function and phase modulation function on the beam 76 (Pair of) second input beam / beam path 77 Angular scanning of the second beam 78 Second input beam wave number vector 80 Second input beam wavefront 81 Gaussian beam profile 82 Generated beam (from a pair of input beams) 84 Generated beam wavefront 86 Generated beam wave number vector 88 Display panel 90 Display panel, front side (proximal) 92 Display panel, back side (distal) 94 Display panel, left end face 96 Display panel, right end face 100 NLO material layer 103 NLO material upper structure spot 104 NLO material spot cluster 106 NLO material cluster spots R, G, and B (106R, 106G, and 106B respectively). 【0188】 108 QPM grating / spatially modulated region 110 Intersection region of the beam within the NLO material layer 120 Amplitude modulator 121 Amplitude modulator controller 122 Phase modulator 123 Phase modulator controller 123a Combined amplitude and phase modulator 124b Controller for the combined amplitude and phase modulator 124 Input beam routing component 125 Rotatable mirror element 128 Air surrounding the element 130 Front filter layer 132 Rear filter layer 134 Light-shielding layer 136 Front mirror layer 138 Rear mirror layer 139 Electrical control element for the electrochromic layer 140 Photochromic / electrochromic layer (for dimming in bright light) 141 Electrical connection 142 Front vision correction lens layer 144 Rear vision correction lens layer 146 Anti-reflection coating (ARC) layer, front 148 Anti-reflection coating (ARC) layer, rear 150 Front spacer layer 152 Rear spacer layer 154 Dimming layer (e.g., photochromic, electrochromic) 156 Dimming layer, electrical connection wire 160 Eye 162 Eyeball 164 Cornea 166 Lens 168 Retina 170 Pupil 172 Iris 174 Optic nerve 180 Object in the natural scene (stick figure) 181 Wavefront from the natural scene Light scattered from the object 182 Ray 184 Discontinuity of the scale 186 Scattered wavefront 188 Insertion diagram of the scattered wavefront 190 Distance 192 Wavefront from point A 188A Wavefront from point B 188B Focus 190 Focus of light from point A 190A Focus of light from point B 190B QPM structure in the NLO material 200 "Up" domain in the NLO material 202 "Down" domain in the NLO material 204 QPM period 206x (red is 206R, green is 206G, blue is 206B) Upper structure QPM period 208 Input single beam 220 Mirror 222x (222A is the first mirror, 222B is the second mirror, 222C is the third mirror) Output beam path 224x (224A is the first output beam path, 224B is the second output beam path, 224C is the third output beam path) Lens 240x (240A is the first lens, 240B is the second lens) Controller for one or more lenses 241 Relay lens 242 Reflection structure 260 Internal mirror layer in the display 262 Eyeglass arm / strap side of the display element 264 Nose / bridge side of the display element 266 Control system for gaze tracking 280 Blanking period 282 Region outside the lens containing the absorber 290 Entrance region for the light beam 292 Lens for VR 300 Input laser wave 317 Exemplary radii of curvature of the input laser wave 318, 319, 320 321 Incoherent emitter array (e.g., micro-LED array) 322, 323, 334 Pixel examples of incoherent emitter array 324 Controller for incoherent emitter array 325 Lens 400 Controller (including processor) 402 Input for controller 404 Output for controller

Claims

[Claim 1] A display device (12) that displays a composite scene image in response to the input of image information, A first beam source (16) for providing a first input beam (70) of a first frequency and bandwidth, A second beam source (18) for providing a second input beam (76) of a second frequency and bandwidth, A display panel (88) comprising a nonlinear optical (NLO) material (100) that is phase-matched to the first and second input beams and phase-matched to a generated beam (82) with a sum frequency equal to the sum of the frequencies of the first and second beams, Input beam routing components (222, 240) arranged to introduce the first and second input beams into the display panel, wherein the first and second input beams traverse the display panel in first and second paths that intersect each other in non-collinear geometry within the NLO material, defining the intersection volume (100) generated by sum frequency generation of the generated beams, A controller (400) is configured to form the composite scene image by amplitude modulating at least one of the first and second input beams and phase modulating at least one of the first and second input beams in response to the input of the image information, thereby setting the amplitude, wave vector, and wavefront curvature radius of the generated beam. A display device is provided that includes the following. [Claim 2] The display device according to claim 1, wherein the first input beam encodes image intensity information that defines the intensity of the generated beam generated in each intersection volume of a given composite scene image, and the second input beam encodes image depth information that defines the wavefront curvature radius of the generated beam generated in all intersection volumes of a given composite scene image. [Claim 3] The display device according to claim 1, wherein the controller is operable to group the composite scene images into image frames, and different composite scene images in a given image frame have different wavefront curvature radii to represent image elements located at different perceptual distances. [Claim 4] The display device according to claim 1, for displaying the composite scene image in color, comprising a plurality of beam sources including first and second input beam sources, wherein the first and second input beams form one of three input beam pairs generated by the plurality of beam sources, and the first to third input beam pairs combine to generate first to third generated beams having first to third sum frequencies that provide first to third primary colors. [Claim 5] The display apparatus according to claim 4, wherein the plurality of beam sources for forming the three input beam pairs comprises an emitter array and first to third lasers, the emitter array provides one of the input beams for all three input beam pairs, the other input beams for each pair are provided by one of the first to third lasers, and the first to third primary colors are provided by time slicing so that the first to third generated beams are generated sequentially. [Claim 6] The display device according to claim 5, wherein the emitter array is driven to amplitude modulate the input beam generated by the emitter array. [Claim 7] The display apparatus according to claim 4, wherein the NLO material comprises first to third spatially modulated regions, each providing quasi-phase matching to the first to third input beam pairs. [Claim 8] The display apparatus according to claim 7, wherein the first to third spatially modulated regions are formed in the NLO material in the first to third depth portions (206) within the display panel. [Claim 9] The display device according to claim 7, wherein the first to third spatially modulated regions are formed in the NLO material as an array of spot clusters (106), and each spot cluster includes adjacent spots in each of the first to third spatially modulated regions. [Claim 10] The display device according to claim 4, wherein the controller is operable to group the composite scene images within an image frame, and the different composite scene images in a given image frame are images of the first to third primary colors, respectively. [Claim 11] The display device according to claim 4, wherein the controller is operable to group the composite scene images within an image frame, and the different composite scene images in a given image frame are images of the first to third primary colors and have different wavefront curvature radii, thereby representing color image elements located at different perceptual distances. [Claim 12] The display apparatus according to any one of claims 1 to 11, wherein the input beam routing component is adjustable to provide a change in at least one of the first and second paths to change the wave vector direction of the generated beam by changing the angle at which the first and second input beams intersect in the NLO material. [Claim 13] The display apparatus according to any one of claims 1 to 11, wherein the input beam routing component is adjustable to provide a change in at least one of the first and second paths to change the location where the first and second paths intersect in the NLO material. [Claim 14] A display device according to any one of claims 1 to 11, further comprising: an amplitude modulator (120) arranged to amplitude modulate the first input beam; and a phase modulator (122) arranged to phase modulate the second input beam. [Claim 15] A display device according to any one of claims 1 to 11, further comprising a combined amplitude and phase modulator (123a) for amplitude and phase modulation of one of the first and second input beams. [Claim 16] The display apparatus according to any one of claims 1 to 11, wherein the NLO material is spatially modulated with respect to its second-order nonlinearity to provide phase matching through quasi-phase matching. [Claim 17] The display apparatus according to any one of claims 1 to 11, wherein the NLO material is homogeneous with respect to its second-order nonlinearity and provides phase matching through birefringence phase matching. [Claim 18] The display device according to any one of claims 1 to 11, wherein the sum frequency is within the visible range and the first and second frequencies are within the infrared range. [Claim 19] The display device according to any one of claims 1 to 11, wherein the ratio of the first frequency to the second frequency is 0.5 to 2.

0. [Claim 20] The display device according to any one of claims 1 to 11, wherein the display panel containing the NLO material is transparent over the visible range. [Claim 21] The display apparatus according to any one of claims 1 to 11, wherein the NLO material is contained within the display panel in an NLO material layer. [Claim 22] The display device according to claim 21, wherein the display panel comprises a front filter layer (130) positioned in front of the NLO material layer, the front filter layer being opaque with respect to the first and second frequencies and transparent with respect to the sum frequency, and preventing the emission of light from the first and second input beams outward from the display panel. [Claim 23] The display device according to claim 21, wherein the display panel comprises a back filter layer disposed on the back of the NLO material layer, the back filter layer being opaque to the first and second frequencies, and preventing the emission of light from the first and second input beams inward from the display panel. [Claim 24] The display device according to claim 23, wherein the back filter layer is transparent to visible frequencies. [Claim 25] The display apparatus according to claim 21, wherein the NLO material is continuously distributed across the NLO material layer so that phase matching and thus generated beam generation can occur at any location in the NLO material layer. [Claim 26] The display apparatus according to claim 21, wherein the NLO material is distributed across the NLO material layer as an array of spots (106) so that phase matching and thus generated beam generation are limited to the locations of those spots. [Claim 27] The display device according to any one of claims 1 to 11, wherein the display panel comprises a light-shielding layer (134) having an array of pixels that can be individually addressed by electrical control lines to switch between a first state in which the pixels are opaque to visible frequencies and a second state in which they are transparent to visible frequencies, thereby allowing natural scene light to be mixed from a selected region of the light-shielding layer. [Claim 28] The display device according to any one of claims 1 to 11, wherein the first and second paths traverse the display panel by continuous reflection. [Claim 29] The display device according to claim 28, wherein the continuous reflection is due to total internal reflection from the front and back of the display panel. [Claim 30] The display apparatus according to claim 28, wherein the continuous reflection is from front and back mirror layers (136, 138) disposed within the display panel, which are located in the front and back of the NLO material, respectively. [Claim 31] The display apparatus according to claim 30, wherein the front and rear mirror layers are reflective at the frequency of the input beam and transparent over the visible frequency range. [Claim 32] The display apparatus according to any one of claims 1 to 11, wherein the NLO material is transparent across visible light. [Claim 33] The display device according to any one of claims 1 to 11, wherein the display panel has a peripheral region containing a material that absorbs light of the first and second frequencies, so that the first and second input beams are absorbed after crossing each other in the NLO material when the first and second paths reach the peripheral region (290). [Claim 34] The display device according to any one of claims 1 to 11, further comprising one or more optical sensors positioned outside the display panel to detect abnormal leakage of input beam light from the display panel indicating structural damage to the display panel. [Claim 35] The display device according to any one of claims 1 to 11, wherein the first and second frequencies are different. [Claim 36] The display device according to any one of claims 1 to 11, wherein the first and second frequencies are equal such that the sum frequency generation results in the generation of the second harmonic. [Claim 37] The display apparatus according to claim 36, wherein when the first and second frequencies are equal, the first and second beam sources are one and the same beam source, and both the first and second input beams are derived from that beam source. [Claim 38] A wearable headset (10) incorporating the display device according to any one of claims 1 to 11, such that the display panel is positioned in front of the wearer's eyes with an intervertex distance of less than 30 mm for image formation close to the eyes. [Claim 39] A display device (12) that displays a composite scene image in response to the input of image information, A plurality of beam sources (16, 18) having respective frequencies and bandwidths for providing first to third input beam pairs of two input beams (70, 76), wherein each input beam pair has a pair of frequencies that, when combined, result in first to third primary color frequencies, and the plurality of beam sources (16, 18) A display panel (80) including a nonlinear optical (NLO) material (100) that is phase-matched to the first to third pairs of input beams and the first to third generated beams (82) at first to third primary color frequencies equal to the sum of the frequencies of the first to third pairs of input beams, Input beam routing components (222, 240) arranged to introduce the two input beams of each input beam pair into the display panel, wherein each pair of input beams traverses the display panel in first and second paths that intersect each other in non-collinear geometry within the NLO material, defining the intersection volume (110) of the generated beams for the input beam pair generated by sum frequency generation, An amplitude modulator (120) capable of operating to amplitude modulate at least one beam of each input beam pair, A phase modulator (122) capable of operating to phase-modulate at least one beam of each input beam pair, A controller (400) is operable to form a color image frame, each image frame including first to third composite scene images generated by the first to third input beam pairs, and the controller is operable to form each of the first to third composite scene images by controlling an amplitude modulator and a phase modulator in response to the input of the image information to set the generated beam amplitude, the generated beam wave vector, and the generated beam wavefront curvature radius. A display device equipped with the following features. [Claim 40] The display apparatus according to claim 39, wherein the plurality of beam sources comprises an emitter array (16) and first to third lasers (18), the emitter array providing one of the input beams for all three input beam pairs, the other input beams for each pair being provided by one of the first to third lasers, and the first to third primary colors being provided by time slicing so that the first to third generated beams are generated sequentially.

Images