Optical system and optical device

Abstract

An optical system is configured for an imaging system comprising a partially occupied microLED array comprising occupied and unoccupied array positions that cause lost image detail. The optical system comprises at least one optical device, wherein each optical device is associated with and positioned relative to a grouping of microLEDs of the partially occupied microLED array. Each optical device comprises at least one transparent plate member configured to refract a light beam from the grouping of microLEDs in the array onto an image plane. Each optical device is positionable so as to cause the at least one transparent plate member to sequentially refract the light beam along more than one light path. The more than one light path corresponds to both occupied and unoccupied array positions, thereby recovering lost image detail.

Description

Optical System and Optical DeviceCross-Reference to Related Applications

[0001] The present application claims priority from Australian Provisional Patent Application No 2024901521 filed on 23 May 2024, the contents of which are incorporated herein by reference.Technical Field

[0002] The present disclosure broadly relates to an optical system and an optical device for recovering image detail lost when using a sparsely occupied microLED array.Background

[0003] MicroLED technology is based on tiny LED devices. MicroLEDs can be used for displays, such as colour displays, where each LED creates a colour pixel. MicroLED displays can be used in flexible displays with excellent image quality. However, the manufacture of MicroLED displays is both complex and expensive.

[0004] A critical roadblock to the commercialisation of microLED displays, is the lack of a reliable high volume transfer process to bond small dies with the precision required. State of the art flip-chip die bonders typically handle devices in the 50 to 70 micron range and struggle to achieve a throughput of a thousand dies per hour. This is some five orders of magnitude short of the transfer rate required for commercial viability. A partial solution to this problem may include relaxing the tolerances on placement accuracy, enabling the use of technology such as massively parallel pick- and-place.

[0005] Although increased pixel spacing in microLED displays may make the manufacture of these displays more practical, increased pixel spacing would also result in reduced image resolution.

[0006] Optical devices for enhancing the resolution of images usually include a transparent plate member, such as a glass window, that refracts a light beam passing through the plate member. The light beam projects an image made up of rows and columns of pixels. The plate members are tilted back and forth between a first and a second position so that the light beam is shifted. The projected image is shifted by a fraction of a pixel, usually by half a pixel.

[0007] Aberration free image-shifting (static or dynamic) through an angled plate is used for resolution enhancement by sub-pixel image-shifting. In such applications an array of square pixels is typically transformed into an array of pixels sub-divided into rectangles with aspect ratio shortened in the direction tangential to the plate rotation. The number of these rectangles (or sub-pixels) that fit into the original square gives the resolution enhancement factor.

[0008] The enhancement of an image by overlapping of pixels is also known as super resolution projection or imaging. This can be done, for example, by splitting a temporal sequence of frames into two sub-frames, and successive sub-frames are then displaced with respect to each other by a fraction of a pixel, typically by a half or a third of a pixel. The sub-frames are projected in a sufficiently fast manner so that they appear to the human eye as if they are being projected simultaneously and superimposed. The illusion of a resolution can be achieved that appears to the human eye to be twice as high.

[0009] Existing resolution enhancement solutions, however, cannot be used to recover the resolution lost when a partially occupied array of microLEDs is used.

[0010] Any discussion of documents, acts, materials, devices, articles or the like which has been included in the present specification is not to be taken as an admission that any or all of these matters form part of the prior art base or were common general knowledge in the field relevant to the present disclosure as it existed before the priority date of each claim of this application.Summary

[0011] Described herein is an optical device for recovering the resolution of an undersampled image by image shifting, whilst also boosting the brightness and colour gamut of that image and applying the vacated space to integrate hybrid elements. Also described in an optical system comprising a plurality of such optical devices.

[0012] The need for this type of solution has not arisen before because the use of partially occupied microLED arrays for display technology is not the norm. In the past detectors and emitters have been point devices, linear arrays, or rectangular arrays - all manufactured monolithically. Imaging devices employing partially occupied image planes are not known, or in any event not common. The reason for this is that, conventionally, microLED displays are fabricated by lithography and there would be no benefit in less than 100% occupancy. However, recent microLED arrays made using pick and place assembly would benefit from the approach described herein.

[0013] The optical system described herein includes a combination of optical devices for recovering the original resolution (or at least some of the lost resolution) of subresolution images degraded by a sparsely populated pixel array that under-samples the optical field. In addition, the optical system can ameliorate the effects of defective pixels and can also enhance brightness and colour gamut by time division multiplexing adjacent pixels using an image shifter to displace the image by integer multiples of the pixel pitch at an integer multiple of the frame rate.

[0014] In one aspect there is provided an optical system configured for an imaging system comprising a partially occupied microLED array comprising occupied and unoccupied array positions that cause lost image detail, the optical system comprising: at least one optical device, wherein each optical device: is associated with and positioned relative to a grouping of microLEDs of the partially occupied microLED array; comprises at least one transparent plate member configured to refract a light beam from the grouping of microLEDs in the array onto an image plane; and is positionable so as to cause the at least one transparent plate member to sequentiallyrefract the light beam along more than one light path, the more than one light path corresponding to both occupied and unoccupied array positions, thereby recovering lost image detail.

[0015] The optical system may comprise a sparse array of optical devices comprising the plurality of optical devices.

[0016] The optical system may comprise fewer optical devices than displayed pixels in the image plane.

[0017] The plurality of optical devices may be configured to effect integer pixel shifts.

[0018] The at least one optical device may comprise two or more transparent plate members. The two or more transparent plate members may have different thicknesses to effect different optical shifts. The two or more transparent plate members may be positioned at different tilt angles to effect different optical shifts. The transparent plate members may consist of monolithic wedges.

[0019] The at least one optical device may comprise a rotating assembly. The rotating assembly may comprise a rotor. The rotating assembly may comprise a frame supporting the at least one transparent plate member, so that as the rotating assembly rotates around an axis orthogonal to the image plane the at least one transparent plate member rotates into and out of an optical path associated with the image plane.

[0020] The optical system may further comprise at least one actuator controlled to effect time division multiplexing of adjacent pixels. The actuator may utilise one or more of: electromagnetic actuation, piezoelectric actuation, and electrostatic actuation.

[0021] The at least one optical device may cause image shifting thereby displacing an image provided by the microLED array by integer multiples of a pixel pitch at an integer multiple of a frame rate.

[0022] In another aspect there is provided an imaging system comprising: a partially occupied microLED array comprising occupied and unoccupied array positions that cause lost image detail; and an optical system comprising: at least one optical device, wherein each optical device: is associated with and positioned relative to a grouping of microLEDs of the partially occupied microLED array; comprises at least one transparent plate member configured to refract a light beam from the grouping of microLEDs in the array onto an image plane; and is positionable so as to cause the at least one transparent plate member to sequentially refract the light beam along more than one light path, the more than one light path corresponding to both occupied and unoccupied array positions, thereby recovering lost image detail.

[0023] The occupied and unoccupied array positions may cause an increase in pitch size. The pitch size may be more than 5 micron.

[0024] The optical device may comprises n transparent plate members, and the partially occupied microLED array may have 1 / n occupancy.

[0025] The partially occupied microLED array may incorporate photodetectors in some or all of the array positions unoccupied by microLEDs.

[0026] The partially occupied microLED array may incorporate optical modulators in some or all of the array positions unoccupied by microLEDs.

[0027] The partially occupied microLED array may incorporate optical amplifiers in some or all of the array positions unoccupied by microLEDs.

[0028] The partially occupied microLED array may incorporate analogue and / or digital electronic components in some or all of the array positions unoccupied by microLEDs.

[0029] The partially occupied microLED array may incorporate acoustic transducers in some or all of the array positions unoccupied by microLEDs.

[0030] Deviations in array order, such vacancies and / or dislocations, may be mapped and compensated for in software and / or firmware.

[0031] Throughout this specification the word “comprise” or variations such as “comprises” or “comprising”, will be understood to imply the inclusion of a stated element, integer or step, or group of elements, integers or steps, but not the exclusion of any other element, integer or step, or group of elements, integers or steps.Brief Description of Drawings

[0032] Embodiments of the disclosure are now described by way of example with reference to the accompanying drawings in which:

[0033] Figure 1 is a schematic representation of the operation of an embodiment of an optical system.

[0034] Figure 2 is a schematic representation of an embodiment of an optical device that forms part of the optical system in Figure 1.

[0035] Figure 3 A is a schematic representation of light refraction where the angle of incidence is zero.

[0036] Figure 3B is a schematic representation of light refraction where the angle of incidence is not zero.

[0037] Figure 4 is a schematic illustration of the operation of an embodiment of an optical device.

[0038] Figure 5 is a schematic illustration of the operation of another embodiment of an optical device.

[0039] Figure 6 is a schematic representation of a resolved image formed using an embodiment of an optical system.

[0040] Figure 7A is a schematic representation of a two-plate embodiment of an optical device.

[0041] Figure 7B is a schematic representation of a four-plate embodiment of an optical device.

[0042] Figure 8 is a schematic representation of a three-plate embodiment of an optical device.

[0043] Figure 9 is a schematic representation of a resolved image formed using the three-plate optical device of Figure 8.

[0044] Figure 10 is a schematic representation of the operation of one four-plate embodiment of an optical device.

[0045] Figure 11 A is a schematic representation of another four-plate embodiment of an optical device.

[0046] Figure 1 IB is a schematic representation of pixel occupancy associated with the four-plate optical device of Figure 1 IB.

[0047] Figure 12 is a schematic representation of an RGB pixel (sub-pixels Red, Green and blue) with a highlight of the usable (enhanced resolution) area.

[0048] Figure 13 is a schematic representation illustrating the enhanced colour gamut achievable using an embodiment of a three-plate optical device.

[0049] Figure 14 illustrates yet another embodiment of an optical device.

[0050] Figure 15A illustrates operation of the optical device of Figure 14 positioned at a first angle.

[0051] Figure 15B illustrates operation of the optical device of Figure 14 positioned at a second angle.

[0052] Figure 16 is a schematic representation of the pixel displacement achieved using the optical device of Figure 14.

[0053] Figure 17 is a schematic representation of one embodiment of a piezoelectric actuator forming part of an optical device.

[0054] Figure 18 is a schematic representation of another embodiment of a piezoelectric actuator forming part of an optical device.

[0055] Figure 19 is a schematic representation of an embodiment of an optical device in the form of a hybrid projection and imaging device.Detailed Description

[0056] System overview:

[0057] Referring to Figure 1 of the drawings, an optical system 100 is configured for an imaging system that comprises a partially occupied microLED array 104. The partially occupied microLED array 104 comprises occupied array positions 106 and unoccupied array positions 108 that cause lost image resolution or reduced image detail. The optical system 100 comprises at least one optical device 110. Each optical device 110 is associated with and positioned relative to a grouping 111 of microLEDs 112 of the partially occupied microLED array 104. In some embodiments, the grouping consists of all the microLEDs of the array 104 (this is illustrated in Figure 1). In other embodiments, the grouping includes a subset of the microLEDs of the array 104. Each optical device comprises at least one transparent plate member 114 configured to refract a light beam 116 from the grouping 111 of microLEDs 112 in the array 104 to form an image 118 in an image plane 102. Each optical device 110 is positionable so as to cause the at least one transparent plate member 114 to sequentially refract the lightbeam 116 along more than one light path 120, the more than one light path 120 corresponding to both occupied array positions 106 and unoccupied array positions 108, thereby recovering lost image detail.

[0058] The plurality of optical devices 110 is in the form of a sparse device array 130 of optical devices. The device array 130 is considered “sparse” because there are fewer optical devices than displayed pixels 132 in the image plane 102.

[0059] In the illustrated embodiment, one optical device 110 is associated with a grouping comprising a plurality of microLEDs in the array 104 (i.e., the device array 130 consists of one device 110). The optical device 110, comprising, for example, two plate members that consist of monolithic wedges, will shift the image of the entire pixel array in parallel with each wedge.

[0060] In other embodiments, the ratio of optical devices 110 to microLEDs may be smaller. In one example, the array 130 has two devices 110, each associated with half of the resulting displayed image. In another example, the number of optical devices 110 may be the same as the number of microLEDs in the array 104 (the number of microLEDs being the same as the number of occupied array positions 106). In such embodiments, the light beam from each microLED is refracted by an associated optical device, thereby forming the image visible on the display.

[0061] In some embodiments, the unoccupied array positions 108 may be used to accommodate other components. For example, for some transceivers photodetectors may be placed in unoccupied array positions. Another example is imaging processors where logic gates or memory elements may be placed in unoccupied array positions.

[0062] Device and Plates:

[0063] Each optical device 110 comprises at least one transparent plate member 114 configured to refract a light beam 116 from the array 104 to form an image 118 in the image plane 102. This effect is illustrated in Figures 3A and 3B, where a plate member114 of thickness T composed of a material with refractive index n, effects a lateral image shift Ax as derived by application of Snell’s law:

[0064] The optical device is configured to effect integer pixel shifts. The reason for this is that a partially occupied microLED array would result in unilluminated pixel positions on the display corresponding to the unoccupied microLED array positions.

[0065] These unilluminated pixel positions are compensated for using integer pixel shifts, as partial pixel shifts (i.e. non-integer shifts) would only partially illuminate the associated pixel positions on the display. The resulting visible shifted pixel may be referred to as a “virtual pixel” or an “interstitial pixel” on account of the resulting visible pixel being a type of mirage or optical illusion created to fill the gap.

[0066] In some embodiments, the optical device may comprise a rotating assembly, for example in the form of a rotor. Figure 2 of the drawings illustrates an embodiment of an optical device 110 that has two plates 114, supported by a frame 202. The first plate 210 and the second plate 212 are rotated into and out of the optical path 204 continually, around an axis 206 orthogonal to the image plane 208. In the exemplary embodiment, the optical device is intended to work in constant movement.

[0067] Figure 4 of the drawings is a schematic representation of an embodiment of an optical device 110 having two plates where the image is passed through alternating plates with thickness differences sufficient to effect the desired shift: T2-T1=AT.

[0068] In alternative embodiments, as illustrated in Figure 5, the two plates vary in tilt angle to effect the integer pixel shift.

[0069] The plate angle and plate thickness variables can be altered in combination to achieve minimum aberrations for the desired image shift.

[0070] The resolved image 600 formed by the alternating insertion of plates at alternate angles, is illustrated in Figure 6 of the drawings. The first pixel array 602 (t=0, 9=91) corresponds to the initial image as seen through the first plate 210 mounted with a first tilt angle 91. The second pixel array 604 (t=l 9 =92) illustrates the image as seen at a later time through the second plate 212 mounted with a second tilt angle 92, once the second plate 212 has rotated into the optical path 204. It can be seen that the entire pixel array (as created by the partially occupied microLED array) appears to have shifted one pixel to the right, from the first array 602 produced by the first plate 210 to the second array 604 produced by the second plate 212. This alternating image shift persists with every rotation and if the rotation rate is sufficiently high, persistence of vision will meld the alternate images into a single perceived image 600 with a resolution double that of the original image, in the direction of image shift.

[0071] The resolution achieved in this way is not “super-resolution”. In this example the microLED array 104 is at half resolution, and this half resolution is compensated for by the optical device 110 (in the direction of image shift), so that the resolution sacrificed by partially populating the microLED array 104 is recovered by means of interlacing alternate shifted images. In effect a deficiency in spatial resolution is compensated for by sacrificing the ultimate frame rate achievable. Given that many imaging systems have frame rates far in excess of that required, this is a sacrifice that is often acceptable.

[0072] Some issues regarding reduced frame rates can be ameliorated by adding extra image shifting plates to the rotor. Figure 7A of the drawings illustrates a two-plate optical device 700. Figure 7B of the drawings illustrates a four-plate optical device 702.

[0073] Operation of such variants may be understood with reference to Figure 8 of the drawings, which illustrates a three-plate embodiment of an optical device 800. In this example, the partially occupied microLED array has one third occupancy. The system recovers at least some of the lost resolution (and in some embodiments may recover the full potential resolution) by multiplexing the images seen through each ofthe three plates. The upper half of Figure 9 illustrates image sub-fields 902, 904 and 906 corresponding to intervals 0,1 and 2 during which the optical path traverses plates at tilt angles 01, 02 and 03 respectively. These image are sequentially displaced by 0, 1 and 2 pixels, in the horizontal direction. The resulting multiplexed image 900 is perceived as a ‘full-resolution’ image.

[0074] Figure 10 illustrate the mode of operation of a four plate implementation with plates of alternating thickness, allowing occupancy of the array, with only half of potential pixel positions active. The apparent image at the image plane (shown in dashed outline) is illustrated in sequence, at the bottom. The active pixels appear in alternate positions twice in each rotation cycle, as the shifting plates with two different thicknesses occlude the image plane twice in each rotation. If each one of the four plates has a different thickness or tilt, as illustrated in Figure 11 A, even sparser occupancy is possible. This is illustrated in Figure 11B where each plate confers a distinct apparent pixel position. The translation pitch is four pixel positions, and the translation step is one pixel position.

[0075] Figure 12 shows another implementation that achieves sparse device populations using a two-plate configuration as illustrated in Figure 7A. The number of devices required for a given resolution is reduced. The sparse microLED array used increases production yield, enables the use of pick and place technology, and enables the use of larger optics for individual subpixels. The optical system allows for easy routing and connectivity between microLEDs. Colour mixing and brightness are enhanced by moving the pixels at P / (2x3). In other words, the size of a sub-pixel becomes the size of a pixel. Advantageously, this relaxes the alignment tolerances of LED wafer bonding to the CMOS backplane considerably.

[0076] Figure 13 shows another implementation that achieves enhanced colour gamut. A three-plate optical device is used, where each plate has a different tilt angle. The resulting usable display is slightly smaller than the full size of the arrays, with the benefit of the perceived colour shift caused by the different tilt angles causing each pixel to be perceived as an RGB (red green blue) emitter.

[0077] Various mechanisms may be used to change the effective optical characteristics of the one or more plate members that refract the light beam(s) from the microLED array. For example, an alternative opto-mechanical embodiment of an optical device 1400 is illustrated in Figure 14 that (in this example) uses one plate 1402 supported by a frame 1404 that rotates about an axis 1406 parallel to the image plane 1408 of the display 1410. This might be the preferred embodiment for systems where space in the image plane is restricted, but space orthogonal to the image plane is not. The mode of operation of this concept may be understood with reference to Figures 15A and 15B, where it can be seen how the length of the optical path 1502 is a function of the tilt angle 0 of the plate 1402. The apparent pixel positions are shown in Figure 16. As the plate 1402 rotates from -9 to 0 to +9 each pixel is displaced between three positions. This allows a pixel occupancy of one third for this particular implementation.

[0078] Actuator:

[0079] The trade-off for improved spatial performance is a penalty in temporal performance. The image shifting device operates at an integer multiple of the frame rate of the pixellated device. This limits the applicability of most voice-coil image shifters driven by the Lorentz force, given their low scanning frequencies. Faster image shifters based on technologies such as piezoelectric actuators or ferroelectric liquid crystal on silicon (LCoS), would be preferred in combination with commercial pixellated devices.

[0080] There are a range of image-shifting devices commonly used to achieve superresolution through sub-pixel image displacement (commonly known as ‘pixel shift’). Typically Lorentz coil actuators are used to implement this pixel shifting, but the greater distances needed for integer pixel shifts, coupled with the high frame rates required for this shift, encourage re-evaluation of component choices.

[0081] An alternative implementation based on a piezoelectric actuator 1700, is illustrated in Figure 17. A plate 1702 with a thickness (T) of 550 microns, tilted by just1 degree, will give a 3 micron pixel shift if the refractive index n is about 1.5. If the plate 1702 is driven by a piezoelectric actuator 1700 placed at a distance (L) 1mm from the fulcrum 1704 supporting that plate, the absolute displacement required to implement that 3 micron pixel shift is just 17.5 microns. The benefit of such small displacements is that they can be driven at very high rates. Commercial examples of such devices can easily attain resonant frequencies of 76 kHz.

[0082] Three alternative modes of actuation for an image shifting plate may be used:1. Electromagnetic (Lorentz Force) actuation, prioritising force and distance over speed;2. Piezoelectric actuation, optimising the balance between force, distance and speed; and3. Electrostatic actuation, prioritising speed over force and distance.

[0083] The first mode is based on galvanomagnetic effects first elucidated by Faraday; but the rates most useful for pixel shifting approach the upper limits of manufacturable system designs. The second mode is based on the direct conversion of electric charge to mechanical motion, mediated by energy interchange between intrinsic electric fields and internal stress in certain non-centrosymmetric crystals. This allows apparent design simplicity, at the cost of employing unusual materials which may constrain design freedom. The third mode attempts to avoid the fabrication issues of the first and material issues of the second, by mediating direct conversion between electric charge and mechanical motion through lithographically fabricated microstructures.

[0084] Figure 18 shows an embodiment of an optical device 1800 using a piezoelectric actuator 1802. In this embodiment two parallel transparent plate members 1804 are separated by a distance AL that is controlled by the piezo-actuator. The plate members are in the form of wedges, and the parallel surfaces of the plate members 1804 are angled, resulting in a displacement of the pixel on the display 1806 of Axgiven by:With p — sin-1[n sin(a)]

[0085] For a wedge angle a=15 degrees made of glass material of index of refraction n=1.45 and a relative displacement of AL=15 microns, a pixel image shift of Ax=3 microns is achieved. The resonant frequency of the example embodiment is 76kHz. This embodiment requires thick plates and high quality anti-reflection coating to avoid formation of ghost images.

[0086] The actuators in the optical device array are controlled to effect time division multiplexing of adjacent pixels. The image shifting optical devices in the array displace the image provided by the microLED array by integer multiples of the pixel pitch at an integer multiple of the frame rate.

[0087] Most existing image shifting solutions focus on enhanced resolution (also called super-resolution). In contrast, the system and devices described herein is aimed at improving fabrication yield of pick and place microLED projection displays by using sub-resolution displays for which assembly is eased by greater pixel separation. This is made possible because the optical devices described herein recover the desired resolution through image shifting. Currently fabrication yield is the critical bottleneck in microLED arrays and the optical devices described herein can be used to address the problem of fabrication yield by employing image shifting to recover image detail lost when sparse microLED arrays are used.

[0088] Cameras can be manufactured at the desired resolutions and yield, but the same principle of sub-resolution arrays of discrete imaging pixels allows mix and match of different spectral and luminance sensitives by using different materials. This enables imaging with unique spectral and luminance sensitivities.

[0089] Sub-resolution layout enables a hybrid projection and imaging device 1900 as illustrated in Figure 19. This can be thought of as an imaging transceiver, by analogy with radio transceivers. The microLED array 1902 in this embodiment includes both emitting diodes 1904 and photo-diodes 1906. This has applications including in optical sensing (e.g., where a common optical path is used), in real-time image correction for projection, and aberration correction for cameras.

[0090] Advantages:

[0091] The optical devices described herein are based on a similar principle used in conventional resolution enhancement. However, the aim of conventional devices is resolution enhancement beyond device dimensions. In contrast, the problem solved by the optical device described herein is to reduce both cost and complexity by reducing the number of devices, i.e. the number of emitters and detectors used. This is made feasible because the original resolution can be recovered using the image-shifting method described herein.

[0092] Existing resolution enhancement solutions cannot be used to recover the resolution lost when a partially occupied array of microLEDs is used. One reason is that partially occupied microLED arrays are not normally used for display technology, and hence the loss in image resolution or image detail resulting from the use of such displays has not yet been addressed. Another reason is that existing solutions cannot be adapted to solve this problem, at least because ‘over-driving’ systems designed for subpixel shifts in order to achieve integer pixel image-shifting would only be beneficial if the coupled device array is sparse enough to benefit from the reduced device numbers. If not, the only effect is to add dark bands across image produced where the dark bands correspond to the regions that unilluminated pixels would have illuminated.

[0093] In any event, existing systems do not typically have sparsely populated optical device arrays, so that there would be limited cost benefit.

[0094] Conventional image resolution enhancement systems aim to improve image quality. They do not, however, address the problems of (a) reducing cost and complexity by reducing device numbers and (b) also reducing the required accuracy of device placement. In contrast, the novel optical device described herein provides a solution that both reduces device count as well as increases the ease of manufacture. This results in a less complex and more cost effective design.

[0095] The optical devices described herein combine integer pixel image-shifting with a sparsely populated device field designed to recover the resolution of a fully populated device field.

[0096] Some existing systems use sub pixel image shifts for image enhancement. In contrast, the system described herein increases the shift from sub-pixel to integer pixel. Whereas overlapping pixels results in higher resolution images, integer shifted pixels result in images with “gaps” due to unilluminated areas. Conventionally, such gaps are undesirable which is one reason why no existing systems exist that function in the manner described herein. However, the resulting image gaps can be eliminated by compensating gaps in the object or projector plane.

[0097] Whereas monolithic displays are not made with gaps, microLED displays cannot avoid them, and the attempt to minimise the gaps increases assembly cost. The system described herein accepts the gaps between microLEDs thus allowing cheap fabrication. This is possible because image-shifting is used to recover the original resolution by masking the gaps: unilluminated areas are illuminated by adjacent shifted pixels.

[0098] Advantageously, some embodiments of the system exploit the gaps, for example using them to embed detection, processing, and / or memory elements into an object plane that does not have to be fully occupied by microLEDs.

[0099] Although monolithic microLED array s / di splays can be made with small pitch and gaps, the bonding of the microLED wafer to the CMoS backplane wafer can beextremely challenging when the microLED pitch is smaller than 5-microns. The present invention can alleviate the issues of wafer-to-wafer bonding by increasing the pitch size through the introduction of gaps between microLEDs.

[0100] The semi-populated pixel layout described herein (for example as illustrated in Figures 6, 9 and 16) are designed to ease manufacture by reducing the tolerances required during pick-and-place assembly. Ultimately this reduces manufacturing costs by increasing yields. Rapid image shifting allows half the pixels (or more) to be omitted, whilst their function is substituted by the remaining pixels shifted to occupy what would have been their virtual positions. The ensuing advantages include, but are not limited to:- reduced number of photoemitters and / or photodetectors for a given image quality,- increased production rate and yield due to relaxed pick-and-place tolerances,- easier connectivity to and between arrayed devices,- increased aperture of pixel and sub-pixel micro-optics and enhanced brightness and colour gamut by overlaying sub-pixels.

[0101] The enhanced brightness and colour gamut is most easily achieved by an integer sub-pixel shift or multiples of a sixth of the pitch. This adds the brightnesses and colours of the shifted and unshifted sub-pixels. This sub-resolution multiplexing to recover parity resolution is in stark contrast with the normal use of image shifting to achieve super-resolution, typically using a half sub-pixel image shift.

[0102] The degree of under-sampling possible to maximise other advantages is almost arbitrary depending on the sparseness with which the matrix is populated. A matrix with only one third occupancy, for example, is described with reference to Figure 9 (compared to the matrix with one half occupancy described with reference to Figure 6). This free space also allows access for efficient testing and repair.

[0103] The vacant regions of the optical field can be used for a multiplicity of applications. The optical system described herein comprises a pixelated array of optical sources or detectors; where the arrays are scanned by image shifting devices. Hence theproposed optical system can enhance the performance of both cameras and projectors. If populated by photodetectors, the combined photoemitter and / or photodetector array is effectively an optical transceiver array, acting as both camera and projector. This has applications in areas such as wavefront sensing, adaptive optics, common path interferometry, optical tomography and free space communication.

[0104] If populated by processing and / or memory elements, the combined photodetector and / or processor array can mimic many of the graded receptive field characteristics of the retina such as image sharpening and colour perception. If populated by waveguides, the vacant regions could be used to mix the outputs of adjacent emitters and introduce additional optical modulation e.g., polarisation to their emission, this in addition to the previously mentioned refinements. If populated by MEMS structures, the possibility of a camera, projector, microphone, and / or speaker arises, as does the possibility of a device that bears its own image shifter, on the areas vacated due to the use of the principle of image shifting.

[0105] Advantageously, image shifting not only eases manufacturing tolerances but also allows the integration of non-imaging devices in the image field. This allows distributed detection, processing and storage.

[0106] Notably, the need for the type of optical device described herein has not arisen before. Prior detectors and emitters have been point devices, linear arrays, or rectangular arrays - all manufactured monolithically. Imaging devices employing partially occupied image planes are not well-known or used to any extent, since the devices are fabricated by lithography and to date there has been no benefit in less than 100% occupancy. Only recent microLED arrays that need pick-and-place assembly would benefit from the approach described herein. One area that might use partial occupancy is pattern matching, where detector layout might match the target pattern. But this is a different application and does not need image-shifting at all.

[0107] An additional advantage of the method described herein relates to epitaxial defects. Epitaxial defects can lead to shifts in wavelength binning in up to 5% of LEDdies, a critical bottleneck for overall display yield. Defective pixels can be mapped and custom image shifting algorithms devised to mask these image defects, offering the possibility to recover devices that would otherwise have to be discarded. If defective dies have to be replaced, the sparse architecture assists existing tools to effect that repair.

[0108] Advantageously, the optical systems and optical devices described herein allow for increased pixel spacing in microLED displays. The low external quantum efficiency of microLEDs (typically 40 to 55%) combined with their high packing densities, result in significant thermal management issues. Increased pixel spacing offers the possibility of increased light extraction and precise beam shaping, by enabling micro-optics with reduced alignment tolerances. Beam shaping is particularly important for micro-displays in augmented reality (AR) applications, given that their typically single figure acceptance angles (measured in degrees) couple poorly to the typically Lambertian emission of microLEDs.

[0109] An advantage of the methods described herein that recover resolution (and in some embodiments full or nearly full resolution) despite the existence of vacant pixels is that the applicability is not limited to designed arrays of such vacancies. It can also be used to compensate for random distributions of such vacancies, arising from limitations in manufacturing processes. For instance fluidic self-assembly has been used to create large arrays of microLED emitters, but its use has been limited by the difficulty in ensuring that these arrays are well ordered. Deviations from order include vacancies and / or dislocations in the pixel arrays, which reduce panel yield. However, such panels can be used when combined with image-shifting simply by mapping those flaws and compensating for them in software and / or firmware. This solution does not just increase manufacturing yield, it allows the adoption of high-throughput non- deterministic assembly.

[0110] In addition to these advantages the optical system described herein allows a range of hybrid devices, by allowing the integration of material systems difficult to combine through standard manufacturing methods.

[0111] The vacant pixel positions in a partially occupied field of micro-emitters can be occupied by photodetectors, allowing a hybrid of projector and / or camera use within the same device. This device could be based on any combination of semiconductor materials, not limited by typical fabrication constraints. Such a hybrid device is not only more compact than standard devices but also covers a far broader range of wavelengths.

[0112] The vacant pixel positions in a partially occupied field of micro-emitters can be occupied by optical modulators, allowing a hybrid emitter with a higher modulation bandwidth than possible with direct carrier injection. The systems described herein are able to incorporate retro-reflectors to ensure that the emitter beam is redirected onto the modulator. Such a hybrid optical system with enhanced bandwidth has applications in free-space optical communications, LIDAR and optical coherence tomography.

[0113] The vacant pixel positions in a partially occupied field of micro-emitters can be occupied by optical amplifiers, allowing a hybrid emitter with a higher intensity than possible with direct carrier injection. This system could incorporate retro-reflectors to ensure that the emitter beam is redirected onto the amplifier. Such a hybrid optical system with enhanced intensity, and being bandwidth tuneable on the pixel level, would have applications in free-space optical communications, LIDAR and optical coherence tomography.

[0114] The vacant pixel positions in a partially occupied field of micro-emitters can be occupied by analogue and / or digital electronic components, allowing a hybrid emitter / detector array incorporating reconfigurable hardware based digital image processing.. Such a hybrid optical system could mimic the retinal pre-processing (e.g. lateral inhibition, contrast enhancement) demonstrating image processing at far higher rates than possible in software alone.

[0115] The vacant pixel positions in a partially occupied field of micro-emitters can be occupied by acoustic transducers, allowing a hybrid optical and acoustic array capable of projecting optical images and / or projecting (with speakers) and / or sampling(with microphones) acoustic fields. Such a hybrid system would combine all the functionalities required of a projector in a more compact form than currently possible. AR / VR glasses are also a possible application.

[0116] It will be understood to persons skilled in the art of the invention that many modifications may be made without departing from the spirit and scope of the invention.

Claims

CLAIMS:

1. An optical system configured for an imaging system comprising a partially occupied microLED array comprising occupied and unoccupied array positions that cause lost image detail, the optical system comprising: at least one optical device, wherein each optical device: is associated with and positioned relative to a grouping of microLEDs of the partially occupied microLED array; comprises at least one transparent plate member configured to refract a light beam from the grouping of microLEDs in the array onto an image plane; and is positionable so as to cause the at least one transparent plate member to sequentially refract the light beam along more than one light path, the more than one light path corresponding to both occupied and unoccupied array positions, thereby recovering lost image detail.

2. The optical system of claim 1, wherein the optical system comprises a sparse array of optical devices comprising the plurality of optical devices.

3. The optical system of claim 1 or claim 2, comprising fewer optical devices than displayed pixels in the image plane.

4. The optical system of any one of the preceding claims, wherein the plurality of optical devices are configured to effect integer pixel shifts.

5. The optical system of any one of the preceding claims, wherein the at least one optical device comprises two or more transparent plate members.

6. The optical system of any one of the preceding claims, wherein the transparent plate members consist of monolithic wedges.

7. The optical system of any one of the preceding claims, wherein the at least one optical device comprises a rotating assembly.

8. The optical system of claim 7, wherein the rotating assembly comprises a rotor.

9. The optical system of claim 7, wherein the rotating assembly comprises a frame supporting the at least one transparent plate member, so that as the rotating assembly rotates around an axis orthogonal to the image plane the at least one transparent plate member rotates into and out of an optical path associated with the image plane.

10. The optical system of claim 5, wherein the two or more transparent plate members have different thicknesses to effect different optical shifts.

11. The optical system of claim 5 or claim 10, wherein the two or more transparent plate members are positioned at different tilt angles to effect different optical shifts.

12. The optical system of any of the preceding claims, further comprising at least one actuator controlled to effect time division multiplexing of adjacent pixels.

13. The optical system of claim 12, wherein the actuator utilises one or more of: electromagnetic actuation, piezoelectric actuation, and electrostatic actuation.

14. The optical system of any one of the preceding claims, wherein the at least one optical device causes image shifting thereby displacing an image provided by the microLED array by integer multiples of a pixel pitch at an integer multiple of a frame rate.

15. An optical system comprising: a partially occupied microLED array comprising occupied and unoccupied array positions that cause lost image detail; and an optical system comprising: at least one optical device, wherein each optical device: is associated with and positioned relative to a grouping of microLEDs of the partially occupied microLED array; comprises at least one transparent plate member configured to refract a light beam from the grouping of microLEDs in the array onto an image plane; and is positionable so as to cause the at least one transparent plate member to sequentially refract the light beam along more than one light path, the more than one light path corresponding to both occupied and unoccupied array positions, thereby recovering lost image detail.

16. The optical system of claim 15, wherein the optical system comprises the optical system of any one of claims 1 to 14.

17. The optical system of, claim 15 or 16 wherein the occupied and unoccupied array positions cause an increase in pitch size.

18. The optical system of claim 17, wherein the pitch size is more than 5 micron.

19. The optical system of any one of claims 15 to 18, wherein the optical device comprises n transparent plate members, and wherein the partially occupied microLED array has 1 / n occupancy.

20. The optical system of any one of the preceding claims, wherein the partially occupied microLED array incorporates photodetectors in some or all of the array positions unoccupied by microLEDs.

21. The optical system of any one of the preceding claims, wherein the partially occupied microLED array incorporates optical modulators in some or all of the array positions unoccupied by microLEDs.

22. The optical system of any one of the preceding claims, wherein the partially occupied microLED array incorporates optical amplifiers in some or all of the array positions unoccupied by microLEDs.

23. The optical system of any one of the preceding claims, wherein the partially occupied microLED array incorporates analogue or digital electronic components in some or all of the array positions unoccupied by microLEDs.

24. The optical system of any one of the preceding claims, wherein the partially occupied microLED array incorporates acoustic transducers in some or all of the array positions unoccupied by microLEDs.

25. The optical system of any one of the preceding claims, wherein deviations in array order comprise one or more of vacancies and dislocations, and the deviations in array order are mapped and / or compensated for in software.

Images