System and method for displaying objects with depth of field

Abstract

An object display system includes a right light signal generator, a left light signal generator, a right combiner, and a left combiner. The right light signal generator generates a plurality of right light signals of an object. The right combiner receives and redirects the plurality of right light signals toward one retina of a user to display a plurality of right pixels of the object. The left light signal generator generates a plurality of left light signals of an object. The left combiner receives and redirects the plurality of left light signals toward another retina of the user to display a plurality of left pixels of the object. The user receives a first redirected right light signal and a corresponding first redirected left light signal to display a first virtual binocular pixel of the object at a first depth of field, the first depth of field being related to a first angle between the first redirected right light signal and the corresponding first redirected left light signal.

Description

Technical Field

[0001] The present invention relates to a system and method for displaying an object with depth of field, and more particularly, to a system and method for displaying an object with depth of field by generating a plurality of right-side light signals and a left-side light signal and redirecting the plurality of right-side light signals and the left-side light signal to a user's retina. Background Art

[0002] Traditional virtual reality (VR) and augmented reality (AR) systems use stereoscopic technology to create a three-dimensional virtual image by simultaneously projecting two parallax images with different perspectives onto display panels on the left and right sides of the user's eyes. The difference in perspective between the two parallax images is interpreted by the brain as the image's depth. However, the user's eyes are actually focused (gaze) on the display panels; due to the parallax images, the depth of the display panels differs from the depth of the image the user sees. When the focus accommodation of depth perception is inconsistent with eye convergence, vergence-accommodation conflict (VAC) occurs. VAC can cause dizziness or headaches in users. Furthermore, using parallax images in mixed reality (MR) prevents users from focusing on both the real object and the virtual image simultaneously, resulting in focal rivalry. Furthermore, displaying virtual image movement using parallax imaging technology places a significant burden on image processing hardware. Summary of the Invention

[0003] The present invention provides a system and method for displaying an object with depth of field in space. Because the depth of field of the object coincides with the location of a user's binocular gaze, vergence-accommodation conflict (VAC) and focal rivalry are avoided. The object display system includes a right light signal generator, a right combiner, a left light signal generator, and a left combiner. The right light signal generator generates a plurality of right light signals for the object. The right combiner receives and redirects the plurality of right light signals toward the user's retina to display a plurality of right pixels of the object. The left light signal generator generates a plurality of left light signals for the object. The left combiner receives and redirects the plurality of left light signals toward the user's other retina to display a plurality of left pixels of the object. Furthermore, the user perceives a first redirected right light signal and a corresponding first redirected left light signal, generating a first virtual binocular pixel of the object at a first depth of field. The first depth of field is related to a first angle between the first redirected right light signal and the corresponding first redirected left light signal. In one embodiment, the first depth of field is determined by the first angle between the optical path extensions of the first redirected right light signal and the corresponding first redirected left light signal.

[0004] When the object is perceived as having multiple depths of field, for a first virtual binocular pixel of the object, the user perceives the second redirected right light signal and the corresponding second redirected left light signal, generating a second virtual binocular pixel of the object at a second depth of field, wherein the second depth of field is related to a second angle between the second redirected right light signal and the corresponding second redirected left light signal.

[0005] Furthermore, the first redirected right-side light signal is not a parallax of the corresponding first redirected left-side light signal. In this case, the right eye and the left eye receive the image of the object from the same perspective, rather than through the traditional method of generating parallax between the right eye and left eye perspectives of 3D images.

[0006] In another embodiment, the first redirected right light signal and the corresponding first redirected left light signal are at substantially the same height on the retinas of both eyes of the user.

[0007] In another embodiment, the plurality of right optical signals generated by the right optical signal generator are reflected only once before entering the user's retina, and the plurality of left optical signals generated by the left optical signal generator are reflected only once before entering the user's retina.

[0008] In one embodiment, the right combiner receives and redirects a plurality of right light signals toward the right retina of the user to display a plurality of right pixels of the object, and the left combiner receives and redirects a plurality of left light signals toward the left retina of the user to display a plurality of left pixels of the object. In another embodiment, the right combiner receives and redirects a plurality of left light signals toward the right retina of the user to display a plurality of right pixels of the object, and the left combiner receives and redirects a plurality of right light signals toward the left retina of the user to display a plurality of left pixels of the object.

[0009] In augmented reality (AR) or mixed reality (MR) applications, the right combiner and the left combiner are transparent to ambient light.

[0010] In AR and MR applications, the object display system further includes a support structure that can be worn on the user's head. A right optical signal generator, a left optical signal generator, a right combiner, and a left combiner are mounted on the support structure. In one embodiment, the support structure is a pair of glasses. In this case, the support structure can be glasses with or without lenses. The lenses can be prescription lenses to correct myopia, hyperopia, etc.

[0011] In an embodiment of the smart glasses, the right optical signal generator can be mounted on the right temple of the frame, while the left optical signal generator can be mounted on the left temple of the frame. Furthermore, the right combiner can be mounted on the right lens, while the left combiner can be mounted on the left lens. There are many different implementations for mounting the combiner. The combiner can be attached or integrated to the lens using detachable or non-detachable means. Furthermore, the combiner can be integrally formed with the lens, including a prescription lens.

[0012] This case uses retinal scanning technology to project right and left light signals to the user's retina, which is different from near-eye displays that are usually placed very close to the user's eyes.

[0013] Other features and advantages of this invention will be more clearly described in the following description, and those skilled in the art will also be able to further understand this invention through practicing this invention. The purposes and advantages of this invention are clearly indicated by the structures and methods indicated in the specification, scope of claims, and diagrams of this invention. The following description is illustrative and intended to explain the scope of the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 A schematic diagram illustrating an embodiment of an object display system according to the present invention.

[0015] Figure 2Schematic diagram illustrating the relationship between a virtual binocular pixel and a corresponding right pixel and left pixel pair according to the present invention.

[0016] Figure 3 A schematic diagram illustrating the optical path from the optical signal generator to the combiner and reaching the user's retina according to the present invention.

[0017] Figure 4 Schematic diagram of a virtual binocular pixel formed by the right light signal and the left light signal according to the present invention.

[0018] Figure 5 A lookup table according to an embodiment of the present invention is described.

[0019] Figure 6 Schematic diagram for illustrating how different virtual binocular pixels display an object according to the present invention.

[0020] Figure 7 A flow chart illustrating a process of displaying an object according to the present invention.

[0021] Figure 8 A schematic diagram illustrating the position of the optical signal generator relative to the combiner according to the present invention.

[0022] Figure 9 FIG. 1 is a schematic diagram illustrating an object display system having an optical replicator according to an embodiment of the present invention.

[0023] Figure 10 FIG. 1 is a schematic diagram illustrating an object display system according to an embodiment of the present invention.

[0024] Figure 11 FIG. 1 is a schematic diagram illustrating an integrated combiner according to the present invention.

[0025] Figure 12 Schematic diagram illustrating an object display system mounted on glasses according to the present invention.

[0026] Figure 13 Schematic diagram illustrating a diopter unit and a combiner according to the present invention.

[0027] Figure 14A -I is a schematic diagram illustrating the display of a moving object according to the present invention. DETAILED DESCRIPTION

[0028] The terms used herein are used to describe the details of specific embodiments of the present invention. All terms should be interpreted in the broadest possible sense. Certain terms will be emphasized below; any limiting terms will be defined in the specific embodiments.

[0029] The present invention relates to systems and methods for displaying objects with depth of field in space. Because the depth of field of the object coincides with the location of a user's binocular gaze, vergence-accommodation conflict (VAC) and focal rivalry are avoided. Embodiments described herein relate to one or more methods, systems, devices, and computer-readable media storing processor-executable steps for displaying objects with depth of field in space. The object display system includes a right light signal generator, a right combiner, a left light signal generator, and a left combiner. The right light signal generator generates a plurality of right light signals for the object. The right combiner receives and redirects the plurality of right light signals toward one user's retina to display a plurality of right pixels of the object. The left light signal generator generates a plurality of left light signals for the object. The left combiner receives and redirects the plurality of left light signals toward the retina of another user to display a plurality of left pixels of the object. Furthermore, the user perceives the first redirected right light signal and the corresponding first redirected left light signal as a first virtual binocular pixel of the object at a first depth of field, where the first depth of field is related to a first angle between the first redirected right light signal and the corresponding first redirected left light signal. In one embodiment, the first depth of field is determined by a first angle between the optical paths of the first redirected right light signal and the corresponding first redirected left light signal.

[0030] A user can perceive the position of an object at multiple depths of field. In addition to the first virtual binocular pixel of the object, the user perceives the second redirected right light signal and the corresponding second redirected left light signal to display the second virtual binocular pixel of the object at a second depth of field. The second depth of field is related to a second angle between the second redirected right light signal and the corresponding second redirected left light signal.

[0031] Furthermore, the first redirected right light signal is not parallax of the corresponding first redirected left light signal. The right eye and the left eye receive the image of the object from the same perspective, rather than receiving the parallax from the right eye perspective and the left eye perspective respectively to generate 3D images.

[0032] In another embodiment, the first redirected right light signal and the corresponding first redirected left light signal are at substantially the same height on the retinas of both eyes of the user.

[0033] In another embodiment, the plurality of right optical signals generated by the right optical signal generator are reflected only once before entering the user's retina, and the plurality of left optical signals generated by the left optical signal generator are reflected only once before entering the user's retina.

[0034] In one embodiment, the right combiner receives and redirects a plurality of right light signals toward the right retina of the user to display a plurality of right pixels of the object, and the left combiner receives and redirects a plurality of left light signals toward the left retina of the user to display a plurality of left pixels of the object. In another embodiment, the right combiner receives and redirects a plurality of left light signals toward the right retina of the user to display a plurality of right pixels of the object, and the left combiner receives and redirects a plurality of right light signals toward the left retina of the user to display a plurality of left pixels of the object.

[0035] In augmented reality (AR) or mixed reality (MR) applications, the right combiner and the left combiner are transparent to ambient light.

[0036] In AR and MR applications, the object display system further includes a support structure that is wearable on the user's head. A right optical signal generator, a left optical signal generator, a right combiner, and a left combiner are mounted on the support structure. In one embodiment, the system is a head-mounted device, particularly eyeglasses. In this case, the support structure may be a frame with or without eyeglass lenses. The lenses may be prescription lenses, such as those used to correct nearsightedness or farsightedness.

[0037] In an embodiment of the smart glasses, the right optical signal generator can be mounted on the right temple of the frame, while the left optical signal generator can be mounted on the left temple of the frame. Furthermore, the right combiner can be mounted on the right lens, while the left combiner can be mounted on the left lens. There are many different implementations for mounting. The combiner can be attached or integrated to the lens using either detachable or non-detachable means. Furthermore, the combiner can be integrally formed with the lens, including a prescription lens.

[0038] like Figure 1As shown, the object display system includes a right optical signal generator 10 for generating a plurality of right optical signals (e.g., 12, 14, and 16), a right combiner 20 for receiving and redirecting the plurality of right optical signals toward a user's right retina 54, a left optical signal generator 30 for generating a plurality of left optical signals (e.g., 32, 34, and 36), and a left combiner 40 for receiving and redirecting the plurality of left optical signals toward a user's left retina 64. The user has a right eye 50 including a right pupil 52 and a right retina 54, while a left eye 60 includes a left pupil 62 and a left retina 64. Generally, the diameter of a human pupil ranges from 2 to 8 mm, depending on the intensity of ambient light. In bright light environments, the diameter of a normal adult pupil varies between 2 and 4 mm, while in darkness, the diameter of a normal adult pupil varies between 4 and 8 mm. The plurality of right optical signals are redirected by the right combiner 20, pass through the right pupil 52, and are ultimately received by the right retina 54. Right light signal 12 is the rightmost light signal visible to the user's right eye on a specific horizontal plane. Right light signal 14 is the leftmost light signal visible to the user's right eye on a specific horizontal plane. When receiving the redirected right light signal, the user can see multiple right pixels of an object in area A defined by the extension of redirected right light signals 12' and 14'. Area A can be referred to as the field of view (FOV) of right eye 50. Similarly, multiple left light signals are redirected by left combiner 40, pass through left pupil 62, and are ultimately received by left retina 64. Left light signal 32 is the rightmost light signal visible to the user's left eye on a specific horizontal plane. Left light signal 34 is the leftmost light signal visible to the user's left eye on a specific horizontal plane. When receiving the redirected left light signal, the user can see multiple left pixels of an object in area B defined by the extension of redirected left light signals 32' and 34'. Area B can be referred to as the field of view (FOV) of left eye 60. When a plurality of right and left pixels are displayed in area C (the overlapping area of area A and area B), at least one right light signal of the displayed right pixel is fused with the corresponding left light signal of the displayed left pixel to display a virtual binocular pixel with depth of field in area C. The depth of field is related to the angle of the redirected right light signal and the redirected left light signal. This angle is also called the convergence angle.

[0039] like Figure 1 and 2As shown, a virtual image of a dinosaur object 70 with multiple depths of field is visible in an area C in front of the user. The image of the dinosaur object 70 includes a first virtual binocular pixel 72 displayed at a first depth of field D1 and a second virtual binocular pixel 74 displayed at a second depth of field D2. The first angle between the first redirected right light signal 16' and the corresponding first redirected left light signal 36' is θ1. The first depth of field D1 is related to the first angle θ1. In particular, the first depth of field D1 of the first virtual binocular pixel of the object can be determined by the first angle θ1 between the optical path extensions of the first redirected right light signal and the corresponding first redirected left light signal. Thereby, the first depth of field D1 of the first virtual binocular pixel 72 can be calculated by the following formula:

[0040]

[0041] The distance between the right pupil 52 and the left pupil 62 is the interpupillary distance (IPD). Similarly, the second angle between the second redirected right light signal 18' and the corresponding second redirected left light signal 38' is θ2. The second depth of field D2 is related to the second angle θ2. In particular, the second depth of field D2 of the second virtual binocular pixel of the object can be determined by the second angle θ2 between the optical path extensions of the second redirected right light signal and the corresponding second redirected left light signal. The user perceives the second virtual binocular pixel 74 as being farther away from the user than the first virtual binocular pixel 72 (i.e., having a greater depth of field), and the second angle θ2 is smaller than the first angle θ1.

[0042] Furthermore, although the redirected right light signal 16' and the corresponding redirected left light signal 36' together display the first virtual binocular pixel 72 at the first depth of field D1, the redirected right light signal 16' is not parallax of the corresponding redirected left light signal 36'. Conventionally, because the right eye sees objects at a different perspective than the left eye, the parallax between the image received by the right eye and the image received by the left eye is used to allow the user to perceive a 3D image with depth. However, in the present invention, the right light signal of the virtual binocular pixel and the corresponding left light signal display an image at the same perspective. Therefore, the red, blue, and green (RBG) light intensities and / or the brightness of the right light signal and the left light signal are substantially the same. In other words, the right pixel is substantially the same as the corresponding left pixel. However, in another embodiment, one or both of the right and left light signals can be adjusted to present other 3D effects, such as shadows. Generally speaking, in the present invention, both the right eye and the left eye receive the image of the object from the same perspective, rather than from the parallax of the right eye perspective and the left eye perspective in the traditional way of generating 3D images. As described above, a plurality of right-side optical signals are generated by the right-side optical signal generator 10, redirected by the right-side merger 20, and scanned directly to the right retina to form a right retinal image on the right retina. Similarly, a plurality of left-side optical signals are generated by the left-side optical signal generator 30, redirected by the left-side merger 40, and scanned directly to the left retina to form a left retinal image on the left retina. In one embodiment, as Figure 2 As shown, right retinal image 80 includes 36 right pixels (6x6 matrix) and left retinal image 90 also includes 36 left pixels (6x6 matrix). In another embodiment, right retinal image 80 includes 921,600 right pixels (1280x720 matrix) and left retinal image 90 also includes 921,600 left pixels (1280x720 matrix). The object display system is configured to generate a plurality of right optical signals and a corresponding plurality of left optical signals, each of which forms a right retinal image on the right retina and a left retinal image on the left retina. Therefore, due to image fusion, the user can view a virtual binocular object with a specific depth of field in area C.

[0043] like Figure 2As shown, the first right optical signal 16 from the right optical signal generator 10 is received and reflected by the right combiner 20. The first redirected right optical signal 16' passes through the right pupil 52 and reaches the user's right retina to display the right pixel R43. The corresponding left optical signal 36 from the left optical signal generator 30 is received and reflected by the left combiner 40. The first redirected optical signal 36' passes through the left pupil 62 and reaches the user's left retina to display the left pixel L33. Due to image fusion, the user can see a virtual binocular object with multiple depths of field, which are determined by the angles between the multiple redirected right optical signals and the corresponding multiple redirected left optical signals of the same object. The angles between the redirected right optical signals and the corresponding left optical signals are determined by the relative horizontal distance between the right pixel and the left pixel. Therefore, the depth of field of a virtual binocular pixel is inversely proportional to the relative horizontal distance between the right pixel and the corresponding left pixel that form the virtual binocular pixel. In other words, the deeper the depth of field of the virtual binocular pixel for the user, the smaller the relative horizontal distance on the X axis between the right pixel and the left pixel forming the virtual binocular pixel. Figure 2 As shown, the second virtual binocular pixel 74 has a deeper depth of field for the user than the first virtual binocular pixel 72 (i.e., it is farther away from the user). Therefore, on the retina, the horizontal distance between the second right pixel and the second left pixel is smaller than the horizontal distance between the first right pixel and the first left pixel. Specifically, the horizontal distance between the second right pixel R41 and the second left pixel L51, which form the second virtual binocular pixel, is 4 pixels long. However, the distance between the first right pixel R43 and the first left pixel L33, which form the first virtual binocular pixel, is 6 pixels long.

[0044] In such Figure 3In the embodiment of FIG. 1 , the optical paths of a plurality of right-side optical signals and a plurality of left-side optical signals from the optical signal generator to the retina are illustrated. The plurality of right-side optical signals generated by the right optical generator are projected to the right combiner 20 to form a right combiner image (RCI) 82. These plurality of right-side optical signals are redirected by the right combiner 20 and converged into a small right pupil image 84, which passes through the right pupil 52 and ultimately reaches the right retina 54 to form a right retinal image 86. Right retinal image 86 comprises ixj pixels. Similarly, the plurality of left-side optical signals generated by the left optical generator 30 are projected to the left combiner 40 to form a left combiner image 92. These plurality of left-side optical signals are redirected by the left combiner 40 and converged into a small left pupil image 94, which passes through the left pupil 62 and ultimately reaches the left retina 64 to form a left retinal image 96. Each of the left retinal images 96 comprises ixj pixels. Pixel (0,0) is the top-leftmost pixel in each image. Pixels in the retinal image are reversed left and right, and upside down, relative to the corresponding pixels in the combiner image. Depending on the appropriate relative position and angle of the optical signal generator and combiner, each optical signal has its own optical path from the optical signal generator to the retina. A right optical signal representing a right pixel on the right retina is combined with a corresponding left optical signal representing a left pixel on the left retina to form a virtual binocular pixel with a specific depth of field. Therefore, a virtual binocular pixel in space can be represented by a pair of right and left pixels, or a pair of 20 pixels from the right combiner and 40 pixels from the left combiner.

[0045] The virtual object seen by the user in area C includes a plurality of virtual binocular pixels. In order to accurately describe the position of the virtual binocular pixels in space, each position in space is represented by three-dimensional (3D) coordinates, such as XYZ coordinates. In other embodiments, different 3D coordinate systems can also be used. Therefore, each virtual binocular pixel has 3D coordinates - horizontal direction, vertical direction, and depth direction. The horizontal direction (or X-axis direction) is along the interpupillary line. The vertical direction (or Y-axis direction) is along the facial midline and is perpendicular to the horizontal direction. The depth direction (or Z-axis direction) is orthogonal to the frontal surface and is perpendicular to the horizontal and vertical directions.

[0046] Figure 4The relationship between pixels in the right combiner image, pixels in the left combiner image, and virtual binocular pixels is shown. As described above, pixels in the right combiner image have a one-to-one relationship with pixels in the right retinal image (right pixels). Pixels in the left combiner image have a one-to-one relationship with pixels in the left retinal image (left pixels). However, the pixels in the retinal image are reversed from their corresponding pixels in the combiner image. For a right retinal image containing 36 (6x6) right pixels and a left retinal image containing 36 (6x6) left pixels, assuming the light signals are within the user's FOV, there are a total of 216 (6x6x6) virtual binocular pixels in region C (shown as a dot matrix). The optical path of the redirected right light signal intersects with the optical path of each redirected left light signal in the same row in the image. Similarly, the optical path of the redirected left light signal intersects with the optical path of each redirected right light signal in the same row in the image. Therefore, there are 36 (6x6) virtual binocular pixels in a layer, and there are a total of 6 layers in space. There is usually a small angle between two adjacent lines, indicating that the light paths extend to intersect and form a virtual binocular pixel (although in Figure 4 (The right and left pixels on each retina are shown as parallel.) Right pixels and corresponding left pixels with approximately the same height on each retina (i.e., right and left retinal images in the same row) are initially fused together. Therefore, right and left pixels in the same row of the retinal image are paired to form virtual binocular pixels.

[0047] like Figure 5As shown, a lookup table is used to easily find the right pixel and left pixel pair of each virtual binocular pixel. For example, 216 virtual binocular pixels, numbered 1 to 216, are composed of 36 (6x6) right pixels and 36 (6x6) left pixels. The first (1st) virtual binocular pixel VBP(1) represents the right pixel RRI(1,1) and the left pixel LRI(1,1) pair. The second (2nd) virtual binocular pixel VBP(2) represents the right pixel RRI(2,1) and the left pixel LRI(1,1) pair. The seventh (7th) virtual binocular pixel VBP(7) represents the right pixel RRI(1,1) and the left pixel LRI(2,1) pair. The thirty-seventh (37th) virtual binocular pixel VBP(37) represents the right pixel RRI(1,2) and the left pixel LRI(1,2) pair. The 216th virtual binocular pixel VBP (216) represents a pair of right pixel RRI (6, 6) and left pixel LRI (6, 6). Therefore, in order to display a specific virtual binocular pixel of an object in space, it is necessary to first determine which right pixel and left pixel pair can be used to generate the corresponding right light signal and left light signal. In addition, each row of virtual binocular pixels in the lookup table includes a pointer to the memory address storing the VBP depth of field (z) and the VBP position (x, y). Other data, such as the VBP scale, the number of overlapping objects, and the continuous depth of field, can be stored. The scale can be the relative size data of the specific VBP compared to the standard VBP. For example, when the object is displayed one meter in front of the user with the standard VBP, the scale can be set to 1. Therefore, the scale of the specific VBP 90 cm in front of the user can be set to 1.2. Similarly, the scale of the specific VBP 1.5 m in front of the user can be set to 0.8. When an object moves from a first depth of field (D1) to a second depth of field (D2), proportional size can be used to determine the size of the displayed object. The object overlap count indicates the number of objects that overlap (objects fully or partially hidden behind another object). The depth order provides depth order information for each overlapping object. For example, consider three overlapping objects. The depth order of the first object in the front can be set to 1, while the depth order of the second object hidden behind the first object can be set to 2. As different objects move, the object overlap count and depth order can be used to determine which object or portion should be displayed.

[0048] like Figure 6 As shown, a virtual object with multiple depths of field, such as a dinosaur, can be displayed in area C by projecting predetermined right and left pixels onto the user's retina. In one embodiment, the object position can be determined by a reference position, and the object's viewing angle can be determined by a rotation angle. Figure 7As shown, in step 710, an object image is created using a reference position. In one embodiment, the object image can be generated by a 2D or 3D module. The reference position can be located at the center of gravity of the object. In step 720, the virtual binocular pixel of the reference position is determined. Using the 3D coordinates of the reference position, the designer can directly determine, for example, through GUI software, the closest virtual binocular pixel, such as VBP (145). In step 730, a pair of right and left pixels corresponding to the virtual binocular pixel is found. The designer can find the corresponding right and left pixel pairs using a lookup table. The designer can also calculate the convergence angle and find the corresponding right and left pixels based on the depth of field of the predetermined reference position, assuming that the reference position is located in front of the user's eyes. The designer can move the reference position to a predetermined X and Y coordinate in the XY plane and find the final corresponding right and left pixels. In step 740, the right light signal and the corresponding left light signal are projected to display the right pixel and the corresponding left pixel respectively as the reference position. Once the right pixel and left pixel pair of the virtual binocular pixel corresponding to the reference position is determined, the entire virtual object can be displayed using 2D or 3D model data.

[0049] The lookup table is generated using the following steps. First, a personal virtual map is obtained based on the user's IPD. This virtual map is generated by the system during initialization and calibration and defines the boundaries of the area C where the user can see objects with depth. Objects with depth are visible to the user due to fusion of the right and left retinal images. Second, the convergence angle for each depth in the Z-axis direction (for each point on the Z-coordinate) is calculated to determine the X- and Y-coordinate positions of right and left pixel pairs in the right and left retinal images, respectively. Third, the right and left pixel pairs are shifted along the X-axis to determine the X- and Z-coordinate positions of each right and left pixel pair at a specific depth. Fourth, the right and left pixel pairs are shifted along the Y-axis to determine the Y-coordinate of each right and left pixel pair. Thus, the 3D coordinate system (e.g., XYZ) of each right and left pixel pair in the right and left retinal images can be determined to generate the lookup table. Furthermore, the third and fourth steps can be interchanged.

[0050] In another embodiment, the designer can determine each necessary virtual binocular pixel to form a virtual object, and then use a lookup table to find each corresponding right and left pixel pair. Right and left light signals can then be generated. The right and left retinal images have the same viewing angle. Parallax is not used to render the 3D image. Therefore, complex and time-consuming computer graphics calculations can be avoided. The relative position of the object in the right and left retinal images determines the user's perceived depth of field.

[0051] Optical signal generators 10 and 20 may subsequently utilize light-emitting diodes (LEDs, including mini and micro LEDs), organic light-emitting diodes (OLEDs), superluminescent diodes (SLDs), liquid crystal on silicon (LcoS), liquid crystal displays (LCDs), or combinations thereof as light sources. In one embodiment, optical signal generators 10 and 20 are laser beam scanning projectors (LBS projectors), which include light sources such as red lasers, green lasers, and blue lasers, light color modulators (such as bidirectional color combiners and polarization combiners), and two-dimensional (2D) adjustable reflectors (such as 2D electromechanical systems (MEMS) mirrors). The 2D adjustable reflectors may be replaced by two one-dimensional (1D) reflectors (such as two 1D MEMS mirrors). The LBS projector sequentially generates and scans optical signals to form a 2D image with a predetermined resolution, for example, 1280x720 pixels per frame. Therefore, a light signal is generated for each pixel at a time and projected one pixel at a time toward combiners 20 and 40. To allow a user to see a 2D image with one eye, the LBS projector must sequentially generate a light signal for each pixel, such as a 1280x720 light signal, within the duration of visual persistence (e.g., 1 / 18 second). Therefore, the dwell time of each light signal is approximately 60.28 nanoseconds.

[0052] In another embodiment, optical signal generators 10 and 30 may be digital light processing projectors ("DLP projectors") that generate one 2D color image at a time. Texas Instruments' DLP technology is one of several technologies used to produce DLP projectors. The entire 2D color image frame, which may comprise, for example, 1280x720 pixels, is projected simultaneously toward combiners 20 and 40.

[0053] Combiners 20 and 40 receive and redirect the plurality of optical signals generated by optical signal generators 10 and 20. In one embodiment, combiners 20 and 40 reflect the plurality of optical signals so that the redirected optical signals are on the same side of combiners 20 and 40 as the incident optical signals. In another embodiment, combiners 20 and 40 refract the plurality of optical signals so that the redirected optical signals are on different sides of combiners 20 and 40 as the incident optical signals. When combiners 20 and 40 function as refractors, the reflectivity can vary considerably, for example, from 20% to 80%, depending in part on the capabilities of the optical signal generators. A person skilled in the art can determine an appropriate reflectivity based on the characteristics of the optical signal generators and combiners. Furthermore, in one embodiment, the surfaces of combiners 20 and 40 opposite to the optical signal incident surfaces thereof can be transparent to ambient light. The degree of transparency can vary significantly depending on the embodiment. In AR / MR applications, a transparency of 50% is preferred, while in other embodiments, 75% is preferred. Furthermore, in order to redirect the light signals, the combiners 20 and 40 may converge the plurality of light signals to form a combiner image so that the combiner image can pass through the pupil and reach the user's retina.

[0054] Combiners 20 and 40 can be constructed from mirrored glass or plastic, coated with a specific material (e.g., metal) to achieve partial light transmission or partial reflection. The advantage of using a reflective combiner is that it eliminates the need for a light guide plate (as in conventional technology) to direct the light signal to the user's eyes, thus addressing undesirable reflection effects such as multiple shadows and color distortion. Combiners 20 and 40 can be holographic combiners, but this is not a preferred embodiment because reflection effects can cause multiple shadows and RGB distortion. In some embodiments, it is desirable to avoid the use of a holographic combiner.

[0055] In one embodiment, the combiners 20 and 40 have ellipsoidal surfaces. In addition, the optical signal generator and the user's eyes are located at the focus of the ellipsoid. Figure 8 As shown, the right combiner 20 has an ellipsoidal surface, the right optical signal generator 10 is located at the right focus, and the user's right eye is located at the left focus of the ellipsoid. Similarly, the left combiner 40 has an ellipsoidal surface, the left optical signal generator 30 is located at the left focus, and the user's left eye is located at the right focus of the ellipsoid. Based on the geometric properties of the ellipsoid, all light beams projected from one focus to the ellipsoidal surface will be reflected to the other focus. Under such circumstances, all light beams projected from the optical signal generator to the ellipsoidal surface combiner will be reflected to the user's eyes. Therefore, in this embodiment, the FOV can be maximized, which is equivalent to the ellipsoidal surface. In another embodiment, the combiners 20 and 40 may have a flat surface and have a holographic film, which is configured to reflect light in an ellipsoidal manner.

[0056] The object display system may use right and left collimators to converge the beams of multiple optical signals, for example, causing the direction of movement to be less aligned with a single direction or reducing the spatial cross-section of the beams. The right collimator may be disposed between the right optical signal generator 10 and the right combiner 20, while the left collimator may be disposed between the left optical signal generator 30 and the left combiner 40. The collimators may be curved mirrors or lenses.

[0057] like Figure 9 As shown, the object display system may further include a right optical replicator (optical replicator) and a left optical replicator. The optical replicator may be disposed between the optical signal generators 10 and 30 and the combiners 20 and 40 to replicate the incident light signal. Thus, the optical replicator can generate a plurality of incident light signals to expand the user's eyebox. The optical replicator may be a beam splitter, a polarizing beam splitter, a half-silvered mirror, a partially reflective mirror, a dichroic mirrored prism, a dichroic mirror, or a dielectric optical coating. The optical replicator 120 may include at least two optical elements to replicate the incident light signal into at least two objects. Each optical element may be a lens, a reflector, a partially reflective mirror, a prism, or any combination thereof.

[0058] The object display system may further include a control unit, which includes all necessary circuitry to control the right optical signal generator 10 and the left optical signal generator 30. The control unit provides electronic signals to the optical signal generators to generate a plurality of optical signals. In one embodiment, the positions and angles of the right optical signal generator 10 and the left optical signal generator 30 can adjust the incident angles of the right and left optical signals and the receiving positions of the right combiner 20 and the left combiner 40. Such adjustments can be implemented by the control unit. The control unit can communicate with an independent image signal provider via a wired or wireless mechanism. Wireless communications include 4G and 5G telecommunications, WiFi, Bluetooth, near-field communication, and the Internet. The control unit may include a processor, memory, and input / output interfaces (I / O interfaces) to communicate with the image signal provider and the user. The object display system further includes a power supply. The power supply can be a battery and / or a wirelessly rechargeable component.

[0059] There are at least two options for arranging the optical path from the optical signal generator to the user's retina. The first option described above is that the right optical signal generator 10 generates a right optical signal, and the right optical signal is redirected to the right retina by the right combiner 20; the left optical signal generator 30 generates a left optical signal, and the left optical signal is redirected to the left retina by the left combiner 40. Figure 10As shown, the second option is that the right optical signal generator 10 generates a right optical signal, which is redirected to the left retina by the left combiner 40; and the left optical signal generator 30 generates a left optical signal, which is redirected to the right retina by the right combiner 20.

[0060] In another embodiment, if Figure 11 As shown, right combiner 20 and left combiner 40 can be integrated into an integrated combiner with specific curvatures for the right and left optical signals. In this embodiment of the large combiner, the right optical signal generated by right optical signal generator 10 is reflected onto the left retina, while the left optical signal generated by left optical signal generator 30 is reflected onto the right retina. By extending the width of the combiner to create a relatively large reflective surface, the FOV and area C of binocular fusion can be increased.

[0061] The object display system may include a support structure wearable on the user's head to carry the right optical signal generator 10, the left optical signal generator 30, the right combiner 20, and the left combiner 40. The right combiner 20 and the left combiner 40 are located within the user's field of view. Therefore, in this embodiment, the object display system is a head-mounted device (HWD). In particular, if Figure 12 As shown, the object display system can be mounted on glasses, i.e., smart glasses. In this case, the support structure can be a glasses frame with or without lenses. The lenses can be prescription lenses, used to correct myopia, hyperopia, etc. The right optical signal generator 10 can be mounted on the right temple 140 of the frame, while the left optical signal generator 30 can be mounted on the left temple 130 of the frame. The right combiner 20 can be mounted on the right lens, while the left combiner 40 can be mounted on the left lens. There are many different mounting methods. The combiner can be attached or integrated to the lens using detachable or non-detachable means. In addition, the combiner can be integrally formed with the lens, wherein the lens includes a prescription lens. When the support structure does not include lenses, the right combiner 20 and the left combiner 40 can be mounted directly on the frame.

[0062] All of the various components of the object display system embodiments described above can be applied to HWDs. Therefore, HWDs, including smart glasses, can be further equipped with other object display system components, such as a control unit, a right collimator, and a left collimator. The right collimator can be located between the right optical signal generator 10 and the right combiner 20, while the left collimator can be located between the left optical signal generator 30 and the left combiner 40. Furthermore, the combiner can be replaced by a beam splitter and a converging lens. The beam splitter's function is to reflect the light signal, while the converging lens' function is to converge the light signal so that it passes through the pupil and reaches the user's retina.

[0063] When the object display system is configured in smart glasses, the smart glasses lenses can have both refractive properties to correct the user's vision and function as a combiner. Smart glasses can have lenses with different prescriptions to correct the vision of users with myopia or hyperopia. In these cases, each lens of the smart glasses can include a refractive element and a combiner. The refractive element and combiner can be made of the same or different materials and integrally molded. The refractive element and combiner can be manufactured as two separate components and assembled together. These two components can be joined or detached, for example, by magnetic attachment, or they can be permanently joined. In both cases, the combiner is provided on the side of the lens closest to the user's eye. If the lens is integrally molded, the combiner forms the inner surface of the lens. If the lens has two components, the combiner forms the inner surface of the lens. The combiner allows the environment to penetrate and reflects the light signal generated by the light signal generator to the user's eye, forming a virtual image in the real environment. The combiner is designed with an appropriate curvature to reflect and focus all light signals from the light signal generator into the pupil and onto the retina of the eye.

[0064] In some embodiments, the curvature of the surface of the diopter unit is determined based on the user's diopter. If the lens is formed in one piece, the curvature with diopter is the outer surface of the lens. If the lens has two parts, the diopter unit forms the outer part of the lens. In this case, the curvature with diopter may be the inner surface or the outer surface of the diopter unit. In order to better match the diopter unit with the combiner, in one embodiment, the diopter unit may be classified into 3 groups based on the degree - more than +3.00 (hyperopia), between -3.0-+3.0, and less than -3.0 (myopia). The combiner may be designed based on the category of the diopter unit. In another embodiment, the diopter units may be classified into 5 groups or 10 groups, each group having a smaller diopter range. As Figure 13As shown, the outer surface 210 of the diopter unit 200 is designed to provide a curvature with a diopter power, and the inner surface 220 of the diopter unit 200 can be designed to have the same curvature as the outer surface 310 of the combiner 300. Therefore, the diopter unit 200 can be more easily combined with the combiner 300. For example, the inner surface 220 of the diopter unit 200 and the outer surface 310 of the combiner 300 can be the same spherical or ellipsoidal surface. In another embodiment, when the curvature of the inner surface 220 of the diopter unit 200 is a diopter power, the outer surface 310 of the combiner 300 can be designed to have the same or similar curvature as the inner surface 220 of the diopter unit 200 to facilitate the combination of the two. However, when the outer surface 310 of the combiner 300 and the inner surface 220 of the diopter unit 200 do not have the same curvature, the outer surface 310 of the combiner 300 and the inner surface 220 of the diopter unit 200 can be combined with each other through a mechanical mechanism such as magnets, adhesive materials, or a joining member. Alternatively, an intermediate material may be used to assemble the diopter unit 200 and the combiner 300. Alternatively, the combiner 300 may be coated on the inner surface 320 of the lens.

[0065] In addition to static virtual objects within the spatial frame, the object display system can also display moving objects. When the right optical signal generator 10 and the left optical signal generator 30 generate optical signals at high speeds, such as 30, 60, or even higher frames per second, the user can see objects moving smoothly within the image due to persistence of vision. Various embodiments for displaying virtual object movement are described below. Figure 14A -I illustrates examples 1-9 of object movement. The objects shown in the right and left combiner images 82 and 92 do not precisely reflect the positions of the corresponding right and left light signals. Furthermore, these examples use the midpoint of the user's interpupillary line as the origin of the XYZ coordinate system. Furthermore, RCI(10,10) and LCI(10,10) are set to the midpoint between the right and left combiner images. Similarly, RRI(10,10) and LRI(10,10) are set to the midpoint between the right and left retinal images. The (0,0) pixel is the top-leftmost pixel in each image.

[0066] Figure 14AExample 1 illustrates the virtual object being moved only in the X-axis direction (rightward), from the first virtual binocular pixel to the second virtual binocular pixel, within the same depth plane. To achieve this, the positions of the right light signal and the corresponding left light signal in the right and left combiner images, respectively, must be shifted to the right by the same distance (pixels) in the X-axis direction. Therefore, the positions of the right light signal and the corresponding left light signal in the right and left retinal images that form the virtual object, respectively, must be shifted to the left by the same distance in the X-axis direction. In other words, the right light signal and the corresponding left light signal from the light signal generator must be projected at different X-coordinate positions on the combiner image. However, since the Y- and Z-coordinates (depth direction) of the virtual object remain unchanged, the right light signal and the corresponding left light signal are projected at the same position (Y- and Z-coordinates) on the combiner image. For example, when the virtual object's XYZ coordinates move from (0,0,100) to (10,0,100), the right light signal on the right combiner image moves from RCI(10,10) to RCI(12,10), while the left light signal on the left combiner image moves from LCI(10,10) to LCI(12,10). Therefore, the right light signal on the right retinal image moves from RRI(10,10) to RRI(8,10), while the left light signal on the left retinal image moves from LRI(10,10) to LRI(8,10).

[0067] Figure 14BExample 2 illustrates that a virtual object is moved only in the Y-axis direction (downward), from the first virtual binocular pixel to the second virtual binocular pixel, within the same depth plane. To achieve this, the right light signal and the corresponding left light signal must be shifted downward by the same distance (pixels) in the Y-axis direction within the right and left combiner images, respectively. Consequently, the right light signal and the corresponding left light signal must each be shifted upward by the same distance in the Y-axis direction within the right and left retinal images that form the virtual object. In other words, the right light signal and the corresponding left light signal from the light signal generator must be projected at different Y-coordinate positions on the combiner image. However, since the virtual object's X- and Z-coordinates (in the depth direction) remain unchanged, the right light signal and the corresponding left light signal are projected at the same position (X- and Z-coordinates) on the combiner image. For example, when the virtual object's XYZ coordinates move from (0,0,100) to (0,-10,100), the right light signal on the right combiner image moves from RCI(10,10) to RCI(10,12), while the left light signal on the left combiner image moves from LCI(10,10) to LCI(10,12). Therefore, the right light signal on the right retinal image moves from RRI(10,10) to RRI(10,8), while the left light signal on the left retinal image moves from LRI(10,10) to LRI(10,8).

[0068] Figure 14CExample 3 illustrates a virtual object that moves only in the Z-axis direction (toward the user), thus moving from the original depth plane to a new depth plane. To achieve this, the positions of the right and left light signals in the right and left combiner images must be closer to each other in the X-axis, as the convergence angle between the optical paths of the right and left light signals increases. Consequently, the positions of the right and left light signals forming the virtual object in the right and left retinal images move farther apart in the X-axis. Generally speaking, as the virtual object moves closer to the user, the relative distance between the positions of the right and left light signals in the combiner image decreases, while the relative distance between the positions of the right and left light signals in the retinal image increases. In other words, the right and left light signals from the optical signal generator must be projected onto two different X-coordinate positions on the combiner image to bring them closer together. However, because the Y-coordinate of the virtual object remains unchanged, the right and left light signals are projected onto the same Y-coordinate position in the combiner image. For example, when the virtual object's XYZ coordinates move from (0,0,100) to (0,0,50), the right light signal on the right combiner image moves from RCI(10,10) to RCI(5,10), while the left light signal on the left combiner image moves from LCI(10,10) to LCI(15,10). Therefore, the right light signal on the right retinal image moves from RRI(10,10) to RRI(15,10), while the left light signal on the left retinal image moves from LRI(10,10) to LRI(5,10).

[0069] However, in order to move the virtual object closer to the user, the x-coordinate of the virtual object is not at the middle of the interpupillary line (the midpoint) (in one embodiment, x-coordinate = 0). The positions of the right light signal and the corresponding left light signal in the right and left combiner images, respectively, must be closer to each other based on a ratio. The ratio is the distance between the position of the right light signal in the right combiner image and its left edge (near the middle of the eyes) compared to the distance between the position of the left light signal in the left combiner image and its right edge (near the middle of the eyes). For example, assuming that the position of the right light signal in the right combiner image is 10 pixels from the left edge (near the middle of the eyes) and the position of the left light signal in the left combiner image is 5 pixels from the right edge (near the middle of the eyes), the ratio of the distance from the right position to the middle to the distance from the left position to the middle is 2:1 (10:5). To move the object closer to the user, due to the 2:1 ratio, if the right position on the right combiner image and the left position on the left combiner image must be 3 pixels closer to each other, the right position must be moved 2 pixels towards the left border, and the left position must be moved 1 pixel towards the right.

[0070] Figure 14D Example 4 illustrates the movement of a virtual object in the same depth-of-field plane in space, in the X-axis direction (to the right) and the Y-axis direction (upward), from a first virtual binocular pixel to a second virtual binocular pixel. To achieve this, the positions of the right light signal and the corresponding left light signal in the right and left combiner images must be shifted to the right and upward relative to their original positions. Consequently, the positions of the right light signal and the corresponding left light signal in forming the right and left retinal images of the virtual object, respectively, must be shifted to the right and upward relative to their original positions. In other words, the right light signal and the corresponding left light signal from the optical signal generator must be projected to new positions to the upper right of the right and left combiner images, while maintaining the same convergence angle between the optical path extensions of the right light signal and the corresponding left light signal. For example, when the virtual object's XYZ coordinates move from (0,0,100) to (10,10,100), the right light signal in the right combiner image moves from RCI(10,10) to RCI(12,8), while the left light signal in the left combiner image moves from LCI(10,10) to LCI(12,8). Therefore, the right light signal in the right retinal image moves from RRI(10,10) to RRI(8,12), while the left light signal in the left retinal image moves from LRI(10,10) to LRI(8,12).

[0071] Figure 14E Example 5 illustrates that a virtual object moves in space in the Y-axis direction (downward) and the Z-axis direction (toward the user), thereby moving from the original depth plane to a new depth plane. To achieve this, the positions of the right light signal and the corresponding left light signal in the right and left combiner images must be moved downward in the Y-axis direction and closer to each other in the X-axis direction, corresponding to a large convergence angle. Therefore, the positions of the right light signal and the corresponding left light signal in the right and left retinal images that form the virtual object must be moved upward in the Y-axis direction and away from each other in the X-axis direction. In other words, the right light signal and the corresponding left light signal from the light signal generator must be projected at different Y-coordinate positions and two different X-coordinate positions (close to each other) in the combiner image. For example, when the virtual object's XYZ coordinates move from (0,0,100) to (0,-10,50), the right light signal in the right combiner image moves from RCI(10,10) to RCI(5,12), while the left light signal in the left combiner image moves from LCI(10,10) to LCI(15,12). Therefore, the right light signal in the right retinal image moves from RRI(10,10) to RRI(15,8), while the left light signal in the left retinal image moves from LRI(10,10) to LRI(5,8).

[0072] However, because the virtual object's X-coordinate remains constant while the virtual object moves toward the user, the respective positions of the right light signal and the corresponding left light signal in the right and left combiner images must be closer to each other based on a ratio. This ratio is calculated by comparing the distance between the right light signal's position in the right combiner image and its left edge (near the center of the eyes) to the distance between the left light signal's position in the left combiner image and its right edge (near the center of the eyes). For example, (assuming the right light signal's position in the right combiner image is 10 pixels from the left edge (near the center of the eyes) and the left light signal's position in the left combiner image is 5 pixels from the right edge (near the center of the eyes), the ratio of the distance from the right position to the center to the distance from the left position to the center is 2:1 (10:5). To move the object closer to the user, due to the 2:1 ratio, if the right position in the right combiner image and the left position in the left combiner image must be 3 pixels closer to each other, the right position must move 2 pixels toward the left edge, while the left position must move 1 pixel toward the right.

[0073] Figure 14F Example 6 illustrates that a virtual object moves in space in the X-axis direction (toward the right) and the Z-axis direction (toward the user), thereby moving from the original depth plane to a new depth plane. To achieve this, the positions of the right light signal and the corresponding left light signal in the right and left combiner images must be moved to the right in the X-axis direction and moved closer to each other in the X-axis direction, corresponding to a large convergence angle. Therefore, the positions of the right light signal and the corresponding left light signal in the right and left retinal images that form the virtual object must be moved to the left in the X-axis direction and moved away from each other in the X-axis direction. In other words, the right light signal and the corresponding left light signal from the light signal generator must be projected at different X-coordinate positions and two different X-coordinate positions (to the right and closer to each other) in the combiner image. Since the Y-coordinate of the virtual object remains unchanged, the right light signal and the corresponding left light signal are projected at the same Y-coordinate position in the combiner image. For example, when the virtual object's XYZ coordinates move from (0,0,100) to (10,0,50), the right light signal on the right combiner image moves from RCI (10,10) to RCI (7,10), and the left light signal on the left combiner image moves from LCI (10,10) to LCI (17,10). Consequently, the right light signal on the right retinal image moves from RRI (10,10) to RRI (13,10), and the left light signal on the left retinal image moves from LRI (10,10) to LRI (3,10).

[0074] Figure 14GExample 7 illustrates the movement of a virtual object in space along the X-axis (toward the right), Y-axis (downward), and Z-axis (toward the user), thereby moving from the original depth plane to a new depth plane. To achieve this, the positions of the right light signal and the corresponding left light signal in the right and left combiner images must be shifted to the right in the X-axis, downward in the Y-axis, and closer to each other in the X-axis, corresponding to a large convergence angle. Therefore, the positions of the right light signal and the corresponding left light signal in the right and left retinal images that form the virtual object must be shifted to the left in the X-axis, upward in the Y-axis, and farther from each other in the X-axis. In other words, the right light signal and the corresponding left light signal from the light signal generator must be projected onto two different X-coordinates (toward the right and closer to each other) and different Y-coordinates in the combiner image. For example, when the virtual object's XYZ coordinates move from (0,0,100) to (10,-10,50), the right light signal on the right combiner image moves from RCI(10,10) to RCI(7,12), while the left light signal on the left combiner image moves from LCI(10,10) to LCI(17,12). Consequently, the right light signal on the right retinal image moves from RRI(10,10) to RRI(13,8), while the left light signal on the left retinal image moves from LRI(10,10) to LRI(3,8).

[0075] Figure 14H Example 8 shows a method for moving a virtual object in the Z-axis direction. The object moves from a depth of field of 1m to a depth of field of 10m (far away from the user), and thus moves from the original depth plane to the new depth plane in space. When the space of region C contains a sufficient number of virtual binocular pixels, the virtual object can move smoothly through many intermediate virtual binocular pixels. In other words, when the right retinal image and the left retinal image contain a sufficient number of right pixels and left pixels, the user can see a large number of virtual binocular pixels in space. Figure 14H In the figure, the object represented by the dot moves from the first virtual binocular pixel at a depth of field of 1 meter to the second virtual binocular pixel at a depth of field of 10 meters, passing through different intermediate virtual binocular pixels. First, the convergence angle between the first redirected right light signal and the first redirected left light signal at the first virtual binocular pixel at a depth of field of 1 meter is 3.4 degrees.

[0076] If IPD = 60 mm, θ = 3.4 degrees.

[0077] Second, the convergence angle between the optical path extensions of the second redirected right light signal and the second redirected left light signal of the second virtual binocular pixel with a depth of field of 10 m is 0.34 degrees.

[0078] If IPD = 60 mm, θ = 0.34 degrees.

[0079] Third, calculate the intermediate virtual binocular pixels. The number of intermediate virtual binocular pixels can be calculated based on the difference in convergence angle between the first and second virtual binocular pixels and the number of pixels in the X-axis per degree of the FOB. The difference between the convergence angle of the first virtual binocular pixel (3.4 degrees) and the convergence angle of the second virtual binocular pixel (0.34 degrees) is 3.06. The number of pixels in the X-axis per degree of the FOB is 32. Assuming the total width of the scanned retinal image is 1280 pixels, its field of view (FOV) covers 40 degrees. Therefore, when a virtual object moves from the first virtual binocular pixel at a depth of field of 1 meter to the second virtual binocular pixel at a depth of field of 10 meters, there are approximately 98 (32 x 3.06) virtual binocular pixels available to represent this movement. These 98 virtual binocular pixels can be found using the aforementioned lookup table. Fourth, in this embodiment, the movement can be represented using 98 intermediate virtual binocular pixels, e.g., divided into 98 small movement steps. These 98 virtual binocular pixel right and left light signals are generated by right light signal generator 10 and left light signal generator 30, respectively, and projected onto the user's right and left retinas. Thus, the user can see the virtual object smoothly move from 1 meter to 10 meters through 98 intermediate positions.

[0080] Figure 14I Example 9 illustrates a method for moving a virtual object from a depth of field of 1m to a depth of field of 20cm (closer to the user) in the Z-axis direction, thereby moving from the original depth plane to the new depth plane in space. When the space of region C contains a sufficient number of virtual binocular pixels, the virtual object can be smoothly moved through many intermediate virtual binocular pixels. In other words, when the right retinal image and the left retinal image contain a sufficient number of right pixels and left pixels, the user can see a large number of virtual binocular pixels in space. (In Figure 14I In the figure, the object represented by the dot moves from the first virtual binocular pixel at a depth of field of 1m to the second virtual binocular pixel at a depth of field of 20cm, passing through different intermediate virtual binocular pixels. First, the convergence angle between the first redirected right light signal and the first redirected left light signal at the first virtual binocular pixel at a depth of field of 1m is 3.4 degrees.

[0081] If IPD = 60 mm, θ = 3.4 degrees.

[0082] Second, the convergence angle between the optical path extensions of the second redirected right light signal and the second redirected left light signal of the second virtual binocular pixel with a depth of field of 20 cm is 17 degrees.

[0083] If IPD = 60 mm, θ = 17 degrees.

[0084] Third, calculate the intermediate virtual binocular pixels. The number of intermediate virtual binocular pixels can be calculated based on the difference in convergence angle between the first and second virtual binocular pixels and the number of pixels in the X-axis per degree of the field of view (FOB). The difference between the convergence angle of the first virtual binocular pixel (3.4 degrees) and the convergence angle of the second virtual binocular pixel (17 degrees) is 13.6. The number of pixels in the X-axis per degree of the FOB is 32. Assuming the total width of the scanned retinal image is 1280 pixels, its field of view (FOV) covers 40 degrees. Therefore, when a virtual object moves from the first virtual binocular pixel at a depth of field of 1 meter to the second virtual binocular pixel at a depth of field of 20 cm, there are approximately 435 (32 x 13.6) virtual binocular pixels available to represent this movement. These 435 virtual binocular pixels can be found using the aforementioned lookup table. Fourth, in this embodiment, the movement can be represented using 435 intermediate virtual binocular pixels, e.g., divided into 435 small movement steps. These 435 virtual binocular pixel right and left light signals are generated by right light signal generator 10 and left light signal generator 30, respectively, and projected onto the user's right and left retinas. Thus, the user can see the virtual object smoothly move from 1 meter to 20 cm through 435 intermediate positions.

[0085] While the numerous technical features and advantages of the present invention have been described above, the disclosed functions and detailed structures are merely illustrative. The broadest interpretation of the claims encompasses modifications to the shape, size, and configuration of components based on the teachings of this specification, without departing from the spirit of the present invention.

[0086] The above embodiments are provided so that people with ordinary knowledge in the field can use this case. Various improvements and changes to this embodiment may be obvious to people with ordinary knowledge in the field. Under the premise that no new technical features are involved, the technical ideas and subject matter described herein can be applied to other embodiments. The scope of rights requested in this case is not used to limit the embodiments described herein and should be interpreted in the broadest scope. Other embodiments encompassed by the spirit of this case may also be included in this case. The scope of rights requested in this case also covers other improvements and changes and equivalents.

Claims

1. A system for displaying an object with depth of field, characterized in that Include: a right optical signal generator for generating a plurality of right optical signals of an object, and redirecting the plurality of right optical signals by a right combiner to directly project the plurality of right optical signals onto a retina of a user and form a retinal image of the object on the retina; and a left optical signal generator for generating a plurality of left optical signals of the object, and redirecting the plurality of left optical signals by a left combiner to directly project the plurality of left optical signals to another retina of a user and form another retinal image of the object on the other retina; Among them, each right optical signal has a corresponding left optical signal, Any right light signal redirected by the right combiner and its corresponding left light signal redirected by the left combiner form a right pixel and a left pixel pair. The right pixel and left pixel pair correspond to specific positions on the first retina and the other retina and a 3D position. When the user's eyes receive the right pixel and left pixel pair, they perceive a first virtual binocular pixel at the 3D position. The 3D position includes information about a first depth of field. The 3D position is the same as the intersection of the light paths of the right light signal redirected by the right combiner and the corresponding left light signal redirected by the left combiner, so that when the user gazes at the first virtual binocular pixel, the first depth of field of the first virtual binocular pixel is the same as the depth of the user's binocular gaze position, and the first depth of field is related to the convergence angle of the redirected right light signal and the redirected left light signal. The right combiner and the left combiner receive and redirect the plurality of right light signals and their corresponding left light signals toward the user to form a plurality of virtual binocular pixels. The right projection angle of the right light signal generator can be adjusted to change the angle of incidence of the plurality of right light signals on the right combiner, and the left projection angle of the left light signal generator can be adjusted to change the angle of incidence of the plurality of left light signals on the left combiner. This changes the convergence angle between each redirected right light signal and its corresponding redirected left light signal to form a plurality of virtual binocular pixels with different depths of field in the same virtual object or the same virtual image.

2. The system according to claim 1, wherein The first depth of field is determined by a first angle between the optical path extensions of a first redirected right light signal and a corresponding first redirected left light signal.

3. The system according to claim 1, wherein: A first redirected right light signal and a corresponding first redirected left light signal are at substantially the same height on the retinas of both eyes of the user.

4. The system according to claim 1, wherein: A first redirected right light signal is not a parallax of a corresponding first redirected left light signal.

5. The system according to claim 1, wherein: The plurality of right optical signals generated by the right optical signal generator are reflected only once before entering the user's retina, and the plurality of left optical signals generated by the left optical signal generator are reflected only once before entering the user's retina.

6. The system according to claim 1, wherein: The right optical signal generator is a right LBS projector, and the multiple right optical signals generated by the right LBS projector are reflected only once by the right combiner before entering the user's retina. The left optical signal generator is a left LBS projector, and the multiple left optical signals generated by the left LBS projector are reflected only once by the left combiner before entering the user's retina.

7. The system according to claim 1, wherein: The right combiner and the left combiner are transparent to ambient light.

8. The system according to claim 1, wherein: The user perceives the second redirected right light signal and the corresponding second redirected left light signal as a second virtual binocular pixel of the object at a second depth of field, wherein the second depth of field is related to a second angle between the second redirected right light signal and the corresponding second redirected left light signal.

9. The system according to claim 1, wherein: The right combiner and the left combiner are elliptical in shape. The right optical signal generator is disposed at a focal position of the right combiner, and the left optical signal generator is disposed at a focal position of the left combiner.

10. The system according to claim 1, wherein: Further including: a support structure wearable on the user's head; Wherein, the right optical signal generator and the left optical signal generator are mounted on the supporting structure; and The right merger and the left merger are mounted on the support structure and are arranged in the user's field of vision.

11. The system according to claim 10, wherein: The support structure is eyeglasses.

12. The system according to claim 11, wherein The lenses of the glasses have a degree and are equipped with the right merger or the left merger.

13. The system according to claim 11, wherein: The lens of the glasses has a degree and is integrally formed with the right merger or the left merger.

14. The system according to claim 11, wherein: A lens with a degree and the right merger or the left merger can be combined with each other and can be detached from each other.

15. The system according to claim 10, wherein: The right merger and the left merger are integrally formed into an integrated merger.

16. A method for displaying an object with depth of field, characterized in that: Include: generating a plurality of right-side optical signals of the object from a right-side optical signal generator; redirecting the plurality of right optical signals to the user's retina by a right combiner, and forming a retinal image of an object on the retina; generating a plurality of left-side optical signals of the object from a left-side optical signal generator; redirecting the plurality of left optical signals to another retina of the user by a left combiner, and forming another retinal image of the object on the other retina; Among them, each right optical signal has a corresponding left optical signal, Any right light signal redirected by the right combiner and its corresponding left light signal redirected by the left combiner form a right pixel and a left pixel pair. The right pixel and left pixel pair correspond to specific positions on the first retina and the other retina and a 3D position. When the user's eyes receive the right pixel and left pixel pair, they perceive a first virtual binocular pixel at the 3D position. The 3D position includes information about a first depth of field. The 3D position is the same as the intersection of the light paths of the right light signal redirected by the right combiner and the corresponding left light signal redirected by the left combiner, so that when the user gazes at the first virtual binocular pixel, the first depth of field of the first virtual binocular pixel is the same as the depth of the user's binocular gaze position, and the first depth of field is related to the convergence angle of the redirected right light signal and the redirected left light signal. The right combiner and the left combiner receive and redirect the plurality of right light signals and their corresponding left light signals toward the user to form a plurality of virtual binocular pixels. The right projection angle of the right light signal generator can be adjusted to change the angle of incidence of the plurality of right light signals on the right combiner, and the left projection angle of the left light signal generator can be adjusted to change the angle of incidence of the plurality of left light signals on the left combiner. This changes the convergence angle between each redirected right light signal and its corresponding redirected left light signal to form a plurality of virtual binocular pixels with different depths of field in the same virtual object or the same virtual image.

17. The method according to claim 16, wherein The first depth of field is determined by a first angle between the optical path extensions of a first redirected right light signal and a corresponding first redirected left light signal.

18. The method according to claim 16, wherein A first redirected right light signal and a corresponding first redirected left light signal are at substantially the same height on the retinas of both eyes of the user.

19. The method according to claim 16, wherein A first redirected right light signal is not a parallax of a corresponding first redirected left light signal.

20. The method of claim 16, wherein: The plurality of right optical signals generated by the right optical signal generator are reflected only once before entering the user's retina, and the plurality of left optical signals generated by the left optical signal generator are reflected only once before entering the user's retina.

21. The method according to claim 16, wherein The right optical signal generator is a right LBS projector, and the multiple right optical signals generated by the right LBS projector are reflected only once by the right combiner before entering the user's retina. The left optical signal generator is a left LBS projector, and the multiple left optical signals generated by the left LBS projector are reflected only once by the left combiner before entering the user's retina.

22. The method of claim 16, wherein: The right combiner and the left combiner are transparent to ambient light.

23. The method of claim 16, wherein: The right combiner and the left combiner are elliptical in shape. The right optical signal generator is disposed at a focal position of the right combiner, and the left optical signal generator is disposed at a focal position of the left combiner.

24. The method of claim 16, wherein : The right optical signal generator and the left optical signal generator are mounted on a support structure, and the support structure can be worn on the head of the user; and The right merger and the left merger are mounted on the supporting structure and are arranged in the user's field of vision.

25. The method of claim 24, wherein: The supporting structure is a pair of glasses, the lenses of the glasses have a degree and are loaded with the right side combiner or the left side combiner.

26. The method of claim 24, wherein: The right merger and the left merger are integrally formed into an integrated merger.

Images