Method for realizing near object display and AR (Augmented Reality) display equipment

Abstract

The invention provides a method for realizing near object display and AR display equipment, and belongs to the technical field of AR augmented reality, a designer sets a target position of virtual image T rendering according to the width of an optical machine eye box, and when eyes watch the virtual image T, the surrounding rotation angles # imgabs0 # and # imgabs1 # of a left optical machine eye box and a right optical machine eye box are calculated according to the target position; current left and right pupil center positions and left and right ray machine eye box center positions are calculated; according to the central positions of the left and right ray machine eye boxes, the ray machine eye boxes are designed at the position parallel to the X axis, then the left ray machine eye box is rotated by # imgabs2 # clockwise, and the right ray machine eye box is rotated by # imgabs3 # anticlockwise, so that the eyeball rotation center, the pupil center, the ray machine eye box center and the virtual image T are located on the same straight line; according to the method and the device, the requirement of a user for displaying a virtual screen at a short distance can be met in a small-field-angle display device only through physical parallax adjustment.

Description

Technical Field

[0001] The present application relates to the field of AR augmented reality technology, and specifically to a method for realizing near-object display and an AR display device. Background Art

[0002] Augmented reality glasses or see-through OST glasses are divided into two categories of optical lenses: (1) free-form surfaces and birdbath BB, collectively known as reflective optics or Reflective Optic; and (2) array optical waveguides and diffraction optical waveguides, collectively known as waveguide optics or Waveguide Optic. Glasses architecture based on reflective optics, such as Figure 1 As shown in Figure 1, the projector / display screen is placed above the lens to project and reflect light downward onto a focused display surface. The display surface has an eye box area of ​​about 10 mm wide and high for the user's eyes to watch. Figure 2 As shown, a projector is placed to the side of the lens and couples an image into it. The image undergoes total internal reflection within the lens and propagates to an area with a refractive structure. This refractive structure creates an eyebox approximately 10mm wide and high toward the eyeball for the user's viewing experience. Alternatively, hybrid waveguide-based eyeglass architectures utilize the layered refraction principle of optical waveguides to make reflective optical lenses thinner or, in other words, to increase the field of view of waveguide optical lenses.

[0003] If the interpupillary distance (IPD) between the two eyes is fixed, a larger FOV (Field of View) results in a larger cross-sectional area, allowing for a wider range of objects to be zoomed in and displayed. Existing AR glasses have a very small field of view, making it impossible to create a left-right parallax effect on the virtual screen, and thus unable to bring the virtual screen closer. They can only display a 100-inch virtual screen at a distance (e.g., 7 meters).

[0004] Because AR glasses have only a small eyebox (small field of view), if the eyebox doesn't move, the image seen through the eyebox will only shrink until it disappears as the image gets closer, gradually reducing the parallax. To meet user needs for different virtual image positions and distances, and to achieve the function of focusing on close objects in close-object display devices with a small field of view, Huawei (CN116009253A) and Oppo (CN119045192A) have each proposed methods for optical waveguide glasses to change the virtual image position by changing the image projection angle. However, both methods have the following problems: (1) Ignoring the eye box: The entire optical waveguide lens is mistakenly regarded as a plane that can display images. It is ignored that only the light guide array at a specific position of the lens can output the image vertically at the set angle, and the focusing area of ​​the light guide (eye box area) is very small. It can only be seen clearly when it is vertically aligned with the user's eyes within a range of about 10mm. If the coupling angle of the projector is changed, the coupling position of the image with the changed projection angle in the medium with the same refractive index will change. If it can only be coupled from the originally designed eye box area, the coupled image will be distorted, and the eye will see a distorted image.

[0005] (2) Confusing pupil distance and parallax: The two patents also mention that if the image is not aligned with the user's eyeball, the image direction can be changed to align with the user's eyeball by changing the optical machine angle. This concept is wrong. Not to mention that it is impossible to export the image outside the eye box, even if it is really possible to export the image outside the eye box, such as Figure 3 As shown, the outcoupling angle of this image is also oblique, and it cannot be projected vertically to the pupil. The oblique image cannot be focused on the macula area in the eyeball, and the user still cannot see the image.

[0006] Guangli (CN115561911A) discloses an AR display device and an AR head-mounted device. The waveguide lens is connected to the device body through a connecting seat; the waveguide lens and the connecting seat are connected by a rotating shaft; the rotating shaft is connected to a driving component; the driving component is used to drive the waveguide lens to swing left and right with the vertical axis as the rotation axis through the rotating shaft. By flipping the waveguide lens left and right, the distance of the combined image can be changed. By flipping the optical lens outward and up and down and rotating left and right with the center of the lens as the central axis (such as Figure 4 As shown in the figure, the projected image can achieve (1) alignment with the eyeball (pupillary distance problem) and (2) change the parallax (image distance). Since the lens does not rotate around the center of the eyeball, but around the center of the optical engine / display module, and the direction of the light coupled out of the eye box is perpendicular to the direction of the lens, when the lens angle changes, the projected image will also change its angle, and it will not be able to enter the user's eyeball vertically and correctly focus on the central area of ​​the macula in the eye. Therefore, the same problem exists as Huawei and Oppo. Summary of the Invention

[0007] The purpose of this application is to provide a method for realizing near-object display and an AR display device. Designers calculate the rotation angles of the left and right optical machine eye boxes, realize physical parallax adjustment by rotating the optical machine eye boxes, and display virtual objects at the target position.

[0008] The present application provides a method for achieving near-object display, which is applicable to scenarios where an optical machine eye box is fixed in position. The optical machine includes a projector and an optical lens having a display focus area. The display focus area of ​​the optical lens or the spatial range in which the human eye can see a focused image is defined as the optical machine eye box. The angle by which the optical machine eye box rotates around the rotation center of the eyeball to achieve physical adjustment of parallax is defined as the optical machine eye box rotation angle, wherein: The XY coordinate system is established with the midpoint of the line connecting the left and right eyeball rotation centers as the origin O(0,0). The line connecting the left and right eyeball rotation centers is set as the X axis. The positions of the left and right eyeball rotation centers are , The pupil distance IPD is the distance between the left and right eyeball rotation centers, and the preset optical machine eye box width K, field of view angle FOV, the distance IPD between the left and right eyeball rotation centers, the distance r between the eyeball rotation center and the pupil center, and the optical machine eye box center (X eb ,Y eb ) and the distance from the pupil center, set and The rotation angle of the left and right optical machine eye boxes; The designer sets the target position (0, D) for rendering the virtual image T according to the width K of the optical machine eye box. D is the vertical distance between the target position and the X axis. When the eyes look forward to infinity, the pupil center is directly above the eyeball rotation center when viewed from a bird's-eye view. That is, the initial position of the pupil center is , When the binoculars are looking at the virtual image T, the left and right optical eye box rotation angles are calculated based on the target position (0, D) and ; Calculate the current left and right pupil center positions and the left and right optical eye box center positions; According to the center position of the left and right optical eye boxes, first design the optical eye box in a position parallel to the X axis, and then rotate the left optical eye box clockwise Rotate the right optical machine eye box counterclockwise ; The positions of the left and right pupil centers, the left and right eyeball rotation centers, the left and right optomechanical eyebox centers, and the virtual image T are known, so that the eyeball rotation center, pupil center, optomechanical eyebox center, and virtual image T are located on the same straight line, and the left and right optomechanical eyeboxes with physically adjusted parallax are fixed.

[0009] The left and right optical eye box rotation angles are calculated based on the target position (0, D) and : If the virtual image T is a single pixel, the rotation angle of the left and right optical eye boxes can be calculated by formula (1): or : (1).

[0010] If the virtual image is not a single pixel point and is a regular object, when calculating the left optical machine eye box rotation angle And the right optical machine rotation angle Before, calculate the center pixel of the virtual image and set it as the virtual image T; If the virtual image is not a single pixel and is an irregular object, when calculating the left optical machine eye box rotation angle, And the right optical machine rotation angle Before, the weighted average position is calculated based on the weight of the three-dimensional position of each pixel's XY and distance Z, and is set as the virtual image T; If the virtual image is a virtual screen, and the display screen covers the entire optical eye box, the leftmost pixel and the rightmost pixel position of the virtual screen are known, and the left optical eye box rotation angle is calculated. And the right optical machine rotation angle Before the image is taken, the mean of the left and right end pixel positions is calculated and set as the virtual image T.

[0011] The left and right optical eye box rotation angles are calculated based on the target position (0, D) and : If the virtual image is a virtual screen, and the pixels of the display screen cover the entire optical eye box, the leftmost pixel of the virtual screen is known to be and the rightmost pixel Position, leftmost pixel and the rightmost pixel The distance value, that is, the width of the virtual screen, is set to the leftmost pixel The distance from the normal to the center of the left eyeball is and the leftmost pixel The rotation angle of the left optical machine eye box for the virtual image is , the leftmost pixel The distance from the normal to the center of the right eyeball is and the leftmost pixel The rotation angle of the right optical machine eye box for the virtual image is , , the rightmost pixel The distance from the normal line of the right eyeball center is and the rightmost pixel The rotation angle of the right optical machine eye box for the virtual image is , the rightmost pixel The distance from the normal to the center of the left eyeball is and the rightmost pixel The rotation angle of the left optical machine eye box for the virtual image is , , calculate the rotation angles of the four optical eye boxes 、 、 and : By pupil distance IPD and rightmost pixel Position, get the rightmost pixel Distance from the normal to the center of the right eyeball , then the right optical machine eye box rotation angle ; By pupil distance IPD and leftmost pixel Position, calculate the leftmost pixel Distance from the normal to the center of the right eyeball , then the right optical machine eye box rotation angle ; By pupil distance IPD and leftmost pixel Position, calculate the leftmost pixel Distance from the normal of the left eyeball center , then the left optical machine eye box rotation angle ; By pupil distance IPD and rightmost pixel Position, calculate the rightmost pixel Distance from the normal of the left eyeball center , then the left optical machine eye box rotation angle ; Rotate the right optical machine eye box around the angle and Calculate the average value to get the rotation angle of the right optical machine eye box , rotate the left optical machine eye box around the angle and Calculate the average value to get the rotation angle of the left optical machine eye box : and .

[0012] Calculate the current left and right pupil center positions and the left and right optical eye box center positions: The left pupil center position is calculated according to formula (2): and the center of the right pupil : (2); According to the distance r between the eyeball rotation center and the pupil center and the center of the optical machine eye box (X eb ,Y eb ) and the distance from the pupil center, and calculate the center position of the optical machine eye box (X eb ,Y eb ) and the distance r from the center of eye rotation eb ,set up , then use formula (2) to calculate the center position of the left optical machine eye box (X Leb ,Y Leb ) and the center position of the right optical machine eye box (X Reb ,Y Reb ).

[0013] If the virtual image is a virtual screen, after fixing the left and right optical eye boxes with physically adjusted parallax, let the display screen fill the entire optical eye box: Calculate the center position of the right optical machine eye box (X Reb ,Y Reb ) and the center position of the left optical machine eye box (X Leb ,Y Leb ), set the two end points of the right optical machine eye box center position as the right eye box ear position (X RE ,Y RE ) and the right eye box nose position (X RB ,Y RB ), the distance between the two end points is the width of the optical eye box, and the two end points passing through the center of the left optical eye box are the left eye box ear position (X LE ,Y LE ) and the left eye box nose position (X LN ,Y LN ), the distance between the two end points is the width of the optical eye box; let w = optical eye box width / 2, the slope of the right optical eye box , left optical machine eye box slope ; Calculate the right eye box position close to the ear (X RE ,Y RE ) and the right eye box nose position (X RB ,Y RB ): (4) Calculate the left eye box position close to the ear (X LE ,Y LE ) and the left eye box nose position (X LN ,Y LN ): (5); Adjust the positions of the two end points of the display screen of the optical eye box, which are close to the ear side and the nose side, so that the display screen covers the entire optical eye box.

[0014] Align the endpoint of the left optical machine eye box on the nose side, and replace the left optical machine eye box with a width of K that has achieved physical parallax adjustment with a width of F L The optical eye box, and the left optical eye box is rotated with the end point on the nose side as the center. , The left optical machine eye box rotation angle, the left optical machine eye box after rotation Parallel to the X axis: (7); Similarly, do the same process for the right optical machine eye box.

[0015] This application provides another method for achieving near-object display, which is applicable to scenarios where an eye box based on eye tracking rotates following changes in the center position of the pupil. The optical machine includes a projector and an optical lens with a display focus area. The display focus area of ​​the optical lens or the spatial range where the human eye can see a focused image is defined as the eye box. The angle at which the eye box rotates around the rotation center of the eyeball to achieve physical adjustment of parallax is defined as the eye box rotation angle, where: The XY coordinate system is established with the midpoint of the line connecting the left and right eyeball rotation centers as the origin O(0,0). The line connecting the left and right eyeball rotation centers is set as the X axis. The positions of the left and right eyeball rotation centers are , , IPD is the distance between the left and right eyeball rotation centers, preset the optical machine eye box width K, field of view angle FOV, the distance IPD between the left and right eyeball rotation centers, the distance r between the eyeball rotation center and the pupil center, and the optical machine eye box center (X eb ,Y eb ) and the distance from the pupil center, set and is the rotation angle of the left and right eye boxes; the target position (0, D) of the virtual image T rendering is set according to the width of the eye box, and D is the vertical distance between the target position and the X axis. When the eyes look forward to infinity, the pupil center is seen from the top view directly above the eye rotation center, that is, the initial position of the pupil center is , ; When binocularly looking at the virtual image T, the position of the right eye tracking sensor facing the right eye is (X C ,Y C ), the right eye tracking sensor is used to collect the angle between the right pupil and the vertical line of the X axis , the right optical machine eye box rotation angle is obtained through triangulation calculation : (6); Where t= ; Similarly, the position of the left eye tracking sensor facing the left eye is (X C ,Y C ), X C Take the negative value on the left side of the origin and use the same formula to calculate the rotation angle of the left optical machine eye box. ; Calculate the current left and right pupil center positions and the left and right optical eye box center positions; According to the center position of the left and right optical eye boxes, first design the optical eye boxes to be parallel to the X-axis; The control device calculates the rotation angle of the left and right optical machine eye boxes and Then, the left and right optical eye boxes are rotated by the optical eye box drive components. and , the left and right pupil center positions, the left and right eyeball rotation center positions, the left and right optomechanical eye box center positions and the virtual image T position are known, so that the eyeball rotation center, pupil center, optomechanical eye box center and virtual image T are located on the same straight line.

[0016] The display screen in each optical machine eye box corresponds to all or part of the virtual screen, and the contents of the display screens in the left and right optical machine eye boxes correspond to all or part of the virtual screen after being spliced.

[0017] The present application provides an AR display device, including a device body, wherein the device body includes a connected optical machine, and the optical machine includes a projector and an optical lens having a display focus area. The display focus area of ​​the optical lens or the spatial range in which the human eye can see a focused image is an optical machine eye box, and the optical machine eye box is designed by any of the above-mentioned methods for achieving near-object display.

[0018] The technical solution of this application does not require special optical design or sacrifice of field of view, and can achieve near-field display of virtual objects through physical parallax adjustment alone. This application can meet users' needs for different virtual image positions and distances, and realize the function of focusing on close objects in display devices with a small field of view, especially meeting users' needs for displaying virtual screens at close range. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 Schematic diagram of the optical path of glasses based on reflective optics; Figure 2 This is a schematic diagram of the optical path of glasses based on diffraction optics; Figure 3 Schematic diagram showing that the image coupled out of the eye box cannot be projected vertically to the pupil due to changing the angle of the projection optical machine; Figure 4 This is a diagram showing how the light particle pattern rotates left and right around the center of the lens to change the distance of the combined image. Figure 5 This is a schematic diagram of the structure of the right eyeball of a human eye; Figure 6 The virtual image T in this application is a top view of a single pixel point located in the vertical center direction of the human eye; Figure 7 A top view of parallax alignment of multiple pixels in the virtual screen of this application; Figure 8For this application, an eye tracking sensor is used to calculate the rotation angle of the left and right optical machine eye boxes. Figure 9 This is a control schematic diagram of the AR display device based on the eye tracking sensor in this application; Figure 10 This is a top view of the eye box following the center of the pupil using the eye tracking optical machine in this application. The left image shows a top view of both eyes looking at the middle of the screen, and the right image shows a top view of both eyes looking at the rightmost side of the screen. Figure 11 A top view showing the unequal distribution of light rays corresponding to the equidistant grid of the virtual screen on the screen displayed by the rotating optical eye box; Figure 12 for Figure 11 Enlarged view of middle area A; Figure 13 This is a top view of the optical machine eye box F set horizontally to simulate the optical machine eye box K rotating around to achieve physical adjustment of the parallax effect; Figure 14 for Figure 13 The enlarged image of the equidistant distribution of light rays corresponding to the equidistant grid of the virtual screen on the screen displayed by the horizontally set optical eye box F; Figure 15 for Figure 13 A magnified image of the uneven distribution of light rays on the screen displayed by the rotating optical machine eye box K; Figure 16 This is a functional block diagram of the AR head display device of Example 3 of this application. DETAILED DESCRIPTION

[0020] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0022] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or solution described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or solutions. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner. The principle of this application is as follows: (1) Definition of professional terms mentioned in the article: Optical machine = display module = display module: refers to the device that combines a projector and an optical lens with a display focus area. The projector generates the projection content, which is guided by the optical lens and then coupled out of the display focus area before entering the human eye. Eyebox: The spatial range within which the human eye can see a focused image. This refers to the spatial range within which the image, refracted from the projector to the optical lens, can be clearly seen. Assuming the distance between the optical lens and the human eye remains constant, manufacturers generally only need to specify the eyebox width. This application uses the common eyebox width of 10mm as an example. In order to illustrate the technical solution more clearly, the complex structure of components such as the projector and optical lenses is omitted in the drawings given in this application, and the optical eye box is used. To indicate the position where the human eye can see the focused image projected by the projector, the optical eye box surface is perpendicular to the central axis of the eyeball.

[0023] like Figure 5As shown, the content seen by the user's eyes, or ambient light, passes through the cornea, through the pupil between the iris, and then through the lens, where it is refracted into the eye's lens. Ambient light is reflected in the center of the eye and projected onto the retina, centered at the macula. Retinal cells detect the ambient light's grayscale (rods) and color (cones), and then transmit it to the brain via the optic nerve. The eyeball can rotate up, down, left, right, and diagonally via six extraocular muscles. During rotation, the center of the eyeball is not at the back, but at the center of the lens, the center point where the ambient light image crosses and inverts. This application refers to the center of the eyeball as the eye's rotation center. As shown in the figure, the distance from the cornea to the lens is 3 mm, and the diameter from the front of the lens to the center of the macula is 20 mm. The radius from the eyeball's rotation center to the cornea is 10 mm + 3 mm = 13 mm. This radius is used as the distance from the eyeball's rotation center to the pupil center for parallax calculations.

[0024] Eyebox rotation angle: This application refers to the display focal area of ​​the optical lens or the spatial range in which the human eye can see the focused image as the eyebox. The eyebox rotation angle refers to the angle through which the eyebox is rotated around the eye's rotation center to physically adjust the parallax. (2) Methods for realizing near-object display: The calculation of this application assumes that the pupil distance has been adjusted for the optical machine eye box.

[0025] The XY coordinate system is established with the midpoint of the line connecting the left and right eyeball rotation centers as the origin O(0,0). The line connecting the left and right eyeball rotation centers is set as the X axis. The positions of the left and right eyeball rotation centers are , , IPD is the distance between the left and right eyeball rotation centers, the preset distance r between the eyeball rotation center and the pupil center (set to 13mm), the optical machine eye box center (X eb ,Y eb ) and the distance between the pupil center (set to 2mm), then the optical eye box center (X eb ,Y eb ) to the eyeball rotation center is 15mm; preset the field of view angle FOV (for example, 50°), the distance between the left and right eyeball rotation centers (equivalent to the pupil distance IPD, set to 70mm) and the optical machine eye box width K (for example, 10mm), set and This is the angle at which the left and right eye boxes rotate. This is also the angle at which the left and right eyeballs need to rotate to align the pupils with the eye boxes. , at this time there is no parallax, and the virtual image is rendered at an infinite distance.

[0026] If the designer wants the virtual image T to be rendered at the target position (0, D), where D is the vertical distance between the target position and the X-axis, the left and right optical eye box rotation angles are calculated based on the target position (0, D). and , then calculate the current left and right pupil center positions and the left and right optical eye box center positions. According to the left and right optical eye box center positions, first design the optical eye box in a position parallel to the X axis, and then rotate the left optical eye box clockwise Rotate the right optical machine eye box counterclockwise Given the positions of the left and right pupil centers, the left and right eyeball rotation centers, the left and right optical eyebox centers, and the virtual image T, make the eyeball rotation centers, pupil centers, optical eyebox centers, and virtual image T lie on the same straight line. Fix the left and right optical eyeboxes with physically adjusted parallax, and give the following specific steps according to different usage scenarios: A. Single pixel method: Step 1 (Calculate the rotation angle of the optical machine eye box): When the eyes look forward to infinity, the pupil center is directly above the eyeball rotation center from a bird's-eye view, which is the initial position of the pupil center. for , ,When binocularly looking at the virtual image T, if the virtual image T is a single pixel, such as Figure 6 As shown, the left and right optical eye box rotation angles are calculated by formula (1) or : (1); Substituting the target position (0, D) of the virtual image T, we can get the angle of rotation of the left and right optical machine eye boxes. ; Step 2 (calculate the current pupil center position and the optical eye box center position): Calculate the left pupil center position according to formula (2) and the center of the right pupil : (2) If the angle of the left and right optical machine eye box is and If they are the same, then formula (2) can be simplified to: (3) According to the distance r between the eyeball rotation center and the pupil center and the center of the optical machine eye box (X eb ,Y eb ) and the distance from the pupil center, and calculate the center position of the optical machine eye box (X eb ,Y eb ) and the distance r from the center of eye rotation eb ,set up , then use formula (2) or formula (3) to calculate the center position of the left optical machine eye box (X Leb ,Y Leb ) and the center position of the right optical machine eye box (X Reb ,Y Reb ); Continue from the previous For example, assuming that the left and right eye boxes have the same rotation angle , the distance from the eyeball rotation center to the pupil center r = 13mm, X Rp =35-13*sin(4º)=34.09mm, Y Rp =13*cos(4º)=12.97mm, then the center position of the right pupil (X Rp ,Y Rp ) is (34.09,12.97); Since the left pupil is the same angle to the left of the origin, the center position of the left pupil (X Lp ,Y Lp )=(-34.09mm,12.97mm), and assuming that the distance r between the center of the optical machine eye box and the center of eye rotation is eb = 25mm, the center position of the right optical machine eye box (X Reb ,Y Reb )=(33.25,24.94), the center position of the left optical machine eye box (X Leb ,Y Leb )=(-33.25,24.94).

[0027] Step 3 (Fix the optical eye box that has achieved physical parallax adjustment): According to the center position of the left and right optical eye boxes, first design the optical eye box to be parallel to the X-axis, and then rotate the left optical eye box clockwise Rotate the right optical machine eye box counterclockwise , the left and right pupil center positions, the left and right eyeball rotation center positions, the left and right optomechanical eye box center positions and the virtual image T position are known, so that the eyeball rotation center, pupil center, optomechanical eye box center and virtual image T are located on the same straight line.

[0028] B. Method where the virtual image is multiple pixels: If the virtual image is not a single pixel point and is a regular object, then the center pixel point of the virtual image is set as the virtual image T, and the left optical machine eye box rotation angle is calculated according to the single pixel method. And the right optical machine rotation angle ; If the virtual image is not a single pixel and is an irregular object, then the weighted average position of all pixels is set as the virtual image T, and the left optical machine eye box rotation angle is calculated according to the single pixel method. And the right optical machine rotation angle The weighted average position of all pixels refers to the pixel position obtained by averaging the weights of the three-dimensional position of each pixel based on its XY and distance Z.

[0029] If the virtual image is a virtual screen, and the pixels of the virtual screen cover the entire optical eye box, the target position of the preset virtual screen is parallel to the X axis and the vertical distance from the X axis is D, the leftmost pixel of the virtual screen is known to be and the rightmost pixel Position, the average of the left and right end pixel positions is taken as the pixel position, and it is set as the virtual image T. The angle of the left optical machine eye box rotation is calculated according to the single pixel method The angle around the right optical machine eye box .

[0030] Another method is given below using the virtual image as a virtual screen as an example.

[0031] Step 1 (calculate the rotation angle of the optical machine eye box): Figure 7 This is a top view of the parallax alignment of multiple pixels on the virtual screen, showing that the pixels of the screen cover the entire optical eye box. The target position of the virtual screen is parallel to the X-axis and the vertical distance from the X-axis is D. It is known that the leftmost pixel of the virtual screen and the rightmost pixel Position (screen pixel coordinates), leftmost pixel and the rightmost pixel The distance value (the width of the virtual screen) is set to the leftmost pixel The distance from the normal to the center of the left eyeball is and the leftmost pixel The rotation angle of the left optical machine eye box for the virtual image is , the leftmost pixel The distance from the normal line of the right eyeball center is and the leftmost pixel The rotation angle of the right optical machine eye box for the virtual image is , , the rightmost pixel The distance from the normal line of the right eyeball center is and the rightmost pixel The rotation angle of the right optical machine eye box for the virtual image is , the rightmost pixel The distance from the normal to the center of the left eyeball is and the rightmost pixel The rotation angle of the left optical machine eye box for the virtual image is , , calculate the rotation angles of the four optical eye boxes 、 、 and : By pupil distance IPD and rightmost pixel Position, get the rightmost pixel Distance from the normal to the center of the right eyeball , then the right optical machine eye box rotation angle ; By pupil distance IPD and leftmost pixel Position, calculate the leftmost pixel Distance from the normal to the center of the right eyeball , then the right optical machine eye box rotation angle ; By pupil distance IPD and leftmost pixel Position, calculate the leftmost pixel Distance from the normal of the left eyeball center , then the left optical machine eye box rotation angle ; By pupil distance IPD and rightmost pixel Position, calculate the rightmost pixel Distance from the normal of the left eyeball center , then the left optical machine eye box rotation angle ; Rotate the right optical machine eye box around the angle and Calculate the average value to get the rotation angle of the right optical machine eye box , rotate the left optical machine eye box around the angle and Calculate the average value to get the rotation angle of the left optical machine eye box : and .

[0032] Steps 2 and 3 are the same as for single pixel. Step 4 (Adjust the position of the display screen on the ear side and nose side in the left and right optical eye boxes so that the display screen covers the entire optical eye box): Figure 6 As shown in step 2, the center position of the right optical machine eye box (X Reb ,Y Reb ) and the center position of the left optical machine eye box (X Leb ,Y Leb ), set the two end points of the right optical machine eye box center position as the right eye box ear position (X RE ,Y RE ) and the right eye box nose position (X RB ,Y RB ), the distance between the two end points is the width K of the optical eye box, and the two end points passing through the center of the left optical eye box are the left eye box ear positions (X LE,Y LE ) and the left eye box nose position (X LN ,Y LN ), the distance between the two end points is also the width of the optical eye box; let w = optical eye box width / 2, the optical eye box slope , then the right optical machine eye box inclination , left optical machine eye box slope , calculate the right eye box position close to the ear (X RE ,Y RE ), right eye box near the nose position (X RB ,Y RB ), the left eye box near the ear position (X LE ,Y LE ) and the left eye box nose position (X LN ,Y LN ); The formula for calculating the positions of the two endpoints of the center position of the right optical machine eye box is: (4) The formula for calculating the positions of the two endpoints of the left optical machine eye box center position is: (5).

[0033] Continuing the previous example, i=35mm, r eb =25mm, X Reb =33.25mm, Y Reb =24.94mm, then: (X RE ,Y RE )=(38.24,25.29),(X RN ,Y RN )=(28.27,24.59); X Leb =-33.25mm, Y Leb =24.94mm, then: (X LE ,Y LE )=(-38.24,25.29),(X LN ,Y LN )=(-28.27,24.59).

[0034] During the design process, the designer (1) presets the target position (0, D) of the virtual screen according to the eye box width; (2) uses the center point of the virtual screen as the virtual image T and uses the single pixel method to calculate the angle of rotation of the left optical machine eye box at a fixed distance D. The angle around the right optical machine eye box ; (3) Adapt the projected video stream pixels to the glasses' optical machine pixels according to the adaptation ratio, where the adaptation ratio = glasses' optical machine pixels / projected video stream pixels. For example, if the glasses' optical machine screen width is 2000 pixels and the eye box width is 10mm, it means that when the center of the pupil is aligned with the center of the optical machine eye box, 2000 pixels can be seen within a width of 10mm. Assuming that the projected video stream pixels are 4000, the projected video stream pixels should be divided by 2 to adapt to the glasses' optical machine pixels. Similarly, if the projected video stream pixels are 1000, the projected video stream pixels should be multiplied by 2 to adapt to the glasses' optical machine pixels; (4) Set the distance r between the center of the optical machine eye box and the center of eyeball rotation. eb = 25mm, the center position of the right optical machine eye box (X Reb ,Y Reb ) = (33.26,24.94), the center position of the left optical machine eye box (X Leb ,Y Leb )=(-33.26,24.94); (5) Design the left and right optical eye boxes at this position and parallel to the X-axis, and rotate the right optical eye box 4° counterclockwise and the left optical eye box 4° clockwise to achieve physical parallax adjustment, and move the screen from infinity to the target position; (6) Adjust the positions of the two end points of the display screen of the optical eye box on the ear side and the nose side so that the display screen covers the entire optical eye box.

[0035] C. Method for calculating the rotation angle of the left and right optical machine eye boxes using the eye tracking sensor: Step 1. Figure 8 As shown, let r be the distance from the eyeball rotation center to the pupil center. When the eyes look forward to infinity, the pupil center is directly above the eyeball rotation center. The initial positions of the left and right pupil centers are for , ,When binocularly looking at the virtual image T, the position of the right eye tracking sensor facing the right eye is (X C ,Y C ), the right eye tracking sensor is used to collect the angle between the right pupil and the vertical line of the X axis , the right optical machine eye box rotation angle is obtained through triangulation calculation : (6); in, ; The left eye tracking sensor position facing the left eye is (X C ,Y C ), because X C It is on the left side of the origin, so it will be a negative value. The same formula can be used to calculate the rotation angle of the left optical machine eye box ; Steps 2 and 3 are the same as the single pixel method.

[0036] The above is a design scheme in which the optical machine eye box is fixed after physically adjusting the parallax.

[0037] According to the principle of method C, eye tracking can allow the optical eye box to adjust its rotation angle according to the changes in the pupil center position, such as Figure 10 As shown, each optical eye box only needs to display part of the virtual screen, and the virtual screens displayed by the left and right optical eye boxes are spliced ​​together to form a screen that is larger than the width of a single optical eye box. This means that Figure 10 In the left picture, the two eyes do not see the entire screen, but only the middle part. Through eye tracking, the left and right optical eye boxes rotate according to the position of the pupil center, so that the center of the eyeball, the center of the pupil, and the center of the optical eye box are always on the same line. In this way, the two eyes can look to the left or right of the larger screen, and the correct video can be displayed according to the viewing angle of the eyeball. Figure 10 In the right image, you can see both eyes looking toward the far right of the screen. The left and right eyeboxes display different screen areas. The right eye sees a smaller, more vertical area than the left eye. The black area in the image represents the area seen by both eyes. Because the left eye is viewing at a more oblique angle, it also sees a gray portion of the screen. Because the eyeboxes are tilted, the screen ratio also adjusts based on distance.

[0038] Since there is a distance between the virtual object and the eyeball, the farther away the pixel is from the eyeball's rotation center, the smaller it should be, and vice versa. Figure 11 As shown in the figure, the virtual screen is divided into several equidistant grids. From the figure, we can see that the light corresponding to the equidistant grids of the virtual screen is unevenly distributed on the screen of the optical machine eye box. Figure 12 From the enlarged image, we can see that the grids are larger when the distance is close, and the grids are narrower when the distance is farther. The display pixels of the optical eye box need to correct the image distortion according to the distance.

[0039] When rendering virtual objects, the pixels displayed on the screen should be based on the position and distance between the eyeball's rotation center and the virtual object. If the eye and the optomechanical eyebox are facing the center of the screen, the virtual object displayed close to the eye will be too large, possibly exceeding the visible range of the eyebox. Conversely, screen pixels farther away will be scaled down, resulting in wasted optomechanical screen pixels on the right side of the screen. Therefore, when the optomechanical eyebox is aimed at a virtual object, we should consider that the closer the pixel, the more important it is. The weighted average position should be calculated based on the weights of the XY and Z three-dimensional position of each pixel of the virtual object. The optomechanical automatic rotation of the glasses should guide the user to view the weighted average position of the three-dimensional pixels of the virtual object.

[0040] D. Method for achieving physical parallax adjustment effect by simulating the rotation of an optomechanical eye box set parallel to the X-axis: The AR optical lens technology will get better and better in the future. When the field of view of the optical eye box becomes larger and larger, Figures 13 to 15 It can be seen that there is no need to rotate the optical eye box, but the optical eye box F set parallel to the X axis can directly simulate the rotation of the optical eye box K to achieve the effect of physical adjustment of parallax. The advantage of this is that the display pixels on the optical eye box F set parallel to the X axis do not need to be like Figure 11 The central rotating optical eye box is used to correct distortion at near and far distances.

[0041] like Figure 13 As shown in the figure, it is known that the width of the optical eye box K = 10mm after the rotation to achieve physical adjustment of parallax, and the rotation angle of the optical eye box , then rotate the optical machine eye box with the end point on the ear side as the center of the circle , so that the optical machine eye box is set parallel to the X-axis. How large does the width F of the optical machine eye box set parallel to the X-axis need to be to simulate the projection effect of the rotating optical machine eye box? (7) F=10*(0.785+0.707*0.707)=14, that is to say, if the optical machine eye box rotates at an angle of 45 degrees, as long as the width F of the optical machine eye box set parallel to the X axis can reach 14 mm, it can simulate the virtual image displayed by the optical machine eye box with a width K of 10 mm after physically adjusting the parallax by rotating 45 degrees. Example 1 The first embodiment of the present application provides a method for realizing near object display. An XY coordinate system is established with the midpoint of the line connecting the left and right eyeball rotation centers as the origin O(0,0). The line connecting the left and right eyeball rotation centers is set as the X axis. The positions of the left and right eyeball rotation centers are , , IPD is the distance between the left and right eyeball rotation centers, preset the optical machine eye box width K, field of view angle FOV, the distance IPD between the left and right eyeball rotation centers, the distance r between the eyeball rotation center and the pupil center, and the optical machine eye box center (X eb ,Y eb ) and the distance from the pupil center, set and The rotation angle of the left and right optical machine eye boxes; The designer sets the target position (0, D) for virtual image T rendering according to the width of the optical machine eye box. D is the vertical distance between the target position and the X axis. When the eyes look forward to infinity, the pupil center is directly above the eyeball rotation center when viewed from a top-down perspective. That is, the initial position of the pupil center is , , the left and right optical eye box rotation angles calculated based on the target position (0, D) and ; Calculate the left and right pupil center positions and the left and right optical eye box center positions; Based on the left and right optical eye box center positions, first design the optical eye box in a position parallel to the X-axis, and then rotate the left optical eye box clockwise Rotate the right optical machine eye box counterclockwise Given the positions of the left and right pupil centers, the left and right eyeball rotation centers, the left and right optomechanical eyebox centers, and the virtual image T, make the eyeball rotation centers, pupil centers, optomechanical eyebox centers, and virtual image T lie on the same straight line. Fix the left and right optomechanical eyeboxes with physically adjusted parallax. Specific steps: Step 1 (calculate the rotation angle of the optical machine eye box): If the virtual image T is a single pixel, the rotation angles of the left and right optical machine eye boxes are calculated by formula (1): or : (1); If the virtual image is not a single pixel point and is a regular object, then the center pixel point of the virtual image is set as the virtual image T, and the left optical machine eye box rotation angle is calculated according to the single pixel method. And the right optical machine rotation angle ; If the virtual image is not a single pixel and is an irregular object, the weighted average position is calculated based on the weight of the three-dimensional position of each pixel's XY and distance Z, and is set as the virtual image T. The left optical machine eye box rotation angle is calculated according to the single pixel method. And the right optical machine rotation angle ; If the virtual image is a virtual screen, and the display screen covers the entire optical eye box, the preset target position of the virtual screen is parallel to the X-axis and the vertical distance from the X-axis is D. The leftmost and rightmost pixel positions of the virtual screen are known. The average of the left and right pixel positions is taken as the pixel position and set as the virtual image T. The left optical eye box rotation angle is calculated according to the single pixel method. and the right optical machine eye box rotation angle ; If the virtual image is a virtual screen, and the pixels of the display screen cover the entire optical eye box, the target position of the preset virtual screen is parallel to the X-axis and the vertical distance from the X-axis is D, such as Figure 7 As shown, the leftmost pixel of the known virtual screen and the rightmost pixel Position, leftmost pixel and the rightmost pixel The distance value, that is, the width of the virtual screen, is set to the leftmost pixel The distance from the normal to the center of the left eyeball is and the leftmost pixel The rotation angle of the left optical machine eye box for the virtual image is , the leftmost pixel The distance from the normal line of the right eyeball center is and the leftmost pixel The rotation angle of the right optical machine eye box for the virtual image is , , the rightmost pixel The distance from the normal line of the right eyeball center is and the rightmost pixel The rotation angle of the right optical machine eye box for the virtual image is , the rightmost pixel The distance from the normal to the center of the left eyeball is and the rightmost pixel The rotation angle of the left optical machine eye box for the virtual image is , , calculate the rotation angles of the four optical eye boxes 、 、 and : By pupil distance IPD and rightmost pixel Position, get the rightmost pixel Distance from the normal to the center of the right eyeball , then the right optical machine eye box rotation angle ; By pupil distance IPD and leftmost pixel Position, calculate the leftmost pixel Distance from the normal to the center of the right eyeball , then the right optical machine eye box rotation angle ; By pupil distance IPD and leftmost pixel Position, calculate the leftmost pixel Distance from the normal of the left eyeball center , then the left optical machine eye box rotation angle ; By pupil distance IPD and rightmost pixel Position, calculate the rightmost pixel Distance from the normal of the left eyeball center , then the left optical machine eye box rotation angle ; Rotate the right optical machine eye box around the angle and Calculate the average value to get the rotation angle of the right optical machine eye box , rotate the left optical machine eye box around the angle and Calculate the average value to get the rotation angle of the left optical machine eye box : and .

[0042] Step 2 (Calculate the pupil center position and the optical eye box center position): Calculate the left pupil center position according to formula (2) and the center of the right pupil : (2) If the angle of the left and right optical machine eye box is and If they are the same, then formula (2) can be simplified to: (3) According to the distance r between the eyeball rotation center and the pupil center and the center of the optical machine eye box (X eb ,Y eb ) and the distance from the pupil center, and calculate the center position of the optical machine eye box (X eb ,Y eb ) and the distance r from the center of eye rotation eb ,set up , then use formula (2) or formula (3) to calculate the center position of the left optical machine eye box (X Leb ,Y Leb ) and the center position of the right optical machine eye box (X Reb ,Y Reb ); Step 3 (Fix the optical eye box that has achieved physical parallax adjustment): According to the center position of the left and right optical eye boxes, first design the optical eye box to be parallel to the X-axis, and then rotate the left optical eye box clockwise Rotate the right optical machine eye box counterclockwise , given the positions of the left and right pupil centers, the left and right eyeball rotation centers, the left and right optomechanical eyebox centers, and the virtual image T, the eyeball rotation center, pupil center, optomechanical eyebox center, and virtual image T are located on the same straight line; If the virtual image is a virtual screen, the process also includes step 4 (adjusting the positions of the display screens on the ear side and the nose side in the left and right optical eye boxes so that the display screens cover the entire optical eye box): Figure 6 As shown in step 2, the center position of the right optical machine eye box (X Reb ,Y Reb ) and the center position of the left optical machine eye box (X Leb ,Y Leb ), set the two end points of the right optical machine eye box center position as the right eye box ear position (X RE ,Y RE ) and the right eye box nose position (X RB ,Y RB), the distance between the two end points is the width of the optical eye box, and the two end points passing through the center of the left optical eye box are the left eye box ear position (X LE ,Y LE ) and the left eye box nose position (X LN ,Y LN ), the distance between the two end points is also the width of the optical eye box; let w = optical eye box width / 2, then the slope of the right optical eye box is , left optical machine eye box slope , calculate the right eye box position close to the ear (X RE ,Y RE ), right eye box near the nose position (X RB ,Y RB ), the left eye box near the ear position (X LE ,Y LE ) and the left eye box nose position (X LN ,Y LN ); The two endpoints of the center position of the right optical machine eye box, i.e. the ear position of the right eye box (X RE ,Y RE ) and the right eye box nose position (X RB ,Y RB ) is: (4) The two endpoints of the center position of the left optical machine eye box, i.e. the left eye box ear position (X LE ,Y LE ) and the left eye box nose position (X LN ,Y LN ) is: (5).

[0043] The virtual image is a virtual screen. The method of fixing the position of the optomechanical eye box is as follows: the designer presets the target position of the virtual screen (0, D) and the distance r between the center of the optomechanical eye box and the center of eyeball rotation according to the width of the eye box. eb ; Take the center point of the virtual screen as the virtual image T, and use the single pixel method to calculate the left optical machine eye box rotation angle at a fixed distance D and the right optical machine eye box rotation angle ; Adapt the video stream pixels of the projection screen to the glasses optical machine pixels according to the adaptation ratio, where the adaptation ratio = glasses optical machine pixels / projection video stream pixels; calculate the center position of the right optical machine eye box (X Reb ,Y Reb ), the center position of the left optical machine eye box (X Leb ,Y Leb ) Design the left and right optical eye boxes at this position and parallel to the X axis, and rotate the right optical eye box counterclockwise Rotate clockwise with the left optical machine eye box To achieve physical parallax adjustment, move the virtual screen from infinity to the target position, and then fix the optomechanical eye box. Adjust the positions of the two end points of the display screen of the optomechanical eye box, namely, on the ear side and on the nose side, so that the display screen covers the entire optomechanical eye box.

[0044] Align the endpoint of the left optical machine eye box on the nose side, and replace the left optical machine eye box with a width of K that has achieved physical parallax adjustment with a width of F L The optical eye box, and the left optical eye box is rotated with the end point on the nose side as the center. , The left optical machine eye box rotation angle, the left optical machine eye box after rotation Parallel to the X axis: (7); Similarly, do the same process for the right optical machine eye box.

[0045] Example 2 The second embodiment of the present application provides another method for realizing near-object display, which is applicable to a scenario in which an optical machine eye box rotates according to the change of the pupil center position based on eye tracking. The optical machine includes a projector and an optical lens with a display focus area. The display focus area of ​​the optical lens or the spatial range in which the human eye can see the focused image is defined as the optical machine eye box. The angle by which the optical machine eye box rotates around the eyeball rotation center to achieve physical adjustment of parallax is defined as the optical machine eye box rotation angle: The XY coordinate system is established with the midpoint of the line connecting the left and right eyeball rotation centers as the origin O(0,0). The line connecting the left and right eyeball rotation centers is set as the X axis. The positions of the left and right eyeball rotation centers are , , IPD is the distance between the left and right eyeball rotation centers, preset the optical machine eye box width K, field of view angle FOV, the distance IPD between the left and right eyeball rotation centers, the distance r between the eyeball rotation center and the pupil center, and the optical machine eye box center (X eb ,Y eb ) and the distance from the pupil center, set and is the rotation angle of the left and right eye boxes; the target position (0, D) of the virtual image T rendering is set according to the width of the eye box, and D is the vertical distance between the target position and the X axis. When the eyes look forward to infinity, the pupil center is seen from the top view directly above the eye rotation center, that is, the initial position of the pupil center is , ; When binocularly looking at the virtual image T, the position of the right eye tracking sensor facing the right eye is (X C ,Y C ), the right eye tracking sensor is used to collect the angle between the right pupil and the vertical line of the X axis , the right optical machine eye box rotation angle is obtained through triangulation calculation : (6); in, ; Similarly, the position of the left eye tracking sensor facing the left eye is (X C ,Y C ), X C Take the negative value on the left side of the origin and use the same formula to calculate the rotation angle of the left optical machine eye box. ; Calculate the current left and right pupil center positions and the left and right optical eye box center positions; According to the center position of the left and right optical eye boxes, first design the optical eye boxes to be parallel to the X-axis; The control device calculates the rotation angle of the left and right optical machine eye boxes and Then, the left and right optical eye boxes are rotated by the optical eye box drive components. and , the left and right pupil center positions, the left and right eyeball rotation center positions, the left and right optomechanical eye box center positions and the virtual image T position are known, so that the eyeball rotation center, pupil center, optomechanical eye box center and virtual image T are located on the same straight line.

[0046] The display screen in each optical machine eye box corresponds to all or part of the virtual screen, and the contents of the display screens in the left and right optical machine eye boxes correspond to all or part of the virtual screen after being spliced.

[0047] Example 3 The third embodiment of the present application provides an AR display device, such as Figure 16 As shown, the device body 100 includes a connected optical machine 200, and the optical machine 200 includes a projector 201 and an optical lens 202 having a display focus area 203. The display focus area 203 of the optical lens 202 or the spatial range in which the human eye can see the focused image is the optical machine eye box, and the optical machine eye box is designed by the method of embodiment one or two.

[0048] The specific implementation methods described above further illustrate the purpose, technical solutions and beneficial effects of this application. It should be understood that the above description is only the specific implementation methods of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of this application should be included in the scope of protection of this application.

Claims

1. A method for achieving near-object display, applicable to scenarios where an optical machine eye box is fixed in position. The optical machine includes a projector and an optical lens having a display focus area. The display focus area of ​​the optical lens, or the spatial range in which the human eye can see a focused image, is defined as the optical machine eye box. The angle by which the optical machine eye box rotates about the eye's rotation center to achieve physical adjustment of parallax is defined as the optical machine eye box rotation angle. The method is characterized by: The XY coordinate system is established with the midpoint of the line connecting the left and right eyeball rotation centers as the origin O(0,0). The line connecting the left and right eyeball rotation centers is set as the X axis. The positions of the left and right eyeball rotation centers are , The pupil distance IPD is the distance between the left and right eyeball rotation centers, and the preset optical machine eye box width K, field of view angle FOV, the distance IPD between the left and right eyeball rotation centers, the distance r between the eyeball rotation center and the pupil center, and the optical machine eye box center (X eb ,Y eb ) and the distance from the pupil center, set and The rotation angle of the left and right optical machine eye boxes; The designer sets the target position (0, D) for rendering the virtual image T according to the width K of the optical machine eye box. D is the vertical distance between the target position and the X axis. When the eyes look forward to infinity, the pupil center is directly above the eyeball rotation center when viewed from a bird's-eye view. That is, the initial position of the pupil center is , When the binoculars are looking at the virtual image T, the left and right optical eye box rotation angles are calculated based on the target position (0, D) and ; Calculate the current left and right pupil center positions and the left and right optical eye box center positions; According to the center position of the left and right optical eye boxes, first design the optical eye box in a position parallel to the X axis, and then rotate the left optical eye box clockwise Rotate the right optical machine eye box counterclockwise ; The positions of the left and right pupil centers, the left and right eyeball rotation centers, the left and right optomechanical eyebox centers, and the virtual image T are known, so that the eyeball rotation center, pupil center, optomechanical eyebox center, and virtual image T are located on the same straight line, and the left and right optomechanical eyeboxes with physically adjusted parallax are fixed.

2. The method for realizing near-object display according to claim 1, characterized in that: The left and right optical eye box rotation angles are calculated based on the target position (0, D) and : If the virtual image T is a single pixel, the rotation angle of the left and right optical eye boxes can be calculated by formula (1): or : （1）。 3. The method for realizing near-object display according to claim 2, characterized in that: If the virtual image is not a single pixel point and is a regular object, when calculating the left optical machine eye box rotation angle and the right optical machine rotation angle Before, calculate the center pixel of the virtual image and set it as the virtual image T; If the virtual image is not a single pixel and is an irregular object, when calculating the left optical machine eye box rotation angle, and the right optical machine rotation angle Before, the weighted average position is calculated based on the weight of the three-dimensional position of each pixel's XY and distance Z, and is set as the virtual image T; If the virtual image is a virtual screen, and the display screen covers the entire optical eye box, the leftmost pixel and the rightmost pixel position of the virtual screen are known, and the left optical eye box rotation angle is calculated. and the right optical machine rotation angle Before the image is taken, the mean of the left and right end pixel positions is calculated and set as the virtual image T.

4. The method for realizing near-object display according to claim 1, characterized in that: The left and right optical eye box rotation angles are calculated based on the target position (0, D) and : If the virtual image is a virtual screen, and the pixels of the display screen cover the entire optical eye box, the leftmost pixel of the virtual screen is known to be and the rightmost pixel Position, leftmost pixel and the rightmost pixel The distance value, that is, the width of the virtual screen, is set to the leftmost pixel The distance from the normal to the center of the left eyeball is and the leftmost pixel The rotation angle of the left optical machine eye box for the virtual image is , the leftmost pixel The distance from the normal to the center of the right eyeball is and the leftmost pixel The rotation angle of the right optical machine eye box for the virtual image is , , the rightmost pixel The distance from the normal to the center of the right eyeball is and the rightmost pixel The rotation angle of the right optical machine eye box for the virtual image is , the rightmost pixel The distance from the normal to the center of the left eyeball is and the rightmost pixel The rotation angle of the left optical machine eye box for the virtual image is , , calculate the rotation angles of the four optical eye boxes 、 、 and : By pupil distance IPD and rightmost pixel Position, get the rightmost pixel Distance from the normal to the center of the right eyeball , then the right optical machine eye box rotation angle ; By pupil distance IPD and leftmost pixel Position, calculate the leftmost pixel Distance from the normal to the center of the right eyeball , then the right optical machine eye box rotation angle ; By pupil distance IPD and leftmost pixel Position, calculate the leftmost pixel Distance from the normal of the left eyeball center , then the left optical machine eye box rotation angle ; By pupil distance IPD and rightmost pixel Position, calculate the rightmost pixel Distance from the normal of the left eyeball center , then the left optical machine eye box rotation angle ; Rotate the right optical machine eye box around the angle and Calculate the average value to get the rotation angle of the right optical machine eye box , rotate the left optical machine eye box around the angle and Calculate the average value to get the rotation angle of the left optical machine eye box : and .

5. The method for realizing near object display according to claim 1, characterized in that: Calculate the current left and right pupil center positions and the left and right optical eye box center positions: The left pupil center position is calculated according to formula (2): and the center of the right pupil : （2）； According to the distance r between the eyeball rotation center and the pupil center and the center of the optical machine eye box (X eb ,Y eb ) and the distance from the pupil center, and calculate the center position of the optical machine eye box (X eb ,Y eb ) and the distance r from the center of eye rotation eb ,set up , then use formula (2) to calculate the center position of the left optical machine eye box (X Leb ,Y Leb ) and the center position of the right optical machine eye box (X Reb ,Y Reb ).

6. The method for realizing near object display according to claim 1, characterized in that: If the virtual image is a virtual screen, after fixing the left and right optical eye boxes with physically adjusted parallax, let the display screen fill the entire optical eye box: Calculate the center position of the right optical machine eye box (X Reb ,Y Reb ) and the center position of the left optical machine eye box (X Leb ,Y Leb ), set the two end points of the right optical machine eye box center position as the right eye box ear position (X RE ,Y RE ) and the right eye box nose position (X RB ,Y RB ), the distance between the two end points is the width of the optical eye box, and the two end points passing through the center of the left optical eye box are the left eye box ear position (X LE ,Y LE ) and the left eye box nose position (X LN ,Y LN ), the distance between the two end points is the width of the optical eye box; let w = optical eye box width / 2, the slope of the right optical eye box , left optical machine eye box slope ; Calculate the right eye box position close to the ear (X RE ,Y RE ) and the right eye box nose position (X RB ,Y RB ): （4） Calculate the left eye box position close to the ear (X LE ,Y LE ) and the left eye box nose position (X LN ,Y LN ): （5）； Adjust the positions of the two end points of the display screen of the optical eye box, which are close to the ear side and the nose side, so that the display screen covers the entire optical eye box.

7. The method for realizing near object display according to claim 1, characterized in that: Align the endpoint of the left optical machine eye box on the nose side, and replace the left optical machine eye box with a width of K that has achieved physical parallax adjustment with a width of F L The optical eye box, and the left optical eye box is rotated with the end point on the nose side as the center. , The left optical machine eye box rotation angle, the left optical machine eye box after rotation Parallel to the X axis: （7）； Similarly, do the same process for the right optical machine eye box.

8. A method for achieving near-object display, applicable to scenarios where an eye-tracking optical machine eye box rotates to follow changes in pupil center position. The optical machine includes a projector and an optical lens with a display focus area. The display focus area of ​​the optical lens, or the spatial range in which the human eye can see a focused image, is defined as the optical machine eye box. The angle by which the optical machine eye box rotates around the eye's rotation center to achieve physical adjustment of parallax is defined as the optical machine eye box rotation angle. The method is characterized by: The XY coordinate system is established with the midpoint of the line connecting the left and right eyeball rotation centers as the origin O(0,0). The line connecting the left and right eyeball rotation centers is set as the X axis. The positions of the left and right eyeball rotation centers are , , IPD is the distance between the left and right eyeball rotation centers, preset the optical machine eye box width K, field of view angle FOV, the distance IPD between the left and right eyeball rotation centers, the distance r between the eyeball rotation center and the pupil center, and the optical machine eye box center (X eb ,Y eb ) and the distance from the pupil center, set and is the rotation angle of the left and right eye boxes; the target position (0, D) of the virtual image T rendering is set according to the width of the eye box, and D is the vertical distance between the target position and the X axis. When the eyes look forward to infinity, the pupil center is seen from the top view directly above the eye rotation center, that is, the initial position of the pupil center is , ; When binocularly looking at the virtual image T, the position of the right eye tracking sensor facing the right eye is (X C ,Y C ), the right eye tracking sensor is used to collect the angle between the right pupil and the vertical line of the X axis , the right optical machine eye box rotation angle is obtained through triangulation calculation : （6）； Where t= ; Similarly, the position of the left eye tracking sensor facing the left eye is (X C ,Y C ), X C Take the negative value on the left side of the origin and use the same formula to calculate the rotation angle of the left optical machine eye box. ; Calculate the current left and right pupil center positions and the left and right optical eye box center positions; According to the center position of the left and right optical eye boxes, first design the optical eye boxes to be parallel to the X-axis; The control device calculates the rotation angle of the left and right optical machine eye boxes and Then, the left and right optical eye boxes are rotated by the optical eye box drive components. and , the left and right pupil center positions, the left and right eyeball rotation center positions, the left and right optomechanical eye box center positions and the virtual image T position are known, so that the eyeball rotation center, pupil center, optomechanical eye box center and virtual image T are located on the same straight line.

9. The method for realizing near-object display according to claim 8, characterized in that: The display screen in each optical machine eye box corresponds to all or part of the virtual screen, and the contents of the display screens in the left and right optical machine eye boxes correspond to all or part of the virtual screen after being spliced.

10. An AR display device, comprising a device body, the device body comprising a connected optical engine, the optical engine comprising a projector and an optical lens having a display focus area, characterized in that: The display focus area of ​​the optical lens or the spatial range in which the human eye can see the focused image is the optical machine eye box, and the optical machine eye box is designed by the method for realizing near-object display according to any one of claims 1 to 9.

Images