A method, device, AR device and storage medium for improving the display effect of virtual objects in an AR device

Abstract

The present invention provides a method, a device, an AR device and a storage medium for improving the display effect of virtual objects of an AR device. The method includes: acquiring a real environment image and constructing a real environment image coordinate system; acquiring perspective information and constructing a perspective coordinate system; acquiring the coordinate system difference between the real environment image coordinate system and the perspective coordinate system; acquiring the eye movement state; adjusting the perspective coordinate system according to the eye movement state; adjusting the coordinate system difference according to the adjusted perspective coordinate system; and adjusting the coordinates of the virtual object according to the adjusted coordinate system difference. After adopting the above technical solution, the superimposition effect of the virtual object of the AR device and the object in the actual scene can be better, and the user can have a better viewing experience.

Description

Technical Field

[0001] The present invention relates to the technical field of AR devices, and in particular, to a method, a device, an AR device, and a storage medium for improving the display effect of virtual objects on an AR device. Background Art

[0002] Augmented Reality (AR) technology is a new technology that "seamlessly" integrates real-world information and virtual-world information. It is to simulate and superimpose entity information (visual information, sound, taste, touch, etc.) that is difficult to experience within a certain time and space range in the real world through scientific and technological means such as computers, and then apply the virtual information to the real world, which is perceived by human senses, so as to achieve a sensory experience beyond reality. The real environment and virtual objects are superimposed on the same picture or space in real time and exist simultaneously.

[0003] Currently, there are mainly two display methods for head-mounted AR devices: video see-through and optical see-through.

[0004] In the video see-through method, a miniature camera installed on the AR device is used to collect images of the real scene. At the same time, the computer superimposes the information and image signals to be added on the video signal of the camera through scene understanding and analysis, and fuses the virtual scene generated by the computer with the real scene. Finally, it is presented to the user through a display.

[0005] In the optical see-through method, a pair of semi-transmissive and semi-reflective optical combiners installed in front of the eyes are used to fuse the external real environment and virtual information. The real scene is directly presented to the user through the semi-reflective semi-lens. The virtual information generated by the computer is magnified by the optical system and then reflected by the semi-transmissive and semi-reflective lens and enters the eyes. The real scene and virtual information converge on the retina to form an imaging effect of virtual and real superposition.

[0006] The problem with existing optical see-through AR devices is that the coordinate positions of virtual information produced by the computer, such as virtual objects, are determined by the coordinate system obtained from the real scene acquired by the camera, that is, determined by the coordinate system constructed according to the camera; while the real scene seen by the user is directly obtained by the human eye, that is, determined by the coordinate system formed by the human eye. Due to the deviation between these two coordinate systems, the virtual objects produced by the computer have a deviation when superimposed on the real scene, resulting in a poor virtual and real superposition effect, that is, the display effect of virtual objects is poor, affecting the user experience.

[0007] Therefore, it is necessary to develop a method, a device, an AR device, and a storage medium that can improve the display effect of virtual objects on an AR device. Summary of the Invention

[0008] To overcome the above technical defects, the object of the present invention is to provide a method, device, AR device and storage medium capable of improving the display effect of virtual objects on AR devices.

[0009] The present invention discloses a method for improving the display effect of virtual objects on AR devices, including:

[0010] Obtain a real environment image and construct a real environment image coordinate system;

[0011] Obtain perspective information and construct a perspective coordinate system;

[0012] Obtain the coordinate difference between the real environment image coordinate system and the perspective coordinate system;

[0013] Obtain the eye movement state;

[0014] Adjust the perspective coordinate system according to the eye movement state;

[0015] Adjust the coordinate difference according to the adjusted perspective coordinate system;

[0016] Adjust the coordinates of the virtual object according to the adjusted coordinate difference.

[0017] Preferably, after obtaining the real environment image and constructing the real environment image coordinate system,

[0018] Add a virtual object and obtain the coordinates of the virtual object in the real environment image coordinate system.

[0019] Preferably, the step of adding a virtual object and obtaining the coordinates of the virtual object in the real environment image coordinate system includes:

[0020] Identify a reference object in the real environment through object recognition technology and obtain the coordinates of the reference object in the real environment image coordinate system;

[0021] Add a virtual object according to the preset positional relationship between the virtual object and the reference object and obtain the coordinates of the virtual object in the real environment image coordinate system.

[0022] Preferably, after adjusting the coordinates of the virtual object according to the adjusted coordinate difference,

[0023] Project the virtual object onto the display screen of the AR device with the adjusted coordinates.

[0024] Preferably, obtain the eye movement state through eye tracking technology;

[0025] The eye tracking technology includes: iris angle change tracking technology, eye and periorbital feature change tracking technology, and infrared tracking technology.

[0026] Preferably, the origin of the perspective coordinate system is the center point of the eye, the x-axis and y-axis of the perspective coordinate system are in the same directions as the x-axis and y-axis of the retina plane or iris plane, and the z-axis of the perspective coordinate system is the line-of-sight observation direction.

[0027] Preferably, obtaining the coordinate system difference between the real environment image coordinate system and the perspective coordinate system includes:

[0028] Identifying a reference object in the real environment through object recognition technology, and obtaining the coordinates of the reference object in the real environment image coordinate system and the perspective coordinate system;

[0029] Calculating the coordinate difference of the reference object in the real environment image coordinate system and the perspective coordinate system;

[0030] The coordinate difference is the coordinate system difference between the real environment image coordinate system and the perspective coordinate system.

[0031] The present invention also discloses a device for improving the display effect of virtual objects of an AR device, including:

[0032] A first acquisition unit for acquiring a real environment image and constructing a real environment image coordinate system;

[0033] A second acquisition unit for acquiring perspective information and constructing a perspective coordinate system;

[0034] A third acquisition unit for acquiring the coordinate system difference between the real environment image coordinate system and the perspective coordinate system;

[0035] A fourth acquisition unit for acquiring the eye movement state;

[0036] A first adjustment unit for adjusting the perspective coordinate system according to the eye movement state;

[0037] A second adjustment unit for adjusting the coordinate system difference according to the adjusted perspective coordinate system;

[0038] A third adjustment unit for adjusting the coordinates of the virtual object according to the adjusted coordinate system difference.

[0039] The present invention also discloses an AR device, including a memory, a processor, and a computer program stored in the memory and configured to be executed by the processor. The processor, when executing the computer program, implements the above method.

[0040] The present invention also discloses a computer-readable storage medium, which includes a stored computer program. When the computer program runs, it controls the device where the computer-readable storage medium is located to execute the above method.

[0041] After adopting the above technical solution, compared with the prior art, it can effectively improve the display effect of virtual objects of the AR device, make the superposition of virtual objects and the real scene more matching, enhance the visual effect of the AR glasses, and thus improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a schematic flowchart of a method for improving the display effect of virtual objects of an AR device in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0043] The advantages of the present invention will be further elaborated below in conjunction with the accompanying drawings and specific embodiments.

[0044] Here, the exemplary embodiments will be described in detail, and the examples are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0045] The terms used in the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure. The singular forms "a", "the" and "said" used in the present disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0046] It should be understood that although the terms first, second, third, etc. may be used in the present disclosure to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0047] In the description of the present invention, it should be understood that the orientation or positional relationships indicated by the terms "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention.

[0048] In the description of the present invention, unless otherwise specified and defined, it should be noted that the terms "mounted", "connected" and "coupled" should be understood in a broad sense. For example, they can be mechanical connections or electrical connections, or the communication inside two elements, can be directly connected, or indirectly connected through an intermediate medium. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0049] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only for the convenience of describing the present invention, and they have no specific meaning themselves. Therefore, "module" and "component" can be used interchangeably.

[0050] See the attached Figure 1 , which is a schematic flow chart of a method for improving the virtual object display effect in an AR device in an embodiment of the present invention.

[0051] The AR device can be an AR glasses or a head-mounted display, and the display mode of the AR device is optical see-through.

[0052] The method includes the following steps:

[0053] S1: Obtain a real environment image and construct a real environment image coordinate system.

[0054] Specifically, a camera is provided on the AR device, and the camera is used to obtain an image of the real environment. The processing module of the AR device calculates and constructs a real environment image coordinate system, that is, a camera coordinate system, according to the image of the real environment obtained by the camera.

[0055] The origin of the real environment image coordinate system is the camera optical center, the x-axis and y-axis are consistent with the x-axis and y-axis of the imaging plane, and the z-axis is the optical axis of the camera, perpendicular to the x-axis and y-axis.

[0056] After step S1, there is also step S1': Add a virtual object and obtain the coordinates of the virtual object in the real environment image coordinate system.

[0057] That is, after obtaining the image of the real environment, according to the obtained real environment image, add the virtual objects that need to be added, and obtain the coordinates of the virtual objects in the coordinate system of the real environment image. It should be noted that the addition here can refer to constructing the corresponding virtual objects according to the real environment image, or it can refer to adding the existing or pre-generated virtual objects to the coordinate system of the real environment image according to the preset addition rules. The virtual objects here can refer to pictures, videos, animations, texts, etc., that is, things added by the processor that do not exist in the real environment image.

[0058] Specifically, in step S1’, first identify the reference object in the real environment through object recognition technology, and obtain the coordinates of the reference object in the coordinate system of the real environment image; then add virtual objects according to the preset positional relationship between the virtual objects and the reference object, and obtain the coordinates of the virtual objects in the coordinate system of the real environment image.

[0059] Object recognition technology is a basic technology in the field of computer vision. Its function is to identify what objects are in the image and report the position and orientation of this object in the scene represented by the image.

[0060] Object recognition technology generally includes the following steps: preprocessing of the image, feature extraction, feature selection, modeling, matching, and positioning.

[0061] The main methods of object recognition include: object classification methods based on statistics, recognition based on object parts, generative methods and discriminative methods, object recognition methods based on models, and object recognition methods based on context.

[0062] The following takes improving the display effect of a virtual round vase on a round table in the real environment as an example to illustrate the above steps.

[0063] In step S1, obtain the real environment image containing the round table through the camera of the AR device, and calculate and construct the coordinate system of the real environment image according to the real environment image. The origin of the coordinate system of the real environment image is the optical center of the camera. The x-axis and y-axis are consistent with the x-axis and y-axis of the imaging plane, and the z-axis is the optical axis of the camera, perpendicular to the x-axis and y-axis.

[0064] In step S1', after obtaining the real environment image, first identify the circular tabletop in the real environment through object recognition technology. The circular tabletop is the above-mentioned reference object. After identifying the circular tabletop, locate the circular tabletop and obtain the coordinates (x1, y1, z1) of the center of the circular tabletop in the coordinate system of the real environment image. Then add a virtual vase according to the preset positional relationship between the virtual circular vase and the circular tabletop. The coordinates of the center of the bottom of the virtual circular vase in the coordinate system of the real environment image are (x2, y2, z2), and it is preset to add a virtual circular vase 3 cm to the right and 5 cm above the center of the circular tabletop. Then add a virtual circular vase according to this preset positional relationship, and the coordinates (x2, y2, z2) of the virtual circular vase in the coordinate system of the real environment image are (x1 + 3, y1 + 5, z1).

[0065] S2: Obtain perspective information and construct a perspective coordinate system.

[0066] Specifically, the perspective information includes information such as the position, state, line-of-sight direction, and perspective of the eyeball, and the perspective information can be obtained through eye-tracking technology.

[0067] The eye-tracking technology includes: iris angle change tracking technology, eyeball and periorbital feature change tracking technology, and infrared tracking technology.

[0068] After obtaining the perspective information, the processor of the VR device calculates and constructs a perspective coordinate system according to the perspective information. Specifically, the origin of the perspective coordinate system is the center point of the eyeball, the x-axis and y-axis of the perspective coordinate system are in the same direction as the x-axis and y-axis of the retina plane or iris plane, and the z-axis of the perspective coordinate system is the line-of-sight observation direction. In some embodiments, the center point of the eyeball may be the center point of the pupil.

[0069] S3: Obtain the coordinate system difference between the real environment image coordinate system and the perspective coordinate system.

[0070] Specifically, after the real environment image coordinate system and the perspective coordinate system are established, according to information such as the positional relationship between the eyeball and the camera and the perspective information, the positional relationship between the real environment image coordinate system and the perspective coordinate system can be directly known, and thus the coordinate system difference between the real environment image coordinate system and the perspective coordinate system can be directly obtained.

[0071] In some embodiments, the coordinate difference between the real environment image coordinate system and the perspective coordinate system can also be obtained by using a reference object. Specifically, the reference object in the real environment is identified through object recognition technology, and the coordinates of the reference object in the real environment image coordinate system and the perspective coordinate system are obtained; the coordinate difference between the reference object in the real environment image coordinate system and the perspective coordinate system is calculated; and the coordinate difference is the coordinate difference between the real environment image coordinate system and the perspective coordinate system.

[0072] Continuing with the example of improving the display effect of a virtual circular vase on a circular tabletop in the real environment, the above steps will be described below.

[0073] In step S2, the perspective information is obtained through eye tracking technology, and the perspective coordinate system is calculated and constructed. The origin of the perspective coordinate system is the center point of the eyeball. The x-axis and y-axis of the perspective coordinate system are in the same directions as the x-axis and y-axis of the retina plane or iris plane, and the z-axis of the perspective coordinate system is the line-of-sight observation direction. Further, since the circular tabletop in the real environment has been identified through object recognition technology in step S1', in this step, the coordinates of the center of the circular tabletop in the perspective coordinate system are obtained as (x3, y3, z3).

[0074] In step S3, based on the coordinates (x1, y1, z1) of the center of the circular tabletop in the real environment image coordinate system and the coordinates (x3, y3, z3) in the perspective coordinate system obtained, the coordinate difference between the center of the circular tabletop in the real environment image coordinate system and the perspective coordinate system is calculated. The coordinate difference is (x3 - x1, y3 - y1, z3 - z1), that is, the coordinate difference between the real environment image coordinate system and the perspective coordinate system is (x3 - x1, y3 - y1, z3 - z1). In some embodiments, at this time, the coordinates of the virtual circular vase can be adjusted according to the coordinate difference, that is, adding the coordinate difference to the coordinates of the virtual circular vase in the real environment image coordinate system, and the coordinates of the virtual circular vase in the perspective coordinate system (x3 - x1 + x2, y3 - y1 + y2, z3 - z1 + z2) can be obtained. Furthermore, the image of the virtual circular vase can be projected onto the display screen of the AR device with these coordinates. In this way, the deviation between the coordinate system constructed based on the camera and the human eye perspective coordinate system can be compensated, and the superimposition effect of the virtual object and the actual scene can be improved, that is, the display effect of the virtual object can be improved.

[0075] S4: Obtain the eye movement state.

[0076] Specifically, the movement state of the eyeball, i.e., the rotation direction, angle, etc. of the eyeball, is tracked through eye tracking technology. Since the line-of-sight direction changes when the eyeball moves. If you want to improve the display effect of virtual objects, it is necessary to adjust the coordinates of the virtual objects according to the movement of the eyeball, i.e., the change in the line-of-sight direction, so that it better matches the line of sight of the human eye.

[0077] S5: Adjust the perspective coordinate system according to the eyeball movement state.

[0078] Specifically, for this step, in some embodiments, based on the initial perspective coordinate system, the change amount of the eyeball movement, i.e., information such as the rotation direction and angle of the eyeball, is obtained, and this change amount is superimposed on the initial perspective coordinate system to obtain the changed perspective coordinate system.

[0079] For this step, in some other embodiments, through eye tracking technology, for the eyeball after rotating a certain direction and angle, its perspective-related information is obtained, and a perspective coordinate system is re-established. The re-established perspective coordinate system is the adjusted perspective coordinate system.

[0080] S6: Adjust the coordinate system difference according to the adjusted perspective coordinate system.

[0081] For this step, in some embodiments, through information such as the position of the eyeball after movement, the implementation direction, i.e., its positional relationship with the camera, etc., the positional relationship between the real environment image coordinate system and the perspective coordinate system adjusted in step S5 can be directly obtained, and thus the coordinate system difference between the real environment image coordinate system and the perspective coordinate system adjusted in step S5 can be directly obtained.

[0082] For this step, in some other embodiments, the coordinate system difference between the real environment image coordinate system and the perspective coordinate system adjusted in step S5 can also be calculated through the coordinate difference of a reference object in the real environment image coordinate system and the reference object in the perspective coordinate system adjusted in step S5. That is, subtract the coordinates of the reference object in these two coordinate systems to obtain the adjusted coordinate system difference.

[0083] S7: Adjust the coordinates of the virtual object according to the adjusted coordinate system difference.

[0084] In this step, adding the coordinate system difference obtained in step S6 to the coordinates of the virtual object in the real environment image coordinate system can obtain the coordinates of the virtual object in the adjusted perspective coordinate system.

[0085] Further, the method further includes step S8: projecting the virtual object onto the display screen of the AR device at the adjusted coordinates. Projecting the virtual object at the adjusted coordinates can effectively compensate for the deviation between the coordinate system constructed based on the camera and the coordinate system of the human eye's perspective. Even with eye movement, the virtual object can be displayed in the appropriate position, improving the superimposition effect of the virtual object on the actual scene, that is, improving the display effect of the virtual object.

[0086] Here, continue to take improving the display effect of a virtual circular vase on a circular table in a real environment as an example to illustrate the above steps.

[0087] In steps S4 and S5, the eye movement state is obtained through eye tracking technology, and the perspective coordinate system is reconstructed based on the information related to the perspective of the eye after movement, that is, the adjusted perspective coordinate system is obtained.

[0088] In step S6, the coordinates (x4, y4, z4) of the center of the circular table in the adjusted perspective coordinate system are obtained, and the coordinate difference between the center of the circular table in the real environment image coordinate system and the adjusted perspective coordinate system is calculated, that is, (x4 - x1, y4 - y1, z4 - z1). This coordinate difference is the coordinate system difference between the real environment image coordinate system and the adjusted perspective coordinate system.

[0089] In step S7, the coordinates of the virtual circular vase in the real environment image coordinate system are added to the adjusted coordinate system difference obtained in step S6 to obtain the coordinates (x4 - x1 + x2, y4 - y1 + y2, z4 - z1 + z2) of the virtual object in the adjusted perspective coordinate system.

[0090] In step S8, the virtual object is projected onto the display screen of the AR device at the adjusted coordinates (x4 - x1 + x2, y4 - y1 + y2, z4 - z1 + z2).

[0091] It should be noted that for steps S1 - S7 of the present invention, there is no strict order requirement. For example, steps S4 and S2 can be completed synchronously; for example, step S3 can be after step S5. As long as the method includes the above steps S1 - S7, even with different orders, it should be included in the protection scope of this application.

[0092] The present invention also discloses a device for improving the display effect of virtual objects on an AR device, including:

[0093] The first acquisition unit is used to acquire a real environment image and construct a real environment image coordinate system;

[0094] The second acquisition unit is used to acquire perspective information and construct a perspective coordinate system;

[0095] A third acquisition unit, configured to acquire the coordinate difference between the real environment image coordinate system and the perspective coordinate system;

[0096] A fourth acquisition unit, configured to acquire the eye movement state;

[0097] A first adjustment unit, configured to adjust the perspective coordinate system according to the eye movement state;

[0098] A second adjustment unit, configured to adjust the coordinate difference according to the adjusted perspective coordinate system;

[0099] A third adjustment unit, configured to adjust the coordinates of the virtual object according to the adjusted coordinate difference;

[0100] Further, the present invention further includes:

[0101] A first addition unit, configured to add a virtual object and acquire the coordinates of the virtual object in the real environment image coordinate system;

[0102] Further, the present invention further includes:

[0103] A projection unit, configured to project the virtual object at the adjusted coordinates onto the display screen of the AR device.

[0104] The present invention also discloses an AR device, including a memory, a processor, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the methods in the above embodiments are implemented.

[0105] In this application, the processor includes one or more processing cores. By running or executing the computer program stored in the memory (including: instructions, programs, code sets, or instruction sets, etc.) and calling the data stored in the memory, various functions of the AR device are executed and data is processed. Optionally, the processor may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). The processor may integrate a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem, etc.

[0106] The memory may include a Random Access Memory (RAM), or may also include a Read-Only Memory. Optionally, the memory includes a non-transitory computer-readable storage medium. The memory can be used to store computer programs (including: instructions, programs, code, code sets, or instruction sets, etc.).

[0107] The AR device further includes:

[0108] A camera for acquiring images of the real environment;

[0109] An eye tracking device for acquiring viewing angle information and eye movement states;

[0110] A display screen for projecting virtual objects. The display screen may be a transparent lens.

[0111] The present invention also discloses a computer-readable storage medium, which includes a stored computer program. When the computer program runs, it controls the device where the computer-readable storage medium is located to execute the method in the above embodiments. Specifically, the computer program can be built into the AR device. In this way, the AR device can implement the steps of the above method by executing the built-in computer program.

[0112] The computer-readable storage medium can be various types of storage media, and is optionally a non-instantaneous storage medium. The computer-readable storage medium can be optionally a mobile storage device, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disc, etc., which are all media that can store program codes.

[0113] It should be noted that the embodiments of the present invention have good implementability and do not impose any form of limitation on the present invention. Any person skilled in the art may use the disclosed technical content to modify or transform it into an equivalent effective embodiment. However, as long as it does not depart from the technical solution of the present invention, any modification, equivalent change, or modification made to the above embodiments based on the technical essence of the present invention still falls within the scope of the technical solution of the present invention.

Claims

1. A method for improving the display effect of virtual objects in an AR device, characterized in that, including: Obtain a real environment image and construct a real environment image coordinate system; Identify a reference object in the real environment through object recognition technology, and obtain the coordinates of the reference object in the real environment image coordinate system; add a virtual object according to a preset positional relationship between the virtual object and the reference object, and obtain the coordinates of the virtual object in the real environment image coordinate system; Obtain perspective information and construct a perspective coordinate system; Identify a reference object in the real environment through object recognition technology, and obtain the coordinates of the reference object in the perspective coordinate system; calculate the coordinate difference of the reference object in the real environment image coordinate system and the perspective coordinate system; the coordinate difference is the coordinate system difference between the real environment image coordinate system and the perspective coordinate system; Obtain the eye movement state; Adjust the perspective coordinate system according to the eye movement state; Adjust the coordinate system difference according to the adjusted perspective coordinate system; Add the adjusted coordinate system difference to the coordinates of the virtual object in the real environment image coordinate system to obtain the coordinates of the virtual object in the adjusted perspective coordinate system.

2. The method according to claim 1, wherein after adjusting the coordinates of the virtual object according to the adjusted coordinate system difference, project the virtual object onto the display screen of the AR device with the adjusted coordinates.

3. The method according to claim 1, wherein obtain the eye movement state through an eye tracking technology; the eye tracking technology includes: iris angle change tracking technology, eye and eye peripheral feature change tracking technology, and infrared tracking technology.

4. The method according to claim 1, wherein the origin of the perspective coordinate system is the center point of the eye, the x-axis and y-axis of the perspective coordinate system are in the same direction as the x-axis and y-axis of the retina plane or iris plane, and the z-axis of the perspective coordinate system is the line-of-sight viewing direction.

5. A device for improving the display effect of virtual objects in an AR device, characterized in that, including: A first obtaining unit, configured to obtain a real environment image and construct a real environment image coordinate system; A first adding unit, configured to identify a reference object in the real environment through object recognition technology, and obtain the coordinates of the reference object in the real environment image coordinate system; add a virtual object according to a preset positional relationship between the virtual object and the reference object, and obtain the coordinates of the virtual object in the real environment image coordinate system; A second obtaining unit, configured to obtain perspective information and construct a perspective coordinate system; A third obtaining unit, configured to identify a reference object in the real environment through object recognition technology, and obtain the coordinates of the reference object in the perspective coordinate system; calculate the coordinate difference of the reference object in the real environment image coordinate system and the perspective coordinate system; the coordinate difference is the coordinate system difference between the real environment image coordinate system and the perspective coordinate system; A fourth obtaining unit, configured to obtain the eye movement state; A first adjusting unit, configured to adjust the perspective coordinate system according to the eye movement state; A second adjusting unit, configured to adjust the coordinate system difference according to the adjusted perspective coordinate system; A third adjustment unit is configured to add the adjusted coordinate difference of the coordinate system to the coordinates of the virtual object in the real environment image coordinate system, so as to obtain the coordinates of the virtual object in the adjusted perspective coordinate system.

6. An AR device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and configured to be executed by the processor. The feature is that when the processor executes the computer program, the method described in any one of claims 1-4 is implemented.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program. When the computer program runs, it controls the device where the computer-readable storage medium is located to execute the method described in any one of claims 1-4.

Images