Virtual object rendering method, electronic equipment and storage medium

Abstract

The invention provides a virtual object rendering method, electronic equipment and a storage medium. The method comprises the steps of obtaining a pose of the electronic equipment at a current rendering moment; according to the pose, determining a first position of a virtual object of the rendering space in a rendering coordinate system at the current rendering moment; according to the first position, determining a second position of the virtual object projected to the display screen of the electronic equipment at the current rendering moment; and displaying the rendering effect of the virtual object at a second position on the display screen at the current rendering moment. In this way, the display position of the rendered virtual object can be changed along with the change of the pose of the electronic equipment.

Description

Technical Field

[0001] The present application relates to the field of rendering technology, and in particular to a virtual object rendering method, electronic device and storage medium. Background Art

[0002] At present, a 3dof (3 Degrees of Freedom) rendering method based on VR (Virtual Reality) is provided according to the MVP (Model, View, Projection) matrix calculation method. However, for AR (Augmented Reality) glasses, when wearing AR glasses for exercise, the above technology cannot change the display position of the rendered virtual object as the posture of the AR glasses changes. Summary of the invention

[0003] Based on this, the present application provides a virtual object rendering method, an electronic device and a storage medium, which can change the display position of the rendered virtual object as the posture of the electronic device changes.

[0004] In a first aspect, the present application provides a virtual object rendering method, the method comprising:

[0005] Get the position and posture of the electronic device at the current rendering moment;

[0006] Determine, according to the position and posture, a first position of the virtual object in the rendering coordinate system at the current rendering moment;

[0007] Determining, according to the first position, a second position at which the virtual object is projected onto the display screen of the electronic device at the current rendering moment;

[0008] The rendering effect of the virtual object is displayed at a second position on the display screen at the current rendering moment.

[0009] In a second aspect, the present application provides an electronic device, comprising a memory and a processor, wherein the memory is used to store a computer program, and the processor is used to execute the computer program and implement the virtual object rendering method as described above when executing the computer program.

[0010] In a third aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the virtual object rendering method as described above.

[0011] The embodiments of the present application provide a virtual object rendering method, an electronic device and a storage medium. The position of the virtual object in the rendering coordinate system at the current rendering time is determined according to the posture of the electronic device at the current rendering time; that is, the position of the virtual object in the rendering coordinate system changes with the change of the posture of the electronic device. The position of the virtual object projected onto the display screen of the electronic device determined thereby also changes with the change of the posture of the electronic device. Therefore, in this way, the display position of the rendered virtual object can be changed with the change of the posture of the electronic device. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a flowchart of an embodiment of a virtual object rendering method of the present application;

[0013] Figure 2 This is a flowchart of another embodiment of the virtual object rendering method of the present application.

[0014] Figure 3 This is a flowchart of another embodiment of the virtual object rendering method of the present application.

[0015] Figure 4 This is a flowchart of another embodiment of the virtual object rendering method of the present application.

[0016] Figure 5 This is a flowchart of another embodiment of the virtual object rendering method of the present application.

[0017] Figure 6 It is a structural schematic diagram of an embodiment of the electronic device of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0020] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no special meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0021] At present, a 3dof (3 Degrees of Freedom) rendering method based on VR (Virtual Reality) is provided according to the MVP (Model, View, Projection) matrix calculation method. However, for AR (Augmented Reality) glasses, when wearing AR glasses for exercise, the above technology cannot change the display position of the rendered virtual object as the posture of the AR glasses changes.

[0022] The embodiments of the present application provide a virtual object rendering method, an electronic device and a storage medium. The position of the virtual object in the rendering coordinate system at the current rendering time is determined according to the posture of the electronic device at the current rendering time; that is, the position of the virtual object in the rendering coordinate system changes with the change of the posture of the electronic device. The position of the virtual object projected onto the display screen of the electronic device determined thereby also changes with the change of the posture of the electronic device. Therefore, in this way, the display position of the rendered virtual object can be changed with the change of the posture of the electronic device.

[0023] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0024] See also Figure 1 , Figure 1 It is a flowchart of an embodiment of a virtual object rendering method of the present application.

[0025] The method is applicable to an electronic device, the posture of the electronic device can be changed, and the electronic device includes a display screen. The posture of the electronic device can be changed, which means that the electronic device itself can change its posture, or the electronic device can be carried on an external object (a user, a car, a robot, etc. using the electronic device), and the posture can change with the change of the external object.

[0026] The electronic devices include, but are not limited to, head-mounted electronic devices and non-head-mounted electronic devices. Head-mounted electronic devices include AR (Augmented Reality) glasses, etc. Non-head-mounted electronic devices include vehicle-mounted / aircraft-mounted head-up display (HUD) devices, etc.

[0027] The method includes: step S101 to step S104.

[0028] Step S101: Obtain the position and posture of the electronic device at the current rendering moment.

[0029] Rendering in computer graphics refers to the process of using software to generate images from models; models are descriptions of three-dimensional objects in a strictly defined language or data structure, which includes geometry, viewpoint, texture, and lighting information; therefore, rendering is the process of projecting the model in a three-dimensional scene into a digital image in two dimensions according to the set environment, lighting, material, and rendering parameters. To briefly summarize it in one sentence, the rendering process is the process of displaying a three-dimensional object in a computer on a two-dimensional plane on a monitor.

[0030] The pose is used to describe the position (displacement) and attitude (rotation) of an object or robot in three-dimensional space. It is the position and orientation of an object or robot relative to a reference coordinate system. The pose of an electronic device can change. When the pose of an electronic device changes, the position of the virtual object in the rendering space projected onto the display screen of the electronic device also changes, which can make the rendered virtual object closer to the real scene in the user's view. Therefore, it is first necessary to obtain the pose of the electronic device at the current rendering moment.

[0031] There are many ways to obtain the position and posture of the electronic device at the current rendering moment: for example, collecting IMU data through an IMU (Inertial Measurement Unit) sensor, and calculating the position and posture of the electronic device through the IMU data; collecting image data of the environment around the electronic device, and calculating the position and posture of the electronic device through the image data; etc.

[0032] Step S102: determining a first position of a virtual object in the rendering space in a rendering coordinate system at the current rendering moment according to the position and posture.

[0033] Virtual objects are set in the rendering space after data calculation. The source of data can be point cloud data obtained by sensors on electronic devices to perform three-dimensional scanning of the surrounding environment. Virtual objects can also be simply set in the rendering space by the observer, such as setting a red cube with a length, width and height of 1m in front of the observer's eyes; virtual objects can be two-dimensional virtual objects or three-dimensional virtual objects. The rendering coordinate system is the coordinate system of the rendering space. Virtual objects can be represented in the form of three-dimensional virtual models in the rendering space, so the rendering space can be the space where the three-dimensional virtual model is located. The rendering coordinate system can be specified. For example, in 3dof (three degrees of freedom) or 6dof (six degrees of freedom), the rendering coordinate system can use a right-handed coordinate system, with the IMU (Inertial Measurement Unit) sensor as the origin, the horizontal right is the x-axis, the upward is the y-axis, and the backward is the z-axis.

[0034] The position of the virtual object in the rendering coordinate system is associated with the posture of the electronic device, so the first position of the virtual object in the rendering coordinate system at the current rendering moment needs to be determined according to the posture of the electronic device at the current rendering moment.

[0035] The posture of the electronic device is usually the posture of the electronic device in a certain coordinate system. Determining the first position of the virtual object in the rendering coordinate system at the current rendering time according to the posture of the electronic device at the current rendering time is equivalent to determining the first position of the virtual object in the rendering coordinate system at the current rendering time according to the posture of the electronic device in a certain coordinate system at the current rendering time. Therefore, it is necessary to determine in advance the conversion relationship between a certain coordinate system and the rendering coordinate system, and the conversion relationship between a certain coordinate system and the rendering coordinate system can be obtained in advance through multiple groups of known related data (i.e., the electronic device is at multiple different times, respectively, in multiple known postures in a certain coordinate system, and the virtual object is at multiple different times at multiple known positions in the rendering coordinate system).

[0036] Step S103: According to the first position, determining a second position of the virtual object projected onto the display screen of the electronic device at the current rendering moment.

[0037] The position of the virtual object projected onto the display screen of the electronic device is also associated with the posture of the electronic device. After the first position is determined according to the posture of the electronic device at the current rendering moment, the second position of the virtual object projected onto the display screen of the electronic device at the current rendering moment can be determined based on the first position.

[0038] In an embodiment of the present application, the conversion relationship from the rendering coordinate system to the screen coordinate system of the display screen of the electronic device can be determined in advance based on multiple groups of known related data (that is, multiple positions of the virtual object in the rendering coordinate system at multiple different times are known, and multiple positions thereof projected onto the display screen of the electronic device are also known); then, based on the position of the virtual object in the rendering coordinate system at the current rendering time, combined with the conversion relationship from the rendering coordinate system to the screen coordinate system of the display screen of the electronic device, the position of the virtual object projected onto the display screen of the electronic device at the current rendering time can be determined.

[0039] Step S104: displaying the rendering effect of the virtual object at a second position on the display screen at the current rendering moment.

[0040] Projecting the virtual object in the rendering space onto the display screen of the electronic device is a rendering process, that is, the process of displaying a three-dimensional virtual object or a two-dimensional virtual object in the computer on a two-dimensional plane on the display screen. When the second position where the virtual object is projected onto the display screen of the electronic device at the current rendering moment is determined, the rendering effect of the virtual object can be displayed at the second position determined on the display screen, and the rendering effect is presented as a two-dimensional image of the virtual object.

[0041] The embodiments of the present application provide a virtual object rendering method, an electronic device and a storage medium. The position of the virtual object in the rendering coordinate system at the current rendering time is determined according to the posture of the electronic device at the current rendering time; that is, the position of the virtual object in the rendering coordinate system changes with the change of the posture of the electronic device. The position of the virtual object projected onto the display screen of the electronic device determined thereby also changes with the change of the posture of the electronic device. Therefore, in this way, the display position of the rendered virtual object can be changed with the change of the posture of the electronic device.

[0042] In some embodiments, the display screen includes a perspective screen, in which case the display screen can see through actual objects in the surrounding environment of the electronic device. When the position of the electronic device changes, the actual objects in its surrounding environment also change, and the actual objects seen through on the display screen of the electronic device also change. Since the display position of the rendered virtual object can change with the change of the position of the electronic device, the actual objects seen through on the display screen of the electronic device also change with the change of the position of the electronic device. Therefore, in this way, when the rendered virtual object is superimposed on the actual object seen through, the rendered virtual object can be anchored together with the changed actual object for display as the position of the electronic device changes, as if the position of the rendered virtual object is also changing, thereby improving the user experience.

[0043] In some embodiments, the electronic device includes an optical waveguide.

[0044] An optical waveguide is a device that transmits light waves from one place to another, that is, a medium device that guides the propagation of light waves in it, and it guides the image to the audience's eyes in the form of light. Unlike traditional display methods, optical waveguides use microdisplays and specially constructed optical waveguide devices to achieve image projection. In this way, users can enjoy virtual images without head-mounted displays (which makes electronic devices using optical waveguides for display have the characteristics of light weight and thinness). A microdisplay is a small and high-resolution device, usually composed of multiple nano-scale LEDs (Light Emitting Diodes) arranged in an array, which emit light of different colors and brightness through electrical signal control, and then inject these lights into the optical waveguide device for guidance. This special design ensures that the image can be transparently projected into the user's field of view (which makes electronic devices using optical waveguides for display have the characteristics of high penetration of external light) without interfering with the surrounding environment.

[0045] Since the process of projecting the virtual object from the position in the rendering coordinate system to the position on the display screen of the electronic device is not a direct conversion process from the rendering coordinate system to the screen coordinate system of the display screen, it needs to be implemented through an optical waveguide in the middle, that is, the conversion process from the rendering coordinate system through the optical waveguide and then to the screen coordinate system of the display screen. It is necessary to correct the second position determined on the display screen so that the rendered and displayed virtual object presents a correct three-dimensional effect. In addition, the optical waveguide itself has a very precise structure and components, and it is necessary to be very careful during the installation process to ensure that the various components are correctly placed and connected, and to perform necessary calibration operations to achieve the best visual effect. If there is an installation error in the optical waveguide, the second position determined on the display screen can be corrected to reasonably correct the installation error of the optical waveguide. That is, step S104, before displaying the rendering effect of the virtual object at the second position on the display screen at the current rendering moment, can also include: step S105, such as Figure 2 shown.

[0046] Step S105: According to the conversion relationship between the observer's eyes and the projection center of the optical machine, the second position of the virtual object projected onto the display screen at the current rendering moment is corrected, the electronic device includes an optical waveguide, and the optical waveguide includes an optical machine.

[0047] In the embodiment of the present application, the process of projecting the virtual object from the first position in the rendering coordinate system to the second position on the display screen of the electronic device at the current rendering moment is not to project the virtual object directly from the first position in the rendering coordinate system to the second position on the display screen, but rather a process of converting the virtual object from the first position in the rendering coordinate system through the optical waveguide and then to the second position in the screen coordinate system of the display screen. Therefore, it is necessary to correct the second position in combination with the conversion relationship between the observer's human eye and the projection center of the optical machine.

[0048] Among them, the conversion relationship between the observer's human eye and the projection center of the optical machine can be obtained in advance based on known relevant data.

[0049] At this time, step S104, displaying the rendering effect of the virtual object at the second position on the display screen at the current rendering moment, may include: displaying the rendering effect of the virtual object at the corrected second position on the display screen at the current rendering moment.

[0050] In some embodiments, if the conversion relationship between the observer's human eye and the projection center of the optical machine is not obtained in advance, it can be obtained in the following manner. That is, step S105, before correcting the second position of the virtual object projected onto the display screen at the current rendering time according to the conversion relationship between the observer's human eye and the projection center of the projector represented by the optical machine, may also include: step S106 and step S107, such as Figure 3 shown.

[0051] Step S106: Projecting a plurality of known mark points to the position where the light is coupled into the optical waveguide via the optical machine to obtain a plurality of first pixel position points corresponding one-to-one to the plurality of known mark points, imaging the plurality of first pixel position points on the imaging plane of the camera to obtain a plurality of second pixel position points corresponding one-to-one to the plurality of first pixel position points, and the posture of the camera is the same as the posture of the observer's eyes.

[0052] Step S107: Determine the conversion relationship between the observer's eye and the projection center of the projector represented by the optical machine based on a plurality of first connection directions from a plurality of first pixel positions to the projection center of the optical machine and a plurality of second connection directions from a plurality of second pixel positions to the optical center of the camera.

[0053] For optical waveguides, the light received by the human eye is parallel light, that is, within the visible range of the human eye, the image received by the human eye is the same, but due to the difference in the angle of reception of the human eye, the imaging position will be at different positions of the human eye. For AR glasses, with the design of the nose pads of the glasses, the angle of reception of the human eye after the observer wears the AR glasses is not much different. At this time, the second position is mainly affected by the assembly error of the projection center of the optical machine.

[0054] The calculation method of the above steps is as follows:

[0055] The optical machine is regarded as a projector, and a calibration plate with a fixed distance is designed. A plurality of marking points are arranged on the calibration plate. The calibration plate is projected and displayed at the position where the optical waveguide light is coupled into the optical waveguide through the optical machine. The position where the optical waveguide light is coupled into the optical waveguide can be regarded as a virtual optical screen. The first connecting line direction from each marking point on the calibration plate displayed on the optical screen to the projection center of the optical machine (that is, the direction in the projector, which is also the direction of the light emitted by the optical machine, and the first connecting line direction can be represented by the vector n→) n→ can be determined.

[0056] The camera is used to replace the human eye. The process of the observer's human eye watching the optical screen is equivalent to the process of imaging the mark points displayed on the optical screen on the imaging plane of the camera. If the internal parameters of the camera are known, the second line direction from each mark point on the imaging plane (which is obtained by imaging each mark point displayed on the optical screen on the imaging plane of the camera) to the optical center of the camera (that is, the direction in the camera, which is also the direction of the light received by the human eye, and the second line direction can be represented by the vector m→) can be obtained. The following relationship is satisfied between these two directions:

[0057] n→=T rv m→

[0058] Through multiple marking points, the conversion relationship T between the observer's eyes and the projection center of the optical machine can be calculated. rv .

[0059] It is known that the fov (field of view) of the optical machine is θ, and the resolution of the optical machine is W*H (W represents the number of pixels in the horizontal direction, and H represents the number of pixels in the vertical direction), then the equivalent internal parameter K matrix of the optical machine can be written as:

[0060]

[0061] The fixed distance mark point projected on the optical screen is a three-dimensional point P (X, Y, Z) in the projector coordinate system. The first pixel position of point P on the optical screen is (u, v):

[0062]

[0063] Normalizing the above formula yields the first line direction n→, which represents the first line direction from the first pixel position on the optical screen to the projection center of the optical machine.

[0064] If a camera is placed at the human eye, the first pixel position on the optical screen is imaged on the imaging plane of the camera to obtain the second pixel position. The coordinates of the second pixel position in the imaging plane coordinate system are (u', v'). Let the intrinsic parameter matrix of the camera be K', then

[0065]

[0066] When the first connecting line direction n→ and the second connecting line direction m→ are known, the conversion relationship T between the observer's eye and the projection center of the optical machine can be calculated. rv .

[0067] In some embodiments, step S102, determining a first position of the virtual object in the rendering space in the rendering coordinate system at the current rendering moment according to the posture, may include: determining the first position according to the position and posture of the virtual object in the rendering space in the rendering coordinate system at the previous rendering moment.

[0068] In an embodiment of the present application, the position of the virtual object in the rendering coordinate system changes with the change of the posture of the electronic device. The first position of the virtual object in the rendering coordinate system at the current rendering moment is determined according to the position of the virtual object in the rendering coordinate system at the previous rendering moment and the posture of the electronic device at the current rendering moment. In this way, the virtual object in the rendering space can be controlled to change from the position in the rendering coordinate system at the previous rendering moment to the position in the rendering coordinate system at the current rendering moment.

[0069] In some embodiments, step S102, determining the first position according to the position and the posture of the virtual object in the rendering coordinate system at the last rendering time, may also include: sub-steps S1021 to S1023, such as Figure 4 shown.

[0070] Sub-step S1021: determining the position of the virtual object in the world coordinate system at the last rendering time according to the position of the virtual object in the rendering coordinate system at the last rendering time.

[0071] Sub-step S1022: Determine the position of the virtual object in the designated coordinate system at the current rendering time according to the posture of the electronic device in the designated coordinate system at the current rendering time and the position of the virtual object in the world coordinate system at the previous rendering time.

[0072] Sub-step S1023: determining a first position of the virtual object in the rendering coordinate system at the current rendering moment according to the position of the virtual object in the designated coordinate system at the current rendering moment.

[0073] The world coordinate system refers to the same coordinate system that can be used by multiple objects. Therefore, the position of an object in the rendering coordinate system and the position of an object in the specified coordinate system can be converted through the world coordinate system.

[0074] The embodiment of the present application first converts the position of the virtual object in the rendering coordinate system at the previous rendering moment to its position in the world coordinate system, and then determines the position of the virtual object in the specified coordinate system at the current rendering moment in combination with the posture of the electronic device in the specified coordinate system at the current rendering moment, and further determines the position of the virtual object in the rendering coordinate system at the current rendering moment.

[0075] In order to perform the above conversion, (1) it is necessary to obtain the conversion relationship T of the position of the object in the rendering coordinate system (i.e., the rendering space) in the world coordinate system based on known relevant data (i.e., multiple known position points in the rendering coordinate system are converted to corresponding multiple known position points in the world coordinate system) in advance. wo , where the world coordinate system is generally the z-axis direction, which is the opposite direction of the gravity acceleration represented by the IMU (Inertial Measurement Unit) when stationary, and the other two directions are determined by the x-axis direction and the y-axis direction of the IMU. The IMU sensor is set on the electronic device; (2) It is necessary to obtain the position conversion relationship T of the object in the world coordinate system in the specified coordinate system (for example, the eye body coordinate system) in advance based on the known relevant data (that is, multiple known position points in the world coordinate system are converted to the corresponding multiple known position points in the specified coordinate system). bw , this conversion relationship T bw It can be determined by sensors, such as the 3dof method of a single IMU, or by integrating IMU with camera, lidar, gnss (Global Navigation Satellite System), etc. The conversion relationship T bw will change with the change of the posture of the electronic device; (3) it is necessary to obtain the position conversion relationship T from the object in the specified coordinate system (for example, the coordinate system of the eyeglass body) to the rendering coordinate system (i.e., the rendering space) at the next rendering time in advance based on known relevant data (i.e., multiple known position points in the specified coordinate system are converted to corresponding multiple known position points in the rendering coordinate system). ob , this position conversion relationship T obThat is also certain.

[0076] In some embodiments, the electronic device includes AR glasses, and the designated coordinate system includes an AR glasses body coordinate system.

[0077] In some embodiments, if the position of the virtual object at the last rendering moment is a position in the local coordinate system, then the position of the virtual object in the local coordinate system at the last rendering moment needs to be converted to a position in the rendering coordinate system. That is, sub-step S1021, before determining the position of the virtual object in the world coordinate system at the last rendering moment according to the position of the virtual object in the rendering coordinate system at the last rendering moment, may also include: determining the position of the virtual object in the rendering coordinate system at the last rendering moment according to the position of the virtual object in the local coordinate system at the last rendering moment.

[0078] The local coordinate system may also be called the object coordinate system, which may refer to a coordinate system established with an object as the center. The rendering space may include more than two virtual objects, and the position of each virtual object is represented by a local coordinate system. On the one hand, the representation method of the position of the virtual object is not uniform, and on the other hand, it is difficult to reflect the spatial relationship between the virtual objects. Therefore, the position of the virtual object in the local coordinate system at the last rendering moment can be converted to the position in the rendering coordinate system. In order to achieve this conversion, it is necessary to obtain the conversion matrix M from the local coordinate system to the rendering coordinate system in advance based on known relevant data.

[0079] In some embodiments, step S103, determining, based on the first position, a second position of the virtual object projected onto the display screen of the electronic device at the current rendering moment, may include: sub-steps S1031 and S1032, such as Figure 5 shown.

[0080] Sub-step S1031: determining the position of the virtual object in the observer coordinate system at the current rendering moment according to the first position.

[0081] Sub-step S1032: determining the second position according to the position of the virtual object in the observer coordinate system at the current rendering moment.

[0082] Before performing the above calculations, (1) it is necessary to obtain in advance the conversion relationship between the position of the object in the rendering coordinate system and the position of the object in the observer coordinate system (i.e., human eye observation) based on known relevant data (i.e., multiple known position points in the rendering coordinate system are converted to corresponding multiple known position points in the observer coordinate system), i.e., the observer matrix V. The observer matrix V can be obtained by referring to the derivation in computer graphics; (2) it is necessary to obtain in advance the conversion relationship from the object in the observer coordinate system to the display screen of the electronic device, i.e., the projection matrix P, based on known relevant data (i.e., multiple known position points in the observer coordinate system are converted to corresponding multiple known position points in the screen coordinate system of the display screen).

[0083] In one embodiment, according to the posture of the electronic device at the current rendering moment, the virtual object in the rendering space is projected onto the display screen of the electronic device, the position of the virtual object in the local coordinate system is S1, and the position projected onto the display screen of the electronic device is S2, then the position conversion relationship is:

[0084] S2=S1*PT rv VT ob T bw T wo M

[0085] Wherein, M represents the transformation matrix from the local coordinate system to the rendering coordinate system; T wo Represents the transformation relationship between the position of an object in the rendering coordinate system (i.e., rendering space) and the position in the world coordinate system; T bw The transformation relationship between the position of an object in the world coordinate system and the specified coordinate system (for example, the eye body coordinate system); T ob It represents the position transformation relationship from the object in the specified coordinate system (for example, the coordinate system of the glasses body) to the rendering coordinate system (i.e., rendering space) at the next rendering moment; V represents the observer matrix; P represents the projection matrix; T rv Indicates the conversion relationship between the observer's human eyes and the projection center of the optical machine. When the display screen of an electronic device uses an optical waveguide for display, in this way, the display position of the rendered virtual object can also be changed as the posture of the electronic device changes, and the assembly error of the projection center of the optical machine used by the optical waveguide can be corrected.

[0086] See also Figure 6 , Figure 61 is a schematic diagram of the structure of an embodiment of an electronic device of the present application, wherein the electronic device 100 includes a display screen 10, and the electronic device 100 also includes a memory 20 and a processor 30, wherein the memory 20 is used to store a computer program, and the processor 30 is used to execute the computer program and implement any of the above-mentioned virtual object rendering methods when executing the computer program. For a detailed description of the relevant content, please refer to the above-mentioned method part, which will not be repeated here.

[0087] The processor 30 may be a microcontroller unit, a central processing unit or a digital signal processor, etc. The memory 20 may be a Flash chip, a read-only memory, a magnetic disk, an optical disk, a USB disk or a mobile hard disk, etc.

[0088] The present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements any of the above-mentioned virtual object rendering methods. For detailed descriptions of the relevant contents, please refer to the relevant contents of the above-mentioned methods, which will not be repeated here.

[0089] The computer-readable storage medium may be an internal storage unit of the electronic device, such as a hard disk or a memory. The computer-readable storage medium may also be an external storage device, such as a plug-in hard disk, a smart memory card, a secure digital card, a flash memory card, etc.

[0090] It should be understood that the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0091] It should also be understood that the term “and / or” used in the specification and appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0092] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A virtual object rendering method, It is characterized in that The method comprises: Get the position and posture of the electronic device at the current rendering moment; Determine, according to the position and posture, a first position of the virtual object in the rendering space in the rendering coordinate system at the current rendering moment; Determining, according to the first position, a second position at which the virtual object is projected onto the display screen of the electronic device at the current rendering moment; The rendering effect of the virtual object is displayed at a second position on the display screen at the current rendering moment.

2. The method according to claim 1, It is characterized in that The display screen includes a see-through screen.

3. The method according to claim 1, It is characterized in that Before displaying the rendering effect of the virtual object at the second position on the display screen at the current rendering moment, the method includes: Correcting a second position of the virtual object projected onto the display screen at the current rendering moment according to a conversion relationship between an observer's human eye and a projection center of the optical machine, wherein the electronic device includes the optical waveguide, and the optical waveguide includes the optical machine; The step of displaying the rendering effect of the virtual object at the second position on the display screen at the current rendering moment includes: The rendering effect of the virtual object is displayed at the corrected second position on the display screen at the current rendering moment.

4. The method according to claim 3, It is characterized in that Before correcting the second position of the virtual object projected onto the display screen at the current rendering moment according to the conversion relationship between the observer's human eyes and the projection center of the optical machine, the method includes: Projecting a plurality of known mark points to the position where the light waveguide is coupled into light through the optical machine to obtain a plurality of first pixel position points corresponding to the plurality of known mark points respectively, imaging the plurality of first pixel position points on an imaging plane of a camera to obtain a plurality of second pixel position points corresponding to the plurality of first pixel position points respectively, and the posture of the camera is the same as the posture of the human eye of the observer; The conversion relationship between the observer's eye and the projection center of the optical machine is determined based on multiple first connection directions from multiple first pixel positions to the projection center of the optical machine and multiple second connection directions from multiple second pixel positions to the optical center of the camera.

5. The method according to claim 1, It is characterized in that Determining, according to the posture, a first position of the virtual object in the rendering space in the rendering coordinate system at the current rendering moment includes: The first position is determined according to the position and the posture of the virtual object in the rendering space in the rendering coordinate system at the last rendering time.

6. The method according to claim 5, It is characterized in that The determining the first position according to the position and the posture of the virtual object in the rendering coordinate system at the last rendering time includes: Determining the position of the virtual object in the world coordinate system at the last rendering time according to the position of the virtual object in the rendering coordinate system at the last rendering time; Determine the position of the virtual object in the specified coordinate system at the current rendering time according to the posture of the electronic device in the specified coordinate system at the current rendering time and the position of the virtual object in the world coordinate system at the previous rendering time; The first position is determined according to the position of the virtual object in the specified coordinate system at the current rendering moment.

7. The method according to claim 5, It is characterized in that Before determining the position of the virtual object in the world coordinate system at the last rendering time according to the position of the virtual object in the rendering coordinate system at the last rendering time, the method includes: The position of the virtual object in the rendering coordinate system at the last rendering time is determined according to the position of the virtual object in the local coordinate system at the last rendering time.

8. The method according to claim 1, It is characterized in that The step of determining, based on the first position, a second position of the virtual object projected onto the display screen of the electronic device at the current rendering moment includes: Determine, according to the first position, a position of the virtual object in the observer coordinate system at the current rendering moment; The second position is determined according to the position of the virtual object in the observer coordinate system at the current rendering moment.

9. An electronic device, It is characterized in that The electronic device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program and implement the virtual object rendering method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor implements the virtual object rendering method according to any one of claims 1 to 8.

Images