Method and device for displaying virtual elements
By calculating and converting the AR coordinates of virtual elements in real time, generating depth maps and using depth tests, the problem of overlap and occlusion of virtual elements and real scenes in complex scenes is solved, and the effect of flexible viewing and realism is achieved.
Patent Information
- Application Number
- CN202510023407.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-07
- Publication Date
- 2025-05-16
AI Technical Summary
The prior art is difficult to flexibly view hidden virtual elements in real scenes in complex scenes, and cannot be close to the real scenes, resulting in the problem of overlapping and obstructing of virtual elements with real scenes.
By setting the bounding box parameters and scene parameters of the virtual element, the AR coordinates of the bounding box are calculated in real time, and converted into screen coordinates, drawing a black and white graph for binarization, generating a depth map, and using depth tests to achieve occlusion.
It realizes flexible viewing and free movement in complex scenarios, and refreshes virtual elements in hidden areas in real time, ensuring the real sense of the combination of virtual and real, avoiding unnecessary interference, and improving user experience.
Smart Images
Figure CN120014156A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field, and in particular to a method and device for displaying virtual elements. Background Art
[0002] Augmented reality (AR) is a technology that combines virtual elements with real scenes. Through AR technology, users can see virtual elements such as computer-generated text, images, 3D models, videos or audio in the real world. When there are many virtual elements in a complex scene, it is common for virtual elements to overlap with the real scene or for virtual elements to block the real scene, such as virtual pipelines buried underground and in walls.
[0003] In the prior art, there is a method of dividing virtual elements into layer groups and controlling the display or hiding of the layer groups to view the virtual elements hidden in the real scene. However, this method can only display or hide the virtual elements as a whole, which has limitations and cannot be close to the real scene. Therefore, how to view the hidden virtual elements in the real scene more flexibly and closer to the real scene has become an urgent problem to be solved. Summary of the invention
[0004] Based on the above problems, the present application provides a method and device for displaying virtual elements, so as to be more flexible and closer to the real scene when viewing AR hidden facilities.
[0005] The present application discloses a method for displaying a virtual element, the method comprising:
[0006] Setting bounding box parameters and scene parameters of the virtual element, and calculating AR coordinates of the bounding box in real time based on the bounding box parameters and the scene parameters;
[0007] Convert the AR coordinates into screen coordinates, and draw a black and white image according to the screen coordinates; the black and white image shows the scene and the bounding box in the scene;
[0008] Binarize the black and white image to obtain a depth map;
[0009] The depth map is rendered and occlusion is achieved using depth testing.
[0010] Optionally, the setting of the bounding box parameters and the scene parameters of the virtual element, and calculating the AR coordinates of the bounding box in real time based on the bounding box parameters and the scene parameters, includes:
[0011] Set the bounding box parameters; the parameters include the length, width and depth of the bounding box;
[0012] Setting the scene parameters to simulate an AR scene; the scene parameters include the length, width, and height of the AR scene;
[0013] Performing collision detection in real time based on the scene parameters to obtain real-time AR collision point coordinates;
[0014] The AR coordinates are calculated in real time by combining the AR collision point coordinates with the bounding box parameters.
[0015] Optionally, converting the AR coordinates into screen coordinates includes:
[0016] Convert the AR coordinates to the camera coordinate system in the world coordinate system;
[0017] Convert the AR coordinates in the camera coordinate system to the image coordinate system;
[0018] The AR coordinates in the image coordinate system are converted into the pixel coordinate system to obtain the screen coordinates.
[0019] Optionally, after drawing the black and white image according to the screen coordinates, the method further includes:
[0020] The scaling factor is set to scale the black-and-white image to reduce the amount of calculation for binarization processing.
[0021] Optionally, rendering the depth map and implementing occlusion using a depth test includes:
[0022] Performing depth calculation according to the depth map, rendering the content within the bounding box, and filtering the content outside the bounding box;
[0023] Using the depth test, it is determined whether the virtual element needs to be occluded by comparing the depth of the pixel.
[0024] Based on the above method for displaying a virtual element, the present application also discloses a device for displaying a virtual element, including: a setting unit, a conversion unit, a processing unit and a rendering unit;
[0025] The setting unit is used to set the bounding box parameters and scene parameters of the virtual element, and calculate the AR coordinates of the bounding box in real time based on the bounding box parameters and the scene parameters;
[0026] The conversion unit is used to convert the AR coordinates into screen coordinates and draw a black and white image according to the screen coordinates; the black and white image shows the scene and the bounding box in the scene;
[0027] The processing unit is used to perform binarization processing on the black and white image to obtain a depth map;
[0028] The rendering unit is used to render the depth map and implement occlusion using a depth test.
[0029] Optionally, the setting unit includes:
[0030] A setting subunit, used to set the bounding box parameters; the parameters include the length, width and depth of the bounding box;
[0031] A simulation subunit, used to set the scene parameters to simulate the AR scene; the scene parameters include the length, width and height of the AR scene;
[0032] A collision subunit, used to perform collision detection in real time based on the scene parameters and obtain real-time AR collision point coordinates;
[0033] The combining subunit is used to combine the AR collision point coordinates with the bounding box parameters to calculate the AR coordinates in real time.
[0034] Optionally, the conversion unit includes:
[0035] A first conversion subunit, configured to convert the AR coordinates from a world coordinate system to a camera coordinate system;
[0036] A second conversion subunit, used to convert the AR coordinates in the camera coordinate system into the image coordinate system;
[0037] The third conversion subunit is used to convert the AR coordinates in the image coordinate system into the pixel coordinate system to obtain the screen coordinates.
[0038] Optionally, the device further comprises:
[0039] The scaling unit is used to set a scaling factor to scale the black and white image to reduce the amount of calculation for binarization processing.
[0040] Optionally, the rendering unit includes:
[0041] A rendering subunit, configured to perform depth calculation according to the depth map, render the content within the bounding box, and filter the content outside the bounding box;
[0042] The occlusion subunit is used to use a depth test to determine whether the virtual element needs to be occluded by comparing the depths of pixels.
[0043] The present application discloses a method and device for displaying virtual elements. The bounding box parameters and scene parameters of the virtual elements are set, and the virtual elements are integrated into the real scene, enriching the information display. The AR coordinates of the bounding box are calculated in real time based on the bounding box parameters and scene parameters, and the AR coordinates are converted into screen coordinates, and a black and white image is drawn according to the screen coordinates to display the scene and the bounding box therein. The black and white image is binarized to obtain a depth map, which is rendered, and occlusion is achieved using a depth test. In order to solve the problem of overlapping and occlusion between virtual elements and real scenes in complex scenes, especially the display of hidden facilities, an innovative solution is proposed. Unlike the existing method of displaying or hiding AR virtual elements in layers, the present application can accurately view and freely move, and refresh AR virtual elements in specific hidden areas in real time, while ensuring the realism of the combination of virtual and real through real-time testing and rendering. The part outside the bounding box displays the real scene normally, avoiding unnecessary interference. A more efficient, flexible and real-life combination of virtual and real scenes is achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.
[0045] Figure 1a A flowchart of a method for displaying virtual elements disclosed in an embodiment of the present application;
[0046] Figure 1b A schematic diagram of a bounding box disclosed in an embodiment of the present application;
[0047] Figure 1c A schematic diagram of the pinhole imaging principle disclosed in the embodiments of the present application;
[0048] Figure 1d A schematic diagram of a black and white image disclosed in an embodiment of the present application;
[0049] Figure 1e A schematic diagram of a result of displaying a virtual element disclosed in an embodiment of the present application;
[0050] Figure 2 A flowchart of another method for displaying virtual elements disclosed in an embodiment of the present application;
[0051] Figure 3 A schematic diagram of the structure of a device for displaying virtual elements disclosed in an embodiment of the present application. DETAILED DESCRIPTION
[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0053] Embodiment 1: This application discloses a method for displaying virtual elements.
[0054] For details, please refer to Figure 1a A method for displaying a virtual element disclosed in this embodiment includes the following steps:
[0055] Step 101: setting bounding box parameters and scene parameters of a virtual element, and calculating AR coordinates of the bounding box in real time based on the bounding box parameters and the scene parameters.
[0056] In the method described in this embodiment, the virtual elements required to be displayed can be flexibly loaded in the AR scene according to the actual work needs. These virtual elements may include but are not limited to complex models such as pipelines, as well as other types such as gltf models, BIM models, video elements, web page elements, etc. The loading process can be implemented through AR technology platforms such as ARCore, etc., to ensure that the virtual elements can be accurately and stably integrated into the real scene. There is no strict limitation on the specific method of loading virtual elements or the type of virtual elements, as long as the loading task can be completed and the actual work needs can be met.
[0057] In the method described in this embodiment, it is necessary to set the bounding box parameters and scene parameters. The bounding box is the part that needs to be displayed on the wall or ground in the AR scene, and can be a rectangular parallelepiped, or a cylinder, a sphere, etc. Figure 1b As shown, the bounding box parameters are used to define the size of the bounding box in the AR scene. When the bounding box is a cuboid, the bounding box parameters include the length, width, and depth of the bounding box. Correspondingly, when the bounding box is a sphere, the bounding box parameters include parameters such as the radius of the bounding box. The shape of the bounding box and the values of the bounding box parameters are not specifically limited here, as long as they can meet actual work requirements.
[0058] Among them, the scene parameters are set according to the actual work site environment. They are parameters of an AR scene to simulate the walls, floors, ceilings, etc. in the actual environment for collision detection calculations. For example, the AR virtual scene can be a cuboid, and its scene parameters can be 6 meters long, 6 meters wide, and 6 meters high. Correspondingly, the AR virtual scene can also be a cylinder, a sphere, etc., in which case its scene parameters can be ground radius, height, etc. There is no specific limitation on the shape of the AR virtual scene and the values of the scene parameters, as long as they can meet the actual work needs.
[0059] In the method described in this embodiment, collision detection is performed in real time based on scene parameters, and real-time and dynamic AR collision point coordinates can be obtained. Specifically, by emitting rays to the wall in the AR scene that is pre-configured according to the scene parameters, the real-time determination of the collision point can be achieved. The starting point of this ray is set to the center point of the screen, and its direction points directly to the wall. When this ray intersects the wall, the intersection between them is defined as the coordinates of the AR collision point. In actual work, the user's perspective often moves continuously with the changes in task requirements, and the center point of the screen also moves with the user's perspective. In order to maintain the accuracy and real-time performance of collision detection, the method described in this embodiment can continuously update the position of the center point of the screen and re-emit rays for collision detection. Thereby, the latest position of the collision between the center point of the screen and the wall can be captured in real time, and the real-time moving AR collision point coordinates can be obtained.
[0060] In the method described in this embodiment, the AR collision point coordinates are combined with the preset bounding box parameters to further achieve precise positioning of the bounding box in the AR scene. When the bounding box is a cuboid, taking the AR collision point coordinates as the starting point, combined with the preset bounding box parameters (length, width, depth), the complete coordinate information of the bounding box in the AR scene can be calculated, and the AR coordinates of the bounding box in the AR scene can be calculated in real time. Specifically, the AR collision point coordinates can be regarded as a corner point of the bounding box, and the coordinates of other corner points are calculated in turn according to the bounding box parameters. These corner point coordinates jointly define the position and shape of the bounding box in the AR scene. Since the collision detection is real-time, the AR coordinates of the bounding box will also be updated in real time with the user's perspective and interactive actions.
[0061] Step 102: convert the AR coordinates into screen coordinates, and draw a black and white image according to the screen coordinates.
[0062] In the method described in this embodiment, the AR coordinates are converted from the world coordinate system to the camera coordinate system, and then the AR coordinates in the camera coordinate system are converted to the image coordinate system. The AR coordinates in the image coordinate system are then converted to the pixel coordinate system, and finally the screen coordinates are obtained. Among them, the conversion of the AR coordinates from the world coordinate system to the camera coordinate system can be specifically achieved through the camera extrinsic matrix, as shown in the following formula:
[0063]
[0064] In the formula, the coordinates of w in the table below are the coordinates in the world coordinate system, the coordinates of c in the table below are the coordinates in the camera coordinate system, and the matrix in the middle is the camera extrinsic parameter matrix.
[0065] In the method described in this embodiment, the AR coordinates in the camera coordinate system are converted to the image coordinate system, which can be achieved by the camera intrinsic parameter matrix, mainly using the pinhole imaging principle. Figure 1c As shown, Figure 1c The left half is represented in the camera coordinate system, where camera centre is the center of the camera coordinate system, image plane is the image plane (two-dimensional plane), and principal axes are the principal axes. Figure 1c The right half is the transformed image coordinate system.
[0066] In the method described in this embodiment, the AR coordinates in the image coordinate system are converted to the pixel coordinate system, which can be achieved by translation transformation. For example, the actual physical size represented by each pixel in the image is determined by the resolution information of the image, and the translation amount from the image coordinate system to the pixel coordinate system is calculated. Then, based on the translation amount, the AR coordinates in the image coordinate system are converted to the pixel coordinate system by translation transformation to obtain the screen coordinates, so as to facilitate the use of the screen coordinates in digital image processing and analysis.
[0067] In the method described in this embodiment, a black and white image can be drawn according to the screen coordinates, such as Figure 1d As shown, the black and white image is used to show the scene and the bounding box in the scene. The white part is the inside of the bounding box, and the black part is the outside of the bounding box, that is, the scene. The bounding box is represented as an ellipse in the black and white image. The specific shape of the bounding box is not limited here. It can also be represented as a rectangle, circle, etc. in the black and white image.
[0068] In the method described in this embodiment, as an optional method, a scaling factor can be set to scale the black and white image to reduce the amount of calculation for binarization. For example, when the scaling factor is set to 0.5, the black and white image will be reduced to half of its original size. At this time, the pixel value required for depth calculation becomes half of the original, reducing the amount of calculation. In actual work, the scaling factor can be set according to demand. The specific value of the scaling factor is not limited here, and it can achieve the effect of meeting actual work needs and reducing the amount of calculation.
[0069] Step 103: binarize the black-and-white image to obtain a depth map.
[0070] In the method described in this embodiment, as an optional method, a black-and-white image can be read, and the specific step of binarizing it can be to set the depth value of the white pixel in the black-and-white image to a preset maximum value and the depth value of the black pixel to 0, thereby obtaining a depth map. The preset maximum value can be set to 32767 based on experience in actual work, and the specific data of the preset maximum value is not limited here, and it is sufficient to obtain a depth map.
[0071] Step 104: Render the depth map and implement occlusion using depth testing.
[0072] In the method described in this embodiment, depth calculation is performed based on the depth map, the content within the bounding box is rendered, and the content outside the bounding box is filtered. As an optional method, the obtained depth map can be passed to the SceneForm rendering engine, and SceneForm can perform depth calculation based on the depth map, only render the content within the bounding box, and filter out the content outside the bounding box.
[0073] In the method described in this embodiment, a depth test is used to determine whether a virtual element needs to be blocked by comparing the depth of pixels. As an optional method, a depth test can be used through SceneForm to achieve a real-time blocking effect. Figure 1e As shown in the figure, the box in the middle of the screen is the bounding box (the area behind the board in the AR scene), and the rest is the AR scene (AR real scene, board, etc.). It can be seen that the bounding box displays virtual elements such as pipelines, bounding box parameters, virtual data parameters, etc. In the AR scene outside the bounding box, only the real scene is displayed.
[0074] The method described in this embodiment utilizes AR technology to load multiple virtual elements in a real scene, including but not limited to glTF models, pipeline models, BIM models, video elements, and web page elements, thereby enhancing the information richness and interactivity of the scene. In view of the problem of overlapping and blocking virtual elements and real scenes in complex AR scenes, the method described in this embodiment is different from the existing method of dividing virtual elements into layer groups for display or hiding. Instead, it is possible to view virtual elements in a specific hidden area, and the area can be freely moved and refreshed in real time, which is suitable for processing hidden facilities such as underground and wall pipelines, providing more precise and flexible control. Among them, by performing real-time collision testing and calculation on the target, it is ensured that when moving the bounding box, it has the effect of sticking to the wall and the ground, and the display content can be updated in real time, maintaining a high degree of virtual-real combination realism. The real scene can be displayed normally for the area outside the bounding box, avoiding unnecessary interference and improving the authenticity and immersion of the user experience.
[0075] Embodiment 2: This application discloses another method for displaying virtual elements. Figure 2 The method described in this embodiment introduces the whole process of real-time display of virtual elements.
[0076] Step 201: Load virtual elements and set bounding box parameters and scene parameters.
[0077] Step 202: Obtain the center point of the screen at the current moment, perform collision detection based on the center point of the screen and scene parameters, and obtain the coordinates of the AR collision point.
[0078] Step 203: Calculate the AR coordinates of the bounding box based on the AR collision point coordinates and the bounding box parameters.
[0079] Step 204: convert the AR coordinates of the bounding box into screen coordinates, and draw a black and white image according to the screen coordinates.
[0080] Step 205: Black and white Figure 2 The depth map is obtained by value processing.
[0081] Step 206: Render the depth map, only render the content within the bounding box, and filter the content outside the bounding box.
[0082] Step 207: Use depth test to implement occlusion, and use the next moment as the current moment. Return to step 202.
[0083] Based on the method for displaying virtual elements disclosed in the above embodiment, this embodiment correspondingly discloses a device for displaying virtual elements. Figure 3 , the device for displaying virtual elements includes: a setting unit 301, a conversion unit 302, a processing unit 303 and a rendering unit 304;
[0084] The setting unit 301 is used to set the bounding box parameters and scene parameters of the virtual element, and calculate the AR coordinates of the bounding box in real time based on the bounding box parameters and the scene parameters;
[0085] The conversion unit 302 is used to convert the AR coordinates into screen coordinates and draw a black and white image according to the screen coordinates; the black and white image shows the scene and the bounding box in the scene;
[0086] The processing unit 303 is used to perform binarization processing on the black and white image to obtain a depth map;
[0087] The rendering unit 304 is used to render the depth map and implement occlusion using a depth test.
[0088] Optionally, the setting unit 301 includes:
[0089] A setting subunit, used to set the bounding box parameters; the parameters include the length, width and depth of the bounding box;
[0090] A simulation subunit, used to set the scene parameters to simulate the AR scene; the scene parameters include the length, width and height of the AR scene;
[0091] A collision subunit, used to perform collision detection in real time based on the scene parameters and obtain real-time AR collision point coordinates;
[0092] The combining subunit is used to combine the AR collision point coordinates with the bounding box parameters to calculate the AR coordinates in real time.
[0093] Optionally, the conversion unit 302 includes:
[0094] A first conversion subunit, configured to convert the AR coordinates from a world coordinate system to a camera coordinate system;
[0095] A second conversion subunit, used to convert the AR coordinates in the camera coordinate system into the image coordinate system;
[0096] The third conversion subunit is used to convert the AR coordinates in the image coordinate system into the pixel coordinate system to obtain the screen coordinates.
[0097] Optionally, the device further comprises:
[0098] The scaling unit is used to set a scaling factor to scale the black and white image to reduce the amount of calculation for binarization processing.
[0099] Optionally, the rendering unit 304 includes:
[0100] A rendering subunit, configured to perform depth calculation according to the depth map, render the content within the bounding box, and filter the content outside the bounding box;
[0101] The occlusion subunit is used to use a depth test to determine whether the virtual element needs to be occluded by comparing the depths of pixels.
[0102] The embodiments in this specification are described in a progressive manner. As for the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the methods.
[0103] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.
[0104] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
[0105] The features described in the embodiments of this specification can be replaced or combined with each other, so that professional and technical personnel in this field can implement or use the present application.
[0106] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. A method for displaying a virtual element, characterized in that: include: Setting bounding box parameters and scene parameters of the virtual element, and calculating AR coordinates of the bounding box in real time based on the bounding box parameters and the scene parameters; Convert the AR coordinates into screen coordinates, and draw a black and white image according to the screen coordinates; The black and white image shows the scene and the bounding box in the scene; Binarize the black and white image to obtain a depth map; The depth map is rendered and occlusion is achieved using depth testing.
2. The method according to claim 1, characterized in that The setting of the bounding box parameters and the scene parameters of the virtual element, and calculating the AR coordinates of the bounding box in real time based on the bounding box parameters and the scene parameters, includes: Set the bounding box parameters; the parameters include the length, width and depth of the bounding box; Setting the scene parameters to simulate an AR scene; the scene parameters include the length, width, and height of the AR scene; Performing collision detection in real time based on the scene parameters to obtain real-time AR collision point coordinates; The AR coordinates are calculated in real time by combining the AR collision point coordinates with the bounding box parameters.
3. The method according to claim 1, characterized in that The converting the AR coordinates into screen coordinates comprises: Convert the AR coordinates to the camera coordinate system in the world coordinate system; Convert the AR coordinates in the camera coordinate system to the image coordinate system; The AR coordinates in the image coordinate system are converted into the pixel coordinate system to obtain the screen coordinates.
4. The method according to claim 1, characterized in that After drawing the black and white image according to the screen coordinates, the method further includes: The scaling factor is set to scale the black-and-white image to reduce the amount of calculation for binarization processing.
5. The method according to claim 1, characterized in that The rendering of the depth map and implementing occlusion using a depth test includes: Performing depth calculation according to the depth map, rendering the content within the bounding box, and filtering the content outside the bounding box; Using the depth test, it is determined whether the virtual element needs to be occluded by comparing the depth of the pixel.
6. A device for displaying virtual elements, characterized in that: include: Setting unit, conversion unit, processing unit and rendering unit; The setting unit is used to set the bounding box parameters and scene parameters of the virtual element, and calculate the AR coordinates of the bounding box in real time based on the bounding box parameters and the scene parameters; The conversion unit is used to convert the AR coordinates into screen coordinates and draw a black and white image according to the screen coordinates; the black and white image shows the scene and the bounding box in the scene; The processing unit is used to perform binarization processing on the black and white image to obtain a depth map; The rendering unit is used to render the depth map and implement occlusion using a depth test.
7. The device according to claim 6, characterized in that The setting unit comprises: A setting subunit, used to set the bounding box parameters; the parameters include the length, width and depth of the bounding box; A simulation subunit, used to set the scene parameters to simulate the AR scene; the scene parameters include the length, width and height of the AR scene; A collision subunit, used to perform collision detection in real time based on the scene parameters and obtain real-time AR collision point coordinates; The combining subunit is used to combine the AR collision point coordinates with the bounding box parameters to calculate the AR coordinates in real time.
8. The device according to claim 6, characterized in that The conversion unit comprises: A first conversion subunit, configured to convert the AR coordinates from a world coordinate system to a camera coordinate system; A second conversion subunit, used to convert the AR coordinates in the camera coordinate system into the image coordinate system; The third conversion subunit is used to convert the AR coordinates in the image coordinate system into the pixel coordinate system to obtain the screen coordinates.
9. The device according to claim 6, characterized in that The device also includes: The scaling unit is used to set a scaling factor to scale the black and white image to reduce the amount of calculation for binarization processing.
10. The device according to claim 6, characterized in that The rendering unit comprises: A rendering subunit, configured to perform depth calculation according to the depth map, render the content within the bounding box, and filter the content outside the bounding box; The occlusion subunit is used to use a depth test to determine whether the virtual element needs to be occluded by comparing the depths of pixels.