Method and device for displaying virtual objects based on spatial position illumination

Abstract

The present application relates to the field of AR technology and provides a method and device for displaying virtual objects based on lighting of spatial positions. When a virtual object is placed on a real picture, a three-dimensional point cloud dataset corresponding to the real picture is constructed, and the two-dimensional coordinates of the virtual object at the target position to be placed are converted into three-dimensional coordinates. A target point cloud dataset contained in a rectangular block with the three-dimensional coordinates as the center and preset length, width and height as sides is obtained from the three-dimensional point cloud dataset, and a lighting information map is determined based on the target point cloud dataset. The lighting information map can accurately reflect the lighting information of the target position in the real environment. Furthermore, based on the lighting information map, the virtual object placed at the target position is rendered so that the surface lighting of the virtual object is consistent with the real environment, thereby realistically superimposing the virtual object on the real picture, improving the authenticity of virtual fusion, and thus enhancing the AR experience.

Description

Technical Field

[0001] The present application relates to the field of augmented reality (AR) technology, and in particular to a method and device for displaying virtual objects based on spatial position illumination. Background Art

[0002] AR technology is a new technology developed on the basis of virtual reality. It is a technology that increases the user's perception of the real world through information provided by the computer system, and superimposes virtual objects, virtual scenes, system prompt information or non-geometric information about real objects generated by the computer system into the real scene, thereby achieving "enhancement" of the real world.

[0003] In AR experiences, users have a subtle perception of lighting, and lighting consistency is often considered a key indicator of virtual-reality integration. Lighting consistency ensures that virtual objects have the same lighting as in the real world. The goal of lighting consistency is to ensure that the lighting conditions of virtual objects match those in the real world, ensuring consistent lighting and shadows between virtual objects and real objects, enhancing the realism of virtual objects.

[0004] Currently, most related technologies are based on input RGB images and use deep learning models to estimate lighting information. Since RGB images are two-dimensional, this method cannot accurately estimate lighting information in 3D space. Summary of the Invention

[0005] Embodiments of the present application provide a method and device for displaying virtual objects based on spatial position illumination, for improving the illumination consistency between virtual objects and the real environment.

[0006] In a first aspect, embodiments of the present application provide a method for displaying virtual objects based on spatial position illumination, which is applied to AR scenarios and includes:

[0007] In response to the operation of placing a virtual object on a real picture, a three-dimensional point cloud dataset corresponding to the real picture is constructed using simultaneous localization and mapping (SLAM) technology;

[0008] Determining the three-dimensional coordinates of the target position for placing the virtual object in the three-dimensional space;

[0009] Acquire, from the three-dimensional point cloud dataset, a target point cloud dataset contained in a cuboid with the three-dimensional coordinates as the center and a preset length, width, and height as the sides;

[0010] Determining a lighting information map corresponding to the target position according to the target point cloud data set;

[0011] According to the illumination information map, a virtual object placed at the target position is rendered, and the virtual object is superimposed and displayed on the real picture.

[0012] In a second aspect, an embodiment of the present application provides a display device that supports augmented reality (AR) functions, including a processor, a memory, a camera, and a display screen, wherein the display screen, the camera, the memory, and the processor are connected via a bus;

[0013] The camera is used to capture real images;

[0014] The memory stores a computer program, and the processor performs the following operations according to the computer program:

[0015] In response to an operation of placing a virtual object on a real image displayed on the display screen, a three-dimensional point cloud dataset corresponding to the real image is constructed using a simultaneous localization and mapping (SLAM) technology;

[0016] Determining the three-dimensional coordinates of the target position for placing the virtual object in the three-dimensional space;

[0017] Acquire, from the three-dimensional point cloud dataset, a target point cloud dataset contained in a cuboid with the three-dimensional coordinates as the center and a preset length, width, and height as the sides;

[0018] Determining a lighting information map corresponding to the target position according to the target point cloud data set;

[0019] A virtual object placed at the target position is rendered according to the lighting information map, and the virtual object is superimposed and displayed on the real picture through the display screen.

[0020] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute a method for displaying virtual objects based on lighting in a spatial position.

[0021] In the above-mentioned embodiment of the present application, when a virtual object is placed on a real screen, the three-dimensional coordinates of the target position where the virtual object is to be placed in the three-dimensional space are determined, and a target point cloud data set contained in a rectangular block with the three-dimensional coordinates as the center and the preset length, width and height as the sides is obtained from the three-dimensional point cloud data set corresponding to the real screen. By describing the real environment in the form of a point cloud, each point cloud data contains the three-dimensional coordinates, color information and intensity information of the corresponding point on the real object, so that the target point cloud data set can be used to generate a lighting information map corresponding to the target position where the virtual object is to be placed. The lighting information map can accurately reflect the lighting information of the real environment. Furthermore, according to the lighting information map, the virtual object placed at the target position is rendered so that the surface lighting of the virtual object is consistent with the real environment, so that the virtual object is realistically superimposed and displayed on the real screen, thereby improving the authenticity of virtual fusion and thus enhancing the AR experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] Figure 1 The lighting effects at different locations in a real environment provided by the embodiments of the present application are exemplarily shown;

[0024] Figure 2 The flowchart of the method for displaying virtual objects based on spatial position illumination provided by an embodiment of the present application is exemplified;

[0025] Figure 3 The following is an exemplary diagram of an interface for placing a virtual object provided in an embodiment of the present application;

[0026] Figure 4 The following is a flowchart of a method for determining an illumination information map based on three-dimensional point cloud data provided by an embodiment of the present application;

[0027] Figure 5 The flowchart of the method for rendering a virtual object provided by an embodiment of the present application is exemplarily shown;

[0028] Figure 6 The AR effect diagram provided by the embodiment of the present application is exemplarily shown;

[0029] Figure 7 The structural diagram of the display device provided in the embodiment of the present application is exemplarily shown. DETAILED DESCRIPTION

[0030] In order to make the purpose, implementation mode and advantages of the present application clearer, the exemplary implementation mode of the present application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0031] All other embodiments derived by persons of ordinary skill in the art based on the exemplary embodiments described herein without inventive effort are within the scope of protection of the claims appended hereto. Furthermore, although the disclosure herein is presented based on one or more exemplary embodiments, it should be understood that each aspect of the disclosure may constitute a complete embodiment on its own.

[0032] In AR scenarios, capturing ambient lighting is crucial to realistically place simulated virtual objects on the real scene. Virtual objects should have consistent lighting effects with the real environment, which requires AR devices to be able to perceive ambient lighting. Specifically, in darker environments, the surface brightness of virtual objects should be dimmed; in brighter environments, the surface brightness of virtual objects should be brighter. If virtual objects have a specular material, their surface can reflect the real scene.

[0033] In a real environment, objects at different locations often receive different lighting information. Figure 1 As shown, in Figure 1 In the example, position A is above the table, so the lighting effect is brighter, while position B is below the table, so the lighting effect is darker. To achieve a more realistic AR fusion effect for virtual objects in the AR scene, different lighting information is required for virtual objects at different positions.

[0034] For example, when a virtual object is placed at position A, the surface of the virtual object is brighter, and when the virtual object is made of a mirror-reflective material, the reflection effect at position A should be reflected; when the virtual object is placed at position B, the surface of the virtual object is darker, and when the virtual object is made of a mirror-reflective material, the reflection effect at position B should be reflected.

[0035] The method and device for displaying virtual objects based on spatial position illumination provided in the embodiments of the present application can achieve a virtual object's illumination effect that changes with placement. Specifically, when a user places a virtual object, the illumination information for the virtual object's placement position is determined based on the virtual object's point cloud data in three-dimensional space. This illumination information enables the virtual object to have correct light and dark variations, and when the virtual object's surface is a mirror-reflective material, it has a correct reflection effect. Furthermore, based on the determined illumination information, the virtual object is realistically superimposed on the real image, improving the authenticity of the virtual-reality fusion and enhancing the user's AR experience.

[0036] See also Figure 2 , is a flow chart of a method for displaying virtual objects based on spatial location illumination provided in an embodiment of the present application. This process is executed by a display device that supports AR functions, including but not limited to smartphones, tablets, laptops, smart TVs, desktops, in-vehicle devices, wearable devices, and other display devices. The process mainly includes the following steps:

[0037] S201: In response to an operation of placing a virtual object on a real picture, SLAM technology is used to construct three-dimensional point cloud data corresponding to the real picture.

[0038] In an optional implementation, taking the display device as a tablet as an example, Figure 3 As shown, the user opens the AR app, which activates the camera to capture and display a color image of the real environment. During the display process, the app detects whether the user has touched or clicked the display. When the user touches or clicks the display, the touch or click location is used as the target location for placing a virtual object in the real image, and a prompt message pops up asking the user whether to place the virtual object. When the user clicks the "Yes" option, the AR app uses SLAM (Smultaneous Localization and Mapping) technology to construct a 3D point cloud dataset corresponding to the real image. Each point cloud data in this 3D point cloud dataset contains the 3D coordinates, color information, and intensity information of the corresponding point on the real object in 3D space.

[0039] S202: Determine the three-dimensional coordinates of the target position for placing the virtual object in the three-dimensional space.

[0040] In an embodiment of the present application, the real picture captured by the camera of the display device is displayed through a two-dimensional display screen. The user touches or clicks the display screen to select the target position for placing the virtual object on the real picture. The display device determines the two-dimensional coordinates of the target position on the display screen based on the touch or click result. In addition, when the display device is displaying the real picture, the AR application will establish a virtual screen corresponding to the display screen in three-dimensional space. After the two-dimensional coordinates are determined, a ray is emitted in a direction perpendicular to the display screen with the two-dimensional coordinates as the starting point, and the three-dimensional coordinates of the intersection of the ray and the virtual screen are determined as the three-dimensional coordinates of the target position.

[0041] S203: Obtaining a target point cloud dataset from the three-dimensional point cloud dataset, which is contained in a cuboid with the three-dimensional coordinates as the center and the preset length, width, and height as the sides.

[0042] Typically, a three-dimensional point cloud dataset covers the entire area of ​​the real picture, while virtual objects are only placed in a local area of ​​the real picture. Therefore, a cuboid can be constructed with the three-dimensional coordinates as the center and the preset length, width, and height as the sides. The preset length, width, and height can be set according to empirical values ​​or virtual scenes. The smaller the volume of the cuboid, the less calculation time is required to estimate the lighting. Optionally, the preset length, width, and height are greater than the length, width, and height of the virtual object. Furthermore, the point cloud data contained in the cuboid is obtained from the point cloud dataset of the real picture, thereby obtaining the point cloud of the real object around the virtual object, thereby improving the accuracy of lighting estimation.

[0043] S204: Determine a lighting information map corresponding to the target position according to the target point cloud dataset.

[0044] In S204, each target point cloud data in the target point cloud data set contains the three-dimensional coordinates, color information, and intensity information of the corresponding point on the real object around the virtual object to be placed in the three-dimensional space, so as to accurately estimate the ambient lighting of the target location and improve the lighting consistency between the virtual object placed at the target location and the real environment. For the specific estimation process of lighting information, see Figure 4 :

[0045] S2041: Determine at least one surface included in the target point cloud dataset and obtain a surface set.

[0046] In an optional embodiment, a Random Sample Consensus (RANSAC) algorithm is used to fit the target point cloud dataset, and a surface set is generated based on at least one fitted surface. The fitted surface can be a plane or a curved surface, and different types of surfaces have different corresponding fitting equations.

[0047] It should be noted that the method for obtaining the surface set also includes implicit function method or triangulation method, etc., and this application does not make any restrictive requirements.

[0048] S2042: Using the three-dimensional coordinates of the target position as the origin, evenly divide multiple directional rays in the three-dimensional space.

[0049] In S2042, a baseline is selected with the three-dimensional coordinates of the target position as the origin, and the 360° three-dimensional space is evenly divided into multiple directional rays at a fixed angle. The more directional rays are divided, the richer the lighting information is, and the more realistic the lighting effect on the surface of the virtual object is.

[0050] S2043: For each directional ray, determine whether the directional ray intersects with the target point cloud data closest to the origin in the cuboid. If so, execute S2044; otherwise, execute S2045.

[0051] In the embodiments of the present application, the origin is the target location for placing a virtual object, and the target point cloud data within the cuboid represents the corresponding point on the real object, including the 3D coordinates, color information, and intensity information of the point. Based on the 3D coordinates of the target location and the 3D coordinates of the target point cloud data within the cuboid, the target point cloud data closest to the origin can be determined. This target point cloud data can accurately reflect the lighting information of the target location. Furthermore, it is determined whether the directional ray intersects with the target point cloud data closest to the origin, and based on the intersection result, the lighting information corresponding to the target location is determined.

[0052] S2044: The color information of the point cloud data closest to the origin is used as the color information of the sub-sphere corresponding to the sphere center angle of the ray in this direction.

[0053] In the embodiments of the present application, a 360° three-dimensional space is divided into multiple directional rays. The sphere center angle corresponding to each directional ray is the same size, 360° / N, where N is the number of directional rays. When a directional ray intersects with the target point cloud data closest to the origin, it indicates that the target point cloud data closest to the origin can be used to estimate the illumination information at the target location. Therefore, the color information of the point cloud data closest to the origin can be used as the color information of the sub-sphere corresponding to the sphere center angle of the directional ray.

[0054] S2045: Determine whether the directional ray intersects with a surface in the surface set. If so, execute S2046; otherwise, execute S2047.

[0055] When the directional ray does not intersect with the target point cloud data closest to the origin, it can be determined whether the directional ray intersects with a surface in the surface set, thereby estimating the lighting information of the target position based on the target point cloud data on the surface.

[0056] S2046: Interpolate the color information of the target point cloud data within a preset range on the surface with the color information of the intersecting point cloud data, and use the interpolated color information as the color information of the sub-sphere corresponding to the sphere center angle of the ray in the direction.

[0057] When the directional ray intersects with a surface in the surface set, an optional implementation method is: interpolating the color information of the target point cloud data adjacent to the intersecting point cloud data on the surface with the color information of the intersecting point cloud data, and using the interpolated color information as the color information of the sub-sphere corresponding to the sphere center angle of the directional ray.

[0058] S2047: Interpolate the color information of the sub-spheres adjacent to the remaining sub-spheres that are not assigned color values ​​on the spherical surface to obtain the color information of the remaining sub-spheres.

[0059] In the embodiments of the present application, when the directional ray does not intersect the target point cloud data closest to the origin within the cuboid, and does not intersect any surface in the surface set, the sub-sphere corresponding to the sphere center angle of the directional ray is not assigned color information. The sub-sphere not assigned color information is referred to as the remaining sub-sphere. For the remaining sub-sphere, the color information of the sub-spheres adjacent to the remaining sub-sphere is interpolated, and the interpolated color information is used as the color information of the remaining sub-sphere.

[0060] S2048: Obtain a panoramic image corresponding to a sphere with the three-dimensional coordinates as the sphere center and a preset length as the radius based on the color information of each sub-sphere determined by the multiple directional rays, and use the panoramic image as the lighting information map corresponding to the target position.

[0061] In an embodiment of the present application, each sub-sphere constitutes a complete sphere with a three-dimensional coordinate as the center and a preset length as the radius. Since the sub-sphere corresponding to each directional ray is assigned color information, a panoramic image corresponding to the sphere is obtained, and the panoramic image is used as the lighting information map corresponding to the target position to estimate the lighting information of the target position in various directions, thereby improving the lighting effect of the virtual object placed at the target position.

[0062] S205: Rendering a virtual object placed at the target location according to the lighting information map, and superimposing the virtual object on the real image.

[0063] The rendering process of virtual objects can be found in Figure 5 :

[0064] S2051: Use the lighting information map as a sky box surrounding the virtual object, and set the sky box to be invisible.

[0065] In the embodiments of this application, the Unity engine on the display device is preconfigured by checking the "Invisible" option to set the skybox to invisible to prevent it from obstructing the real image on the display device. The skybox consists of six surfaces: top, bottom, left, right, and front. By mapping the lighting information map to the skybox surrounding the virtual object, the lighting texture of the virtual object in all directions at the target location can be obtained. This allows accurate determination of shadows and reflections on the virtual object's surface, improving the realism of the lighting.

[0066] S2052: Rendering a virtual object placed at the target position according to the lighting textures in all directions of the sky box.

[0067] Generally, the shapes of virtual objects are irregular, and the surface shapes of different virtual objects are different. The skybox is a regular hexahedron that can surround virtual objects. In this way, by rendering virtual objects with the help of the skybox, the lighting textures in the six directions of up and down, left and right, front and back can be applied to virtual objects of different shapes, thereby improving the compatibility of virtual objects.

[0068] When rendering and displaying virtual objects using the method provided in the embodiments of the present application, the lighting information map at the spatial position can be accurately estimated according to the spatial position of the virtual object, and the lighting information map can be set as the sky box in the Unity engine to change the surface brightness of the virtual object. When the virtual object is a mirror-reflective material, it can reflect the image on the lighting information map, thereby achieving the effect of reflecting the real environment, improving the lighting consistency between the virtual object and the real environment, making the fusion of virtual and real more realistic, and helping to enhance the AR experience.

[0069] See also Figure 6 , which is an AR effect diagram obtained according to the method provided in the embodiment of the present application, wherein the teapot is an added virtual object, and the surface brightness and light and shadow of the teapot are consistent with the light intensity of the real environment, thereby improving the authenticity of the picture.

[0070] Based on the same technical concept, an embodiment of the present application provides a display device that supports AR function and can implement the method steps of displaying virtual objects based on spatial position illumination in the above embodiment, and can achieve the same technical effect.

[0071] See also Figure 7 The display device includes a processor 701, a memory 702, a camera 703, and a display screen 704, wherein the display screen 704, the camera 703, the memory 702 and the processor 701 are connected via a bus 705;

[0072] The camera 703 is used to capture real images;

[0073] The memory 702 stores a computer program, and the processor 701 performs the following operations according to the computer program stored in the memory 702:

[0074] In response to an operation of placing a virtual object on a real image displayed on the display screen 704, a three-dimensional point cloud dataset corresponding to the real image is constructed using a simultaneous localization and mapping (SLAM) technology;

[0075] Determining the three-dimensional coordinates of the target position for placing the virtual object in the three-dimensional space;

[0076] Acquire, from the three-dimensional point cloud dataset, a target point cloud dataset contained in a cuboid with the three-dimensional coordinates as the center and a preset length, width, and height as the sides;

[0077] Determining a lighting information map corresponding to the target position according to the target point cloud data set;

[0078] According to the illumination information map, a virtual object placed at the target position is rendered, and the virtual object is superimposed and displayed on the real picture through the display screen 704.

[0079] Optionally, the processor 701 determines the illumination information map corresponding to the target position according to the target point cloud dataset, specifically by:

[0080] Determine at least one surface included in the target point cloud data set to obtain a surface set, where the surfaces in the surface set are planes or curved surfaces;

[0081] Taking the three-dimensional coordinate as the origin, evenly dividing a plurality of directional rays in the three-dimensional space;

[0082] For each directional ray, if the directional ray intersects with the target point cloud data closest to the origin in the cuboid, the color information of the closest target point cloud data is used as the color information of the sub-sphere corresponding to the sphere center angle of the directional ray; or, if the directional ray intersects with a surface in the surface set, the color information of the target point cloud data within a preset range on the surface is interpolated with the color information of the intersecting point cloud data, and the interpolated color information is used as the color information of the sub-sphere corresponding to the sphere center angle of the directional ray;

[0083] According to the color information of each sub-sphere determined by the multiple directional rays, a panoramic image corresponding to a sphere with the three-dimensional coordinates as the center and a preset length as the radius is obtained, and the panoramic image is used as the lighting information image corresponding to the target position.

[0084] Optionally, when there are remaining sub-spheres in the sphere that are not assigned color information, the processor 701 further performs the following operations:

[0085] Color information of sub-spherical surfaces adjacent to the remaining sub-spherical surfaces on the spherical surface is interpolated to obtain color information of the remaining sub-spherical surfaces.

[0086] Optionally, the processor 701 renders a virtual object placed at the target location according to the illumination information map, specifically by performing the following operations:

[0087] The lighting information map is used as a skybox surrounding the virtual object, and the skybox is set to be invisible to avoid blocking the real picture, wherein the skybox contains lighting textures in six directions: up and down, left and right, and front and back;

[0088] Rendering a virtual object placed at the target position according to the lighting textures in all directions of the skybox.

[0089] Optionally, the processor 701 determines the three-dimensional coordinates of the target position of the virtual object in the three-dimensional space, specifically by:

[0090] Determining the two-dimensional coordinates of a target position for placing the virtual object on the display screen;

[0091] Starting from the two-dimensional coordinates, a ray is emitted in a direction perpendicular to the display screen, and the three-dimensional coordinates of the intersection of the ray and the virtual screen corresponding to the display screen in three-dimensional space are determined as the three-dimensional coordinates of the target position.

[0092] Embodiments of the present application Figure 7 The processor involved may be a central processing unit (CPU), a general-purpose processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute the various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like. The memory may be integrated into the processor or may be provided separately from the processor.

[0093] It should be noted that Figure 7 This is only an example, and provides the necessary hardware for a display device with AR function to execute the method steps for displaying virtual objects based on spatial position illumination provided in the embodiments of the present application. Not shown, the display device also includes common hardware of human-computer interaction devices, such as speakers, microphones, mice, keyboards, etc.

[0094] The embodiment of the present application also provides a computer-readable storage medium for storing some instructions, which, when executed, can complete the method of the aforementioned embodiment.

[0095] An embodiment of the present application further provides a computer program product for storing a computer program, wherein the computer program is used to execute the method of the aforementioned embodiment.

[0096] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0097] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0098] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0099] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0100] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A method for displaying virtual objects based on spatial position illumination, characterized in that: Applied to augmented reality (AR) scenarios, including: In response to the operation of placing a virtual object on a real picture, a three-dimensional point cloud dataset corresponding to the real picture is constructed using simultaneous localization and mapping (SLAM) technology; Determining the three-dimensional coordinates of the target position for placing the virtual object in the three-dimensional space; Acquire, from the three-dimensional point cloud dataset, a target point cloud dataset contained in a cuboid with the three-dimensional coordinates as the center and a preset length, width, and height as the sides; Determining a lighting information map corresponding to the target position according to the target point cloud data set; Rendering a virtual object placed at the target location according to the illumination information map, and superimposing and displaying the virtual object on the real image; Wherein, determining the illumination information map corresponding to the target position according to the target point cloud data set includes: Determine at least one surface included in the target point cloud data set to obtain a surface set, where the surfaces in the surface set are planes or curved surfaces; Taking the three-dimensional coordinate as the origin, evenly dividing a plurality of directional rays in the three-dimensional space; For each directional ray, if the directional ray intersects with the target point cloud data closest to the origin in the cuboid, the color information of the closest target point cloud data is used as the color information of the sub-sphere corresponding to the sphere center angle of the directional ray; or, if the directional ray intersects with a surface in the surface set, the color information of the target point cloud data within a preset range on the surface is interpolated with the color information of the intersecting point cloud data, and the interpolated color information is used as the color information of the sub-sphere corresponding to the sphere center angle of the directional ray; According to the color information of each sub-sphere determined by the multiple directional rays, a panoramic image corresponding to a sphere with the three-dimensional coordinates as the center and a preset length as the radius is obtained, and the panoramic image is used as the lighting information image corresponding to the target position.

2. The method according to claim 1, wherein When there are remaining sub-spheres in the sphere that are not assigned color information, the method further includes: Color information of sub-spherical surfaces adjacent to the remaining sub-spherical surfaces on the spherical surface is interpolated to obtain color information of the remaining sub-spherical surfaces.

3. The method according to claim 1, wherein The rendering of the virtual object placed at the target position according to the illumination information map includes: The lighting information map is used as a skybox surrounding the virtual object, and the skybox is set to be invisible to avoid blocking the real picture, wherein the skybox contains lighting textures in six directions: up and down, left and right, and front and back; Rendering a virtual object placed at the target position according to the lighting textures in all directions of the skybox.

4. The method according to claim 1, wherein Determining the three-dimensional coordinates of the target position for placing the virtual object in the three-dimensional space includes: Determining the two-dimensional coordinates of the target position for placing the virtual object on the display screen; Starting from the two-dimensional coordinates, a ray is emitted in a direction perpendicular to the display screen, and the three-dimensional coordinates of the intersection of the ray and the virtual screen corresponding to the display screen in three-dimensional space are determined as the three-dimensional coordinates of the target position.

5. A display device, characterized in that: Supporting augmented reality (AR) function, including a processor, a memory, a camera, and a display screen, wherein the display screen, the camera, the memory, and the processor are connected via a bus; The camera is used to capture real images; The memory stores a computer program, and the processor performs the following operations according to the computer program: In response to an operation of placing a virtual object on a real image displayed on the display screen, a three-dimensional point cloud dataset corresponding to the real image is constructed using a simultaneous localization and mapping (SLAM) technology; Determining the three-dimensional coordinates of the target position for placing the virtual object in the three-dimensional space; Acquire, from the three-dimensional point cloud dataset, a target point cloud dataset contained in a cuboid with the three-dimensional coordinates as the center and a preset length, width, and height as the sides; Determining a lighting information map corresponding to the target position according to the target point cloud data set; Rendering a virtual object placed at the target location according to the illumination information map, and displaying the virtual object superimposed on the real image through the display screen; The processor determines the illumination information map corresponding to the target position according to the target point cloud data set, and the specific operation is: Determine at least one surface included in the target point cloud data set to obtain a surface set, where the surfaces in the surface set are planes or curved surfaces; Taking the three-dimensional coordinate as the origin, evenly dividing a plurality of directional rays in the three-dimensional space; For each directional ray, if the directional ray intersects with the target point cloud data closest to the origin in the cuboid, the color information of the closest target point cloud data is used as the color information of the sub-sphere corresponding to the sphere center angle of the directional ray; or, if the directional ray intersects with a surface in the surface set, the color information of the target point cloud data within a preset range on the surface is interpolated with the color information of the intersecting point cloud data, and the interpolated color information is used as the color information of the sub-sphere corresponding to the sphere center angle of the directional ray; According to the color information of each sub-sphere determined by the multiple directional rays, a panoramic image corresponding to a sphere with the three-dimensional coordinates as the center and a preset length as the radius is obtained, and the panoramic image is used as the lighting information image corresponding to the target position.

6. The display device according to claim 5, wherein When there are remaining sub-spheres in the sphere that are not assigned color information, the processor further performs the following operations: Color information of sub-spherical surfaces adjacent to the remaining sub-spherical surfaces on the spherical surface is interpolated to obtain color information of the remaining sub-spherical surfaces.

7. The display device according to claim 5, wherein The processor renders a virtual object placed at the target position according to the illumination information map, specifically by: The lighting information map is used as a skybox surrounding the virtual object, and the skybox is set to be invisible to avoid blocking the real picture, wherein the skybox contains lighting textures in six directions: up and down, left and right, and front and back; Rendering a virtual object placed at the target position according to the lighting textures in all directions of the skybox.

8. The display device according to claim 5, wherein The processor determines the three-dimensional coordinates of the target position of the virtual object in the three-dimensional space, specifically by: Determining the two-dimensional coordinates of a target position for placing the virtual object on the display screen; Starting from the two-dimensional coordinates, a ray is emitted in a direction perpendicular to the display screen, and the three-dimensional coordinates of the intersection of the ray and the virtual screen corresponding to the display screen in three-dimensional space are determined as the three-dimensional coordinates of the target position.

Images