Scene display method and device, virtual reality terminal and storage medium

Abstract

The invention provides a scene display method and device, a virtual reality terminal and a storage medium, and the method comprises the steps: obtaining a target virtual reality scene and a to-be-fused object corresponding to the target virtual reality scene, and the to-be-fused object is an object needing to be displayed in a real scene; performing superposition fusion on the target virtual reality scene and the to-be-fused object to obtain a virtual-real fusion scene corresponding to the target virtual reality scene; and displaying the virtual-real fusion scene in the virtual reality terminal.

Description

Technical Field

[0001] The present application relates to the field of scene display, and in particular to a scene display method, device, virtual reality terminal and storage medium. Background Art

[0002] With the development of science and technology, teaching methods have become diversified. In addition to traditional offline teaching using multimedia screens in the classroom, teachers and students can also take online courses through the Internet and use virtual reality (VR) equipment for virtual reality teaching.

[0003] Among several teaching methods, offline teaching and online teaching use two-dimensional screens to complete teaching, that is, the content related to teaching is displayed through the two-dimensional screen, which belongs to teaching from a two-dimensional perspective; virtual reality teaching uses virtual reality equipment to complete teaching, that is, the content related to teaching is displayed in the virtual reality equipment in a three-dimensional way, bringing a three-dimensional immersive feeling. At present, two-dimensional screen teaching and virtual reality teaching are two independent teaching methods that need to be implemented separately. The required operation steps are cumbersome and complicated, and the user experience is not good. Summary of the invention

[0004] The present application provides a scene display method, device, virtual reality terminal and storage medium to solve the technical problem of cumbersome and complicated operation steps required for simultaneously realizing two-dimensional screen teaching and virtual reality teaching.

[0005] In a first aspect, a scene display method is provided, which is applied to a virtual reality terminal, comprising:

[0006] Acquire a target virtual reality scene and an object to be fused corresponding to the target virtual reality scene, wherein the object to be fused is an object to be displayed in the real scene;

[0007] Superimposing and fusing the target virtual reality scene with the object to be fused to obtain a virtual-reality fusion scene corresponding to the target virtual reality scene;

[0008] The virtual-reality fusion scene is displayed in the virtual reality terminal.

[0009] In this technical solution, by obtaining the target virtual reality scene and the object to be fused corresponding to the target virtual reality scene, the object to be fused is the object to be displayed in the real scene; then the target virtual reality scene and the object to be fused are superimposed and fused to obtain the virtual-reality fusion scene corresponding to the target virtual reality scene; and the virtual-reality fusion scene is displayed in the virtual reality terminal, realizing the combined display of the real scene and the virtual scene. When the object to be fused includes a two-dimensional screen, the two-dimensional screen is superimposed and displayed with the target virtual reality scene, and the user only needs to wear the virtual reality terminal to implement the two-dimensional screen teaching and virtual reality teaching at the same time, eliminating the steps of taking off and wearing the virtual reality terminal, which can save operation time.

[0010] In combination with the first aspect, in a possible implementation, the object to be fused includes a two-dimensional screen in a real scene; the superimposing and fusing the target virtual reality scene with the object to be fused to obtain a virtual-reality fusion scene corresponding to the target virtual reality scene includes: obtaining a two-dimensional screen picture stream corresponding to the two-dimensional screen, the two-dimensional screen picture stream being obtained based on capturing and encoding the picture in the two-dimensional screen; decoding the two-dimensional screen picture stream, and superimposing and rendering the decoded two-dimensional screen picture stream in the target virtual reality scene to obtain the virtual-reality fusion scene. By obtaining the two-dimensional screen picture stream in the real scene and rendering and outputting the two-dimensional screen picture stream, the two-dimensional screen picture is put into the virtual reality terminal for display in the form of video streaming, which can ensure the clarity of the two-dimensional screen picture displayed in the virtual reality terminal.

[0011] In combination with the first aspect, in a possible implementation, a first object surface is reserved in the target virtual reality scene, and the first object surface is used to carry a screen image; the two-dimensional screen image stream obtained by fusion and superposition rendering in the target virtual reality scene to obtain the virtual-reality fusion scene includes: in the target virtual reality scene, the decoded two-dimensional screen image stream is set as the texture map of the first object surface, and the target virtual reality scene is rendered to obtain the virtual-reality fusion scene. By setting the decoded two-dimensional screen image stream as the texture map of the object surface reserved in the virtual three-dimensional scene, the two-dimensional screen image can be placed in the virtual three-dimensional scene for display, thereby improving the display effect of the two-dimensional screen image in the virtual reality terminal.

[0012] In combination with the first aspect, in a possible implementation, the decoding and superimposing of the two-dimensional screen image stream in the target virtual reality scene to obtain the virtual-reality fusion scene includes: obtaining a first virtual space position of the target user corresponding to the virtual reality terminal in the virtual space corresponding to the target virtual reality scene; creating a second object surface at a second virtual space position in the target virtual reality scene, setting the decoded two-dimensional screen image stream as a texture map of the second object surface, rendering the target virtual reality scene, and obtaining the virtual-reality fusion scene, wherein the distance between the second virtual space position and the first virtual space position in the virtual space corresponding to the target virtual reality scene is less than a preset distance threshold. By creating an object surface near the participant and setting the decoded two-dimensional screen image stream as a texture map of the object surface and rendering it, the two-dimensional screen image can be placed in the virtual three-dimensional scene for display, thereby improving the display effect of the two-dimensional screen image in the virtual reality terminal.

[0013] In conjunction with the first aspect, in a possible implementation, the scene theme corresponding to the target virtual reality scene matches the picture theme corresponding to the decoded two-dimensional screen picture stream. The scene theme of the virtual three-dimensional scene displayed in the virtual reality terminal matches the picture theme corresponding to the two-dimensional screen picture stream, which can provide an immersive experience to the user when displaying the two-dimensional screen picture and deepen the user's impression of the two-dimensional screen picture.

[0014] In combination with the first aspect, in a possible implementation, the object to be fused includes a real environment in a real scene; the superimposing and fusing the target virtual reality scene with the object to be fused to obtain a virtual-reality fusion scene corresponding to the target virtual reality scene includes: obtaining a real environment picture corresponding to the real environment; superimposing and rendering the target virtual reality scene in the real environment picture to obtain the virtual-reality fusion scene. By fusing the target virtual reality scene with the real environment picture to obtain the virtual-reality fusion scene, the information of the target virtual reality scene can be enriched, creating a stronger immersive atmosphere for the user.

[0015] In combination with the first aspect, in a possible implementation, before the target virtual reality scene is superimposed and rendered in the real environment picture to obtain the virtual-reality fusion scene, the method further includes: in the target virtual reality scene, three-dimensionally reconstructing the real objects in the real environment picture to obtain the three-dimensionally reconstructed virtual objects corresponding to the real objects, and setting the material of the three-dimensionally reconstructed virtual objects to occlusion culling. By three-dimensionally reconstructing the real objects in the real environment picture and setting the material of the reconstructed three-dimensionally reconstructed virtual objects to occlusion culling, the real objects in the real environment picture can visually occlude the virtual objects in the target virtual reality scene, thereby enhancing the sense of immersion.

[0016] In combination with the first aspect, in a possible implementation, after the real object in the real environment screen is three-dimensionally reconstructed in the target virtual reality scene to obtain the three-dimensional reconstructed virtual object corresponding to the real object, the method further includes: searching for a virtual object plane corresponding to the three-dimensional reconstructed virtual object in the target virtual reality scene; and setting part or all of the original virtual objects in the target virtual reality scene on the virtual object plane. By searching for a plane in the reconstructed three-dimensional reconstructed virtual object and setting the original virtual object in the target virtual reality scene on the found plane, it is possible to realize dynamic positioning display of objects in the virtual three-dimensional scene, thereby improving the realism of the virtual-reality fusion scene.

[0017] In combination with the first aspect, in a possible implementation manner, part or all of the original virtual objects are interactive virtual objects in the target virtual reality scene.

[0018] In combination with the first aspect, in a possible implementation, the object to be fused includes a two-dimensional screen and a real environment in a real scene; the superimposing and fusing the target virtual reality scene with the object to be fused to obtain a virtual-reality fusion scene corresponding to the target virtual reality scene includes: obtaining a two-dimensional screen image stream corresponding to the two-dimensional screen and a real environment image corresponding to the real environment, wherein the two-dimensional screen image stream is obtained based on image capture and encoding of the image in the two-dimensional screen; decoding the two-dimensional screen image stream, superimposing and rendering the decoded two-dimensional screen image stream in the target virtual reality scene to obtain an initial fusion scene; and superimposing and rendering the initial fusion scene in the real environment image to obtain the virtual-reality fusion scene. By fusing the two-dimensional screen, the real environment image and the target virtual reality scene, the information of the target virtual reality scene can be enriched, creating a stronger immersive atmosphere for the user.

[0019] In combination with the first aspect, in a possible implementation manner, the position and orientation of the two-dimensional screen in the virtual-reality fusion scene are obtained based on the position and orientation of an object in the real environment picture.

[0020] In a second aspect, a scene display device is provided, which is applied to a virtual reality terminal, comprising:

[0021] An acquisition module, used to acquire a target virtual reality scene and an object to be fused corresponding to the target virtual reality scene, wherein the object to be fused is an object to be displayed in the real scene;

[0022] A fusion module, used to superimpose and fuse the target virtual reality scene with the object to be fused to obtain a virtual-reality fusion scene corresponding to the target virtual reality scene;

[0023] A display module is used to display the virtual-reality fusion scene in the virtual reality terminal.

[0024] In a third aspect, a virtual reality terminal is provided, comprising a memory, a display panel, and one or more processors, wherein the memory and the display panel are connected to the one or more processors, and the one or more processors are used to execute one or more computer programs stored in the memory. When the one or more processors execute the one or more computer programs, the virtual reality terminal implements the scene display method of the first aspect.

[0025] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the scene display method of the first aspect.

[0026] The present application can achieve the following technical effects: it realizes the combined display of real scenes and virtual scenes. When the object to be fused includes a two-dimensional screen, the two-dimensional screen and the target virtual reality scene are superimposed and displayed. The user only needs to wear a virtual reality terminal to implement two-dimensional screen teaching and virtual reality teaching at the same time, eliminating the steps of taking off and wearing the virtual reality terminal, which can save operation time. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A schematic diagram of a network architecture of a virtual reality network provided in an embodiment of the present application;

[0028] Figure 2 A schematic diagram of a flow chart of a scene display method provided in an embodiment of the present application;

[0029] Figure 3 A schematic diagram of a virtual reality scene provided in an embodiment of the present application;

[0030] Figure 4 A schematic diagram of a virtual fusion scene provided in an embodiment of the present application;

[0031] Figure 5 is a structural schematic diagram of a scene display device provided in an embodiment of the present application;

[0032] Figure 6 It is a structural schematic diagram of a virtual reality terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0034] The technical solution of the present application is applicable to teaching scenarios, and in particular to teaching scenarios where virtual reality teaching needs to be completed using a virtual reality terminal.

[0035] The technical solution of the present application can be applied to a virtual reality network composed of multiple virtual reality terminals, which are used by users, and one virtual reality terminal corresponds to one user. The number of virtual reality terminals depends on the number of users in the teaching scene, and the users in the teaching scene can refer to students or teachers.

[0036] For ease of understanding, the virtual reality network involved in this application is first introduced. Figure 1 , Figure 1 A schematic diagram of a network architecture of a virtual reality network provided in an embodiment of the present application. Figure 1 As shown, the virtual reality network includes multiple virtual reality terminals 101. The virtual reality terminal 101 is a terminal with a virtual reality function, that is, the virtual reality terminal 101 has the ability to render and display a target virtual reality scene. Specifically, the virtual reality terminal 101 can be VR glasses, VR helmets, and the like.

[0037] The virtual reality terminal 101 may also have a camera, which is used to shoot the environment outside the virtual reality terminal 101, obtain the external environment picture, and present the external environment picture in the virtual reality terminal 101 to restore the external environment seen by the left and right eyes of the naked eye. Specifically, the number of cameras in the virtual reality terminal 101 can be two, the distance between the two cameras can be 65 mm, and the focal length of the two cameras can be 50 mm, so that the pictures shot and presented by the cameras meet the viewing requirements of the human eye. Further, the two cameras can also be full-color cameras with high resolution, so that the pictures shot and presented by the cameras can be clear enough and the color restoration of the external environment picture can be achieved. Among them, when the number, distance, focal length and resolution of the cameras meet the requirements of the above introduction, the virtual reality terminal 101 shooting and presenting the external environment picture can be called "full-color perspective". Further, the distance between the two cameras in the virtual reality terminal 101 can also change with the change of the imaging wheelbase of the virtual reality terminal, so that the perspective imaging effect can be improved.

[0038] The multiple virtual reality terminals 101 can be divided into teacher terminals and student terminals according to the identities of users in the teaching scene.

[0039] The virtual reality network may further include a target display device 102, wherein the target display device 102 is used to display images on a two-dimensional screen. The target display device 102 may specifically be a display screen, a display curtain, and the like.

[0040] The virtual reality network may further include a network device 103, which may be connected to the virtual reality terminal 101 and the target display device 102 respectively, and is used to establish a connection network between the virtual reality terminal 101 and the target display device 102. The network device 103 may include a router; optionally, the network device 103 may also include a server, which may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0041] based on Figure 1 The virtual reality network shown can implement the technical solution of the present application. The technical solution of the present application can be specifically applied to any virtual reality terminal in the virtual reality network.

[0042] The technical solution of this application is described in detail below.

[0043] See also Figure 2 , Figure 2 A schematic diagram of a scene display method provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, the method comprises the following steps:

[0044] S201, obtaining a target virtual reality scene and an object to be fused corresponding to the target virtual reality scene.

[0045] The target virtual reality scene is the virtual reality scene to be displayed currently, and the virtual reality scene is a virtual three-dimensional scene created based on virtual reality technology. The object to be fused corresponding to the target virtual reality scene refers to the object to be displayed currently in the real scene, and the real scene can be a teaching scene, and the object to be fused can be a two-dimensional screen in the teaching scene, or a real environment in the teaching scene, or a two-dimensional screen and a real environment in the teaching scene.

[0046] In a possible case, the target virtual reality scene and the object to be fused corresponding to the target virtual reality scene may be set by a user.

[0047] Specifically, before using the virtual reality terminal, the user can set the virtual reality terminal. The user can set the target virtual reality scene and the object to be fused corresponding to the target virtual reality scene that the virtual reality terminal needs to display during course teaching, and the virtual reality terminal can save the user settings. When the user uses the virtual reality terminal, the virtual reality terminal can obtain the target virtual reality scene and the object to be fused corresponding to the target virtual reality scene according to the saved user settings.

[0048] Optionally, the target virtual reality scene and the object to be fused corresponding to the target virtual reality scene may also be automatically set by the virtual reality terminal according to specific circumstances.

[0049] In a feasible implementation, the virtual reality terminal may determine the target virtual reality scene and the object to be fused corresponding to the target virtual reality scene according to the current course node or course link.

[0050] In a specific implementation, the virtual reality terminal can obtain the correspondence between the preset course nodes and the virtual reality scenes, and the correspondence between the course nodes and the virtual reality scenes can be preset by the user and saved in the virtual reality network; the virtual reality terminal can determine the target virtual reality scene corresponding to the currently triggered target course node according to the correspondence between the course nodes and the virtual reality scenes; and determine the object to be fused corresponding to the target virtual reality scene according to the target course node. If the target course node is a course node related to two-dimensional screen teaching, the virtual reality terminal determines that the object to be fused includes the two-dimensional screen; if the target course node is other course nodes unrelated to two-dimensional screen teaching, the virtual reality terminal determines that the object to be fused includes the real environment.

[0051] In another feasible implementation, the virtual reality terminal may also determine the target virtual reality scene and the object to be fused corresponding to the target virtual reality scene according to the environment picture outside the virtual reality terminal.

[0052] In a specific implementation manner, the virtual reality terminal can capture the external environment picture through a camera and determine whether there is a two-dimensional screen in the external environment picture. If there is a two-dimensional screen, it is determined that the object to be fused corresponding to the target virtual reality scene includes the two-dimensional screen; if there is no two-dimensional screen, it is determined that the object to be fused corresponding to the target virtual reality scene includes the real environment.

[0053] In another feasible implementation, the virtual reality terminal may also determine the target virtual reality scene and the object to be fused corresponding to the target virtual reality scene in combination with the current course node and the environment outside the virtual reality terminal.

[0054] In a specific implementation manner, the virtual reality terminal can determine the virtual reality scene corresponding to the currently triggered target course node based on the correspondence between the preset course nodes and the virtual reality scenes; then obtain the external environment picture, and determine whether there is a two-dimensional screen in the external environment picture. If there is a two-dimensional screen, it is determined that the object to be fused corresponding to the target virtual reality scene includes the two-dimensional screen; if there is no two-dimensional screen, it is determined that the object to be fused corresponding to the target virtual reality scene includes the real environment.

[0055] Further optionally, the target virtual reality scene and the object to be fused corresponding to the target virtual reality scene may be set in combination with user settings and specific circumstances.

[0056] In a specific implementation, the virtual reality terminal can determine the virtual reality scene corresponding to the currently triggered target course node according to the correspondence between the preset course node and the virtual reality scene; the virtual reality terminal can determine the object to be fused according to the user setting. Alternatively, the virtual reality terminal can also determine the target virtual reality scene according to the user setting, and determine the object to be fused corresponding to the target virtual reality scene according to the current course node.

[0057] Not limited to the above implementation, for example, the target virtual reality scene and the object to be fused corresponding to the target virtual reality scene may be determined according to the geographical location of the virtual reality terminal; or, the object to be fused may be determined based on the above-mentioned situation, and then the target virtual reality scene may be determined according to the object to be fused. This application does not make any limitation.

[0058] S202, superimposing and fusing the target virtual reality scene with the object to be fused, to obtain a virtual-reality fusion scene corresponding to the target virtual reality scene.

[0059] Here, superimposing and fusing the target virtual reality scene with the object to be fused means fusing the target virtual reality scene and the object to be fused together through image processing technology, so that the target virtual reality scene and the object to be fused coexist in the same image.

[0060] Among them, superimposing and fusing the target virtual reality scene with the object to be fused specifically includes the following three situations:

[0061] Case 1: Overlay and merge the target virtual reality scene with the two-dimensional screen in the real environment;

[0062] Case 2: Superimpose and merge the target virtual reality scene with the real environment;

[0063] Case 3: Superimpose and integrate the target virtual reality scene, real environment and two-dimensional screen.

[0064] For the specific superposition and fusion methods involved in the above three situations, please refer to the subsequent description, which will not be described in detail here.

[0065] S203, displaying the virtual-reality fusion scene in the virtual reality terminal.

[0066] Specifically, the virtual reality terminal can output the three-dimensional scene image obtained by rendering the virtual-reality fusion scene to the display panel of the virtual reality terminal for display, thereby realizing the display of the virtual-reality fusion scene.

[0067] exist Figure 2In the corresponding technical solution, the target virtual reality scene and the object to be fused corresponding to the target virtual reality scene are obtained, and the object to be fused is the object to be displayed in the real scene; then the target virtual reality scene and the object to be fused are superimposed and fused to obtain the virtual-reality fusion scene corresponding to the target virtual reality scene; and the virtual-reality fusion scene is displayed in the virtual reality terminal, thereby realizing the combined display of the real scene and the virtual scene. When the object to be fused includes a two-dimensional screen, the two-dimensional screen is superimposed and displayed with the target virtual reality scene, and the user only needs to wear the virtual reality terminal to implement the two-dimensional screen teaching and virtual reality teaching at the same time, eliminating the steps of taking off and wearing the virtual reality terminal, which can save operation time.

[0068] The following introduces the specific situation of superimposing and fusing the target virtual reality scene with the object to be fused.

[0069] Case 1: The object to be fused is the two-dimensional screen in the teaching scene. The target virtual reality scene is superimposed and fused with the two-dimensional screen in the real environment.

[0070] The above step S202 specifically includes the following steps A1-A2.

[0071] A1. Obtain a two-dimensional screen image stream corresponding to the two-dimensional screen.

[0072] The two-dimensional screen picture stream is obtained based on capturing and encoding the pictures in the two-dimensional screen.

[0073] The two-dimensional screen image stream can be obtained by capturing and encoding the two-dimensional screen image displayed in the target display device by the target display device in the virtual reality network. The target display device can capture the two-dimensional screen image displayed in the target display device at a preset capture frequency based on full-screen capture or window capture, and then any encoding method among encoding methods such as H264, H265, VP8 and VP9 can be used to encode the captured two-dimensional screen image in sequence to form a two-dimensional screen image stream, which is a data stream to be transmitted in the virtual reality network, and then any video streaming protocol among HTTP live streaming (HLS) protocol, HTTP dynamic adaptive streaming over HTTP (DASH) protocol, real time messaging protocol (RTMP) protocol, web real time communication (WebRTC) protocol, and HTTP smooth stream protocol is used to establish a data stream transmission channel with the virtual reality terminal, and finally the two-dimensional screen image stream is transmitted to the virtual reality terminal through the data stream transmission channel, so that the virtual reality terminal obtains the two-dimensional screen image stream corresponding to the two-dimensional screen.

[0074] A2. Decode the two-dimensional screen picture stream corresponding to the two-dimensional screen.

[0075] Specifically, the virtual reality terminal decodes the acquired two-dimensional screen image stream using a decoding method corresponding to the encoding method used by the target display device.

[0076] A3. Overlay the two-dimensional screen image stream obtained by rendering and decoding in the target virtual reality scene to obtain a virtual-reality fusion scene.

[0077] In some possible cases, a first object surface is reserved in the target virtual reality scene, and the first object surface is used to carry the video image. Figure 3 , Figure 3 The virtual reality scene is shown in FIG. A "bulletin board" object w1 is reserved in the virtual three-dimensional scene. The "bulletin board" object w1 is used to carry the video screen, that is, to display the video screen.

[0078] In this case, the decoded two-dimensional screen image stream can be set as a texture map of the surface of the first object in the target virtual reality scene, and the target virtual reality scene can be rendered to obtain a virtual-reality fusion scene.

[0079] Specifically, the virtual reality terminal can set the material of the surface of the first object, add a video playback component to the surface of the first object, then set the uniform resource locator (URL) attribute of the video playback component to the URL of the decoded two-dimensional screen image stream, and play the video, so as to set the decoded two-dimensional screen image stream as the texture map of the surface of the first object, and then render the target virtual reality scene, so as to obtain a virtual-real fusion scene. Among them, the material of the surface of the first object can be set to "texture", so that the rendering of the two-dimensional screen image stream does not change with the light and shadow of the objects in the first virtual three-dimensional scene, so that the two-dimensional screen image stream is close to the two-dimensional screen image viewed by the naked eye. The material of the surface of the first object can also be set to a standard principled material, so that the rendering of the two-dimensional screen image stream changes with the light and shadow of the objects in the first virtual three-dimensional scene, so that the two-dimensional screen image stream is integrated with the objects in the first virtual three-dimensional scene, creating a stronger experience for the user.

[0080] By setting the decoded two-dimensional screen image stream as the texture map of the object surface reserved in the virtual three-dimensional scene, the two-dimensional screen image can be placed in the virtual three-dimensional scene for display, thereby improving the display effect of the two-dimensional screen image in the virtual reality terminal.

[0081] In other possible situations, no object surface is reserved in the target virtual reality scene for carrying the video image.

[0082] In this case, the first virtual space position of the target user corresponding to the virtual reality terminal in the virtual space corresponding to the target virtual reality scene can be obtained; a second object surface is created at the second virtual space position in the target virtual reality scene, and the decoded two-dimensional screen image stream is set as the texture map of the second object surface, and the target virtual reality scene is rendered to obtain a virtual-real fusion scene. Among them, the distance between the second virtual space position and the first virtual space position in the virtual space corresponding to the target virtual reality scene is less than the preset distance threshold, that is, the second empty space is near the first virtual space position, that is, the second object surface is created around the target user in the virtual space, so that the two-dimensional screen image corresponding to the two-dimensional screen image stream appears around the target user in the virtual reality terminal. The target user corresponding to the virtual reality terminal refers to the user who uses the virtual reality terminal. The first virtual space position of the target user corresponding to the virtual reality terminal in the virtual space corresponding to the target virtual reality scene can be obtained by the sensor in the virtual reality terminal.

[0083] The principle of setting the decoded two-dimensional screen image stream as the texture map of the second object surface is the same as the principle of setting the decoded two-dimensional screen image stream as the texture map of the first object surface, and will not be repeated here.

[0084] By creating an object surface around the participant and setting the decoded two-dimensional screen image stream as a texture map of the object surface and rendering it, the two-dimensional screen image can be placed in a virtual three-dimensional scene for display, thereby improving the display effect of the two-dimensional screen image in the virtual reality terminal.

[0085] In some possible cases, the scene theme corresponding to the target virtual reality scene may also match the picture theme corresponding to the decoded two-dimensional screen picture stream.

[0086] Specifically, a virtual reality scene library may be preset in the virtual reality network, and the virtual reality scene library stores a variety of virtual reality scenes and scene themes corresponding to the various virtual reality scenes. After decoding and obtaining the two-dimensional screen image stream, the virtual reality terminal may obtain the image theme of the decoded two-dimensional screen image stream, and obtain a virtual reality scene whose scene theme matches the image theme from the virtual reality scene library as the target virtual reality scene. The image theme of the two-dimensional screen image stream may be obtained by the virtual reality terminal by identifying the content of the decoded two-dimensional screen image, or may be pre-marked by the producer of the two-dimensional screen image content and transmitted to the virtual reality terminal together with the two-dimensional screen image stream, and the virtual reality terminal may obtain the image theme pre-marked by the producer while decoding and obtaining the two-dimensional screen image stream.

[0087] For example, assuming that the content in the two-dimensional screen image stream is screen content related to the "little painter in the snow", the image theme of the two-dimensional screen image stream can be snow, and a virtual snow three-dimensional scene can be obtained from the virtual reality scene library as the target virtual reality scene.

[0088] Case 2: The object to be integrated is the real environment in the classroom teaching scene, and the target virtual reality scene is superimposed and integrated with the real environment.

[0089] The above step S202 specifically includes the following steps B1-B2.

[0090] B1. Obtain a real environment picture corresponding to the real environment.

[0091] Specifically, the camera of the virtual reality terminal can obtain the picture of the external environment of the virtual reality terminal to obtain the real environment picture.

[0092] Among them, when the camera of the virtual reality terminal meets the hardware requirements of the full-color perspective requirements of the virtual reality terminal 101, the real environment picture obtained by the virtual reality terminal is a full-color perspective picture; when the camera of the virtual reality terminal does not meet the aforementioned hardware requirements, the real environment picture obtained by the virtual reality terminal is an ordinary perspective picture.

[0093] B2. Overlay the rendered target virtual reality scene on the real environment image to obtain a virtual-reality fusion scene.

[0094] Specifically, the real environment picture can be used as the image bottom layer, and the target virtual reality scene can be superimposed on the real environment picture by using Alpha overlay, so as to overlay and render the target virtual reality scene in the real environment picture.

[0095] By integrating the target virtual reality scene with the real environment picture, a virtual-reality fusion scene is obtained, which can enrich the information of the target virtual reality scene and create a stronger immersive atmosphere for users.

[0096] In some possible situations, before the target virtual reality scene is rendered in the real environment picture and the virtual-reality fusion scene is obtained, the real objects in the real environment picture can be three-dimensionally reconstructed in the target virtual reality scene to obtain the three-dimensional reconstructed virtual objects corresponding to the real objects, and the material of the three-dimensional reconstructed virtual objects can be set to occlusion culling.

[0097] Specifically, the virtual reality terminal can perform three-dimensional reconstruction of real objects in the real environment screen through any one of the three-dimensional reconstruction methods, such as the three-dimensional reconstruction method based on visual geometry (such as the structure from motion (SFM) method) and the three-dimensional reconstruction method based on deep learning (such as stereo vision (multi view stereo, MVS)).

[0098] By performing three-dimensional reconstruction of real objects in the real environment picture and setting the material of the reconstructed three-dimensional reconstructed virtual object to occlusion culling, the real objects in the real environment picture can visually occlude the virtual objects in the target virtual reality scene, thereby enhancing the sense of immersion.

[0099] Furthermore, after obtaining the three-dimensional reconstructed virtual object corresponding to the real object through three-dimensional reconstruction, the virtual object plane corresponding to the three-dimensional reconstructed virtual object can be searched in the target virtual reality scene; and part or all of the original virtual objects in the target virtual reality scene can be set on the virtual object plane.

[0100] The original virtual object refers to a three-dimensional object that originally exists in the virtual reality scene, that is, a three-dimensional object that is not a three-dimensional object reconstructed from the current real environment image, that is, an object in a non-real environment. The virtual object plane used to place the original virtual object is usually a horizontal plane.

[0101] Specifically, the virtual object plane corresponding to the three-dimensional reconstructed virtual object can be found in the target virtual reality scene through the following steps (1)-(4):

[0102] (1) Obtain multiple three-dimensional point cloud data corresponding to the three-dimensional reconstructed virtual object.

[0103] Among them, three-dimensional point cloud data refers to data used to reflect the spatial position of three-dimensional reconstructed virtual objects in the virtual space corresponding to the target virtual reality scene. The three-dimensional point cloud data can be represented as (x, y, z), where x and y represent the position of the three-dimensional reconstructed virtual object on the spatial plane, hereinafter referred to as plane data, and z represents the position of the three-dimensional reconstructed object at a spatial height, hereinafter referred to as height data.

[0104] (2) According to the height data in each 3D point cloud data, the multiple 3D point cloud data are grouped to obtain multiple 3D point cloud data groups.

[0105] Each three-dimensional point cloud data group includes one or more three-dimensional point cloud data, and the height data difference between two three-dimensional point cloud data in the same three-dimensional point cloud group is less than a preset height threshold.

[0106] (3) For each 3D point cloud data group, determine the plane according to the following steps (31)-(33):

[0107] (31) Perform plane fitting on the three-dimensional point cloud data in the three-dimensional point cloud data group to obtain the plane equation corresponding to the three-dimensional point cloud group.

[0108] The plane equation is expressed as: A*x+B*y+C*z+D=0, and A, B, C, and D are coefficients of the plane equation, which are obtained by plane fitting.

[0109] (32) Substitute each 3D point cloud data in the 3D point cloud data group into the plane equation and solve the total error of the plane equation.

[0110] The total error calculation formula is:

[0111] n is the total number of 3D point cloud data in the 3D point cloud data group, (x i ,y i , z i ) is the i-th three-dimensional point cloud data in the three-dimensional point cloud data group, a0 = -A / C, a1 = -B / C, a2 = -D / C.

[0112] (33) If the total error is less than the preset error, the spatial distance between each 3D point cloud data in the 3D point cloud data group and the fitting plane corresponding to the plane equation is calculated, and the 3D point cloud data with a spatial distance less than a preset distance threshold is divided into the fitting plane in the 3D point cloud data group.

[0113] (4) Repeat the above steps (2)-(3) until each 3D point cloud data corresponding to the 3D reconstructed virtual object is divided into the fitting plane, or the number of fitting planes exceeds a preset number.

[0114] In the above steps (1)-(4), after obtaining multiple 3D point cloud data corresponding to the 3D reconstructed virtual object, the 3D point cloud data is first grouped according to the height data of the 3D point cloud data. After obtaining the 3D point cloud data grouping, the 3D point cloud data grouping is used to determine the plane of the 3D reconstructed virtual object, so that the found plane is a plane in the horizontal direction, which is convenient for the subsequent placement of the original object. In addition, after the 3D point cloud data is grouped according to the height data, the plane is determined. Determining the plane within the group can reduce the amount of calculation and increase the speed of finding the horizontal plane.

[0115] Furthermore, after finding the virtual object plane corresponding to the three-dimensional reconstructed virtual object, the original virtual object can be set on a virtual object plane whose height matches the object properties of the original virtual object according to the object properties of the original virtual object in the target virtual reality scene and the height of the virtual object plane corresponding to the three-dimensional reconstructed virtual object.

[0116] For example, if the original virtual object in the target virtual reality scene is a lamp, and the lamp should be placed on the ceiling, the virtual object plane corresponding to the ceiling can be determined from the virtual object plane corresponding to the three-dimensional reconstructed virtual object according to the height of the object plane corresponding to the three-dimensional reconstructed virtual object, and the lamp can be placed on the virtual object plane corresponding to the ceiling. For another example, if the original virtual object in the target virtual reality scene is a book, and the book should be placed on the desk, the virtual object plane corresponding to the desktop can be determined from the virtual object plane corresponding to the three-dimensional reconstructed virtual object according to the height of the object plane corresponding to the three-dimensional reconstructed virtual object, and the book can be placed on the virtual object plane corresponding to the desk.

[0117] By arranging the original virtual object on a virtual object plane whose height matches the object properties of the original virtual object, the original virtual object in the target virtual reality scene can be arranged reasonably.

[0118] Specifically, according to the spatial range of the found virtual object plane in the target virtual reality scene, part or all of the original virtual objects in the target virtual reality scene can be moved into the spatial range, thereby setting part or all of the original virtual objects in the target virtual reality scene on the virtual object plane.

[0119] In some possible cases, the original virtual object set on the virtual object plane may be an interactive virtual object in the target virtual reality scene, that is, the interactive virtual object in the target virtual reality scene is set on the virtual object plane corresponding to the three-dimensional reconstructed virtual object.

[0120] Among them, the interactive virtual object refers to a three-dimensional virtual object that can interact with the user, that is, an object whose physical state can change with the user's operation. For example, it can be a three-dimensional virtual object that can move (such as moving, bouncing, rotating, etc.) in the target virtual reality scene, a three-dimensional virtual object whose shape and size can change (such as becoming larger, smaller, rounder, squarer, etc.), a three-dimensional virtual object whose color can change (such as becoming darker, lighter, etc.), etc., and is not limited to the description here.

[0121] When the interactive virtual object in the target virtual reality scene is set on the virtual object plane corresponding to the three-dimensional reconstructed virtual object, the object collision range of the interactive virtual object can also be set according to the virtual object plane corresponding to the three-dimensional reconstructed virtual object, so as to simulate the physical collision effect of the object in the target virtual reality scene.

[0122] Case 3: The objects to be integrated include the two-dimensional screen and the real environment in classroom teaching. The target virtual reality scene is superimposed and integrated with the objects to be integrated.

[0123] The above step S202 specifically includes the following steps C1-C3.

[0124] C1. Obtain a two-dimensional screen image stream corresponding to the two-dimensional screen and a real environment image corresponding to the real environment.

[0125] Here, regarding the specific implementation method of obtaining the two-dimensional screen image stream and the real environment image, please refer to the description of the aforementioned steps A1 and B1, which will not be repeated here.

[0126] C2. Decode the two-dimensional screen image stream, and overlay and render the decoded two-dimensional screen image stream in the target virtual reality scene to obtain an initial fusion scene.

[0127] Here, regarding the specific implementation method of step C2, please refer to the description of the aforementioned step A2, which will not be repeated here.

[0128] C3. Overlay rendering of the initial fusion scene in the real environment image to obtain a virtual-real fusion scene.

[0129] Here, regarding the specific implementation method of step C3, please refer to the relevant description of the aforementioned step A3, which will not be repeated here.

[0130] Among them, in the process of superimposing the initial fusion scene on the real environment picture, the real objects in the real environment picture can also be three-dimensionally reconstructed and the positions of the original virtual objects in the target virtual scene can be set by referring to the method described in the aforementioned situation 2, which will not be repeated here.

[0131] In some possible cases, in the process of decoding the two-dimensional screen image stream and overlaying and rendering the decoded two-dimensional screen image stream in the target virtual reality scene to obtain an initial fusion scene, the position and orientation of the two-dimensional screen image corresponding to the two-dimensional screen image stream in the initial fusion scene can also be determined according to the position and orientation of objects in the real environment picture, that is, the position and orientation of the two-dimensional screen image corresponding to the two-dimensional screen image stream in the virtual-reality fusion scene are obtained based on the position and orientation of objects in the real environment picture.

[0132] For an example, see Figure 4 , Figure 4 A schematic diagram of a virtual fusion scene provided in an embodiment of the present application, Figure 4 Since the chairs are facing multiple directions around the round table, the two-dimensional screen image is set as the texture map of the four surfaces of the cube L1 in the virtual reality scene, facing four directions in the virtual-reality fusion scene.

[0133] The position and orientation of the two-dimensional screen in the virtual-reality fusion scene are determined based on the position and orientation of objects in the real environment, so that the displayed virtual-reality fusion scene meets user needs and improves user experience.

[0134] The above introduces the method of the present application. To facilitate understanding of the technical solution of the present application, the application of the technical solution of the present application is introduced below through specific scenario examples.

[0135] Scenario example: During the teaching process, two-dimensional screen teaching and virtual reality teaching are overlapped to complete the entire course teaching.

[0136] This scenario involves the following steps:

[0137] 1-1. The teacher and students meet in the classroom and start the lesson.

[0138] In this stage, teachers and students can wear virtual reality terminals, which can superimpose and fuse the virtual reality scene with the real environment in the manner described in the second situation above to obtain a virtual fusion scene.

[0139] 1-2. The teacher introduces the knowledge points of this class and subsequent immersive content through the two-dimensional screen.

[0140] In this step, the virtual reality terminal superimposes and fuses the virtual reality scene with the two-dimensional screen in the manner described in the first case above to obtain a virtual fusion scene.

[0141] 1-3. Teachers and students conduct immersive experience of virtual reality content related to this class.

[0142] Immersive virtual reality content is related to specific courses. Immersive experience virtual reality content can be experimental content, such as conducting experiments in physics, chemistry, and biology courses; it can also be practical content, such as conducting practical operations in physical education, firefighting, and other courses; it can also be scene simulation, such as simulating dialogue scenes in English classes, simulating scenes involved in Chinese, geography, and history classes, etc.; it can also be artistic conception, such as restoring and feeling a certain artistic conception in Chinese and history classes, etc.

[0143] In this stage, the virtual reality terminal can superimpose and merge the virtual reality scene with the real environment in the manner described in the above situation 2 to obtain a virtual-reality fusion scene.

[0144] 1-4. The teacher summarizes the content of the virtual reality experience through the two-dimensional screen and reviews the knowledge points

[0145] In this stage, the virtual reality terminal can superimpose and merge the virtual reality scene with the two-dimensional screen and the real environment in the manner described in the above situation three to obtain a virtual-reality fusion scene.

[0146] 1-5. The class ends, or you repeat the process of steps 1-2 to 1-4 on the next knowledge point.

[0147] Based on the above scenario examples, it can be seen that the present application can use the virtual reality terminal to complete various teaching methods in all links by superimposing and integrating the virtual reality scene with at least one of the two-dimensional screen and the real environment. The user can complete all teaching methods after wearing the virtual reality terminal, and there is no need to frequently take off the virtual reality terminal, thereby saving operation time and improving efficiency.

[0148] The method of the present application is introduced above, and the device of the present application is introduced below.

[0149] See also Figure 5 , Figure 5 Schematic diagram of a scene display device provided in an embodiment of the present application. Figure 5 As shown, the scene display device 30 includes:

[0150] An acquisition module 301 is used to acquire a target virtual reality scene and an object to be fused corresponding to the target virtual reality scene, wherein the object to be fused is an object to be displayed in a real scene;

[0151] A fusion module 302 is used to superimpose and fuse the target virtual reality scene with the object to be fused to obtain a virtual-reality fusion scene corresponding to the target virtual reality scene;

[0152] The display module 303 is used to display the virtual-reality fusion scene in the virtual reality terminal.

[0153] In a possible design, the object to be fused includes a two-dimensional screen in a real scene; the fusion module 302 is specifically used to: obtain a two-dimensional screen image stream corresponding to the two-dimensional screen, wherein the two-dimensional screen image stream is obtained based on capturing and encoding the image in the two-dimensional screen; decode the two-dimensional screen image stream, and overlay and render the decoded two-dimensional screen image stream in the target virtual reality scene to obtain the virtual-reality fusion scene.

[0154] In a possible design, a first object surface is reserved in the target virtual reality scene, and the first object surface is used to carry the screen image; the above-mentioned fusion module 302 is specifically used to: in the target virtual reality scene, set the decoded two-dimensional screen image stream as the texture map of the first object surface, and render the target virtual reality scene to obtain the virtual-reality fusion scene.

[0155] In a possible design, the fusion module 302 is specifically used to: in the target virtual reality scene, set the decoded two-dimensional screen image stream as the texture map of the surface of the first object, and render the target virtual reality scene to obtain the virtual-reality fusion scene.

[0156] In a possible design, the scene theme corresponding to the target virtual reality scene matches the picture theme corresponding to the decoded two-dimensional screen picture stream.

[0157] In a possible design, the object to be fused includes a real environment in a real scene; the fusion module 302 is specifically used to: superimpose and fuse the target virtual reality scene with the object to be fused to obtain a virtual-reality fusion scene corresponding to the target virtual reality scene, including: obtaining a real environment picture corresponding to the real environment; superimposing and rendering the target virtual reality scene in the real environment picture to obtain the virtual-reality fusion scene.

[0158] In a possible design, the fusion module 302 is also used to perform three-dimensional reconstruction of real objects in the real environment picture in the target virtual reality scene, obtain a three-dimensional reconstructed virtual object corresponding to the real object, and set the material of the three-dimensional reconstructed virtual object to occlusion culling.

[0159] In a possible design, the fusion module 302 is further used to search for a virtual object plane corresponding to the three-dimensional reconstructed virtual object in the target virtual reality scene; and to set part or all of the original virtual objects in the target virtual reality scene on the virtual object plane.

[0160] In a possible design, part or all of the original virtual objects are interactive virtual objects in the target virtual reality scene.

[0161] In a possible design, the objects to be fused include a two-dimensional screen and a real environment in a real scene; the fusion module 302 is specifically used to: obtain a two-dimensional screen picture stream corresponding to the two-dimensional screen and a real environment picture corresponding to the real environment, wherein the two-dimensional screen picture stream is obtained based on picture capture and encoding of the picture in the two-dimensional screen; decode the two-dimensional screen picture stream, and overlay and render the decoded two-dimensional screen picture stream in the target virtual reality scene to obtain an initial fusion scene; overlay and render the initial fusion scene in the real environment picture to obtain the virtual-reality fusion scene.

[0162] In a possible design, the position and orientation of the two-dimensional screen in the virtual-reality fusion scene are obtained based on the position and orientation of the object in the real environment picture.

[0163] It should be noted that Figure 5 For the contents not mentioned in the corresponding embodiments, please refer to the description of the aforementioned method embodiments, which will not be repeated here.

[0164] The above device obtains the target virtual reality scene and the object to be fused corresponding to the target virtual reality scene, where the object to be fused is the object to be displayed in the real scene; then the target virtual reality scene is superimposed and fused with the object to be fused to obtain the virtual-reality fusion scene corresponding to the target virtual reality scene; and the virtual-reality fusion scene is displayed in the virtual reality terminal, thereby realizing the combined display of the real scene and the virtual scene. When the object to be fused includes a two-dimensional screen, the two-dimensional screen is superimposed and displayed with the target virtual reality scene, and the user only needs to wear the virtual reality terminal to implement the two-dimensional screen teaching and virtual reality teaching at the same time, eliminating the steps of taking off and wearing the virtual reality terminal, which can save operation time.

[0165] See also Figure 6 , Figure 6 4 is a structural diagram of a virtual reality terminal provided in an embodiment of the present application, wherein the virtual reality terminal 40 includes a processor 401, a memory 402, and a display panel 403. The memory 402 and the display panel 403 are connected to the processor 401, for example, through a bus.

[0166] The processor 401 is configured to support the virtual reality terminal 40 to perform the corresponding functions of the method in the above method embodiment. The processor 401 can be a central processing unit (CPU), a network processor (NP), a hardware chip or any combination thereof. The above hardware chip can be an application specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0167] The memory 402 is used to store program codes, etc. The memory 402 may include a volatile memory (VM), such as a random access memory (RAM); the memory 402 may also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory 402 may also include a combination of the above-mentioned types of memories.

[0168] The display panel 403 is used to display the virtual-real fusion scene.

[0169] The processor 401 may call the program code to perform the following operations:

[0170] Acquire a target virtual reality scene and an object to be fused corresponding to the target virtual reality scene, wherein the object to be fused is an object to be displayed in the real scene;

[0171] Superimposing and fusing the target virtual reality scene with the object to be fused to obtain a virtual-reality fusion scene corresponding to the target virtual reality scene;

[0172] The virtual-reality fusion scene is displayed in the virtual reality terminal.

[0173] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method described in the above embodiment.

[0174] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0175] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A scene display method, characterized in that: Applied to a virtual reality terminal, the method comprises: Acquire a target virtual reality scene and an object to be fused corresponding to the target virtual reality scene, wherein the object to be fused is an object to be displayed in the real scene; Superimposing and fusing the target virtual reality scene with the object to be fused to obtain a virtual-reality fusion scene corresponding to the target virtual reality scene; The virtual-reality fusion scene is displayed in the virtual reality terminal.

2. The method according to claim 1, characterized in that The object to be fused includes a two-dimensional screen in a real scene; The step of superimposing and fusing the target virtual reality scene with the object to be fused to obtain a virtual-reality fusion scene corresponding to the target virtual reality scene includes: Acquire a two-dimensional screen picture stream corresponding to the two-dimensional screen, wherein the two-dimensional screen picture stream is obtained by capturing and encoding pictures in the two-dimensional screen; The two-dimensional screen image stream is decoded, and the decoded two-dimensional screen image stream is overlaid and rendered in the target virtual reality scene to obtain the virtual-reality fusion scene.

3. The method according to claim 2, characterized in that A first object surface is reserved in the target virtual reality scene, and the first object surface is used to carry a screen image; The step of fusing the two-dimensional screen image stream obtained by superimposing rendering in the target virtual reality scene to obtain the virtual-reality fusion scene comprises: In the target virtual reality scene, the decoded two-dimensional screen image stream is set as a texture map of the surface of the first object, and the target virtual reality scene is rendered to obtain the virtual-reality fusion scene.

4. The method according to claim 2, characterized in that: The step of superimposing the two-dimensional screen image stream obtained by rendering and decoding in the target virtual reality scene to obtain the virtual-reality fusion scene comprises: Acquire a first virtual space position of a target user corresponding to the virtual reality terminal in a virtual space corresponding to the target virtual reality scene; A second object surface is created at a second virtual space position in the target virtual reality scene, and the decoded two-dimensional screen image stream is set as a texture map of the second object surface, and the target virtual reality scene is rendered to obtain the virtual-reality fusion scene, and the distance between the second virtual space position and the first virtual space position in the virtual space corresponding to the target virtual reality scene is less than a preset distance threshold.

5. The method according to claim 3 or 4, characterized in that: The scene theme corresponding to the target virtual reality scene matches the picture theme corresponding to the decoded two-dimensional screen picture stream.

6. The method according to claim 1, characterized in that The object to be fused includes a real environment in a real scene; The step of superimposing and fusing the target virtual reality scene with the object to be fused to obtain a virtual-reality fusion scene corresponding to the target virtual reality scene includes: Acquire a real environment picture corresponding to the real environment; The target virtual reality scene is superimposed and rendered in the real environment picture to obtain the virtual-reality fusion scene.

7. The method according to claim 6, characterized in that Before the target virtual reality scene is superimposed and rendered in the real environment picture to obtain the virtual-reality fusion scene, the method further includes: In the target virtual reality scene, the real objects in the real environment picture are three-dimensionally reconstructed to obtain the three-dimensionally reconstructed virtual objects corresponding to the real objects, and the material of the three-dimensionally reconstructed virtual objects is set to occlusion culling.

8. The method according to claim 7, characterized in that After the real object in the real environment picture is three-dimensionally reconstructed in the target virtual reality scene to obtain the three-dimensionally reconstructed virtual object corresponding to the real object, the method further includes: Searching for a virtual object plane corresponding to the three-dimensional reconstructed virtual object in the target virtual reality scene; Part or all of the original virtual objects in the target virtual reality scene are arranged on the virtual object plane.

9. The method according to claim 8, characterized in that Some or all of the original virtual objects are interactive virtual objects in the target virtual reality scene.

10. The method according to claim 1, characterized in that The objects to be fused include a two-dimensional screen and a real environment in a real scene; The step of superimposing and fusing the target virtual reality scene with the object to be fused to obtain a virtual-reality fusion scene corresponding to the target virtual reality scene includes: Acquire a two-dimensional screen picture stream corresponding to the two-dimensional screen and a real environment picture corresponding to the real environment, wherein the two-dimensional screen picture stream is obtained by capturing and encoding a picture in the two-dimensional screen; Decoding the two-dimensional screen image stream, and overlaying and rendering the decoded two-dimensional screen image stream in the target virtual reality scene to obtain an initial fusion scene; The initial fusion scene is overlaid and rendered in the real environment picture to obtain the virtual-real fusion scene.

11. The method according to claim 10, characterized in that The position and orientation of the two-dimensional screen image corresponding to the two-dimensional screen image stream in the virtual-reality fusion scene are obtained based on the position and orientation of the object in the real environment image.

12. A scene display device, characterized in that: Applied to virtual reality terminals, including: An acquisition module, used to acquire a target virtual reality scene and an object to be fused corresponding to the target virtual reality scene, wherein the object to be fused is an object to be displayed in the real scene; A fusion module, used to superimpose and fuse the target virtual reality scene with the object to be fused to obtain a virtual-reality fusion scene corresponding to the target virtual reality scene; A display module is used to display the virtual-reality fusion scene in the virtual reality terminal.

13. A virtual reality terminal, characterized in that: The virtual reality terminal comprises a memory, a processor and a display panel, wherein the memory and the display panel are connected to the processor, and the processor is used to execute one or more computer programs stored in the memory. When the processor executes the one or more computer programs, the virtual reality terminal implements the method according to any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is caused to perform the method according to any one of claims 1 to 11.

Images