Method and apparatus for drawing three-dimensional scene, device, and medium

Abstract

Embodiments of the present disclosure provide a method and apparatus for drawing a three-dimensional scene, a device, and a storage medium. The method comprises: on the basis of a received data stream, acquiring image data corresponding to a target two-dimensional image; on the basis of stitching information associated with the image data, using the image data to reconstruct a plurality of two-dimensional images, the stitching information indicating the positions of the plurality of two-dimensional images in the target two-dimensional image; and on the basis of display information corresponding to the plurality of two-dimensional images, drawing the plurality of two-dimensional images in a three-dimensional scene, the display information indicating display positions of the plurality of two-dimensional images in the three-dimensional scene. In this way, in the embodiments of the present disclosure, two-dimensional images can be used as carriers to draw pictures in a three-dimensional scene, thereby improving the efficiency of drawing the three-dimensional scene.

Description

Method, device, equipment and medium for rendering three-dimensional scene

[0001] The present application claims priority to the Chinese patent application No. 202410437956.1, filed on April 11, 2024, entitled “Method, device, equipment and storage medium for rendering three-dimensional scene”, the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The example embodiments of the present disclosure generally relate to the field of computer, and in particular, to a method, device, equipment and computer readable storage medium for rendering a three-dimensional scene. BACKGROUND

[0003] Extended Reality (XR) is widely studied and applied. XR combines hardware devices and various technical means to fuse virtual content and real scenes, providing users with unique sensory experiences. XR includes, for example, Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR).

[0004] Thus, a virtual environment can be used to provide an immersive interactive experience for users. SUMMARY

[0005] In a first aspect of the present disclosure, a method for rendering a three-dimensional scene is provided. The method comprises: obtaining image data corresponding to a target two-dimensional image based on a received data stream; reconstructing a plurality of two-dimensional images using the image data based on stitching information associated with the image data, the stitching information indicating positions of the plurality of two-dimensional images in the target two-dimensional image; and rendering the plurality of two-dimensional images in the three-dimensional scene based on display information corresponding to the plurality of two-dimensional images, the display information indicating display positions of the plurality of two-dimensional images in the three-dimensional scene.

[0006] In a second aspect of the disclosure, an apparatus for rendering a three-dimensional scene is provided. The apparatus comprises: an obtaining module configured to obtain image data corresponding to a target two-dimensional image based on a received data stream; a reconstructing module configured to reconstruct a plurality of two-dimensional images based on stitching information associated with the image data, the stitching information indicating positions of the plurality of two-dimensional images in the target two-dimensional image, using the image data; and a rendering module configured to render the plurality of two-dimensional images in a three-dimensional scene based on display information corresponding to the plurality of two-dimensional images, the display information indicating display positions of the plurality of two-dimensional images in the three-dimensional scene.

[0007] In a third aspect of the disclosure, an electronic device is provided. The device comprises at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. The instructions, when executed by the at least one processing unit, cause the device to perform the method of the first aspect.

[0008] In a fourth aspect of the disclosure, a computer-readable storage medium is provided. The computer-readable storage medium has stored thereon a computer program, which is executable by a processor to implement the method of the first aspect.

[0009] It should be understood that all statements herein made regarding the disclosure are not intended to define key or essential features of embodiments of the disclosure or to limit the scope of the disclosure. Other features of the disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0010] The above and other features, aspects and advantages of embodiments of the disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings. In the drawings, like or similar elements are referred to by like or similar reference numbers, in which:

[0011] FIG. 1 shows a schematic diagram of an example environment in which embodiments of the disclosure can be implemented;

[0012] FIGS. 2A-2D show schematic diagrams of examples of a virtual environment, respectively, according to some embodiments of the disclosure;

[0013] FIG. 3 shows a flowchart of an example process for rendering a three-dimensional scene, according to some embodiments of the disclosure;

[0014] FIG. 4 shows a block diagram of an example apparatus for rendering a three-dimensional scene, according to some embodiments of the disclosure; and

[0015] FIG. 5 shows a block diagram of a device capable of implementing embodiments of the disclosure. DETAILED DESCRIPTION

[0016] It can be understood that, before using the technical solutions disclosed in the embodiments of the present disclosure, the type of personal information involved in the present disclosure, the use range, the use scenario, etc. should be informed to the user and the authorization of the user should be obtained through appropriate means according to relevant laws and regulations.

[0017] For example, in response to receiving the active request of the user, prompt information is sent to the user to explicitly prompt the user that the operation requested to be performed will require obtaining and using the personal information of the user. Thus, the user can voluntarily choose whether to provide the personal information to the software or hardware such as the electronic device, the application program, the server or the storage medium, etc. performing the operation of the technical solutions of the present disclosure according to the prompt information.

[0018] As an optional but non-limiting implementation manner, in response to receiving the active request of the user, the manner of sending the prompt information to the user may, for example, be the manner of a pop-up window, and the prompt information may, for example, be presented in the form of text in the pop-up window. In addition, the pop-up window may, for example, also carry a selection control for the user to select “agree” or “disagree” to provide the personal information to the electronic device.

[0019] It can be understood that the above notification and obtaining of the authorization of the user are only illustrative, and do not limit the implementation manners of the present disclosure, and other manners meeting the relevant laws and regulations can also be applied to the implementation manners of the present disclosure.

[0020] It can be understood that the data (including but not limited to the data itself, the acquisition or use of the data) involved in the present technical solutions should comply with the requirements of the relevant laws and regulations and the relevant provisions.

[0021] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms, and should not be interpreted as being limited to the embodiments set forth herein, rather, these embodiments are provided to make the present disclosure more thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes, and are not intended to limit the scope of protection of the present disclosure.

[0022] It should be noted that the titles of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and any type of embodiment can be included under any section / subsection. Furthermore, embodiments described in any section / subsection can be combined with any other embodiment described in the same section / subsection and / or in a different section / subsection in any manner.

[0023] In the description of embodiments of the disclosure, the term "includes" and its conjugates are to be open-ended, i.e., "including but not limited to". The term "based on" is to be open-ended, i.e., "based, at least in part, on". The term "one embodiment" or "an embodiment" is to be understood as "at least one embodiment". The term "some embodiments" is to be understood as "at least some embodiments". Other explicit or implicit definitions can also be included below. The terms "first", "second", etc. can refer to different or the same objects. Other explicit or implicit definitions can also be included below.

[0024] With the development of XR technology, live streaming technology based on XR technology is also developing rapidly. Generally speaking, XR live streaming includes panoramic live streaming and live streaming based on volumetric video. Panoramic live streaming relies on the shooting of panoramic cameras and does not support observer position adjustment. For volumetric video, it needs to rely on complex encoding, transmission and decoding processes such as array shooting and three-dimensional modeling.

[0025] Embodiments of the present disclosure propose a scheme for drawing a three-dimensional scene. According to various embodiments of the present disclosure, based on a received data stream, image data corresponding to a target two-dimensional image is obtained; based on stitching information associated with the image data, a plurality of two-dimensional images is reconstructed using the image data, the stitching information indicating positions of the plurality of two-dimensional images in the target two-dimensional image; and based on display information corresponding to the plurality of two-dimensional images, the plurality of two-dimensional images is drawn in the three-dimensional scene, the display information indicating display positions of the plurality of two-dimensional images in the three-dimensional scene.

[0026] Thus, embodiments of the present disclosure can use two-dimensional images as carriers to draw pictures in a three-dimensional scene, thereby improving the drawing efficiency of the three-dimensional scene.

[0027] Example environment

[0028] FIG. 1 shows a block diagram of an environment 100 according to some embodiments of the present disclosure. As shown, the environment 100 can include a user 130, who can wear an XR device 113, for example. The XR device 113 can communicate with an electronic device 110 to reconstruct a virtual scene or to fuse virtual content (also referred to as a virtual environment) and a real environment (also referred to as a physical environment) for the user 130.

[0029] In this disclosure, a virtual environment reconstructed based on VR technology, and an environment in which virtual content is fused with a real environment based on AR technology or MR technology, are collectively referred to as a virtual environment 120. The XR device 113 can be a head-mounted or wearable near-eye display device, such as a head-mounted display, smart glasses, etc., supporting VR, AR, MR, etc. technology. The XR device 113 can include an image generation component and an optical display component for reconstructing the virtual environment 120 in a monocular or binocular field of view and displaying virtual components and / or virtual avatars in the virtual environment.

[0030] For example, the XR device 113 can display one or more images, such as image 124 and image 126, in the virtual environment 120. As will be described in detail below, such an image 124 or image 126 can be rendered in a corresponding three-dimensional scene in the virtual environment 120 based on a received data stream.

[0031] The electronic device 110 can be a separate device capable of communicating with the XR device 113 and / or other image capture devices, such as a server, a computing node, etc. for image or data processing, and can also be integrated with the XR device 113 and / or other image capture devices. In some embodiments, the electronic device 110 can be implemented as the XR device 113, i.e. in this case, the XR device 113 can implement all the functions of the electronic device 110. It should be understood that the above description of the electronic device 110 is merely exemplary and not limiting, and the electronic device 110 can be implemented as a device in multiple forms, structures or categories, and embodiments of the present disclosure have no limitation in this regard.

[0032] It should be understood that the structure and function of the environment 100 are described for illustrative purposes only, and do not imply any limitation on the scope of the present disclosure.

[0033] Example interaction process

[0034] FIGS. 2A-2D respectively illustrate schematic diagrams of rendering a three-dimensional scene, according to some embodiments of the present disclosure. For ease of discussion, an example process according to some embodiments of the present disclosure will be described with reference to at least the environment 100 of FIG. 1.

[0035] In some embodiments, in order to efficiently render images in a virtual scene, embodiments of the present disclosure can generate a separate data stream, such as a bitstream, by stitching images.

[0036] As shown in FIG. 2A, an encoding device of the bitstream can form a single two-dimensional image 202 by stitching a plurality of two-dimensional images, such as image 205, image 210 and image 215. Further, the encoding device can encode the two-dimensional image 202 to obtain corresponding image encoding data.

[0037] In some embodiments, the two-dimensional image 202 can be stitched based on two or more images. The disclosure is not intended to limit the specific way of stitching the images.

[0038] In some embodiments, one or more of the image 205, the image 210 and the image 215 can be associated with a video stream. For example, the image 205 can be a video frame in a video stream. In some embodiments, such a video stream may, for example, be a real-time video stream, e.g., a live video stream. Alternatively, such a video stream can also be a video stream corresponding to an existing video content, e.g., a local video file, etc.

[0039] Taking the live scenario as an example, such multiple video streams may, for example, include video content uploaded by the anchor end and / or video content recorded by the anchor end in real time, etc.

[0040] In some embodiments, the encoding device can stitch the respective video frames of the multiple video streams according to the timing alignment to obtain multiple two-dimensional images.

[0041] Further, the encoding device can encode the multiple two-dimensional images based on any appropriate manner to obtain a corresponding data stream, e.g., a bitstream.

[0042] In some embodiments, to facilitate the XR device 113 to extract the corresponding two-dimensional image from the image data, the encoding device can further determine and generate stitching information associated with the two-dimensional image 202. Such stitching information can indicate the positions of the image 205, the image 210 and the image 215 in the two-dimensional image 205. For example, the stitching information can indicate the four vertex coordinates of the image 205 in the two-dimensional image 202, or two corresponding vertex coordinates.

[0043] In some embodiments, to facilitate the XR device 113 to render the corresponding two-dimensional image in the three-dimensional scene, the encoding device can further determine and generate display information associated with the multiple two-dimensional images. Such display information may, for example, indicate the display positions of the image 205, the image 210 or the image 215 in the three-dimensional scene.

[0044] In some embodiments, the encoding device may, for example, send the stitching information and / or the display information to the XR device 113 independently of the data stream. Alternatively, the encoding device may, for example, also encode the stitching information and / or the display information in the data stream. As an example, the encoding device may, for example, encode the stitching information and / or the display information in the Supplemental Enhancement Information (SEI) of the bitstream.

[0045] Accordingly, the XR device 113, after receiving the data stream, can decode image data corresponding to the two-dimensional image 202 from the data stream.

[0046] Further, the XR device 113 can reconstruct the multiple images in the two-dimensional image 202, e.g., the image 205, the image 210, and the image 215, based on the stitching information obtained.

[0047] For example, the XR device 113 can decode the stitching information in the SEI of the bitstream to determine the positions of the image 205, the image 210, and the image 215 in the two-dimensional image 202, and can accordingly reconstruct the image 205, the image 210, and the image 215.

[0048] Further, the XR device 113 can also render the reconstructed multiple images, e.g., the image 205, the image 210, and the image 215, in the three-dimensional scene accordingly based on the display information obtained.

[0049] For example, the XR device 113 can decode the display information in the SEI of the bitstream to determine the display positions of the image 205, the image 210, and the image 215 in the three-dimensional scene, and can accordingly render the image 205, the image 210, and the image 215.

[0050] As shown in FIG. 2B, the XR device 113 can render the image 210 and the image 215 in the corresponding planes in the three-dimensional scene. Such an image 210 or image 215 may, for example, also be referred to as a partial image.

[0051] In some embodiments, the image 205 can correspond to a panoramic image. Accordingly, the XR device 113 may, for example, determine that the image 205 corresponds to a panoramic image based on the SEI in the bitstream, and can render the panoramic image as a background image of the three-dimensional scene.

[0052] Taking FIG. 2B as an example, the XR device 113 can render a corresponding portion of the image 205 as a viewed background 220 of the three-dimensional scene according to a virtual viewing position 225, e.g., a position of a virtual camera corresponding to the three-dimensional scene.

[0053] Thus, embodiments of the present disclosure can utilize a two-dimensional image as a carrier to render a picture in a three-dimensional scene, thereby improving the rendering efficiency of the three-dimensional scene.

[0054] In some embodiments, embodiments of the present disclosure can also support multi-degree-of-freedom interaction in the three-dimensional scene. For example, FIG. 2C shows a picture 200C of the three-dimensional scene as viewed by a user in a first state of a virtual camera.

[0055] As shown in FIG. 2C, the XR device 110 can determine to present the image 210, the image 215 and the background 220 according to the parameters (e.g., position and pose information) of the virtual camera. In this way, the XR device 110 can present a depth-of-field effect similar to a volumetric video, e.g., the image 210 can be partially occluded from the image 215 based on the display position.

[0056] Further, as shown in FIG. 2C, the XR device 110 can also adjust the frame of the three-dimensional scene according to the motion of the virtual camera. As shown in FIG. 2D, when the virtual camera moves forward and passes the display plane of the image 210 according to the user operation, the XR device 110 can present a frame 200D as shown in FIG. 2D.

[0057] As shown in FIG. 2D, the XR device 110 can determine to stop displaying the image 210 in the frame 200D according to the comparison of the updated parameters of the virtual camera and the display position of the image 210. Additionally, the XR device 110 may, for example, also adjust the display size of the image 215 accordingly.

[0058] It should be understood that the XR device 110 may, for example, also change the display direction of the image 210 or the image 215 accordingly according to the change of the pose of the virtual camera, etc.

[0059] In this way, the embodiments of the present disclosure can support multi-degree-of-freedom interaction in the rendered three-dimensional scene, thereby improving the user’s interactive experience.

[0060] It should be understood that when the image 205, the image 210 or the image 215 corresponds to a video frame of a video stream, the XR device 110 can render the corresponding video frame in the frame 200C or the frame 200D, thereby providing the user with a video viewing effect in the three-dimensional scene.

[0061] Example process

[0062] FIG. 3 illustrates an interaction process 300 in a virtual scene, which can be implemented by the XR device 113, according to some embodiments of the present disclosure.

[0063] At block 310, the XR device 113 obtains image data corresponding to a target two-dimensional image based on a received data stream.

[0064] At block 320, the XR device 113 reconstructs a plurality of two-dimensional images from the image data based on stitching information associated with the image data, the stitching information indicating positions of the plurality of two-dimensional images in the target two-dimensional image.

[0065] At block 330, the XR device 113 renders the plurality of two-dimensional images in the three-dimensional scene based on the display information corresponding to the plurality of two-dimensional images, the display information indicating display positions of the plurality of two-dimensional images in the three-dimensional scene.

[0066] In some embodiments, the process 300 further includes decoding the stitching information and / or the display information from the data stream.

[0067] In some embodiments, the stitching information and / or the display information are encoded in supplemental enhancement information of the data stream.

[0068] In some embodiments, the plurality of two-dimensional images includes panoramic images, and rendering the plurality of two-dimensional images in the three-dimensional scene includes rendering the panoramic images in the three-dimensional scene as a background of the three-dimensional scene.

[0069] In some embodiments, the plurality of two-dimensional images includes local images, and rendering the plurality of two-dimensional images in the three-dimensional scene includes rendering the local images in corresponding planes of the three-dimensional scene based on the display information.

[0070] In some embodiments, the process 300 further includes displaying a frame of the three-dimensional scene with a target device, wherein the frame is determined based on a virtual camera corresponding to the target device, and adjusting the frame based on movement of the virtual camera.

[0071] In some embodiments, adjusting the frame based on the movement of the virtual camera includes determining a display status of at least one of the plurality of two-dimensional images based on a comparison of updated parameters of the virtual camera and the display positions of the at least one of the plurality of two-dimensional images.

[0072] In some embodiments, the display status indicates at least one of: whether the at least one of the plurality of two-dimensional images is displayed in the frame; a size of the at least one of the plurality of two-dimensional images in the frame; and a direction of the at least one of the plurality of two-dimensional images in the frame.

[0073] In some embodiments, the target two-dimensional image is generated by stitching corresponding video frames of a plurality of video streams, and rendering the plurality of two-dimensional images in the three-dimensional scene includes rendering a plurality of video frames corresponding to the plurality of video streams in the three-dimensional scene.

[0074] In some embodiments, the plurality of video streams includes video streams corresponding to existing video content and / or real-time captured video streams.

[0075] In some embodiments, the plurality of two-dimensional images includes at least one two-dimensional image associated with live content.

[0076] Example apparatuses and devices

[0077] FIG. 4 illustrates a schematic structural block diagram of an apparatus 400 for interaction in a virtual scene, according to certain embodiments of the present disclosure. The apparatus 400 can be implemented as or included in the XR device 113. Various modules / components in the apparatus 400 can be implemented by hardware, software, firmware, or any combination thereof.

[0078] As shown, the apparatus 400 includes an obtaining module 410 configured to obtain image data corresponding to a target two-dimensional image based on a received data stream; a reconstructing module 420 configured to reconstruct a plurality of two-dimensional images based on stitching information associated with the image data, the stitching information indicating positions of the plurality of two-dimensional images in the target two-dimensional image, using the image data; and a rendering module 430 configured to render the plurality of two-dimensional images in a three-dimensional scene based on display information corresponding to the plurality of two-dimensional images, the display information indicating display positions of the plurality of two-dimensional images in the three-dimensional scene.

[0079] In some embodiments, the apparatus 400 further includes a decoding module configured to decode the stitching information and / or the display information from the data stream.

[0080] In some embodiments, the stitching information and / or the display information are encoded in a supplemental enhancement information of the data stream.

[0081] In some embodiments, the plurality of two-dimensional images include panoramic images, and the rendering module 430 is further configured to render the panoramic images in the three-dimensional scene as a background of the three-dimensional scene.

[0082] In some embodiments, the plurality of two-dimensional images include local images, and the rendering module 430 is further configured to render the local images in corresponding planes of the three-dimensional scene based on the display information.

[0083] In some embodiments, the apparatus 400 further includes an adjusting module configured to display a frame of the three-dimensional scene using a target device, wherein the frame is determined based on a virtual camera corresponding to the target device, and adjust the frame based on a movement of the virtual camera.

[0084] In some embodiments, the adjusting module is further configured to determine a display state of at least one of the plurality of two-dimensional images based on a comparison of an updated parameter of the virtual camera and a display position of the at least one of the plurality of two-dimensional images.

[0085] In some embodiments, the display state indicates at least one of: whether the at least one of the plurality of two-dimensional images is displayed in the frame; a size of the at least one of the plurality of two-dimensional images in the frame; and a direction of the at least one of the plurality of two-dimensional images in the frame.

[0086] In some embodiments, the target two-dimensional image is generated by stitching corresponding video frames of a plurality of video streams, and rendering the plurality of two-dimensional images in the three-dimensional scene comprises:

[0087] rendering a plurality of video pictures corresponding to the plurality of video streams in the three-dimensional scene.

[0088] In some embodiments, the plurality of video streams comprises video streams corresponding to existing video content and / or real-time captured video streams.

[0089] In some embodiments, the plurality of two-dimensional images comprises at least one two-dimensional image associated with live content.

[0090] FIG. 5 illustrates a block diagram of a computing device 500 in which one or more embodiments of the disclosure can be implemented. It should be appreciated that the computing device 500 illustrated in FIG. 5 is merely an example and should not be construed as any limitation of the functionality and scope of the embodiments described herein. The computing device 500 illustrated in FIG. 5 can be used to implement the XR device 113 of FIG. 1.

[0091] As illustrated in FIG. 5, the computing device 500 is in the form of a general- purpose computing device. Components of the computing device 500 can include, but are not limited to, one or more processors or processing units 510, a memory 520, a storage device 530, one or more communication units 540, one or more input devices 550, and one or more output devices 550. The processing unit 510 can be a real or virtual processor and is capable of executing various processing in accordance with programs stored in the memory 520. In a multi-processing system, multiple processing units execute computer-executable instructions in parallel to improve processing performance of the computing device 500.

[0092] The computing device 500 typically includes a plurality of computer storage media. Such media can be volatile and / or non-volatile storage media and can be removable and / or non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules, and other data. The memory 520 can be volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 530 can be a removable or non-removable media and can include machine readable media such as flash drives, disks, or any other media capable of storing information and / or data and accessible by the computing device 500.

[0093] The computing device 500 can further include additional detachable / non-detachable, volatile / non-volatile storage media. Although not shown in FIG. 5, a disk drive for reading from or writing to a detachable, non-volatile magnetic disk (e.g., a "floppy disk"), and an optical disk drive for reading from or writing to a detachable, non-volatile optical disk can be provided. In these instances, each drive can be connected to the bus (not shown) by one or more data media interfaces. The memory 520 can include a computer program product 525 having one or more program modules configured to carry out the various methods or acts of the various embodiments of the present disclosure.

[0094] The communication unit 540 enables communication with other computing devices over a communication medium. Additionally, the functionality of the components of the computing device 500 can be implemented in a single computing cluster or multiple computer machines that are capable of communicating over a communication connection. Thus, the computing device 500 can operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node in the case of a distributed system environment.

[0095] The input device 550 can be one or more input devices, such as a mouse, a keyboard, a trackball, etc. The output device 550 can be one or more output devices, such as a display, a speaker, a printer, etc. The computing device 500 can also communicate with one or more external devices (not shown) such as a storage device, a display device, etc. through the communication unit 540, as needed, one or more devices that enable a user to interact with the computing device 500, or any devices (e.g., a network card, a modem, etc.) that enable the computing device 500 to communicate with one or more other computing devices. Such communication can be carried out via an input / output (I / O) interface (not shown).

[0096] According to an example implementation of the present disclosure, a computer readable storage medium having computer executable instructions stored thereon is provided, where the computer executable instructions are executed by a processor to implement the method described above. According to an example implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer readable medium and includes computer executable instructions, where the computer executable instructions are executed by a processor to implement the method described above.

[0097] Various aspects of the disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0098] The computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data, programs, program modules, e.g., instructions for operation, or digital content stored thereon or therein for a short time or not at all. The computer readable storage medium can also have other meanings inhered therein by those skilled in the art (e.g., the computer readable storage medium can be a random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory, or any memory that can be used for storage of information or retrieving information). The instructions can be executed by one or more processors of a computer or other programmable data processing apparatus to produce a computer implemented process such that the instructions which execute via the one or more processors of the computer or other programmable data processing apparatus create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable data processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable data processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0099] The computer readable program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable data processing apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable data processing apparatus implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0100] The flow and block diagrams in the drawings show architectural, functional, and operational representations of possible implementations of systems, methods, and computer program products according to the present disclosure. In this regard, each block in the flow and block diagrams can represent a module, a segment, or a portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may

[0101] implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The above description is illustrative only and not restrictive. As used herein, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or". That is, unless specified otherwise, or clear from the context, the phrase "X employs A or B" is intended to mean that the process of X at least employs one of the features A or also employs the other feature B. Likewise, the terms "comprising," "including," containing," or "carrying" are to be construed in an inclusive manner, i.e., as "comprising" or "containing" but not "consisting of." Moreover, the descriptions of the implementations have been presented for purposes of illustration and explanation, and such descriptions are not intended to limit the implementations in scope to the exact embodiments described. Accordingly, modifications and variations of the described implementations are possible in light of the above teachings.

Claims

1. A method for drawing a three-dimensional scene, comprising: acquiring image data corresponding to a target two-dimensional image based on the received data stream; reconstructing a plurality of two-dimensional images using the image data based on stitching information associated with the image data, the stitching information indicating positions of the plurality of two-dimensional images in the target two-dimensional image; as well as The plurality of two-dimensional images are rendered in a three-dimensional scene based on display information corresponding to the plurality of two-dimensional images, the display information indicating display positions of the plurality of two-dimensional images in the three-dimensional scene.

2. The method according to claim 1, further comprising: The splicing information and / or the display information are obtained by decoding the data stream.

3. The method according to claim 2, wherein the splicing information and / or the display information is encoded in supplemental enhancement information of the data stream.

4. The method of claim 1 , wherein the plurality of two-dimensional images include panoramic images, and rendering the plurality of two-dimensional images in a three-dimensional scene comprises: The panoramic image is drawn in the three-dimensional scene to serve as a background of the three-dimensional scene.

5. The method of claim 1 , wherein the plurality of two-dimensional images include partial images, and rendering the plurality of two-dimensional images in a three-dimensional scene comprises: Based on the display information, the partial image is drawn in a corresponding plane of the three-dimensional scene.

6. The method according to claim 1, further comprising: Using a target device to display a picture of the three-dimensional scene, wherein the picture is determined based on a virtual camera corresponding to the target device; as well as The picture is adjusted based on the movement of the virtual camera.

7. The method according to claim 6, wherein adjusting the frame based on the movement of the virtual camera comprises: Based on a comparison between the updated parameters of the virtual camera and a display position of at least one two-dimensional image among the plurality of two-dimensional images, a display state of the at least one two-dimensional image is determined.

8. The method of claim 7, wherein the display status indicates at least one of the following: whether the at least one two-dimensional image is displayed in the picture; the size of the at least one two-dimensional image in the picture; The orientation of the at least one two-dimensional image in the picture.

9. The method according to claim 1 , wherein the target two-dimensional image is generated by splicing corresponding video frames of a plurality of video streams, and rendering the plurality of two-dimensional images in the three-dimensional scene comprises: A plurality of video frames corresponding to the plurality of video streams are drawn in the three-dimensional scene.

10. The method of claim 9, wherein the plurality of video streams comprises: A video stream corresponding to the existing video content; and / or Real-time video streaming.

11. The method of claim 1, wherein the plurality of two-dimensional images includes at least one two-dimensional image associated with live content.

12. A device for drawing a three-dimensional scene, comprising: an acquisition module configured to acquire image data corresponding to a target two-dimensional image based on the received data stream; a reconstruction module configured to reconstruct a plurality of two-dimensional images using the image data based on stitching information associated with the image data, the stitching information indicating positions of the plurality of two-dimensional images in the target two-dimensional image; as well as The drawing module is configured to draw the multiple two-dimensional images in a three-dimensional scene based on display information corresponding to the multiple two-dimensional images, where the display information indicates display positions of the multiple two-dimensional images in the three-dimensional scene.

13. An electronic device comprising: at least one processing unit; as well as At least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 11 when executed by the at least one processing unit.

14. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 11.

Images