XR technology-based combined scene experience generation method and system, and storage medium

Abstract

Disclosed in the present invention are an XR technology-based combined scene experience generation method and system, and a medium. The experience generation method comprises: a first picture is an imaged picture of a first sub-scene of a combined scene; and the first picture and a second picture imaging other content of the combined scene are synthesized by means of occlusion calculation into a user experience picture of the combined scene, wherein the pixel depth value of a first pixel for imaging an object point can be represented by depth information of a first display range of the combined scene, and the first display range is a three-dimensional space covered by the first sub-scene presented in the combined scene space or a user experience space. In the present invention, after a stereoscopic vision picture of a sub-scene of the combined scene is generated, a combined scene experience can be correctly generated for a user without the need to generate and transmit a depth image of the corresponding picture, thereby remarkably saving the bandwidth for transmitting the depth image.

Description

[Corrected 03.09.2024 according to Rule 26] A method, system and storage medium for generating combined scene experience based on XR technology Technical Field The present invention relates to the field of virtual reality technology, and in particular to a method, system and medium for generating a combined scene experience based on extended reality (XR) technology. Background Art Virtual reality (VR), augmented reality (AR), mixed reality (MR) and other technologies are interconnected and integrated, collectively referred to as extended reality technology (XR). Based on XR technology, a combined scene composed of two or more sub-scenes can be constructed, where the sub-scene can be a virtual scene, a virtual-real fusion scene or a real scene. Combined scenes can break down the barriers between scenes and construct a variety of experience activity types. For example, in the classroom, multiple single-person teaching activity scenes or multi-person collaborative teaching activity scenes can be combined to generate a teaching combined scene containing multiple scenes. By changing the types and number of scenes included, a wide variety of combined scenes can be constructed, including: "learning by doing" scenes, "demonstration-practical training" scenes, "cognition-practical training" scenes, practical training assistance scenes, assessment competition scenes, multi-group practical demonstrations, etc. The sub-scenes of a combined scene can be either a stand-alone experience of a certain user or a collaborative experience of multiple users. When the sub-scenes in the combined scene are rendered separately, the user experience screens of each sub-scene need to be synthesized through occlusion calculation to form the user's experience screen of the XR combined scene. In the patent application "A method, system and platform for constructing a new dimensional space based on XR technology" (application number CN202210022608.9), a corresponding depth image is generated while rendering and generating the user experience screen of each sub-scene, and the user experience screen of each sub-scene is occluded according to the depth image. In order to reduce the generation and transmission of depth images, in the patent application "Ubiquitous training campus construction method, system and storage medium based on XR technology" (application number: CN202210908795.0), it is pre-specified that there must be a dividing surface between each sub-scene, and the occlusion relationship between each virtual scene experience screen can be calculated without the need for a depth image based on the position of the dividing surface of the scene. However, many XR combined scenes in actual application requirements cannot meet the requirement that each sub-scene has a dividing surface. Summary of the invention The main purpose of the present invention is to provide a method, system and medium for generating a combined scene experience based on XR technology. Even when the combined scene does not satisfy the existence of a dividing surface between all sub-scenes and the sub-scenes cannot transmit depth images, the occlusion synthesis of the experience picture can be correctly completed. To achieve the above-mentioned purpose, the present invention proposes a method for generating a combined scene experience based on XR technology, wherein the combined scene includes two or more sub-scenes, and multiple sub-scenes can be presented in three dimensions at the same time, and the first picture is an imaging picture of the first sub-scene of the combined scene, and the first picture and the second picture that images other contents of the combined scene are synthesized through occlusion calculation to obtain a user experience picture of the combined scene, wherein the pixel depth value of the first pixel that images the object point in the first picture can be represented by the depth information of the first display interval of the combined scene, and the first display interval is the three-dimensional space covered by the first sub-scene presented in the combined scene space or the user experience space, and the first pixel and the second pixel in the second picture are compared by depth value to obtain an occlusion relationship, and the first pixel and the second pixel share the user's line of sight, and the first sub-scene is any one of a real scene, a virtual scene, and a virtual-real fusion scene. Among them, the first picture only images the scene content within the first imaging interval of the first sub-scene, the first imaging interval is a three-dimensional space, the first picture is a binocular stereoscopic vision image, and the mapping relationship between the first imaging interval and the first display interval satisfies stereoscopic vision. As a further improvement of the present invention, the depth information of the first display interval is represented by the surface depth value of the positive side or negative side of the first display interval facing the user, or the depth information of the first display interval is represented by the depth value between the surface depth value of the positive side and the surface depth value of the negative side. As a further improvement of the present invention, in the user's combined scene experience, the first sub-scene has occlusion consistency with other contents in the combined scene, or has occlusion consistency with other contents in the combined scene except the user interaction tool and / or the user's hand. As a further improvement of the present invention, the occlusion consistency is specifically: in the user's combined scene experience, any object point in the scene content of the combined scene except the first sub-scene has exactly the same occlusion relationship with all object points presented in the combined scene by the first sub-scene. As a further improvement of the present invention, the three-dimensional imaging interval of the first sub-scene is set as a convex interval, and the imaging intervals of other sub-scenes of the combined scene are set so that the display intervals corresponding to the other sub-scenes do not overlap with the first display interval, and the second screen only images the scene content of the other sub-scenes that meets the imaging interval constraints or only images the interactive tools or the user's hands in the other sub-scenes and the scene content that meets the imaging interval constraints. Among them, the binocular camera posture for generating the first picture does not change with the posture of the first user, specifically, the binocular camera posture and pupil distance for generating the first picture are set to constant values, so that the first sub-scene can be fixed in the first user's field of view during the display interval of the combined scene; or, the binocular camera posture for generating the first picture changes with the posture of the first user, specifically, the rotation, translation and scaling relationship between the first sub-scene coordinate system and the first user experience space coordinate system is a constant, and the pupil distance and posture of the first user are transformed through the rotation, translation and scaling relationship between the coordinate systems to obtain the posture and pupil distance of the binocular camera for generating the first picture, so that the first sub-scene can be fixed in the first user experience space during the display interval of the combined scene. To achieve the above objectives, the present invention also proposes a combined scene experience generation system based on extended reality technology, the system comprising a memory, a processor, and a combined scene experience generation method program based on extended reality technology stored in the memory, and the combined scene experience generation method program based on extended reality technology executes the steps of the method described above when executed by the processor. To achieve the above objectives, the present invention also proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is called by a processor, the steps of the combined scene experience generation method based on extended reality technology are executed as described above. The beneficial effects of the combined scene experience generation method, system and storage medium based on XR technology of the present invention are: The present invention adopts the above technical solution. When the virtual sub-scene has occlusion consistency with other sub-scenes of the combined scene, there is no need to generate and transmit depth images, and the combined scene experience can be correctly generated for the user, thereby significantly saving the bandwidth for transmitting depth images. BRIEF DESCRIPTION OF THE DRAWINGS FIG1 is a flow chart of a method for generating a combined scene experience according to a first embodiment of the present invention. FIG. 2 is a schematic diagram of the system structure of the first embodiment of the present invention. FIG. 3 is a schematic diagram of the system structure of the second embodiment of the present invention. FIG. 4 is a schematic diagram of the system structure of the third embodiment of the present invention. FIG. 5 is a schematic diagram of the system structure of the fourth embodiment of the present invention. FIG6 is a flow chart of a method for generating a combined scene experience according to a fifth embodiment of the present invention. FIG. 7 is a schematic diagram of the system structure of the fifth embodiment of the present invention. In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. DETAILED DESCRIPTION It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention. In the following, in combination with Figures 1 to 7 and the first to seventh embodiments, a method for generating a combined scene experience based on XR technology of the present invention is described in detail. Important technical terms involved in the present invention are explained below. Convex interval make is a three-dimensional convex interval, then it has the following properties: Let There are two points a and b, where the coordinates of point a are The coordinates of point b are Point c is on the line connecting points a and b and between points a and b. The coordinates of point c can be expressed as Among them, 1>λ>0, then point c must also belong to Scenarios and Scenario Examples A scene defines the objects, object states, object operation logic, and interaction logic between objects contained in a three-dimensional space; a scene instance is a program process or thread that the system calls on computer processors, memory, graphics cards and other computing resources to run in real time according to the scene definition. This program process or thread calculates the state of each object in the scene in real time, renders the screen, and responds to user interaction. When a single scene has multiple user experiences at the same time, if the computing resources available to a single scene instance cannot generate experience screens for all users in real time, it is necessary to generate multiple scene instances for this scene and allocate them to each user. These scene instances establish communication connections to synchronize the state of objects in the scene, and each scene instance generates an experience screen for the corresponding user in real time, so that each user can experience this scene together. XR scene imaging range User p in the combined scene For an immersive interactive experience, i To form a combined scene One of the sub-scenes of When experiencing i Maybe only part of the scene content needs to be in the combined scene In order to limit s i Need to be in the combined scene The scene content presented to user p is set in a three-dimensional interval, scene si Only the content in this three-dimensional interval needs to be imaged when generating the user p's experience screen. This set three-dimensional interval is the XR scene imaging interval. Among them, the imaging interval can be infinite, or it can be any three-dimensional shape interval such as a cube or a sphere. Display range of XR scenes in combined scenes Combination Scene Contains multiple sub-scenes. In order to make the sub-scenes No overlap or reduced overlap, for any sub-scene s i , when setting the combined scene Set a three-dimensional interval Ω i Used to present i , limit s i exist When setting the combined scene Coordinate system and subscenes i When the rotation and translation scaling relationship between the coordinate systems is used, according to this rotation and translation scaling relationship, the three-dimensional interval Ω i Mapped to sub-scenes i The obtained three-dimensional interval Can be used as scenes i The imaging interval, in turn, according to this rotation, translation and scaling relationship, the scene s i Imaging range Mapping to combined scenes The obtained three-dimensional interval Ω i Can be used as scenes i In the combined scene Display interval. The principle of the present invention, especially occlusion consistency, is described below. For any user p, the three-dimensional display interval of scene s in the user p experience space is Ω, and the interval outside Ω is b0 is any value in the user p's experience space. The prerequisite for the occlusion between b0 and any point c0 in Ω is that the two points are in the same line of sight of the user p. If b0 has the same occlusion relationship with all points in Ω that may be occluded, then when the depth value of point c0 is unknown, if c1 is any point in Ω that shares the same line of sight with b0 and c0, and the depth value of c1 is known, the occlusion relationship between b0 and c1 is calculated, and this occlusion relationship is the occlusion relationship between b0 and c0. The occlusion relationship between the inner point b0 and all the points in Ω that may be occluded is the same, which is called the occlusion consistency between the point b0 and Ω. First embodiment: This embodiment proposes a method for generating a combined scene experience based on XR technology, wherein the combined scene includes two or more sub-scenes, and multiple sub-scenes can be presented in three dimensions at the same time, and the first picture is an imaging picture of the first sub-scene of the combined scene, and the first picture and the second picture that images other contents of the combined scene are synthesized through occlusion calculation to obtain a user experience picture of the combined scene, wherein the pixel depth value of the first pixel that images the object point in the first picture can be represented by the depth information of the first display interval of the combined scene, and the first display interval is the three-dimensional space covered by the first sub-scene presented in the combined scene space or the user experience space, and the first pixel and the second pixel in the second picture are compared by depth value to obtain an occlusion relationship, and the first pixel and the second pixel share the user's line of sight. Among them, the first picture only images the scene content within the first imaging interval of the first sub-scene, the first imaging interval is a three-dimensional space, the first picture is a binocular stereoscopic vision image, and the mapping relationship between the first imaging interval and the first display interval satisfies stereoscopic vision. The depth information of the first display interval is represented by the surface depth value of the positive side or negative side of the first display interval facing the user, or by the depth value between the surface depth value of the positive side and the surface depth value of the negative side. In the user's combined scene experience, the first sub-scene has occlusion consistency with respect to other contents in the combined scene, or has occlusion consistency with respect to other contents in the combined scene except for the user interaction tool and / or the user's hand. The occlusion consistency specifically means that in the user's combined scene experience, any object point in the scene content of the combined scene except the first sub-scene has exactly the same occlusion relationship with all object points presented in the combined scene by the first sub-scene. The three-dimensional imaging interval of the first sub-scene is set as a convex interval, and the imaging intervals of other sub-scenes of the combined scene are set so that the display intervals corresponding to the other sub-scenes do not overlap with the first display interval, and the second screen only images the scene content of the other sub-scenes that meets the imaging interval constraints or only images the interactive tools or user's hands in the other sub-scenes and the scene content that meets the imaging interval constraints. Among them, the posture of the binocular camera for generating the first picture changes with the posture of the first user. Specifically, the rotation, translation and scaling relationship between the first sub-scene coordinate system and the first user experience space coordinate system is a constant. After the pupil distance and posture of the first user are transformed through the rotation, translation and scaling relationship between the coordinate systems, the posture and pupil distance of the binocular camera for generating the first picture are obtained, so that the first sub-scene can be fixed in the first user experience space during the display interval of the combined scene. The combined scene of this embodiment is a virtual-real mixed combined scene, which is composed of a real scene s0 and N virtual sub-scenes, where N is a natural number greater than 1, and these virtual sub-scenes are s1, s2, ..., s N As shown in FIG2 , in this embodiment, the combined scene experience generation system includes: a real scene imaging module, a virtual scene depth information calculation module, an image synthesis module, a combined scene parameter setting module, an instance of the first virtual sub-scene, an instance of the second virtual sub-scene, ..., an instance of the Nth virtual sub-scene. The instance of the first virtual sub-scene is an instance of the virtual sub-scene s1, the instance of the second virtual sub-scene is an instance of the virtual sub-scene s2, and the instance of the Nth sub-scene is an instance of the virtual sub-scene s NFor video perspective MR head-mounted displays, the real scene imaging module generates a binocular stereoscopic vision picture of the user and the corresponding depth image of the real scene through the camera and other optoelectronic devices on the user's MR head-mounted display terminal. For optical perspective MR head-mounted displays, the real scene imaging module does not need to generate a binocular stereoscopic vision picture but needs to generate a binocular stereoscopic depth image through the camera and other optoelectronic devices on the user's MR head-mounted display terminal (it can also be considered that a binocular stereoscopic vision picture is generated, and the color value of each pixel in the binocular stereoscopic vision picture image is 0, that is, a completely black image). The instance of the virtual sub-scene receives the user's real-time posture and interactive operation information sent by the user interface, responds to the received interactive operation information, and renders the user's binocular stereoscopic vision experience picture of the scene imaging interval in real time under the user's real-time posture, and sends it to the image synthesis module. The image synthesis module performs occlusion calculations between the pictures based on the corresponding depth image of the real scene stereoscopic vision picture, the depth image generated by the virtual sub-scene instance, or the depth information of each virtual sub-scene in the combined scene calculated by the virtual scene depth information calculation module, and synthesizes the user's combined scene experience picture, and displays it to the user through the user interface. The user or the system can set the imaging interval of each virtual sub-scene in the combined scene, the rotation, translation and scaling relationship between the coordinate system of each virtual sub-scene and the combined scene coordinate system, and other parameters through the combined scene parameter setting module. The real scene imaging module, the virtual scene depth information calculation module, the image synthesis module, and the combined scene parameter setting module are all deployed on the user terminal, and the instance of the first virtual sub-scene, the instance of the second virtual sub-scene, ..., the instance of the Nth virtual sub-scene are deployed and run on the cloud rendering server in the cloud. Combination Scene It consists of a real scene s0 and N virtual sub-scenes, where N is a natural number greater than 1. These virtual sub-scenes are s1, s2, ..., s N , generating a combined scene for user p in real time Immersive experience, in which, for simplicity, real scene s0 can be combined with the scene As shown in FIG1 , the experience generation method of this embodiment includes the following steps: Step S00: setting the combined scene experience generation parameters, including setting the imaging interval of each virtual sub-scene, the pupil distance of the user p in each virtual sub-scene, the rotation, translation, and scaling transformation relationship between the combined scene coordinate system and the coordinate system of each virtual sub-scene, the display interval of each sub-scene in the combined scene, and whether the scene instances of each sub-scene generate corresponding depth images while rendering the generated experience screen; Step S10: Generate binocular stereoscopic visual experience images of each sub-scene of the combined scene, wherein all arbitrary virtual sub-scenes si The example obtains the position of user p in its coordinate system in real time, and uses this real-time position and s i The combination of imaging interval, user p in s i The pupil distance in the scene space generates the user p's response to the scene s i Binocular stereoscopic vision experience picture; Step S20: performing occlusion calculation on the binocular stereoscopic vision experience images of each sub-scene to synthesize the combined scene experience image and display it to the user; Step S30: The combined scene receives interactive input from user p and determines whether the interactive input is an interaction with a sub-scene. If it is determined that the interactive input is an interaction with any sub-scene s, i interaction, then this interaction input is converted into scene s i Interactive input in coordinate system, s i Respond to the converted interactive input. The following is a detailed introduction to the specific implementation of each step: Step S00 is specifically implemented as follows: In the combined scene parameter setting module, set the combined scene parameters: set each virtual sub-scene s1, s2, ..., s N The imaging intervals are Ω1, Ω2, ..., Ω N ; Set the three-dimensional interval Ω′0 in the combined scene where the virtual scene cannot be deployed. For example, the interval occupied by the wall, equipment and other items in the real scene cannot be deployed with the virtual scene; Set the user p in each virtual sub-scene s1, s2, ..., s N The pupil distances are d1, d2, ..., d N ; The user's pupil distance in the combined scene is Set each virtual sub-scene s1, s2, ..., s N In the combined scene The display intervals are Ω1′, Ω′2, ..., Ω′ N ; Set the rotation, translation and scaling relationship of each virtual sub-scene coordinate system to the combined scene coordinate system. For any virtual sub-scene s i (1≤i≤N), scene s i The rotation, translation and scaling relationship between the coordinate system and the combined scene coordinate system is: i , β i , α i , translate along the X, Y, and Z axes respectively The scaling factor of the three axes Z, X, and Y is λ i; Set whether each virtual sub-scene instance needs to generate a depth image, that is, whether to generate a corresponding depth image at the same time when rendering and generating a binocular stereoscopic visual experience picture. The above parameters are not completely independent and need to meet the following constraints: the pupil distance used by the user p's terminal to generate a real scene stereoscopic visual experience screen or depth image is d0 (d0 needs to be as close to the real pupil distance of user p as possible), and the user Pupillary distance The pupil distance for any virtual sub-scene s i ,have Any virtual sub-scenes i , its imaging interval Ω i For any point b0 in the i Coordinate system to combined scene After the coordinate system is transformed by rotation, translation and scaling, the point Belong to Ω i ′; Display interval Ω1′, Ω′2, ..., Ω′ N There is no overlap between them, Ω1′, Ω′2, ..., Ω′ N do not overlap with Ω′0; for any sub-virtual scene s i , if it displays the interval Ω i ′for Ω′0,Ω1′,Ω′2,...,Ω′ N Divide Ω i ′, all other intervals have occlusion consistency, then set s i The scene instance does not need to generate the corresponding depth image while rendering the binocular stereoscopic visual experience picture, otherwise set s i The scene instance needs to generate the corresponding depth image while rendering the binocular stereoscopic visual experience picture. Let the set of virtual sub-scenes that do not need to generate depth images for the scene instance be Λ, and the set of virtual sub-scenes that need to generate depth images be The first sub-scene is the scene in Λ. Among them, judging any display interval Ω i ' vs Ω' j Is there a method for occlusion consistency: In Ω i ′ and Ω′ j Under the premise of non-overlap, if Ω i ′ is a convex interval, then Ω i ' vs Ω' j With occlusion consistency, if Ω i′ is a non-convex interval, first obtain the interval Ω surrounding this non-convex interval i The minimum convex interval of ′ like and Ω′ j If there is no overlap, Ω′ j With occlusion consistency, there is Ω i ' vs Ω' j It has occlusion consistency, otherwise it does not have occlusion consistency. Among them, we get the non-convex interval Ω i The minimum convex interval of ′ The method can be determined manually or automatically by the system. The automatic determination method of the system is: traverse Ω i For any pair of boundary points on the boundary, if any point between all the points does not belong to Ω i ′, are all incorporated into Ω i 'You get Step S10 is specifically implemented as follows: Any time t j , the pose of user p in the combined scene coordinate system is [W j Q j ], each virtual sub-scene s1, s2, ..., s N In this example, the real-time pose of user p is received from the user interface, and the pose of the user in each virtual sub-scene coordinate system is calculated according to the rotation, translation and scaling relationship between each coordinate system and the combined scene coordinate system. For s1, s2, ..., s N Any virtual scene in i , according to s i Coordinate system and combined scene The rotation, translation and scaling relationship of the coordinate system is calculated to obtain the user p in s i The pose in the coordinate system is Calculating the position and posture of the user p in the virtual sub-scene coordinate system by the rotation, translation and scaling relationship of the coordinate system is well known to those skilled in the art and will not be described in detail here. N Any virtual scene in i , according to the user in s i The pupil distance d i , pose The imaging interval Ω i The scene content in the image is imaged to generate an immersive binocular stereoscopic visual experience screen, where s iThe left and right eyes of the stereoscopic vision picture are respectively and It indicates that, and It is necessary to identify which pixels correspond to empty scene contents, for example, the values ​​of pixels with empty scene contents in the experience screen image are set to a specific value τ null (e.g. blue), or it can be identified by a specific array, each element in this array corresponds to a pixel in the image, and the element can only take the value of 0 or 1 to identify whether the corresponding pixel is imaging the scene content is empty. In the embodiment of the present invention, the experience screen image is selected and The value of the pixel with empty scene content is set to a specific value τ null . For a set of virtual sub-scenes that need to generate corresponding depth images while rendering and generating binocular stereoscopic visual experience images Any scene in k , the left and right eye images of the stereoscopic visual experience screen generated by rendering are and It is also necessary to generate the corresponding depth image and The depth value of the pixel corresponding to the empty scene content is set to a specific value δ, which can be infinite. and That is s k In the combined scene The depth image in the coordinate system, where λ k For scenes k Coordinate system to combined scene In this embodiment, the imaging module of the XR terminal collects the stereoscopic visual picture of the real scene s0 in real time to generate the left and right eye stereoscopic visual images. and (For the optical perspective MR helmet, it can be considered that all pixel RGB color values ​​are generated

[0000] Left and right eye stereo vision images), corresponding to the depth image and In the embodiments of the present invention, it is assumed that the imaging field of view and resolution of all images are the same, so that only pixels of the same image coordinates between different images will have an occlusion relationship; if the imaging field of view and resolution between different images are different, the imaging field of view and resolution between the images can be converted to the same by interpolation or sampling, which is a common method used by technicians in the field. Each virtual sub-scene transmits the generated stereoscopic visual experience screen image to the image synthesis module in real time. If a depth image is generated synchronously, the depth image also needs to be transmitted to the image synthesis module in real time. Step S20 is specifically implemented as follows: The image synthesis module receives a stereoscopic visual experience picture binocular stereoscopic visual image generated by the real scene and the virtual sub-scene instance contained in the combined scene, and also receives The virtual sub-scenes in the set have depth images corresponding to the real scene experience screen, while the virtual sub-scenes in the set Λ do not transmit depth images. In the combined scene The display interval in is Real-time calculation The depth information can be used Surface depth value representation of the positive side facing the user The depth information can also be Surface depth value representation of the dark side facing the user The depth information of the positive side or the negative side can be represented by any value between the surface depth value of the positive side and the surface depth value of the negative side. depth information. The depth information can also be represented by a depth image. At any time t j , The depth information of the left eye depth image of user p is used Indicates that the right eye depth image is represented by Indicates. Surface depth value representation of the positive side (facing the user p) When the depth information of the virtual scene is obtained, the calculation method is as follows: the virtual scene depth information calculation module reads the depth information from the combined scene parameter setting module. In the combined scene The display interval in is User is combining scenes The pupil distance is At any time t j , the pose of user p in the combined scene coordinate system is read from the user interface as [W j Q j ], clear the combined scene Only keep The surface of the user p is oriented with the surface normal facing outwards. The left and right eye images of the user p are generated. The images are only The surface imaging of the positive part, the corresponding depth image is and The depth value of the pixel that does not image the surface is represented by a certain value δ; Surface depth value representation of the dark side (facing away from the user p) When the depth information of the virtual scene is obtained, the calculation method is as follows: the virtual scene depth information calculation module reads the depth information from the combined scene parameter setting module. In the combined scene The display interval in is User is combining scenes The pupil distance is At any time t j , the pose of user p in the combined scene coordinate system is read from the user interface as [W j Q j ], clear the combined scene Only keep The surface of the user p is oriented with the surface normal pointing inwards. The left and right eye images of the user p are generated. The images are only The surface imaging of the dark side, the corresponding depth image is and The depth value of the pixel whose surface is not imaged is represented by a certain specific value δ; the value of each pixel in the depth image can also be arbitrarily between the corresponding surface depth value of the positive side part and the surface depth value of the negative side part. For combined scenes Any sub-scene in the virtual sub-scene set Λ If any pixel τ of the binocular stereoscopic vision experience screen image has an image of an object point, In the combined scene Display interval The depth information of represents the depth of τ, and then The depth values ​​of the pixels in the user's line of sight are compared with the other components of the stereoscopic vision picture to obtain the occlusion relationship between the pixels. Among them, the display interval is calculated in real time according to the user's posture. The depth information can be used to The sun side or the dark side imaging corresponds to a depth map representation, and can also be represented by any depth map having a depth value between the sun side or the dark side imaging corresponding to a depth map representation. If any pixel τ of the experience screen images an object point, the depth value of the pixel point where the display interval depth image and τ share the user's line of sight is represented as the depth value of τ. The image synthesis module receives in real time the binocular stereoscopic visual experience screen images generated by the real scene and the virtual sub-scene contained in the combined scene, and The depth image corresponding to the virtual sub-scene in the set and the real scene experience screen is also read in real time from the virtual scene depth information calculation module to obtain the depth information of all scenes in the combined scene. After the depth information representation value of the display interval is obtained, image synthesis can be performed. The specific synthesis method is: at any time t j , the left and right eye depth images of the received stereoscopic visual experience picture of the real scene s0 are and Corresponding depth image and For collections Any scene in k , the stereoscopic vision images received by the left and right eyes are and The corresponding depth image is and (s k In the combined scene The depth image in the coordinate system is and λ k For scenes k Coordinate system to combined scene Coordinate system scaling factor), for any scene in the set Λ The received stereoscopic vision images are and The corresponding display interval depth information representation image read from the virtual scene depth information calculation module is: and The combined scene after synthesis The left eye image of the stereoscopic visual experience screen of user p is The right eye image is The corresponding depth images are Initial Setup Iterating over a collection For all sub-scenes in k , traverse All pixels, for any pixel if but: Traversal All pixels, for any pixel if but: Traverse all sub-scenes in the set Λ, for any sub-scene Traversal All pixels, for any pixel If the pixel has an image of the scene content, Find the corresponding depth value in When this depth value is less than hour, Block out Original pixels, specifically: and but and Traversal All pixels, for any pixel if and but: Notice: The value is τ null The corresponding depth value cannot be and The value in , its depth value should be considered infinite. After completing the traversal, the resulting image and That is, user p pairs of combination scenarios The binocular stereoscopic vision experience screen is displayed to the user p through the user interface. Step S30 is specifically implemented as follows: User p uses the user interface to combine scenes Perform interactive operations and generate interactive operation command a. The position parameter of the interactive operation is The attitude angle parameter is User interface traversal combined scene All virtual sub-scenes in the display interval Ω1′, Ω′2, ..., Ω′ N , if the coordinate value is determined to be The point is in the display interval Ω k ', where Ω k ′ is the virtual sub-scene s k The display interval is based on s k Coordinate system and combined scene The rotation, translation and scaling relationship of the coordinate system Pose parameters in the coordinate system and Convert to s k The pose parameters in the coordinate system are assigned to the pose parameter components of the interactive operation command a, and the interactive command a with the converted pose parameters is sent to the virtual sub-scene s k Scene instance, virtual sub-scene s k The scene instance responds to this interactive operation command. In the embodiment of the present invention, the imaging intervals of all virtual sub-scenes may also be set as convex intervals, so that all virtual sub-scenes do not need to transmit depth images to the image synthesis module. The beneficial effects of this embodiment are as follows: This embodiment is applicable to combined scenes of virtual-real fusion, especially to MR head displays. When the virtual sub-scene has occlusion consistency in the display interval of the combined scene, the display intervals of other sub-scenes, and the three-dimensional interval in which the virtual scene cannot be deployed in the combined scene, there is no need to transmit the depth image, only the stereoscopic visual experience picture needs to be transmitted to the terminal, so that the occlusion calculation image can be correctly synthesized with other contents of the combined scene. In particular, when the imaging intervals of all virtual sub-scenes are set to convex intervals, all virtual sub-scenes do not need to transmit depth images to the user terminal, so as to generate an immersive experience of the virtual fusion combined scene for the user, thereby significantly saving network bandwidth. Second embodiment: On the basis of the first embodiment, the system configuration is adjusted to obtain the second embodiment. As shown in Figure 3, the virtual scene depth information calculation module is adjusted to be deployed on the cloud server, and a virtual scene image synthesis module is added. The virtual scene image synthesis module is also deployed on the cloud server. The stereoscopic visual experience picture generated by each virtual sub-scene is first subjected to occlusion calculation in the virtual scene image synthesis module to obtain a synthetic image containing only the contents of all virtual sub-scenes. The synthetic image of all virtual sub-scene contents is transmitted to the virtual-real image synthesis module on the user terminal. The virtual-real image synthesis module performs occlusion calculation on the synthetic image containing all virtual sub-scene contents and the real scene user stereoscopic visual experience picture to obtain the final virtual-real fusion combined scene experience picture, which is displayed to the user through the user interface. Referring to the image synthesis method of the first embodiment, the specific implementation method for generating a synthetic image containing only the contents of all virtual sub-scenes is: at any time t j , let the left eye synthetic image of the stereoscopic visual experience screen of user p, which only contains all the virtual sub-scene contents, be The right eye composite image is The corresponding depth images are Initial Setup in, is the virtual sub-scene i The generated left and right eye stereoscopic visual experience picture of user p, if s i The scene instance has a corresponding depth image generated synchronously Then the initial setting If the corresponding depth image is not generated synchronously, s is calculated from the virtual scene depth information calculation module. i Depth information representation value displayed in the combined scene: Depth image Traversal For all pixels, if any pixel The value is τ null ,but (δ can be as large as possible, even infinite), otherwise Traversal For all pixels, if any pixel The value is τ null ,but (δ can be as large as possible, even infinite), otherwise , and then traverse the set Λ and All sub-scenes in the occlusion calculation image synthesis are performed. The occlusion calculation image synthesis process is the same as the first embodiment and will not be repeated. After synthesis, the user p stereoscopic visual experience screen image containing only the contents of all virtual sub-scenes is obtained. And the corresponding depth image Transmitted to the virtual-real image synthesis module, and the stereoscopic visual experience image of the real scene generated in real time And the corresponding depth image Perform occlusion calculation and synthesis to obtain a virtual-real fusion combined scene User experience screen. Beneficial effects: Compared with the first embodiment, the occlusion calculation and image synthesis work between virtual sub-scenes is placed in the cloud, so this embodiment will reduce the amount of calculation for image synthesis on the terminal. The cost is that the cloud performs image synthesis once and the terminal also performs image synthesis once, which may increase latency. Third embodiment: The combined scene is changed to be composed of only virtual sub-scenes and no longer includes real scenes. The system structure is shown in Figure 4, which only removes the real scene imaging module compared with the system diagram of the first embodiment in Figure 2. Compared with the experience generation method of the first embodiment, the experience generation method of this embodiment removes the calculation work related to the real scene, and its step 20 is the same as the second embodiment, which will not be repeated here. Beneficial effects: Can be adapted to virtual reality (VR) terminals. Fourth embodiment: The combined scene is also composed of only virtual sub-scenes and no longer contains real scenes. The system structure is shown in Figure 5, which only removes the real scene imaging module compared with the system diagram of the second embodiment in Figure 3. The experience generation method of this embodiment is the same as the experience generation method of the second embodiment except that the calculation work related to the real scene is removed, and it will not be repeated here. Beneficial effects: It can be adapted to VR terminals. Compared with the fourth embodiment, the occlusion calculation and image synthesis work between virtual sub-scenes is placed in the cloud, so this embodiment will completely eliminate the amount of calculation required for image synthesis in the terminal. Fifth embodiment: The first picture is an imaged picture of the first sub-scene of the combined scene. The first picture and the second picture that images other contents of the combined scene are synthesized by occlusion calculation to obtain the user experience picture of the combined scene. The first pixel of the first picture that images the object point can be represented by the depth information of the first display interval of the combined scene. The first display interval is the three-dimensional space covered by the first sub-scene in the combined scene space or the user experience space. The first pixel and the second pixel in the second picture are compared by depth value to obtain the occlusion relationship. The first pixel and the second pixel share the user's line of sight. The first sub-scene is any one of a real scene, a virtual scene, and a virtual-real fusion scene. The binocular camera posture generated for the first picture does not change with the posture of the first user. Specifically, the binocular camera posture and pupil distance generated for the first picture are set to constant values, so that the first sub-scene can be fixed in the first user's field of view in the display interval of the combined scene. In this embodiment, since the binocular camera posture generated for the first picture does not change with the posture of the user, the picture of the first sub-scene can be used to synthesize the combined scene experience picture of different users, and the user can broadcast live. In this implementation, the first sub-scene is called a live broadcast scene. As shown in FIG7 , the live broadcast system of this embodiment is composed of an anchor end, an audience end, a live broadcast parameter setting module, and a live broadcast scene display interval depth information calculation module. The anchor end runs a live broadcast scene instance, obtains live broadcast parameters from the live broadcast parameter setting module, generates a live broadcast scene binocular stereoscopic vision live broadcast screen image in real time according to the live broadcast parameters, and can receive interactive input from the audience end and respond to the interactive input of the audience. The system, anchor, and audience can set live broadcast parameters through the live broadcast parameter setting module, and the changed live broadcast parameters will be sent to the anchor end, the audience end, and the live broadcast scene display interval depth information calculation module. The live broadcast scene display interval depth information calculation module calculates the display interval depth information according to the display interval of the live broadcast scene in the audience's field of view. When the audience end initially receives the live broadcast picture from the anchor end or the live broadcast parameters change, the audience end obtains the live broadcast scene display interval depth information from the live broadcast scene display interval depth information calculation module. The audience end receives the live broadcast scene stereoscopic vision live broadcast picture from the anchor end, and performs occlusion calculation synthesis on this live broadcast picture and the stereoscopic vision picture of other contents in the audience experience space, synthesizes the experience picture of watching live broadcast while interacting with other contents and displays it to the user. As shown in FIG6 , the construction method of this embodiment includes the following steps: Step S00: setting live broadcast parameters, including setting the live broadcast imaging interval of the live broadcast scene and setting the pupil distance and posture of the binocular camera for generating a stereoscopic live broadcast picture; Step S10: according to the live imaging interval of the live scene and the pupil distance and posture of the binocular camera for generating the stereoscopic live picture, a stereoscopic live picture is generated by real-time rendering, and the stereoscopic live picture is sent to each viewer terminal; Step S20: any viewer terminal receives a stereoscopic visual live picture, and the stereoscopic visual live picture is combined with a stereoscopic visual picture of other experience contents in the viewer experience space to form a stereoscopic visual picture for viewing the live broadcast while experiencing other contents; Step S30: converting the audience's interactive operations on the live scene in their experience space into interactive operations within the live scene, and the live scene responds to the converted interactive operations. Among them, when the step S20 performs image synthesis, the occlusion calculation method of the stereoscopic visual live broadcast image and the stereoscopic visual image of other experience contents in the audience experience space is as follows: if any pixel τ of the live broadcast image has an image of an object point, the depth of τ is represented by the depth information of the live broadcast scene in the audience's field of view display interval, and then the depth value is compared with the pixels of the stereoscopic visual image of other experience contents in the audience experience space that share the audience's line of sight to obtain the occlusion relationship between the pixels. The specific implementation of each step is described in detail below. Step S00 is specifically implemented as follows: In the live broadcast parameter setting module, set the benchmark audience The pupil distance is The live imaging area defined in s is Ω m The rotation, translation and scaling relationship between the live scene coordinate system and the reference audience field of view coordinate system is: x ,t y ,t z , the scaling factor of the three axes Z, X, and Y is λ m , the pupil distance of the benchmark audience in the live broadcast scene is The position of the reference audience in its field of view coordinates is Generally default The pose in the live scene is The display interval of the live scene in the reference audience field of view is Ω user These parameters are not completely independent and need to satisfy the following constraints: For Ω m Any point b0 in the scene is transformed according to the rotation, translation and scaling relationship from the live scene coordinate system to the reference audience field of view coordinate system. Must belong to Ω user , and The rotation, translation and scaling relationship between the live scene coordinate system and the reference audience field of view coordinate system must also be satisfied. The live parameters such as posture and scaling can be reset during the live broadcast as needed. and posture It is the pupil distance and posture of the binocular camera that generates the stereoscopic vision live broadcast picture. Step S10 is specifically implemented as follows: The anchor terminal obtains the live imaging interval Ω of the live scene s from the live parameter setting module m , and the benchmark audience The pose value of s in the live scene And pupil distance Under this pose and pupil distance value, the real-time rendering scene s imaging interval Ω m The binocular stereo vision picture. Among them, is the left eye image of the stereoscopic vision picture, is the right eye image of the stereoscopic picture, and When any pixel in the scene s is not imaged for any object point in the scene s, the pixel value is assigned with a special color value C0 (for example, blue), or can be identified with a specific array, in which each element corresponds to a pixel in the image, and the element can only take the value of 0 or 1 to identify whether the corresponding pixel images the object point. In this embodiment of the present invention, and Any pixel in the scene s m When any object point is imaged, the pixel value is assigned with a special color value C0. The stereoscopic visual image is sent to each viewer. Compared with the inventor's previous invention patent application "Method, system and medium for constructing immersive interactive live broadcast based on XR technology" (application number: CN202210906282.6), the generated stereoscopic visual image no longer needs to contain depth information, which significantly reduces the bandwidth requirement for transmitting stereoscopic visual images. Step S20 is implemented as follows: Any audience kThe corresponding live broadcast end receives the stereoscopic live broadcast picture of s in real time. When the left and right eye pictures of the live broadcast picture are displayed to the left and right eyes of the audience respectively, the audience h k The three-dimensional imaging interval Ω of the scene s will be directly obtained m However, when the audience is watching the live broadcast, they are also experiencing other immersive experiences (immersive experiences can be AR, VR, MR and other XR experiences). At this time, the live stereoscopic visual images need to be synthesized with the stereoscopic visual experience images of other contents to generate the audience h k A complete stereoscopic visual experience picture is created to integrate the live scene into the audience's immersive experience space. When synthesizing the live stereoscopic picture with the stereoscopic visual experience picture of other content, it is necessary to perform occlusion calculation between images. Only pixels on the same line of sight of the user will be occluded. In the embodiment of the present invention, it is assumed that the same field of view and resolution are used when rendering and generating all stereoscopic visual images and depth images. When the field of view and resolution are different, the image field of view and resolution can be made the same through interpolation. Therefore, pixels with the same image coordinates between multiple left-eye or right-eye pictures are on the same line of sight of the user. The stereoscopic visual live broadcast picture and the stereoscopic visual pictures of other experience contents in the audience's experience space are synthesized into a stereoscopic visual picture for watching stereoscopic visual live broadcast while experiencing other content. The specific implementation is as follows. When generating a stereoscopic visual image of other experience contents in the audience's immersive experience space, the three-dimensional display interval of the live scene in the audience's experience space is calculated, and the content of other experience contents in the audience's experience space in this three-dimensional interval is not imaged (the content of other experience contents in the audience's experience space in this three-dimensional interval can also be imaged, but the occlusion of the content of other experience contents in the audience's experience space in this three-dimensional interval with the live scene cannot be ensured to be correct). Obtain the display interval Ω of the live scene under the reference audience field of view from the live parameter setting module user , for any viewer user h k , whose pupil distance is d k Due to the difference in pupil distance, the live virtual scene s is k The display interval in the field of view is That is Ω user The coordinates of all points in the scale are After scaling, we get For the detailed derivation process, please refer to paragraph 125 of the specification of the invention patent application "Method, system and medium for constructing immersive interactive live broadcast based on XR technology" (application number: 202210906282.6). For any user h k , while watching the live broadcast of scene s, you can also experience other content, such as kFor an immersive experience. As has been calculated above, the live virtual scene is k The three-dimensional display range in the field of view is Audience experience space Other experience content in the three-dimensional area The contents inside are not imaged. The live scene display interval depth information calculation module calculates the reference audience field of view Ω user The depth information can be obtained by using Ω user The surface depth value of the positive side represents Ω user The depth information can also be Ω user The surface depth value of the dark side represents Ω user The depth information of the positive surface part or the negative surface part can be represented by any value between the surface depth value of the positive surface part and the surface depth value of the negative surface part. user The depth information of the reference audience field of view Ω user The depth information of can also be represented by a depth image, Ω user Depth information for benchmark viewers The left eye depth image is used Indicates that the right eye depth image is represented by The depth image and the stereoscopic vision image of the live scene use the same field of view and resolution. For any user h k , due to its pupil distance d k Interpupillary distance The difference between the live broadcast scene and the k The display range in the field of view is correspond Depth information available Indicates. Without changing, Unchanged, can be reused. If any pixel τ of the live broadcast screen has an image of an object point (the image of an object point means that the imaging content is not empty), the depth information of the live broadcast scene in the audience's field of view display interval is used to represent the depth of τ, and then the depth value is compared with the pixels of the stereoscopic visual screen of other experience contents in the audience's experience space that are shared by the audience's line of sight to obtain the occlusion relationship between the pixels. The image synthesis method is specifically as follows. Let, in addition to s, user h k The left eye image of the experience screen generated by other content in the experience space is The right eye image is In h k The corresponding depth value of the field of view is and The depth value of the pixel that does not image the object point is represented by a specific value δ. The received live stereoscopic image of scene s and Need and Synthesized as user h k The complete stereoscopic visual picture of watching live broadcast and experiencing other contents in the experience space at the same time. When synthesizing pictures, occlusion calculation needs to be performed. Let the left eye picture of the synthesized experience picture be Right eye picture The corresponding depth image is initialization Traverse any pixel in the image, if there is and but Otherwise, no processing is done, that is, when Any pixel in If the value of this pixel is not the special color value C0, that is, the object point is imaged, and Corresponding pixels will be blocked by the surface of the 3D display area, then the pixel value of the synthesized image is the pixel value of the live screen; similarly, if there is and but Otherwise, no processing is performed. After all pixels in the image are traversed according to the above calculation, the picture synthesis is completed. The synthesized picture is displayed to the audience h k look. Compared with the inventor's previous invention patent application "Immersive interactive live broadcast construction method, system and medium based on XR technology" (application number: 202210906282.6), the implementation of this step does not need to use the depth information of the transmitted live stereoscopic visual picture when performing occlusion calculation. Instead, the occlusion calculation is performed using a depth map representing the depth information of the three-dimensional display interval of the live scene. This depth map is unchanged and can be reused when the three-dimensional display interval of the live scene remains unchanged. For the specific implementation of step s30, please refer to step S50 (paragraphs 127-140 of the specification) of the patent application "Immersive interactive live broadcast construction method, system and medium based on XR technology" (application number: CN202210906282.6), which will not be repeated here. Sixth embodiment: The preferred embodiment 5 is improved to obtain the preferred embodiment 6. Compared with the preferred embodiment 5, step S20 has the following differences compared with the preferred embodiment 5: the audience h k In addition to the live broadcast scene, the experience space m In addition, other experience content is in the three-dimensional area The content in the image is not imaged, but the interactive tool or hand can be imaged in this interval. user The depth information is limited by Ω user The depth value of the dark side is represented. The depth image of the dark side of the live scene facing the audience at the boundary of the display interval in the audience's field of view is calculated. If any pixel τ of the live screen has an image of an object point, the depth image of the dark side of the display interval boundary and the depth value of the pixel point of the audience's line of sight τ are represented as the depth value of τ. When imaging, it will not be blocked by the live scene content. Beneficial effect: Except for the interactive tools or hands, other contents in the audience experience space and the live broadcast scene will be correctly blocked. When the interactive tools or hands enter the live broadcast scene display area, they will be presented, and the audience can interact with the live broadcast scene through the interactive tools or hands. In addition, to achieve the above-mentioned objectives, the present invention proposes a combined scene experience generation system based on XR technology, the system comprising a memory, a processor, and a combined scene experience generation method program based on XR technology stored on the processor, and the combined scene experience generation method program based on XR technology executes the steps of the combined scene experience generation method described in all the above embodiments when run by the processor. To achieve the above objectives, the present invention further proposes a computer-readable storage medium, on which a computer program is stored. When the computer program is called by a processor, the steps of the combined scene experience generation method described in all the above embodiments are executed. The present invention proposes a method, system and medium for generating a combined scene experience based on XR technology, wherein the experience generating method comprises: a first picture is an imaging picture of a first sub-scene of a combined scene, and the first picture and a second picture imaging other contents of the combined scene are synthesized by occlusion calculation to obtain a user experience picture of the combined scene, wherein the first pixel imaging an object point in the first picture has a pixel depth value that can be represented by the depth information of the first display interval of the combined scene, and the first display interval is the three-dimensional space covered by the first sub-scene presented in the combined scene space or the user experience space. The present invention enables the sub-scene in the combined scene to correctly generate a combined scene experience for the user without generating and transmitting the depth image of the corresponding picture after generating a stereoscopic visual picture, thereby significantly saving the bandwidth for transmitting the depth image. The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Equivalent structures or equivalent process changes made by the scheme using the contents of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for generating a combined scene experience based on extended reality technology, characterized in that: The combined scene includes two or more sub-scenes, and multiple sub-scenes can be presented in three dimensions at the same time. The first picture is an imaging picture of the first sub-scene of the combined scene. The first picture and the second picture that images other contents of the combined scene are synthesized through occlusion calculation to obtain a user experience picture of the combined scene, wherein the pixel depth value of the first pixel that images the object point in the first picture can be represented by the depth information of the first display interval of the combined scene. The first display interval is the three-dimensional space covered by the first sub-scene presented in the combined scene space or the user experience space. The first pixel and the second pixel in the second picture are compared by depth value to obtain an occlusion relationship, the first pixel and the second pixel share the user's line of sight, and the first sub-scene is any one of a real scene, a virtual scene, and a virtual-real fusion scene.

2. The generation method according to claim 1, characterized in that: The first picture only images the scene content within the first imaging interval of the first sub-scene, the first imaging interval is a three-dimensional space, the first picture is a binocular stereoscopic vision image, and the mapping relationship between the first imaging interval and the first display interval satisfies stereoscopic vision.

3. The generation method according to claim 2, characterized in that: The depth information of the first display interval is represented by the surface depth value of the positive side or negative side of the first display interval facing the user, or by the depth value between the surface depth value of the positive side and the surface depth value of the negative side.

4. The generation method according to claim 3, characterized in that: In the user's combined scene experience, the first sub-scene has occlusion consistency with respect to other contents in the combined scene, or has occlusion consistency with respect to other contents in the combined scene except for the user interaction tool and / or the user's hand.

5. The generation method according to claim 4, characterized in that: The occlusion consistency specifically means that in the user's combined scene experience, any object point in the scene content of the combined scene except the first sub-scene has exactly the same occlusion relationship with all object points presented in the combined scene by the first sub-scene.

6. The generation method according to claim 5, characterized in that: The three-dimensional imaging interval of the first sub-scene is set as a convex interval, and the imaging intervals of other sub-scenes of the combined scene are set so that the display intervals corresponding to the other sub-scenes do not overlap with the first display interval, and the second screen only images the scene content of the other sub-scenes that meets the imaging interval constraints or only images the interactive tools or user's hands in the other sub-scenes and the scene content that meets the imaging interval constraints.

7. The generation method according to any one of claims 1 to 6, characterized in that: The binocular camera posture for generating the first picture does not change with the posture of the first user. Specifically, the binocular camera posture and the pupil distance for generating the first picture are set to constant values, so that the first sub-scene can be fixed in the first user's field of view during the display interval of the combined scene.

8. The generation method according to any one of claims 1 to 6, characterized in that: The position and posture of the binocular camera for generating the first picture changes with the position and posture of the first user. Specifically, the rotation, translation and scaling relationship between the first sub-scene coordinate system and the first user experience space coordinate system is a constant. After the pupil distance and position and posture of the first user are transformed through the rotation, translation and scaling relationship between the coordinate systems, the position and pupil distance of the binocular camera for generating the first picture are obtained, so that the first sub-scene can be fixed in the first user experience space during the display interval of the combined scene.

9. A combined scene experience generation system based on extended reality technology, characterized in that: The system includes a memory, a processor, and a combined scene experience generation method program based on extended reality technology stored on the processor. When the combined scene experience generation method program based on extended reality technology is executed by the processor, the steps of the method described in any one of claims 1 to 8 are executed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is called by the processor, the steps of the combined scene experience generation method based on extended reality technology described in any one of claims 1 to 8 are executed.

Images