XR combined scene construction method, system and medium based on image cropping

Abstract

The present invention discloses a method for constructing a combined scene of extended reality technology (XR) based on image cropping, characterized in that the part of the binocular stereoscopic vision picture of the first sub-scene that meets the image cropping constraint is subjected to image synthesis with the binocular stereoscopic vision pictures of other contents in the combined scene to obtain the binocular stereoscopic vision picture of the first user of the combined scene, wherein the image cropping constraint is an imaging interval constraint and / or a display interval constraint, and the first sub-scene is one of a virtual scene, a virtual-reality fusion scene, and a real scene. The present invention does not need to limit the imaging interval when the first sub-scene is imaged, so that the experience picture rendering conflicts of multiple users of the first sub-scene will not be caused due to the limitation of the imaging interval, especially a group of binocular stereoscopic vision pictures of the first sub-scene can be cropped with different contents for different users respectively, so that the different experience requirements of a large number of users for the first sub-scene can be met at the same time.

Description

Technical Field

[0001] The present invention relates to the field of XR technology, and in particular to an XR combined scene construction method, system and medium. Background Art

[0002] Virtual reality (VR), augmented reality (AR), mixed reality (MR) and other technologies are interconnected and integrated, collectively referred to as extended reality technology (XR). A combined scene can be constructed by combining two or more single-person experience scenes and / or multi-person collaborative experience scenes. The combined scene is a sub-scene of the combined scene, and the sub-scene can be one or more of a virtual scene, a real scene, and a virtual-real mixed scene. When a user experiences a combined scene, the user can simultaneously view the stereoscopic three-dimensional information of all sub-scenes in the combined scene, and can further have a stereoscopic interactive experience with all sub-scenes. The combined scene can break the isolation between scenes and will play an important role in XR application fields such as teaching.

[0003] One very important technical path in the method of constructing a combined scene is: the sub-scene generates a binocular stereoscopic vision picture, and the binocular stereoscopic vision picture generated by the sub-scene is synthesized with the binocular stereoscopic vision pictures of other sub-scenes in the combined scene after occlusion processing to obtain the user's combined scene experience picture. References [1]-[7] all include the construction of combined scenes, among which the construction of new dimensional space and ubiquitous training campus in references [1][2] are both specific forms of combined scenes. In the combined scene construction method in references [1-5], imaging intervals are set for each sub-scene, and the user's posture in each sub-scene is calculated according to the user's real-time posture. Further, the user's stereoscopic vision experience picture of each sub-scene imaging interval is rendered under the calculated user posture, and then the experience pictures of each sub-scene are synthesized to obtain the combined scene user experience picture. In the literature [6][7], imaging intervals are also set for sub-scenes, and each sub-scene images the imaging interval separately. However, the user's imaging posture in the sub-scene is not updated according to the user's real-time posture, but can be a pre-set fixed value. Therefore, in the synthesized combined scene experience screen, the display interval of the sub-scene in the user's field of view does not change with the user's real-time posture.

[0004] The above methods all involve setting an imaging interval for a sub-scene, and the sub-scene renders the scene content in the imaging interval to generate a screen. When a scene is experienced by multiple users, users may need to set different imaging intervals in the scene. Different imaging intervals will conflict, resulting in the same scene instance being unable to simultaneously generate experience screens for multiple users with different imaging interval requirements. It is necessary to generate multiple scene instances with synchronized states to image different imaging interval settings. For example, user A experiences in scene 1, and user B experiences in a combined scene composed of scene 1 and scene 2. For user A, the imaging interval of scene 1 is the entire scene, while for user B, there is only a specific three-dimensional interval in his field of view or experience space that can be used to present scene 1. Therefore, user A and user B have different requirements for the imaging interval of scene 1. Scene 1 requires two scene instances with synchronized scene states to generate experience screens for user A and user B respectively. Summary of the invention

[0005] The main purpose of the present invention is to provide an XR combined scene construction method, which does not limit the imaging range of the rendering picture in the sub-scene instance. The user experience picture of a specific imaging range can be obtained by simply cropping the sub-scene experience picture, and further synthesized with the experience pictures of other contents in the combined scene through occlusion calculation to obtain the combined scene experience picture.

[0006] To achieve the above-mentioned purpose, the present invention proposes a method for constructing an XR combined scene based on image cropping, the method comprising: performing image synthesis on a part of a binocular stereoscopic vision picture of a first sub-scene that satisfies an image cropping constraint, and a binocular stereoscopic vision picture of other contents in the combined scene, to obtain a first user binocular stereoscopic vision picture of the combined scene, wherein the image cropping constraint is an imaging interval constraint and / or a display interval constraint, and the first sub-scene is one of a virtual scene, a virtual-reality fusion scene, and a real scene.

[0007] In some optional embodiments, the imaging interval constraint is: the imaging content in the binocular stereoscopic vision picture of the first subscene belongs to all pixels of the first imaging interval of the first subscene; the display interval constraint is: in the binocular stereoscopic vision picture of the first subscene, the positions displayed in the combined scene belong to all pixels of the first display interval of the combined scene.

[0008] In some optional embodiments, the first imaging interval must satisfy: the first point is any point within the first imaging interval, the second point is any point on the line connecting the first point and the left eye optical center or the right eye optical center of the binocular camera that images the first subscene, if the second point is imaged, the second point must also belong to the first imaging interval, wherein the binocular camera is used to generate a binocular stereoscopic vision picture of a synthetic combined scene; the first display interval must satisfy: in the first subscene imaging interval corresponding to the first display interval, the first point is any point in this imaging interval, the second point is any point on the line connecting the first point and the left eye optical center or the right eye optical center of the binocular camera that images the first subscene, if the second point is imaged, the second point must also belong to this imaging interval, wherein the binocular camera is used to generate a binocular stereoscopic vision picture of a synthetic combined scene.

[0009] In some optional embodiments, the binocular stereoscopic vision picture of the first subscene is generated by real-time rendering of the first subscene according to the pupil distance and posture of the binocular camera imaging. Before the first subscene generates the binocular stereoscopic vision picture, it is necessary to set the pupil distance and posture of the binocular camera imaging used for synthesizing the combined scene of the first subscene. Before the image is synthesized, it is necessary to set the image cropping constraint.

[0010] Among them, the binocular camera posture setting method for synthesizing the combined scene of the first sub-scene is one of the following methods: setting the posture to an arbitrary posture constant value, setting the posture to follow the posture changes of the first user, setting the posture to follow the posture changes of the second user who experiences the first sub-scene, wherein following means that the posture between the follower and the followed is equal or there is a posture mapping relationship.

[0011] In some optional embodiments, when performing the image synthesis, the pixels in the binocular stereoscopic vision picture of the first sub-scene that meet the image cropping constraints are saved in the binocular vision stereoscopic picture of the first user combined scene or must be subjected to occlusion calculation together with the pixels of the first user's line of sight in the binocular stereoscopic vision picture of other contents in the combined scene.

[0012] Among them, the occlusion calculation method is: for the first pixel that meets the image cropping constraint in the binocular stereoscopic vision picture of the first sub-scene, the second pixel in the binocular stereoscopic vision picture of other contents in the combined scene shares the user line of sight with the first pixel, and the depth value is compared according to the corrected depth value under the same pupil distance. If the corrected depth value of the first pixel is less than the depth value of the second pixel under the same pupil distance, the first pixel occludes the second pixel, otherwise, the second pixel occludes the first pixel; or, the occlusion calculation method is: for the first pixel that meets the image cropping constraint in the binocular stereoscopic vision picture of the first sub-scene, the second pixel in the binocular stereoscopic vision picture of other contents in the combined scene shares the user line of sight with the first pixel, the display interval depth information is queried to obtain the depth representation value of the first pixel, and the first A depth representation value of a pixel is compared with a depth value of a second pixel corrected under the combined scene imaging pupil distance. If the depth representation value of the first pixel is less than the corrected depth value of the second pixel, the first pixel blocks the second pixel; otherwise, the second pixel blocks the first pixel, wherein the display interval is a combined scene display interval for setting a first sub-scene binocular stereoscopic image cropping or a combined scene display interval corresponding to a first sub-scene imaging interval for setting a first sub-scene binocular stereoscopic image cropping; the first imaging interval is an interval set in a first sub-scene coordinate system or an interval in a first sub-scene binocular camera coordinate system, and the coordinate system used in setting the first display interval is one of the combined scene coordinate system, the first user field of view coordinate system, and the combined scene binocular camera coordinate system.

[0013] In some optional embodiments, the first user can perform interactive operations in the combined scene, and the interactive operations are one of the following interactive methods: performing interactive operations on objects in the first sub-scene, using objects in the second sub-scene to perform cross-scene operations on objects in the first sub-scene, and using objects in the first sub-scene to perform cross-scene operations on objects in the second sub-scene.

[0014] To achieve the above-mentioned purpose, a XR combined scene construction system based on image cropping is provided, characterized in that the system includes a memory, a processor, and an XR combined scene construction method program based on image cropping stored in the memory, and the XR combined scene construction method program based on image cropping executes the above-mentioned XR combined scene construction method based on image cropping when run by the processor.

[0015] To achieve the above-mentioned purpose, a computer-readable storage medium is provided, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is called by a processor, the above-mentioned XR combined scene construction method based on image cropping is executed.

[0016] The beneficial effects of the XR combined scene construction method based on image cropping of the present invention are as follows: the method does not need to limit the imaging interval when the first sub-scene is imaged, so that the experience screen rendering conflicts of multiple users of the first sub-scene will not be caused due to the limited imaging interval. In particular, a group of binocular stereoscopic vision pictures of the first sub-scene can be cropped with different contents for different users, thereby satisfying the different experience requirements of a large number of users for the first sub-scene at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is an example of a combination scenario of the present invention.

[0018] Figure 2 An example of cropping a binocular stereo image.

[0019] Figure 3 It is the image coordinate system of the embodiment of the present invention.

[0020] Figure 4 This is the binocular camera coordinate system of the embodiment of the present invention.

[0021] Figure 5 This is example 1 of imaging interval setting for image cropping according to the present invention.

[0022] Figure 6 This is an example of binocular camera imaging.

[0023] Figure 7 An example of occlusion before the imaging interval.

[0024] Figure 8 This is example 2 of imaging interval setting for image cropping according to the present invention.

[0025] Fig. 9 This is example 3 of setting the imaging interval for image cropping according to the present invention.

[0026] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. DETAILED DESCRIPTION

[0027] Combining two or more single-player experience scenarios and / or multi-player collaborative experience scenarios can construct a combined scenario. Figure 1 For example, in (a), user A is experiencing an immersive experience in scene 1, and in (b), user B is experiencing an immersive interactive experience in scene 2. In (d), part of scene 1 is presented in three dimensions in user B's experience space, constructing a combined scene experience for user B that includes scene 1 and scene 2. While user B is experiencing scene 2 normally, he can watch the live scene of scene 1 in three dimensions and interact with scene 1.

[0028] When the user experiences the combined scene, each sub-scene only occupies part of the three-dimensional space of the user experience space or the user's field of view. The present invention cuts the binocular stereoscopic vision picture of the sub-scene and synthesizes it with the picture of other contents of the combined scene, thereby realizing the construction of the combined scene. Figure 2 For example, a group of binocular stereoscopic images of any sub-scene s0 has a certain image interval cropped out of the left and right eye images, and the cropped intervals in the left and right eye images correspond to a radial three-dimensional imaging interval of the sub-scene s0. Point p is in the s0 imaging interval corresponding to the right eye image cropping interval, but not in the imaging interval corresponding to the left eye image cropping interval. Therefore, when the cropped image is shown to the human eye, the point p that can be seen by both the left and right eyes of the human eye is only seen by the right eye after cropping. On this basis, if it is synthesized with the images of other contents of the combined scene, it may appear that at the position of point p, the right eye of the person sees an object of the sub-scene s0, and the left eye of the person sees an object of another sub-scene, which cannot form a correct stereoscopic vision and may cause dizziness. Therefore, the cropping of the binocular stereoscopic image of the sub-scene s0 should preferably satisfy: the s0 three-dimensional imaging interval corresponding to the cropping interval of the left eye image of any sub-scene s0 is the same as the s0 three-dimensional imaging interval corresponding to the cropping interval of its right eye image.

[0029] Important technical terms involved in the present invention are explained below.

[0030] Scenarios and Scenario Examples

[0031] A scene defines the objects, object states, object operation logic, and interaction logic between objects contained in a three-dimensional space; a scene instance is a program process or thread that the system calls on computer processors, memory, graphics cards and other computing resources to run in real time according to the scene definition. This program process or thread calculates the state of each object in the scene in real time, renders the screen, and responds to user interaction. When a single scene has multiple user experiences at the same time, if the computing resources available to a single scene instance cannot generate experience screens for all users in real time, it is necessary to generate multiple scene instances for this scene and allocate them to each user. These scene instances establish communication connections to synchronize the state of objects in the scene, and each scene instance generates an experience screen for the corresponding user in real time, so that each user can experience this scene together.

[0032] Image coordinate system in pixel units and image coordinate system in physical units

[0033] Refer to Zhang Guangjun’s classic book “Machine Vision” [8], such as Figure 3As shown, a rectangular coordinate system u and v are defined on the image. The coordinates (uv) of each pixel are the column number and row number of the pixel in the image array, so (uv) is the image coordinate system coordinate in pixel units. An image coordinate system X and Y in physical units (such as millimeters) are defined on the image, where the X axis and Y axis are parallel to the u axis and v axis, respectively. The origin O1 is the intersection of the camera optical axis and the image plane. If the coordinates of O1 in the image coordinate system in pixel units are (u0 v0), and the physical dimensions of each pixel in the X and Y axis directions are dX and dY, respectively, then the coordinates of any pixel in the image in the two coordinate systems have the following relationship:

[0034]

[0035] Among them, u0, v0, is the camera internal parameter (f is the camera focal length), which can be obtained through calibration. The calibration method is shown in reference [8]. If the image is an image of a virtual scene, since virtual camera imaging generally has no distortion or offset, u0 and v0 are generally the coordinate values ​​of the center point of the image, and It can be calculated from the field of view and resolution. For example, the horizontal field of view of the camera is α, the vertical field of view is β, and the image resolution is n. col Column row OK, then for for If the virtual camera imaging has distortion and offset, since the distortion and offset parameters are set manually, they can be accurately known, and it is easy to determine the internal parameters, which will not be elaborated here.

[0036] Pupillary distance and stereoscopic vision

[0037] When the human eye looks at an object, the left eye and the right eye form an image of the object respectively. The image positions of the same object point in the left eye and the right eye are different. This difference is parallax. Parallax allows the human eye to correctly perceive the three-dimensional information of the object point. When the left and right eye stereoscopic pictures are shown to people, the stereoscopic visual information perceived by the person is affected by the ratio of his or her own pupil distance to the pupil distance of the picture imaging. The image depth information perceived by the human eye needs to be corrected according to this pupil distance ratio. The attachment of the invention patent application "Method, system and medium for constructing immersive interactive live broadcast based on XR technology" (China invention patent application number: 202210906282.6) Figure 5 This conclusion is explained in detail in paragraph 79 of the specification and will not be repeated here.

[0038] Binocular camera coordinate system

[0039] like Figure 4As shown, in the embodiment of the present invention, the binocular camera coordinate system is preferably set as follows: on the line connecting the optical centers of the binocular cameras, the midpoint between the binocular centers is used as the origin of the coordinate system, the direction of the binocular line from the left eye camera to the right eye camera is defined as the positive direction of the x-axis, and the camera direction is defined as the positive direction of the z-axis, which is perpendicular to the x-axis and the z-axis and satisfies the definition of the positive direction of the y-axis in the left-hand coordinate system.

[0040] Field of view and coordinate system

[0041] The field of view is the spatial range visible to the user. The user's field of view changes with the movement of the user's head and eyeballs. The embodiment of the present invention defines the viewer's field of view coordinate system with reference to the viewer's head, so that the movement of the viewer's head and eyeballs does not affect the position relationship of the viewer's field of view coordinate system relative to the viewer's head. As a result, the movement of the viewer's head and eyeballs will not affect the generation and presentation of the live broadcast image. In the embodiment of the present invention, the setting method of the field of view coordinate system is the same as the setting method of the binocular camera coordinate system: on the line connecting the binocular center of the human eye, the midpoint between the binocular center of the binocular eye is used as the origin of the coordinate system, the direction of the binocular line connecting the human eye from the left eye to the right eye is defined as the positive direction of the x-axis, and the direction of the face is defined as the positive direction of the z-axis, and the positive direction of the y-axis is defined to be perpendicular to the x-axis and the z-axis and satisfy the left-hand coordinate system.

[0042] User experience space and coordinate system

[0043] The user experience space is a three-dimensional space for the user's immersive experience. The user experience space coordinate system can be pre-set or automatically constructed by the XR headset. In the embodiment of the present invention, the user experience space coordinate system is set as follows: this coordinate system defines the positive direction of the y-axis in the vertical upward direction of the horizontal plane, and defines the x-axis and z-axis on the horizontal plane.

[0044] Combined scene space and coordinate system

[0045] The coordinate system of the combined space composed of multiple sub-scenes can share the coordinate system with the user experience space, or share a coordinate system with one of the sub-scenes of the user immersive experience. The coordinate system of the combined scene can also be newly constructed according to needs.

[0046] Convex interval

[0047] make is a three-dimensional convex interval, then it has the following properties: Let There are two points a and b, where the coordinates of point a are The coordinates of point b are Point c is on the line connecting points a and b and between points a and b. The coordinates of point c can be expressed as Among them, 1>λ>0, then point c must also belong to

[0048] In the embodiments of the present invention, it is assumed that the imaging field of view and resolution of all images are the same, so that pixels of the same image coordinates between different images share the user's line of sight and there will be an occlusion relationship; if the imaging field of view and resolution between different images are different, the imaging field of view and resolution between the images can be converted to the same by interpolation or sampling, which is a common method used by technicians in the field.

[0049] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Figures 1 to 9 , and the first to eighteenth embodiments describe in detail the XR combined scene construction method based on image cropping of the present invention.

[0050] First embodiment

[0051] The construction method of this embodiment includes: performing image synthesis on a portion of a binocular stereoscopic vision picture of a first sub-scene that satisfies an image cropping constraint and a binocular stereoscopic vision picture of other contents in a combined scene to obtain a first user binocular stereoscopic vision picture of the combined scene, wherein the image cropping constraint is an imaging interval constraint, and the first sub-scene is one of a virtual scene, a virtual-reality fusion scene, and a real scene.

[0052] When performing the image synthesis, the pixels in the binocular stereoscopic vision picture of the first sub-scene that meet the image cropping constraint must be occluded with the pixels of the first user's line of sight in the binocular stereoscopic vision picture of other contents in the combined scene. Specifically, the occlusion calculation method is: for the first pixel in the binocular stereoscopic vision picture of the first sub-scene that meets the image cropping constraint, the second pixel in the binocular stereoscopic vision picture of other contents in the combined scene shares the user's line of sight with the first pixel, and the depth values ​​are compared based on the depth value corrected at the same pupil distance. If the corrected depth value of the first pixel is less than the depth value of the second pixel at the same pupil distance, the first pixel occludes the second pixel, otherwise, the second pixel occludes the first pixel.

[0053] The imaging interval constraint is: the imaging content in the binocular stereoscopic vision picture of the first sub-scene belongs to all pixels in the first imaging interval of the first sub-scene. Wherein, the first imaging interval is an interval set in the first sub-scene coordinate system or an interval in the first sub-scene binocular camera coordinate system.

[0054] by Figure 5 For example, for the first sub-scene s0, the first imaging interval Ω′0 is set in the binocular camera coordinate system. This imaging interval does not limit and does not affect the generation of binocular stereoscopic vision images of the sub-scene s0, but limits the cropping of the binocular stereoscopic vision images of the sub-scene s0. For the right eye image, any pixel I retained after cropping R(uv) must satisfy that its imaging content belongs to the interval Ω′0, specifically: I R During imaging, through pixel I R The right eye camera sight line of (uv) passes through Ω′0, and I R The depth value corresponding to (uv) satisfies the object point whose image belongs to Ω′0. Similarly, for the left eye image, any pixel I retained after cropping L (uv) must satisfy that its imaging content belongs to Ω′0.

[0055] The combined scene construction method of this embodiment is described in detail below.

[0056] Scene s0 is a combined scene The first sub-scene of , where s0 is one of a virtual scene, a virtual-reality fusion scene, and a real scene. and It is a set of binocular stereo vision images generated by rendering or shooting scene s0, and the corresponding depth image is and in, is the left eye image of the stereoscopic vision picture, is the right eye image of the stereoscopic vision picture, is the corresponding left eye depth image, is the corresponding right eye depth image, and the pupil distance (the distance between the optical centers of the binocular cameras) for generating this binocular stereoscopic vision image is d0.

[0057] First user h0 experience combination scenario set up The pupil distance for generating the experience screen for h0 is h0 is the actual pupil distance of the human eye or a manually specified value (such as 60mm). The first user h0, let Except for s0, the other sub-scenes generate the left eye image of h0 experience screen: The right eye image is And according to the pupil distance The corresponding depth image after correction of the stereoscopic visual information perceived under and Set the first imaging interval Ω′0 in s0 as the binocular stereo vision image and Cropped imaging interval constraint, Ω′0 is the imaging interval of s0 defined in the binocular camera coordinate system Ψ′0 (not limiting imaging but only limiting image cropping), binocular stereo vision image of s0 and Need to be cut and Synthesize into user h0 pair combination scene The binocular camera is used to generate a stereoscopic visual image for synthesizing the combined scene.

[0058] The left eye picture of the synthesized combined scene experience screen is Right eye picture The corresponding depth image is initialization

[0059] Traverse s0 left eye image For all pixels in Any pixel in Its depth value is calculate The coordinate value format in the binocular camera coordinate system Ψ′0 is: The image coordinate system coordinate value in physical units is (X Left Y P ),like Figure 6 As shown, The coordinate value of the imaged object point P in the binocular camera coordinate system is but Among them, from formula (1), we can get thus Under the premise that the left eye camera internal parameters have been obtained, that is, u0, v0, In the case of When When The imaging interval Ω′0 constraint is satisfied, otherwise the imaging interval Ω′0 constraint is not satisfied.

[0060] Traverse s0 right eye image For all pixels in Any pixel in Its depth value is calculate The coordinate value in the binocular camera coordinate system is as follows: The image coordinate system coordinate value (X Right Y P ),make The coordinate value of the imaged object point P in the binocular camera coordinate system is but From formula (1), we can get thus Under the premise that the internal parameters of the right eye camera have been obtained, that is, u0, v0, In the case of When When The imaging interval Ω′0 constraint is satisfied, otherwise the imaging interval Ω′0 constraint is not satisfied.

[0061] Let the synthesized experience screen left eye picture be Right eye picture The corresponding depth image is like If the imaging interval Ω′0 constraint is not satisfied, then it is discarded. like If the imaging interval Ω′0 constraint is satisfied, it needs to be synthesized with the experience images of other scene contents in the combined scene. Assign to But it is more preferable to determine by occlusion calculation Is it assigned to

[0062] Due to the rendering s0 generates a binocular stereo vision image and The pupil distance of the combined scene imaging is d0, and the pupil distance of the combined scene imaging is So be sure Depth value Make corrections, after correction The depth value presented in the combined scene is

[0063] Therefore, if Then the pixel Block the corresponding pixels of other sub-scenes in the combined scene and record them to the combined scene image, that is and otherwise Obscured, abandoned After traversing all pixels in the image according to the above calculation, the synthesis of the left eye image of the combined scene is completed. Similarly, the synthesis of the right eye combined scene image is realized. The synthesized combined scene picture is displayed to the audience h k look.

[0064] In the above implementation of the embodiment of the present invention, the experience screens of the sub-scenes other than the first sub-scene are first synthesized, and then the experience screen of the first sub-scene is synthesized. Alternatively, the experience screens of the first sub-scene and one or more other sub-scenes may be synthesized first, and then the synthesized screen may be synthesized with the experience screens of other unsynthesized sub-scenes. The method of synthesizing the experience screens of the first sub-scene and one or more other sub-scenes can be easily obtained from the implementation of the present embodiment, and will not be described in detail here.

[0065] Second embodiment

[0066] The interval Ω0 of the scene coordinate system Ψ0 of the first sub-scene s0 is set as the image cropping constraint. It is known that s0 generates the posture of the binocular camera in Ψ0 for constructing the combined scene image, wherein the attitude angle and position value of this posture value are respectively the rotation angle and translation amount of the coordinate system transformation relationship between the binocular camera coordinate system Ψ0 and the s0 coordinate system Ψ0. Based on this coordinate system transformation relationship, Ω0 is converted into the interval Ω′0 defined in the binocular camera coordinate system Ψ′0. On this basis, referring to the combined scene construction method of the first embodiment, it is easy to obtain the combined scene construction method of setting the imaging interval Ω0 in the s0 coordinate system Ψ0 as the image cropping constraint, which will not be repeated here.

[0067] Third embodiment

[0068] The construction method of this embodiment includes: performing image synthesis on a portion of a binocular stereoscopic vision picture of a first sub-scene that satisfies an image cropping constraint and a binocular stereoscopic vision picture of other contents in a combined scene to obtain a first user binocular stereoscopic vision picture of the combined scene, wherein the image cropping constraint is a display interval constraint, and the first sub-scene is one of a virtual scene, a virtual-reality fusion scene, and a real scene.

[0069] The display interval constraint is: in the binocular stereoscopic vision picture of the first sub-scene, all pixels whose positions displayed in the combined scene belong to the first display interval of the combined scene.

[0070] Among them, when performing the image synthesis, the pixels that meet the image cropping constraint in the binocular stereoscopic vision picture of the first sub-scene must be occluded with the pixels of the first user's line of sight in the binocular stereoscopic vision picture of other contents in the combined scene. Specifically, the occlusion calculation method is: for the first pixel that meets the image cropping constraint in the binocular stereoscopic vision picture of the first sub-scene, the second pixel in the binocular stereoscopic vision picture of other contents in the combined scene shares the user's line of sight with the first pixel, and the depth values ​​are compared according to the depth value corrected at the same pupil distance. If the corrected depth value of the first pixel is less than the depth value of the second pixel at the same pupil distance, the first pixel occludes the second pixel, otherwise, the second pixel occludes the first pixel.

[0071] The coordinate system used in the first display interval is set to be the first user field of view coordinate system or the combined scene binocular camera coordinate system.

[0072] In the embodiment of the present invention, when the pupil distance of the combined scene binocular camera is equal to the pupil distance of the first user, the combined scene binocular camera coordinate system is aligned with the first user field of view coordinate system. Therefore, the following only introduces the specific implementation method of setting the coordinate system used in the first display interval to be the combined scene binocular camera coordinate system.

[0073] The position of the pixel in the binocular stereoscopic vision picture of the first sub-scene displayed in the combined scene refers to the three-dimensional position of the imaging content of this pixel perceived by the first user and presented in the combined scene. This position information is obtained from the stereoscopic vision information corresponding to the binocular stereoscopic vision picture at the pupil distance of the combined scene generating the experience picture for the first user, and can be further converted into the three-dimensional position information of the imaging content of the binocular stereoscopic vision picture.

[0074] Set the combined scene binocular camera coordinate system Lower display interval As image cropping constraints. Combined scene stereo camera coordinate system The binocular camera coordinate system Ψ′0 in s0 is preferably constructed in the same way as the coordinate system. Any point in Mapped to Ψ′0, the corresponding coordinate system is [x0 y0 z0], then in, The pupil distance for generating the experience screen for h0 is The pupil distance of the binocular stereoscopic vision image generated by s0 for constructing the combined scene is d0. According to this mapping relationship, By converting to Ψ0 in the coordinate system, we obtain an s0 imaging interval constraint defined by Ψ′0 in the coordinate system. Thus, the combined scene construction method of this embodiment can be easily obtained by referring to the first embodiment, which will not be described in detail here.

[0075] Fourth embodiment

[0076] The embodiment of the present invention sets the combined scene coordinate system Lower display interval As image cropping constraints. It is known that the combined scene stereo camera is in the combined scene coordinate system The posture under this condition is preferably taken as the value of user h0 at The real-time pose under the condition, the pose angle and position of this pose value are respectively the combined scene binocular camera coordinate system Combined scene coordinate system Based on the transformation relationship, we can get exist Corresponding display range Therefore, referring to the third embodiment, the combined scene construction method of this embodiment can be easily obtained, and will not be described in detail here.

[0077] Fifth to Eighth Embodiments

[0078] The first to fourth embodiments are improved, the binocular stereoscopic vision picture of the first sub-scene is generated by real-time rendering of the first sub-scene according to the pupil distance and posture of the binocular camera imaging, before the first sub-scene generates the binocular stereoscopic vision picture, it is necessary to set the pupil distance and posture of the binocular camera imaging for the first sub-scene synthesis combined scene, and before the image synthesis, it is necessary to set the image cropping constraint, and the binocular camera posture setting method for the first sub-scene synthesis combined scene is one of the following methods: setting the posture to an arbitrary posture constant value, setting the posture to follow the posture change of the first user, and also setting the posture to follow the posture change of the second user who experiences the first sub-scene, wherein following means that the posture between the follower and the followed is equal or there is a posture mapping relationship, wherein the posture mapping relationship can be an arbitrary function relationship, so that the transformation relationship between the s0 coordinate system Ψ0 and the binocular camera coordinate system Ψ′0 changes in real time.

[0079] In the above manner, after improving the first to fourth embodiments, the fifth, sixth, seventh and eighth embodiments are obtained respectively.

[0080] Ninth to Sixteenth Embodiments

[0081] by Figure 7 For example, in Figure 7 When the imaging interval for limited image cropping is set to Ω′0, there is an object h that is not in Ω′0, but is blocked between the right eye camera and Ω′0, so the pixels imaging h in the binocular stereoscopic right eye image do not meet the Ω′0 constraint and need to be cropped and discarded, but the cropped right eye image will not contain the scene content of s0 behind h, which will cause the imaging interval corresponding to the cropped right eye image to be different from the imaging interval corresponding to the left eye image, which will cause the sub-scene s0 object q visible to the left eye to appear in the synthesized combined scene image. Since h is not presented in the combined scene, there will be a situation where there is no object in the combined scene to block q, and the left eye can see q but the right eye cannot see q, thereby forming an abnormal stereoscopic visual effect, which makes the human eye easily fatigued. Therefore, in the s0 binocular stereoscopic image used to synthesize the combined scene image, it is preferred that there should be no imaging content blocked in front of the interval Ω′0.

[0082] To ensure that there is no imaging content that blocks the s0 imaging interval Ω′0, one way is: when an object blocks Ω′0, set this object not to be imaged, which is to limit the imaging interval for generating stereoscopic vision images. Another way is: when setting the imaging interval Ω′0 for image cropping, Ω′0 satisfies that any imaging point on the line between any point in its interval and the optical center of the left or right eye of the binocular camera belongs to Ω′0. Figure 8 As shown, and is the line of sight of the right eye, Passing through the optical center of the left eye O0, Rotate clockwise around point O1 to In the covered interval, all imaging points on the line from any point to the right eye are within this interval. and is the line of sight of the left eye, Passing through the optical center of the right eye O1, Rotate clockwise around point O0 to In the covered area, all imaging points on the line from any point to the left eye are within this area. Rotate clockwise around point O1 to The interval covered is Rotate clockwise around point O0 to The intersection of the covered intervals is Ω′0. Any point in Ω′0 and any point on the line connecting the left eye and the right eye are all in Ω′0. On this basis, Figure 8 The interval O0O1cd close to the human eye is intercepted as the imaging interval for limited cropping.

[0083] In VR applications, an imaging near cross-section is often set in the binocular camera field of view. The object will only be imaged when the depth distance from the object to the human eye camera exceeds this near cross-section, thereby avoiding objects that are too close to the human eye and cause discomfort. Fig. 9 This is an example of setting an imaging interval for limiting image cropping based on setting an imaging near cross section. Fig. 9 In the figure, the imaging near section Γ is set, and the right eye line of sight Rotate clockwise around the right eye optical center O1 to the line of sight The interval covered and behind the near section l is Ω′ R ,Ω′ R All imaging points on the line connecting any point to the right eye are within this interval. Similarly, the left eye's line of sight Rotate clockwise around the left eye's optical center O0 to the line of sight The interval covered and behind the near section Γ is Ω′ L ,Ω′ L All imaging points on the line connecting any point to the left eye are within this interval, and the intersection interval of these two intervals is Any point in the left eye and any point on the line connecting the left eye and the right eye is within this interval. Fig. 9 The quadrilateral region abcd near the near cross section is intercepted as the imaging interval for limiting cropping. Based on the above method, the method of setting the imaging interval abcd can be further simplified to set a line of sight for each eye. and and If they intersect, the intersection point must be behind the near section l. and The interval between and after the near section satisfies that any point in the interval and any imaging point on the line connecting the left eye and the right eye are within this interval (where, and If they intersect after the near section, The interval surrounded by the near cross section meets the requirements), and then and Select a point after the near section, connect point c and point b, and form an area abcd close to the near section as the imaging interval for limiting cropping. Among them, the line connecting point c and point b is a straight line or a curve. If the line connecting point c and point b is a curve, it is necessary to ensure that any point in area abcd and any imaging point on the line connecting the left eye or the right eye are still in this area. For example, area abcd is a convex interval. Further, based on this two-dimensional interval, a three-dimensional imaging interval for limiting cropping is set. Taking the example that the line connecting point c and point b is a straight line, find two parallel lines l on the near section that pass through point a and point d respectively. a With l d , With l d Coplanar Γ d , With l a Coplanar Γ a , select the passing point c and A straight line l perpendicular to and parallel to the near section Γ c , l c and Coplanar Γ bc , select two parallel The two straight lines are directly connected to Coplanar Γ0 and Γ1, Γ0, Γ1, Γ d , Γ a , Γ bc and Γ together surround a three-dimensional interval, and this three-dimensional interval satisfies that any point in the interval and any imaging point on the line connecting the left eye and the right eye are within this interval, and this interval can be used as a three-dimensional imaging interval for limiting image cropping. The above method of setting the three-dimensional imaging interval is one of many three-dimensional interval setting methods that satisfy that any point in the interval and any imaging point on the line connecting the optical center of the left eye or the right eye belong to this interval, and examples are not given here one by one.

[0084] The first, second, fifth and sixth embodiments are improved, and the first imaging interval must satisfy: the first point is any point within the first imaging interval, and the second point is any point on the line connecting the first point and the left eye optical center or the right eye optical center of the binocular camera imaging the first sub-scene, and if the second point is imaged, the second point must also belong to the first imaging interval, wherein the binocular camera is used to generate a binocular stereoscopic vision picture of a synthetic combined scene. After improving the first, second, fifth and sixth embodiments, the ninth, tenth, thirteenth and fourteenth embodiments are obtained respectively.

[0085] Among them, in the fourteenth embodiment, the binocular camera posture setting method for the first sub-scene synthesis combined scene is to set the posture to follow the posture change of the first user or to set the posture to follow the posture change of the second user who experiences the first sub-scene, and the transformation relationship between the binocular camera coordinate system Ψ′0 for the first sub-scene synthesis combined scene and the first sub-scene s0 coordinate system Ψ0 is changed in real time, thereby setting the imaging interval Ω0 for image cropping in the first sub-scene s0 coordinate system Ψ0, and the corresponding imaging interval in the binocular camera coordinate system Ψ′0 is Ω′0, and Ω′0 is changed in real time. In order to ensure that Ω′0 always satisfies that any point in the interval and any imaging point on the line connecting the left eye and the right eye are within this interval, in the fourteenth embodiment, it is preferred to set the imaging interval Ω0 for image cropping in the first sub-scene s0 coordinate system Ψ0 as a convex interval, and limit the position of the binocular camera for the first sub-scene synthesis combined scene to be within Ω0, so as to achieve that even if Ω′0 changes in real time, it always satisfies that any point in the interval and any imaging point on the line connecting the left eye and the right eye are within this interval.

[0086] The third, fourth, seventh and eighth embodiments are improved, and the first display interval must satisfy: according to the conversion relationship between the coordinate system where the first display interval is located and the coordinate system of the first sub-scene binocular camera or the scene coordinate system, the imaging interval corresponding to the first display interval in the first sub-scene is obtained, the first point is an arbitrary point in this imaging interval, and the second point is an arbitrary point on the line connecting the first point and the left eye optical center or the right eye optical center of the binocular camera imaging the first sub-scene. If the second point is imaged, the second point must also belong to this imaging interval, wherein the binocular camera is used to generate a binocular stereoscopic vision picture of a synthetic combined scene, and the imaging interval is an interval set in the first sub-scene coordinate system or an interval in the first sub-scene binocular camera coordinate system. After improving the third, fourth, seventh and eighth embodiments, the eleventh, twelfth, fifteenth and sixteenth embodiments are obtained respectively.

[0087] Among them, in the sixteenth embodiment, the combined scene binocular camera coordinate system Combined scene coordinate system The transformation relationship between them changes in real time, so in the combined scene coordinate system The following is the display interval for image cropping in the first sub-scene s0: The corresponding imaging interval in the s0 binocular camera coordinate system Ψ′0 is Ω′0, which changes in real time. In order to ensure that Ω′0 always satisfies that any point in the interval and any imaging point on the line connecting the left eye and the right eye are within this interval, in the sixteenth embodiment, it is preferred to combine the scene coordinate system The display range set below is a convex interval, and the position of the first user must be within Thus, even if Ω′0 changes in real time, it always satisfies that any point in the interval and any imaging point on the line connecting the left eye and the right eye are within this interval.

[0088] Seventeenth Embodiment

[0089] The fifth to sixteenth embodiments are improved, and the first user can perform interactive operations in the combined scene, and the interactive operations are one of the following interactive methods: interactive operations on objects in the first sub-scene, cross-scene operations on objects in the first sub-scene using objects in the second sub-scene, and cross-scene operations on objects in the second sub-scene using objects in the first sub-scene.

[0090] The specific implementation of the first user's interactive operation on the object in the first sub-scene in the combined scene is as follows.

[0091] In the eighth and sixteenth embodiments, the image cropping constraints are set to the combined scene Coordinate system The display range below The image cropping constraint interval set in other embodiments can be obtained by coordinate system transformation in the combined scene Coordinate system The corresponding display interval The first user h0 in the combined scene Perform interactive operations and generate interactive operation commands a. The interactive operations are performed in the combined scene. Coordinate system The position parameters are The attitude angle parameter is Determine the coordinate value Is it like According to the s0 scene coordinate system Ψ0 and the combined scene Coordinate system The coordinate system transformation relationship is Pose parameters in the coordinate system and The pose parameters are converted into pose parameters in the s0 scene coordinate system Ψ0, the converted pose parameters are assigned to the pose parameter components of the interactive operation command a, and the interactive command a with the converted pose parameters is sent to the scene instance of the sub-scene s0, and the scene instance of the sub-scene s0 responds to the interactive operation command. Thus, the first user can perform interactive operations on the objects in the first sub-scene in the combined scene.

[0092] The first user uses objects in the second sub-scene to perform cross-scene operations on objects in the first sub-scene in the combined scene, or uses objects in the first sub-scene to perform cross-scene operations on objects in the second sub-scene. For specific implementation, please refer to patent application "A cross-scene interaction method, system and medium for XR combined scenes" application number: CN2023115431377, which will not be repeated here.

[0093] Eighteenth Embodiment

[0094] The calculation method of the inter-pixel occlusion method of the first to seventeenth embodiments is improved. The occlusion calculation method is: for the first pixel in the binocular stereoscopic vision picture of the first sub-scene that meets the image cropping constraint, the second pixel in the binocular stereoscopic vision picture of other contents in the combined scene shares the user's line of sight with the first pixel, query the display interval depth information to obtain the depth representation value of the first pixel, and compare the depth representation value of the first pixel with the depth value of the second pixel corrected under the combined scene imaging pupil distance. If the depth representation value of the first pixel is less than the corrected depth value of the second pixel, the first pixel occludes the second pixel, otherwise, the second pixel occludes the first pixel, wherein the display interval is a combined scene display interval that sets the first sub-scene binocular stereoscopic image cropping or a combined scene display interval corresponding to the first sub-scene imaging interval that sets the first sub-scene binocular stereoscopic image cropping.

[0095] The relevant technologies for combining the depth information of the display interval within the scene and querying the depth information of the display interval to obtain the depth representation value of the pixel are described in detail in the inventor's previous patent applications, please refer to references

[10] and

[11] , and will not be repeated here.

[0096] In the first to eighteenth embodiments, the image cropping constraint is an imaging interval constraint or a display interval constraint. By combining the above two types of constraint combination scene construction methods, it is easy to obtain the image cropping constraint as an imaging interval constraint and a display interval constraint combination scene construction method, which will not be described in detail here. In addition, it is preferred to avoid overlapping of other sub-scenes of the combined scene with the first sub-scene in the display interval of the combined scene, and the display intervals of each sub-scene in the combined scene are respectively non-overlapping.

[0097] The beneficial effects of the XR combined scene construction method based on image cropping of the present invention are as follows: the method does not need to limit the imaging interval when the first sub-scene is imaged, so that the experience screen rendering conflicts of multiple users of the first sub-scene will not be caused due to the limited imaging interval. In particular, a group of binocular stereoscopic vision pictures of the first sub-scene can be cropped with different contents for different users, thereby satisfying the different experience requirements of a large number of users for the first sub-scene at the same time.

[0098] To achieve the above-mentioned purpose, a system for constructing an XR combined scene based on image cropping is provided, characterized in that the system includes a memory, a processor, and an XR combined scene construction method program based on image cropping stored in the memory, and the XR combined scene construction method program based on image cropping executes the above-mentioned XR combined scene construction method based on image cropping when the program is run by the processor. The system may be a distributed system, in which the rendering of the first sub-scene and the generation of other content experience screens of the combined scene are processed by different processors, and / or the rendering of the sub-scene and the synthesis of the combined scene experience screen are processed by different processors, and the processors may be installed on the same computing device or on multiple computing devices.

[0099] To achieve the above-mentioned purpose, a computer-readable storage medium is provided, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is called by a processor, the above-mentioned XR combined scene construction method based on image cropping is executed.

[0100] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Equivalent structures or equivalent process changes made by the scheme using the contents of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

[0101] References

[0102] [1] Patent application “A method, system and platform for constructing a new dimensional space based on XR technology” (application number CN202210022608.9)

[0103] [2] Patent application “Ubiquitous training campus construction method, system and storage medium based on XR technology” (application number: CN202210908795.0)

[0104] [3] Patent application “A method, system and storage medium for generating XR combined scene experience” (Application No.: CN2023110288353)

[0105] [4] Patent application “A method, system and medium for generating combined scene experience based on XR technology” (Application No.: 2023110286220)

[0106] [5] Patent application “A method, system, terminal and medium for generating XR combined scene experience images” (Application No.: 2023112052542)

[0107] [6] Patent application “Method, system and medium for constructing immersive interactive live broadcast based on XR technology” (Application number: CN202210906282.6)

[0108] [7] Patent application "A method, system, host terminal, client terminal and medium for constructing XR stereoscopic interactive live broadcast" Application number: CN2023110283716

[0109] [8] Zhang Guangjun. Machine Vision. Beijing: Science Press, 2005.25-27

[0110] [9] Patent application “A cross-scene interaction method, system and medium for XR combined scenes” Application number: CN2023115431377

[0111]

[10] Patent application “A method, system, host terminal, audience terminal and medium for constructing XR stereoscopic interactive live broadcast” Application number: CN2023110283716

[0112]

[11] Patent application “A method, system and medium for generating combined scene experience based on XR technology” Application number: CN2023110286220.

Claims

1. A method for constructing a scene using an extended reality technology combination based on image cropping, characterized in that: Two or more single-person experience scenes and / or multi-person collaborative experience scenes are combined to construct a combined scene, wherein the combined scene is a sub-scene of the combined scene, and a portion of a binocular stereoscopic vision picture of a first sub-scene that satisfies an image cropping constraint is subjected to image synthesis with a binocular stereoscopic vision picture of other contents in the combined scene to obtain a first user binocular stereoscopic vision picture of the combined scene, wherein the image cropping constraint is a three-dimensional imaging interval constraint of the first sub-scene and / or a display interval constraint of the combined scene, and the display interval constraint is a constraint on a three-dimensional position of pixel imaging content in the binocular stereoscopic vision picture of the first sub-scene perceived by the first user and presented in the combined scene, the first sub-scene is one of a virtual scene, a virtual-reality fusion scene, and a real scene, the imaging interval constraint is all pixels in the binocular stereoscopic vision picture of the first sub-scene whose imaging content belongs to a first imaging interval of the first sub-scene, and the display interval constraint is: all pixels in the binocular stereoscopic vision picture of the first sub-scene whose positions displayed in the combined scene belong to the first display interval of the combined scene.

2. The combined scene construction method according to claim 1, characterized in that: The first imaging interval must satisfy: the first point is an arbitrary point within the first imaging interval, the second point is an arbitrary point on the line connecting the first point and the left eye optical center or the right eye optical center of the binocular camera that images the first subscene, if the second point is imaged, the second point must also belong to the first imaging interval, wherein the binocular camera is used to generate a binocular stereoscopic vision picture of a synthetic combined scene; the first display interval must satisfy: in the first subscene imaging interval corresponding to the first display interval, the first point is an arbitrary point in this imaging interval, the second point is an arbitrary point on the line connecting the first point and the left eye optical center or the right eye optical center of the binocular camera that images the first subscene, if the second point is imaged, the second point must also belong to this imaging interval, wherein the binocular camera is used to generate a binocular stereoscopic vision picture of a synthetic combined scene.

3. The combined scene construction method according to claim 2, characterized in that: The binocular stereoscopic vision picture of the first sub-scene is generated by real-time rendering of the first sub-scene based on the pupil distance and posture of the binocular camera imaging. Before the first sub-scene generates the binocular stereoscopic vision picture, it is necessary to set the pupil distance and posture of the binocular camera imaging used for synthesizing the combined scene of the first sub-scene. Before the image is synthesized, it is necessary to set the image cropping constraint.

4. The combined scene construction method according to claim 3, characterized in that: The binocular camera posture setting method for synthesizing the combined scene of the first sub-scene is one of the following methods: setting the posture to an arbitrary posture constant value, setting the posture to follow the posture changes of the first user, setting the posture to follow the posture changes of the second user who experiences the first sub-scene, wherein following means that the posture between the follower and the followed is equal or there is a posture mapping relationship.

5. The combined scene construction method according to any one of claims 1 to 4, characterized in that: When performing the image synthesis, the pixels in the binocular stereoscopic vision picture of the first sub-scene that meet the image cropping constraints are saved in the binocular vision stereoscopic picture of the first user combined scene or must be subjected to occlusion calculation together with the pixels of the first user's line of sight in the binocular stereoscopic vision picture of other contents in the combined scene.

6. The combined scene construction method according to claim 5, characterized in that: The occlusion calculation method is as follows: for a first pixel that satisfies the image cropping constraint in the binocular stereoscopic vision picture of the first subscene, a second pixel in the binocular stereoscopic vision picture of other contents in the combined scene shares the user line of sight with the first pixel, and a depth value comparison is performed based on the depth value corrected at the same pupil distance. If the corrected depth value of the first pixel is less than the depth value of the second pixel at the same pupil distance, the first pixel occludes the second pixel, otherwise, the second pixel occludes the first pixel; or, the occlusion calculation method is as follows: for a first pixel that satisfies the image cropping constraint in the binocular stereoscopic vision picture of the first subscene, a second pixel in the binocular stereoscopic vision picture of other contents in the combined scene shares the user line of sight with the first pixel, the depth information of the display interval is queried to obtain the depth representation value of the first pixel, and the first pixel The depth representation value of the first pixel is compared with the depth value of the second pixel corrected under the combined scene imaging pupil distance. If the depth representation value of the first pixel is less than the corrected depth value of the second pixel, the first pixel blocks the second pixel, otherwise, the second pixel blocks the first pixel, wherein the display interval is a combined scene display interval set to limit the cropping of the first sub-scene binocular stereoscopic vision image or a combined scene display interval corresponding to the first sub-scene imaging interval that limits the cropping of the first sub-scene binocular stereoscopic vision image; the first imaging interval is an interval set in the first sub-scene coordinate system or an interval in the first sub-scene binocular camera coordinate system, and the coordinate system used in the first display interval is one of the combined scene coordinate system, the first user field of view coordinate system, and the combined scene binocular camera coordinate system.

7. The combined scene construction method according to claim 5, characterized in that: The first user can perform interactive operations in the combined scene, and the interactive operations are one of the following interactive methods: performing interactive operations on objects in the first sub-scene, using objects in the second sub-scene to perform cross-scene operations on objects in the first sub-scene, and using objects in the first sub-scene to perform cross-scene operations on objects in the second sub-scene.

8. An extended reality technology combined scene construction system based on image cropping, characterized in that: The system includes a memory, a processor, and a method program for constructing a combined scene of extended reality technology based on image cropping stored in the memory. When the method program for constructing a combined scene of extended reality technology based on image cropping is executed by the processor, the method described in any one of claims 1 to 7 is executed.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is called by the processor, the method for constructing a combined scene of extended reality technology based on image cropping according to any one of claims 1 to 7 is executed.

Images