A refined three-dimensional modeling method and system based on an XR shed

Abstract

The present invention provides a refined 3D modeling method and system based on an XR studio, which relates to the field of image processing technology. The system of the present invention includes a 3D modeling unit, an image generation unit, a texture processing unit and a modeling generation unit. The method steps are used to obtain a 3D scene through the 3D modeling unit, and then a playback module image for XR studio display is generated through the image generation unit. The resolution difference between the original model texture and the mapped model texture is compared based on the soft module image. The modeling generation unit controls the texture processing unit to perform image upsampling or downsampling on the model texture according to the human eye resolution threshold, obtains a refined resolution texture and sets the display control logic. In actual use, the model texture selects the refined resolution texture of the corresponding resolution for display according to the preset display control logic, thereby reducing the resolution difference phenomenon caused by different resolutions.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a refined three-dimensional modeling method and system based on an XR studio. Background Art

[0002] The XR studio is a new type of content display device. Its appearance resembles a cube with one side opened. Inside the box is a playback module for displaying content. It uses "3D model + panoramic background image" to display content. Viewers can get an immersive three-dimensional viewing experience by standing at the opening.

[0003] like Figure 2 As shown, the playback module is usually composed of a hard module + a soft module; among them, the hard module refers to a playback module that does not need to be bent or deformed, and the hard module can directly play the desired material, and the soft module is a playback module that is bent and deformed at the edge or diagonal; since the soft module has certain deformation and bending, image processing is required before playing the material to adapt to the impact of deformation and bending on the image.

[0004] During image processing, the original image is scaled according to the bending conditions, thereby changing the original image resolution. This results in inconsistent resolution at different locations, a phenomenon we call resolution discrepancy. That is, when the resolution discrepancy is large, the human eye can easily detect loss of detail, blurred edges, or pixelation. This is especially evident in the imaging of three-dimensional models.

[0005] In the "3D model + panoramic background image" combination, since the panoramic background image is usually displayed as secondary content, it is far away from the observation point and the observation angle usually does not change much; therefore, its resolution difference phenomenon is not much concerned by people and is not so obvious; when imaging the 3D model, since it is the main display content, it is far away from the observation point, and the observation angle will change greatly with the movement of the model or observation point. Therefore, its resolution difference phenomenon will directly affect the user's viewing experience.

[0006] Existing technologies rely on post-processing manual adjustments to address resolution differences, or do not address them at all. This results in high processing costs, poor efficiency, and the inability to perform real-time calculations and imaging.

[0007] Therefore, it is necessary to provide a refined three-dimensional modeling method and system based on the XR studio to solve the technical problem of poor imaging effect of three-dimensional models (especially dynamic three-dimensional models) on the XR studio. Summary of the Invention

[0008] The purpose of the present invention is to overcome the shortcomings of the existing technology and provide a refined three-dimensional modeling method and system based on an XR studio, aiming to solve the technical problems that the existing technology relies on post-manual adjustment to deal with or does not deal with the resolution difference phenomenon, which leads to high processing costs, poor efficiency and inability to perform real-time calculation and imaging functions.

[0009] To achieve the above objectives, this application proposes a refined 3D modeling system based on an XR studio, comprising:

[0010] A three-dimensional modeling unit, configured to perform three-dimensional modeling to obtain a three-dimensional scene; wherein the three-dimensional scene includes a panoramic background image, a three-dimensional model, and a model texture;

[0011] An image generation unit, configured to generate a playback module image for display in an XR studio; wherein the playback module image includes a hard module image and a soft module image;

[0012] A texture processing unit, used to perform upsampling of the model texture to increase the image resolution, or downsampling to reduce the image resolution;

[0013] The modeling generation unit is used to control the texture processing unit to perform image upsampling or downsampling on the model texture according to the human eye resolution threshold, obtain a refined resolution texture and set the display control logic.

[0014] As a further solution, the three-dimensional modeling unit includes a panoramic background image editing module, a three-dimensional model editing module and a model texture editing module; wherein, the panoramic background image editing module is used to edit the panoramic background image used, the three-dimensional model editing module is used to edit the three-dimensional model, and the model texture editing module is used to edit the model texture used by the three-dimensional model.

[0015] As a further solution, the image generation unit includes a spherical projection mapping module, a cubic mapping module and an image block division module; wherein, the spherical projection mapping module is used to project the three-dimensional scene onto a 360-degree sphere and obtain a spherical plane image; the cubic mapping module is used to map the spherical plane image onto a cube based on the XR studio prototype to obtain a cubic plane image; the image block division module divides the cubic plane image into corresponding playback modules to obtain a hard module image and a soft module image.

[0016] As a further solution, the mapping processing unit includes a resolution acquisition module, an image upsampling module and an image downsampling module; wherein the resolution acquisition module is used to obtain the resolution parameters of the image, the image upsampling module is used to upsample the original image to obtain a corresponding high-resolution image, and the image downsampling module is used to downsample the original image to obtain a corresponding low-resolution image.

[0017] As a further solution, the modeling generation unit includes a resolution division unit, a map generation unit and a layer setting unit; wherein, the resolution division unit is used to divide the corresponding human eye resolution threshold according to the characteristics of the human eye; the map generation unit is used to control the image downsampling module or the image downsampling module, and generate a refined resolution map with a resolution corresponding to the human eye resolution threshold; the layer setting unit is used to set the display control logic of the refined resolution map.

[0018] In another aspect, the present invention provides a method for refined 3D modeling based on an XR studio, which is applied to a refined 3D modeling system based on an XR studio as described in any of the above items, and comprises the following steps:

[0019] Step 1: Performing 3D modeling by a 3D modeling unit to obtain a 3D scene; wherein the 3D scene includes a panoramic background image, a 3D model, and a model texture;

[0020] Step 2: Generate a playback module image for XR studio display through an image generation unit; wherein the playback module image includes a hard module image and a soft module image;

[0021] Step 3: The modeling generation unit controls the texture processing unit to perform image upsampling or downsampling on the model texture according to the human eye resolution threshold;

[0022] Step 4: Obtain a refined resolution map through the modeling generation unit and set the corresponding display control logic; apply the refined resolution map to the 3D model.

[0023] As a further solution, the 3D modeling unit performs 3D modeling through the following steps:

[0024] Edit the used panoramic background image through the panoramic background image editing module;

[0025] Edit the 3D model through the 3D model editing module;

[0026] Edit the model texture used by the 3D model through the model texture editing module.

[0027] As a further solution, the image generation unit generates the playback module image through the following steps:

[0028] The spherical projection mapping module projects the three-dimensional scene onto a 360-degree sphere and obtains a spherical plane image;

[0029] The spherical plane image is mapped onto a cube based on the XR studio through the cube mapping module to obtain a cubic plane image;

[0030] The cubic plane image is divided into corresponding playback modules through the image block division module to obtain hard module images and soft module images.

[0031] As a further solution, the modeling generation unit generates a refined resolution map through the following steps:

[0032] Acquire a soft module image and divide the model map existing in the soft module image to obtain a mapping model map;

[0033] Obtain the resolution of the original model texture and the mapped model texture through the resolution acquisition module;

[0034] The resolution division unit divides the corresponding human eye resolution threshold according to the human eye characteristics;

[0035] Determine whether the resolution of the original model map and the mapped model map exceeds the human eye resolution threshold;

[0036] If it exceeds, the exceeding part is threshold-divided by the human eye resolution threshold until it is all classified within the human eye resolution threshold;

[0037] Otherwise, proceed to the next step;

[0038] The map generation unit controls the image upsampling module or the image downsampling module according to the threshold division to perform image upsampling or downsampling processing on the model map to obtain a refined resolution map of the corresponding division;

[0039] Among them, downsampling is performed when the resolution is higher than the original model map, and upsampling is performed when the resolution is lower than the original model map.

[0040] As a further solution, the modeling generation unit sets the display control logic:

[0041] Obtain the resolution of the refined resolution map corresponding to the threshold division, and obtain the refined resolution threshold;

[0042] Set the refinement resolution threshold as the trigger condition;

[0043] When the playback module displays content, the resolution of each soft module image is obtained through the resolution acquisition module;

[0044] Each refined resolution map is triggered according to the resolution of the soft module image.

[0045] The refined resolution map with the smallest resolution difference is selected and displayed in the soft module image.

[0046] Compared with related technologies, the XR studio-based refined 3D modeling method and system provided by the present invention has the following advantages:

[0047] The present invention obtains a three-dimensional scene through a three-dimensional modeling unit, and then generates a playback module image for XR studio display through an image generation unit; then, based on the soft module image, the resolution difference between the original model map and the mapped model map is compared, and the modeling generation unit controls the map processing unit to perform image upsampling or downsampling on the model map according to the human eye resolution threshold, thereby obtaining a refined resolution map and setting the display control logic; in actual use, the model map selects the refined resolution map of the corresponding resolution for display according to the preset display control logic, thereby reducing the resolution difference phenomenon caused by different resolutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0050] Figure 1 A schematic diagram of the structure of a refined 3D modeling system based on an XR studio provided by the present invention;

[0051] Figure 2 This is a schematic diagram of the playback module structure provided by the present invention;

[0052] Figure 3 A schematic diagram of a three-dimensional scene displayed in the XR studio provided by the present invention;

[0053] Figure 4 A schematic diagram of the steps of a refined three-dimensional modeling method based on an XR studio provided by the present invention.

[0054] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0055] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0056] Example 1

[0057] See also Figure 1 , the embodiment of the present application provides a refined three-dimensional modeling system based on an XR studio, including:

[0058] A three-dimensional modeling unit, configured to perform three-dimensional modeling to obtain a three-dimensional scene; wherein the three-dimensional scene includes a panoramic background image, a three-dimensional model, and a model texture;

[0059] An image generation unit, configured to generate a playback module image for display in an XR studio; wherein the playback module image includes a hard module image and a soft module image;

[0060] A texture processing unit, used to perform upsampling of the model texture to increase the image resolution, or downsampling to reduce the image resolution;

[0061] The modeling generation unit is used to control the texture processing unit to perform image upsampling or downsampling on the model texture according to the human eye resolution threshold, obtain a refined resolution texture and set the display control logic.

[0062] It should be noted that: Figure 3 As shown, this embodiment aims to obtain a three-dimensional scene through a three-dimensional modeling unit, and then generate a playback module image for XR studio display through an image generation unit; based on this, a soft module image can be further obtained, and then based on the soft module image, the resolution difference between the original model map and the mapped model map can be compared. If the resolution difference exceeds the human eye resolution threshold, it means that the resolution difference is large.

[0063] The resolution difference phenomenon is mainly due to the fact that the resolution of the model map changes due to reasons such as bending, resulting in inconsistent resolution and thus resolution difference. The model map needs to be processed in advance, and the resolution of the original model map is static; therefore, we use the modeling generation unit to control the map processing unit to upsample or downsample the model map according to the human eye resolution threshold, obtain a refined resolution map and set the display control logic; in actual use, the model map selects the refined resolution map of the corresponding resolution for display according to the preset display control logic, thereby reducing the resolution difference phenomenon caused by different resolutions.

[0064] Furthermore, the three-dimensional modeling unit includes a panoramic background image editing module, a three-dimensional model editing module and a model texture editing module; wherein, the panoramic background image editing module is used to edit the panoramic background image used, the three-dimensional model editing module is used to edit the three-dimensional model, and the model texture editing module is used to edit the model texture used by the three-dimensional model.

[0065] Furthermore, the image generation unit includes a spherical projection mapping module, a cubic mapping module and an image block division module; wherein the spherical projection mapping module is used to project the three-dimensional scene onto a 360-degree sphere and obtain a spherical plane image; the cubic mapping module is used to map the spherical plane image onto a cube based on the XR studio to obtain a cubic plane image; the image block division module divides the cubic plane image into corresponding playback modules to obtain a hard module image and a soft module image.

[0066] Furthermore, the mapping processing unit includes a resolution acquisition module, an image upsampling module and an image downsampling module; wherein the resolution acquisition module is used to acquire the resolution parameters of the image, the image upsampling module is used to upsample the original image to obtain a corresponding high-resolution image, and the image downsampling module is used to downsample the original image to obtain a corresponding low-resolution image.

[0067] In addition, the modeling generation unit includes a resolution division unit, a map generation unit and a layer setting unit; wherein the resolution division unit is used to divide the corresponding human eye resolution threshold according to the characteristics of the human eye; the map generation unit is used to control the image downsampling module or the image downsampling module, and generate a refined resolution map with a resolution corresponding to the human eye resolution threshold; the layer setting unit is used to set the display control logic of the refined resolution map.

[0068] Example 2

[0069] See also Figure 4 The present invention provides a refined three-dimensional modeling method based on an XR studio, which is applied to a refined three-dimensional modeling system based on an XR studio as described in any of the above items, comprising the following steps:

[0070] Step 1: Performing 3D modeling by a 3D modeling unit to obtain a 3D scene; wherein the 3D scene includes a panoramic background image, a 3D model, and a model texture;

[0071] Step 2: Generate a playback module image for XR studio display through an image generation unit; wherein the playback module image includes a hard module image and a soft module image;

[0072] Step 3: The modeling generation unit controls the texture processing unit to perform image upsampling or downsampling on the model texture according to the human eye resolution threshold;

[0073] Step 4: Obtain a refined resolution map through the modeling generation unit and set the corresponding display control logic; apply the refined resolution map to the 3D model.

[0074] Furthermore, the three-dimensional modeling unit performs three-dimensional modeling through the following steps:

[0075] Edit the used panoramic background image through the panoramic background image editing module;

[0076] Edit the 3D model through the 3D model editing module;

[0077] Edit the model texture used by the 3D model through the model texture editing module.

[0078] Furthermore, the image generation unit generates the playback module image through the following steps:

[0079] The spherical projection mapping module projects the three-dimensional scene onto a 360-degree sphere and obtains a spherical plane image;

[0080] The spherical plane image is mapped onto a cube based on the XR studio through the cube mapping module to obtain a cubic plane image;

[0081] The cubic plane image is divided into corresponding playback modules through the image block division module to obtain hard module images and soft module images.

[0082] Furthermore, the modeling generation unit generates a refined resolution map through the following steps:

[0083] Acquire a soft module image and divide the model map existing in the soft module image to obtain a mapping model map;

[0084] Obtain the resolution of the original model texture and the mapped model texture through the resolution acquisition module;

[0085] The resolution division unit divides the corresponding human eye resolution threshold according to the human eye characteristics;

[0086] Determine whether the resolution of the original model map and the mapped model map exceeds the human eye resolution threshold;

[0087] If it exceeds, the exceeding part is threshold-divided by the human eye resolution threshold until it is all classified within the human eye resolution threshold;

[0088] Otherwise, proceed to the next step;

[0089] The map generation unit controls the image upsampling module or the image downsampling module according to the threshold division to perform image upsampling or downsampling processing on the model map to obtain a refined resolution map of the corresponding division;

[0090] Among them, downsampling is performed when the resolution is higher than the original model map, and upsampling is performed when the resolution is lower than the original model map.

[0091] It should be noted that the human eye resolution threshold can be obtained based on multiple experimental measurements and is a preset parameter value. We use the human eye resolution threshold to divide the exceeding part until it is all divided into the human eye resolution threshold. It can be generally understood as: the numerical range with large resolution difference is divided into several small segments through the human eye resolution threshold. Each small segment is used as a division node to perform image upsampling or downsampling on the model map to obtain the corresponding divided refined resolution map.

[0092] As a further solution, the modeling generation unit sets the display control logic:

[0093] Obtain the resolution of the refined resolution map corresponding to the threshold division, and obtain the refined resolution threshold;

[0094] Set the refinement resolution threshold as the trigger condition;

[0095] When the playback module displays content, the resolution of each soft module image is obtained through the resolution acquisition module;

[0096] Each refined resolution map is triggered according to the resolution of the soft module image.

[0097] The refined resolution map with the smallest resolution difference is selected and displayed in the soft module image.

[0098] like Figure 3 As shown, in a specific embodiment, the texture of the side building will change at the soft module. If it is not processed, it will lead to a large difference in resolution. We will address this phenomenon in advance and divide it into several small segments through the human eye resolution threshold. Each small segment is used as a division node. The model texture is upsampled or downsampled to obtain the corresponding refined resolution texture. During actual display, the resolution of each soft module image will be obtained in real time, which is equivalent to setting up multiple layers of textures but only selecting the refined resolution texture with the smallest resolution difference and displaying it in the soft module image, thereby alleviating the resolution difference.

[0099] The above are only some embodiments of the present application and are not intended to limit the patent scope of the present application. All equivalent structural transformations made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A refined 3D modeling system based on XR studio, characterized by: include: A three-dimensional modeling unit, configured to perform three-dimensional modeling to obtain a three-dimensional scene; wherein the three-dimensional scene includes a panoramic background image, a three-dimensional model, and a model texture; An image generation unit, configured to generate a playback module image for display in an XR studio; wherein the playback module image includes a hard module image and a soft module image; A texture processing unit, used to perform upsampling of the model texture to increase the image resolution, or downsampling to reduce the image resolution; The modeling generation unit is used to control the texture processing unit to perform image upsampling or downsampling on the model texture according to the human eye resolution threshold, obtain a refined resolution texture and set the display control logic; The modeling generation unit generates a refined resolution map through the following steps: Acquire a soft module image and divide the model map existing in the soft module image to obtain a mapping model map; Obtain the resolution of the original model texture and the mapped model texture through the resolution acquisition module; The resolution division unit divides the corresponding human eye resolution threshold according to the human eye characteristics; Determine whether the resolution of the original model map and the mapped model map exceeds the human eye resolution threshold; If it exceeds, the exceeding part is threshold-divided by the human eye resolution threshold until it is all classified within the human eye resolution threshold; Otherwise, proceed to the next step; The map generation unit controls the image upsampling module or the image downsampling module according to the threshold division to perform image upsampling or downsampling processing on the model map to obtain a refined resolution map of the corresponding division; Among them, downsampling is performed when the resolution is higher than the original model map, and upsampling is performed when the resolution is lower than the original model map; The modeling generation unit sets the display and control logic: Obtain the resolution of the refined resolution map corresponding to the threshold division, and obtain the refined resolution threshold; Set the refinement resolution threshold as the trigger condition; When the playback module displays content, the resolution of each soft module image is obtained through the resolution acquisition module; Each refined resolution map is triggered according to the resolution of the soft module image. The refined resolution map with the smallest resolution difference is selected and displayed in the soft module image.

2. The refined 3D modeling system based on XR studio according to claim 1, characterized in that: The three-dimensional modeling unit includes a panoramic background image editing module, a three-dimensional model editing module and a model texture editing module; wherein, the panoramic background image editing module is used to edit the panoramic background image used, the three-dimensional model editing module is used to edit the three-dimensional model, and the model texture editing module is used to edit the model texture used by the three-dimensional model.

3. The refined 3D modeling system based on XR studio according to claim 1, characterized in that: The image generation unit includes a spherical projection mapping module, a cubic mapping module and an image block division module; wherein the spherical projection mapping module is used to project a three-dimensional scene onto a 360-degree sphere and obtain a spherical plane image; the cubic mapping module is used to map the spherical plane image onto a cube based on the XR studio prototype to obtain a cubic plane image; the image block division module divides the cubic plane image into corresponding playback modules to obtain a hard module image and a soft module image.

4. The refined 3D modeling system based on XR studio according to claim 1, characterized in that: The mapping processing unit includes a resolution acquisition module, an image upsampling module and an image downsampling module; wherein the resolution acquisition module is used to obtain the resolution parameters of the image, the image upsampling module is used to upsample the original image to obtain a corresponding high-resolution image, and the image downsampling module is used to downsample the original image to obtain a corresponding low-resolution image.

5. The refined 3D modeling system based on XR studio according to claim 4, characterized in that: The modeling generation unit includes a resolution division unit, a map generation unit and a layer setting unit; wherein the resolution division unit is used to divide the corresponding human eye resolution threshold according to the characteristics of the human eye; the map generation unit is used to control the image downsampling module or the image downsampling module, and generate a refined resolution map with a resolution corresponding to the human eye resolution threshold; the layer setting unit is used to set the display control logic of the refined resolution map.

6. A refined 3D modeling method based on an XR studio, applied to a refined 3D modeling system based on an XR studio as claimed in any one of claims 1 to 5, characterized in that: The steps include: Step 1: Performing 3D modeling by a 3D modeling unit to obtain a 3D scene; wherein the 3D scene includes a panoramic background image, a 3D model, and a model texture; Step 2: Generate a playback module image for XR studio display through an image generation unit; wherein the playback module image includes a hard module image and a soft module image; Step 3: The modeling generation unit controls the texture processing unit to perform image upsampling or downsampling on the model texture according to the human eye resolution threshold; Step 4: Obtain a refined resolution map through the modeling generation unit and set the corresponding display control logic; apply the refined resolution map to the 3D model; Acquire a soft module image and divide the model map existing in the soft module image to obtain a mapping model map; Obtain the resolution of the original model texture and the mapped model texture through the resolution acquisition module; The resolution division unit divides the corresponding human eye resolution threshold according to the human eye characteristics; Determine whether the resolution of the original model map and the mapped model map exceeds the human eye resolution threshold; If it exceeds, the exceeding part is threshold-divided by the human eye resolution threshold until it is all classified within the human eye resolution threshold; Otherwise, proceed to the next step; The map generation unit controls the image upsampling module or the image downsampling module according to the threshold division to perform image upsampling or downsampling processing on the model map to obtain a refined resolution map of the corresponding division; Among them, downsampling is performed when the resolution is higher than the original model map, and upsampling is performed when the resolution is lower than the original model map; The modeling generation unit sets the display and control logic: Obtain the resolution of the refined resolution map corresponding to the threshold division, and obtain the refined resolution threshold; Set the refinement resolution threshold as the trigger condition; When the playback module displays content, the resolution of each soft module image is obtained through the resolution acquisition module; Each refined resolution map is triggered according to the resolution of the soft module image. The refined resolution map with the smallest resolution difference is selected and displayed in the soft module image.

7. The method for refined three-dimensional modeling based on XR studio according to claim 6, characterized in that: The three-dimensional modeling unit performs three-dimensional modeling through the following steps: Edit the used panoramic background image through the panoramic background image editing module; Edit the 3D model through the 3D model editing module; Edit the model texture used by the 3D model through the model texture editing module.

8. The method for refined three-dimensional modeling based on XR studio according to claim 7, characterized in that: The image generation unit generates the playback module image through the following steps: The spherical projection mapping module projects the three-dimensional scene onto a 360-degree sphere and obtains a spherical plane image; The spherical plane image is mapped onto a cube based on the XR studio through the cube mapping module to obtain a cubic plane image; The cubic plane image is divided into corresponding playback modules through the image block division module to obtain hard module images and soft module images.

Images