User-defined background model rendering method and device, equipment and storage medium

Abstract

The invention provides a user-defined background model rendering method and device, equipment and a storage medium. The method comprises the steps of determining background visual elements of a target scene according to a target picture; wherein the background visual element comprises at least part of the image of the target picture; binding a texture coordinate system of the target picture with a screen space coordinate system so as to enable the background visual elements to keep a fixed perspective relationship in the operation interface; obtaining a target model; in response to transformation operation for the camera and / or the target model, determining camera parameters and spatial pose parameters of the target model; obtaining a target scene according to the spatial pose parameter, the camera parameter and the background visual element of the target model; and generating a rendering graph of the target scene. According to the scheme disclosed by the invention, the background image can be kept locked when the model or the camera is adjusted, so that the target scene matched with the perspective effect can be quickly and conveniently obtained.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and particularly to the fields of data processing and 3D design software technologies. Background Art

[0002] Currently, when performing image rendering in 3D rendering software and tools, it may be necessary to splice the rendered image with a real scene photo. To achieve this purpose, users usually rely on a semi-transparent screenshot tool to take a screenshot of the rendered image and then place it in Photoshop or other image editing software to manually adjust the perspective and match the real scene photo. However, this method is cumbersome to operate, and users cannot directly adjust the perspective angle of the rendered image in the rendering interface, thus affecting the splicing accuracy and efficiency. Summary of the Invention

[0003] The present disclosure provides a method, apparatus, device, and storage medium for rendering a model with a custom background to solve or alleviate one or more technical problems in the prior art.

[0004] In a first aspect, the present disclosure provides a method for rendering a model with a custom background, including:

[0005] Determining background visual elements of a target scene according to a target picture; wherein, the background visual elements include at least part of the image of the target picture;

[0006] Binding the texture coordinate system of the target picture to the screen space coordinate system so that the background visual elements maintain a fixed perspective relationship in the operation interface;

[0007] Obtaining a target model;

[0008] Responding to a transformation operation on the camera and / or the target model, and determining camera parameters and spatial pose parameters of the target model;

[0009] Obtaining a target scene according to the spatial pose parameters of the target model, the camera parameters, and the background visual elements;

[0010] Generating a rendered image of the target scene.

[0011] In a second aspect, the present disclosure provides a device for rendering a model with a custom background, including:

[0012] A picture acquisition module, configured to determine background visual elements of a target scene according to a target picture; wherein, the background visual elements include at least part of the image of the target picture;

[0013] A background binding module, configured to bind the texture coordinate system of the target picture to the screen space coordinate system so that the background visual elements maintain a fixed perspective relationship in the operation interface;

[0014] A model acquisition module, configured to acquire a target model;

[0015] An interaction adjustment module, configured to determine camera parameters and spatial pose parameters of the target model in response to a transformation operation on the camera and / or the target model;

[0016] A scene determination module, configured to obtain a target scene according to the spatial pose parameters of the target model, the camera parameters, and background visual elements;

[0017] A generation module, configured to generate a rendering of the target scene.

[0018] In a third aspect, an electronic device is provided, including:

[0019] At least one processor; and

[0020] A memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute any method in the embodiments of the present disclosure.

[0022] In a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute any method in the embodiments of the present disclosure.

[0023] In a fifth aspect, a computer program product is provided, including a computer program which, when executed by a processor, implements any method in the embodiments of the present disclosure.

[0024] The beneficial effects of the technical solution provided by the present disclosure at least include:

[0025] Any planar photo or picture can be used as a custom background. When adjusting the model and the camera, the background can be locked, which is convenient for fusing the background image with the 3D model and realizing the splicing of the model and the picture.

[0026] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. Description of the Drawings

[0027] In the drawings, unless otherwise specified, the same reference numerals throughout the drawings denote the same or similar components or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments provided by the present disclosure and should not be regarded as limiting the scope of the present disclosure.

[0028] Figure 1 is a schematic flowchart of a method for rendering a model with a custom background provided by an embodiment of the present disclosure;

[0029] Figure 2 is a schematic flowchart of a method for rendering a model with a custom background provided by another embodiment of the present disclosure;

[0030] Figure 3 is a schematic structural diagram of a device for rendering a model with a custom background provided by an embodiment of the present disclosure;

[0031] Figure 4 is a block diagram of an electronic device for implementing the method for rendering a model with a custom background according to an embodiment of the present disclosure. Detailed implementation manners

[0032] The present disclosure will be further described in detail below with reference to the accompanying drawings. The same reference numerals in the drawings denote elements with the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings do not have to be drawn to scale unless otherwise specified.

[0033] In addition, for a better illustration of the present disclosure, numerous specific details are given in the following detailed implementation manners. Those skilled in the art should understand that the present disclosure can also be implemented without some specific details. In some instances, methods, means, elements, and circuits well known to those skilled in the art are not described in detail so as to highlight the gist of the present disclosure.

[0034] In some 3D design software or rendering software, such as C4D + Vray, a function of setting a background image as the viewport background is provided. A photo of a real scene can be used as the background image, which is merged with the model during rendering to obtain a rendered image, so as to simulate the effect of the model in a real scene. However, the perspective effect of the background image itself usually does not match the initial pose of the model and the camera view. As a planar image, the perspective effect of the background image itself cannot be changed, and the user needs to adjust the pose of the model, the position and view of the camera, etc., so that the perspective effect of the model matches the perspective effect of the background image. However, in the related art, each time the model or the camera is adjusted, the background image also changes accordingly, resulting in the problem that the background image and the model cannot be accurately docked.

[0035] To at least partially solve one or more of the above problems and other potential problems, embodiments of the present disclosure provide a method for rendering a model with a custom background. By using the technical solution of the embodiments of the present disclosure, any planar photo or picture can be used as a custom background. When adjusting the model and the camera, the background is locked, which is convenient for the fusion of the background image and the 3D model, and realizes the splicing of the model and the picture.

[0036] Figure 1 It is a schematic flowchart of a model rendering method for customizing a background according to an embodiment of the present disclosure. As Figure 1 shown, the method at least includes the following steps:

[0037] S101. Determine the background visual elements of the target scene according to the target picture. Among them, the background visual elements include at least part of the image of the target picture.

[0038] In an embodiment of the present disclosure, the target picture may be a real-scene photo uploaded by the user or a scene picture preset by the system (such as in JPG, PNG format, etc.), serving as the background reference for rendering the scene. The background visual elements can be understood as at least part of the image area extracted from the target picture, which is used to synthesize the final rendering with the 3D model. The target picture can be cropped or scaled to obtain a partial image area.

[0039] In one example, the user uploads a local picture through the operation interface or selects a picture from the system preset library. The picture can be cropped and scaled to fit the aspect ratio of the rendering scene.

[0040] If the target picture contains an alpha channel (such as in PNG format), the system automatically recognizes the transparent area and retains the transparent part in the subsequent synthesis to ensure no occlusion when the background is superimposed on the model.

[0041] The user can select a local area of the target picture (such as the wall and floor in a real-scene photo) through the selection tool, and generate a sub-texture according to the selection coordinates as a fixed background element.

[0042] S102. Bind the texture coordinate system of the target picture to the screen space coordinate system, so that the background visual elements maintain a fixed perspective relationship in the operation interface.

[0043] In an embodiment of the present disclosure, the texture coordinate system is a two-dimensional coordinate system with the upper left corner of the target picture as the origin (0, 0) and the lower right corner as (1, 1), which is used to locate the background image pixels.

[0044] The screen space coordinate system is a two-dimensional coordinate system with the upper left corner of the operation interface window as the origin (0, 0) and the lower right corner as (screen_width, screen_height), which describes the actual position of the pixels on the screen.

[0045] In one example, the UV coordinates (u, v) of the background texture can be linearly mapped to the screen space coordinates (u * screen_width, v * screen_height) to ensure a one-to-one correspondence between the background image pixels and the screen pixels.

[0046] Maintaining a fixed perspective relationship can be understood as perspective locking. In the rendering pipeline, the background layer is independent of the model view matrix and projection matrix of the 3D scene and only depends on the screen space coordinate system. Regardless of how the camera or model is transformed, the background visual elements always maintain a fixed perspective relationship.

[0047] S103. Obtain the target model.

[0048] In the embodiments of the present disclosure, the user can import or select a 3D model file (such as OBJ, GLTF format) from the resource library, which contains data such as meshes, materials, and textures. After import, the vertex, normal, and texture coordinate data of the model file are parsed, the model matrix is initialized to the identity matrix, and it is placed at the origin of the scene coordinate system. In another example, the target model can also be directly created by the user through editing in the interface.

[0049] It should be noted that the order of S101 and S103 can be swapped. The model can be imported first and then the target image, that is, S103 can be before S101 and S102.

[0050] S104. In response to a transformation operation on the camera and / or the target model, determine the camera parameters and the spatial pose parameters of the target model.

[0051] In the embodiments of the present disclosure, the camera parameters can be understood as the parameters describing the camera view angle, usually including the position (x, y, z), the viewing direction (look_at_x, look_at_y, look_at_z), the focal length fov, etc. The spatial pose parameters can be understood as the parameters describing the position and rotation state of the model in the 3D space.

[0052] After importing the model, it is necessary to adjust the camera or the target model according to the user's interaction operations. The interaction operation responses can include:

[0053] Camera adjustment: When the user rotates the view angle by dragging the mouse and zooms the field of view by using the scroll wheel, the camera position and view angle parameters are calculated in real time.

[0054] Model transformation: When the user drags the model by using the translation / rotation tool, the system updates the translation matrix T and rotation matrix R of the model, and synthesizes the model pose matrix M = T×R.

[0055] S105. Obtain the target scene according to the spatial pose parameters of the target model, the camera parameters, and the background visual elements.

[0056] After the adjustment in S104, the new camera position and view angle, and the model position and pose are obtained. At this time, in the view angle of the camera, the position and pose of the model should be similar to or match the perspective effect of the target image, so as to obtain the target scene.

[0057] S106. Generate a rendering of the target scene.

[0058] In the embodiments of the present disclosure, after the user clicks the "Render Image" button, the background visual elements and the target model included in the target scene can be rendered to render the final image at a high resolution, which supports export in formats such as PNG and JPEG, avoiding the multi-step operations of screenshotting and manual stitching in the traditional process.

[0059] During rendering, through rendering layer separation, the background layer can be preferentially drawn as the bottom rendering channel, disabling depth testing and lighting calculation; the model layer is enabled for depth testing as the upper channel to ensure that the model is always superimposed on the background.

[0060] According to the solution of the embodiments of the present disclosure, through texture-screen coordinate binding, it is ensured that the background image is not affected by camera / model transformation, solving the problem of stitching distortion caused by background offset in the prior art.

[0061] In a possible implementation manner, the rendering method further includes the steps of:

[0062] When switching from the current operation interface to the target operation interface, according to the camera parameters and model parameters of the current operation interface, synchronize the camera parameters and model parameters of the target operation interface. Wherein, the current operation interface is one of a model editing interface or a rendering pop-up window interface, and the target operation interface is the other of the two.

[0063] In the embodiments of the present disclosure, the model parameters include spatial pose parameters and vertex parameters. When the user switches from the current operation interface (such as a model editing interface) to the target operation interface (such as a rendering pop-up window interface), the system automatically synchronizes the camera parameters, model vertex parameters, and model spatial pose parameters of the current interface to the target interface, ensuring that the editing of the camera or the model can be synchronized to the other interface, so that when switching between the two interfaces, the perspective relationship between the model and the background is consistent.

[0064] The model editing interface and the rendering pop-up window interface are two operation interfaces with clear division of labor, respectively targeting the two core processes of modeling and rendering. The model editing interface supports users to import, create, or edit the geometric structure (such as vertex, edge, and face editing), material properties (such as color, texture mapping), and animation parameters of 3D models. Tools such as translation, rotation, and scaling are provided to allow users to adjust the position, orientation, and scale of the model in 3D space. When users need to finely adjust the model structure to match the objects in the background image, they can do so in the model editing interface. For example, align the sofa model with the living room layout in the real scene photo.

[0065] The rendering pop-up interface allows users to set lighting (such as light source type, intensity, direction), camera parameters (such as focal length, depth of field, anti-aliasing level), and post-processing effects (such as ambient occlusion, depth of field blur). After the user finishes adjusting the model, they need to quickly preview the rendering effect and generate the final image that seamlessly composes with the background image.

[0066] During the modeling stage, users need to frequently adjust the model details, with high requirements for real-time interaction but relatively low requirements for rendering accuracy. During the rendering stage, users need to focus on fine-tuning the visual effects (such as matching the lighting angle with the background perspective). At this time, hiding the modeling tools can reduce interference.

[0067] The two interfaces maintain data consistency through a parameter synchronization mechanism (such as camera position, model pose). For example, when the user adjusts the model rotation in the model editing interface and then switches to the rendering pop-up window, the model state is automatically inherited, and the background perspective remains locked to avoid repeated adjustments.

[0068] In one example, the user imports a sofa model in the model editing interface, adjusts its size and position according to the background image, and aligns it with the living room layout in the real scene photo.

[0069] Click the "Enter Rendering Mode" button, and the system automatically switches to the rendering pop-up interface, synchronizes the current model pose and camera parameters, and locks the background perspective.

[0070] In the rendering pop-up window, the user adjusts the lighting direction to match the light and shadow effect of the real scene photo, enables depth of field blur to enhance the sense of reality, and then clicks "Render Image" to generate the final composite image.

[0071] If it is found that the model scale does not match, the user can return to the model editing interface to continue adjusting, and the parameters in the rendering pop-up window (such as lighting settings) will be retained to ensure the continuity of the process.

[0072] According to the solution of the embodiments of the present disclosure, the division of labor design between the model editing interface and the rendering pop-up interface not only meets the efficient operation requirements during the modeling stage but also realizes professional control during the rendering stage. At the same time, the seamless connection between the two is ensured through the parameter synchronization mechanism. This design significantly reduces the user's learning cost, avoids misoperations caused by function mixing, and ultimately improves the overall efficiency and accuracy from model adjustment to rendering image generation. The adjustment of the model and camera parameters is real-time feedback to the other interface, supporting continuous editing across interfaces, improving the overall process efficiency from model adjustment to rendering image generation, and facilitating rapid iteration of the design scheme.

[0073] In one possible implementation, the rendering method further includes the steps of:

[0074] In the rendering pop-up interface, update the camera parameters and / or the spatial pose parameters of the target model according to the transformation operations on the camera and / or the target model.

[0075] Render the target model according to the spatial pose parameters and camera parameters of the target model.

[0076] In the embodiments of the present disclosure, in the rendering pop-up window interface, the user can continue to adjust the spatial pose of the target model and the camera parameters, and the system updates the parameters in real time and renders a preview to ensure that the user can intuitively observe the matching effect between the model and the background, and finally provides the ability of dynamic adjustment for generating high-quality renderings.

[0077] According to the solution of the embodiments of the present disclosure, when the user adjusts the model or the camera, the preview screen is updated immediately, significantly shortening the trial-and-error cycle.

[0078] In one possible implementation, step S105 further includes steps of obtaining a target scene according to the spatial pose parameters, camera parameters, and background visual elements of the target model:

[0079] S1051. When the operation interface is the rendering pop-up window interface, render the target model on top of the background visual elements according to the spatial pose parameters, model texture map, lighting parameters, and camera parameters of the target model.

[0080] In the embodiments of the present disclosure, the background visual elements are drawn into the screen space coordinate system, and the perspective relationship is fixed (not changed with the transformation of the camera or the model). If the background contains transparent areas (such as window parts), keep its Alpha channel for subsequent synthesis.

[0081] Calculate the coordinates of the vertices of the target model in the world coordinate system according to the translation matrix T, rotation matrix R, and scaling matrix S of the target model.

[0082] Construct a view-projection matrix based on the camera parameters (Pos, LookAt, FOV), and transform the model vertices to the screen coordinate system.

[0083] Apply the material texture map and lighting parameters (such as the direction of parallel light, attenuation of point light source), and perform shading calculations (such as Phong lighting model) to generate the model color and depth information.

[0084] Overlay the model layer (including depth buffer) on the background layer, and enable depth testing to ensure that the model correctly occludes the background (such as the model is in front of the table in the background).

[0085] The applied post-processing effects include: anti-aliasing (MSAA), ambient occlusion (SSAO), depth of field blur (dynamically adjusted according to the camera focal length), to generate the final preview screen.

[0086] S1052. In response to the transformation operation on the camera and / or the target model, update the camera parameters and the spatial pose parameters of the target model.

[0087] In the embodiments of the present disclosure, by capturing interaction events in the operation interface, a change operation on the model or the camera is determined:

[0088] The model transformation operation may be that after the user selects a target model, by dragging a handle (translation / rotation / scaling Gizmo) or directly dragging the surface of the model, the pose update of the model is triggered.

[0089] For example: Drag the translation handle to move the model along the X-axis, and the system calculates the displacement ΔX in real time and updates the translation matrix T.

[0090] The camera transformation operation may be that the user holds down the right mouse button and drags to rotate the camera view, or scrolls the mouse wheel to adjust the camera focal length (FOV).

[0091] For example: When dragging the mouse horizontally, the yaw angle yaw of the camera increases, and the camera viewing direction LookAt vector is updated.

[0092] Model pose update:

[0093] According to the operation type (translation / rotation / scaling) and the displacement of the input device (mouse / touchpad), the T, R, and S matrices of the model are updated.

[0094] Example: During a rotation operation, the mouse movement distance is converted into a rotation angle θ around the selected axis (such as the Y-axis), and the rotation matrix R is updated to R_new = R_old × RotateY(θ).

[0095] Camera parameter update:

[0096] According to the mouse drag distance and the sensitivity coefficient, the camera position Pos or the viewing direction LookAt is updated.

[0097] Example: When dragging the right mouse button, according to the screen Δx and Δy, the change amount of the camera Euler angle is calculated, and the LookAt vector is updated to a new direction.

[0098] S1053. Obtain a target scene according to the camera parameters, the background visual elements, and the spatial pose parameters of the target model.

[0099] Based on the updated camera parameters, model pose parameters, and background visual elements, re-perform rendering to generate a new target scene.

[0100] In the rendering pop-up window interface, the user can dynamically adjust the spatial pose of the target model and the camera parameters through interactive operations. The system updates the parameters in real time and based on the latest parameters, fuses and renders the model and the background visual elements into a "target scene", finally forming a high-precision preview image synchronized with the user's operations. This process ensures the perspective consistency between the model and the background and the authenticity of the visual effect through the mechanism of parameter update - scene reconstruction - real-time rendering.

[0101] According to the solution of the embodiments of the present disclosure, the background perspective is fixed, and the model and camera parameters are dynamically updated according to the user's operations, ensuring the geometric authenticity of the synthesized scene.

[0102] In a possible implementation manner, step S105 obtains the target scene according to the spatial pose parameters, camera parameters, and background visual elements of the target model, and further includes the steps of:

[0103] S1054. When the operation interface is a model editing interface, display the target model in an editable state according to the spatial pose parameters and camera parameters of the target model.

[0104] In the embodiments of the present disclosure, when the user switches from the rendering pop-up window interface to the model editing interface, the system synchronizes the spatial pose parameters (T, R, S matrices) of the current target model and the camera parameters (such as Pos, LookAt) to the editing interface to ensure that the initial state of the model is consistent with the preview in the rendering pop-up window. In the model editing interface, the target model is in an editable state. The model mesh vertices are displayed with interactive markers (such as highlighted dots). The world coordinate system of the model editing interface is exactly the same as the coordinate system in the rendering pop-up window, avoiding model pose offset caused by interface switching.

[0105] S1055. Adjust the vertex parameters of the target model according to the editing operations on the target model.

[0106] In an example, adjusting the model vertex parameters according to the editing operations may include: the user selects one or more vertices by means of box selection or point selection, and the selected vertices are marked in a highlighted state; drag the selected vertices to move along the screen coordinate system or the model local coordinate system, or adjust the vertex tangent direction through the control handle to modify the model surface shape, etc. According to the mouse drag distance (screen Δx, Δy), combined with the inverse operation of the current camera view-projection matrix, convert the screen displacement into the vertex displacement amount (ΔX, ΔY, ΔZ) in the model local coordinate system. Example: When dragging a vertex to move horizontally along the screen, the system calculates its displacement amount on the local X axis of the model and updates the vertex coordinate V_new = V_old + (ΔX, 0, 0).

[0107] S1056. Obtain the target scene according to the camera parameters, background visual elements, and the adjusted target model.

[0108] After several adjustments, the perspective effect of the target model is made similar to or matched with the perspective effect of the target picture under the perspective of the camera, so as to obtain the target scene that meets the user's requirements.

[0109] In the model editing interface, the user can directly edit the geometric vertex data of the target model (such as adjusting the shape, vertex position, etc.). The system updates the model vertex parameters in real time according to the editing operations, and dynamically renders the target scene after the model is fused with the background based on the current camera parameters and background visual elements. Through the process of vertex-level editing - parameter update - scene reconstruction, an integrated operation of synchronously modifying the geometric structure of the model and previewing the synthesis effect in the model editing interface is realized.

[0110] In a possible implementation, when the current interface is a rendering pop-up window interface, S1051 renders the target model on top of the background visual elements according to the spatial pose parameters, model texture map, lighting parameters, and camera parameters of the target model, and further includes:

[0111] S1051a: Determine the projection-view matrix of the target model based on the perspective parameters and camera parameters of the screen space coordinate system.

[0112] S1051b: Render the background visual elements to the bottom layer of the target scene preferentially according to the mapping relationship between the bound texture coordinate system and the screen space coordinate system.

[0113] S1051c: Render the target model on top of the bottom layer of the target scene according to the model parameters, model texture map, and projection-view matrix of the target model.

[0114] In the embodiments of the present disclosure, determining the projection-view matrix of the target model includes: defining the size of the screen viewport (such as 1920×1080 pixels), the origin position (upper left corner or center point), and the perspective parameters (such as orthographic projection or perspective projection mode). Calculate the transformation matrix (ViewMatrix) from the world coordinate system to the camera coordinate system according to the camera position (Pos) and the viewing direction (LookAt). Generate a perspective projection matrix according to the FOV, aspect ratio of the viewport, and near / far clipping planes of the screen space coordinate system. Multiply the transformation matrix by the perspective projection matrix, so as to convert the model vertices from the world coordinate system to the screen space coordinate system.

[0115] As described above, a one-to-one mapping is established between the UV coordinate system of the background visual elements and the four corner vertices of the screen space coordinate system, so as to realize directly mapping the background visual elements as an environment texture map to the screen space coordinate system. When rasterizing the mapped screen coordinates, the 1:1 correspondence between the background image texture coordinates and the screen pixels is maintained. After rendering the background visual elements to the bottom layer of the target scene, the model is then rendered on top of the bottom layer, so as to fuse the 2D picture with the 3D model.

[0116] In a possible implementation, step S1051c: Render the target model on top of the bottom layer of the target scene according to the model parameters, model texture map, and projection-view matrix of the target model, and further includes:

[0117] Transform the vertex data of the target model to screen space through a projection-view matrix to generate projection geometry information including screen space coordinates and fragment depth values.

[0118] Sample the model texture map based on the texture coordinates interpolated from the screen space coordinates, and determine the pixel color value in combination with the lighting parameters.

[0119] According to the comparison result between the fragment depth value and the background depth buffer, cover the pixel color value on top of the underlying layer of the target scene.

[0120] In the embodiments of the present disclosure, the vertex shader inputs the model vertex data, applies the spatial pose parameters of the model, and calculates the world coordinates. The world coordinates are converted to clip space coordinates through a projection-view matrix, and perspective division is performed to normalize the clip space coordinates to screen space coordinates. Finally, the normalized coordinates are converted to actual pixel coordinates, and the linear depth value of the fragment is recorded. The vertex shader outputs the screen space coordinates (PixelX, PixelY), depth value Depth, interpolated texture coordinates (u, v), and normal vector (nx, ny, nz) to the rasterization stage.

[0121] Divide the screen pixel area covered by the model triangle patches into fragments, and interpolate the attributes (texture coordinates, normal vector) of each fragment according to the barycentric coordinates. Perform perspective-corrected interpolation on the texture coordinates (u, v) and the normal vector to avoid texture stretching problems caused by perspective projection. Sample the base color from the model texture map (such as DiffuseMap) according to the interpolated texture coordinates (u, v). Then calculate the pixel color through the lighting model. Finally, each fragment outputs the final color value FinalColor and depth value Depth for use in the subsequent synthesis stage.

[0122] Before rendering the model, load the depth values of the background visual elements (such as through pre-rendering or depth maps) into the depth buffer for depth comparison. Only when the depth value of the model fragment is less than the depth value at the corresponding position in the background depth buffer, is it allowed to cover the pixel. If the depth test passes, update the value in the depth buffer to the current fragment depth value at the same time to ensure correct occlusion of subsequent fragments. Finally, write the synthesized pixel color into the frame buffer to generate the rendering result of the target scene.

[0123] According to the solution of the embodiments of the present disclosure, the geometric data of the target model is converted into screen space coordinates, combined with texture sampling and lighting calculation to generate pixel colors, and then the model pixels are accurately superimposed on the background based on the depth test results. This process ensures the correct depth relationship between the model and the background, and at the same time realizes high-fidelity material and light and shadow effects, and core solves the problems of occlusion errors between the model and the background, texture distortion, and lighting mismatch in traditional rendering.

[0124] In a possible implementation, S101 determines the background visual elements of the target scene according to the target picture, which further includes the steps of:

[0125] Load the corresponding scene picture according to the selected scene determined from multiple preset scenes.

[0126] Determine the background visual elements of the target scene according to the scene picture.

[0127] Or

[0128] Obtain the uploaded custom picture from the server side.

[0129] Determine the background visual elements of the target scene according to the custom picture.

[0130] In the embodiments of the present disclosure, multiple preset scenes can be provided as a scene library. The preset scene library can include various different scene types, such as "indoor living room", "outdoor park", "virtual exhibition hall". Display a thumbnail list of the preset scenes on the client interface or the web page interface. After the user clicks on the selected scene, a loading request is triggered.

[0131] In another way, the flexibility of the scene can be improved by supporting the user to customize the background by uploading the custom picture provided by the user to the server.

[0132] Determine the background visual elements of the target scene through a dual-path mechanism of matching with the preset scene library or dynamically loading the custom picture. This step solves the problems of single background selection and complex user customization adaptation in the traditional solution, and realizes the balance between fast scene switching and personalized background integration.

[0133] Figure 2 It is a schematic flow chart of the model rendering method for customizing the background provided by another embodiment of the present disclosure. As Figure 2 shown, the method includes the following steps:

[0134] The user selects a preset scene or uploads a real scene photo as the background picture.

[0135] The system automatically sets the background picture as the fixed background of the rendering scene and ensures that the perspective of the background picture is locked.

[0136] The user adjusts the rendering model through the interface, and the system automatically adapts the perspective of the background image and the camera angle.

[0137] After the user completes the matching of the model and the background image, the target scene is determined, and the user can save or continue to perform rendering to obtain the rendering of the target scene.

[0138] In the embodiments of the present disclosure, the front-end implementation includes:

[0139] Providing preset scenes: A series of selected preset scenes are provided, which are designed based on actual usage requirements, enabling users to quickly select and directly apply them to rendering projects, improving efficiency and simplifying operations.

[0140] Providing preset scenes and customizing the background image: Users can upload a background image in formats such as JPG and set it as the fixed background image in the rendering pop-up window. The image upload is deeply integrated with cloud object storage technology to ensure that the uploaded image can be securely and quickly stored in the cloud and efficiently called during the rendering process.

[0141] Background image loading and locking function: The background image perspective adopts the screen mapping type to ensure that the perspective effect of the background image does not change due to the rotation of the camera or the model during the rendering process.

[0142] Model rotation and background image adaptation: The user rotates the rendering model (such as a door head, a building, etc.) in the rendering pop-up window interface, and through the dynamic adjustment of the system, the perspective effect of the background image is adapted to the new rendering angle in real time to ensure that the rendering and the real scene photo always maintain the same perspective relationship.

[0143] Synchronization of the background image and the model coordinates: The coordinate system of the background image is bound to the coordinate system of the rendering model to ensure that each rotation of the model will automatically adjust the position and perspective effect of the background image to match the new camera angle.

[0144] Backend implementation: Use environment mapping to implement the rendering of the background image.

[0145] Figure 3 It is a schematic structural diagram of a model rendering device with a custom background provided by an embodiment of the present disclosure. As Figure 3 shown, the device includes:

[0146] An image acquisition module 301, configured to determine the background visual elements of the target scene according to the target image; wherein, the background visual elements include at least part of the image of the target image;

[0147] A background binding module 302, configured to bind the texture coordinate system of the target image to the screen space coordinate system, so that the background visual elements maintain a fixed perspective relationship in the operation interface;

[0148] The model acquisition module 303 is configured to acquire a target model;

[0149] The interaction adjustment module 303 is configured to determine camera parameters and spatial pose parameters of the target model in response to a transformation operation on the camera and / or the target model;

[0150] The scene determination module 305 is configured to obtain a target scene according to the spatial pose parameters of the target model, the camera parameters, and the background visual elements;

[0151] The generation module 306 is configured to generate a rendering of the target scene.

[0152] For the specific functions and examples of the modules and sub - modules of the device in the embodiments of the present disclosure, reference may be made to the relevant descriptions of the corresponding steps in the above - mentioned method embodiments, which will not be elaborated herein.

[0153] In a possible implementation manner, the device further includes:

[0154] The synchronization module is configured to synchronize the camera parameters and model parameters of the target operation interface according to the camera parameters and model parameters of the current operation interface when switching from the current operation interface to the target operation interface; wherein, the current operation interface is one of a model editing interface or a rendering pop - up window interface, and the target operation interface is the other one.

[0155] In a possible implementation manner, the device further includes:

[0156] The real - time rendering module is configured to update the camera parameters and / or the spatial pose parameters of the target model according to a transformation operation on the camera and / or the target model in the rendering pop - up window interface;

[0157] Render the target model according to the spatial pose parameters of the target model and the camera parameters.

[0158] In a possible implementation manner, the scene determination module 205 is configured to:

[0159] In the case where the operation interface is a rendering pop - up window interface, render the target model above the background visual elements according to the spatial pose parameters of the target model, the model texture, the lighting parameters, and the camera parameters;

[0160] In response to a transformation operation on the camera and / or the target model, update the camera parameters and the spatial pose parameters of the target model;

[0161] Obtain a target scene according to the camera parameters, the background visual elements, and the spatial pose parameters of the target model.

[0162] In a possible implementation, the scene determination module 205 is configured to:

[0163] When the operation interface is a model editing interface, display the target model in an editable state according to the spatial pose parameters and camera parameters of the target model;

[0164] Adjust the vertex parameters of the target model according to the editing operations on the target model;

[0165] Obtain a target scene according to the camera parameters, the background visual elements, and the adjusted target model.

[0166] In a possible implementation, the scene determination module 205 is configured to:

[0167] Determine the projection-view matrix of the target model based on the perspective parameters and camera parameters of the screen space coordinate system;

[0168] Render the background visual elements to the bottom layer of the target scene preferentially according to the mapping relationship between the bound texture coordinate system and the screen space coordinate system;

[0169] Render the target model above the bottom layer of the target scene according to the model parameters, model textures, and the projection-view matrix of the target model.

[0170] In a possible implementation, the scene determination module 205 is configured to:

[0171] Transform the vertex data of the target model to the screen space through the projection-view matrix to generate projection geometry information including screen space coordinates and fragment depth values;

[0172] Sample the model textures based on the texture coordinates interpolated from the screen space coordinates, and determine the pixel color values in combination with the lighting parameters;

[0173] Overlay the pixel color values above the bottom layer of the target scene according to the comparison result between the fragment depth values and the background depth buffer.

[0174] Figure 4 It is a structural block diagram of an electronic device according to an embodiment of the present disclosure. As Figure 4As shown, the electronic device includes: a memory 410 and a processor 420. The memory 410 stores a computer program that can run on the processor 420. The number of the memory 410 and the processor 420 can be one or more. The memory 410 can store one or more computer programs. When the one or more computer programs are executed by the electronic device, the electronic device is caused to execute the method provided in the foregoing method embodiments. The electronic device may further include: a communication interface 430, configured to communicate with external devices and perform data interaction and transmission.

[0175] If the memory 410, the processor 420, and the communication interface 430 are implemented independently, the memory 410, the processor 420, and the communication interface 430 can be interconnected through a bus and complete communication with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 only a thick line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0176] Optionally, in a specific implementation, if the memory 410, the processor 420, and the communication interface 430 are integrated on a chip, the memory 410, the processor 420, and the communication interface 430 can complete communication with each other through an internal interface.

[0177] It should be understood that the foregoing processor can be a Central Processing Unit (CPU), or can also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc. It is worth noting that the processor can be a processor that supports the Advanced RISC Machines (ARM) architecture.

[0178] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory, and may further include a non-volatile random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink dynamic random access memory (SLDRAM), and direct rambus random access memory (DR RAM).

[0179] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present disclosure are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wirelessly (such as infrared, Bluetooth, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a Digital Versatile Disc (DVD)), or a semiconductor medium (such as a Solid State Disk (SSD)), etc. It should be noted that the computer-readable storage medium mentioned in the present disclosure can be a non-volatile storage medium, in other words, it can be a non-transitory storage medium.

[0180] Those of ordinary skill in the art can understand that all or part of the steps for implementing the above embodiments can be completed by hardware, or can be completed by a program instructing relevant hardware. The program can be stored in a computer-readable storage medium, and the storage medium mentioned above can be a read-only memory, a magnetic disk, or an optical disc, etc.

[0181] In the description of the embodiments of the present disclosure, the descriptions with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present disclosure. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0182] In the description of the embodiments of the present disclosure, unless otherwise specified, " / " means "or". For example, A / B may mean A or B. The "and / or" herein is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B may mean: A exists alone, A and B exist simultaneously, and B exists alone.

[0183] In the description of the embodiments of the present disclosure, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the embodiments of the present disclosure, unless otherwise specified, "a plurality of" means two or more.

[0184] The foregoing are only exemplary embodiments of the present disclosure and are not intended to limit the present disclosure. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A model rendering method for a custom background, comprising: Determine the background visual elements of the target scene according to the target picture; wherein the background visual elements include at least a partial image of the target picture; Binding the texture coordinate system of the target image to the screen space coordinate system so that the background visual elements maintain a fixed perspective relationship in the operation interface; Get the target model; In response to a transformation operation on a camera and / or the target model, determining camera parameters and spatial pose parameters of the target model; Obtaining a target scene according to the spatial pose parameters of the target model, the camera parameters and the background visual elements; A rendering of the target scene is generated.

2. The method according to claim 1, further comprising: When switching from the current operation interface to the target operation interface, the camera parameters and model parameters of the target operation interface are synchronized according to the camera parameters and model parameters of the current operation interface; wherein the current operation interface is one of the model editing interface and the rendering pop-up interface, and the target operation interface is the other of the two.

3. The method according to claim 1, further comprising: In the rendering pop-up window interface, updating the camera parameters and / or the spatial pose parameters of the target model according to the transformation operation on the camera and / or the target model; Render the target model according to the spatial pose parameters and camera parameters of the target model.

4. The method according to claim 1, wherein: The step of obtaining a target scene according to the spatial pose parameters of the target model, the camera parameters and the background visual elements comprises: In the case where the operation interface is a rendering pop-up window interface, the target model is rendered on the background visual element according to the spatial pose parameters, model map, lighting parameters, and camera parameters of the target model; In response to a transformation operation on a camera and / or the target model, updating the camera parameters and the spatial pose parameters of the target model; A target scene is obtained according to the camera parameters, the background visual elements and the spatial pose parameters of the target model.

5. The method according to claim 1, wherein: The step of obtaining a target scene according to the spatial pose parameters of the target model, the camera parameters and the background visual elements comprises: In the case where the operation interface is a model editing interface, displaying the target model in an editable state according to the spatial posture parameters and camera parameters of the target model; According to the editing operation on the target model, adjusting the vertex parameters of the target model; A target scene is obtained according to the camera parameters, the background visual elements and the adjusted target model.

6. The method according to claim 4, wherein: In the case where the current interface is a rendered pop-up window interface, rendering the target model on the background visual element according to the spatial pose parameters, model map, lighting parameters, and camera parameters of the target model includes: Determining a projection-view matrix of the target model based on perspective parameters and camera parameters of the screen space coordinate system; The background visual elements are preferentially rendered to the bottom layer of the target scene according to the mapping relationship between the bound texture coordinate system and the screen space coordinate system; The target model is rendered on the bottom layer of the target scene according to the model parameters of the target model, the model map and the projection-view matrix.

7. The method according to claim 6, wherein: The step of rendering the target model on the bottom layer of the target scene according to the model parameters of the target model, the model map and the projection-view matrix comprises: Transforming the vertex data of the target model to the screen space through the projection-view matrix to generate projection geometry information including screen space coordinates and fragment depth values; Based on the texture coordinates interpolated from the screen space coordinates, the model map is sampled, and the pixel color value is determined in combination with the lighting parameters; According to the comparison result between the fragment depth value and the background depth buffer, the pixel color value is overlaid onto the bottom layer of the target scene.

8. The method according to claim 1, wherein: Determining the background visual elements of the target scene according to the target image includes: According to a selected scene determined from a plurality of preset scenes, a corresponding scene picture is loaded; Determining background visual elements of a target scene according to the scene image; or Get the uploaded custom image from the server; Based on the custom image, background visual elements of the target scene are determined.

9. A model rendering device with a customized background, comprising: An image acquisition module, used to determine background visual elements of a target scene according to a target image; wherein the background visual elements include at least a partial image of the target image; A background binding module, used to bind the texture coordinate system of the target image to the screen space coordinate system, so that the background visual elements maintain a fixed perspective relationship in the operation interface; Model acquisition module, used to acquire the target model; An interactive adjustment module, configured to determine camera parameters and spatial pose parameters of the target model in response to a transformation operation on the camera and / or the target model; A scene determination module, used to obtain a target scene according to the spatial pose parameters of the target model, the camera parameters and the background visual elements; A generation module is used to generate a rendering of the target scene.

10. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

11. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.

12. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.

Images