Image rendering method and apparatus, and related device

Abstract

Embodiments of the present application relate to the technical field of computer vision, and disclose an image rendering method and apparatus, and a related device. In the embodiments of the present application, for any object to be rendered, a three-dimensional geometric model and three-dimensional material information of the object is extracted by means of an implicit object model of the object, an explicit object model of the object is displayed on a front-end interaction interface on the basis of the three-dimensional geometric model and the three-dimensional material information, the display attribute of the explicit object model is edited, and the implicit object model is rendered on the basis of the edited display attribute. In other words, only one implicit object model needs to be obtained, and an image rendering scheme in which an implicit object model and an explicit object model are combined can be realized by means of the method provided by the embodiments of the present application, the editability of an image rendering process can be improved, and the high realism of a rendering result can be ensured.

Description

Image rendering method, device and related equipment

[0001] This application claims priority to Chinese patent applications with application number 202311287935.8 filed on September 28, 2023, and invention name “A rendering method, device and system”, and application number 202311785372.5 filed on December 22, 2023, and invention name “Image rendering method, device and related equipment”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of computer vision technology, and in particular to an image rendering method, apparatus, and related equipment. Background Art

[0003] In computer vision, for any object to be rendered, a two-dimensional image of the object at a specific viewing angle and pose can be rendered based on its implicit object model. However, the process of rendering images using implicit object models is limited in editability, resulting in rendering results that often fail to meet user needs.

[0004] Summary of the Invention

[0005] The embodiments of the present application provide an image rendering method, apparatus, and related devices that can improve the editability of the image rendering process. The technical solution is as follows:

[0006] In a first aspect, an image rendering method is provided, in which a first implicit object model is obtained, the first implicit object model being used to predict an arbitrary perspective image of a first object; a first three-dimensional geometric model and first three-dimensional material information of the first three-dimensional geometric model are determined based on the first implicit object model, the first three-dimensional geometric model being the three-dimensional geometric model of the first object; a first explicit object model of the first object is displayed on a front-end interactive interface based on the first three-dimensional geometric model and the first three-dimensional material information; display properties of the first explicit object model in the front-end interactive interface are determined in response to an editing operation of the user on the front-end interactive interface; and the first implicit object model is rendered based on the display properties of the first explicit object model in the front-end interactive interface to obtain at least one first object image.

[0007] In the image rendering method provided in the embodiment of the present application, for any object to be rendered, the three-dimensional geometric model and three-dimensional material information of the object are extracted through the implicit object model of the object, and the explicit object model of the object is displayed on the front-end interactive interface based on the three-dimensional geometric model and three-dimensional material information. The display properties of the explicit object model are edited, and the implicit object model is rendered based on the edited display properties. In other words, only an implicit object model needs to be obtained, and the method provided in the embodiment of the present application can realize an image rendering solution that combines the implicit object model with the explicit object model, which can not only improve the editability of the image rendering process, but also ensure the high realism of the rendering results.

[0008] Based on the method provided in the first aspect, in one possible implementation, a first coordinate transformation relationship may also be determined, where the first coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the first implicit object model and the object coordinate system of the first three-dimensional geometric model. In this scenario, the implementation process of rendering the first implicit object model based on the display properties of the first explicit object model in the front-end interactive interface may include rendering the first implicit object model based on the first coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface.

[0009] When determining the first 3D geometric model and the first 3D material information of the first 3D geometric model based on the first implicit object model, a first coordinate transformation relationship can also be determined to facilitate subsequent accurate rendering of the first implicit object model based on the first coordinate transformation relationship.

[0010] Based on the method provided in the first aspect, in one possible implementation, a second coordinate transformation relationship may also be determined, where the second coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the first explicit object model and the interface coordinate system of the front-end interactive interface. In this scenario, the implementation process of rendering the first implicit object model based on the display properties of the first explicit object model in the front-end interactive interface may be: rendering the first implicit object model based on the second coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface.

[0011] Since the display properties of the determined first explicit object model in the front-end interactive interface refer to the display properties in the interface coordinate system of the front-end interactive interface, and the interface coordinate system and the object coordinate system of the first explicit object model may not be the same coordinate system, the second coordinate transformation relationship can also be determined to facilitate the subsequent accurate rendering of the first implicit object model based on the second coordinate transformation relationship.

[0012] Based on the method provided in the first aspect, in one possible implementation, the implementation process of determining the first three-dimensional material information of the first three-dimensional geometric model can be: mapping the first three-dimensional geometric model through a texture mapping algorithm based on an arbitrary perspective image of the first object output by the first implicit object model to obtain the first three-dimensional material information.

[0013] The first 3D material information obtained in this manner includes the texture values ​​at each point in the first 3D geometric model. The texture of the object can subsequently be displayed directly on the front-end interactive interface based on the texture values. Therefore, the first 3D material information obtained in this manner can also be referred to as explicit material information. The first 3D material information obtained in this manner can increase the speed of subsequent display of the first explicit object model, thereby improving the efficiency of rendering the first explicit object model on the front-end interactive interface.

[0014] Based on the method provided in the first aspect, in one possible implementation, the implementation process of determining the first three-dimensional material information of the first three-dimensional geometric model can be: determining the first material model based on the first implicit object model, and the first material model is used to predict the arbitrary perspective texture of the first object.

[0015] This approach yields a first material model capable of predicting the texture of the first object at any viewing angle. The first 3D material information obtained in this scenario can also be referred to as implicit material information or neural material. This first material model can improve the realism of the first explicit object model when subsequently displaying it.

[0016] Based on the method provided in the first aspect, in a possible implementation manner, the implementation process of obtaining the first implicit object model may be: training the first implicit object model based on multiple first sample data, where the multiple first sample data are used to indicate multiple perspective imaging information of the first object.

[0017] In an embodiment of the present application, a first implicit object model can be obtained through sample data training to implement an image rendering solution that combines the implicit object model with the explicit object model based on the first implicit object model.

[0018] Based on the method provided in the first aspect, in one possible implementation, a second implicit object model can also be obtained, and the second implicit object model is used to predict an arbitrary perspective image of the second object; a second explicit object model of the second object input by the user based on the front-end interactive interface is obtained; the second explicit object model is displayed on the front-end interactive interface; in response to the user's editing operation on the front-end interactive interface, the display properties of the second explicit object model in the front-end interactive interface are determined; based on the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered to obtain at least one second object image; and at least one first object image and at least one second object image are fused to obtain panoramic simulation data.

[0019] In the image rendering method provided in the embodiment of the present application, in addition to obtaining the explicit object model of the object through the implicit object model, the user can also directly input an explicit object model to improve the flexibility of the embodiment of the present application.

[0020] Based on the method provided in the first aspect, in one possible implementation, a third coordinate transformation relationship may also be determined, where the third coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the second implicit object model and the object coordinate system of the second explicit object model. In this scenario, the second implicit object model may be rendered based on the display properties of the second explicit object model in the front-end interactive interface to obtain at least one second object image. This may be achieved by rendering the second implicit object model based on the third coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface.

[0021] Since the object coordinate system of the second explicit object model input by the user may not be the same as the object coordinate system of the second implicit object model obtained above, in this embodiment of the application, a third coordinate transformation relationship may also be determined to facilitate subsequent accurate rendering of the second implicit object model based on the third coordinate transformation relationship.

[0022] Based on the method provided in the first aspect, in one possible implementation, the implementation process of determining the third coordinate transformation relationship can be: outputting at least one image and the pose of at least one image in the object coordinate system of the second implicit object model based on the second implicit object model; locating the pose of at least one image in the object coordinate system of the second explicit object model; and determining the third coordinate transformation relationship based on the pose of at least one image in the object coordinate system of the second explicit object model and the pose of at least one image in the object coordinate system of the second implicit object model.

[0023] The third coordinate transformation relationship can be determined by the position and posture of the same image in the second explicit object model and the second implicit object model, thereby improving the flexibility of the embodiment of the present application.

[0024] Based on the method provided in the first aspect, in one possible implementation, a fourth coordinate transformation relationship may also be determined, where the fourth coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the second explicit object model and the interface coordinate system of the front-end interactive interface. In this scenario, the implementation process of rendering the second implicit object model based on the display properties of the second explicit object model in the front-end interactive interface may be: rendering the second implicit object model based on the fourth coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface.

[0025] Since the display properties of the determined second explicit object model in the front-end interactive interface refer to the display properties in the interface coordinate system of the front-end interactive interface, and the interface coordinate system and the object coordinate system of the second explicit object model may not be the same coordinate system, the fourth coordinate transformation relationship can also be determined to facilitate the subsequent accurate rendering of the second implicit object model based on the fourth coordinate transformation relationship.

[0026] In a second aspect, an image rendering device is provided, wherein the image rendering device has the function of implementing the image rendering method of the first aspect. The image rendering device includes at least one module, wherein the at least one module is used to implement the image rendering method of the first aspect.

[0027] In a third aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the image rendering method provided in the first aspect above.

[0028] In a fourth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device cluster, the computing device cluster executes the image rendering method provided in the first aspect.

[0029] In a fifth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the image rendering method provided in the first aspect.

[0030] The technical effects obtained in the above-mentioned second, third, fourth and fifth aspects are similar to those obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] FIG1 is a schematic diagram of the architecture of an image rendering system provided by an embodiment of the present application;

[0032] FIG2 is a schematic diagram of another architecture of the image rendering system shown in FIG1 provided in an embodiment of the present application;

[0033] FIG3 is a flow chart of an image rendering method provided in an embodiment of the present application;

[0034] FIG4 is a flow chart of another image rendering method provided by an embodiment of the present application;

[0035] FIG5 is a schematic structural diagram of an image rendering device provided in an embodiment of the present application;

[0036] FIG6 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0037] FIG7 is a schematic diagram of the architecture of a computing device cluster provided in an embodiment of the present application;

[0038] FIG8 is a schematic diagram of the architecture of another computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0040] Before explaining the embodiments of the present application, the application scenarios of the embodiments of the present application are first introduced.

[0041] In scenarios such as autonomous driving simulation, game simulation, augmented reality (AR) and virtual reality (VR), it is necessary to produce a large number of three-dimensional (3D) scenes with rich content and high realism. The representation methods of objects in 3D scenes can be divided into explicit and implicit. Explicit representation refers to directly giving the geometry, material, lighting and other information of the object in the scene through physically meaningful parameters, that is, creating an explicit object model of the object, where the explicit object model can also be called explicit material, and examples of explicit object models include mesh and other models. In contrast, implicit representation refers to giving the color and density and other information of the object at the corresponding point in the scene through a mapping function, that is, creating an implicit object model of the object, where the implicit object model can also be called implicit material, and examples of implicit object models include neural radiance field (NeRF) and other models.

[0042] Current scene editing systems for creating 3D scenes include modeling software like Blender (an open-source 3D software) and game engines like Unity (a real-time content development platform). These scene editing systems support flexible translation, rotation, and scaling of different objects in a 3D scene, but only support editing explicit object models. However, due to limitations in modeling accuracy and rendering algorithms, the rendered results of explicit object models still differ from what the human eye sees in the real world, resulting in low rendering fidelity.

[0043] Implicit object models, such as the NeRF model, can provide very realistic rendering effects. However, scene editing systems such as Blender and Unity cannot directly edit implicit object models, which means that the editability of the rendering process is very low.

[0044] Based on this, an embodiment of the present application provides an image rendering method. In the image rendering method provided in the embodiment of the present application, for any object to be rendered, the explicit object model of the object is displayed on the front-end interactive interface through the implicit object model of the object, the display properties of the explicit object model are edited, and the implicit object model is rendered based on the edited display properties. In other words, it is only necessary to obtain an implicit object model, and through the method provided in the embodiment of the present application, an image rendering solution that combines the implicit object model with the explicit object model can be implemented, which can not only improve the editability of the image rendering process, but also ensure the high realism of the rendering results.

[0045] The image rendering method provided in the embodiments of the present application can be applied to cloud scenarios, such as autonomous driving scenarios. In this scenario, the image rendering method is executed by a cloud platform, that is, the cloud platform provides cloud services to users, and the cloud services are used to implement the image rendering method provided in the embodiments of the present application. Optionally, the image rendering method provided in the embodiments of the present application can also be executed by a terminal device. The embodiments of the present application do not limit the application scenarios of the image rendering method.

[0046] The image rendering system provided in the embodiment of the present application is explained below.

[0047] Figure 1 is a schematic diagram of the architecture of an image rendering system provided by an embodiment of the present application. As shown in Figure 1, the image rendering system 00 includes an implicit modeling module 01, an explicit and implicit alignment module 02, an explicit and implicit combined proxy scene editing module 03, and an explicit and implicit combined scene rendering module 04.

[0048] The implicit modeling module 01 is used to create an implicit object model for each of one or more objects, and output the implicit object model to the explicit and implicit alignment module 02 .

[0049] For example, for any object to be modeled, the user can provide multiple images of the object to the implicit modeling module 01, and the implicit modeling module 01 creates an implicit object model of the object based on these images. Optionally, in addition to providing multiple images of the object to the implicit modeling module 01, the user can also provide point cloud information and / or depth information of the object, and the implicit modeling module 01 creates an implicit object model of the object based on these images, the point cloud information, and / or the depth information.

[0050] The implicit modeling module 01 may use NeRF, instant neural graphics primitives (InstantNGP), or reconstruction based on differentiable rendering to create an implicit object model, which will not be described one by one here.

[0051] In the embodiment of the present application, the objects to be modeled can be objects (such as vehicles, road obstacles), people, spaces (such as city streets, highway sections), etc., and they will not be illustrated one by one here.

[0052] The explicit and implicit alignment module 02 is used to receive the implicit object model of each object in one or more objects output by the explicit and implicit alignment module 02, and output the explicit and implicit combination materials of each object in one or more objects to the explicit and implicit combination agent scene editing module 03 and the explicit and implicit combination scene rendering module 04 respectively.

[0053] For example, for any object, the explicit and implicit alignment module 02 can extract the object's 3D geometric model and 3D material information from the implicit object model to facilitate the subsequent display of the object's explicit object model on the front-end interactive interface. The module then aligns the object coordinate system of the 3D geometric model with the object coordinate system of the implicit object model to form a combined explicit and implicit material. In this scenario, the combined explicit and implicit material includes the implicit object model, the object's 3D geometric model and 3D material information, and the conversion relationship between the object coordinate systems of the implicit object model and the 3D geometric model.

[0054] Among them, aligning the object coordinate system of the three-dimensional geometric model and the object coordinate system of the implicit object model can be understood as: determining the transformation relationship between the two object coordinate systems, which may include translation parameters and / or rotation parameters and / or scaling parameters, etc.

[0055] Optionally, if the user already has an explicit object model of the object that has been pre-modeled, the user can input the explicit object model of the object into the explicit-implicit alignment module 02. In this scenario, the explicit-implicit alignment module 02 can align the object coordinate system of the explicit object model input by the user with the object coordinate system of the implicit object model output by the implicit modeling module 01 to jointly form the explicit-implicit combined material. In this scenario, the explicit-implicit combined material includes the implicit object model, the explicit object model input by the user, and the conversion relationship between the object coordinate systems of these two object models.

[0056] The scene editing module 03 of the visible and invisible combination agent is used to receive the visible and invisible combination material of each object in the one or more objects output by the visible and invisible alignment module 02, and output the visible and invisible combination scene to the visible and invisible combination scene rendering module 04.

[0057] For example, the scene editing module 03 of the combined explicit and implicit proxy can display the explicit object model of each object on a front-end interactive interface. The front-end interactive interface can be an editing interface based on web technologies such as Three.js (a 3D engine encapsulated and running based on the native web graphics library (WebGL)) or Babylon.js (an open source web 3D rendering engine). The front-end interactive interface can also be an editing interface based on client technologies such as Blender (an open source 3D production software).

[0058] The user combines and edits the explicit object models in the front-end interactive interface. After the user combines and edits the explicit object models in the front-end interactive interface, the scene editing module 03 of the explicit and hidden combined agent determines the display attributes of each explicit object model in the front-end interactive interface, which display attributes include the position, size, orientation, and other attributes of the explicit object model in the front-end interactive interface. The scene editing module 03 of the explicit and hidden combined agent combines the display attributes of each explicit object model and the conversion relationship between the object coordinate system of each explicit object model and the interface coordinate system of the front-end interactive interface into a explicit and hidden combined scene.

[0059] The visible and invisible combined scene rendering module 04 is used to receive the visible and invisible combined scene output by the scene editing module 03 of the visible and invisible combined agent, and the visible and invisible combined material of each object in one or more objects output by the visible and invisible alignment module 02, and obtain camera parameter information. Based on the visible and invisible combined scene, the visible and invisible combined material of each object in one or more objects and the camera parameter information, the implicit object model of each object is rendered to obtain a highly realistic image.

[0060] The camera parameter information exemplarily includes a rendering perspective pose, which may include one perspective pose or multiple perspective poses.

[0061] FIG2 is a schematic diagram of another architecture of the image rendering system shown in FIG1 provided in an embodiment of the present application.

[0062] As shown in FIG. 2 , the explicit and implicit alignment module 02 includes an explicit material extraction module 021 , an explicit material input module 022 and an explicit and implicit alignment representation module 023 .

[0063] For any object, the explicit material extraction module 021 is used to extract a 3D geometric model and 3D material information from the implicit object model of the object. In the embodiment of the present application, the 3D material information extracted by the explicit material extraction module 021 can be explicit material information such as a texture. In this scenario, the 3D geometric model and 3D material information extracted from the implicit object model of the object can be referred to as an explicit object model. In other words, in the embodiment of the present application, an explicit object model can be directly extracted from the implicit object model.

[0064] Optionally, the extracted three-dimensional material information can be implicit material information that is seen but not obtained by a mapping function, etc. In this scenario, the three-dimensional geometric model and three-dimensional material information extracted from the implicit object model of the object can be called an explicit and implicit combined object model. In other words, in an embodiment of the present application, an explicit and implicit combined object model can be extracted from the implicit object model.

[0065] The explicit material input module 022 is used to obtain an explicit object model of an object input by the user in response to the user's operation on the front-end interactive interface.

[0066] The explicit and implicit alignment representation module 023 is used to align the object coordinate system of the three-dimensional geometric model determined by the explicit material extraction module 021 with the object coordinate system of the implicit object model, or to align the object coordinate system of the explicit object model determined by the explicit material input module 022 with the object coordinate system of the implicit object model.

[0067] As shown in FIG2 , the scene editing module 03 of the explicit and implicit combination proxy includes a explicit and implicit combination material combination module 031 , an explicit and implicit combination proxy editing module 032 and a scene alignment module 033 .

[0068] Among them, the explicit and implicit combined material combination module 031 is used to respond to the user adding one or more explicit object models of objects in the scene of the front-end interactive interface, determine the position of the explicit object model of each object on the front-end interactive interface, and locate the conversion relationship between the object coordinate system and the interface coordinate system of each explicit object model.

[0069] The explicit and implicit combined proxy editing module 032 is used to respond to the user's editing operation on a certain explicit object model, determine the display properties of the explicit object model, such as position, size, and orientation, and locate the conversion relationship between the object coordinate system of the explicit object model and the interface coordinate system.

[0070] The scene alignment module 033 is used to integrate the positions and transformation relationships of each explicit object model output by the explicit and implicit combined material combination module 031, and the display properties and transformation relationships of each explicit object model output by the explicit and implicit combined proxy editing module 032, to obtain a scene combination material, which includes the display properties of the explicit object model of each object in the scene, and the transformation relationship between the object coordinate system of the explicit object model and the interface coordinate system.

[0071] It should be noted that, for the same object, when the explicit and implicit combined material combination module 031 outputs the position and conversion relationship of the explicit object model of the object, and the explicit and implicit combined proxy editing module 032 also outputs the display attributes and conversion relationship of the explicit object model of the object, the position and conversion relationship of the explicit object model of the object output by the explicit and implicit combined material combination module 031 corresponds to the user operation time, and the display attributes and conversion relationship of the explicit object model of the object output by the explicit and implicit combined proxy editing module 032 also corresponds to the user operation time. Therefore, the scene alignment module 033 can obtain the position and conversion relationship of the explicit object model of the object with a later user operation time as the position and conversion relationship of the explicit object model of the object in the final scene combined material.

[0072] The above transformation relationships all refer to the transformation relationships between the object coordinate system of the explicit object model and the interface coordinate system.

[0073] Among them, when the user operates on the front-end interactive interface, he usually first adds an explicit object model of an object to the scene of the front-end interactive interface through the explicit and implicit combination material combination module 031, and then edits the display properties of the explicit object model through the explicit and implicit combination proxy editing module 032. In this scenario, the position and conversion relationship of the explicit object model of the object in the scene combination material are the position and conversion relationship in the display properties output by the explicit and implicit combination proxy editing module 032. Optionally, after adding an explicit object model of an object to the scene of the front-end interactive interface through the explicit and implicit combination material combination module 031, the user can also not edit the display properties of the explicit object model through the explicit and implicit combination proxy editing module 032. In this scenario, the position and conversion relationship of the explicit object model of the object in the scene combination material are the position and conversion relationship output by the explicit and implicit combination material combination module 031.

[0074] In addition, the scene combination material can be represented by, for example, a graphic library transmission format (glTF), a universal scene description (USD), or other formats, which is not limited in the embodiments of the present application.

[0075] It should be noted that the image rendering system shown in FIG. 2 is used to illustrate the image rendering system shown in FIG. 1 , and does not constitute a limitation on the image rendering system shown in FIG. 1 .

[0076] In addition, the image rendering system 00 can be exemplified as a cloud platform. In this scenario, the image rendering system 00 can be implemented by one or more computing devices. Among them, the implicit modeling module 01, the explicit and implicit alignment module 02, the explicit and implicit combined agent scene editing module 03, the explicit and implicit combined scene rendering module 04, and other modules can be integrated on the same computing device or deployed on different computing devices, which is not limited in this embodiment of the present application.

[0077] The image rendering method provided in the embodiment of the present application is explained below.

[0078] Figure 3 is a flow chart of an image rendering method provided by an embodiment of the present application. As shown in Figure 3, the method includes the following steps.

[0079] Step 301: Acquire a first implicit object model, where the first implicit object model is used to predict an arbitrary-viewing angle image of a first object.

[0080] In embodiments of the present application, a first implicit object model can be obtained through training with sample data, thereby implementing an image rendering solution that combines the implicit object model with the explicit object model based on the first implicit object model. Based on this, in some embodiments, obtaining the first implicit object model can be achieved by training the first implicit object model based on multiple first sample data, where the multiple first sample data are used to indicate imaging information of the first object from multiple perspectives.

[0081] For example, the plurality of first sample data may include a plurality of pictures collected for the first object. Optionally, the plurality of first sample data may also include a plurality of pictures collected for the first object, and point cloud information and / or depth information collected for the first object.

[0082] The first implicit object model can be a NeRF model. The basic principle of training a NeRF model is as follows: for any object, sample images of the object are first collected from different perspectives, and the camera pose of each sample image is determined. The sample images and corresponding camera poses are then fed into the NeRF model as sample data for training. The trained NeRF model can then be fed with two-dimensional images of the object from any perspective.

[0083] Optionally, the first implicit object model may also be another type of implicit model. The embodiment of the present application does not limit the type of the first implicit object model.

[0084] Alternatively, in other embodiments, obtaining the first implicit object model may be achieved by: in response to a user input operation on a front-end interactive interface, obtaining the first implicit object model input by the user. In other words, the first implicit object model may be a pre-modeled implicit object model.

[0085] Step 302: Determine a first three-dimensional geometric model and first three-dimensional material information of the first three-dimensional geometric model based on the first implicit object model, where the first three-dimensional geometric model is a three-dimensional geometric model of a first object.

[0086] Determining the first three-dimensional geometric model based on the first implicit object model can be achieved by a geometry extraction algorithm. The geometry extraction algorithm can be, for example, a marching cube (MC) algorithm, a baked signed distance field (BakedSDF) algorithm for gridded neural networks for real-time view synthesis, and the like, which are not described one by one here.

[0087] In some embodiments, the first three-dimensional material information of the first three-dimensional geometric model can be determined by mapping the first three-dimensional geometric model through a texture mapping algorithm based on an arbitrary perspective image of the first object output by the first implicit corresponding model to obtain the first three-dimensional material information.

[0088] The first 3D material information obtained in this manner includes the texture values ​​at each point in the first 3D geometric model. The texture of the object can subsequently be displayed directly on the front-end interactive interface based on the texture values. Therefore, the first 3D material information obtained in this manner can also be referred to as explicit material information. The first 3D material information obtained in this manner can increase the speed of subsequent display of the first explicit object model, thereby improving the efficiency of rendering the first explicit object model on the front-end interactive interface.

[0089] Among them, the arbitrary perspective image of the first object output based on the first implicit corresponding model is mapped to the first three-dimensional geometric model through a texture mapping algorithm. For example, it can be achieved through steps such as projection mapping, transformation function, texture sampling and texture conversion. The embodiments of this application do not provide detailed descriptions of this.

[0090] In other embodiments, determining the first three-dimensional material information of the first three-dimensional geometric model may be implemented by: determining a first material model based on a first implicit object model, the first material model being used to predict the arbitrary perspective texture of the first object; the first three-dimensional material information includes the first material model.

[0091] This approach yields a first material model capable of predicting the texture of the first object at any viewing angle. The first three-dimensional material information obtained in this scenario can also be referred to as a neural material. The first material model can be used to improve the realism of the first explicit object model when subsequently displaying it.

[0092] The method for determining the first material model based on the first implicit object model may be, for example, baking the first implicit object model into the first material model using BakedsSDF, NeRF2Mesh (a technology for recovering fine texture meshes from NeRF through adaptive surface refinement), etc. This embodiment of the present application will not be described in detail.

[0093] The above is used to illustrate how to obtain the first three-dimensional material information. Optionally, the first three-dimensional material information can also be obtained from the first implicit object model through other methods, which will not be explained here one by one.

[0094] In addition, in some embodiments, when determining the first three-dimensional geometric model and the first three-dimensional material information of the first three-dimensional geometric model based on the first implicit object model, a first coordinate transformation relationship can also be determined. The first coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the first implicit object model and the object coordinate system of the first three-dimensional geometric model, so as to facilitate the subsequent accurate rendering of the first implicit object model based on the transformation relationship between the two object coordinate systems.

[0095] For example, since the first 3D geometric model is determined based on the first implicit object model, the object coordinate system of the first 3D geometric model and the object coordinate system of the first implicit object model can be the same coordinate system. In this case, the values ​​of parameters such as translation parameters and rotation parameters in the first coordinate transformation relationship indicating the transformation relationship between the object coordinate system of the first implicit object model and the object coordinate system of the first 3D geometric model can all be 0, indicating that the object coordinate system of the first 3D geometric model and the object coordinate system of the first implicit object model are the same coordinate system.

[0096] Optionally, in some scenarios, when constructing the first 3D geometric model based on the first implicit object model, the object coordinate system of the first 3D geometric model and the object coordinate system of the first implicit object model may not be the same coordinate system. In this case, the object coordinate system of the first 3D geometric model and the object coordinate system of the first implicit object model can be converted to the same coordinate system through the first coordinate transformation relationship to facilitate subsequent rendering of the first implicit object model.

[0097] Optionally, if the object coordinate system of the first three-dimensional geometric model and the object coordinate system of the first implicit object model are the same coordinate system, the first coordinate transformation relationship can also be recorded. If the first coordinate transformation relationship is not found in the rendered image later, the object coordinate system of the first three-dimensional geometric model and the object coordinate system of the first implicit object model are assumed to be the same coordinate system.

[0098] Step 303: Displaying a first explicit object model of the first object on a front-end interactive interface based on the first three-dimensional geometric model and the first three-dimensional material information.

[0099] In an embodiment of the present application, in order to improve the high editability of images rendered through implicit object models, after obtaining the first three-dimensional geometric model and the first three-dimensional material information through the first implicit object model, the first explicit object model of the first object is displayed on the front-end interactive interface based on the first three-dimensional geometric model and the first three-dimensional material information, so that the user can flexibly edit the display properties based on the first explicit object model displayed on the front-end interactive interface.

[0100] In some embodiments, when the first 3D material information includes texture values ​​at each point in the first 3D geometric model, the first 3D geometric model in a default viewing angle and the texture values ​​at each point in the first 3D geometric model in a default angular position can be directly displayed on the front-end interactive interface to obtain the first display object model. Subsequently, when rotation of the first display object model is detected, the first 3D geometric model in other viewing angles and the texture values ​​at each point in the first 3D geometric model in other viewing angles can be displayed.

[0101] In other embodiments, when the first three-dimensional material information includes a first material model, in response to the need to display the first object on the front-end interactive interface, the texture information of the first object under the default viewing angle posture is predicted through the first material model, and the texture information includes the texture values ​​at each position point in the first object under the default viewing angle posture, and the first three-dimensional geometric model is displayed according to the default viewing angle posture, and the texture information of the first object is mapped in the first three-dimensional geometric model to obtain the first explicit object model. Subsequently, when a change in the viewing angle posture is detected, the texture information of the first object under the changed viewing angle posture is output through the first material model, and the first three-dimensional geometric model is displayed according to the changed viewing angle posture, and the texture information of the first object is mapped in the first three-dimensional geometric model to obtain the first explicit object model under the changed viewing angle posture. This process can be called obtaining the first explicit object model through real-time rendering of the first material model.

[0102] Step 304: In response to the user's editing operation on the front-end interactive interface, determine the display properties of the first explicit object model in the front-end interactive interface.

[0103] The editing operation performed by the user on the front-end interactive interface may include, for example, one or more of movement, rotation, and scaling. The display attributes of the determined first explicit object model on the front-end interactive interface may include, for example, display size and display posture, wherein the display posture includes position and orientation.

[0104] In addition, the display properties of the first explicit object model determined in step 304 in the front-end interactive interface refer to the display properties in the interface coordinate system of the front-end interactive interface, and the interface coordinate system and the object coordinate system of the first explicit object model may not be the same coordinate system. Based on this, in some embodiments, a second coordinate transformation relationship can also be determined in step 304. The second coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the first explicit object model and the interface coordinate system of the front-end interactive interface, so as to facilitate the subsequent accurate rendering of the first implicit object model based on the second coordinate transformation relationship.

[0105] Optionally, if the object coordinate system of the first explicit object model and the interface coordinate system of the front-end interactive interface are the same coordinate system, a second coordinate transformation relationship can also be determined. In this case, the values ​​of parameters such as the translation parameters and rotation parameters in the second coordinate transformation relationship indicating the transformation relationship between the object coordinate system of the first explicit object model and the interface coordinate system can all be 0, indicating that the object coordinate system of the first explicit object model and the interface coordinate system are the same coordinate system.

[0106] Optionally, if the object coordinate system of the first explicit object model and the interface coordinate system of the front-end interactive interface are the same coordinate system, the second coordinate transformation relationship may not be recorded. If the second coordinate transformation relationship is not found in the rendered image later, it is assumed that the object coordinate system of the first explicit object model and the interface coordinate system of the front-end interactive interface are the same coordinate system.

[0107] Step 305: Based on the display properties of the first explicit object model in the front-end interactive interface, the first implicit object model is rendered to obtain at least one first object image.

[0108] In some embodiments, rendering the first implicit object model based on the display properties of the first explicit object model in the front-end interactive interface may be implemented by rendering the first implicit object model based on a first coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface. The first coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the first implicit object model and an object coordinate system of the first three-dimensional geometric model.

[0109] Among them, the implementation method of rendering the first implicit object model based on the first coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface can be: based on the display properties of the first explicit object model in the front-end interactive interface, determine the display properties of the first explicit object model in the world coordinate system; based on the first coordinate transformation relationship, convert the display properties of the first display object model in the world coordinate system into the display properties of the first implicit object model in the world coordinate system; and render the first implicit object model based on the display properties of the first implicit object model in the world coordinate system.

[0110] For example, when the interface coordinate system of the front-end interactive interface is the world coordinate system, the display properties of the first explicit object model in the front-end interactive interface can be directly used as the display properties of the first explicit object model in the world coordinate system.

[0111] Optionally, when the interface coordinate system of the front-end interactive interface is not the world coordinate system, it is necessary to use coordinate conversion to convert the display properties of the first explicit object model in the front-end interactive interface into the display properties of the first explicit object model in the world coordinate system. The specific implementation method will be described in detail later and will not be expanded here.

[0112] In other embodiments, rendering the first implicit object model based on the display properties of the first explicit object model in the front-end interactive interface may be implemented by rendering the first implicit object model based on a second coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface. The second coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the first explicit object model and an interface coordinate system of the front-end interactive interface.

[0113] Among them, rendering the first implicit object model based on the second coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface can be understood as: based on the second coordinate transformation relationship, the display properties of the first explicit object model in the front-end interactive interface are converted into the display properties of the first explicit object model in the world coordinate system, and then the first implicit object model is rendered based on the display properties of the first explicit object model in the world coordinate system.

[0114] For example, if the object coordinate systems of the first implicit object model and the first three-dimensional geometric model used to generate the first explicit object model are the same coordinate system, then after determining the display properties of the first explicit object model in the world coordinate system, the display properties of the first explicit object model in the world coordinate system can be used as the display properties of the first implicit object model in the world coordinate system, and then the first implicit object model can be rendered based on the display properties of the first implicit object model in the world coordinate system.

[0115] As another example, if the object coordinate systems of the first implicit object model and the first three-dimensional geometric model used to generate the first explicit object model are not the same coordinate system, after determining the display properties of the first explicit object model in the world coordinate system, the display properties of the first explicit object model in the world coordinate system can be converted into the display properties of the first implicit object model in the world coordinate system through coordinate system conversion, and then the first implicit object model can be rendered based on the display properties of the first implicit object model in the world coordinate system.

[0116] The display attributes of the first implicit object model in the world object coordinate system include information such as display size and display pose. By inputting the display attributes and the view pose to be rendered into the first implicit object model, the first implicit object model can output a first object image corresponding to the view pose.

[0117] The viewing angle pose indicates the pose of the rendering camera in the world coordinate system. The first implicit object model can use the viewing angle pose to convert the display properties of the first implicit object model in the world object coordinate system into the display properties of the first implicit object model in the camera coordinate system of the rendering camera. Based on the display properties of the first implicit object model in the camera coordinate system, a first object image corresponding to the viewing angle pose can be obtained. The specific implementation method is not described in detail here.

[0118] It should be noted that the above-mentioned conversion process of display attributes in different coordinate systems is for illustration only. In the embodiment of the present application, display attributes can also be converted in different coordinate systems through other methods, which will not be illustrated one by one here.

[0119] In summary, in the image rendering method provided in the embodiment of the present application, for any object to be rendered, the three-dimensional geometric model and three-dimensional material information of the object are extracted through the implicit object model of the object, and the explicit object model of the object is displayed on the front-end interactive interface based on the three-dimensional geometric model and three-dimensional material information. The display properties of the explicit object model are edited, and the implicit object model is rendered based on the edited display properties. In other words, it is only necessary to obtain an implicit object model, and the method provided in the embodiment of the present application can realize an image rendering solution that combines the implicit object model with the explicit object model, which can not only improve the editability of the image rendering process, but also ensure the high realism of the rendering results.

[0120] Based on the image rendering system shown in Figures 1 and 2, it can be seen that in addition to obtaining the explicit object model of the object through the implicit object model, the user can also directly input an explicit object model to improve the flexibility of the embodiments of the present application. Therefore, when the system shown in Figure 1 or Figure 2 renders the objects in the scene through the embodiment shown in Figure 3, it can further render other objects in the scene through the embodiment shown in Figure 4 below. It should be noted that the embodiments shown in Figures 3 and 4 can be implemented separately or in combination.

[0121] Step 401: Acquire a second implicit object model, where the second implicit object model is used to predict an arbitrary-viewing angle image of a second object.

[0122] The implementation of step 401 may refer to step 301 in the embodiment shown in FIG3 , and will not be described in detail here.

[0123] Step 402: Obtain a second explicit object model of a second object input by the user based on the front-end interactive interface.

[0124] The second explicit object model may be a mesh model or the like.

[0125] Because the object coordinate system of the second explicit object model input by the user and the object coordinate system of the second implicit object model obtained in step 401 may not be the same coordinate system, in some embodiments, a third coordinate transformation relationship may be determined. The third coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the second implicit object model and the object coordinate system of the second explicit object model. This facilitates subsequent accurate rendering of the second implicit object model based on the third coordinate transformation relationship.

[0126] For example, determining the third coordinate transformation relationship may be implemented by: outputting at least one image and the pose of at least one image in the object coordinate system of the second implicit object model based on the second implicit object model; locating the pose of at least one image in the object coordinate system of the second explicit object model; and determining the third coordinate transformation relationship based on the pose of at least one image in the object coordinate system of the second explicit object model and the pose of at least one image in the object coordinate system of the second implicit object model.

[0127] The third coordinate transformation relationship can be determined by the position and posture of the same image in the second explicit object model and the second implicit object model, thereby improving the flexibility of the embodiment of the present application.

[0128] The positioning of the at least one image in the object coordinate system of the second explicit object model can be achieved, for example, by a 6-degree-of-freedom (dof) positioning algorithm, which will not be described in detail here.

[0129] Optionally, in embodiments of the present application, the third coordinate transformation relationship may be determined by other means. For example, when a user inputs the second explicit object model, the user may manually align the object coordinate system of the second explicit object model with the object coordinate system of the second implicit object model in the front-end interactive interface. The system determines the third coordinate transformation relationship based on operations such as translation, rotation, and scaling during the alignment process, which will not be described in detail here.

[0130] Step 403: Display the second explicit object model on the front-end interactive interface.

[0131] Since the explicit object model of the second object, ie, the second explicit object model, has been acquired through step 402 , the second explicit object model can be directly displayed on the front-end interactive interface.

[0132] Step 404: In response to the user's editing operation on the front-end interactive interface, determine the display properties of the second explicit object model in the front-end interactive interface.

[0133] The implementation of step 404 may refer to step 304 in the embodiment of FIG3 , and will not be described in detail here.

[0134] Similar to step 304, since the display properties of the second explicit object model determined in the front-end interactive interface refer to the display properties in the interface coordinate system of the front-end interactive interface, and the interface coordinate system and the object coordinate system of the second explicit object model may not be a coordinate system, a fourth coordinate transformation relationship can also be determined to facilitate the subsequent accurate rendering of the second implicit object model based on the fourth coordinate transformation relationship. The fourth coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the second explicit object model and the interface coordinate system of the front-end interactive interface. The relevant implementation method can refer to step 304 in the embodiment of Figure 3, and will not be repeated here.

[0135] Step 405: Based on the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered to obtain at least one second object image.

[0136] Among them, in some embodiments, the second implicit object model is rendered based on the display properties of the second explicit object model in the front-end interactive interface, and the implementation method for obtaining at least one second object image can be: based on the third coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered.

[0137] In other embodiments, the implementation method of rendering the second implicit object model based on the display properties of the second explicit object model in the front-end interactive interface can be: rendering the second implicit object model based on the fourth coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface.

[0138] The above implementation method can refer to step 305 in the embodiment of FIG3 , which will not be described in detail here.

[0139] Step 406: Fusing at least one first object image and at least one second object image to obtain panoramic simulation data.

[0140] At least one first object image corresponds to at least one viewing angle and pose, and at least one second object image also corresponds to at least one viewing angle and pose. Based on this, in step 406, the first object image and the second object image corresponding to the same viewing angle and pose can be fused to obtain at least one panoramic image frame, which constitutes the panoramic simulation data.

[0141] The embodiment of the present application does not limit the implementation method of fusing the first object image and the second object image corresponding to the same viewing angle and posture.

[0142] In summary, in the image rendering method provided in the embodiment of the present application, in addition to being able to obtain the explicit object model of the object through the implicit object model, the user can also directly input an explicit object model, which improves the flexibility of the embodiment of the present application.

[0143] The following is an explanation of the relevant devices and equipment provided in the embodiments of the present application.

[0144] The present application also provides an image rendering device, as shown in FIG5 , wherein the device 500 includes:

[0145] The first acquisition module 501 is used to acquire a first implicit object model, which is used to predict an arbitrary-viewing angle image of a first object. For specific implementation, reference may be made to step 301 in the embodiment of FIG. 3 .

[0146] The first determination module 502 is used to determine a first three-dimensional geometric model and first three-dimensional material information of the first three-dimensional geometric model based on the first implicit object model. The first three-dimensional geometric model is a three-dimensional geometric model of the first object. For specific implementation methods, please refer to step 302 in the embodiment of Figure 3.

[0147] The first display module 503 is used to display the first explicit object model of the first object on the front-end interactive interface based on the first three-dimensional geometric model and the first three-dimensional material information; for specific implementation, please refer to step 303 in the embodiment of Figure 3.

[0148] The second determining module 504 is used to determine the display properties of the first explicit object model in the front-end interactive interface in response to the user's editing operation in the front-end interactive interface; for specific implementation methods, please refer to step 304 in the embodiment of Figure 3.

[0149] The first rendering module 505 is configured to render the first implicit object model based on the display properties of the first explicit object model in the front-end interactive interface to obtain at least one first object image. Specific implementation methods may refer to step 305 in the embodiment of FIG. 3 .

[0150] Optionally, the device further comprises:

[0151] a third determining module, configured to determine a first coordinate transformation relationship, where the first coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the first implicit object model and an object coordinate system of the first three-dimensional geometric model;

[0152] The first rendering module is used to:

[0153] The first implicit object model is rendered based on the first coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface.

[0154] Optionally, the device further comprises:

[0155] a fourth determining module, configured to determine a second coordinate transformation relationship, where the second coordinate transformation relationship indicates a transformation relationship between the object coordinate system of the first explicit object model and the interface coordinate system of the front-end interactive interface;

[0156] The first rendering module is used to:

[0157] The first implicit object model is rendered based on the second coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface.

[0158] Optionally, the first determining module is configured to:

[0159] The arbitrary perspective image of the first object outputted based on the first implicit object model is mapped to the first three-dimensional geometric model through a texture mapping algorithm to obtain first three-dimensional material information.

[0160] Optionally, the first determining module is configured to:

[0161] A first material model is determined based on the first implicit object model, where the first material model is used to predict an arbitrary viewing angle texture of the first object.

[0162] Optionally, the first acquisition module is used to:

[0163] A first implicit object model is trained based on a plurality of first sample data, where the plurality of first sample data is used to indicate imaging information of a first object from multiple perspectives.

[0164] Optionally, the device further comprises:

[0165] a second acquisition module, configured to acquire a second implicit object model, wherein the second implicit object model is used to predict an arbitrary perspective image of a second object;

[0166] A third acquisition module is used to acquire a second explicit object model of a second object input by the user based on the front-end interactive interface;

[0167] A second display module, configured to display a second explicit object model on a front-end interactive interface;

[0168] a fifth determining module, configured to determine display properties of the second explicit object model in the front-end interactive interface in response to an editing operation performed by a user in the front-end interactive interface;

[0169] A second rendering module is configured to render the second implicit object model based on display properties of the second explicit object model in the front-end interactive interface to obtain at least one second object image;

[0170] The fusion module is used to fuse at least one first object image and at least one second object image to obtain panoramic simulation data.

[0171] Optionally, the device further comprises:

[0172] a sixth determining module, configured to determine a third coordinate transformation relationship, where the third coordinate transformation relationship indicates a transformation relationship between the object coordinate system of the second implicit object model and the object coordinate system of the second explicit object model;

[0173] The second rendering module is used to:

[0174] The second implicit object model is rendered based on the third coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface.

[0175] Optionally, the sixth determining module is configured to:

[0176] outputting at least one image and a pose of the at least one image in an object coordinate system of the second implicit object model based on the second implicit object model;

[0177] localizing a pose of the at least one image in an object coordinate system of a second explicit object model;

[0178] A third coordinate transformation relationship is determined based on the pose of the at least one image in the object coordinate system of the second explicit object model and the pose of the at least one image in the object coordinate system of the second implicit object model.

[0179] Optionally, the device further comprises:

[0180] a seventh determining module, configured to determine a fourth coordinate transformation relationship, where the fourth coordinate transformation relationship indicates a transformation relationship between the object coordinate system of the second explicit object model and the interface coordinate system of the front-end interactive interface;

[0181] The second rendering module is used to:

[0182] The second implicit object model is rendered based on the fourth coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface.

[0183] The first acquisition module 501, the first determination module 502, the first display module 503, the second determination module 504, and the first rendering module 505 can all be implemented via software or hardware. For example, the implementation of the first acquisition module 501 will be described below using the first acquisition module 501 as an example. Similarly, the implementation of the first determination module 502, the first display module 503, the second determination module 504, and the first rendering module 505 can refer to the first acquisition module 501.

[0184] As an example of a software functional unit, the first acquisition module 501 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the first acquisition module 501 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0185] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0186] As an example of a hardware functional unit, the first acquisition module 501 may include at least one computing device, such as a server. Alternatively, the first acquisition module 501 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0187] The multiple computing devices included in the first acquisition module 501 can be distributed in the same region or in different regions. The multiple computing devices included in the first acquisition module 501 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the first acquisition module 501 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0188] It should be noted that, in other embodiments, the first acquisition module 501 can be used to execute any step in the image rendering method, the first determination module 502 can be used to execute any step in the image rendering method, the first display module 503 can be used to execute any step in the image rendering method, the second determination module 504 can be used to execute any step in the image rendering method, and the first rendering module 505 can be used to execute any step in the image rendering method. The steps that the first acquisition module 501, the first determination module 502, the first display module 503, the second determination module 504, and the first rendering module 505 are responsible for implementing can be specified as needed. The full functionality of the image rendering device is achieved by having the first acquisition module 501, the first determination module 502, the first display module 503, the second determination module 504, and the first rendering module 505 respectively implement different steps in the image rendering method.

[0189] The present embodiment also provides a computing device 600. As shown in Figure 6, computing device 600 includes a bus 602, a processor 604, a memory 606, and a communication interface 608. Processor 604, memory 606, and communication interface 608 communicate with each other via bus 602. Computing device 600 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in computing device 600.

[0190] Bus 602 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG6 shows a single bus line, but this does not imply a single bus or type of bus. Bus 602 may include a path for transmitting information between various components of computing device 600 (e.g., memory 606, processor 604, and communication interface 608).

[0191] The processor 604 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0192] The memory 606 may include volatile memory, such as random access memory (RAM). The processor 604 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0193] The memory 606 stores executable program code. The processor 604 executes the executable program code to implement the functions of the first acquisition module 501, the first determination module 502, the first display module 503, the second determination module 504, and the first rendering module 505, thereby implementing the image rendering method. In other words, the memory 606 stores instructions for executing the image rendering method.

[0194] Alternatively, the memory 606 stores executable codes, and the processor 604 executes the executable codes to respectively implement the functions of the aforementioned image rendering apparatus, thereby implementing the image rendering method. That is, the memory 606 stores instructions for executing the image rendering method.

[0195] The communication interface 608 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 600 and other devices or a communication network.

[0196] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0197] As shown in Fig. 7, the computing device cluster includes at least one computing device 600. The memory 606 in one or more computing devices 600 in the computing device cluster may store the same instructions for executing the image rendering method.

[0198] In some possible implementations, the memory 606 of one or more computing devices 600 in the computing device cluster may also store some instructions for executing the image rendering method. In other words, the combination of one or more computing devices 600 can jointly execute the instructions for executing the image rendering method.

[0199] It should be noted that the memory 606 in different computing devices 600 in the computing device cluster can store different instructions, each for executing a portion of the functions of the image rendering apparatus. That is, the instructions stored in the memory 606 in different computing devices 600 can implement the functions of one or more of the first acquisition module 501, the first determination module 502, the first display module 503, the second determination module 504, and the first rendering module 505.

[0200] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network or a local area network, etc. FIG8 shows a possible implementation. As shown in FIG8 , two computing devices 600A and 600B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 606 in the computing device 600A stores instructions for executing the functions of the first acquisition module 501, the first determination module 502, and the first rendering module 505. At the same time, the memory 606 in the computing device 600B stores instructions for executing the functions of the first display module 503 and the second determination module 504.

[0201] The connection method between the computing device clusters shown in Figure 8 can be that considering that the image rendering method provided in this application needs to interact with the user and a large amount of calculation is required in the background, it is considered to hand over the functions implemented by the first acquisition module 501, the first determination module 502 and the first rendering module 505 to the computing device 600A for execution, and the functions implemented by the first display module 503 and the second determination module 504 to the computing device 600B for execution.

[0202] It should be understood that the functionality of the computing device 600A shown in FIG8 may also be implemented by multiple computing devices 600. Similarly, the functionality of the computing device 600B may also be implemented by multiple computing devices 600.

[0203] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes an image rendering method.

[0204] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the image rendering method.

[0205] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. An image rendering method, characterized in that: The method comprises: Acquire a first implicit object model, where the first implicit object model is used to predict an arbitrary viewing angle image of a first object; Determine a first three-dimensional geometric model and first three-dimensional material information of the first three-dimensional geometric model based on the first implicit object model, where the first three-dimensional geometric model is a three-dimensional geometric model of the first object; Displaying a first explicit object model of the first object on a front-end interactive interface based on the first three-dimensional geometric model and the first three-dimensional material information; In response to a user's editing operation on the front-end interactive interface, determining a display attribute of the first explicit object model in the front-end interactive interface; Based on the display properties of the first explicit object model in the front-end interactive interface, the first implicit object model is rendered to obtain at least one first object image.

2. The method according to claim 1, characterized in that The method further comprises: Determining a first coordinate transformation relationship, wherein the first coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the first implicit object model and an object coordinate system of the first three-dimensional geometric model; The rendering of the first implicit object model based on the display attribute of the first explicit object model in the front-end interactive interface includes: The first implicit object model is rendered based on the first coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface.

3. The method according to claim 1 or 2, characterized in that The method further comprises: Determining a second coordinate transformation relationship, where the second coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the first explicit object model and an interface coordinate system of the front-end interactive interface; The rendering of the first implicit object model based on the display attribute of the first explicit object model in the front-end interactive interface includes: The first implicit object model is rendered based on the second coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface.

4. The method according to any one of claims 1 to 3, characterized in that The determining of the first three-dimensional material information of the first three-dimensional geometric model includes: The first three-dimensional geometric model is mapped using a texture mapping algorithm based on the arbitrary viewing angle image of the first object output by the first implicit object model to obtain the first three-dimensional material information.

5. The method according to any one of claims 1 to 3, characterized in that: The first three-dimensional material information includes the first material model, and the determining the first three-dimensional material information of the first three-dimensional geometric model includes: The first material model is determined based on the first implicit object model, where the first material model is used to predict an arbitrary viewing angle texture of the first object.

6. The method according to any one of claims 1 to 5, characterized in that: The obtaining of the first implicit object model comprises: A first implicit object model is trained based on a plurality of first sample data, where the plurality of first sample data is used to indicate imaging information of the first object from multiple viewing angles.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Acquire a second implicit object model, where the second implicit object model is used to predict an arbitrary viewing angle image of a second object; Acquire a second explicit object model of the second object input by the user based on the front-end interactive interface; Displaying the second explicit object model on the front-end interactive interface; In response to the user's editing operation on the front-end interactive interface, determining the display of the second explicit object model in the front-end interactive interface Display attributes; Rendering the second implicit object model based on the display properties of the second explicit object model in the front-end interactive interface to obtain at least one second object image; The at least one first object image and the at least one second object image are fused to obtain panoramic simulation data.

8. The method according to claim 7, characterized in that The method further comprises: determining a third coordinate transformation relationship, the third coordinate transformation relationship indicating a transformation relationship between an object coordinate system of the second implicit object model and an object coordinate system of the second explicit object model; The rendering of the second implicit object model based on the display attribute of the second explicit object model in the front-end interactive interface to obtain at least one second object image includes: Based on the third coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered.

9. The method according to claim 8, characterized in that Determining the third coordinate transformation relationship includes: Outputting at least one image and a pose of the at least one image in an object coordinate system of the second implicit object model based on the second implicit object model; locating a pose of the at least one image in an object coordinate system of the second explicit object model; The third coordinate transformation relationship is determined based on the pose of the at least one image in the object coordinate system of the second explicit object model and the pose of the at least one image in the object coordinate system of the second implicit object model.

10. The method according to any one of claims 7 to 9, characterized in that: The method further comprises: Determining a fourth coordinate transformation relationship, where the fourth coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the second explicit object model and an interface coordinate system of the front-end interactive interface; The rendering of the second implicit object model based on the display attribute of the second explicit object model in the front-end interactive interface includes: Based on the fourth coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered.

11. An image rendering device, characterized in that: The device comprises: A first acquisition module, used to acquire a first implicit object model, where the first implicit object model is used to predict an arbitrary viewing angle image of a first object; A first determining module, configured to determine a first three-dimensional geometric model and first three-dimensional material information of the first three-dimensional geometric model based on the first implicit object model, wherein the first three-dimensional geometric model is a three-dimensional geometric model of the first object; A first display module, configured to display a first explicit object model of the first object on a front-end interactive interface based on the first three-dimensional geometric model and the first three-dimensional material information; A second determination module, configured to determine a display attribute of the first explicit object model in the front-end interaction interface in response to an editing operation of the user in the front-end interaction interface; The first rendering module is used to render the first implicit object model based on the display properties of the first explicit object model in the front-end interactive interface to obtain at least one first object image.

12. The device according to claim 11, characterized in that The device also includes: a third determining module, configured to determine a first coordinate transformation relationship, wherein the first coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the first implicit object model and an object coordinate system of the first three-dimensional geometric model; The first rendering module is used for: The first implicit object model is rendered based on the first coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface.

13. The device according to claim 11 or 12, characterized in that The device also includes: a fourth determining module, configured to determine a second coordinate transformation relationship, wherein the second coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the first explicit object model and an interface coordinate system of the front-end interactive interface; The first rendering module is used for: The first implicit object model is rendered based on the second coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface.

14. The device according to any one of claims 11 to 13, characterized in that The first determining module is used for: The first three-dimensional geometric model is mapped using a texture mapping algorithm based on the arbitrary viewing angle image of the first object output by the first implicit object model to obtain the first three-dimensional material information.

15. The device according to any one of claims 11 to 13, characterized in that The first determining module is used for: The first material model is determined based on the first implicit object model, where the first material model is used to predict an arbitrary viewing angle texture of the first object.

16. The device according to any one of claims 11 to 15, characterized in that The first acquisition module is used for: A first implicit object model is trained based on a plurality of first sample data, where the plurality of first sample data is used to indicate imaging information of the first object from multiple viewing angles.

17. The device according to any one of claims 11 to 16, characterized in that The device also includes: A second acquisition module, used to acquire a second implicit object model, where the second implicit object model is used to predict an arbitrary viewing angle image of a second object; A third acquisition module, used to acquire a second explicit object model of the second object input by the user based on the front-end interactive interface; A second display module, used for displaying the second explicit object model on the front-end interactive interface; a fifth determining module, configured to determine a display attribute of the second explicit object model in the front-end interactive interface in response to an editing operation of the user in the front-end interactive interface; A second rendering module, configured to render the second implicit object model based on the display properties of the second explicit object model in the front-end interactive interface to obtain at least one second object image; The fusion module is used to fuse the at least one first object image and the at least one second object image to obtain panoramic simulation data.

18. The device according to claim 17, characterized in that The device also includes: a sixth determining module, configured to determine a third coordinate transformation relationship, wherein the third coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the second implicit object model and an object coordinate system of the second explicit object model; The second rendering module is used for: Based on the third coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered.

19. The device according to claim 18, characterized in that The sixth determination module is used for: Outputting at least one image and a pose of the at least one image in an object coordinate system of the second implicit object model based on the second implicit object model; locating a pose of the at least one image in an object coordinate system of the second explicit object model; The third coordinate transformation relationship is determined based on the pose of the at least one image in the object coordinate system of the second explicit object model and the pose of the at least one image in the object coordinate system of the second implicit object model.

20. The device according to any one of claims 17 to 19, characterized in that: The device also includes: a seventh determining module, configured to determine a fourth coordinate transformation relationship, wherein the fourth coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the second explicit object model and an interface coordinate system of the front-end interactive interface; The second rendering module is used for: Based on the fourth coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered.

21. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1-10.

22. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 10.

23. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 10.

Images