Image rendering method and device and related equipment

Abstract

The embodiment of the invention discloses an image rendering method and device and related equipment, and belongs to the technical field of computer vision. In the embodiment of the invention, for any to-be-rendered object, the three-dimensional geometric model and the three-dimensional material information of the object are extracted through the implicit object model of the object, and the explicit object model of the object is displayed on the front-end interaction interface based on the three-dimensional geometric model and the three-dimensional material information. And editing the display attribute of the explicit object model, and rendering the implicit object model based on the edited display attribute. In other words, only one implicit object model needs to be acquired, and an image rendering scheme combining the implicit object model and the explicit object model can be realized through the method provided by the embodiment of the invention, so that the editability of an image rendering process can be improved, and the high trueness of a rendering result can be ensured.

Description

[0001] This application claims priority to Chinese patent application No. 202311287935.8 filed on September 28, 2023, with invention name “A rendering method, device and system”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of computer vision technology, and in particular to an image rendering method, apparatus, and related equipment. Background Art

[0003] In computer vision technology, for any object to be rendered, a two-dimensional image of the object in a certain viewing angle and position can be rendered based on the implicit object model of the object. However, the process of rendering images through implicit object models is not very editable, resulting in the rendering results often being difficult to meet user needs. Summary of the invention

[0004] The embodiment of the present application provides an image rendering method, apparatus and related equipment, which can improve the editability of the image rendering process. The technical solution is as follows:

[0005] In a first aspect, an image rendering method is provided, in which a first implicit object model is obtained, the first implicit object model being used to predict an arbitrary perspective image of a first object; a first three-dimensional geometric model and first three-dimensional material information of the first three-dimensional geometric model are determined based on the first implicit object model, the first three-dimensional geometric model being the three-dimensional geometric model of the first object; a first explicit object model of the first object is displayed on a front-end interactive interface based on the first three-dimensional geometric model and the first three-dimensional material information; display properties of the first explicit object model in the front-end interactive interface are determined in response to an editing operation of a user on the front-end interactive interface; and the first implicit object model is rendered based on the display properties of the first explicit object model in the front-end interactive interface to obtain at least one first object image.

[0006] In the image rendering method provided in the embodiment of the present application, for any object to be rendered, the three-dimensional geometric model and three-dimensional material information of the object are extracted through the implicit object model of the object, and the explicit object model of the object is displayed on the front-end interactive interface based on the three-dimensional geometric model and the three-dimensional material information, the display properties of the explicit object model are edited, and the implicit object model is rendered based on the edited display properties. In other words, only an implicit object model needs to be obtained, and an image rendering solution combining an implicit object model with an explicit object model can be implemented through the method provided in the embodiment of the present application, which can not only improve the editability of the image rendering process, but also ensure the high authenticity of the rendering result.

[0007] Based on the method provided in the first aspect, in a possible implementation, a first coordinate transformation relationship may also be determined, the first coordinate transformation relationship indicating the transformation relationship between the object coordinate system of the first implicit object model and the object coordinate system of the first three-dimensional geometric model. In this scenario, based on the display properties of the first explicit object model in the front-end interactive interface, the implementation process of rendering the first implicit object model may be: based on the first coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface, the first implicit object model is rendered.

[0008] When determining the first 3D geometric model and the first 3D material information of the first 3D geometric model based on the first implicit object model, the first coordinate transformation relationship can also be determined to facilitate subsequent accurate rendering of the first implicit object model based on the first coordinate transformation relationship.

[0009] Based on the method provided in the first aspect, in a possible implementation, a second coordinate transformation relationship can also be determined, and the second coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the first explicit object model and the interface coordinate system of the front-end interactive interface. In this scenario, based on the display properties of the first explicit object model in the front-end interactive interface, the implementation process of rendering the first implicit object model can be: based on the second coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface, the first implicit object model is rendered.

[0010] Since the display properties of the determined first explicit object model in the front-end interactive interface refer to the display properties in the interface coordinate system of the front-end interactive interface, and the interface coordinate system and the object coordinate system of the first explicit object model may not be the same coordinate system, a second coordinate transformation relationship can also be determined to facilitate the subsequent accurate rendering of the first implicit object model based on the second coordinate transformation relationship.

[0011] Based on the method provided in the first aspect, in one possible implementation, the process of determining the first three-dimensional material information of the first three-dimensional geometric model can be: mapping the first three-dimensional geometric model through a texture mapping algorithm based on an arbitrary perspective image of the first object output by the first implicit object model to obtain the first three-dimensional material information.

[0012] The first three-dimensional material information obtained in the above manner includes the texture value at each position point in the first three-dimensional geometric model. The texture of the object can be directly displayed based on the texture value in the front-end interactive interface. Therefore, the first three-dimensional material information obtained in this manner can also be called explicit material information. The first three-dimensional material information obtained in this manner can increase the speed of the subsequent display of the first explicit object model, that is, it can increase the efficiency of rendering the first explicit object model in the front-end interactive interface.

[0013] Based on the method provided in the first aspect, in a possible implementation manner, the process of determining the first three-dimensional material information of the first three-dimensional geometric model may be: determining the first material model based on the first implicit object model, and the first material model is used to predict the arbitrary perspective texture of the first object.

[0014] In the above manner, a first material model capable of predicting the texture of the first object at any viewing angle can be obtained. The first three-dimensional material information obtained in this scenario can also be called implicit material information or neural material. The authenticity of the first explicit object model can be improved when the first explicit object model is subsequently displayed using the first material model.

[0015] Based on the method provided in the first aspect, in a possible implementation manner, the implementation process of obtaining the first implicit object model may be: training the first implicit object model based on multiple first sample data, where the multiple first sample data are used to indicate multiple perspective imaging information of the first object.

[0016] In an embodiment of the present application, a first implicit object model can be obtained through sample data training to implement an image rendering solution that combines the implicit object model with the explicit object model based on the first implicit object model.

[0017] Based on the method provided in the first aspect, in a possible implementation, a second implicit object model can also be obtained, and the second implicit object model is used to predict an arbitrary perspective image of the second object; a second explicit object model of the second object input by the user based on the front-end interactive interface is obtained; the second explicit object model is displayed on the front-end interactive interface; in response to the user's editing operation on the front-end interactive interface, the display properties of the second explicit object model in the front-end interactive interface are determined; based on the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered to obtain at least one second object image; at least one first object image and at least one second object image are fused to obtain panoramic simulation data.

[0018] In the image rendering method provided in the embodiment of the present application, in addition to obtaining the explicit object model of the object through the implicit object model, the user can also directly input an explicit object model to improve the flexibility of the embodiment of the present application.

[0019] Based on the method provided in the first aspect, in a possible implementation, a third coordinate transformation relationship can also be determined, and the third coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the second implicit object model and the object coordinate system of the second explicit object model. In this scenario, based on the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered to obtain at least one second object image. The implementation process can be: based on the third coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered.

[0020] Since the object coordinate system of the second explicit object model input by the user and the object coordinate system of the second implicit object model obtained above may not be the same coordinate system, a third coordinate transformation relationship may also be determined in the embodiment of the present application, so as to facilitate the subsequent accurate rendering of the second implicit object model based on the third coordinate transformation relationship.

[0021] Based on the method provided in the first aspect, in a possible implementation manner, the implementation process of determining the third coordinate transformation relationship can be: outputting at least one image and the pose of at least one image in the object coordinate system of the second implicit object model based on the second implicit object model; locating the pose of at least one image in the object coordinate system of the second explicit object model; determining the third coordinate transformation relationship based on the pose of at least one image in the object coordinate system of the second explicit object model and the pose of at least one image in the object coordinate system of the second implicit object model.

[0022] The third coordinate transformation relationship can be determined through the position and posture of the same image in the second explicit object model and the second implicit object model, thereby improving the flexibility of the embodiments of the present application.

[0023] Based on the method provided in the first aspect, in a possible implementation, a fourth coordinate transformation relationship can also be determined, and the fourth coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the second explicit object model and the interface coordinate system of the front-end interactive interface. In this scenario, based on the display properties of the second explicit object model in the front-end interactive interface, the implementation process of rendering the second implicit object model can be: based on the fourth coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered.

[0024] Since the display properties of the determined second explicit object model in the front-end interactive interface refer to the display properties in the interface coordinate system of the front-end interactive interface, and the interface coordinate system and the object coordinate system of the second explicit object model may not be a coordinate system, the fourth coordinate transformation relationship can also be determined to facilitate the subsequent accurate rendering of the second implicit object model based on the fourth coordinate transformation relationship.

[0025] In a second aspect, an image rendering device is provided, wherein the image rendering device has the function of implementing the image rendering method in the first aspect. The image rendering device comprises at least one module, wherein the at least one module is used to implement the image rendering method provided in the first aspect.

[0026] In a third aspect, a computing device cluster is provided, comprising at least one computing device, each computing device comprising a processor and a memory; the processor of at least one computing device is used to execute instructions stored in the memory of at least one computing device, so that the computing device cluster executes the image rendering method provided in the first aspect above.

[0027] In a fourth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device cluster, the computing device cluster executes the image rendering method provided in the first aspect.

[0028] In a fifth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the image rendering method provided in the first aspect.

[0029] The technical effects obtained by the above-mentioned second, third, fourth and fifth aspects are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a schematic diagram of the architecture of an image rendering system provided by an embodiment of the present application;

[0031] Figure 2 This is a method provided by the embodiment of the present application. Figure 1 Another schematic diagram of the architecture of the image rendering system shown;

[0032] Figure 3 is a flow chart of an image rendering method provided by an embodiment of the present application;

[0033] Figure 4 is a flow chart of another image rendering method provided by an embodiment of the present application;

[0034] Figure 5 is a structural schematic diagram of an image rendering device provided in an embodiment of the present application;

[0035] Figure 6 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0036] Figure 7 It is a schematic diagram of the architecture of a computing device cluster provided in an embodiment of the present application;

[0037] Figure 8 It is a schematic diagram of the architecture of another computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0039] Before explaining the embodiments of the present application, the application scenarios of the embodiments of the present application are first introduced.

[0040] In scenes such as autonomous driving simulation, game simulation, augmented reality (AR) and virtual reality (VR), a large number of rich and highly realistic three-dimensional (3D) scenes need to be produced. The representation methods of objects in 3D scenes can be divided into explicit and implicit. Explicit representation refers to directly giving the geometry, material, lighting and other information of the object in the scene through physically meaningful parameters, that is, creating an explicit object model of the object, where the explicit object model can also be called explicit material, and the explicit object model includes models such as mesh. In contrast, implicit representation refers to giving the color and density of the object at the corresponding point in the scene through a mapping function, that is, creating an implicit object model of the object, where the implicit object model can also be called implicit material, and the implicit object model includes models such as neural radiance field (NeRF) and so on.

[0041] At present, scene editing systems for making 3D scenes can include modeling software such as Blender (an open source 3D software) and game engines such as Unity (a real-time content development platform). These scene editing systems support flexible editing of translation, rotation, and scaling of different objects in 3D scenes, but only support editing of explicit object models. However, due to limitations such as modeling accuracy and rendering algorithms, the rendering results of explicit object models are still different from the images seen by the human eye in the real world, that is, the rendering realism is low.

[0042] Implicit object models such as the NeRF model can provide very realistic rendering effects. However, implicit object models cannot be directly edited in scene editing systems such as Blender and Unity, which means that the editability of the rendering process is very low.

[0043] Based on this, an image rendering method is provided in an embodiment of the present application. In the image rendering method provided in an embodiment of the present application, for any object to be rendered, the explicit object model of the object is displayed on the front-end interactive interface through the implicit object model of the object, the display properties of the explicit object model are edited, and the implicit object model is rendered based on the edited display properties. In other words, it is only necessary to obtain an implicit object model, and an image rendering solution combining an implicit object model with an explicit object model can be implemented through the method provided in an embodiment of the present application, which can not only improve the editability of the image rendering process, but also ensure the high authenticity of the rendering results.

[0044] Among them, the image rendering method provided in the embodiment of the present application can be applied to a cloud scene, and the cloud scene can be an example of an autonomous driving scene, etc. In this scenario, the image rendering method is executed by a cloud platform, that is, the cloud platform provides cloud services to users, and the cloud service is used to implement the image rendering method provided in the embodiment of the present application. Optionally, the image rendering method provided in the embodiment of the present application can also be executed by a terminal device. The embodiment of the present application does not limit the application scenario of the image rendering method.

[0045] The image rendering system provided in the embodiment of the present application is explained below.

[0046] Figure 1 Schematic diagram of the architecture of an image rendering system provided by an embodiment of the present application. Figure 1 As shown, the image rendering system 00 includes an implicit modeling module 01, an explicit and implicit alignment module 02, an explicit and implicit combined proxy scene editing module 03, and an explicit and implicit combined scene rendering module 04.

[0047] The implicit modeling module 01 is used to create an implicit object model for each of one or more objects, and output the implicit object model to the explicit and implicit alignment module 02 .

[0048] For example, for any object to be modeled, the user can provide multiple pictures of the object to the implicit modeling module 01, and the implicit modeling module 01 creates an implicit object model of the object based on these pictures. Optionally, in addition to providing multiple pictures of the object to the implicit modeling module 01, the user can also provide point cloud information and / or depth information of the object, and the implicit modeling module 01 creates an implicit object model of the object based on these pictures, point cloud information and / or depth information.

[0049] The implicit modeling module 01 may use NeRF, instant neural graphics primitives (InstantNGP), or reconstruction based on differentiable rendering to create an implicit object model, which will not be described one by one here.

[0050] In the embodiments of the present application, the objects to be modeled may be objects (such as vehicles, road obstacles), people, spaces (such as city streets, highway sections), etc., which will not be illustrated one by one here.

[0051] The explicit and implicit alignment module 02 is used to receive the implicit object model of each object in one or more objects output by the explicit and implicit alignment module 02, and output the explicit and implicit combination materials of each object in one or more objects to the explicit and implicit combination agent scene editing module 03 and the explicit and implicit combination scene rendering module 04 respectively.

[0052] For example, for any object, the explicit and implicit alignment module 02 can extract the 3D geometric model and 3D material information of the object from the implicit object model, so as to facilitate the subsequent display of the explicit object model of the object on the front-end interactive interface, and align the object coordinate system of the 3D geometric model with the object coordinate system of the implicit object model to jointly form the explicit and implicit combined material. In this scenario, the explicit and implicit combined material includes the implicit object model, the 3D geometric model and 3D material information of the object, and the conversion relationship between the object coordinate systems of the implicit object model and the 3D geometric model.

[0053] Among them, aligning the object coordinate system of the three-dimensional geometric model and the object coordinate system of the implicit object model can be understood as: determining the transformation relationship between the two object coordinate systems, and the transformation relationship may include translation parameters and / or rotation parameters and / or scaling parameters, etc.

[0054] Optionally, if the user already has an explicit object model of the object that has been modeled in advance, the user can input the explicit object model of the object into the explicit and implicit alignment module 02. In this scenario, the explicit and implicit alignment module 02 can align the object coordinate system of the explicit object model input by the user and the object coordinate system of the implicit object model output by the implicit modeling module 01 to jointly form the explicit and implicit combined material. In this scenario, the explicit and implicit combined material includes the implicit object model, the explicit object model input by the user, and the conversion relationship between the object coordinate systems of the two object models.

[0055] The scene editing module 03 of the visible and invisible combination agent is used to receive the visible and invisible combination material of each object in the one or more objects output by the visible and invisible alignment module 02, and output the visible and invisible combination scene to the visible and invisible combination scene rendering module 04.

[0056] For example, the scene editing module 03 of the explicit and implicit combination agent can display the explicit object model of each object on the front-end interactive interface. The front-end interactive interface can be an editing interface of Web technologies such as Three.js (a three-dimensional engine based on the native web graphics library (WebGL) encapsulation operation) and Babylon.js (an open source Web 3D rendering engine). The front-end interactive interface can also be an editing interface based on client technologies such as Blender (an open source 3D production software).

[0057] The user performs combination editing on the explicit object models in the front-end interactive interface. After the user performs combination editing on the explicit object models in the front-end interactive interface, the scene editing module 03 of the explicit and hidden combination agent determines the display attributes of each explicit object model in the front-end interactive interface, and the display attributes include the position, size, orientation and other attributes of the explicit object model in the front-end interactive interface. The scene editing module 03 of the explicit and hidden combination agent combines the display attributes of each explicit object model and the conversion relationship between the object coordinate system of each explicit object model and the interface coordinate system of the front-end interactive interface into an explicit and hidden combination scene.

[0058] The visible and invisible combined scene rendering module 04 is used to receive the visible and invisible combined scene output by the scene editing module 03 of the visible and invisible combined agent, and the visible and invisible combined material of each object in one or more objects output by the visible and invisible alignment module 02, and obtain camera parameter information, and render the implicit object model of each object based on the visible and invisible combined scene, the visible and invisible combined material of each object in one or more objects and the camera parameter information to obtain a highly realistic image.

[0059] The camera parameter information exemplarily includes a rendering perspective pose, and the rendering perspective pose may include one perspective pose or multiple perspective poses.

[0060] Figure 2 This is a method provided by the embodiment of the present application. Figure 1 Another architectural diagram of an image rendering system is shown.

[0061] like Figure 2 As shown, the explicit and implicit alignment module 02 includes an explicit material extraction module 021 , an explicit material input module 022 and an explicit and implicit alignment representation module 023 .

[0062] Among them, for any object, the explicit material extraction module 021 is used to extract the three-dimensional geometric model and three-dimensional material information from the implicit object model of the object. In the embodiment of the present application, the three-dimensional material information extracted by the explicit material extraction module 021 can be explicit material information such as a texture, which is what you see is what you get. In this scenario, the three-dimensional geometric model and three-dimensional material information extracted from the implicit object model of the object can be called an explicit object model. In other words, in the embodiment of the present application, an explicit object model can be directly extracted from the implicit object model.

[0063] Optionally, the extracted three-dimensional material information can be implicit material information that is seen but not obtained by a mapping function, etc. In this scenario, the three-dimensional geometric model and three-dimensional material information extracted from the implicit object model of the object can be called an explicit and implicit combined object model. In other words, in an embodiment of the present application, an explicit and implicit combined object model can be extracted from the implicit object model.

[0064] The explicit material input module 022 is used to obtain an explicit object model of an object input by the user in response to the user's operation on the front-end interactive interface.

[0065] The explicit and implicit alignment representation module 023 is used to align the object coordinate system of the three-dimensional geometric model determined by the explicit material extraction module 021 with the object coordinate system of the implicit object model, or to align the object coordinate system of the explicit object model determined by the explicit material input module 022 with the object coordinate system of the implicit object model.

[0066] like Figure 2 As shown, the scene editing module 03 of the combination of explicit and implicit agents includes a combination of explicit and implicit material combining module 031 , a combination of explicit and implicit agents editing module 032 and a scene alignment module 033 .

[0067] Among them, the explicit and implicit combined material combination module 031 is used to respond to the user adding one or more explicit object models of objects in the scene of the front-end interactive interface, determine the position of the explicit object model of each object on the front-end interactive interface, and locate the conversion relationship between the object coordinate system and the interface coordinate system of each explicit object model.

[0068] The explicit and implicit combined proxy editing module 032 is used to respond to the user's editing operation on a certain explicit object model, determine the display properties of the explicit object model, such as position, size, and orientation, and locate the conversion relationship between the object coordinate system and the interface coordinate system of the explicit object model.

[0069] The scene alignment module 033 is used to integrate the positions and transformation relationships of each explicit object model output by the explicit and implicit combined material combination module 031, and the display properties and transformation relationships of each explicit object model output by the explicit and implicit combined proxy editing module 032, to obtain a scene combination material, which includes the display properties of the explicit object model of each object in the scene, and the transformation relationship between the object coordinate system of the explicit object model and the interface coordinate system.

[0070] It should be noted that, for the same object, when the explicit and implicit combined material combination module 031 outputs the position and conversion relationship of the explicit object model of the object, and the explicit and implicit combined proxy editing module 032 also outputs the display attributes and conversion relationship of the explicit object model of the object, the position and conversion relationship of the explicit object model of the object output by the explicit and implicit combined material combination module 031 corresponds to the user operation time, and the display attributes and conversion relationship of the explicit object model of the object output by the explicit and implicit combined proxy editing module 032 also corresponds to the user operation time. Therefore, the scene alignment module 033 can obtain the position and conversion relationship of the explicit object model of the object with a later user operation time as the position and conversion relationship of the explicit object model of the object in the final scene combined material.

[0071] The above conversion relationships all refer to the conversion relationship between the object coordinate system of the explicit object model and the interface coordinate system.

[0072] Among them, when the user operates on the front-end interactive interface, he usually first adds an explicit object model of an object to the scene of the front-end interactive interface through the explicit and implicit combination material combination module 031, and then edits the display properties of the explicit object model through the explicit and implicit combination proxy editing module 032. In this scenario, the position and conversion relationship of the explicit object model of the object in the scene combination material are the position and conversion relationship in the display properties output by the explicit and implicit combination proxy editing module 032. Optionally, after the user adds an explicit object model of an object to the scene of the front-end interactive interface through the explicit and implicit combination material combination module 031, he may not edit the display properties of the explicit object model through the explicit and implicit combination proxy editing module 032. In this scenario, the position and conversion relationship of the explicit object model of the object in the scene combination material are the position and conversion relationship output by the explicit and implicit combination material combination module 031.

[0073] In addition, the scene combination material can be represented by, for example, a graphic library transmission format (glTF), a universal scene description (USD), or the like, which is not limited in the embodiments of the present application.

[0074] It should be noted that Figure 2 The image rendering system shown is used to Figure 1 The image rendering system shown is for illustrative purposes only and does not constitute a Figure 1 Definitions of the image rendering system shown.

[0075] In addition, the image rendering system 00 can be exemplified as a cloud platform. In this scenario, the image rendering system 00 can be implemented by one or more computing devices. Among them, the implicit modeling module 01, the explicit and implicit alignment module 02, the explicit and implicit combined agent scene editing module 03, the explicit and implicit combined scene rendering module 04 and other modules can be integrated on the same computing device, or deployed on different computing devices, which is not limited in the embodiments of the present application.

[0076] The image rendering method provided in the embodiment of the present application is explained below.

[0077] Figure 3 This is a flow chart of an image rendering method provided by an embodiment of the present application. Figure 3 As shown, the method includes the following steps.

[0078] Step 301: Acquire a first implicit object model, where the first implicit object model is used to predict an arbitrary viewing angle image of a first object.

[0079] In an embodiment of the present application, a first implicit object model can be obtained through sample data training, so as to implement an image rendering scheme combining the implicit object model with the explicit object model based on the first implicit object model. Based on this, in some embodiments, the first implicit object model can be obtained by: training the first implicit object model based on multiple first sample data, and the multiple first sample data are used to indicate multiple perspective imaging information of the first object.

[0080] For example, the plurality of first sample data may include a plurality of pictures collected for the first object. Optionally, the plurality of first sample data may also include a plurality of pictures collected for the first object, and point cloud information and / or depth information collected for the first object.

[0081] The first implicit object model may be a NeRF model. The basic principle of training a NeRF model is as follows: for any object, first collect sample images of the object from different perspectives, determine the camera pose of each sample image, and send the sample image and the corresponding camera pose as sample data to the NeRF model for training. The trained NeRF model can then input the two-dimensional image of the object from any perspective.

[0082] Optionally, the first implicit object model may also be other types of implicit models, and the embodiment of the present application does not limit the type of the first implicit object model.

[0083] Optionally, in some other embodiments, the first implicit object model can be obtained by: in response to a model input operation of the user in the front-end interactive interface, obtaining the first implicit object model input by the user. In other words, the first implicit object model can be an implicit object model that has been modeled in advance.

[0084] Step 302: Determine a first three-dimensional geometric model and first three-dimensional material information of the first three-dimensional geometric model based on the first implicit object model, where the first three-dimensional geometric model is a three-dimensional geometric model of a first object.

[0085] Wherein, determining the first three-dimensional geometric model based on the first implicit object model can be implemented by a geometry extraction algorithm. The geometry extraction algorithm can be, for example, a marching cube (MC), a baked signed distance field (BakedSDF) algorithm for gridded neural networks for real-time view synthesis, etc., which will not be described one by one here.

[0086] In some embodiments, determining the first three-dimensional material information of the first three-dimensional geometric model may be achieved by mapping the first three-dimensional geometric model through a texture mapping algorithm based on an arbitrary perspective image of the first object output by the first implicit corresponding model to obtain the first three-dimensional material information.

[0087] The first three-dimensional material information obtained in the above manner includes the texture value at each position point in the first three-dimensional geometric model. The texture of the object can be directly displayed based on the texture value in the front-end interactive interface. Therefore, the first three-dimensional material information obtained in this manner can also be called explicit material information. The first three-dimensional material information obtained in this manner can increase the speed of the subsequent display of the first explicit object model, that is, it can increase the efficiency of rendering the first explicit object model in the front-end interactive interface.

[0088] Among them, the arbitrary perspective image of the first object output based on the first implicit corresponding model is mapped to the first three-dimensional geometric model through a texture mapping algorithm. For example, it can be achieved through steps such as projection mapping, transformation function, texture sampling and texture conversion, which is not described in detail in the embodiments of the present application.

[0089] In other embodiments, determining the first three-dimensional material information of the first three-dimensional geometric model may be implemented by: determining a first material model based on a first implicit object model, the first material model being used to predict an arbitrary viewing angle texture of the first object; and the first three-dimensional material information includes the first material model.

[0090] In this way, a first material model capable of predicting the texture of the first object at any viewing angle can be obtained. The first three-dimensional material information obtained in this scenario can also be called a neural material. The first material model can be used to improve the authenticity of the first explicit object model when the first explicit object model is subsequently displayed.

[0091] The implementation method of determining the first material model based on the first implicit object model may be, for example, baking the first implicit object model into the first material model through BakedsSDF, NeRF2Mesh (a technology for recovering fine texture meshes from NeRF through adaptive surface refinement), etc. This embodiment of the application will not be described in detail.

[0092] The above is used to illustrate how to obtain the first three-dimensional material information. Optionally, the first three-dimensional material information can also be obtained from the first implicit object model in other ways, which will not be illustrated one by one here.

[0093] In addition, in some embodiments, when determining the first three-dimensional geometric model and the first three-dimensional material information of the first three-dimensional geometric model based on the first implicit object model, a first coordinate transformation relationship can also be determined. The first coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the first implicit object model and the object coordinate system of the first three-dimensional geometric model, so as to facilitate the subsequent accurate rendering of the first implicit object model based on the transformation relationship between the two object coordinate systems.

[0094] For example, since the first three-dimensional geometric model is determined based on the first implicit object model, the object coordinate system of the first three-dimensional geometric model and the object coordinate system of the first implicit object model can be the same coordinate system. In this case, the values ​​of the translation parameters, rotation parameters and other parameters indicating the conversion relationship between the object coordinate system of the first implicit object model and the object coordinate system of the first three-dimensional geometric model in the first coordinate conversion relationship can all be 0, so as to indicate that the object coordinate system of the first three-dimensional geometric model and the object coordinate system of the first implicit object model are the same coordinate system.

[0095] Optionally, in some scenarios, when constructing the first 3D geometric model based on the first implicit object model, the object coordinate system of the first 3D geometric model and the object coordinate system of the first implicit object model may not be the same coordinate system. In this case, through the first coordinate conversion relationship, the object coordinate system of the first 3D geometric model and the object coordinate system of the first implicit object model can be converted to the same coordinate system to facilitate subsequent rendering of the first implicit object model.

[0096] Optionally, if the object coordinate system of the first three-dimensional geometric model and the object coordinate system of the first implicit object model are the same coordinate system, the first coordinate transformation relationship may also be recorded. If the first coordinate transformation relationship is not found in the rendered image later, it is assumed that the object coordinate system of the first three-dimensional geometric model and the object coordinate system of the first implicit object model are the same coordinate system.

[0097] Step 303: Displaying a first explicit object model of the first object on a front-end interactive interface based on the first three-dimensional geometric model and the first three-dimensional material information.

[0098] In an embodiment of the present application, in order to improve the high editability of images rendered through an implicit object model, after obtaining a first three-dimensional geometric model and first three-dimensional material information through a first implicit object model, a first explicit object model of the first object is displayed on a front-end interactive interface based on the first three-dimensional geometric model and the first three-dimensional material information, so that a user can flexibly edit display properties based on the first explicit object model displayed on the front-end interactive interface.

[0099] In some embodiments, when the first three-dimensional material information includes the texture value at each position point in the first three-dimensional geometric model, the first three-dimensional geometric model in the default viewing angle position and the texture values ​​at each position point in the first three-dimensional geometric model in the default angular position can be directly displayed on the front-end interactive interface to obtain the first explicit object model. Subsequently, when the rotation of the first display object model is detected, the first three-dimensional geometric model in other viewing angle positions and the texture values ​​at each position point in the first three-dimensional geometric model in other viewing angle positions can be displayed.

[0100] In other embodiments, when the first three-dimensional material information includes a first material model, in response to the need to display the first object on the front-end interactive interface, the texture information of the first object under the default viewing angle posture is predicted through the first material model, and the texture information includes the texture values ​​at each position point in the first object under the default viewing angle posture, and the first three-dimensional geometric model is displayed according to the default viewing angle posture, and the texture information of the first object is mapped in the first three-dimensional geometric model to obtain the first explicit object model. Subsequently, when a change in the viewing angle posture is detected, the texture information of the first object under the changed viewing angle posture is output through the first material model, and the first three-dimensional geometric model is displayed according to the changed viewing angle posture, and the texture information of the first object is mapped in the first three-dimensional geometric model to obtain the first explicit object model under the changed viewing angle posture. This process can be called obtaining the first explicit object model by real-time rendering through the first material model.

[0101] Step 304: In response to the user's editing operation on the front-end interactive interface, determine the display properties of the first explicit object model in the front-end interactive interface.

[0102] The editing operation of the user in the front-end interactive interface may include one or more of the operations of moving, rotating and scaling. The display attributes of the determined first explicit object model in the front-end interactive interface may include display size and display posture, wherein the display posture includes position and orientation.

[0103] In addition, the display properties of the first explicit object model determined in step 304 in the front-end interactive interface refer to the display properties in the interface coordinate system of the front-end interactive interface, and the interface coordinate system and the object coordinate system of the first explicit object model may not be the same coordinate system. Based on this, in some embodiments, a second coordinate transformation relationship can also be determined in step 304, and the second coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the first explicit object model and the interface coordinate system of the front-end interactive interface, so as to facilitate the subsequent accurate rendering of the first implicit object model based on the second coordinate transformation relationship.

[0104] Optionally, if the object coordinate system of the first explicit object model and the interface coordinate system of the front-end interactive interface are the same coordinate system, the second coordinate transformation relationship can also be determined. In this case, the values ​​of the translation parameters, rotation parameters and other parameters indicating the transformation relationship between the object coordinate system of the first explicit object model and the interface coordinate system in the second coordinate transformation relationship can all be 0, to indicate that the object coordinate system of the first explicit object model and the interface coordinate system are the same coordinate system.

[0105] Optionally, if the object coordinate system of the first explicit object model and the interface coordinate system of the front-end interactive interface are the same coordinate system, the second coordinate transformation relationship may not be recorded. If the second coordinate transformation relationship is not found in the rendered image later, it is assumed that the object coordinate system of the first explicit object model and the interface coordinate system of the front-end interactive interface are the same coordinate system.

[0106] Step 305: Based on the display properties of the first explicit object model in the front-end interactive interface, the first implicit object model is rendered to obtain at least one first object image.

[0107] In some embodiments, based on the display properties of the first explicit object model in the front-end interactive interface, the first implicit object model may be rendered by rendering the first implicit object model based on the first coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface. The first coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the first implicit object model and the object coordinate system of the first three-dimensional geometric model.

[0108] Among them, the implementation method of rendering the first implicit object model based on the first coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface can be: based on the display properties of the first explicit object model in the front-end interactive interface, determine the display properties of the first explicit object model in the world coordinate system; based on the first coordinate transformation relationship, convert the display properties of the first display object model in the world coordinate system into the display properties of the first implicit object model in the world coordinate system; and render the first implicit object model based on the display properties of the first implicit object model in the world coordinate system.

[0109] For example, when the interface coordinate system of the front-end interactive interface is the world coordinate system, the display properties of the first explicit object model in the front-end interactive interface can be directly used as the display properties of the first explicit object model in the world coordinate system.

[0110] Optionally, when the interface coordinate system of the front-end interactive interface is not the world coordinate system, it is necessary to convert the display attributes of the first explicit object model in the front-end interactive interface into the display attributes of the first explicit object model in the world coordinate system through coordinate conversion. The specific implementation method will be described in detail later and will not be expanded here.

[0111] In other embodiments, based on the display properties of the first explicit object model in the front-end interactive interface, the first implicit object model may be rendered by rendering the first implicit object model based on the second coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface. The second coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the first explicit object model and the interface coordinate system of the front-end interactive interface.

[0112] Among them, rendering the first implicit object model based on the second coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface can be understood as: based on the second coordinate transformation relationship, the display properties of the first explicit object model in the front-end interactive interface are converted into the display properties of the first explicit object model in the world coordinate system, and then the first implicit object model is rendered based on the display properties of the first explicit object model in the world coordinate system.

[0113] For example, if the object coordinate systems of the first implicit object model and the first three-dimensional geometric model used to generate the first explicit object model are the same coordinate system, then after determining the display properties of the first explicit object model in the world coordinate system, the display properties of the first explicit object model in the world coordinate system can be used as the display properties of the first implicit object model in the world coordinate system, and then the first implicit object model can be rendered based on the display properties of the first implicit object model in the world coordinate system.

[0114] As another example, if the object coordinate systems of the first implicit object model and the first three-dimensional geometric model used to generate the first explicit object model are not the same coordinate system, then after determining the display properties of the first explicit object model in the world coordinate system, the display properties of the first explicit object model in the world coordinate system can be converted into the display properties of the first implicit object model in the world coordinate system by means of coordinate system conversion, and then the first implicit object model can be rendered based on the display properties of the first implicit object model in the world coordinate system.

[0115] The display attributes of the first implicit object model in the world object coordinate system include information such as display size and display posture. The display attributes and the view posture to be rendered are input into the first implicit object model, and the first implicit object model can output a first object image corresponding to the view posture.

[0116] The viewing angle pose indicates the pose of the rendering camera in the world coordinate system. The first implicit object model can convert the display attributes of the first implicit object model in the world object coordinate system into the display attributes of the first implicit object model in the camera coordinate system of the rendering camera through the viewing angle pose, and the first object image corresponding to the viewing angle pose can be obtained based on the display attributes of the first implicit object model in the camera coordinate system. The specific implementation method will not be described in detail here.

[0117] It should be noted that the above-mentioned conversion process of display attributes in different coordinate systems is used for illustrative purposes. In the embodiments of the present application, display attributes can also be converted in different coordinate systems in other ways, which will not be illustrated one by one here.

[0118] In summary, in the image rendering method provided in the embodiment of the present application, for any object to be rendered, the three-dimensional geometric model and three-dimensional material information of the object are extracted through the implicit object model of the object, and the explicit object model of the object is displayed on the front-end interactive interface based on the three-dimensional geometric model and the three-dimensional material information, the display properties of the explicit object model are edited, and the implicit object model is rendered based on the edited display properties. In other words, only an implicit object model needs to be obtained, and an image rendering solution combining an implicit object model with an explicit object model can be implemented through the method provided in the embodiment of the present application, which can not only improve the editability of the image rendering process, but also ensure the high authenticity of the rendering result.

[0119] based on Figure 1 and Figure 2 It can be seen from the image rendering system shown that the explicit object model of the object can be obtained through the implicit object model, and the user can also directly input an explicit object model to improve the flexibility of the embodiment of the present application. Figure 1 or Figure 2 The system shown is Figure 3 When rendering objects in the scene in the embodiment shown in the figure, the following steps can be further performed: Figure 4 The embodiment shown renders other objects in the scene. It should be noted that Figure 3 and Figure 4 The embodiments shown can be implemented individually or in combination.

[0120] Step 401: Acquire a second implicit object model, where the second implicit object model is used to predict an arbitrary viewing angle image of a second object.

[0121] The implementation of step 401 can refer to Figure 3 Step 301 in the illustrated embodiment will not be described in detail here.

[0122] Step 402: Obtain a second explicit object model of a second object input by a user based on the front-end interactive interface.

[0123] The second explicit object model may be a mesh model or the like.

[0124] Since the object coordinate system of the second explicit object model input by the user and the object coordinate system of the second implicit object model obtained in step 401 may not be the same coordinate system, in some embodiments, a third coordinate transformation relationship may be determined, the third coordinate transformation relationship indicating the transformation relationship between the object coordinate system of the second implicit object model and the object coordinate system of the second explicit object model, so as to accurately render the second implicit object model based on the third coordinate transformation relationship.

[0125] By way of example, determining the third coordinate transformation relationship may be implemented by: outputting at least one image and the pose of at least one image in the object coordinate system of the second implicit object model based on the second implicit object model; locating the pose of at least one image in the object coordinate system of the second explicit object model; and determining the third coordinate transformation relationship based on the pose of at least one image in the object coordinate system of the second explicit object model and the pose of at least one image in the object coordinate system of the second implicit object model.

[0126] The third coordinate transformation relationship can be determined through the position and posture of the same image in the second explicit object model and the second implicit object model, thereby improving the flexibility of the embodiments of the present application.

[0127] The positioning of the at least one image in the object coordinate system of the second explicit object model may be achieved, for example, by a 6-degree-of-freedom (dof) positioning algorithm, which will not be described in detail herein.

[0128] Optionally, in the embodiment of the present application, the third coordinate transformation relationship may also be determined by other methods. For example, when the user inputs the second explicit object model, the user may also manually align the object coordinate system of the second explicit object model with the object coordinate system of the second implicit object model in the front-end interactive interface, and the system determines the third coordinate transformation relationship based on operations such as translation, rotation, and scaling during the alignment process, which will not be described in detail here.

[0129] Step 403: Display the second explicit object model on the front-end interactive interface.

[0130] Since the explicit object model of the second object, ie, the second explicit object model, has been acquired through step 402, the second explicit object model can be directly displayed on the front-end interactive interface.

[0131] Step 404: In response to the user's editing operation on the front-end interactive interface, the display properties of the second explicit object model in the front-end interactive interface are determined.

[0132] The implementation of step 404 can refer to Figure 3 Step 304 in the embodiment will not be described in detail here.

[0133] Similar to step 304, since the display properties of the second explicit object model determined in the front-end interactive interface refer to the display properties in the interface coordinate system of the front-end interactive interface, and the interface coordinate system and the object coordinate system of the second explicit object model may not be the same coordinate system, a fourth coordinate transformation relationship can also be determined to facilitate the subsequent accurate rendering of the second implicit object model based on the fourth coordinate transformation relationship. The fourth coordinate transformation relationship indicates the transformation relationship between the object coordinate system of the second explicit object model and the interface coordinate system of the front-end interactive interface. For related implementation methods, please refer to Figure 3 Step 304 in the embodiment is also not described in detail here.

[0134] Step 405: Based on the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered to obtain at least one second object image.

[0135] Among them, in some embodiments, the second implicit object model is rendered based on the display properties of the second explicit object model in the front-end interactive interface, and the implementation method for obtaining at least one second object image can be: based on the third coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered.

[0136] In other embodiments, the implementation method of rendering the second implicit object model based on the display properties of the second explicit object model in the front-end interactive interface can be: rendering the second implicit object model based on the fourth coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface.

[0137] The above implementation can refer to Figure 3 Step 305 in the embodiment will not be described in detail here.

[0138] Step 406: Fusing at least one first object image and at least one second object image to obtain panoramic simulation data.

[0139] Among them, at least one first object image corresponds to at least one viewing angle and posture, and at least one second object image also corresponds to at least one viewing angle and posture. Based on this, in step 406, the first object image and the second object image corresponding to the same viewing angle and posture can be fused respectively to obtain at least one panoramic image, and the at least one panoramic image constitutes the panoramic simulation data.

[0140] The embodiment of the present application does not limit the implementation method of fusing the first object image and the second object image corresponding to the same viewing angle and posture.

[0141] In summary, in the image rendering method provided in the embodiment of the present application, in addition to being able to obtain the explicit object model of the object through the implicit object model, an explicit object model can also be directly input by the user, which improves the flexibility of the embodiment of the present application.

[0142] The following is an explanation of the relevant devices and equipment provided in the embodiments of the present application.

[0143] The present application also provides an image rendering device, such as Figure 5 As shown, the device 500 includes:

[0144] The first acquisition module 501 is used to acquire a first implicit object model, and the first implicit object model is used to predict an arbitrary viewing angle image of the first object. For specific implementation methods, please refer to Figure 3 Step 301 in the embodiment.

[0145] The first determination module 502 is used to determine a first 3D geometric model and first 3D material information of the first 3D geometric model based on the first implicit object model, where the first 3D geometric model is a 3D geometric model of the first object. For specific implementation methods, please refer to Figure 3 Step 302 in the embodiment.

[0146] The first display module 503 is used to display the first explicit object model of the first object on the front-end interactive interface based on the first three-dimensional geometric model and the first three-dimensional material information; the specific implementation method can refer to Figure 3 Step 303 in the embodiment.

[0147] The second determination module 504 is used to determine the display attributes of the first explicit object model in the front-end interactive interface in response to the user's editing operation in the front-end interactive interface; the specific implementation method can refer to Figure 3 Step 304 in an embodiment.

[0148] The first rendering module 505 is used to render the first implicit object model based on the display properties of the first explicit object model in the front-end interactive interface to obtain at least one first object image. Figure 3 Step 305 in the embodiment.

[0149] Optionally, the device further comprises:

[0150] a third determining module, configured to determine a first coordinate transformation relationship, wherein the first coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the first implicit object model and an object coordinate system of the first three-dimensional geometric model;

[0151] The first rendering module is used to:

[0152] The first implicit object model is rendered based on the first coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface.

[0153] Optionally, the device further comprises:

[0154] a fourth determining module, configured to determine a second coordinate transformation relationship, wherein the second coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the first explicit object model and an interface coordinate system of the front-end interactive interface;

[0155] The first rendering module is used to:

[0156] Based on the second coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface, the first implicit object model is rendered.

[0157] Optionally, the first determining module is used to:

[0158] The first three-dimensional geometric model is mapped using a texture mapping algorithm based on an arbitrary viewing angle image of the first object outputted by the first implicit object model to obtain first three-dimensional material information.

[0159] Optionally, the first determining module is used to:

[0160] A first material model is determined based on the first implicit object model, where the first material model is used to predict an arbitrary viewing angle texture of the first object.

[0161] Optionally, the first acquisition module is used to:

[0162] A first implicit object model is trained based on a plurality of first sample data, where the plurality of first sample data are used to indicate imaging information of a first object from a plurality of viewing angles.

[0163] Optionally, the device further comprises:

[0164] A second acquisition module, used to acquire a second implicit object model, where the second implicit object model is used to predict an arbitrary viewing angle image of a second object;

[0165] A third acquisition module, used to acquire a second explicit object model of a second object input by a user based on the front-end interactive interface;

[0166] A second display module, used for displaying a second explicit object model on the front-end interactive interface;

[0167] A fifth determination module, configured to determine display properties of the second explicit object model in the front-end interactive interface in response to an editing operation of the user in the front-end interactive interface;

[0168] A second rendering module, configured to render the second implicit object model based on display properties of the second explicit object model in the front-end interactive interface to obtain at least one second object image;

[0169] The fusion module is used to fuse at least one first object image and at least one second object image to obtain panoramic simulation data.

[0170] Optionally, the device further comprises:

[0171] a sixth determining module, configured to determine a third coordinate transformation relationship, wherein the third coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the second implicit object model and an object coordinate system of the second explicit object model;

[0172] The second rendering module is used to:

[0173] Based on the third coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered.

[0174] Optionally, the sixth determining module is used to:

[0175] Outputting at least one image and a pose of the at least one image in an object coordinate system of the second implicit object model based on the second implicit object model;

[0176] localizing a pose of at least one image in an object coordinate system of a second explicit object model;

[0177] A third coordinate transformation relationship is determined based on the pose of at least one image in the object coordinate system of the second explicit object model and the pose of at least one image in the object coordinate system of the second implicit object model.

[0178] Optionally, the device further comprises:

[0179] a seventh determination module, configured to determine a fourth coordinate transformation relationship, wherein the fourth coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the second explicit object model and an interface coordinate system of the front-end interactive interface;

[0180] The second rendering module is used to:

[0181] Based on the fourth coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered.

[0182] Among them, the first acquisition module 501, the first determination module 502, the first display module 503, the second determination module 504 and the first rendering module 505 can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the first acquisition module 501 is described below by taking the first acquisition module 501 as an example. Similarly, the implementation of the first determination module 502, the first display module 503, the second determination module 504 and the first rendering module 505 can refer to the first acquisition module 501.

[0183] As an example of a software functional unit, the first acquisition module 501 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the first acquisition module 501 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with similar geographical locations. Among them, usually a region can include multiple AZs.

[0184] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0185] As an example of a hardware functional unit, the first acquisition module 501 may include at least one computing device, such as a server, etc. Alternatively, the first acquisition module 501 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0186] The multiple computing devices included in the first acquisition module 501 can be distributed in the same region or in different regions. The multiple computing devices included in the first acquisition module 501 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the first acquisition module 501 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0187] It should be noted that, in other embodiments, the first acquisition module 501 can be used to execute any step in the image rendering method, the first determination module 502 can be used to execute any step in the image rendering method, the first display module 503 can be used to execute any step in the image rendering method, the second determination module 504 can be used to execute any step in the image rendering method, and the first rendering module 505 can be used to execute any step in the image rendering method. The steps that the first acquisition module 501, the first determination module 502, the first display module 503, the second determination module 504, and the first rendering module 505 are responsible for implementing can be specified as needed, and the first acquisition module 501, the first determination module 502, the first display module 503, the second determination module 504, and the first rendering module 505 respectively implement different steps in the image rendering method to realize the full functions of the image rendering device.

[0188] The present application embodiment also provides a computing device 600. Figure 6 As shown, computing device 600 includes: bus 602, processor 604, memory 606 and communication interface 608. Processor 604, memory 606 and communication interface 608 communicate through bus 602. Computing device 600 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in computing device 600.

[0189] The bus 602 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The bus 602 may include a path for transmitting information between various components of the computing device 600 (eg, the memory 606, the processor 604, and the communication interface 608).

[0190] The processor 604 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0191] The memory 606 may include a volatile memory, such as a random access memory (RAM). The processor 604 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0192] The memory 606 stores executable program codes, and the processor 604 executes the executable program codes to respectively implement the functions of the first acquisition module 501, the first determination module 502, the first display module 503, the second determination module 504, and the first rendering module 505, thereby implementing the image rendering method. That is, the memory 606 stores instructions for executing the image rendering method.

[0193] Alternatively, the memory 606 stores executable codes, and the processor 604 executes the executable codes to respectively implement the functions of the aforementioned image rendering apparatus, thereby implementing the image rendering method. That is, the memory 606 stores instructions for executing the image rendering method.

[0194] The communication interface 608 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 600 and other devices or communication networks.

[0195] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0196] like Figure 7 As shown, the computing device cluster includes at least one computing device 600. The memory 606 in one or more computing devices 600 in the computing device cluster may store the same instructions for executing the image rendering method.

[0197] In some possible implementations, the memory 606 of one or more computing devices 600 in the computing device cluster may also store partial instructions for executing the image rendering method. In other words, the combination of one or more computing devices 600 may jointly execute instructions for executing the image rendering method.

[0198] It should be noted that the memory 606 in different computing devices 600 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the image rendering apparatus. That is, the instructions stored in the memory 606 in different computing devices 600 may implement the functions of one or more of the first acquisition module 501, the first determination module 502, the first display module 503, the second determination module 504 and the first rendering module 505.

[0199] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Figure 8 A possible implementation is shown. Figure 8 As shown, two computing devices 600A and 600B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 606 in the computing device 600A stores instructions for executing the functions of the first acquisition module 501, the first determination module 502, and the first rendering module 505. At the same time, the memory 606 in the computing device 600B stores instructions for executing the functions of the first display module 503 and the second determination module 504.

[0200] Figure 8 The connection method between the computing device clusters shown can be that considering that the image rendering method provided in the present application needs to interact with the user and a large amount of calculation is required in the background, it is considered that the functions implemented by the first acquisition module 501, the first determination module 502 and the first rendering module 505 are executed by the computing device 600A, and the functions implemented by the first display module 503 and the second determination module 504 are executed by the computing device 600B.

[0201] It should be understood that Figure 8 The functions of the computing device 600A shown in FIG. 6A may also be completed by multiple computing devices 600. Similarly, the functions of the computing device 600B may also be completed by multiple computing devices 600.

[0202] The embodiment of the present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the image rendering method.

[0203] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the image rendering method.

[0204] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. An image rendering method, characterized in that: The method comprises: Acquire a first implicit object model, where the first implicit object model is used to predict an arbitrary viewing angle image of a first object; Determine a first three-dimensional geometric model and first three-dimensional material information of the first three-dimensional geometric model based on the first implicit object model, where the first three-dimensional geometric model is a three-dimensional geometric model of the first object; Displaying a first explicit object model of the first object on a front-end interactive interface based on the first three-dimensional geometric model and the first three-dimensional material information; In response to a user's editing operation on the front-end interactive interface, determining a display attribute of the first explicit object model in the front-end interactive interface; Based on the display properties of the first explicit object model in the front-end interactive interface, the first implicit object model is rendered to obtain at least one first object image.

2. The method according to claim 1, characterized in that The method further comprises: Determining a first coordinate transformation relationship, wherein the first coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the first implicit object model and an object coordinate system of the first three-dimensional geometric model; The rendering of the first implicit object model based on the display attribute of the first explicit object model in the front-end interactive interface includes: The first implicit object model is rendered based on the first coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface.

3. The method according to claim 1 or 2, characterized in that The method further comprises: Determining a second coordinate transformation relationship, where the second coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the first explicit object model and an interface coordinate system of the front-end interactive interface; The rendering of the first implicit object model based on the display attribute of the first explicit object model in the front-end interactive interface includes: The first implicit object model is rendered based on the second coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface.

4. The method according to any one of claims 1 to 3, characterized in that The determining of the first three-dimensional material information of the first three-dimensional geometric model includes: The first three-dimensional geometric model is mapped using a texture mapping algorithm based on the arbitrary viewing angle image of the first object output by the first implicit object model to obtain the first three-dimensional material information.

5. The method according to any one of claims 1 to 3, characterized in that: The first three-dimensional material information includes the first material model, and the determining the first three-dimensional material information of the first three-dimensional geometric model includes: The first material model is determined based on the first implicit object model, where the first material model is used to predict an arbitrary viewing angle texture of the first object.

6. The method according to any one of claims 1 to 5, characterized in that: The obtaining of the first implicit object model comprises: A first implicit object model is trained based on a plurality of first sample data, where the plurality of first sample data is used to indicate imaging information of the first object from multiple viewing angles.

7. The method according to any one of claims 1 to 6, characterized in that: The method further comprises: Acquire a second implicit object model, where the second implicit object model is used to predict an arbitrary viewing angle image of a second object; Acquire a second explicit object model of the second object input by the user based on the front-end interactive interface; Displaying the second explicit object model on the front-end interactive interface; In response to a user's editing operation on the front-end interactive interface, determining a display attribute of the second explicit object model in the front-end interactive interface; Rendering the second implicit object model based on the display properties of the second explicit object model in the front-end interactive interface to obtain at least one second object image; The at least one first object image and the at least one second object image are fused to obtain panoramic simulation data.

8. The method according to claim 7, characterized in that The method further comprises: determining a third coordinate transformation relationship, the third coordinate transformation relationship indicating a transformation relationship between an object coordinate system of the second implicit object model and an object coordinate system of the second explicit object model; The rendering of the second implicit object model based on the display attribute of the second explicit object model in the front-end interactive interface to obtain at least one second object image includes: Based on the third coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered.

9. The method according to claim 8, characterized in that Determining the third coordinate transformation relationship includes: Outputting at least one image and a pose of the at least one image in an object coordinate system of the second implicit object model based on the second implicit object model; locating a pose of the at least one image in an object coordinate system of the second explicit object model; The third coordinate transformation relationship is determined based on the pose of the at least one image in the object coordinate system of the second explicit object model and the pose of the at least one image in the object coordinate system of the second implicit object model.

10. The method according to any one of claims 7 to 9, characterized in that: The method further comprises: Determining a fourth coordinate transformation relationship, where the fourth coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the second explicit object model and an interface coordinate system of the front-end interactive interface; The rendering of the second implicit object model based on the display attribute of the second explicit object model in the front-end interactive interface includes: Based on the fourth coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered.

11. An image rendering device, characterized in that: The device comprises: A first acquisition module, used to acquire a first implicit object model, where the first implicit object model is used to predict an arbitrary viewing angle image of a first object; A first determining module, configured to determine a first three-dimensional geometric model and first three-dimensional material information of the first three-dimensional geometric model based on the first implicit object model, wherein the first three-dimensional geometric model is a three-dimensional geometric model of the first object; A first display module, configured to display a first explicit object model of the first object on a front-end interactive interface based on the first three-dimensional geometric model and the first three-dimensional material information; A second determination module, configured to determine a display attribute of the first explicit object model in the front-end interaction interface in response to an editing operation of the user in the front-end interaction interface; The first rendering module is used to render the first implicit object model based on the display properties of the first explicit object model in the front-end interactive interface to obtain at least one first object image.

12. The device according to claim 11, characterized in that The device also includes: a third determining module, configured to determine a first coordinate transformation relationship, wherein the first coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the first implicit object model and an object coordinate system of the first three-dimensional geometric model; The first rendering module is used for: The first implicit object model is rendered based on the first coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface.

13. The device according to claim 11 or 12, characterized in that The device also includes: a fourth determining module, configured to determine a second coordinate transformation relationship, wherein the second coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the first explicit object model and an interface coordinate system of the front-end interactive interface; The first rendering module is used for: The first implicit object model is rendered based on the second coordinate transformation relationship and the display properties of the first explicit object model in the front-end interactive interface.

14. The device according to any one of claims 11 to 13, characterized in that The first determining module is used for: The first three-dimensional geometric model is mapped using a texture mapping algorithm based on the arbitrary viewing angle image of the first object output by the first implicit object model to obtain the first three-dimensional material information.

15. The device according to any one of claims 11 to 13, characterized in that The first determining module is used for: The first material model is determined based on the first implicit object model, where the first material model is used to predict an arbitrary viewing angle texture of the first object.

16. The device according to any one of claims 11 to 15, characterized in that The first acquisition module is used for: A first implicit object model is trained based on a plurality of first sample data, where the plurality of first sample data is used to indicate imaging information of the first object from multiple viewing angles.

17. The device according to any one of claims 11 to 16, characterized in that The device also includes: A second acquisition module, used to acquire a second implicit object model, where the second implicit object model is used to predict an arbitrary viewing angle image of a second object; A third acquisition module, used to acquire a second explicit object model of the second object input by the user based on the front-end interactive interface; A second display module, used for displaying the second explicit object model on the front-end interactive interface; a fifth determining module, configured to determine a display attribute of the second explicit object model in the front-end interactive interface in response to an editing operation of the user in the front-end interactive interface; A second rendering module, configured to render the second implicit object model based on the display properties of the second explicit object model in the front-end interactive interface to obtain at least one second object image; The fusion module is used to fuse the at least one first object image and the at least one second object image to obtain panoramic simulation data.

18. The device according to claim 17, characterized in that The device also includes: a sixth determining module, configured to determine a third coordinate transformation relationship, wherein the third coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the second implicit object model and an object coordinate system of the second explicit object model; The second rendering module is used for: Based on the third coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered.

19. The device according to claim 18, characterized in that The sixth determination module is used for: Outputting at least one image and a pose of the at least one image in an object coordinate system of the second implicit object model based on the second implicit object model; locating a pose of the at least one image in an object coordinate system of the second explicit object model; The third coordinate transformation relationship is determined based on the pose of the at least one image in the object coordinate system of the second explicit object model and the pose of the at least one image in the object coordinate system of the second implicit object model.

20. The device according to any one of claims 17 to 19, characterized in that: The device also includes: a seventh determining module, configured to determine a fourth coordinate transformation relationship, wherein the fourth coordinate transformation relationship indicates a transformation relationship between an object coordinate system of the second explicit object model and an interface coordinate system of the front-end interactive interface; The second rendering module is used for: Based on the fourth coordinate transformation relationship and the display properties of the second explicit object model in the front-end interactive interface, the second implicit object model is rendered.

21. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1-10.

22. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 10.

23. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 10.

Images