Image rendering method, device, apparatus and storage medium

Abstract

This invention provides an image rendering method, apparatus, device, and storage medium. The method includes: acquiring object attributes of an object to be rendered; predicting smoothing parameters based on a 3D image to be shaded corresponding to the object with these object attributes; and using these smoothing parameters for shading, thus completing the rendering of the object. The predicted smoothing parameters reflect the material of the object to be rendered. Using this scheme, the smoothing parameters used in the shading process are predicted and specifically targeted at the material of the object to be rendered. Therefore, the material can be more accurately incorporated into the rendering process, thereby improving the realism of the rendering.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image rendering method, apparatus, device, and storage medium. Background Technology

[0002] With the development of computers, 3D modeling has been applied to more and more fields. For example, in the fields of animation or game production, all objects in animation or game scenes can be obtained through 3D modeling. In the live streaming field, virtual anchors have recently emerged, where anchors can create their own virtual avatars using application software (APP) and use these avatars to broadcast live.

[0003] Taking the virtual avatar of a virtual anchor as an example, this avatar needs to be rendered before it is displayed on the screen. Therefore, how to improve the realism of the rendering becomes an urgent problem to be solved. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide an image rendering method, apparatus, device, and storage medium to ensure the realism of the rendering.

[0005] In a first aspect, embodiments of the present invention provide an image rendering method, comprising:

[0006] Get the object properties of the object to be rendered;

[0007] Based on the 3D image of the object to be rendered corresponding to the object having the object attributes, predict the smoothing parameters that reflect the material of the object to be rendered.

[0008] The three-dimensional image to be colored is rendered according to the smoothing parameters.

[0009] In a second aspect, embodiments of the present invention provide an image rendering apparatus, comprising:

[0010] The acquisition module is used to retrieve the object properties of the object to be rendered.

[0011] The prediction module is used to predict smoothing parameters reflecting the material of the object to be rendered based on the 3D image of the object to be rendered corresponding to the object having the object attributes.

[0012] The rendering module is used to render the 3D image to be colored according to the smoothing parameters.

[0013] Thirdly, embodiments of the present invention provide an electronic device, including a processor and a memory, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the image rendering method described in the first aspect. The electronic device may also include a communication interface for communicating with other devices or communication networks.

[0014] Fourthly, embodiments of the present invention provide a non-transitory machine-readable storage medium storing executable code, which, when executed by a processor of an electronic device, enables the processor to at least implement the image rendering method as described in the first aspect.

[0015] Fifthly, embodiments of the present invention provide an image rendering method, comprising:

[0016] Retrieve the object properties of the object to be rendered in the live stream page;

[0017] Based on the 3D image to be shaded corresponding to the object to be rendered, which has the object attributes, predict the smoothing parameters that reflect the material of the object to be rendered.

[0018] Render the 3D image to be colored according to the smoothing parameters;

[0019] The live stream page displays the shading 3D image of the object to be rendered.

[0020] Sixthly, embodiments of the present invention provide an image rendering apparatus, comprising:

[0021] The acquisition module is used to retrieve the object properties of the object to be rendered in the live streaming page.

[0022] The prediction module is used to predict smoothing parameters that reflect the material of the object to be rendered based on the 3D image to be shaded corresponding to the object to be rendered, which has the object attributes.

[0023] The rendering module is used to render the 3D image to be colored according to the smoothing parameters;

[0024] The display module is used to display the shading 3D image corresponding to the object to be rendered on the live streaming page.

[0025] In a seventh aspect, embodiments of the present invention provide an electronic device, including a processor and a memory, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the image rendering method in the fifth aspect described above. The electronic device may also include a communication interface for communicating with other devices or communication networks.

[0026] Eighthly, embodiments of the present invention provide a non-transitory machine-readable storage medium storing executable code, which, when executed by a processor of an electronic device, enables the processor to at least implement the image rendering method as described in the fifth aspect.

[0027] Ninthly, embodiments of the present invention provide an image rendering method, comprising:

[0028] Obtain the object properties of the target human body part of the user;

[0029] Based on the three-dimensional image to be colored corresponding to the target human body part having the object attributes, predict the smoothing parameters reflecting the material of the target human body part;

[0030] Render the 3D image to be colored according to the smoothing parameters;

[0031] Display the colored 3D image corresponding to the target human body part.

[0032] In a tenth aspect, embodiments of the present invention provide an image rendering apparatus, comprising:

[0033] The acquisition module is used to acquire the object properties of the user's target human body parts;

[0034] The prediction module is used to predict smoothing parameters reflecting the material of the target human body part based on the three-dimensional image to be colored corresponding to the target human body part having the object attributes.

[0035] The rendering module is used to render the 3D image to be colored according to the smoothing parameters;

[0036] The display module is used to display the colored 3D image corresponding to the target human body part.

[0037] Eleventhly, embodiments of the present invention provide an electronic device, including a processor and a memory, wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the image rendering method of the ninth aspect above. The electronic device may further include a communication interface for communicating with other devices or communication networks.

[0038] In a twelfth aspect, embodiments of the present invention provide a non-transitory machine-readable storage medium storing executable code, which, when executed by a processor of an electronic device, enables the processor to at least implement the image rendering method as described in the ninth aspect.

[0039] In a thirteenth aspect, embodiments of the present invention provide an image rendering method, comprising:

[0040] Retrieve the object properties of the object to be rendered in the road image;

[0041] Based on the 3D image to be shaded corresponding to the object to be rendered, which has the object attributes, predict the smoothing parameters that reflect the material of the object to be rendered.

[0042] Render the 3D image to be colored according to the smoothing parameters;

[0043] A 3D map is generated based on the shaded 3D image corresponding to the object to be rendered.

[0044] In a fourteenth aspect, embodiments of the present invention provide an image rendering apparatus, comprising:

[0045] The acquisition module is used to obtain the object properties of the objects to be rendered in the road image;

[0046] The prediction module is used to predict smoothing parameters that reflect the material of the object to be rendered based on the 3D image to be shaded corresponding to the object to be rendered, which has the object attributes.

[0047] The rendering module is used to render the 3D image to be colored according to the smoothing parameters;

[0048] The display module is used to generate a 3D map based on the shaded 3D image corresponding to the object to be rendered.

[0049] In a fifteenth aspect, embodiments of the present invention provide an electronic device including a processor and a memory, the memory being used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the image rendering method of the ninth aspect described above. The electronic device may further include a communication interface for communicating with other devices or communication networks.

[0050] In a sixteenth aspect, embodiments of the present invention provide a non-transitory machine-readable storage medium storing executable code, which, when executed by a processor of an electronic device, enables the processor to at least implement the image rendering method as described in the ninth aspect.

[0051] In a seventeenth aspect, embodiments of the present invention provide a model training method, comprising:

[0052] Obtain a sample image containing the sample object, wherein the sample image is a two-dimensional image;

[0053] Based on the image features of the sample image, generate a corresponding 3D image to be colored;

[0054] Generate a two-dimensional image to be colored based on the three-dimensional image to be colored corresponding to the sample image;

[0055] The orientation information of the pixels in the two-dimensional image to be colored is input into the first prediction model, so that the first prediction model outputs smoothing parameters that reflect the material of the sample object.

[0056] The two-dimensional image to be colored is rendered according to the smoothing parameters to obtain the colored two-dimensional image;

[0057] The model parameters of the first prediction model are adjusted based on the color difference between the sample image and the colored two-dimensional image.

[0058] In an eighteenth aspect, embodiments of the present invention provide a model training apparatus, comprising:

[0059] The acquisition module is used to acquire a sample image containing the sample object, wherein the sample image is a two-dimensional image;

[0060] The generation module is used to generate a three-dimensional image to be colored corresponding to the sample image based on the image features of the sample image, and to generate a two-dimensional image to be colored corresponding to the sample image based on the three-dimensional image to be colored corresponding to the sample image.

[0061] The input module is used to input the orientation information of the pixels in the two-dimensional image to be colored into the first prediction model, so that the first prediction model outputs smoothing parameters that reflect the material of the sample object.

[0062] The rendering module is used to render the two-dimensional image to be colored according to the smoothing parameters to obtain the colored two-dimensional image;

[0063] The adjustment module is used to adjust the model parameters of the first prediction model based on the color difference between the sample image and the colored two-dimensional image.

[0064] In a nineteenth aspect, embodiments of the present invention provide an electronic device including a processor and a memory, the memory being used to store one or more computer instructions, wherein the one or more computer instructions, when executed by the processor, implement the model training method of the seventeenth aspect described above. The electronic device may further include a communication interface for communicating with other devices or communication networks.

[0065] In a twentieth aspect, embodiments of the present invention provide a non-transitory machine-readable storage medium storing executable code, which, when executed by a processor of an electronic device, enables the processor to at least implement the model training method as described in the seventeenth aspect.

[0066] The image rendering method provided in this invention obtains the object attributes of the object to be rendered, and then predicts smoothing parameters reflecting the material of the object to be rendered based on the corresponding 3D image to be colored, which contains these object attributes. The object to be rendered is displayed in a three-dimensional style in the 3D image to be colored, but without color. Finally, the object is colored using these smoothing parameters, and the coloring result is displayed on the screen, thus completing the rendering of the object.

[0067] When light shines on objects of different materials, the degree of refraction and reflection of light on the object's surface varies, resulting in color deviations. In the above approach, the smoothing parameters used in the shading process are predicted and specifically tailored to the material of the object being rendered. Therefore, material factors can be more accurately incorporated into the rendering process, thereby improving the realism of the rendering. Attached Figure Description

[0068] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0069] Figure 1 A flowchart of an image rendering method provided in an embodiment of the present invention;

[0070] Figure 2 for Figure 1 A flowchart illustrating the specific implementation of step 104 in the image rendering method provided in the illustrated embodiment;

[0071] Figure 3 for Figure 2 A flowchart illustrating the specific implementation of step 201 in the image rendering method provided in the illustrated embodiment;

[0072] Figure 4 A flowchart of the training process of the second prediction model provided in an embodiment of the present invention;

[0073] Figure 5 for Figure 4 A schematic diagram of the training process of the second prediction model provided in the illustrated embodiment;

[0074] Figure 6 A flowchart of a model training method provided in an embodiment of the present invention;

[0075] Figure 7 A flowchart of another image rendering method provided in an embodiment of the present invention;

[0076] Figure 8 A flowchart illustrating yet another image rendering method provided in an embodiment of the present invention;

[0077] Figure 9 A flowchart illustrating yet another image rendering method provided in an embodiment of the present invention;

[0078] Figure 10 This invention provides a schematic diagram illustrating the application of an image rendering method in an entertainment scenario.

[0079] Figure 11 This is a schematic diagram of the structure of an image rendering device provided in an embodiment of the present invention;

[0080] Figure 12 To and Figure 11 A schematic diagram of the electronic device corresponding to the image rendering apparatus provided in the illustrated embodiment;

[0081] Figure 13 This is a schematic diagram of another image rendering apparatus provided in an embodiment of the present invention;

[0082] Figure 14 To and Figure 13 A schematic diagram of the electronic device corresponding to the image rendering apparatus provided in the illustrated embodiment;

[0083] Figure 15 This is a schematic diagram of the structure of another image rendering device provided in an embodiment of the present invention;

[0084] Figure 16 To and Figure 15 A schematic diagram of the electronic device corresponding to the image rendering apparatus provided in the illustrated embodiment;

[0085] Figure 17 This is a schematic diagram of the structure of another image rendering device provided in an embodiment of the present invention;

[0086] Figure 18 To and Figure 17 A schematic diagram of the electronic device corresponding to the image rendering apparatus provided in the illustrated embodiment;

[0087] Figure 19 This is a schematic diagram of the structure of another model training device provided in an embodiment of the present invention;

[0088] Figure 20 To and Figure 19 The illustrated embodiment provides a schematic diagram of the electronic device corresponding to the model training apparatus. Detailed Implementation

[0089] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0090] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention and the appended claims are also intended to include the plural forms, unless the context clearly indicates otherwise. “Multiple” generally includes at least two, but does not exclude the inclusion of at least one.

[0091] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0092] Depending on the context, the words “if” or “suppose” as used here can be interpreted as “when” or “in response to determination” or “in response to identification.” Similarly, depending on the context, the phrases “if determination” or “if identification (of the condition or event of the statement)” can be interpreted as “when determination” or “in response to determination” or “when identification (of the condition or event of the statement)” or “in response to identification (of the condition or event of the statement).”

[0093] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a product or system comprising a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a product or system. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the product or system that includes said element.

[0094] The image rendering method provided in the embodiments of the present invention will be described below. Before proceeding, the practical significance of this image rendering needs to be explained by example:

[0095] In practical applications, we often need to create 3D models of objects in real-world scenes and then render the corresponding 3D models onto the screen. Based on this requirement, the image rendering methods provided in the various embodiments of this invention can be used.

[0096] For example, in the entertainment industry, users can generate their own virtual avatars through 3D modeling. Similarly, in the live streaming industry, streamers can generate their own virtual avatars through 3D modeling, and the products for sale and the live stream background can all be virtual 3D objects. In the medical field, human organs can be 3D modeled and rendered on the screen to facilitate further use of 3D organ models for academic discussions, disease analysis, and auxiliary diagnosis. In the navigation field, a certain area can be 3D modeled and rendered to obtain a 3D map for navigation, thus realizing navigation functionality. Furthermore, as mentioned in the background technology section regarding animation and game production, objects in real-world scenes can be directly modeled and rendered to complete the construction of animation or game scenes.

[0097] Of course, the above scenarios are just illustrations, and this invention does not limit the application scenarios of the image rendering method.

[0098] Based on the above description, some embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Where there is no conflict between the embodiments, the following embodiments and features can be combined with each other. Furthermore, the timing of the steps in the following method embodiments is merely an example and not a strict limitation.

[0099] Figure 1 This is a flowchart illustrating an image rendering method provided in an embodiment of the present invention. This image rendering method can be executed by a rendering device. It is understood that the rendering device can be implemented as software, or a combination of software and hardware. Figure 1 As shown, the method includes the following steps:

[0100] S101, Get the object properties of the object to be rendered.

[0101] The object to be rendered can be any object in the real-world scene, and its properties can include the object's shape, texture, and so on. For example, when the object to be rendered is a human face, its properties can include the face shape, the shape of the facial features, the texture of the skin, and even the facial expression.

[0102] One alternative method for obtaining object attributes is to use a specific scanning device to scan the object to be rendered in order to obtain the object attributes. With this method, the image rendering method provided in the embodiments of this invention can directly generate a three-dimensional image of an object in space.

[0103] Alternatively, the object to be rendered can be photographed first to obtain a corresponding two-dimensional image. Then, image features are extracted from the two-dimensional image, including the object attributes of the object to be rendered. The extraction of image features can be achieved using a prediction model, such as a Markov Random Field (MRF) model, a fractal model, etc. With this acquisition method, the image rendering method provided in the embodiments of this invention can directly generate a three-dimensional image corresponding to a two-dimensional image of an object in space.

[0104] Of course, the above methods for obtaining object properties are just examples, and this invention does not limit the methods for obtaining object properties.

[0105] S102, based on the 3D image to be shaded corresponding to the object to be rendered with object attributes, predict the smoothing parameters that reflect the material of the object to be rendered.

[0106] Based on the 3D image to be shaded corresponding to the object to be rendered, which has object attributes, predict the smoothing parameters that reflect the material of the object to be rendered.

[0107] S103, render the 3D image to be colored according to the smoothing parameters.

[0108] After determining the object attributes of the object to be rendered, a 3D image corresponding to the object can be generated based on these attributes. Then, a smoothing parameter reflecting the material of the object to be rendered is predicted based on the generated 3D image. Finally, this smoothing parameter is used to color the generated 3D image. The object to be rendered is displayed in a 3D stereoscopic style in the generated 3D image; however, the stereoscopic object is not colored. Therefore, the 3D image generated based on the object attributes can be called a 3D image to be colored. Furthermore, the rendering method provided in the various embodiments of this invention can be understood as the process of coloring a 3D image to be colored and displaying the colored 3D image on the screen.

[0109] The generation of the 3D image to be shaded and the prediction of smoothing parameters are explained below. For the generation of the 3D image to be shaded, optionally, template images corresponding to different types of objects to be rendered can be obtained in advance. These template images are also 3D images, meaning the objects contained within them are displayed in a stereoscopic style. In this case, the template image corresponding to the object to be rendered can be determined first based on its type. Then, the object attributes of the object to be rendered can be used as adjustment parameters to adjust the template image corresponding to the object, thereby obtaining the 3D image to be shaded corresponding to the object to be rendered.

[0110] For example, when the object to be rendered is a face, its corresponding template image is also a 3D image containing a face. The face is displayed in a 3D style in the template image, except that the face in the template image and the face to be rendered are two different faces with different face shapes, facial features, and expressions. In this case, the object attributes of the object to be rendered, such as the face shape, expression, shape of facial features, skin texture, etc., can be used as parameters to adjust the template image. This makes the face in the adjusted template image have the same object attributes as the face to be rendered, and they are the same face. This adjusted template image can also be considered as the 3D image to be colored corresponding to the object to be rendered.

[0111] As is easily understood, the smallest unit constituting a two-dimensional image is a pixel. Similarly, the smallest unit constituting a three-dimensional image can be a polygon, meaning that an object in a three-dimensional image is composed of multiple polygons. Optionally, the shape of a polygon can be a polygon, with triangular polygons being the most common.

[0112] For predicting the smoothing parameters, optionally, based on the generated 3D image to be shaded, the orientation information of each facet in the 3D image to be shaded can be obtained, where the orientation information of the facet can be the normal direction of the facet. On the other hand, the coordinates, texture features, and orientation information of the vertices on each facet can also be obtained. The coordinates of the vertices correspond to the world coordinate system and include a depth value. The orientation information of the vertices is also the normal information of the vertices, and the orientation information of the facet can be determined based on the orientation information of the vertices in the facet.

[0113] At this point, the smoothing parameters reflecting the material of the object to be rendered can be predicted based on the orientation information of each facet. Optionally, the prediction of smoothing parameters is usually achieved using a prediction model. In practice, the prediction model can specifically be a neural network model composed of at least one fully connected layer. The normal information of each facet surrounding the object in the 3D image is input into this prediction model, so that the prediction model outputs smoothing parameters reflecting the material of the object. The training process of this prediction model can be found in [reference needed]. Figure 4 The description in the illustrated embodiment.

[0114] The principle by which normal information reflects material properties can be understood as follows: the normal information of a surface element represents its degree of unevenness. The smoother the material, the less obvious the unevenness, meaning the changes in the normal information of the surface elements are less noticeable. This unevenness can create the textures of different materials, such as the texture of skin or fabric. Furthermore, the process of predicting smoothing parameters using a prediction model can be described as follows: inputting the normal information in the form of a two-dimensional vector into the prediction model, and then calculating this vector along with the weight values ​​of nodes in at least one fully connected layer in the prediction model, thereby predicting the smoothing parameters.

[0115] After predicting the smoothing parameters, these parameters can be further used to color the 3D image to be colored. Specifically, the color value of the 3D points contained in each facet of the 3D image to be colored is first calculated based on the smoothing parameters. Then, the 3D image to be colored is colored according to these color values; that is, the pixel points on the screen that correspond to the 3D points in the image to be colored are colored using these color values. The colored 3D image will then be displayed on the screen. The above process of displaying after coloring can be considered as the process of rendering the 3D image to be colored.

[0116] Furthermore, since the smoothing parameter, which reflects the material of the object being rendered, is a variable predicted value—meaning it changes depending on the object being rendered—it effectively achieves differentiability in the shading process. Moreover, using the smoothing parameter during shading allows for more accurate calculation of the color value to be rendered for each pixel on the screen, ensuring that the color value reflects the material of the object being rendered, thus improving rendering realism.

[0117] For example, under the same environment, when light shines on objects of different materials, it undergoes varying degrees of refraction and reflection on the object's surface, resulting in color variations. For objects like glass and ceramics, light spots often appear on their surfaces after being illuminated. However, by incorporating smoothing parameters into the shading process, the effect of light on the surface of the object being rendered, as well as any light spots that may appear due to illumination, will be rendered, thus achieving realistic rendering.

[0118] In this embodiment, the object attributes of the object to be rendered are obtained, and then a corresponding 3D image to be shaded is generated based on these attributes. Furthermore, since the 3D image of the object to be rendered is composed of multiple facets, smoothing parameters can be determined based on the orientation information of each facet. Finally, these smoothing parameters are used for shading, thus completing the rendering of the object. When using the above scheme, since the smoothing parameters used in the shading process are specifically designed for the material of the object to be rendered, the material can be effectively incorporated into the rendering process, thereby improving the realism of the rendering.

[0119] Additionally, after predicting the smoothing parameters in step 102, in practical applications, one optional method is to follow... Figure 1 The method provided in the illustrated embodiment renders the 3D image to be colored directly based on the predicted smoothing parameters.

[0120] Alternatively, the predicted smoothing parameters can be displayed on the user interface, allowing the user to determine, based on experience, whether the current smoothing parameters are appropriate and to modify them. The rendering device can then execute step 103 based on the user-modified smoothing parameters.

[0121] Alternatively, the rendering device can first determine the required rendering scene for the object to be rendered, and further determine the pre-selected parameter range corresponding to this rendering scene. If the predicted smoothing parameters exceed the preset parameter range, the predicted smoothing parameters are displayed for the user to modify, and then step 103 is executed based on the user-modified smoothing parameters. For example, when the object to be rendered is a face, the rendering scene could be an entertainment or live streaming scene. When the object to be rendered is a vehicle or building, the rendering scene could be a traffic scene.

[0122] In practical applications, to further ensure the realism of the rendering, in addition to the smoothing parameters, other rendering parameters can be used simultaneously to complete the coloring of the 3D image to be colored. Optionally, such as... Figure 2 As shown, one optional implementation of step 103, namely the specific process of coloring the 3D image to be colored, may include the following steps:

[0123] S201, through coordinate transformation, determine the correspondence between the target 3D point in the 3D image to be colored and the target pixel in the screen. The target 3D point is any 3D point in the 3D image to be colored.

[0124] Each facet in the 3D image to be colored contains multiple 3D points, including vertices. Obtaining the 3D image also allows us to obtain the coordinates of the vertices on each facet in the world coordinate system. Then, we can interpolate the vertex coordinates to obtain the coordinates of the non-vertices in the 3D image in the world coordinate system. Both vertices and non-vertices within each facet of the 3D image can be considered 3D points.

[0125] Next, the coordinate values ​​of each 3D point are transformed, that is, they are transformed from the world coordinate system to the screen coordinate system, thereby establishing the correspondence between the 3D points in the 3D image to be colored and the pixels in the screen, which completes the rasterization process.

[0126] Optionally, the coordinate transformation process can be specifically as follows: transforming the coordinate values ​​from the world coordinate system to the camera coordinate system, then from the camera coordinate system to the Normalized Device Coordinates (NDC) coordinate system, and finally from the NDC coordinate system to the screen coordinate system. Each step of the coordinate transformation can be achieved using a preset transformation matrix. The coordinate transformation described above is a standard process in the field of image rendering, and the specific process will not be elaborated here.

[0127] It should be noted that the correspondence between the 3D points and pixels established above can also reflect the correspondence between each facet in the 3D image to be colored and the pixel area on the screen. A softening coefficient can also be obtained during the process of establishing the correspondence. Optionally, this softening coefficient can be used to soften the pixel area corresponding to each facet, thereby reducing the blockiness of object edges in the image, making the edges of the rendered object smooth, and improving the quality of image rendering.

[0128] S202, determine the rendering parameters of the target 3D point based on the smoothing parameters.

[0129] The rendering parameters for the target 3D point may include texture parameters, diffuse reflection parameters, and specular reflection parameters. For texture parameters, the determination method may be as follows:

[0130] Obtaining the 3D image to be colored also yields the texture features of each vertex in the 3D image, which can be specifically represented as texture parameters. Similar to step 201, the texture parameters of the vertices can then be interpolated to obtain the texture parameters of each 3D point in the 3D image to be colored, i.e., the texture parameters of the target 3D point in the 3D image to be colored. The target 3D point can be any 3D point in the 3D image to be colored.

[0131] The diffuse reflection parameters and specular reflection parameters can be determined in the following ways:

[0132] The diffuse reflection parameters and specular reflection parameters of the target 3D point can be calculated based on the position of the light source in the 2D image shooting environment and the orientation information of the surface element to which the target 3D point belongs, specifically the normal information, and with the help of the bidirectional reflection distribution function (BRDF).

[0133] Specifically, the diffuse parameter diffuse can be expressed as: diffuse = DisneyDiffuse(n, I, h, s).

[0134] The specular parameter can be expressed as:

[0135] specular=π*SmithGGXVisibilityTerm(n,I,v,s)*GGXTerm(n,h,s).

[0136] Here, DisneyDiffuse is the preset diffuse reflection calculation function, and SmithGGXVisibilityTerm and GGXTerm are the preset specular reflection calculation functions. It can be considered a type of BRDF function. n represents the orientation information of the facet containing the target 3D point, I represents the position of the light source, h is the half-angle tensor, s is the smoothing parameter, and v is the shooting angle of the 2D image. Specifically, the orientation information n of the facet containing the target 3D point is obtained simultaneously with the 3D image to be colored. Furthermore, the position I of the light source, the half-angle tensor h, and the shooting angle v are all preset known values. The half-angle tensor h is the result of averaging and normalizing the shooting angle v and the position I of the light source.

[0137] In the above calculation method, the calculation of diffuse reflection parameters and specular reflection parameters requires the use of smoothing parameters that reflect the material of the object to be rendered. This allows both reflection parameters to indirectly reflect the material of the object to be rendered.

[0138] S203 determines the color value of the target 3D point based on the rendering parameters.

[0139] Next, the color values ​​of the target 3D points in the 3D image to be colored are further determined based on the rendering parameters. Since the method for determining the color value of each 3D point is the same, the explanation will focus on the target 3D point.

[0140] Based on the two reflection parameters and texture parameters of the target 3D point, the color value of the target 3D point can optionally be calculated according to the following formula: Color = (ambient + diffuse) * texels + specular

[0141] Here, diffuse is the diffuse reflection parameter, specular is the specular reflection parameter, texels is the texture parameter, and ambient is the preset ambient light parameter. Generally speaking, ambient can be called the self-illumination term, diffuse can be called the diffuse reflection term, and specular can be called the specular reflection term.

[0142] The color value of each three-dimensional point in the three-dimensional image to be colored can be calculated using the method described above.

[0143] S204, based on the correspondence, render the target pixels on the screen using the color values ​​of the target 3D points.

[0144] Finally, based on the correspondence between the three-dimensional points and pixels obtained in step 201, the color value of each three-dimensional point can be rendered onto the corresponding pixel position on the screen, thereby completing the image rendering, i.e., the coloring process of the three-dimensional image to be colored.

[0145] In this embodiment, the correspondence between 3D points in the 3D image to be colored and pixels on the screen is first established. Then, color values ​​are calculated for image rendering using predicted smoothing parameters and the orientation information of facets. Similar to the above embodiment, smoothing parameters that reflect the material of the object to be rendered are used when calculating color values ​​to ensure the realism of the rendering. Furthermore, the orientation information of facets is also used when calculating color values. Under the same lighting conditions, facets at different locations on the object to be rendered often exhibit color differences, and the orientation information of facets reflects their position on the object. Therefore, introducing orientation information into the color value calculation process can effectively reflect the color differences of 3D points on different facets, thereby ensuring the realism of the rendering.

[0146] In addition, the above Figure 2 In step 201 of the illustrated embodiment, interpolation can be used to establish the correspondence between 3D points and pixels. Optionally, step 201 can be implemented in the following way: Figure 3 As shown, the following steps may be included:

[0147] S301, Based on the depth values ​​of the vertices of the face elements in the three-dimensional image to be colored, determine a preset number of target face elements whose depth values ​​meet the requirements.

[0148] While obtaining the 3D image to be rendered, we also obtain the depth values ​​of the facet vertices that enclose the object to be rendered in this image, as well as the orientation information and texture features (i.e., texture parameters) of the facet vertices. Next, we determine a preset number of target facets whose depth values ​​meet the requirements.

[0149] Assuming the preset number is M, first sort the vertices in ascending order according to their depth values, determine the M vertices corresponding to the M smallest depth values, and then determine the face elements to which these M vertices belong. If the M vertices belong to exactly M face elements, then these M face elements are the target face elements. Where M > 1.

[0150] If M vertices belong to N face elements, where M > N, then we can continue to determine the depth value of one of the remaining depth values ​​according to the sorting, and determine the face element to which this minimum depth value belongs as the target face element, and so on, until M face elements are determined.

[0151] S302, determine the centroid coordinates of the target surface element based on the coordinates of its vertex values.

[0152] S303, based on the centroid coordinates, interpolate the vertex coordinates to obtain the coordinates of each 3D point in the 3D image to be colored.

[0153] The centroid coordinates of each target surface element are determined based on the coordinates of its vertices, resulting in M ​​centroid coordinates. Then, based on these M centroid coordinates, the 3D points of the vertices in the 3D image to be shaded are interpolated to obtain the coordinates of the non-vertices.

[0154] It should be noted that when M=1, it indicates that only one facet corresponding to the minimum depth value is considered during interpolation, which is obviously incomplete. Therefore, in this embodiment, M is set to be greater than 1, meaning that multiple facets are considered simultaneously during interpolation. By setting the value of M, the above can be achieved. Figure 2 The softening effect in the illustrated embodiment.

[0155] Using barycentric coordinates, various information can be interpolated from the vertices contained in a face element. Besides the coordinates mentioned above, texture features (i.e., texture parameters) can also be interpolated. Similarly, M barycentric coordinates can be used to interpolate the texture parameters of the vertices in a face element to obtain the texture parameters of the non-vertices in the face element, which is equivalent to obtaining the texture parameters of the target 3D points in the 3D image to be colored. This achieves... Figure 2 Step 202 is shown in the implementation diagram.

[0156] S304 determines the correspondence between each 3D point in the 3D image to be colored and the pixel points on the screen through coordinate transformation.

[0157] The execution process of step 403 can be found in the following example: Figure 2 The relevant descriptions in the illustrated embodiments will not be repeated here.

[0158] In this embodiment, the coordinates and texture parameters of vertices in the 3D image to be shaded are interpolated using the centroids of multiple facets with the smallest depth values, thus obtaining the coordinates and texture parameters of non-vertices in the 3D image to be shaded. By using the centroid coordinates of multiple facets, the interpolation results are more accurate, thereby further ensuring the realism of the final image rendering.

[0159] In addition, Figure 1In the illustrated embodiment, it has been mentioned that image features can be extracted from a 2D image containing the object to be rendered using a prediction model. It has also been mentioned that smoothing parameters reflecting the material of the object to be rendered can be predicted based on orientation information. Optionally, similar to the feature extraction process, the prediction process of smoothing parameters can also be implemented using a neural network-based prediction model. In practical applications, the above two processes can be implemented by independent or non-independent prediction models. When the two prediction models are implemented by two independent models, for clarity, the prediction model used to extract image features can be called the first prediction model, and the prediction model that outputs the smoothing parameters can be called the second prediction model. Furthermore, the gradient is transferred from the first prediction model to the second prediction model, thereby ensuring the realism of the rendering.

[0160] For the training process of the second prediction model, optionally, as follows: Figure 4 As shown, the following steps may be included:

[0161] S401, Obtain a sample image containing the sample object. The sample image is a two-dimensional image.

[0162] S402, Generate the 3D image to be colored corresponding to the sample image based on the image features of the sample image.

[0163] Collect two-dimensional images containing sample objects, i.e., sample images. The sample objects are the same as the objects to be rendered, and can be any object in a real-world scene, such as the human face mentioned in the above embodiments.

[0164] Next, image features are extracted from the sample image. Continuing with the example of a face, these image features reflect the face shape, expression, facial features, and skin texture, etc. This image feature extraction can be achieved through a first prediction model. That is, the sample image is input into the first prediction model, and the first prediction model outputs the image features.

[0165] Then, based on the image features, the corresponding 3D image to be colored for the sample object can be generated. The generation process can utilize a template image; the specific details are as follows... Figure 1 The description in step 102 of the illustrated embodiment is similar and will not be repeated here.

[0166] S403: Generate a two-dimensional image to be colored based on the three-dimensional image to be colored corresponding to the sample image.

[0167] In the 3D image to be shaded corresponding to the sample object, the coordinates of each 3D point in the world coordinate system are known. By transforming the coordinates from the world coordinate system to the screen coordinate system, the position of the corresponding pixel on the screen can be calculated. In other words, the correspondence between the 3D points and pixels in the 3D image to be shaded can be established. Based on this correspondence, the 2D image to be shaded corresponding to the sample object can be generated, and the orientation information of each pixel in the 2D image to be shaded, i.e., the normal information of the pixel, can also be obtained.

[0168] S404, input the orientation information of the pixels in the two-dimensional image to be colored into the second prediction model, so that the second prediction model outputs the smoothing parameters.

[0169] S405: Render the two-dimensional image to be colored according to the smoothing parameters to obtain the colored two-dimensional image.

[0170] Next, the orientation information of each pixel can be input into the second prediction model, which then outputs smoothing parameters that reflect the material of the sample object. These smoothing parameters are then used to render the corresponding 2D image of the sample object to obtain the shaded 2D image. Optionally, as... Figure 1 As shown in the embodiment, the second prediction model can be a neural network model consisting of at least one fully connected layer.

[0171] The coloring process for a 2D image is similar to the calculation process for coloring a 3D image, except that the coloring process uses the orientation information of pixels in the 2D image and the orientation information of 3D points in the 3D image, respectively. For details, please refer to [link to relevant documentation]. Figure 2 , Figure 3 The relevant descriptions in the illustrated embodiments.

[0172] For any content not described in detail in steps 401 to 405 above, please refer to the above embodiments, except that the coloring object is changed from a three-dimensional image to be colored containing the object to be rendered to a two-dimensional image to be colored containing the sample object.

[0173] S406, Adjust the model parameters of the second prediction model based on the color difference between the sample image and the colored two-dimensional image.

[0174] Next, the loss value is calculated based on the difference in color values ​​of each pixel between the original sample image and the colored 2D image. The model parameters of the second prediction model are then adjusted based on this loss value, thereby completing the optimization of the second prediction model.

[0175] Optionally, while optimizing the second prediction model, the first prediction model can also be optimized at the same time. That is, the model parameters of the first prediction model can be adjusted according to the calculated loss value, which means that the two models can be jointly trained.

[0176] Furthermore, the training process provided in this embodiment can also be combined with... Figure 5 understand.

[0177] In this embodiment, a three-dimensional image to be colored corresponding to the sample image is first generated. Then, a two-dimensional image to be colored and a colored two-dimensional image are generated sequentially based on this three-dimensional image. The color difference of each pixel in the colored two-dimensional image and the original and sample images is further calculated to optimize the model, thereby ensuring that the second prediction model can output accurate smoothing parameters, thus further ensuring the accuracy of image rendering.

[0178] Figure 4 The model training method shown is explained during the introduction of image rendering. The smoothing parameters that the model outputs, which reflect the material of the object, may also be applied to other scenarios. Therefore, the training process of the model that can output smoothing parameters can be introduced separately without the premise of image rendering.

[0179] Figure 6 This is a flowchart illustrating a model training method provided in an embodiment of the present invention. This model training method can be executed by a training device. It is understood that the training device can be implemented as software, or a combination of software and hardware. Figure 6 As shown, the method includes the following steps:

[0180] S501, Obtain a sample image containing the sample object. The sample image is a two-dimensional image.

[0181] S502, Generate the 3D image to be colored corresponding to the sample image based on the image features of the sample image.

[0182] Obtain a sample image containing the sample object. The sample image is a two-dimensional image, and the sample object in the image can be any object in the real scene. Then, extract the image features of the sample image to generate a three-dimensional image to be colored corresponding to the sample object.

[0183] Image feature extraction can be achieved through a prediction model. For clarity, in this embodiment, the model that outputs smoothing parameters is referred to as the first prediction model, and the model that performs image feature extraction is referred to as the second prediction model.

[0184] After obtaining the image features output by the second prediction model, optionally, these image features and a preset template image can be used to generate a 3D image to be colored corresponding to the sample image, wherein the template image corresponds to the type of the sample object. For the specific generation process, please refer to [link to documentation]. Figure 1 The relevant description in step 102 of the illustrated embodiment.

[0185] S503, Generate a two-dimensional image to be colored based on the three-dimensional image to be colored corresponding to the sample image.

[0186] The execution process of step 503 can be found in the following example: Figure 4 The relevant descriptions in the illustrated embodiments will not be repeated here.

[0187] S504, the orientation information of the pixels in the two-dimensional image to be colored is input into the first prediction model, so that the first prediction model outputs smoothing parameters that reflect the material of the sample object.

[0188] S505 renders the two-dimensional image to be colored according to the smoothing parameters to obtain the colored two-dimensional image.

[0189] S506, Adjust the model parameters of the first prediction model based on the color difference between the sample image and the colored two-dimensional image.

[0190] Next, the orientation information of the pixels in the two-dimensional image to be colored is input into the first prediction model, so that the first prediction model outputs a smoothing parameter, which can reflect the material of the sample object in the sample image.

[0191] The 2D image to be colored is then rendered using smoothing parameters to obtain the colored 2D image. Finally, the color differences between pixels in the original sample image and the colored 2D image are used to adjust the first prediction model.

[0192] Optionally, the second prediction model can also be adjusted simultaneously based on the color differences mentioned above, that is, the joint optimization of the two prediction models can be achieved.

[0193] It should be noted that the calculation process performed by the first prediction model in this embodiment is different from... Figures 1-4 The second prediction model in the illustrated embodiment is the same, and the calculation process performed by the second prediction model in this embodiment is the same. Figures 1-4 The first prediction model in the illustrated embodiment is the same.

[0194] Additionally, for parts not described in detail in this embodiment, please refer to the description of... Figures 4-5 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figures 4-5 The descriptions in the illustrated embodiments will not be repeated here.

[0195] As mentioned in the above description, the image rendering provided in the various embodiments can be applied to different scenarios, such as live streaming, entertainment, or transportation scenarios. Therefore, in a live streaming scenario... Figure 7 A flowchart illustrating yet another image rendering method provided in an embodiment of the present invention. For example... Figure 7 As shown, the method includes the following steps:

[0196] S601, retrieve the object properties of the object to be rendered in the live streaming page.

[0197] In a live streaming scenario, all content displayed on the live streaming page can be considered as objects to be rendered. This includes the items being traded (i.e., the products for sale), the live streaming background, and even the streamer. The live streaming background can also vary depending on the type of item being traded. For example, if the item is snacks, the background could be a cozy living room setting; if the item is a specialty product from a particular region, the background could be a famous building or scene from that region.

[0198] To obtain the object attributes of the object to be rendered, alternatively, they can be obtained by extracting the image features of their respective two-dimensional images, or they can be obtained by scanning with other scanning devices.

[0199] S602, based on the 3D image to be shaded corresponding to the object to be rendered with object attributes, predicts smoothing parameters that reflect the material of the object to be rendered.

[0200] After obtaining the object's attributes, a 3D image to be shaded corresponding to the object to be rendered can be generated based on these attributes. Then, smoothing parameters reflecting the material can be predicted based on this 3D image. Optionally, the orientation information of the facets surrounding the object to be rendered in the 3D image can be input into the prediction model, so that the prediction model outputs the smoothing parameters.

[0201] For the generation of the 3D image to be colored, optionally, the 3D image to be colored corresponding to the object to be rendered can be generated based on the image features of the object to be rendered and the template image, wherein the template image corresponds to the type of the object to be rendered, and the image features can be regarded as object attributes.

[0202] S603 renders the 3D image to be colored based on the smoothing parameters.

[0203] S604 displays the shaded 3D image of the object to be rendered on the live streaming page.

[0204] Finally, the objects to be rendered on the live stream page are colored according to their respective smoothing parameters to obtain their respective colored 3D images, which are then displayed on the screen of the terminal device.

[0205] For example, the objects to be rendered on the live stream page could include specialty product B from region A, and the live stream background could include building C from region A. After rendering to the screen, both specialty product B and building C will appear as realistic 3D models on the live stream page. Additionally, the streamer displayed on the live stream page could be a 3D virtual avatar of the streamer themselves.

[0206] For any content not described in detail in this embodiment, as well as the technical effects that can be achieved, please refer to the above. Figures 1-5 The relevant descriptions in the illustrated embodiments will not be repeated here.

[0207] Similar to live streaming scenarios, in entertainment scenarios, Figure 8 A flowchart illustrating yet another image rendering method provided in an embodiment of the present invention. For example... Figure 8 As shown, the method includes the following steps:

[0208] S701, obtain the object properties of the target human body part of the user.

[0209] User A can generate their own 3D virtual avatar using an application (APP) installed on their terminal device. Specifically, in response to a user-triggered launch, the APP can optionally access the terminal device's camera to capture a 2D image of a target body part of the user. This target body part can typically be the face, upper body, or the entire user's body, or any part such as the limbs. Then, the image features of the 2D image can be extracted and used as object attributes for the target body part.

[0210] Alternatively, users can use the scanning function of a terminal device with an installed app to obtain object attributes by scanning the target human body parts.

[0211] S702, based on the 3D image of the target human body part with object attributes, predict the smoothing parameters that reflect the material of the target human body part.

[0212] S703 renders the 3D image to be colored based on the smoothing parameters.

[0213] S704 displays the colored 3D image corresponding to the target human body part.

[0214] The generation of the 3D image to be colored corresponding to the target human body part can be based on the template image corresponding to the target human body part and the image features extracted as object attributes as mentioned above. Then, smoothing parameters reflecting the material of the target human body part can be predicted based on the 3D image to be colored.

[0215] The material of the target human body part can be: if the target human body part is the face, then the material is actually the skin. If the target human body part is the upper body, then the material is the material of the clothes the user is wearing.

[0216] Optionally, the smoothing parameters can be predicted using a prediction model. For example, the orientation information, i.e., the normal information, of the facets surrounding the target human body part in the 3D image to be colored can be input into the prediction model, so that the prediction model can output the smoothing parameters.

[0217] Finally, the target human body part can be further colored according to the smoothing parameters in the 3D image to be colored, and the colored 3D image can be displayed on the terminal device.

[0218] For details not described in this embodiment, please refer to [the relevant documentation]. Figure 10 The descriptions in the illustrated embodiments are for illustrative purposes only. Furthermore, any details not described in detail in this embodiment, as well as the achievable technical effects, can be found above. Figures 1-5 The relevant descriptions in the illustrated embodiments will not be repeated here.

[0219] In traffic scenarios, Figure 9 A flowchart illustrating yet another image rendering method provided in an embodiment of the present invention. For example... Figure 9 As shown, the method includes the following steps:

[0220] S801, obtain the object properties of the object to be rendered in the road image.

[0221] The road sections for which 3D maps need to be generated are photographed to obtain road images. Objects in the road images, such as vehicles, trees, and road signs, can be considered as objects to be rendered. Then, image features can be extracted using a model, and these image features can be used as object attributes of the objects to be rendered.

[0222] S802, based on the 3D image to be shaded corresponding to the object to be rendered with object attributes, predict the smoothing parameters that reflect the material of the object to be rendered.

[0223] S803 renders the 3D image to be colored based on the smoothing parameters.

[0224] S804 generates a 3D map based on the shaded 3D image corresponding to the object to be rendered.

[0225] Next, based on the 3D images of the objects to be rendered, which also have object attributes, the smoothing parameters reflecting the material of the objects to be rendered are predicted. The 3D images to be rendered are then colored according to the predicted smoothing parameters, thus completing the rendering of the 3D images. Each object to be rendered in the road image is displayed on the terminal device screen in a realistic 3D style. The colored 3D images of each object to be rendered from multiple 2D images of different road segments can form a 3D map of that road segment.

[0226] For any content not described in detail in this embodiment, as well as the technical effects that can be achieved, please refer to the above. Figures 1-5 The relevant descriptions in the illustrated embodiments will not be repeated here.

[0227] To facilitate understanding, the specific implementation process of the image rendering method provided above will be illustrated with examples in the following different application scenarios.

[0228] (1) In the game production scene, if we need to build a bedroom scene, in order to ensure the realism of each object in the bedroom scene, we can use a real bedroom to model and color it and finally display the coloring result on the screen.

[0229] Based on this requirement, objects such as beds, wardrobes, and desks in a real bedroom can be photographed to obtain their respective two-dimensional images. Then, the image rendering method provided by this invention can be used to generate three-dimensional images of each object, such as beds, wardrobes, and desks, to be colored and displayed on the screen.

[0230] The modeling and shading process will be explained using a wardrobe in a bedroom scene as an example:

[0231] After capturing a 2D image of the wardrobe in the bedroom, it can be input into a first prediction model. The first prediction model outputs image features of the 2D image, which can reflect information such as the shape and texture of the wardrobe. Then, the image features output by the first prediction model can be used as parameters to modify the wardrobe template (i.e.,... Figure 1 The template image in the illustrated embodiment is adjusted to obtain a 3D wardrobe image to be colored. The wardrobe in this image is colorless and displayed in a 3D style, meaning it is composed of multiple triangular facets. The vertices and non-vertices of the triangular facets forming the wardrobe can be collectively referred to as 3D points in the 3D wardrobe image to be colored.

[0232] Next, the normal information of multiple triangular facets in the 3D wardrobe image to be colored is input into the second prediction model, which outputs smoothing parameters reflecting the wardrobe material. Finally, these smoothing parameters are used to color each 3D point in the 3D wardrobe image to be colored, so that a colored, 3D wardrobe is displayed on the screen.

[0233] Since the coloring process is the same for every 3D point, we will take any 3D point, i.e., the target 3D point, as an example to illustrate the coloring process:

[0234] The diffuse and specular reflection parameters of the target 3D point can be calculated based on the normal information of the surface element containing the target 3D point, the position of the light source, and the smoothing parameters output by the second prediction model. Then, these two reflection parameters, along with the texture parameters of the target 3D point, are used to calculate the color value of the target 3D point. In a bedroom, the light source position can be the location of the window. For detailed calculation methods of the two reflection parameters, please refer to [link to relevant documentation]. Figure 2 The description in the illustrated embodiment.

[0235] Simultaneously, since the coordinates and texture parameters of the vertices in the 3D wardrobe image to be colored are known, interpolation can be performed on the coordinates and texture parameters of each vertex in the image to obtain the interpolated coordinates and texture parameters of the non-vertices. Then, through coordinate transformation, the coordinates of the vertices and non-vertices in the world coordinate system are transformed to the screen coordinate system to establish the correspondence between the 3D points in the image to be colored and the pixels on the screen. Finally, based on this correspondence, the color values ​​of the target 3D points are used to render the corresponding pixels, thus achieving the coloring of the target 3D points, and the coloring result will be displayed on the screen.

[0236] The diffuse and specular reflection parameters used in calculating color values ​​are obtained using the smoothing parameters output by the second prediction model. Introducing smoothing parameters ensures that the wardrobe's material is considered during shading. Simultaneously, the normal information of the facets is also incorporated into the shading process, which effectively reflects the color differences of 3D points on different triangular facets surrounding the wardrobe. Both of these aspects guarantee the realism of the rendering.

[0237] Furthermore, the first and second prediction models used in the above coloring process can also be jointly trained. For details on the training process, please refer to [link to training documentation]. Figures 4-5 The example shown.

[0238] (2) In an entertainment scenario, user A can use an app installed on a terminal device to generate their own 3D virtual avatar. The process of creating a 3D virtual avatar using an app can be as follows:

[0239] In response to user A's launch action, the app can simultaneously access the device's camera to capture a two-dimensional image of the user's face. This image is then input into a pre-defined prediction model to extract facial features. These features reflect user A's face shape, facial features, expression, and skin texture. The app can then use these features, along with a facial template (i.e.,...), to further refine the facial image. Figure 1 The template image in the illustrated embodiment is used to generate a 3D face image to be colored. In this image, the face is displayed in a 3D style but without color, and the face is composed of multiple triangular facets. The vertices and non-vertices of the triangular facets surrounding the face can be collectively referred to as 3D points in the 3D face image to be colored.

[0240] Then, the normal information of the multiple triangular face elements surrounding the face in the 3D face image to be colored is input into the second prediction model, which outputs the smoothing parameters corresponding to the face. Finally, each 3D point in the 3D face image to be colored can be colored according to the smoothing parameters, and the 3D virtual image corresponding to user A can be displayed on the screen of the terminal device.

[0241] Since the coloring process is the same for every 3D point, we will take any 3D point, i.e., the target 3D point, as an example to illustrate the coloring process:

[0242] The diffuse and specular reflection parameters of the target 3D point can be calculated based on the normal information of the surface element containing the target 3D point, the position of the light source, and the smoothing parameters output by the second prediction model. Then, these two reflection parameters, along with the texture parameters of the target 3D point, are used to calculate the color value of the target 3D point. In the scene where the 3D virtual avatar is generated, the position of the light source varies depending on the user's environment. For example, in an indoor scene, the light source could be a window or the location of indoor lighting equipment; in an outdoor scene, the light source could be the location of outdoor lighting equipment, and so on.

[0243] Simultaneously, since the coordinates and texture parameters of the vertices in the 3D face image to be colored are known, interpolation can be performed on these vertices to obtain the coordinates and texture parameters of the non-vertices for interpolation. Then, through coordinate transformation, the coordinates of the vertices and non-vertices in the world coordinate system are converted to the screen coordinate system to establish the correspondence between the 3D points in the image to be colored and the pixels on the screen. Finally, based on this correspondence, the color values ​​of the target 3D points are used to render the corresponding pixels, thus achieving the coloring of the target 3D points, and the coloring result will be displayed on the screen.

[0244] The process of creating a 3D virtual character scene described above can also be combined with... Figure 7 I understand. Furthermore, the process not described in detail here can be found in the relevant descriptions within the "Game Development" scenario mentioned above.

[0245] Of course, in the live streaming field mentioned in the background technology section, the streamer can also use a live streaming app to capture their own two-dimensional facial image and further create their own three-dimensional virtual avatar, which they then use for live streaming. Simultaneously, the trading objects and the live streaming background in the live stream can also be captured using the app to generate corresponding three-dimensional images.

[0246] The creation process of the 3D virtual avatar of the streamer, the trading partner, and the corresponding 3D images of the live stream background can also be combined with... Figure 7 And the description in the "entertainment scenarios" above.

[0247] (3) In traffic scenarios, we can model and color multiple road segments in a certain area and display the coloring results on the screen so that users can see a three-dimensional map on the screen.

[0248] The specific process can be as follows: First, the road segment is photographed to obtain a two-dimensional road image. Then, the road image is input into a first prediction model to extract its image features. Assuming the road image contains a vehicle, the image features can include the vehicle's shape and texture. Next, a three-dimensional vehicle image to be colored is generated based on the extracted image features and the vehicle template. In this image, the vehicle is displayed in a 3D style but without color, and the vehicle is composed of multiple triangular facets. The vertices and non-vertices of the triangular facets surrounding the vehicle can be collectively referred to as three-dimensional points in the three-dimensional face image to be colored.

[0249] Next, the normal information of multiple triangular facets in the 3D vehicle image to be colored is input into the second prediction model, which then outputs smoothing parameters reflecting the vehicle's material. Finally, each 3D point in the 3D vehicle image to be colored is colored according to the smoothing parameters, thus establishing a colored 3D model of the vehicle, which completes the coloring of the vehicle in the road image. After coloring all objects in the captured road image as described above, a 3D map can be viewed on the screen. The coloring process for any 3D point in the 3D vehicle image to be colored can be analogous to the two scenario embodiments described above, and will not be repeated here.

[0250] Besides the scenarios exemplified above, the image rendering methods provided in the embodiments of this invention can also be applied to Virtual Reality (VR) and Augmented Reality (AR) scenarios. For instance, in educational settings, teachers often need to explain knowledge using two-dimensional images. In this case, the image rendering methods described above can be used to render the two-dimensional images into three-dimensional images, allowing students to understand the knowledge by combining the three-dimensional images with their understanding. Similarly, in exhibitions or museums, merchandise can be rendered into three-dimensional images for 3D display. Furthermore, in medical settings, two-dimensional diagnostic images of patients can be rendered into three-dimensional images to enable doctors to make more accurate diagnoses.

[0251] The image rendering apparatus of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will understand that these image rendering apparatuses can all be configured using commercially available hardware components through the steps taught in this solution.

[0252] Figure 11This is a schematic diagram of the structure of an image rendering device provided in an embodiment of the present invention, as shown below. Figure 11 As shown, the device includes:

[0253] Module 11 is used to obtain the object properties of the object to be rendered.

[0254] The prediction module 12 is used to predict smoothing parameters that reflect the material of the object to be rendered based on the three-dimensional image to be shaded corresponding to the object to be rendered, which has the object attributes.

[0255] The rendering module 13 is used to render the three-dimensional image to be colored according to the smoothing parameters.

[0256] Optionally, the acquisition module 11 is specifically used to: acquire a two-dimensional image containing the object to be rendered; and input the two-dimensional image into a first prediction model so that the first prediction model outputs image features as attributes of the object.

[0257] Optionally, the prediction module 13 specifically includes:

[0258] The generation unit 121 is used to generate a three-dimensional image to be colored corresponding to the object to be rendered based on the object attributes.

[0259] The prediction unit 122 is used to predict the smoothing parameters based on the orientation information of the face elements surrounding the object to be rendered in the three-dimensional image to be colored.

[0260] Optionally, the generation unit 121 is specifically used to: generate a three-dimensional image to be colored corresponding to the object to be rendered based on the image features and the template image, wherein the template image corresponds to the type of the object to be rendered.

[0261] Optionally, the prediction module 12 is specifically used to: input the direction information into the second prediction model so that the second prediction model outputs the smoothing parameter.

[0262] Optionally, the rendering module 13 specifically includes:

[0263] The first determining unit 131 is used to determine the correspondence between a target three-dimensional point in the three-dimensional image to be colored and a target pixel in the screen through coordinate transformation, wherein the target three-dimensional point is any three-dimensional point in the three-dimensional image to be colored.

[0264] The second determining unit 132 is used to determine the rendering parameters of the target 3D point based on the smoothing parameters.

[0265] The third determining unit 133 is used to determine the color value of the target three-dimensional point according to the rendering parameters.

[0266] The rendering unit 134 is used to render the target pixel on the screen using the color value of the target three-dimensional point according to the correspondence.

[0267] Optionally, the first determining unit 131 is specifically used to: determine a preset number of target face elements whose depth values ​​meet the requirements based on the depth values ​​of the vertices of the face elements in the three-dimensional image to be colored;

[0268] The centroid coordinates of the target surface element are determined based on the coordinates of its vertices.

[0269] Based on the centroid coordinates, the coordinates of the vertices are interpolated to obtain the coordinates of each 3D point in the 3D image to be colored.

[0270] By transforming coordinates, the correspondence between each three-dimensional point in the three-dimensional image to be colored and the pixel points on the screen is determined.

[0271] Optionally, the second determining unit 132 is specifically used to: determine the texture parameters of the target three-dimensional point; and determine the diffuse reflection parameters and specular reflection parameters of the target three-dimensional point based on the smoothing parameters, the orientation information of the surface element where the target three-dimensional point is located, and the position of the light source, wherein the light source is located in the shooting environment corresponding to the two-dimensional image.

[0272] Optionally, the device further includes a determining module 14 and a display module 15.

[0273] The determining module 14 is used to determine the rendering scene to which the object to be rendered belongs.

[0274] The acquisition module 11 is also used to acquire the preset parameter range corresponding to the rendering scene.

[0275] The display module 15 is used to display the smoothing parameter if the predicted smoothing parameter exceeds the preset parameter range, so that the user can modify it.

[0276] Optionally, the image rendering apparatus further includes a generation module 16, an input module 17, and an adjustment module 18.

[0277] The acquisition module 11 is further configured to acquire a sample image containing the sample object, wherein the sample image is a two-dimensional image.

[0278] The generation module 16 is further configured to generate a three-dimensional image to be colored corresponding to the sample image based on the image features of the sample image; and to generate a two-dimensional image to be colored corresponding to the sample image based on the three-dimensional image to be colored corresponding to the sample image.

[0279] The input module 17 is used to input the orientation information of the pixels in the two-dimensional image to be colored into the second prediction model, so that the second prediction model outputs smoothing parameters.

[0280] The rendering module 13 is used to render the two-dimensional image to be colored according to the smoothing parameters to obtain the colored two-dimensional image.

[0281] The adjustment module 18 is used to adjust the model parameters of the second prediction model based on the color difference between the sample image and the colored two-dimensional image.

[0282] Optionally, the adjustment module 18 is further configured to adjust the model parameters of the first prediction model based on the color difference between the sample image and the colored two-dimensional image.

[0283] Figure 11 The device shown can perform Figures 1 to 5 For the methods shown in the embodiments, the parts not described in detail in this embodiment can be referred to the following: Figures 1 to 5 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figures 1 to 5 The descriptions in the illustrated embodiments will not be repeated here.

[0284] The above describes the internal functions and structure of the image rendering device. In one possible design, the image rendering device can be implemented as an electronic device, such as... Figure 12 As shown, the electronic device may include a processor 21 and a memory 22. The memory 22 is used to store data supporting the electronic device in performing the above-described actions. Figures 1 to 5 The image rendering method program provided in the illustrated embodiment is configured by the processor 21 to execute the program stored in the memory 22.

[0285] The program includes one or more computer instructions, wherein when the one or more computer instructions are executed by the processor 21, they can perform the following steps:

[0286] Get the object properties of the object to be rendered;

[0287] Based on the 3D image of the object to be rendered corresponding to the object having the object attributes, predict the smoothing parameters that reflect the material of the object to be rendered.

[0288] The three-dimensional image to be colored is rendered according to the smoothing parameters.

[0289] Optionally, the processor 21 is further configured to perform the aforementioned Figures 1 to 5 All or part of the steps in the illustrated embodiments.

[0290] The structure of the electronic device may also include a communication interface 23 for the electronic device to communicate with other devices or communication networks.

[0291] In addition, embodiments of the present invention provide a computer storage medium for storing computer software instructions used by the aforementioned electronic device, which includes instructions for executing the above-mentioned... Figures 1 to 5 The procedure involved in the image rendering method in the illustrated embodiment.

[0292] Figure 13 This is a schematic diagram of another image rendering apparatus provided in an embodiment of the present invention, as shown below. Figure 13 As shown, the device includes:

[0293] Module 31 is used to obtain the object properties of the object to be rendered in the live streaming page.

[0294] The prediction module 32 is used to predict smoothing parameters that reflect the material of the object to be rendered based on the 3D image to be shaded corresponding to the object to be rendered, which has the object attributes.

[0295] The rendering module 33 is used to render the three-dimensional image to be colored according to the smoothing parameters.

[0296] The display module 34 is used to display the shading 3D image corresponding to the object to be rendered on the live streaming page.

[0297] Optionally, the prediction module 32 specifically includes:

[0298] The generation unit 321 is used to generate a three-dimensional image to be colored corresponding to the object to be rendered based on the object attributes.

[0299] The input unit 322 is used to input the orientation information of the face elements surrounding the object to be rendered in the three-dimensional image to be colored into the prediction model, so that the prediction model outputs the smoothing parameters.

[0300] The object to be rendered includes at least one of the following: the transaction object on the live streaming page, the virtual live streaming background, and the virtual anchor.

[0301] Optionally, the generation unit 321 is specifically used to: acquire a two-dimensional image containing the object to be rendered; and generate a three-dimensional image to be colored corresponding to the object to be rendered based on a template image and the image features of the two-dimensional image, wherein the template image corresponds to the type of the object to be rendered, and the image features are used as the object attributes.

[0302] Figure 13 The device shown can perform Figure 7 For the methods shown in the embodiments, the parts not described in detail in this embodiment can be referred to the following: Figure 7The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figure 7 The descriptions in the illustrated embodiments will not be repeated here.

[0303] The above describes the internal functions and structure of the image rendering device. In one possible design, the image rendering device can be implemented as an electronic device, such as... Figure 14 As shown, the electronic device may include a processor 35 and a memory 36. The memory 36 is used to store data supporting the electronic device in performing the above-described actions. Figure 7 The image rendering method program provided in the illustrated embodiment is configured by the processor 35 to execute the program stored in the memory 36.

[0304] The program includes one or more computer instructions, wherein when the one or more computer instructions are executed by the processor 35, they can perform the following steps:

[0305] Retrieve the object properties of the object to be rendered in the live stream page;

[0306] Based on the 3D image to be shaded corresponding to the object to be rendered, which has the object attributes, predict the smoothing parameters that reflect the material of the object to be rendered.

[0307] Render the 3D image to be colored according to the smoothing parameters;

[0308] The live stream page displays the shading 3D image of the object to be rendered.

[0309] Optionally, the processor 35 is further configured to perform the aforementioned Figure 7 All or part of the steps in the illustrated embodiments.

[0310] The structure of the electronic device may also include a communication interface 37 for the electronic device to communicate with other devices or communication networks.

[0311] In addition, embodiments of the present invention provide a computer storage medium for storing computer software instructions used by the aforementioned electronic device, which includes instructions for executing the above-mentioned... Figure 7 The procedure involved in the image rendering method in the illustrated embodiment.

[0312] Figure 15 This is a schematic diagram of the structure of another image rendering device provided in an embodiment of the present invention, as shown below. Figure 15 As shown, the device includes:

[0313] The acquisition module 41 is used to acquire the object attributes of the user's target human body parts.

[0314] The prediction module 42 is used to predict smoothing parameters reflecting the material of the target human body part based on the three-dimensional image to be colored corresponding to the target human body part having the object attributes.

[0315] Rendering module 43 is used to render the three-dimensional image to be colored according to the smoothing parameters.

[0316] Display module 44 is used to display the colored 3D image corresponding to the target human body part.

[0317] Optionally, the acquisition module 41 is specifically used to: in response to the user-triggered shooting operation, acquire a two-dimensional image containing the target human body part; and extract the image features of the two-dimensional image as the object attribute.

[0318] Optionally, the prediction module 42 is specifically used to: generate a three-dimensional image to be colored corresponding to the target human body part based on the template image and the image features, wherein the template image contains the target human body part; and input the orientation information of the face elements surrounding the target human body part in the three-dimensional image to be colored into the prediction model, so that the prediction model outputs the smoothing parameters.

[0319] Figure 15 The device shown can perform Figure 8 For the methods shown in the embodiments, the parts not described in detail in this embodiment can be referred to the following: Figure 8 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figure 8 The descriptions in the illustrated embodiments will not be repeated here.

[0320] The above describes the internal functions and structure of the image rendering device. In one possible design, the image rendering device can be implemented as an electronic device, such as... Figure 16 As shown, the electronic device may include a processor 45 and a memory 46. The memory 46 is used to store data supporting the electronic device in performing the above-described actions. Figure 8 The image rendering method program provided in the illustrated embodiment is configured by the processor 45 to execute the program stored in the memory 46.

[0321] The program includes one or more computer instructions, wherein when the one or more computer instructions are executed by the processor 45, they can perform the following steps:

[0322] Obtain the object properties of the target human body part of the user;

[0323] Based on the three-dimensional image to be colored corresponding to the target human body part having the object attributes, predict the smoothing parameters reflecting the material of the target human body part;

[0324] Render the 3D image to be colored according to the smoothing parameters;

[0325] Display the colored 3D image corresponding to the target human body part.

[0326] Optionally, the processor 45 is further configured to perform the aforementioned Figure 8 All or part of the steps in the illustrated embodiments.

[0327] The structure of the electronic device may also include a communication interface 47 for the electronic device to communicate with other devices or communication networks.

[0328] In addition, embodiments of the present invention provide a computer storage medium for storing computer software instructions used by the aforementioned electronic device, which includes instructions for executing the above-mentioned... Figure 8 The procedure involved in the image rendering method in the illustrated embodiment.

[0329] Figure 17 This is a schematic diagram of the structure of another image rendering device provided in an embodiment of the present invention, as shown below. Figure 17 As shown, the device includes:

[0330] The acquisition module 51 is used to acquire the object attributes of the object to be rendered in the road image.

[0331] The prediction module 52 is used to predict smoothing parameters that reflect the material of the object to be rendered based on the 3D image to be shaded corresponding to the object to be rendered, which has the object attributes.

[0332] The rendering module 53 is used to render the three-dimensional image to be colored according to the smoothing parameters.

[0333] The display module 54 is used to generate a 3D map based on the shading 3D image corresponding to the object to be rendered.

[0334] Figure 17 The device shown can perform Figure 9 For the methods shown in the embodiments, the parts not described in detail in this embodiment can be referred to the following: Figure 9 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figure 9 The descriptions in the illustrated embodiments will not be repeated here.

[0335] The above describes the internal functions and structure of the image rendering device. In one possible design, the image rendering device can be implemented as an electronic device, such as... Figure 18 As shown, the electronic device may include a processor 55 and a memory 56. The memory 56 is used to store data supporting the electronic device in performing the above-described actions. Figure 9The image rendering method program provided in the illustrated embodiment is configured by the processor 55 to execute the program stored in the memory 56.

[0336] The program includes one or more computer instructions, wherein when the one or more computer instructions are executed by the processor 55, they can perform the following steps:

[0337] Retrieve the object properties of the object to be rendered in the road image;

[0338] Based on the 3D image to be shaded corresponding to the object to be rendered, which has the object attributes, predict the smoothing parameters that reflect the material of the object to be rendered.

[0339] Render the 3D image to be colored according to the smoothing parameters;

[0340] A 3D map is generated based on the shaded 3D image corresponding to the object to be rendered.

[0341] Optionally, the processor 55 is further configured to perform the aforementioned Figure 9 All or part of the steps in the illustrated embodiments.

[0342] The structure of the electronic device may also include a communication interface 57 for the electronic device to communicate with other devices or communication networks.

[0343] In addition, embodiments of the present invention provide a computer storage medium for storing computer software instructions used by the aforementioned electronic device, which includes instructions for executing the above-mentioned... Figure 9 The procedure involved in the image rendering method in the illustrated embodiment.

[0344] The model training apparatus of one or more embodiments of the present invention will be described in detail below. Those skilled in the art will understand that these service-providing apparatuses can all be configured using commercially available hardware components through the steps taught in this solution.

[0345] Figure 19 This is a schematic diagram of the structure of a model training device provided in an embodiment of the present invention, as shown below. Figure 19 As shown, the device includes:

[0346] The acquisition module 61 is used to acquire a sample image containing the sample object, wherein the sample image is a two-dimensional image.

[0347] The generation module 62 is used to generate a three-dimensional image to be colored corresponding to the sample image based on the image features of the sample image, and to generate a two-dimensional image to be colored corresponding to the sample image based on the three-dimensional image to be colored corresponding to the sample image.

[0348] The input module 63 is used to input the orientation information of the pixels in the two-dimensional image to be colored into the first prediction model, so that the first prediction model outputs smoothing parameters that reflect the material of the sample object.

[0349] The rendering module 64 is used to render the two-dimensional image to be colored according to the smoothing parameters to obtain the colored two-dimensional image.

[0350] The adjustment module 65 is used to adjust the model parameters of the first prediction model based on the color difference between the sample image and the colored two-dimensional image.

[0351] Figure 19 The device shown can perform Figure 6 For the methods shown in the embodiments, the parts not described in detail in this embodiment can be referred to the following: Figure 6 The relevant descriptions of the illustrated embodiments are provided below. For the execution process and technical effects of this technical solution, please refer to [link / reference]. Figure 6 The descriptions in the illustrated embodiments will not be repeated here.

[0352] The above describes the internal functions and structure of the model training device. In one possible design, the model training device can be implemented as an electronic device, such as... Figure 20 As shown, the electronic device may include a processor 66 and a memory 67. The memory 67 is used to store data supporting the electronic device in performing the above-described actions. Figure 6 The program for the model training method provided in the illustrated embodiment is such that the processor 66 is configured to execute the program stored in the memory 67.

[0353] The program includes one or more computer instructions, wherein when the one or more computer instructions are executed by the processor 66, they can perform the following steps:

[0354] Obtain a sample image containing the sample object, wherein the sample image is a two-dimensional image;

[0355] Based on the image features of the sample image, generate a corresponding 3D image to be colored;

[0356] Generate a two-dimensional image to be colored based on the three-dimensional image to be colored corresponding to the sample image;

[0357] The orientation information of the pixels in the two-dimensional image to be colored is input into the first prediction model, so that the first prediction model outputs smoothing parameters that reflect the material of the sample object.

[0358] The two-dimensional image to be colored is rendered according to the smoothing parameters to obtain the colored two-dimensional image;

[0359] The model parameters of the first prediction model are adjusted based on the color difference between the sample image and the colored two-dimensional image.

[0360] Optionally, the processor 66 is further configured to perform the aforementioned Figure 6 All or part of the steps in the illustrated embodiments.

[0361] The structure of the electronic device may also include a communication interface 68 for communication between the electronic device and other devices or communication networks.

[0362] In addition, embodiments of the present invention provide a computer storage medium for storing computer software instructions used by the aforementioned electronic device, which includes instructions for executing the above-mentioned... Figure 6 The procedure involved in the model training method in the illustrated embodiment.

[0363] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image rendering method, characterized in that, include: Get the object properties of the object to be rendered; The orientation information of the face elements that form the object to be rendered in the three-dimensional image to be colored is input into the second prediction model, so that the second prediction model outputs smoothing parameters that reflect the material of the object to be rendered. The three-dimensional image to be colored is the three-dimensional image corresponding to the object to be rendered that has the object attributes. The three-dimensional image to be colored is rendered according to the smoothing parameters.

2. The method according to claim 1, characterized in that, The process of obtaining the object properties of the object to be rendered includes: Obtain a two-dimensional image containing the object to be rendered; The two-dimensional image is input into a first prediction model, and the first prediction model outputs image features as attributes of the object.

3. The method according to claim 2, characterized in that, The method further includes: Generate the 3D image to be colored corresponding to the object to be rendered based on the object's attributes.

4. The method according to claim 3, characterized in that, The step of generating the 3D image to be shaded corresponding to the object to be rendered based on the object attributes includes: Based on the image features and the template image, a three-dimensional image to be colored corresponding to the object to be rendered is generated, wherein the template image corresponds to the type of the object to be rendered.

5. The method according to claim 3, characterized in that, The step of rendering the 3D image to be colored according to the smoothing parameters includes: By transforming coordinates, the correspondence between the target 3D point in the 3D image to be colored and the target pixel in the screen is determined, wherein the target 3D point is any 3D point in the 3D image to be colored; The rendering parameters of the target 3D point are determined based on the smoothing parameters; The color value of the target 3D point is determined based on the rendering parameters; Based on the correspondence, the color value of the target 3D point is used to render the target pixel on the screen.

6. The method according to claim 5, characterized in that, The step of determining the correspondence between target 3D points in the 3D image to be colored and target pixels on the screen through coordinate transformation includes: Based on the depth values ​​of the vertices of the face elements in the three-dimensional image to be colored, determine a preset number of target face elements whose depth values ​​meet the requirements; The centroid coordinates of the target surface element are determined based on the coordinates of its vertices. Based on the centroid coordinates, the coordinates of the vertices are interpolated to obtain the coordinates of each 3D point in the 3D image to be colored. By transforming coordinates, the correspondence between each three-dimensional point in the three-dimensional image to be colored and the pixel points on the screen is determined.

7. The method according to claim 5, characterized in that, Determining the rendering parameters of the target 3D point based on the smoothing parameters includes: Determine the texture parameters of the target 3D point; Based on the smoothing parameters, the orientation information of the surface element where the target 3D point is located, and the position of the light source, the diffuse reflection parameters and specular reflection parameters of the target 3D point are determined, and the light source is located in the shooting environment corresponding to the 2D image.

8. The method according to claim 1, characterized in that, The method further includes: Determine the rendering scene to which the object to be rendered belongs; Obtain the preset parameter range corresponding to the rendering scene; If the predicted smoothing parameter exceeds the preset parameter range, the smoothing parameter is displayed for the user to modify.

9. The method according to claim 3, characterized in that, The method further includes: Obtain a sample image containing the sample object, wherein the sample image is a two-dimensional image; Based on the image features of the sample image, generate a corresponding 3D image to be colored; Generate a two-dimensional image to be colored based on the three-dimensional image to be colored corresponding to the sample image; The orientation information of the pixels in the two-dimensional image to be colored is input into the second prediction model so that the second prediction model outputs smoothing parameters. The two-dimensional image to be colored is rendered according to the smoothing parameters to obtain the colored two-dimensional image; The model parameters of the second prediction model are adjusted based on the color difference between the sample image and the colored 2D image.

10. An image rendering method, characterized in that, include: Retrieve the object properties of the object to be rendered in the live stream page; The orientation information of the face elements that form the object to be rendered in the three-dimensional image to be colored is input into the second prediction model, so that the second prediction model outputs smoothing parameters that reflect the material of the object to be rendered. The three-dimensional image to be colored is the three-dimensional image corresponding to the object to be rendered that has the object attributes. Render the 3D image to be colored according to the smoothing parameters; The live stream page displays the shading 3D image of the object to be rendered.

11. The method according to claim 10, characterized in that, The objects to be rendered include at least one of the following: the transaction object on the live streaming page, the virtual live streaming background, and the virtual anchor.

12. The method according to claim 10 or 11, characterized in that, The method further includes: Generate the 3D image to be colored corresponding to the object to be rendered based on the object's attributes.

13. The method according to claim 12, characterized in that, The step of generating the 3D image to be shaded corresponding to the object to be rendered based on the object attributes includes: Obtain a two-dimensional image containing the object to be rendered; Based on the template image and the image features of the two-dimensional image, a three-dimensional image to be colored corresponding to the object to be rendered is generated. The template image corresponds to the type of the object to be rendered, and the image features are used as the object attributes.

14. An image rendering method, characterized in that, include: Obtain the object properties of the target human body part of the user; The orientation information of the face elements that form the target human body part in the three-dimensional image to be colored is input into the second prediction model, so that the second prediction model outputs smoothing parameters that reflect the material of the target human body part. The three-dimensional image to be colored is the three-dimensional image corresponding to the target human body part that has the object attribute. Render the 3D image to be colored according to the smoothing parameters; Display the colored 3D image corresponding to the target human body part.

15. The method according to claim 14, characterized in that, The method of obtaining the object attributes of the user's target human body parts includes: In response to the user-triggered shooting operation, a two-dimensional image containing the target human body part is acquired; Extract the image features of the two-dimensional image to serve as the object attributes.

16. The method according to claim 15, characterized in that, The method further includes: Based on the template image and the image features, a three-dimensional image to be colored corresponding to the target human body part is generated, wherein the template image contains the target human body part.

17. An image rendering method, characterized in that, include: Retrieve the object properties of the object to be rendered in the road image; The orientation information of the face elements that form the object to be rendered in the three-dimensional image to be colored is input into the second prediction model, so that the second prediction model outputs smoothing parameters that reflect the material of the object to be rendered. The three-dimensional image to be colored is the three-dimensional image corresponding to the object to be rendered that has the object attributes. Render the 3D image to be colored according to the smoothing parameters; A 3D map is generated based on the shaded 3D image corresponding to the object to be rendered.

18. An image rendering apparatus, characterized in that, include: The acquisition module is used to retrieve the object properties of the object to be rendered. The prediction module is used to input the orientation information of the face elements surrounding the object to be rendered in the three-dimensional image to be colored into the second prediction model, so that the second prediction model outputs smoothing parameters that reflect the material of the object to be rendered. The three-dimensional image to be colored is the three-dimensional image corresponding to the object to be rendered that has the object attributes. The rendering module is used to render the 3D image to be colored according to the smoothing parameters.

19. An electronic device, characterized in that, include: A memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor performs the image rendering method as described in any one of claims 1 to 9.

20. A non-transitory machine-readable storage medium, characterized in that, The non-transitory machine-readable storage medium stores executable code that, when executed by a processor of an electronic device, causes the processor to perform the image rendering method as described in any one of claims 1 to 9.

21. An image rendering apparatus, characterized in that, include: The acquisition module is used to retrieve the object properties of the object to be rendered in the live streaming page. The prediction module is used to input the orientation information of the face elements surrounding the object to be rendered in the three-dimensional image to be colored into the second prediction model, so that the second prediction model outputs smoothing parameters that reflect the material of the object to be rendered. The three-dimensional image to be colored is the three-dimensional image corresponding to the object to be rendered that has the object attributes. The rendering module is used to render the 3D image to be colored according to the smoothing parameters; The display module is used to display the shading 3D image corresponding to the object to be rendered on the live streaming page.

22. An electronic device, characterized in that, include: A memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor performs the image rendering method as described in any one of claims 10 to 13.

23. A non-transitory machine-readable storage medium, characterized in that, The non-transitory machine-readable storage medium stores executable code that, when executed by a processor of an electronic device, causes the processor to perform the image rendering method as described in any one of claims 10 to 13.

24. An image rendering apparatus, characterized in that, include: The acquisition module is used to acquire the object properties of the user's target human body parts; The prediction module is used to input the orientation information of the face elements that enclose the target human body part in the three-dimensional image to be colored into the second prediction model, so that the second prediction model outputs smoothing parameters that reflect the material of the target human body part. The three-dimensional image to be colored is the three-dimensional image corresponding to the target human body part that has the object attribute. The rendering module is used to render the 3D image to be colored according to the smoothing parameters; The display module is used to display the colored 3D image corresponding to the target human body part.

25. An electronic device, characterized in that, include: A memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor performs the image rendering method as described in any one of claims 14 to 16.

26. A non-transitory machine-readable storage medium, characterized in that, The non-transitory machine-readable storage medium stores executable code that, when executed by a processor of an electronic device, causes the processor to perform the image rendering method as described in any one of claims 14 to 16.

27. An image rendering apparatus, characterized in that, include: The acquisition module is used to obtain the object properties of the objects to be rendered in the road image; The prediction module is used to input the orientation information of the face elements surrounding the object to be rendered in the three-dimensional image to be colored into the second prediction model, so that the second prediction model outputs smoothing parameters that reflect the material of the object to be rendered. The three-dimensional image to be colored is the three-dimensional image corresponding to the object to be rendered that has the object attributes. The rendering module is used to render the 3D image to be colored according to the smoothing parameters; The display module is used to generate a 3D map based on the shaded 3D image corresponding to the object to be rendered.

28. An electronic device, characterized in that, include: A memory and a processor; wherein the memory stores executable code, and when the executable code is executed by the processor, the processor performs the image rendering method as described in claim 17.

29. A non-transitory machine-readable storage medium, characterized in that, The non-transitory machine-readable storage medium stores executable code that, when executed by a processor of an electronic device, causes the processor to perform the image rendering method as described in claim 17.

Images