Method, device, electronic device and storage medium for drawing an image

Abstract

Embodiments of the present disclosure provide a method and device for rendering an image, electronic equipment and a storage medium. The method comprises: obtaining a plurality of images to be processed; wherein the light angles between the light sources and target objects in each image to be processed are different; processing the image to be processed according to a target object attribute determination model to obtain object attribute information of the target object; and determining a target three-dimensional view corresponding to the target object based on the object attribute information. The technical solution provided by the embodiments of the present disclosure not only realizes accurate estimation of object attribute information, but also can perform reverse rendering on the object based on the information.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, electronic device, and storage medium for drawing images. Background Technology

[0002] Reverse rendering is one of the more important research directions in computer graphics. Using reverse rendering technology, the geometry and material of an object can be recovered from a single image of the object.

[0003] However, in the process of reverse rendering, because the objects in the images usually have complex structures or characteristics, there are many obstacles in the process of building object models based on existing methods, making it impossible to accurately reflect the display effect of the objects in reality. Summary of the Invention

[0004] This disclosure provides a method, apparatus, electronic device, and storage medium for drawing images, which not only achieves accurate estimation of object attribute information, but also enables reverse drawing of objects based on this information.

[0005] In a first aspect, embodiments of this disclosure provide a method for drawing an image, the method comprising:

[0006] Acquire multiple images to be processed; in each image, the illumination angle between the light source and the target object is different.

[0007] The model determines the target object's attributes and processes the image to be processed to obtain the target object's attribute information.

[0008] Based on the object attribute information, a target 3D view corresponding to the target object is determined.

[0009] Secondly, embodiments of this disclosure also provide an apparatus for drawing images, the apparatus comprising:

[0010] The image acquisition module is used to acquire multiple images to be processed; in each image, the illumination angle between the light source and the target object is different.

[0011] The object attribute information determination module is used to process the image to be processed according to the target object attribute determination model to obtain the object attribute information of the target object;

[0012] The target 3D view determination module is used to determine the target 3D view corresponding to the target object based on the object attribute information.

[0013] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0014] One or more processors;

[0015] Storage device for storing one or more programs.

[0016] When the one or more programs are executed by the one or more processors, the one or more processors implement the method of drawing an image as described in any embodiment of this disclosure.

[0017] Fourthly, embodiments of this disclosure also provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform a method for drawing an image as described in any of the embodiments of this disclosure.

[0018] The technical solution of this disclosure first acquires multiple images of the target object under different lighting angles, then determines the model to process the images based on the target object's attributes to obtain the object attribute information, and finally determines the target 3D view corresponding to the target object based on the object attribute information. This not only achieves accurate estimation of the object attribute information, but also enables reverse drawing of the object based on this information. At the same time, it ensures that the drawn target 3D view is as close as possible to the object's actual display effect in reality, so that users can appreciate the actual presentation effect of the object just by looking at the target 3D view, thus improving the user experience. Attached Figure Description

[0019] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0020] Figure 1 This is a flowchart illustrating a method for drawing an image provided in Embodiment 1 of this disclosure;

[0021] Figure 2 This is a flowchart illustrating a method for drawing an image provided in Embodiment 2 of this disclosure;

[0022] Figure 3 This is a flowchart illustrating a method for drawing an image provided in Embodiment 3 of this disclosure;

[0023] Figure 4 This is a network structure diagram of an image drawing method provided in Embodiment 3 of this disclosure;

[0024] Figure 5 This is a structural block diagram of an image drawing apparatus provided in Embodiment 4 of this disclosure;

[0025] Figure 6This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of this disclosure. Detailed Implementation

[0026] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0027] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0028] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0029] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0030] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0031] Before introducing this technical solution, we can first illustrate the application scenarios. This technical solution can be applied to any situation where it is necessary to determine the three-dimensional view of an object. For example, in live streaming, art design, or specific applications, when it is necessary to display a semi-transparent object from all angles, multiple images of the object can be obtained by taking pictures of the object from multiple lighting angles. In this case, based on this technical solution, the attribute information of the semi-transparent object can be determined using the captured images, and then the three-dimensional view of the object can be reverse-engineered.

[0032] Example 1

[0033] Figure 1This is a flowchart illustrating a method for drawing an image according to Embodiment 1 of this disclosure. This embodiment is applicable to situations where a corresponding three-dimensional view is constructed based on multiple images related to an object. This method can be executed by an image drawing device, which can be implemented in the form of software and / or hardware. The hardware can be an electronic device, such as a mobile terminal, a PC, or a server.

[0034] like Figure 1 The method in this embodiment includes:

[0035] S110. Acquire multiple images to be processed.

[0036] In this embodiment, the image to be processed includes at least a specific object (i.e., the target object). In order to construct a three-dimensional view corresponding to the object, multiple images to be processed are required. Those skilled in the art should understand that multiple images to be processed can reflect the shape and structure information of the target object from multiple angles, or reflect the shape and structure information of the target object under different environmental conditions from one or more angles.

[0037] In this embodiment, the image to be processed may also include a translucent material object for which a 3D view is desired to be constructed. This translucent material object can be an object with translucent properties, such as jade, candles, or plant leaves. In optics, transparency is the physical property that allows light to pass through a material without being scattered. On a macroscopic scale, photons obey Snell's law. Transparency is a superset of transparency; it allows light to pass through but does not necessarily conform to Snell's law, and photons can be scattered at either of the two interfaces. Since the interaction between light and material includes scattering and absorption, the translucent property of an object can also be understood as follows: transparency occurs when light enters the object, is refracted, and then exits. Subsurface scattering occurs when light enters the object, is reflected and absorbed internally, and then some light exits from the surface, resulting in a translucent effect. In this embodiment, the translucent object in the image can be represented by the target object.

[0038] In this embodiment, to accurately represent the actual appearance of the object during the subsequent 3D view construction process, the illumination angle between the light source and the target object is different in each image to be processed. It can be understood that when the image shooting angle remains constant, the target object in each image is under different illumination angles, thus the target object in each image will also present different effects. For example, after pointing the shooting device at a piece of jade and fixing the relative positions of the device and the jade, a flash is placed to the left, right, and behind the jade and taken pictures respectively. The jade images obtained under the three illumination effects are the images to be processed.

[0039] In this embodiment, there are multiple ways to acquire multiple images to be processed. For example, specific images can be retrieved from a repository containing multiple images according to preset rules as images to be processed. Those skilled in the art should understand that the retrieved multiple images should present the target object from multiple lighting angles. Alternatively, a camera device can be used to capture images of the target object based on the aforementioned requirements for different lighting angles, thereby obtaining multiple images to be processed in real time. For example, after fixing the target object, multiple shooting angles are preset at different positions. After the camera device captures images of the target object based on these shooting angles, multiple images to be processed corresponding to the object can be obtained. Those skilled in the art should understand that the specific method of acquiring images to be processed should be selected according to the actual situation, and this embodiment does not impose specific limitations here.

[0040] S120. Based on the target object's attributes, determine the model to process the image to be processed, and obtain the target object's attribute information.

[0041] The target object attribute determination model can be a pre-trained deep learning model capable of processing multiple images and outputting corresponding results. Once the input images are identified, the model processes them to output the attribute information of the target object within the images. It should be noted that multiple images are acquired; therefore, the target object attribute determination model can use this set of images as input, process them uniformly, and then output the object attribute information.

[0042] In this embodiment, object attribute information refers to parameters that reflect the effect of an object in reality from multiple dimensions, including the object's material information, shape information (geometric information), color, texture, smoothness, and transparency. Continuing with the above example, when the images of jade under three lighting effects are determined as the images to be processed, these images can be input into the target object attribute determination model. After model processing, the material information, color, texture, and transparency of the jade can be obtained.

[0043] It should be noted that after obtaining the object attribute information of the target object by determining the model based on the target object's attributes, this information can be stored in a specific repository and marked accordingly, so that it can be directly called in subsequent image processing, avoiding the waste of computing resources caused by determining the object attribute information of the target object multiple times.

[0044] S130. Based on the object attribute information, determine the target 3D view corresponding to the target object.

[0045] In this embodiment, after obtaining the object attribute information of the target object through the target object attribute determination model, a target 3D view corresponding to the target object can be determined. The target 3D view is a projection of the 3D model corresponding to the target object, observed from different viewpoints in 3D space. On one hand, the target 3D view can realistically reflect the effect of the target object in reality in a three-dimensional manner; on the other hand, based on the target 3D view, users can observe the target object from multiple directions. For example, users can select different viewpoints and observe the front view, side view, and top view of the target object through the projection of the 3D view.

[0046] Specifically, various computer software can be used to construct a target 3D view. For example, first, use CAD or Revit software to create an engineering project, import the determined object attribute information into the software, and then set the corresponding configuration items in the software based on this information. After the software parameters are set, a rendering simulation operation can be performed on the target object to construct a target 3D view corresponding to the target object.

[0047] The technical solution of this embodiment first acquires multiple images of the target object under different lighting angles, then determines the model to process the images based on the target object's attributes to obtain the object attribute information, and finally determines the target 3D view corresponding to the target object based on the object attribute information. This not only achieves accurate estimation of object attribute information, but also enables reverse drawing of the object based on this information. At the same time, it ensures that the drawn target 3D view is as close as possible to the object's real-world display effect, so that users can appreciate the actual presentation effect of the object just by looking at the target 3D view, thus improving the user experience.

[0048] Example 2

[0049] Figure 2 This is a flowchart illustrating a method for drawing an image according to Embodiment 2 of this disclosure. Based on the aforementioned embodiments, a camera device captures images of a target object based on lighting angles from a preset set of lighting angles to obtain the reflection and refraction effects of light on the object under different lighting conditions. For different types of images to be processed, differentiated processing is performed based on reflection attribute determination sub-models and refraction attribute determination sub-models to obtain information on the front and back of the object in multiple dimensions. This allows the target 3D view constructed based on this information to more accurately reflect the effect of the object under actual light sources. Specific implementation methods can be found in the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0050] like Figure 2 As shown, the method specifically includes the following steps:

[0051] S210. Illuminate the target object sequentially based on the illumination angles in the preset illumination angle set, and acquire the image to be processed including the target object.

[0052] To accurately construct a 3D view of a target object with semi-transparent properties, it is first necessary to acquire images of the target object under various lighting angles. Therefore, in this embodiment, one approach is to aim the camera device at the target object and fix the relative position of the camera device and the target object. Simultaneously, multiple lighting angles are preset to sequentially illuminate the target object. Under each lighting angle, the camera device acquires one or more images containing the target object to be processed, thereby obtaining images reflecting the effect of the object under different lighting conditions. Another approach is to fix the target object and the light source, draw a circle with the target object as the center, select any radius as the 0° baseline, and preset multiple lighting angles in a clockwise or counterclockwise direction. The camera device moves along the circle based on each lighting angle, and captures the corresponding image as the image to be processed when the device reaches each lighting angle. It can be understood that after the camera device rotates around the target object once, multiple images to be processed under the preset lighting angles can be obtained. Those skilled in the art should understand that the set of multiple illumination angles mentioned above is the preset illumination angle set. At the same time, regardless of which method is used to acquire the image to be processed, after the camera device finishes capturing the image, the obtained image can be stored in a specific server so that it can be directly obtained and used in the process of reconstructing the three-dimensional view of the target object.

[0053] In practical applications, multiple angles in the preset lighting angle set can be set based on specific angle intervals / angle adjustment steps. For example, after aiming the camera at the target object and fixing their positions, a circle with a radius of 2m and the target object as the center can be drawn. Six points on the circle with an angle interval of 60° can be randomly selected as flash mounting points. The flash is turned on sequentially at these six mounting points, and the camera captures the corresponding images, thus obtaining the images of the target object under different lighting angles. Alternatively, a circle with a radius of 2m and the target object as the center can be drawn. A point on the circle can be randomly selected as the flash mounting point. After turning on the flash and capturing the first image to be processed, the position of the flash on the arc can be adjusted in 20° increments, clockwise / counterclockwise. When the flash moves to the first adjusted position, the flash is turned on, and the camera captures the second image to be processed. This process continues until all 18 images to be processed have been captured by the camera. It should be noted that, in this embodiment, in order to accurately obtain the reflection and refraction effect of light on a translucent material object, the flash can always be pointed at the target object (even if the light from the center of the flash always shines directly on the target object).

[0054] S220. Based on the target object's attributes, determine the model to process the image to be processed, and obtain the target object's attribute information.

[0055] When light shines on a translucent object, there are two phenomena: reflection and refraction. Reflection refers to the phenomenon where light changes its direction of propagation at the interface between different materials and returns to the original material; this is divided into specular reflection and diffuse reflection. Refraction refers to the phenomenon where light changes its direction of propagation when it obliquely enters another medium from one medium, causing the light to bend at the interface between the two media.

[0056] Based on this, in the process of determining the object attribute information of the target object, it is necessary to clarify the reflection and refraction effects of light on the object. Correspondingly, the target object attribute determination model includes a reflection attribute determination sub-model and a refraction attribute determination sub-model. These models can be pre-trained deep learning-based models. The two sub-models in the target object attribute determination model will be explained in detail below.

[0057] Optionally, the reflection attribute determination sub-model is used to process the image to be processed based on the reflection type to obtain the reflection attribute association information corresponding to the target object; the reflection attribute determination sub-model is used to process the reflection attribute association information and the image to be processed based on the refraction type to obtain the refraction attribute association information corresponding to the target object; and the object attribute information is determined based on the reflection association attribute information and the refraction association attribute information.

[0058] It should be noted that images to be processed under different lighting angles can be further divided into reflection type and refraction type. This can be understood as follows: for images of the reflection type, the light reflected from the target object by the flash can be determined; for images of the refraction type, the light refracted from the target object by the flash can be determined. Taking six images acquired at 60° intervals as an example, it is known that the positions of the six images taken by the flash are all on a circle centered on the target object, and the angular interval between adjacent points is 60° (i.e., multiple images are under different lighting angles). A line is drawn between the camera and the target object, and this line is marked as line segment 0. Lines are also drawn between each flash point and the target object, and these are marked as line segments 1 through 6. The angles between each of these six line segments and line segment 0 are determined. When the angle is between -90° and 90°, the image corresponding to these line segments is of the reflection type. The images to be processed, for example, line segments 1 to 3 with included angles of 0°, -60°, and 60°, contain light reflected by the object and can at least reflect the reflected image of the front of the target object. When the included angle is in [-180°, -90°] or [90°, 180°], it can be determined that the images to be processed corresponding to these line segments are refraction-type images to be processed. For example, line segments 4 to 6 with included angles of -120°, 120°, and 180°, these images contain not only light reflected by the object but also light refracted by the object and can at least reflect the refracted image of the back of the target object.

[0059] In practical applications, based on the image shooting methods in the above examples, it can be determined that images taken by the flash in the area between the target object and the camera device are reflection-type images to be processed, while images taken by the flash behind the target object are refraction-type images to be processed.

[0060] In this embodiment, the image to be processed with the reflection type is determined and used as input. After being processed by the reflection attribute determination sub-model, the reflection attribute association information output by the model can be obtained. Specifically, the reflection association attribute information includes the reflection material of the target object, the reflection color, the normal map of the image to be processed with the reflection type, and the depth value of each pixel.

[0061] Among them, the reflective material information represents the information corresponding to the material of the reflected light in the image, such as texture and smoothness; the reflective color information represents the color presented in the image when the target object reflects light; the normal map of the image to be processed can be understood as a normal map, that is, a normal is drawn at each point on the concave and convex surface of the original object, and the direction of the normal is marked by the RGB color channel. It can be understood as another different surface parallel to the original concave and convex surface. In practical applications, by determining the normal map, surfaces with low detail can be rendered with high-detail precision in lighting direction and reflection. For example, after baking a normal map from a high-detail model, even if it's applied to the normal map channel of a low-detail model, its surface can still have a rendering effect with light and shadow distribution. Simultaneously, using normal maps reduces the number of faces and computational complexity required for object rendering, thus optimizing the rendering effect. The depth value of each pixel reflects at least the depth of each pixel constituting the target object in the image (i.e., the distance between the camera lens on the object and the point corresponding to that pixel during image capture). Those skilled in the art should understand that during the construction of a 3D view of the target, these depth values ​​can also determine the distance between the corresponding pixel and the viewpoint. It should be noted that the aforementioned reflection-related attribute information characterizes information about the front of the object (i.e., the side facing the camera).

[0062] In this embodiment, the process of the reflection attribute determination sub-model processing the reflection-type images can be implemented based on an encoder-decoder framework, a deep learning model framework. In practical applications, this framework allows multiple reflection-type images to be concatenated using a specific programming language, followed by encoding to obtain the corresponding latent variables. These latent variables are unobservable random variables, typically inferred only from the posterior probability of collected samples. Furthermore, after determining the latent variables corresponding to multiple reflection-type images, four decoding processes yield the reflection material, reflection color, normal map of the reflection-type images, and depth values ​​of each pixel.

[0063] Correspondingly, after determining the type of refraction in the image to be processed, the aforementioned reflection attribute association information can be used as the model input. After processing by the refraction attribute determination sub-model, the refraction association attribute information output by the model can be obtained. Specifically, the refraction association attribute information includes the refraction material of the target object, the normal map of the image to be processed with the refraction type, and the depth value of each pixel.

[0064] The refractive material characterizes information corresponding to the material of the refracted light rays in the image; the normal map of the image to be processed and the depth value of each pixel are similar to those of the reflection type, and will not be described again in this embodiment. It should be noted that the above-mentioned refractive related attribute information characterizes information about the reverse side of the object (i.e., the side facing away from the camera device).

[0065] In this embodiment, the process by which the refraction attribute determination sub-model processes the image to be processed based on the refraction type and the associated information of the reflection attribute can also be implemented based on an encoder-decoder framework. In practical applications, the image of the refraction type and the predicted frontal color, normal, and depth of the object can be merged together and processed through an encoding process to obtain the corresponding latent variables. Furthermore, by performing three decoding processes on the latent variables, the normal maps of the refraction material and the image to be processed based on the refraction type, as well as the depth values ​​of each pixel, can be obtained respectively.

[0066] By performing differentiated processing on different types of images, we can obtain the reflection attribute association information of the front side of a translucent material object and the refraction attribute association information of the back side, so that the target 3D view constructed in the subsequent process can more accurately reflect the display effect of the object under actual light source.

[0067] In this embodiment, after obtaining the reflection-related attribute information and refraction-related attribute information of the target object, these information are integrated to obtain the corresponding object attribute information. The object attribute information includes at least one of material information, shape information, color information, texture information, smoothness information, and transparency information.

[0068] S230: Using the editor as drawing parameters, material information, shape information, color information, texture information, smoothness information, and transparency information, the target 3D view is drawn.

[0069] In this embodiment, after determining the object attribute information of the target object, a pre-written script can be used to import the object attribute information into 3D animation rendering and production software to draw a target 3D view corresponding to the target object; alternatively, the determined object attribute information can be used as drawing reference data, and relevant personnel can draw in the software based on the drawing reference data to obtain the corresponding target 3D view. Those skilled in the art should understand that the specific drawing method can be selected according to the actual situation, and this embodiment does not impose specific limitations.

[0070] Specifically, after obtaining the material, shape, color, texture, smoothness, and transparency information of a piece of jade, a corresponding project can be created in the working interface of a 3D animation rendering software (such as 3ds Max) based on a pre-written script. Further, the data associated with these multi-dimensional information is extracted, and the corresponding parameters in the software are set based on the extracted data. For example, based on the material information, the material editor can determine the material that best matches the actual appearance of the jade; based on the shape information, the edges and corners of the software-generated model can be outlined (e.g., by setting the smoothness parameters of the geometry in 3ds Max), thus making the model resemble the shape of the jade; furthermore, based on the color, smoothness, and transparency information, the specific values ​​of parameters such as diffuse color, specular level, gloss, reflection, refraction, and transparency of the model can be edited in the material editor. Simultaneously, the texture map can be determined for the model based on the texture information. Finally, after setting the above parameters in the software, the model can be rendered to obtain the target 3D view of the jade.

[0071] The technical solution of this embodiment involves a camera device capturing images of a target object based on a preset set of illumination angles to obtain the reflection and refraction effects of the object on light under different illumination conditions. For different types of images to be processed, a sub-model for determining reflection properties and a sub-model for determining refraction properties are used to perform differentiated processing to obtain information on the front and back of the object in multiple dimensions. This allows the target three-dimensional view constructed based on this information to more accurately reflect the effect of the object under actual light sources.

[0072] Example 3

[0073] Figure 3 This is a flowchart illustrating a method for drawing an image according to Embodiment 3 of this disclosure. Based on the aforementioned embodiments, multiple training samples are obtained. The training samples are used to train a sub-model for determining reflection attributes and a sub-model for determining refraction attributes. After the model training is completed, object attribute information corresponding to the object is obtained based on these models, thereby constructing a target 3D view that more closely resembles the real-world effect of the object. Specific implementation methods can be found in the technical solution of this embodiment. Technical terms that are the same as or corresponding to those in the above embodiments will not be repeated here.

[0074] like Figure 3 As shown, the method specifically includes the following steps:

[0075] S310. Obtain multiple training samples; for each training sample, input multiple images to be trained from the current training sample into the object attribute determination model to be trained, and obtain the actual associated attribute information corresponding to the object to be processed.

[0076] In this embodiment, to obtain the associated attribute information of the target object in the above example, it is necessary to pre-train the item attribute determination model. Since the item attribute determination model includes a reflection attribute determination sub-model and a refraction attribute determination sub-model, in practical applications, it is necessary to train the two sub-models separately. Those skilled in the art should understand that for the above two models, the specific process of obtaining the training results includes steps such as constructing a training set, model training, and model parameter tuning. These steps are described in detail below.

[0077] In this embodiment, each training sample includes multiple training images. As described in Embodiment 2, the training images are images of the object to be processed under illumination angles within a preset set of illumination angles. For the target object, after acquiring multiple training images, the set constructed based on these images is the training set corresponding to the target object. For example, when both sub-models to be trained are deep learning networks, and the object to be processed is a piece of jade, a camera can be aimed at the jade, and multiple training images can be acquired using the camera under illumination angles within the preset set of illumination angles. The set constructed based on these images is the training set for the two sub-models.

[0078] It should be noted that, in the technical solution disclosed herein, when using the model to determine object attribute information in the subsequent process, in order to ensure the consistency of the model input in logic and form, the number of multiple images to be processed as input should be consistent with the number of multiple images to be trained in each training sample during model training; correspondingly, the number of illumination angles in the preset illumination angle set should also be consistent with the number of the aforementioned images to be trained.

[0079] In this embodiment, the object attribute determination model to be trained includes a sub-model to be trained for reflection attribute determination and a sub-model to be trained for refraction attribute determination. Those skilled in the art should understand that once the sub-model to be trained for reflection attribute determination is completed, it becomes the reflection attribute determination sub-model, and once the sub-model to be trained for refraction attribute determination is completed, it becomes the refraction attribute determination sub-model. Corresponding to the two sub-models to be trained, the output actual associated attribute information also includes actual trained refraction attribute information and actual trained reflection attribute information. It should be noted that before the sub-models to be trained are fully trained, the output actual trained refraction attribute information and actual trained reflection attribute information may not accurately reflect the reflection and refraction of light by the object to be processed, and based on the output results, it may not be possible to accurately construct a target 3D view of the object to be processed.

[0080] It should also be noted that after identifying multiple training images in the training set, they can be uniformly input into the item attribute determination model to train the model. For example, the six jade images under different lighting angles in the above example can be input into the item attribute determination model to train. Alternatively, images of specific types in the training set can be input into the corresponding sub-models to train each sub-model. For example, the six jade images under different lighting angles in the above example can be divided into three training images of the reflection type and three training images of the refraction type. Furthermore, the three training images of the reflection type can be input into the reflection attribute determination sub-model to train, and the three training images of the refraction type can be input into the refraction attribute determination sub-model to train.

[0081] Specifically, based on the sub-model for determining the reflection attributes to be trained, the training images of the reflection type in the current training samples are processed to obtain the actual training reflection attribute information corresponding to the training object. Similarly, based on the sub-model for determining the refraction attributes to be trained, the training images of the refraction type in the current training samples and the actual training reflection attribute information are processed to obtain the actual training refraction attribute information. Continuing with the example of jade, after inputting the processing image of the reflection type related to the jade into the sub-model for determining the reflection attributes to be trained, the model can output the jade's reflection material, reflection color, the normal map of the processing image of the reflection type, and the depth value of each pixel as the actual training reflection attribute information. Similarly, after inputting the processing image of the refraction type related to the jade into the sub-model for determining the refraction attributes to be trained, the model can output the jade's refraction material, the normal map of the processing image of the refraction type, and the depth value of each pixel as the actual training refraction attribute information.

[0082] S320. Based on actual associated attribute information, determine multiple actual drawn images; by processing the actual drawn images and the image to be trained at the same illumination angle, obtain the target object attribute determination model.

[0083] In this embodiment, the actual training reflection attribute information and the actual training refraction attribute information are integrated to obtain the actual associated attribute information. It can be understood that, in order to correct the model parameters, the number of actually drawn images should be the same as the number of training images in the current training samples; simultaneously, since the parameters in the model can be adjusted using the controlled variable method in practical applications, the illumination angle of the object to be processed in the actually drawn images is the same as the illumination angle of the training images.

[0084] In this embodiment, the actual drawn image includes the actual reflection drawn image and the actual refraction drawn image. The actual reflection drawn image is the image drawn based on the actual trained reflection attribute information, and the actual refraction drawn image is the image drawn based on the actual trained refraction attribute information.

[0085] Specifically, for training images of each reflection type, the target illumination angle of the light source in the current training image for the object to be processed is determined, and based on the actual training reflection attribute information in the actual associated attribute information, an actual reflection rendering image consistent with the target illumination angle is determined. For training images of each refraction type, the target illumination angle of the light source in the current training image for the object to be processed is determined, and based on the actual training refraction attribute information in the actual associated attribute information, an actual refraction rendering image consistent with the target illumination angle is determined.

[0086] Continuing with the above example, after identifying the reflection type image from the jade-related images to be processed, it is also necessary to determine the target illumination angle generated by the light source projecting onto the object in each image. Furthermore, after determining the actual reflection rendering image of the jade based on the actual training reflection attribute information, the image can be associated with the corresponding target illumination angle; for example, each actual reflection rendering image can be labeled with a corresponding illumination angle. The processing method for the jade refraction type image is similar to that for the reflection type image described above, and will not be repeated here.

[0087] Furthermore, the actual reflection rendering image and the image to be trained at the same illumination angle are processed to determine the loss value. Based on the loss value and the first preset loss function corresponding to the reflection attribute determination sub-model to be trained, the model parameters in the reflection attribute determination sub-model to be trained are adjusted. Additionally, the actual refraction rendering image and the image to be trained at the same illumination angle are processed to determine the loss value. Based on the loss value and the second preset loss function corresponding to the reflection attribute determination sub-model to be trained, the model parameters in the reflection attribute determination sub-model to be trained are adjusted. The convergence of the first preset loss function and the second preset loss function is used as the training objective to obtain the target item attribute determination model.

[0088] Continuing with the example above, after obtaining the marked actual reflection rendering image of the jade, it is also necessary to obtain the actual training images of the jade under various lighting angles. Further, keeping the lighting angle constant, the actual reflection rendering image corresponding to that lighting angle is obtained through label filtering. This image is then processed together with the training image corresponding to that lighting angle to obtain the loss value of the model function. Based on the loss value and the first preset loss function corresponding to the sub-model for determining the reflection attributes to be trained, the model parameters in the sub-model can be adjusted, and the corresponding adjustment results can be obtained. In this embodiment, the convergence of the first preset loss function can be used as the training objective. The first preset loss function is associated with the sub-model for determining the reflection attributes to be trained. Based on this, it can be understood that when the first preset loss function is determined not to have converged, it indicates that the adjustment result does not meet the requirements of the scheme, and it is necessary to select other lighting angles and continue training the model in the above manner. When the first preset loss function is determined to have converged, it indicates that the adjustment result meets the requirements of the scheme, and the sub-model for determining the reflection attributes required for subsequently determining the 3D view of the target has been obtained.

[0089] In this embodiment, the second preset loss function is associated with the refraction property determination sub-model to be trained. The specific process of obtaining the refraction property determination sub-model based on the actual refraction drawing image and the image to be trained is similar to the above process, and will not be described again in this embodiment.

[0090] The following is combined with Figure 4 The network structure described above illustrates the process exemplarily. After fixing the positions of the camera device and the target object, a circle is drawn with the target object as the center and the distance between them as the radius. The flash moves clockwise or counterclockwise along this circle. Simultaneously, using the line connecting the camera device and the target object as the 0° baseline, six illumination angles—0°, 60°, 120°, 180°, -60°, and -120°—are preset. When the flash moves to the corresponding illumination angle on the circle, the camera device is controlled to capture an image containing the target object, thus obtaining multiple images of the target object at different illumination angles.

[0091] See also Figure 4 The multiple images to be processed are classified. When the illumination angle is 0°, 60°, and -60°, the resulting images only reflect the reflection effect of the object on the light from the front. Therefore, these three images are considered as reflection-type images to be processed. When the illumination angle is 120°, 180°, and -120°, the resulting images not only reflect the reflection effect of the object on the light but also reflect the refraction effect of the object on the light from the back. Therefore, these three images can be considered as both reflection-type and refraction-type images to be processed.

[0092] See also Figure 4 During model training, reflection-type images can be input into the reflection attribute determination sub-model. These images are merged using a specific programming language and then encoded to obtain the corresponding latent variables. Further, after four decoding processes, the object's reflective material, reflective color, normal map of the reflection-type image, and depth values ​​of each pixel are obtained. Based on the reflection attribute association information, the object's reflection map can be obtained. Similarly, the obtained reflection attribute association information, combined with the refraction-type image, is input into the refraction attribute determination sub-model. Through a similar processing procedure, the object's refraction material, normal map of the refraction-type image, and depth values ​​of each pixel are obtained. Based on the refraction attribute association information, the object's refraction map can be obtained. Those skilled in the art should understand that the processing procedures for reflection-type and refraction-type images are the same as described above in subsequent model applications, and will not be repeated here.

[0093] In this embodiment, multiple images can be drawn based on the reflection attribute information and the illumination angle corresponding to the reflection image, serving as reflection images. Similarly, multiple refraction images can be drawn based on the refraction attribute information and the illumination angle corresponding to the refraction image.

[0094] See also Figure 4 During model training, based on the obtained reflection map, refraction map, and the training image of the object, the sub-models for determining reflection and refraction attributes can be modified respectively. Specifically, the actual reflection / refraction rendering images and the training image can be processed at illumination angles of 0°, 60°, 120°, 180°, -60°, and -120° to determine the corresponding loss value for each illumination angle. For example, when the illumination angle is 120°, the loss value under reflection and the loss value under refraction can be determined. Further, the parameters in the training reflection attribute determination model can be adjusted based on the first preset loss function. The parameters in the training refraction attribute determination model can be adjusted based on the second preset loss function. When the loss function converges, it is determined that the corresponding training model meets the requirements of practical application. If it does not converge, the training image and the rendering image corresponding to one of the illumination angles can be processed again, for example... When the illumination angle is 60°, a first preset loss function can be determined based on the loss value under the reflection condition corresponding to the illumination angle and the reflection attribute determination sub-model to be trained. A second preset loss function can be determined based on the loss value under the refraction condition corresponding to the illumination angle and the refraction attribute determination sub-model to be trained. Furthermore, when both loss functions converge, it is determined that the training model under the illumination angle has been completed.

[0095] S330: Acquire multiple images to be processed.

[0096] S340. Based on the target object's attributes, determine the model to process the image to be processed, and obtain the target object's attribute information.

[0097] S350. Based on the object attribute information, determine the target 3D view corresponding to the target object.

[0098] The technical solution of this embodiment obtains multiple training samples, and trains the sub-model for determining the reflection attribute and the sub-model for determining the refraction attribute based on the training samples. After the model training is completed, the object attribute information corresponding to the object is obtained based on these models, and then a target three-dimensional view that is closer to the real effect of the object is constructed.

[0099] Example 4

[0100] Figure 5 This is a structural block diagram of an image drawing apparatus provided in Embodiment 4 of this disclosure. It can execute the image drawing method provided in any embodiment of this disclosure, and possesses the corresponding functional modules and beneficial effects for executing the method. For example... Figure 5 As shown, the device specifically includes: an image acquisition module 410, an object attribute information determination module 420, and a target 3D view determination module 430.

[0101] The image acquisition module 410 is used to acquire multiple images to be processed; wherein the illumination angle between the light source and the target object is different in each image to be processed.

[0102] The object attribute information determination module 420 is used to process the image to be processed according to the target object attribute determination model to obtain the object attribute information of the target object.

[0103] The target 3D view determination module 430 is used to determine the target 3D view corresponding to the target object based on the object attribute information.

[0104] Optionally, the image acquisition module 410 is further configured to sequentially illuminate the target object based on the illumination angles in a preset illumination angle set, and acquire an image to be processed including the target object.

[0105] Based on the above technical solutions, the target object attribute determination model includes a reflection attribute determination sub-model and a refraction attribute determination sub-model, and the object attribute information determination module 420 includes a reflection attribute association information determination unit, a refraction attribute association information determination unit, and an object attribute information determination unit.

[0106] The reflection attribute association information determination unit is used to process the image to be processed based on the reflection attribute determination sub-model to obtain the reflection attribute association information corresponding to the target object.

[0107] The refraction attribute association information determination unit is used to process the reflection attribute association information and the image to be processed based on the refraction attribute determination sub-model to obtain the refraction attribute association information corresponding to the target object.

[0108] The object attribute information determination unit is used to determine the object attribute information based on the reflection-related attribute information and the refraction-related attribute information.

[0109] Based on the above technical solutions, the reflection-related attribute information includes the reflective material, reflective color, reflection type, normal map of the image to be processed, and depth value of each pixel of the target object; the refraction-related attribute information includes the refraction material, normal map of the image to be processed, and depth value of each pixel of the target object.

[0110] Based on the above technical solutions, the image rendering device also includes a training sample acquisition module, an actual associated attribute information determination module, an actual rendered image determination module, and an object attribute determination model determination module.

[0111] The training sample acquisition module is used to acquire multiple training samples; wherein each training sample includes multiple images to be trained, and the images to be trained are taken of the object to be processed under illumination angles in a preset set of illumination angles.

[0112] The actual association attribute information determination module is used to input multiple images to be trained in the current training sample into the object attribute determination model to be trained, and obtain the actual association attribute information corresponding to the object to be processed.

[0113] The actual drawn image determination module is used to determine multiple actual drawn images based on the actual associated attribute information; wherein the number of actual drawn images is the same as the number of training images in the current training sample, and the illumination angle of the object to be processed in the actual drawn images is the same as the illumination angle of the training images.

[0114] The object attribute determination model module is used to obtain the target object attribute determination model by processing the actual drawn image and the image to be trained at the same illumination angle.

[0115] Based on the above technical solutions, the model for determining the attributes of the object to be trained includes a sub-model for determining the reflection attributes to be trained and a sub-model for determining the refraction attributes to be trained. The actual associated attribute information includes actual training refraction attribute information and actual training reflection attribute information. The actual associated attribute information determination module includes an actual training reflection attribute information determination unit and an actual training refraction attribute information determination unit.

[0116] The actual training reflection attribute information determination unit is used to process the image of the object to be trained in the current training sample based on the sub-model for determining the reflection attribute to be trained, so as to obtain the actual training reflection attribute information corresponding to the object to be trained.

[0117] The actual training refraction attribute information determination unit is used to process the image to be trained with refraction type in the current training sample and the actual training reflection attribute information based on the refraction attribute determination sub-model to be trained, so as to obtain the actual training refraction attribute information.

[0118] Based on the above technical solutions, the actual drawn image includes an actual reflection drawn image and an actual refraction drawn image, and the actual drawn image determination module includes an actual reflection drawn image determination unit and an actual refraction drawn image determination unit.

[0119] The actual reflection rendering image determination unit is used to determine the target illumination angle of the light source in the current training image for each type of reflection, and to determine the actual reflection rendering image that is consistent with the target illumination angle based on the actual training reflection attribute information in the actual associated attribute information.

[0120] The actual refraction drawing image determination unit is used to determine the target illumination angle of the light source in the current training image for each type of refraction, and to determine the actual refraction drawing image that is consistent with the target illumination angle based on the actual training refraction attribute information in the actual associated attribute information.

[0121] Optionally, the actual drawn image determination module is further configured to process the actual reflected drawn image and the image to be trained at the same illumination angle, determine the loss value, and adjust the model parameters in the sub-model to be trained based on the loss value and a first preset loss function corresponding to the sub-model to be trained for reflecting attribute determination; and to process the actual refracted drawn image and the image to be trained at the same illumination angle, determine the loss value, and adjust the model parameters in the sub-model to be trained for reflecting attribute determination based on the loss value and a second preset loss function corresponding to the sub-model to be trained for reflecting attribute determination; and to converge the first preset loss function and the second preset loss function as the training objective to obtain the target item attribute determination model.

[0122] Based on the above technical solutions, the object attribute information includes at least one of material information, shape information, color information, texture information, smoothness information, and transparency information.

[0123] Optionally, the target 3D view determination module 430 is also used to draw the target 3D view based on the editor using the material information, shape information, color information, texture information, smoothness information and transparency information as drawing parameters.

[0124] The technical solution provided in this embodiment first acquires multiple images of the target object under different lighting angles, then determines the model to process the images based on the target object's attributes to obtain the object attribute information, and finally determines the target 3D view corresponding to the target object based on the object attribute information. This not only achieves accurate estimation of object attribute information, but also enables reverse drawing of the object based on this information. At the same time, it ensures that the drawn target 3D view is as close as possible to the object's actual display effect in reality, so that users can appreciate the actual presentation effect of the object just by looking at the target 3D view, thus improving the user experience.

[0125] The image drawing apparatus provided in this disclosure can execute the image drawing method provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the method.

[0126] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0127] Example 5

[0128] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of this disclosure. Refer to the following... Figure 6 It illustrates an electronic device suitable for implementing embodiments of the present disclosure (e.g., Figure 6 The diagram below shows the structure of the terminal device or server 500. The terminal device in this embodiment may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and vehicle terminals (e.g., vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 6 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0129] like Figure 6 As shown, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 506 into a random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An edit / output (I / O) interface 505 is also connected to the bus 504.

[0130] Typically, the following devices can be connected to I / O interface 505: editing devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 506 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 6 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0131] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 509, or installed from storage device 506, or installed from ROM 502. When the computer program is executed by processing device 501, it performs the functions defined in the methods of embodiments of this disclosure.

[0132] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0133] The electronic device provided in this embodiment and the image drawing method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0134] Example 6

[0135] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the image drawing method provided in the above embodiments.

[0136] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0137] In some implementations, clients and servers may communicate using any currently known or future-developed network protocol, such as Hypertext Transfer Protocol (HTTP), and may interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0138] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0139] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to:

[0140] Acquire multiple images to be processed; in each image, the illumination angle between the light source and the target object is different.

[0141] The model determines the target object's attributes and processes the image to be processed to obtain the object attribute information of the target object.

[0142] Based on the object attribute information, a target 3D view corresponding to the target object is determined.

[0143] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0144] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0145] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0146] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0147] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0148] According to one or more embodiments of this disclosure, [Example 1] provides a method for drawing an image, the method comprising:

[0149] Acquire multiple images to be processed; in each image, the illumination angle between the light source and the target object is different.

[0150] The model determines the target object's attributes and processes the image to be processed to obtain the object attribute information of the target object.

[0151] Based on the object attribute information, a target 3D view corresponding to the target object is determined.

[0152] According to one or more embodiments of this disclosure, [Example 2] provides a method for drawing an image, further comprising:

[0153] The target object is illuminated sequentially based on the lighting angles in the preset lighting angle set, and an image to be processed including the target object is obtained.

[0154] According to one or more embodiments of this disclosure, [Example 3] provides a method for drawing an image, further comprising:

[0155] Based on the reflection attribute determination sub-model, the image to be processed is of the reflection type to obtain the reflection attribute association information corresponding to the target object;

[0156] Based on the refraction property determination sub-model, the reflection property association information and the image to be processed based on the refraction type are processed to obtain the refraction property association information corresponding to the target object;

[0157] Based on the reflection-related attribute information and the refraction-related attribute information, the object attribute information is determined.

[0158] According to one or more embodiments of this disclosure, [Example 4] provides a method for drawing an image, further comprising:

[0159] Optionally, the reflection-related attribute information includes the reflective material, reflective color, reflection type, normal map of the image to be processed, and depth value of each pixel of the target object; the refraction-related attribute information includes the refraction material, normal map of the image to be processed, and depth value of each pixel of the target object.

[0160] According to one or more embodiments of this disclosure, [Example 5] provides a method for drawing an image, further comprising:

[0161] Multiple training samples are acquired; each training sample includes multiple images to be trained, which are images of the object to be processed under illumination angles in a preset set of illumination angles.

[0162] For each training sample, multiple images to be trained in the current training sample are input into the object attribute determination model to be trained, so as to obtain the actual associated attribute information corresponding to the object to be processed;

[0163] Based on the actual association attribute information, multiple actual drawn images are determined; wherein, the number of actual drawn images is the same as the number of images to be trained in the current training sample, and the illumination angle of the object to be processed in the actual drawn images is the same as the illumination angle of the images to be trained.

[0164] By processing the actual drawn image and the image to be trained at the same lighting angle, the target object attribute determination model is obtained.

[0165] According to one or more embodiments of this disclosure, [Example Six] provides a method for drawing an image, further comprising:

[0166] The model for determining the attributes of an object to be trained includes a sub-model for determining the reflection attributes to be trained and a sub-model for determining the refraction attributes to be trained. The actual associated attribute information includes actual training refraction attribute information and actual training reflection attribute information.

[0167] Optionally, a sub-model is determined based on the reflection attributes to be trained, and the images of the reflection type to be trained in the current training samples are processed to obtain the actual training reflection attribute information corresponding to the item to be trained.

[0168] Based on the refraction attribute determination sub-model, the refraction type of the training image in the current training sample and the actual training reflection attribute information are processed to obtain the actual training refraction attribute information.

[0169] According to one or more embodiments of this disclosure, [Example Seven] provides a method for drawing an image, further comprising:

[0170] The actual drawn image includes the actual reflection drawn image and the actual refraction drawn image.

[0171] Optionally, for each type of reflection in the training image, the target illumination angle of the light source in the current training image is determined, and based on the actual training reflection attribute information in the actual associated attribute information, an actual reflection drawing image consistent with the target illumination angle is determined.

[0172] For each type of refraction in the training image, the target illumination angle of the light source in the current training image is determined, and based on the actual training refraction attribute information in the actual associated attribute information, the actual refraction drawing image that is consistent with the target illumination angle is determined.

[0173] According to one or more embodiments of this disclosure, [Example Eight] provides a method for drawing an image, further comprising:

[0174] Optionally, the actual reflection rendering image and the image to be trained at the same illumination angle are processed to determine a loss value, and the model parameters in the sub-model corresponding to the reflection attribute to be trained are adjusted based on the loss value and a first preset loss function.

[0175] The actual refraction image and the image to be trained at the same illumination angle are processed to determine the loss value. Based on the loss value and the second preset loss function corresponding to the sub-model of the reflection attribute to be trained, the model parameters in the sub-model of the reflection attribute to be trained are adjusted.

[0176] The convergence of the first preset loss function and the second preset loss function is used as the training objective to obtain the target item attribute determination model.

[0177] According to one or more embodiments of this disclosure, [Example Nine] provides a method for drawing an image, further comprising:

[0178] Optionally, the object attribute information includes at least one of material information, shape information, color information, texture information, smoothness information, and transparency information.

[0179] According to one or more embodiments of this disclosure, [Example 10] provides a method for drawing an image, further comprising:

[0180] Optionally, the target 3D view can be drawn using the editor with the material information, shape information, color information, texture information, smoothness information, and transparency information as drawing parameters.

[0181] According to one or more embodiments of this disclosure, [Example 11] provides an apparatus for drawing an image, comprising:

[0182] The image acquisition module is used to acquire multiple images to be processed; in each image, the illumination angle between the light source and the target object is different.

[0183] The object attribute information determination module is used to process the image to be processed according to the target object attribute determination model to obtain the object attribute information of the target object;

[0184] The target 3D view determination module is used to determine the target 3D view corresponding to the target object based on the object attribute information.

[0185] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0186] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0187] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for drawing an image, characterized in that, include: Multiple images to be processed are acquired; wherein the illumination angle between the light source and the target object is different in each image to be processed, and the target object is a semi-transparent material. The model determines the target object's attributes and processes the image to be processed to obtain the target object's attribute information. Based on the object attribute information, a target 3D view corresponding to the target object is determined; The target object attribute determination model includes a reflection attribute determination sub-model and a refraction attribute determination sub-model. The process of processing the image to be processed according to the target object attribute determination model to obtain the object attribute information of the target object includes: Based on the reflection attribute determination sub-model, the image to be processed is of the reflection type to obtain the reflection attribute association information corresponding to the target object; Based on the refraction property determination sub-model, the image to be processed is processed with the reflection property association information and refraction type to obtain the refraction property association information corresponding to the target object; Based on the reflection attribute association information and the refraction attribute association information, the object attribute information is determined.

2. The method according to claim 1, characterized in that, Acquire images of the target object under different lighting angles, including: The target object is illuminated sequentially based on the lighting angles in the preset lighting angle set, and an image to be processed including the target object is obtained.

3. The method according to claim 1, characterized in that, The reflection attribute association information includes the reflective material, reflective color, reflection type, normal map of the image to be processed, and depth value of each pixel of the target object; the refraction attribute association information includes the refraction material, refraction type, normal map of the image to be processed, and depth value of each pixel of the target object.

4. The method according to claim 1, characterized in that, Also includes: Multiple training samples are acquired; each training sample includes multiple images to be trained, which are images of the object to be processed under illumination angles in a preset set of illumination angles. For each training sample, multiple images to be trained in the current training sample are input into the object attribute determination model to be trained, so as to obtain the actual associated attribute information corresponding to the object to be processed; Based on the actual association attribute information, multiple actual drawn images are determined; wherein, the number of actual drawn images is the same as the number of images to be trained in the current training sample, and the illumination angle of the object to be processed in the actual drawn images is the same as the illumination angle of the images to be trained. By processing the actual drawn image and the image to be trained at the same lighting angle, the target object attribute determination model is obtained.

5. The method according to claim 4, characterized in that, The object attribute determination model to be trained includes a sub-model for determining reflection attributes and a sub-model for determining refraction attributes. The actual associated attribute information includes actual training refraction attribute information and actual training reflection attribute information. The step of inputting multiple images to be trained from the current training samples into the object attribute determination model to be trained, and obtaining the actual associated attribute information corresponding to the object to be processed, includes: Based on the reflection attributes to be trained, a sub-model is determined, and the reflection type of the image to be trained in the current training sample is processed to obtain the actual training reflection attribute information corresponding to the object to be trained. Based on the refraction attribute determination sub-model, the refraction type of the training image in the current training sample and the actual training reflection attribute information are processed to obtain the actual training refraction attribute information.

6. The method according to claim 5, characterized in that, The actual drawn image includes an actual reflection drawn image and an actual refraction drawn image. The determination of multiple actual drawn images based on the actual associated attribute information includes: For each type of reflection in the training image, the target illumination angle of the light source in the current training image is determined, and based on the actual training reflection attribute information in the actual associated attribute information, the actual reflection drawing image that is consistent with the target illumination angle is determined. For each type of refraction in the training image, the target illumination angle of the light source in the current training image is determined, and based on the actual training refraction attribute information in the actual associated attribute information, the actual refraction drawing image that is consistent with the target illumination angle is determined.

7. The method according to claim 6, characterized in that, The process of processing actual drawn images and training images at the same illumination angle to obtain the target object attribute determination model includes: The actual reflection image and the image to be trained at the same illumination angle are processed to determine a loss value. Based on the loss value and a first preset loss function corresponding to the reflection attribute to be trained, the model parameters in the reflection attribute to be trained sub-model are adjusted. The actual refraction image and the image to be trained at the same illumination angle are processed to determine the loss value. Based on the loss value and the second preset loss function corresponding to the sub-model of the reflection attribute to be trained, the model parameters in the sub-model of the reflection attribute to be trained are adjusted. The convergence of the first preset loss function and the second preset loss function is used as the training objective to obtain the target object attribute determination model.

8. The method according to claim 1, characterized in that, The object attribute information includes at least one of the following: material information, shape information, color information, texture information, smoothness information, and transparency information.

9. The method according to claim 8, characterized in that, Determining the target 3D view corresponding to the target object based on the object attribute information includes: The editor uses the material information, shape information, color information, texture information, smoothness information, and transparency information as drawing parameters to draw the target 3D view.

10. An apparatus for drawing images, characterized in that, include: The image acquisition module is used to acquire multiple images to be processed; wherein, the illumination angle between the light source and the target object is different in each image to be processed, and the target object is a semi-transparent material. The object attribute information determination module is used to process the image to be processed according to the target object attribute determination model to obtain the object attribute information of the target object; The target 3D view determination module is used to determine the target 3D view corresponding to the target object based on the object attribute information; The target object attribute determination model includes a reflection attribute determination sub-model and a refraction attribute determination sub-model, and the object attribute information determination module includes a reflection attribute association information determination unit, a refraction attribute association information determination unit, and an object attribute information determination unit. The reflection attribute association information determination unit is used to process the image to be processed based on the reflection attribute determination sub-model to obtain the reflection attribute association information corresponding to the target object; The refraction attribute association information determination unit is used to process the image to be processed based on the refraction attribute determination sub-model, the reflection attribute association information and the refraction type, to obtain the refraction attribute association information corresponding to the target object. The object attribute information determination unit is used to determine the object attribute information based on the reflection attribute association information and the refraction attribute association information.

11. An electronic device, characterized in that, The electronic device includes: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method of drawing an image as described in any one of claims 1-9.

12. A storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to perform the method of drawing an image as described in any one of claims 1-9.

Images