Shadow estimation method and device, electronic equipment and readable storage medium

Abstract

This application discloses a shadow estimation method, apparatus, electronic device, and readable storage medium, belonging to the field of artificial intelligence technology. The method involves: acquiring a first virtual image corresponding to a target virtual space; rendering a target virtual object in the target virtual space using the first virtual image as a background to obtain a first mapping image corresponding to the target virtual space; determining a material estimation image of the first mapping image, the material estimation image indicating the material reflectivity of the object in the first mapping image; and estimating the shadow parameters of the target virtual object based on the material estimation image.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to a shadow estimation method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] With the development of technology, augmented reality (AR) is being used more and more widely.

[0003] AR (Augmented Reality) is a technology that renders virtual objects in AR space. Among the key aspects, the rendering of shadows for virtual objects is a crucial criterion for assessing the realism of the rendering. However, current technologies lack a specific method for obtaining the shadow parameters of virtual objects, hindering accurate shadow rendering in AR space. Therefore, determining how to obtain these shadow parameters is a pressing issue that needs to be addressed. Summary of the Invention

[0004] The purpose of this application is to provide a shadow estimation method, apparatus, electronic device, and readable storage medium capable of obtaining shadow parameters of virtual objects.

[0005] In a first aspect, embodiments of this application provide a shadow estimation method, the method comprising: acquiring a first virtual image corresponding to a target virtual space; and rendering a target virtual object in the target virtual space with the first virtual image as a background to acquire a first mapping image corresponding to the target virtual space; determining a material estimation image of the first mapping image, the material estimation image being used to indicate the material reflectivity of the object in the first mapping image; and estimating shadow parameters of the target virtual object based on the material estimation image.

[0006] Secondly, embodiments of this application provide a shadow estimation device, which includes an acquisition module and a processing module. The acquisition module is configured to acquire a first virtual image corresponding to a target virtual space; the processing module is configured to render a target virtual object in the target virtual space using the first virtual image acquired by the acquisition module as a background; the acquisition module is further configured to acquire a first mapping image corresponding to the target virtual space after the processing module renders the target virtual object; the processing module is further configured to estimate a material estimation image of the first mapping image acquired by the acquisition module, the material estimation image indicating the material reflectivity of the object in the first mapping image; the processing module is further configured to estimate shadow parameters of the target virtual object based on the material estimation image.

[0007] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0008] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0009] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0010] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0011] In this embodiment, a first virtual image corresponding to the target virtual space is obtained; and using the first virtual image as a background, a target virtual object is rendered once in the target virtual space to obtain a first mapping image corresponding to the target virtual space; a material estimation image of the first mapping image is determined, which is used to indicate the material reflectivity of the object in the first mapping image; and the shadow parameters of the target virtual object are estimated based on the material estimation image. Through this scheme, since the first virtual image corresponding to the target virtual space is used as a background, after rendering the target virtual object once in the target virtual space, a first mapping image including the first virtual image and the target virtual object can be obtained. Therefore, the material estimation image output after inputting the first mapping image into the material estimation model can reflect the material reflectivity of the target virtual object and the material reflectivity of the object in the first virtual image, thereby ensuring that the shadow parameters of the target virtual object can be accurately estimated based on the material estimation image. Attached Figure Description

[0012] Figure 1 This is one of the flowcharts illustrating the shadow estimation method provided in the embodiments of this application;

[0013] Figure 2 This is a schematic diagram of the structure of the general material estimation model in the shadow estimation method provided in the embodiments of this application;

[0014] Figure 3 This is a second schematic flowchart of the shadow estimation method provided in the embodiments of this application;

[0015] Figure 4 This is a schematic diagram of the rendering process of virtual objects in AR space based on the shadow estimation method provided in the embodiments of this application;

[0016] Figure 5 This is a schematic diagram of the shadow estimation device provided in the embodiments of this application;

[0017] Figure 6 This is one of the structural schematic diagrams of an electronic device provided based on an embodiment of this application;

[0018] Figure 7 This is a second schematic diagram of the structure of an electronic device provided in the embodiments of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0020] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0021] The technical terms involved in the technical solutions provided in the embodiments of this application will be explained below.

[0022] Augmented Reality (AR): Also known as Augmented Reality, AR is a relatively new technology that integrates information from the real world and the virtual world. It uses computer technology to simulate and overlay virtual information onto the real world, making it perceptible to human senses and creating a sensory experience that transcends reality. The real environment and virtual objects can coexist simultaneously in the same scene and space after being superimposed.

[0023] Hard shadows: Sharp shadows on the edges of virtual objects in AR scenes, typically the shadow effect of virtual objects under strong light.

[0024] Soft shadows: Compared to hard shadows, soft shadows are the shadows with blurred edges of virtual objects in AR scenes. They are generally the shadow effects of virtual objects under relatively soft lighting conditions.

[0025] Self-shadow: The shadow rendering effect caused by the reflective properties of the material itself of a virtual object.

[0026] Reflectivity (Albedo): An essential property of a material, physically defined as the percentage of radiant energy (such as light energy) reflected by the material out of the total radiant energy.

[0027] The shadow estimation method, apparatus, electronic device, and readable storage medium provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0028] Currently, rendering realistic virtual digital objects in real-world scenes remains a significant challenge for augmented reality. Besides more realistic modeling, the industry's main approach involves using neural network models to predict lighting parameters in real time and then providing these parameters to the renderer to achieve more realistic rendering of virtual digital objects. However, shadows are another crucial factor in the realism of virtual digital object rendering, and related technologies have not adequately addressed this issue. Specifically, they lack a concrete solution for determining the shadows of virtual digital objects.

[0029] The purpose of this application is to provide a shadow estimation method. For a virtual object that needs to be rendered in virtual space, it can first be rendered without shadows in virtual space (hereinafter referred to as the first rendering for ease of description), and then a first mapping image corresponding to the virtual space can be obtained, and the material estimation map of the first mapping image can be estimated. Since the first mapping image includes the virtual object, the material estimation map can reflect the material reflectivity of the virtual object. Thus, the shadow parameters of the target virtual object can be estimated based on the material estimation map.

[0030] The first rendering process is as follows: based on the time-series images (i.e., preview images, i.e., virtual images corresponding to the virtual space) provided by the device's camera acquisition module, the virtual objects are rendered for the first time in the virtual space. At this time, the rendering method has no shadow parameters, and the rendering result is scene image 1 containing virtual objects (e.g., AR scene image). Scene image 1 is preprocessed to obtain the first mapping image.

[0031] Therefore, the first mapped image can be input into the general material estimation model, and the output will be the material estimation image corresponding to the first mapped image.

[0032] After completing the first rendering, the device can estimate whether self-shadows need to be rendered for the virtual object based on the material estimation image, the estimated intensity coefficient of the main light source in the virtual space, and the first mask image corresponding to the virtual object in the first mapping image.

[0033] Furthermore, when it is necessary to estimate the shadow type or shadow intensity of a virtual object, the device can render the initial shadow (specifically, a hard shadow) of the virtual object in the virtual space based on the estimated main light source direction of the virtual space and the spatial information of the virtual object (which can indicate the shape, volume, and three-dimensional contour information of the virtual object's position in the target virtual space). The rendering result is a scene image 2 containing the initial shadow (hard shadow) of the virtual object; then, the second mask image corresponding to the initial shadow is segmented from the scene image 2.

[0034] After completing the second rendering, the device can combine the second mask image and the estimated material map to estimate the shadow type of the virtual object; and / or, based on the estimated main light source intensity coefficient of the virtual space, the estimated material map and the second mask image, estimate the shadow intensity of the virtual object, the shadow intensity including soft shadow or hard shadow.

[0035] It is understood that the shadow parameters of a virtual object include at least one of the following: whether to render a shadow for the virtual object, the shadow type of the virtual object, and the shadow intensity of the virtual object.

[0036] After obtaining the shadow parameters of the virtual object, these shadow parameters can be used as a step in the third rendering input of the renderer to achieve a realistic shadow rendering effect for the virtual object.

[0037] This application provides a shadow estimation method. Figure 1 A flowchart of a shadow estimation method provided in an embodiment of this application is shown, with an electronic device as an example of the method being implemented. Figure 1 As shown, the shadow estimation method provided in this application embodiment may include the following steps 101 to 104.

[0038] Step 101: The electronic device acquires the first virtual image corresponding to the target virtual space.

[0039] Alternatively, the target virtual space can be created based on the three-dimensional space of the environment in which the electronic device is located.

[0040] When using electronic devices, the device can overlay a target virtual space onto its surrounding environment to create an augmented reality space. This results in a display effect where a virtual space is superimposed on the environment in which the electronic device is located.

[0041] In this embodiment of the application, the first virtual image is an image of the environment in which the electronic device is located, captured in real time by the camera of the electronic device.

[0042] Optionally, the first virtual image can be a depth image captured in real time by a depth camera, or it can be an image synthesized from images captured in real time by multiple cameras by an electronic device. This allows for the rendering of realistic real-time shadow effects for virtual objects in virtual space.

[0043] It can be seen that the first virtual image reflects the real environment in which the electronic device is located.

[0044] Optionally, after the electronic device captures an image of the real environment through a camera, preprocessing operations can be performed on the captured image. Specifically, these preprocessing operations may include: cropping, rotating based on the screen orientation of the electronic device, resizing the image, and converting the image format.

[0045] Step 102: The electronic device renders the target virtual object once in the target virtual space with the first virtual image as the background, so as to obtain the first mapped image corresponding to the target virtual space.

[0046] In this embodiment, the electronic device can use an image of the real environment in which the electronic device is located (i.e., the first virtual image) acquired in real time as the background to render a target virtual object in the target virtual space. This rendering process does not involve any shadow effects; it is solely for rendering the target virtual object in the target virtual space. This rendering process can also be considered as compositing the first virtual image with the target virtual object to obtain an AR scene image (hereinafter referred to as AR scene image 1), i.e., a virtual-real combined image, or a rendered image. Then, the electronic device can perform mapping processing on the AR scene image 1 to obtain a first mapped image.

[0047] The electronic device can perform mapping processing on the AR scene image 1 along the user's line of sight to obtain a first mapped image.

[0048] It can be seen that the first mapped image includes the target virtual object and the first virtual image, specifically the mapped target virtual object and the mapped first virtual image.

[0049] In this embodiment of the application, the first mapped image is a two-dimensional image.

[0050] Optionally, the first mapped image is a red (R) green (G) blue (B) image.

[0051] Optionally, the electronic device can input the spatial information of the first virtual image, the target virtual object, and the target virtual space into the rendering model for rendering, and output the AR scene image 1 mentioned above through the rendering model.

[0052] Step 103: The electronic device estimates the material estimation image of the first mapping map.

[0053] The aforementioned material estimation image can be used to indicate the material reflectivity of objects in the first mapped image.

[0054] It can be understood that "the reflectance of the material of an object in an image" can be understood as the reflectance of the material of the object in the image.

[0055] In this embodiment of the application, the material estimation image can indicate the material reflectance of each pixel in the first mapped image.

[0056] In this embodiment of the application, since the first mapping image includes the target virtual object and the first virtual image, the material estimation image can indicate the material reflectivity of the target virtual object and the material reflectivity of each object in the first virtual image.

[0057] In this embodiment, the material estimation image is a single-channel image.

[0058] Alternatively, the electronic device can estimate the material estimation map of the first mapped image using a general material estimation model.

[0059] Specifically, the electronic device can input the first mapped image into the general material estimation model and output the material estimation model of the first mapped image.

[0060] Optionally, the electronic device can input the RGB vector of the first mapped image into a general material estimation model and output single-channel material estimation data (i.e., material estimation image).

[0061] For example, assuming the first mapped image is a 572*572*3 image, the electronic device can input the RGB vector of the first mapped image into the general material estimation model and output a vector of 484*484*1 (i.e., material estimation data). The output vector represents the reflectivity of each object in the first mapped image. It can be seen that the first mapped image is a 3-channel image with a size of 572*572, while the material estimation image is a single-channel image with a size of 484*484.

[0062] This application provides a general material estimation model capable of estimating the material of virtual objects. The output of this general material estimation model is a complete material estimation image that reflects the material reflectivity of each object (including virtual objects) in the input image.

[0063] Alternatively, the general material estimation model can be a model trained on a neural network based on a training dataset.

[0064] The training dataset includes a training image set and a material image set of the training image set. The material reflectance set of the training image set includes the material image of each training image in the training image set. The material image of each training image can indicate the reflectance of each pixel in each training image.

[0065] The material images for training images can be acquired from the target device, which is a device capable of capturing the ground truth properties of images of different materials.

[0066] Optionally, such as Figure 2 As shown, the general material estimation model can include 7 network modules, namely, the first convolution module 21, the second convolution module 22, the third convolution module 23, the fourth convolution module 24, the first deconvolution module 25, the second deconvolution module 26, and the third deconvolution module 27.

[0067] For example, with Figure 2 The process of estimating the material estimation map of the initial image using the general material estimation model shown is explained. Specifically, for an initial input image, the input image is first processed by convolutional modules 21, 22, 23, and 24 in sequence. Then, the first deconvolution module 25 performs deconvolution processing on the output image of the fourth convolution module based on the output image of the third convolution module 23; the second deconvolution module 26 continues to perform deconvolution processing on the output image of the first deconvolution module based on the output image of the second convolution module 22; and the third deconvolution module 27 continues to perform deconvolution processing on the output image of the second deconvolution module based on the output image of the first convolution module 21, and outputs the material estimation map of the initial input image. Figure 2 The double-dotted line in the image indicates a copy operation. Figure 2 Images 28, 29, and 30 are copies of the output images of the third convolutional module 23, the second convolutional module 22, and the first convolutional module 21, respectively.

[0068] Of course, the above material estimation model can also include a loss function, such as the L2 Loss function.

[0069] For a description of training a general material estimation model and the purpose of the loss function, see the relevant description in the related techniques.

[0070] Step 104: The electronic device estimates the shadow parameters of the target virtual object based on the material estimation image.

[0071] Optionally, the shadow parameters of the target virtual object may include at least one of the following: a first parameter, shadow type, and shadow intensity.

[0072] The first parameter can be used to indicate whether the target virtual object requires rendering a shadow, the shadow type including soft shadow or hard shadow.

[0073] Thus, since at least one of the first parameter, shadow type, and shadow intensity of the target virtual object can be estimated, the flexibility in estimating the shadow parameters of the target virtual object can be improved.

[0074] For a detailed description of self-shadow, soft shadow, and hard shadow, please refer to the relevant descriptions in the glossary section above.

[0075] In the shadow estimation device provided in this application embodiment, since the first virtual image corresponding to the target virtual space is used as the background, after rendering the target virtual object in the target virtual space once, a first mapping image including the first virtual image and the target virtual object can be obtained. Therefore, the material estimation image output after inputting the first mapping image into the material estimation model can reflect the material reflectivity of the target virtual object and the material reflectivity of the object in the first virtual image, thereby ensuring that the shadow parameters of the target virtual object can be accurately estimated based on the material estimation image.

[0076] The shadow estimation method provided in the embodiments of this application will be described in detail below using three methods.

[0077] Optionally, in method 1, assuming the shadow parameters of the target virtual object include a first parameter, the electronic device estimates the shadow parameters of the target virtual object based on the material estimation image, which may include the following step A.

[0078] Step A: The electronic device estimates the first parameter of the target virtual object based on the material estimation image, the intensity coefficient of the target main light source, and the first mask image corresponding to the target virtual object in the first mapping image.

[0079] Among them, the target main light source intensity coefficient is the main light source intensity coefficient of the target virtual space estimated by the electronic device.

[0080] In this embodiment of the application, after obtaining the first mapped image, the electronic device obtains the first mask image of the target virtual object in the first mapped image based on the first mapped image.

[0081] In this embodiment of the application, the pixel value of the image region in the first mask image corresponding to the target virtual object in the first mapping image is a fixed value, for example, 1. This image region can be called a non-zero region; the pixel value of other regions in the first mask image other than this image region is 0.

[0082] In this embodiment, the target main light source intensity coefficient can specifically indicate the intensity of the main light source in the environment where the electronic device is located. For methods of estimating the target main light source intensity coefficient for electronic devices, please refer to related technologies; this application does not limit such methods.

[0083] Thus, since the electronic device can estimate the first parameter of the target virtual object based on the material estimation image, the intensity coefficient of the target main light source, and the first mask image, the electronic device can know whether it is necessary to render a self-shadow for the target virtual object in the target virtual space with the first virtual image as the background, thereby improving the rendering effect of the target virtual object.

[0084] Optionally, step A above can be implemented through steps A1 to A3 below.

[0085] Step A1: The electronic device determines the temporary light source reflection intensity image corresponding to the target virtual space based on the target main light source intensity coefficient and material estimation image.

[0086] Specifically, the electronic device can multiply the target main light source intensity coefficient by the pixel value of each pixel in the material estimation image to obtain a temporary light source reflection intensity image corresponding to the target virtual space. The temporary light source reflection intensity image indicates the reflection intensity of the main light source in the target virtual space.

[0087] Step A2: Based on the first mask image, the electronic device determines the first image region in the temporary light source reflection intensity image that corresponds to the target virtual object.

[0088] Specifically, the electronic device can perform a pixel-by-pixel AND operation (i.e., multiplication) on the temporary light source reflection intensity image and the first mask image, and determine the pixel image region where the pixel value of the temporary light source reflection intensity image is non-zero as the first image region. It can be understood that the purpose of performing the pixel-by-pixel AND operation on the temporary light source reflection intensity image and the first mask image is to retain the image region in the temporary light source reflection intensity image that corresponds to the non-zero region in the first mask image.

[0089] Step A3: The electronic device determines the first parameter based on the average pixel value of the first image region and the first pixel threshold.

[0090] Specifically, the electronic device can compare the average pixel value of the first image region with a first pixel threshold. If the average pixel value of the first image region is greater than the first pixel threshold, the electronic device determines that the first parameter indicates that the target virtual object needs to render a self-shadow; otherwise, the electronic device determines that the first parameter indicates that the target virtual object does not render a self-shadow.

[0091] In this embodiment of the application, the average pixel value of the first image region can be: the sum of the pixel values ​​of all pixels in the first image region divided by the number of pixels in the first image region.

[0092] The first pixel threshold can be set according to actual usage requirements, and is not limited in the embodiments of this application.

[0093] Thus, since the first image region is a temporary light source reflection intensity map of the target virtual object, if the average pixel value of the first image region is greater than the first pixel threshold, it indicates that the reflectivity of the target virtual object is relatively high; otherwise, it indicates that the reflectivity of the target virtual object is relatively low. Based on the average pixel value of the first image region and the first pixel threshold, it can be accurately determined whether the target virtual object requires rendering self-shadows.

[0094] Optionally, in method 2, assuming the shadow parameters of the target virtual object include the shadow type of the target virtual object, then before step 104 above, the shadow estimation method provided in this application embodiment may further include step 105 as described below. Furthermore, the electronic device estimating the shadow parameters of the target virtual object based on the material estimation image may include step B as described below.

[0095] Step 105: Based on the direction of the target main light source and the spatial information of the target virtual object, the electronic device renders the initial shadow of the target virtual object in the target virtual space to obtain the second mapping image corresponding to the target virtual space.

[0096] The target main light source direction refers to the direction of the main light source in the target virtual space estimated by the electronic device. This direction can indicate the direction of the main light source in the environment in which the electronic device is located. For methods used by the electronic device to estimate the target main light source direction, please refer to related technologies; this application does not limit such methods.

[0097] Optionally, the initial shadow of the target virtual object is specifically the hard shadow of the target virtual object.

[0098] In this embodiment, the electronic device can input the spatial information of the target virtual object, the spatial information of the target virtual space, and the direction of the target main light source into the rendering model, so as to render the initial shadow of the target virtual object in the target virtual space through the rendering model, and output an AR scene image (hereinafter referred to as AR scene image 2) through the rendering model. Then, the electronic device can perform mapping processing on the AR scene image 2 to obtain a second mapped image.

[0099] The electronic device can perform mapping processing on the AR scene image 2 along the user's line of sight to obtain a second mapped image.

[0100] It can be seen that the second mapped image includes the target virtual object, specifically the mapped target virtual object.

[0101] In this embodiment of the application, the second mapped image is a two-dimensional image.

[0102] In this embodiment of the application, the spatial information of the target virtual object can indicate the shape, volume, and position of the target virtual object in the target virtual space.

[0103] Step B: The electronic device estimates the shadow type based on the second mask image corresponding to the initial shadow in the material estimation image and the second mapping image.

[0104] In this embodiment of the application, after obtaining the second mapped image, the electronic device obtains the second mask image corresponding to the initial shadow in the second mapped image based on the second mapped image.

[0105] In this embodiment of the application, the pixel value of the image region in the second mask image corresponding to the initial shadow in the second mapped image is a fixed value, for example, 1, and the pixel value of other regions in the second mask image other than this image region is 0.

[0106] Thus, since the direction of the target main light source can indicate the direction of the main light source in the environment where the electronic device is located, the initial shadow of the target virtual object can be accurately rendered in the target virtual space based on the direction of the target main light source and the spatial information of the target virtual object. Therefore, the material estimation image and the mask image corresponding to the initial shadow can accurately determine the shadow type of the target virtual object.

[0107] Optionally, step B above can be implemented through steps B1 and B2 as described below.

[0108] Step B1: The electronic device determines the second image region in the material estimation image that corresponds to the initial shadow based on the second mask image.

[0109] Specifically, the electronic device can perform a pixel-by-pixel AND operation (i.e., multiplication) on the material estimation image and the second mask image, and determine the pixel image region with non-zero pixel values ​​in the resulting image as the first image region. It can be understood that the purpose of performing the pixel-by-pixel AND operation on the material estimation image and the second mask image is to retain the image region in the material estimation image that corresponds to the non-zero region in the second mask image.

[0110] Step B2: The electronic device determines the shadow type based on the average pixel value of the second image region and the second pixel threshold.

[0111] Specifically, the electronic device can compare the average pixel value of the second image region with a second pixel threshold. If the average pixel value of the second image region is greater than the second pixel threshold, the electronic device determines that the shadow type of the target virtual object is a hard shadow; otherwise, it determines that the shadow type of the target virtual object is a soft shadow.

[0112] The second pixel threshold can be the same as or different from the first pixel threshold, depending on the actual usage requirements.

[0113] In this embodiment, assuming the initial shadow of the target virtual object is rendered in space 1 of the target virtual space, then: the second image region can indicate the material reflectivity of the object in space 1 before the initial shadow is rendered. If the average pixel value of the second image region is greater than the second pixel threshold, it indicates that the material reflectivity of the object in space 1 is relatively high before the initial shadow is rendered, thus the impact on the material reflectivity of the object in space 1 is significant after the target virtual object blocks the light illuminating space 1; otherwise, the material reflectivity of the object in space 1 is relatively low before the initial shadow is rendered, thus the impact on the material reflectivity of the object in space 1 is also relatively low after the target virtual object blocks the light illuminating space 1.

[0114] Thus, since the second image region corresponds to the initial shadow, it can indicate the material reflectivity of the object in the target virtual space corresponding to the initial shadow before rendering the shadow of the target virtual object. Therefore, if the average pixel value of the second image region is greater than the second pixel threshold, it indicates that the shadow of the target virtual object has a significant impact on the reflectivity of the area used to render the initial shadow; otherwise, it indicates that the shadow of the target virtual object has a relatively small impact on the reflectivity of the area used to render the initial shadow. Based on the average pixel value of the second image region and the second pixel threshold, the shadow type of the target virtual object can be accurately determined.

[0115] Optionally, in method 3, assuming the shadow parameters of the target virtual object include the shadow type of the target virtual object, then before step 104 above, the shadow estimation method provided in this application embodiment may further include the following step 106. The electronic device estimating the shadow parameters of the target virtual object based on the material estimation image may include the following step C.

[0116] Step 106: Based on the direction of the target main light source and the spatial information of the target virtual object, the electronic device renders the initial shadow of the target virtual object in the target virtual space to obtain the second mapping image corresponding to the target virtual space.

[0117] For a detailed description of step 106, please refer to the relevant description in step 105 above. To avoid repetition, it will not be repeated here.

[0118] Step C: The electronic device estimates the shadow intensity based on the target main light source intensity coefficient, the material estimation image, and the second mask image corresponding to the initial shadow in the second mapping image.

[0119] Wherein, the target main light source direction is the main light source direction of the target virtual space estimated by the electronic device, and the target main light source intensity coefficient is the main light source intensity coefficient of the target virtual space estimated by the electronic device.

[0120] It should be noted that electronic devices can estimate the direction and intensity coefficient of the target's main light source based on illumination estimation algorithms.

[0121] For further descriptions of the target main light source direction and the target main light source intensity coefficient, please refer to the relevant descriptions in the above embodiments.

[0122] Thus, since the direction of the target main light source can indicate the direction of the main light source in the environment where the electronic device is located, the initial shadow of the target virtual object can be accurately rendered in the target virtual space based on the direction of the target main light source and the spatial information of the target virtual object. Furthermore, since the intensity coefficient of the target main light source can indicate the intensity of the main light source in the environment where the electronic device is located, the shadow intensity of the target virtual object can be accurately determined by combining the intensity coefficient of the target main light source, the material estimation image, and the mask image corresponding to the initial shadow.

[0123] Optionally, step C above can be implemented through steps C1 and C2 as described below.

[0124] Step C1: The electronic device determines the second image region where the shadow is located in the material estimation image based on the second mask image.

[0125] For a detailed description of step C1, please refer to the relevant description of step B1 in the above embodiments.

[0126] Step C2: The electronic device determines the shadow intensity based on the target main light source intensity coefficient and the average pixel value of the second image region.

[0127] Specifically, the electronic device can use the product of the average pixel value of the second image region and the intensity coefficient of the target main light source as the shadow intensity of the target virtual object.

[0128] It should be noted that electronic devices can also use the product of the target main light source intensity coefficient and the square root of the pixels in the second image region as the shadow intensity of the target virtual object.

[0129] Thus, since the second image region corresponds to the initial shadow, the second image region can indicate the material reflectivity of the object in the target virtual space corresponding to the initial shadow before rendering the shadow of the target virtual object. Therefore, the shadow intensity of the target virtual object can be accurately determined based on the target main light source intensity coefficient and the average pixel value of the second image region.

[0130] It should be noted that methods 1 to 3 above are illustrated using the example where the shadow parameters include the first parameter, shadow intensity, and shadow type, respectively. In actual implementation, when the shadow parameters include both shadow intensity and shadow type, steps 105 and 106 above are the same, that is, only the second mapping image is acquired once.

[0131] Optionally, after step 104 above, the shadow estimation method provided in this application embodiment may further include step 107 below.

[0132] Step 107: Based on the shadow parameters of the target virtual object, the electronic device performs secondary rendering of the target virtual object in the target virtual space with the first virtual image as the background, so as to display the spatial scene of the target virtual space.

[0133] Optionally, the electronic device can use the shadow parameters and lighting parameters (e.g., the direction and intensity coefficient of the target main light source), the target virtual object, the spatial information of the target virtual space, and the first virtual image as input to the renderer, so as to perform secondary rendering of the target virtual object in the target virtual space by the renderer, so as to display the spatial scene of the target virtual space rendered by the renderer.

[0134] Specifically, the renderer can apply parameters such as shadow intensity, shadow type, and whether to render self-shadows (i.e., the first parameter) in the main light source's attribute settings. Shadow intensity affects the overall brightness of the rendered shadows. Based on different shadow types, the renderer will select different shaders, and can render self-shadows or not based on the first parameter. This ultimately achieves a realistic shadow rendering effect. Thus, it can provide real-time, dynamically variable shadow parameters in AR scenes, including shadow type, shadow intensity, and self-shadow detection. These shadow parameters can be applied to the renderer, making the rendered virtual objects more realistic, thereby improving the user experience.

[0135] It should be noted that the electronic device can acquire shadow parameters once for each first virtual image acquired.

[0136] Thus, since the target virtual object can be rendered in the target virtual space based on the shadow parameters of the target virtual object and with the first virtual image as the background, a more realistic rendering effect of the target virtual object can be achieved.

[0137] The shadow estimation method provided in this application will be illustrated below with reference to specific embodiments.

[0138] This application proposes a shadow (parameter) estimation method. The shadow parameters may include shadow intensity, shadow type (soft shadow / hard shadow) and whether to render self-shadow (i.e., the first parameter). The aim is to solve the problem of rendering realistic shadows of digital objects in different AR scenes and provide a more realistic AR experience.

[0139] To achieve realistic real-time shadow effects in AR scene rendering, this application provides a real-time shadow parameter estimation method, such as... Figure 3 As shown, the method may include steps 301 to 306 as described below. The rendering process of virtual objects in AR space by the electronic device is as follows: Figure 4 As shown.

[0140] Step 301: Obtain camera preview image

[0141] In this embodiment, a camera preview image (e.g., a first virtual image) is acquired in real time from the camera preview stream. This acquired camera preview image is used in the next processing step. After acquiring the camera preview image, it can be preprocessed. The main preprocessing steps include image cropping, rotation based on the screen orientation of the mobile phone (i.e., electronic device), resizing the image, and image format transformation.

[0142] Step 302: Render the virtual object using the camera preview image as the background.

[0143] Using the preprocessed camera preview image from step 301 as a background, a virtual object is rendered in AR space. This rendering process does not involve any shadow effects; it is solely for rendering the virtual object in AR space. After rendering is complete, the electronic device can map the AR space containing the rendered virtual object along the viewing direction to obtain a mapped image 1. This mapped image 1 is used for subsequent material map estimation. This rendering can be completed using an AR renderer.

[0144] After obtaining the mapped image 1, the mask image of the virtual objects in the mapped image 1 can be further determined.

[0145] A mask image is a binary image where the masked region has a non-zero fixed value, and the non-masked region is set to zero.

[0146] Step 303: Obtain the material map containing the virtual object.

[0147] The rendered image synthesized in step 302 (i.e., mapped image 1) is used as input to the trained general material estimation model (also known as the general material estimation network) to obtain a material map containing the virtual object. This material map is single-channel data, representing the reflectivity properties of different materials in mapped image 1. This material map is used to find the intersection of the image with the shadow range of the virtual object.

[0148] For a description of the general material estimation model and how to train the general material model, please refer to the relevant descriptions in the above embodiments.

[0149] Step 304: Render and segment the shadows of the virtual object.

[0150] Based on the estimated main light source direction from the lighting estimation results, a second rendering is performed in the AR space. Specifically, this second rendering renders the initial shadow (specifically, a hard shadow) of the virtual object in the AR space. This rendering process only requires the spatial information of the virtual object (also known as 3D information), the main light source direction from the lighting estimation results, and the current spatial information of the AR scene as rendering input. The purpose of the second rendering is to obtain the mask image of the virtual object's shadow. After the second rendering is completed, the electronic device can perform mapping processing on the AR space containing the rendered virtual object along the viewing direction to obtain a mapped image 2. The electronic device can then determine the mask image of the virtual object in mapped image 2 and use this mask image as the shadow mask image of the virtual object.

[0151] Understandably, the second rendering was also done through the AR renderer.

[0152] Step 305: Obtain shadow parameters

[0153] The shadow parameters in this application embodiment include three parts: shadow intensity, shadow type (soft shadow / hard shadow), and whether to render self-shadow.

[0154] The following sections will explain the methods for estimating shadow intensity, shadow type, and whether to render self-shadows.

[0155] Step 305a: Estimate the shadow intensity of the virtual object.

[0156] Based on the estimated main light source intensity coefficient from the illumination estimation results, the material map from step 303, and the virtual object's mask map from step 302, the shadow intensity of the virtual object is estimated. The estimation process is as follows: The estimated pixel value is multiplied by the pixel value in the material map to obtain a temporary light source reflection intensity map. Then, a pixel-by-pixel AND operation is performed using this temporary light source reflection intensity map and the virtual object's mask map. This involves retaining the image regions corresponding to the non-zero regions in the temporary light source reflection intensity map and the virtual object's mask map, and calculating the average pixel value of the retained image regions by dividing the sum of all pixel values ​​in the retained image regions by the number of pixels in the retained image regions. If the determined average pixel value is greater than a preset pixel threshold, it is determined that the virtual object needs to render a self-shadow; otherwise, it is determined that the virtual object does not need to render a self-shadow. It can be seen that this estimation process yields parameter results for whether or not self-shadows are rendered.

[0157] Step 305b: Estimate the shadow type of the virtual object.

[0158] The shadow mask image obtained in step 304 is ANDed with the material image obtained in step 303. This operation is described in detail as follows: the material image and the shadow mask image are ANDed pixel-by-pixel, meaning the image regions in the material image corresponding to the non-zero regions in the retained shadow mask image are used. Then, the average pixel value of the retained image regions is calculated. If the determined average pixel value is greater than a set pixel threshold, it is determined that the virtual object requires hard shadows; otherwise, it is determined that the virtual object requires soft shadows. This process yields the shadow type of the virtual object.

[0159] Step 305c: Estimate the shadow intensity of the virtual object.

[0160] The product of the average pixel value determined in step 305b and the intensity coefficient of the main light source in the illumination estimation result is used to determine the shadow intensity of the virtual object.

[0161] Step 306: Render virtual objects based on shadow parameters

[0162] The shadow parameters estimated in step 305, including shadow intensity, shadow type (soft / hard shadow), and whether to render self-shadows, are used as input to the AR renderer. A third rendering is then performed using the AR renderer. This rendering process is described in detail as follows: The AR renderer receives these shadow parameters and applies them to the main light source's attribute parameters, including shadow intensity, shadow type, and whether to render self-shadows. This rendering process utilizes these parameters; specifically, shadow intensity affects the overall brightness and darkness of the shadow rendering result; based on different shadow types, the renderer selects different shaders and renders or does not render self-shadows based on the result of whether to render them. This achieves a realistic shadow rendering effect.

[0163] The shadow estimation method provided in this application can be executed by a shadow estimation device. This application uses the example of a shadow estimation device executing the shadow estimation method to illustrate the shadow estimation device provided in this application.

[0164] This application also provides a shadow estimation device. Figure 5 A schematic diagram of the structure of the shadow estimation device provided in an embodiment of this application is shown, as follows: Figure 5 As shown, the shadow estimation device 50 may include an acquisition module 51 and a processing module 52.

[0165] The acquisition module 51 can be used to acquire the first virtual image corresponding to the target virtual space;

[0166] The processing module 52 can be used to render the target virtual object once in the target virtual space, using the first virtual image obtained by the acquisition module 51 as the background.

[0167] The acquisition module 51 can also be used to acquire the first mapping image corresponding to the target virtual space after the processing module 52 renders the target virtual object once.

[0168] The processing module 52 can also be used to estimate the material estimation image of the first mapped image obtained by the acquisition module 51. The material estimation image can be used to indicate the material reflectivity of the object in the first mapped image.

[0169] The processing module 52 can also be used to estimate the shadow parameters of the target virtual object based on the material estimation image.

[0170] In one possible implementation, the shadow parameters mentioned above include at least one of the following: a first parameter, a shadow type, and a shadow intensity;

[0171] The first parameter can be used to indicate whether the target virtual object requires rendering a shadow, and the shadow type includes soft shadow or hard shadow.

[0172] In one possible implementation, the shadow parameter includes the first parameter;

[0173] The processing module 52 can be specifically used to estimate the first parameter based on the material estimation image, the target main light source intensity coefficient and the first mask image corresponding to the target virtual object in the first mapping image;

[0174] Wherein, the target main light source intensity coefficient is the estimated main light source intensity coefficient of the target virtual space.

[0175] In one possible implementation, the processing module 52 described above can be used to: determine a temporary light source reflection intensity image corresponding to the target virtual space based on the target main light source intensity coefficient and the material estimation image; and determine a first image region in the temporary light source reflection intensity image corresponding to the target virtual object based on the first mask image; and determine the first parameter based on the average pixel value of the first image region and a first pixel threshold.

[0176] In one possible implementation, the shadow parameters include the shadow type;

[0177] The processing module 52 can also be used to render the initial shadow of the target virtual object in the target virtual space based on the target main light source direction and the spatial information of the target virtual object before estimating the shadow parameters of the target virtual object based on the material estimation image;

[0178] The acquisition module 51 can also be used to acquire the second mapping image corresponding to the target virtual space after the processing module 52 renders the initial shadow of the target virtual object in the target virtual space;

[0179] The processing module 52 can be specifically used to estimate the shadow type based on the second mask image corresponding to the initial shadow in the material estimation image and the second mapping image;

[0180] Wherein, the direction of the target main light source is the estimated direction of the main light source in the target virtual space.

[0181] In one possible implementation, the processing module 52 described above can be used to: determine a second image region in the material estimation image corresponding to the initial shadow based on the second mask image; and determine the shadow type based on the average pixel value and the second pixel threshold of the second image region.

[0182] In one possible implementation, the shadow parameters include the shadow intensity;

[0183] The processing module 52 can also be used to render the initial shadow of the target virtual object in the target virtual space based on the target main light source direction and the spatial information of the target virtual object before estimating the shadow parameters of the target virtual object based on the material estimation image, so as to obtain the second mapping image corresponding to the target virtual space;

[0184] The processing module 52 can be specifically used to estimate the shadow intensity based on the target main light source intensity coefficient, the material estimation image and the second mask image corresponding to the initial shadow in the second mapping image;

[0185] Wherein, the target main light source direction is the estimated main light source direction of the target virtual space, and the target main light source intensity coefficient is the estimated main light source intensity coefficient of the target virtual space.

[0186] In one possible implementation, the processing module 52 can be specifically used to determine the second image region in the material estimation image corresponding to the initial shadow based on the second mask image; and to determine the shadow intensity based on the target main light source intensity coefficient and the average pixel value of the second image region.

[0187] In one possible implementation, the device 50 further includes a display module;

[0188] The processing module 52 can also be used to perform secondary rendering of the target virtual object in the target virtual space based on the shadow parameters of the target virtual object after estimating the shadow parameters based on the material estimation image and using the first virtual image as the background.

[0189] The display module can be used to display the spatial scene of the target virtual space after the processing module 52 performs secondary rendering of the target virtual object in the target virtual space.

[0190] In the shadow estimation device provided in this application embodiment, since the first virtual image corresponding to the target virtual space is used as the background, after rendering the target virtual object in the target virtual space once, a first mapping image including the first virtual image and the target virtual object can be obtained. Therefore, the material estimation image output after inputting the first mapping image into the material estimation model can reflect the material reflectivity of the target virtual object and the material reflectivity of the object in the first virtual image, thereby ensuring that the shadow parameters of the target virtual object can be accurately estimated based on the material estimation image.

[0191] The shadow estimation device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0192] The shadow estimation device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system used.

[0193] The shadow estimation device provided in this application embodiment can achieve... Figures 1 to 4 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0194] Optionally, such as Figure 6 As shown, this application embodiment also provides an electronic device 600, including a processor 601 and a memory 602. The memory 602 stores a program or instructions that can run on the processor 601. When the program or instructions are executed by the processor 601, they implement the various steps of the above-described shadow estimation method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0195] Figure 7 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0196] The electronic device 7000 includes, but is not limited to, the following components: radio frequency unit 7001, network module 7002, audio output unit 7003, input unit 7004, sensor 7005, display unit 7006, user input unit 7007, interface unit 7008, memory 7009, and processor 7010.

[0197] Those skilled in the art will understand that the electronic device 7000 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 7010 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 7 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0198] The processor 7010 can be used to obtain a first virtual image corresponding to the target virtual space, and with the first virtual image as the background, render the target virtual object once in the target virtual space to obtain a first mapping image corresponding to the target virtual space.

[0199] The processor 7010 can also be used to estimate a material estimation image of the first mapped image, which can be used to indicate the material reflectivity of an object in the first mapped image;

[0200] The processor 7010 can also be used to estimate the shadow parameters of a target virtual object based on a material estimation image.

[0201] In one possible implementation, the shadow parameters mentioned above include at least one of the following: a first parameter, a shadow type, and a shadow intensity;

[0202] The first parameter can be used to indicate whether the target virtual object requires rendering a shadow, and the shadow type includes soft shadow or hard shadow.

[0203] In one possible implementation, the shadow parameters include a first parameter; the processor 7010 can be specifically used to estimate the first parameter based on the material estimation image, the intensity coefficient of the target main light source, and the first mask image corresponding to the target virtual object in the first mapping image;

[0204] Among them, the target main light source intensity coefficient is the estimated main light source intensity coefficient of the target virtual space.

[0205] In one possible implementation, the processor 7010 can be specifically used to: determine the temporary light source reflection intensity image corresponding to the target virtual space based on the target main light source intensity coefficient and the material estimation image; and determine the first image region corresponding to the target virtual object in the temporary light source reflection intensity image based on the first mask image; and determine the first parameter based on the average pixel value of the first image region and the first pixel threshold.

[0206] In one possible implementation, the shadow parameters mentioned above include the shadow type;

[0207] The processor 7010 can also be used to render the initial shadow of the target virtual object in the target virtual space based on the direction of the target main light source and the spatial information of the target virtual object before estimating the shadow parameters of the target virtual object based on the material estimation image;

[0208] The processor 7010 can also be used to obtain the second mapping image corresponding to the target virtual space after rendering the initial shadow of the target virtual object in the target virtual space;

[0209] The processor 7010 can be used to estimate the shadow type based on the second mask image corresponding to the initial shadow in the material estimation image and the second mapping image;

[0210] The direction of the main light source of the target is the estimated direction of the main light source in the virtual space of the target.

[0211] In one possible implementation, the processor 7010 can be specifically used to: determine a second image region in the material estimation image corresponding to the initial shadow based on the second mask image; and determine the shadow type based on the average pixel value of the second image region and a second pixel threshold.

[0212] In one possible implementation, the shadow parameters mentioned above include shadow intensity;

[0213] The processor 7010 can also be used to render the initial shadow of the target virtual object in the target virtual space based on the direction of the target main light source and the spatial information of the target virtual object before estimating the shadow parameters of the target virtual object based on the material estimation image, so as to obtain the second mapping image corresponding to the target virtual space;

[0214] The processor 7010 can be used to estimate shadow intensity based on the second mask image corresponding to the initial shadow in the target main light source intensity coefficient, material estimation image, and second mapping image;

[0215] Wherein, the target main light source direction is the estimated target virtual space main light source direction, and the target main light source intensity coefficient is the estimated target virtual space main light source intensity coefficient.

[0216] In one possible implementation, the processor 7010 can be specifically used to determine a second image region in the material estimation image corresponding to the initial shadow based on the second mask image; and to determine the shadow intensity based on the target main light source intensity coefficient and the average pixel value of the second image region.

[0217] In one possible implementation, the processor 7010 can also be used to perform secondary rendering of the target virtual object in the target virtual space based on the shadow parameters of the target virtual object after estimating the shadow parameters of the target virtual object based on the material estimation image and with the first virtual image as the background.

[0218] Display unit 7006 can be used to display the spatial scene of the target virtual space after the processor 7010 performs secondary rendering of the target virtual object in the target virtual space.

[0219] In the electronic device provided in this application embodiment, since the first virtual image corresponding to the target virtual space is used as the background, after rendering the target virtual object in the target virtual space once, a first mapping image including the first virtual image and the target virtual object can be obtained. Therefore, the material estimation image output after inputting the first mapping image into the material estimation model can reflect the material reflectivity of the target virtual object and the material reflectivity of the object in the first virtual image, thereby ensuring that the shadow parameters of the target virtual object can be accurately estimated based on the material estimation image.

[0220] It should be understood that, in this embodiment, the input unit 7004 may include a graphics processing unit (GPU) 70041 and a microphone 70042. The GPU 70041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 7006 may include a display panel 70061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 7007 includes at least one of a touch panel 70071 and other input devices 70072. The touch panel 70071 is also called a touch screen. The touch panel 70071 may include a touch detection device and a touch controller. Other input devices 70072 may include, but are not limited to, a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, and a joystick, which will not be described in detail here.

[0221] The memory 7009 can be used to store software programs and various data. The memory 7009 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 7009 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 7009 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0222] Processor 7010 may include one or more processing units; optionally, processor 7010 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 7010.

[0223] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described shadow estimation method embodiments and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0224] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0225] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described shadow estimation method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0226] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0227] This application provides a computer program product stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the shadow estimation method embodiments described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0228] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0229] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0230] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A method of shadow estimation, characterized by, The method comprises: acquiring a first virtual image corresponding to a target virtual space; rendering a target virtual object in the target virtual space with the first virtual image as a background to acquire a first mapping image corresponding to the target virtual space; estimating a material estimation image of the first mapping image, the material estimation image being used to indicate a material reflectivity of an object in the first mapping image; estimating a shadow parameter of the target virtual object based on the material estimation image; the shadow parameter comprises a first parameter, the first parameter being used to indicate whether the target virtual object needs to render self-shadow; the estimating of the shadow parameter of the target virtual object based on the material estimation image comprises: determining a temporary light source reflection intensity image corresponding to the target virtual space based on a target main light source intensity coefficient and the material estimation image; determining a first image region corresponding to the target virtual object in the temporary light source reflection intensity image based on a first mask image corresponding to the target virtual object in the first mapping image; determining the first parameter based on an average pixel value of the first image region and a first pixel threshold value; wherein the target main light source intensity coefficient is an estimated main light source intensity coefficient of the target virtual space.

2. The method of claim 1, wherein, The shadow parameter comprises a shadow type, and the shadow type comprises a soft shadow or a hard shadow. Before the estimating of the shadow parameter of the target virtual object based on the material estimation image, the method further comprises: rendering an initial shadow of the target virtual object in the target virtual space based on a target main light source direction and spatial information of the target virtual object to acquire a second mapping image corresponding to the target virtual space, the spatial information of the target virtual object being used to indicate a shape of the target virtual object, a volume of the target virtual object, and a position of the target virtual object in the target virtual space; the estimating of the shadow parameter of the target virtual object based on the material estimation image comprises: estimating the shadow type based on the material estimation image and a second mask image corresponding to the initial shadow in the second mapping image; wherein the target main light source direction is an estimated main light source direction of the target virtual space.

3. The method of claim 2, wherein, The estimating of the shadow type based on the material estimation image and the second mask image corresponding to the initial shadow in the second mapping image comprises: determining a second image region corresponding to the initial shadow in the material estimation image based on the second mask image; determining the shadow type based on an average pixel value of the second image region and a second pixel threshold value.

4. The method of claim 1, wherein, The shadow parameter comprises a shadow intensity; Before the estimating of the shadow parameter of the target virtual object based on the material estimation image, the method further comprises: render an initial shadow of the target virtual object in the target virtual space based on a target main light source direction and spatial information of the target virtual object, to obtain a second mapping image corresponding to the target virtual space, the spatial information of the target virtual object being used to indicate a shape of the target virtual object, a volume of the target virtual object, and a position of the target virtual object in the target virtual space; the estimating the shadow parameter of the target virtual object based on the material estimation image comprises: estimating a shadow intensity based on a target main light source intensity coefficient, the material estimation image, and a second mask image corresponding to the initial shadow in the second mapping image; wherein the target main light source direction is an estimated main light source direction of the target virtual space, and the target main light source intensity coefficient is an estimated main light source intensity coefficient of the target virtual space.

5. The method of claim 4, wherein, the estimating the shadow intensity based on the target main light source intensity coefficient, the material estimation image, and the second mask image corresponding to the initial shadow in the second mapping image comprises: determining a second image region corresponding to the initial shadow in the material estimation image based on the second mask image; determining the shadow intensity based on the target main light source intensity coefficient and an average pixel value of the second image region.

6. The method of claim 1, wherein, after the estimating the shadow parameter of the target virtual object based on the material estimation image, the method further comprises: performing secondary rendering on the target virtual object in the target virtual space based on the shadow parameter and taking the first virtual image as a background, to display a spatial scene of the target virtual space.

7. A shadow estimation apparatus characterized by comprising: The device comprises an acquisition module and a processing module. The acquisition module is configured to acquire a first virtual image corresponding to a target virtual space. The processing module is configured to perform primary rendering on a target virtual object in the target virtual space, taking the first virtual image acquired by the acquisition module as a background. The acquisition module is further configured to acquire a first mapping image corresponding to the target virtual space after the processing module performs primary rendering on the target virtual object. The processing module is further configured to estimate a material estimation image of the first mapping image acquired by the acquisition module, the material estimation image being used to indicate a material reflectivity of an object in the first mapping image, and estimate a shadow parameter of the target virtual object based on the material estimation image. The shadow parameter comprises a first parameter, which is used to indicate whether the target virtual object needs to render self-shadow. The processing module is specifically configured to determine a temporary light source reflection intensity image corresponding to the target virtual space based on a target main light source intensity coefficient and the material estimation image, determine a first image region corresponding to the target virtual object in the temporary light source reflection intensity image based on a first mask image corresponding to the target virtual object in the first mapping image, and determine the first parameter based on an average pixel value of the first image region and a first pixel threshold. The target main light source intensity coefficient is a main light source intensity coefficient of the target virtual space estimated.

8. The apparatus of claim 7, wherein, The shadow parameter includes a shadow type, and the shadow type includes a soft shadow or a hard shadow. The processing module is further configured to, before estimating the shadow parameter of the target virtual object based on the material estimation image, render an initial shadow of the target virtual object in the target virtual space based on a target main light source direction and spatial information of the target virtual object, and the spatial information of the target virtual object is used to indicate a shape of the target virtual object, a volume of the target virtual object, and a position of the target virtual object in the target virtual space. The obtaining module is further configured to, after the processing module renders the initial shadow of the target virtual object in the target virtual space, obtain a second mapping image corresponding to the target virtual space. The processing module is specifically configured to estimate the shadow type based on the material estimation image and a second mask image corresponding to the initial shadow in the second mapping image. The target main light source direction is a main light source direction of the target virtual space estimated.

9. The apparatus of claim 8, wherein, The processing module is specifically configured to determine a second image region corresponding to the initial shadow in the material estimation image based on the second mask image, and determine the shadow type based on an average pixel value of the second image region and a second pixel threshold.

10. The apparatus of claim 7, wherein, The shadow parameter includes a shadow intensity. The processing module is further configured to, before estimating the shadow parameter of the target virtual object based on the material estimation image, render an initial shadow of the target virtual object in the target virtual space based on a target main light source direction and spatial information of the target virtual object, and the spatial information of the target virtual object is used to indicate a shape of the target virtual object, a volume of the target virtual object, and a position of the target virtual object in the target virtual space. The processing module is specifically configured to estimate the shadow intensity based on a target main light source intensity coefficient, the material estimation image, and a second mask image corresponding to the initial shadow in the second mapping image. The target main light source direction is a main light source direction of the target virtual space estimated, and the target main light source intensity coefficient is a main light source intensity coefficient of the target virtual space estimated.

11. The apparatus of claim 10, wherein, The processing module is specifically configured to determine a second image region corresponding to the initial shadow in the material estimation image based on the second mask image, and determine the shadow intensity based on the target main light source intensity coefficient and an average pixel value of the second image region.

12. The apparatus of claim 7, wherein, The device further includes a display module. The processing module is further configured to, after estimating the shadow parameter of the target virtual object based on the material estimation image, perform secondary rendering on the target virtual object in the target virtual space based on the shadow parameter and taking the first virtual image as a background. The display module is configured to display a space scene of the target virtual space after the processing module performs secondary rendering on the target virtual object in the target virtual space.

13. An electronic device, comprising: A processor and a memory are included, the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the shadow estimation method according to any one of claims 1 to 6.

14. A readable storage medium, characterized by, The readable storage medium stores programs or instructions, and the programs or instructions, when executed by a processor, implement the steps of the shadow estimation method according to any one of claims 1 to 6.

Images