Shadow map generation method and device, electronic equipment and computer storage medium

Abstract

The invention provides a shadow map generation method and device, electronic equipment and a computer storage medium, and the method comprises the steps: determining an influence value of a light source on an entity for each light source according to the illumination intensity of the light source and the distance between the light source and the entity after obtaining the light source information of each light source; the method comprises the following steps: taking a light source with the maximum influence value on an entity in all light sources as a main light source, then obtaining a depth map of the entity shot by a depth camera at the main light source and a rendering picture of a virtual camera, and then generating a scene depth map under the main light source according to the rendering picture; and finally, generating a first shadow map according to the depth map of the entity and the scene depth map under the main light source. In the actual application process, after the first shadow map is obtained, the first shadow map acts on the output picture of the virtual camera, so that the output virtual scene and the entity are combined more naturally, and the shooting effect is better.

Description

Technical Field

[0001] The present application relates to the field of virtual reality technology, and in particular to a method, device, electronic device and computer storage medium for generating a shadow map. Background Art

[0002] At present, virtual shooting has gradually become an important shooting method in various game and film and television industries. However, there is always a sense of division when virtual assets interact with real objects. One of the important reasons is that the projection of real light sources cannot be mapped to virtual scenes during green screen shooting. In particular, when real light sources are used as the main light sources of entities (people, objects, etc.), accurate shadows cannot be mapped in virtual scenes. At the same time, under the influence of different lights in virtual and real scenes, it is impossible to determine the main light source affecting the entity, which will make the final output shadow very unrealistic. Summary of the invention

[0003] In view of this, the present application provides a method, device, electronic device and computer storage medium for generating a shadow map, which accurately determines the main light source and generates a shadow map, so that the virtual scene and the entity are combined more naturally and the shooting effect is better.

[0004] The first aspect of the present application provides a method for generating a shadow map, comprising:

[0005] Obtaining light source information of each light source; wherein the light source is divided into a real light source and a virtual light source; the light source information includes light intensity and a distance between the light source and the entity;

[0006] For each light source, determine the impact value of the light source on the entity according to the light intensity of the light source and the distance between the light source and the entity;

[0007] Obtain a depth map of the entity captured by a depth camera at the main light source and a rendering image of the virtual camera; wherein the main light source is the light source with the greatest impact on the entity among all light sources;

[0008] Generate a scene depth map under the main light source according to the rendered image;

[0009] A first shadow map is generated according to the depth map of the entity and the scene depth map under the main light source.

[0010] Optionally, generating a scene depth map under the main light source according to the rendered image of the virtual camera includes:

[0011] For each fragment in the rendered image, performing coordinate system conversion on the world coordinates of the fragment to obtain texture coordinates of the depth map of the fragment;

[0012] Generate the scene depth map under the main light source based on the texture coordinates of the depth map of all fragments.

[0013] Optionally, for each fragment in the rendered image, performing coordinate system conversion on the world coordinates of the fragment to obtain texture coordinates of the depth map of the fragment includes:

[0014] For each fragment in the rendered image, convert the world coordinates of the fragment into coordinates in the light source view coordinate system of the main light source to obtain the light source view coordinates of the fragment;

[0015] Convert the light source view coordinates of the fragment into light source clipping coordinates to obtain the light source clipping coordinates of the fragment;

[0016] Performing standardized coordinate conversion on the light source clipping coordinates of the fragment to obtain the standardized coordinates of the fragment;

[0017] The normalized coordinates of the fragment are converted into depth coordinates to obtain texture coordinates of the depth map of the fragment.

[0018] Optionally, generating a first shadow map according to the depth map of the entity and the scene depth map under the main light source includes:

[0019] For each group of fragments, determine the shadow pixel output result of the group of fragments according to the depth value of the second fragment and the depth value of the first fragment; wherein the fragments at the same position in the depth map of the entity and the scene depth map under the main light source are a group of fragments; the group of fragments includes the first fragment and the second fragment; the depth value of the first fragment is the depth value of the fragment in the depth map of the entity; the depth value of the second fragment is the depth value of the fragment in the scene depth map under the main light source;

[0020] According to the output results of all shadow pixels, the first shadow map is drawn.

[0021] Optionally, after generating the first shadow map according to the depth map of the entity and the scene depth map under the main light source, the method further includes:

[0022] For each secondary light source, generating a scene depth map under the secondary light source according to the rendered image;

[0023] generating a second shadow map according to the depth map of the entity and the depth map of the scene under the secondary light source;

[0024] Performing optimization analysis based on the first shadow map and the second shadow map to obtain optimization analysis results;

[0025] If the optimization analysis result indicates that the first shadow map needs to be optimized, the pixels in the first shadow map are corrected using the pixels in the second shadow map to obtain a third shadow map.

[0026] Optionally, performing optimization analysis according to the first shadow map and the second shadow map to obtain an optimization analysis result includes:

[0027] Taking the quotient of the union number of pixels in the first shadow map and the second shadow map and the number of all pixels in the second shadow map as the overlap ratio between the first shadow map and the second shadow map;

[0028] If the overlap rate between the first shadow map and the second shadow map is greater than or equal to the overlap rate threshold, determining that the first shadow map needs to be optimized;

[0029] If the overlap ratio between the first shadow map and the second shadow map is less than the overlap ratio threshold, it is determined that the first shadow map does not need to be optimized.

[0030] Optionally, if the optimization analysis result indicates that the first shadow map needs to be optimized, using pixels in the second shadow map to correct pixels in the first shadow map to obtain a third shadow map includes:

[0031] If the optimization analysis result indicates that the first shadow map needs to be optimized, for each group of pixels, determine whether the first pixel is smaller than the second pixel; wherein pixels at the same position in the first shadow map and the second shadow map are a group of pixels; the first pixel is a pixel in the first shadow map; and the second pixel is a pixel in the second shadow map;

[0032] If it is determined that the first pixel is smaller than the second pixel, the first pixel is corrected according to the second pixel, the exposure parameter, the influence value of the main light source, and the influence value of the secondary light source to obtain a third pixel;

[0033] Determining whether the second pixel is smaller than the first pixel;

[0034] If it is determined that the second pixel is smaller than the first pixel, the third pixel is corrected according to the first pixel, the exposure parameter, the influence value of the main light source, and the influence value of the secondary light source to obtain a fourth pixel;

[0035] A third shadow map is generated based on all the uncorrected third pixels and all the fourth pixels.

[0036] A second aspect of the present application provides a shadow map generation device, comprising:

[0037] A first acquisition unit is used to acquire light source information of each light source; wherein the light source is divided into a real light source and a virtual light source; the light source information includes light intensity and a distance between the light source and the entity;

[0038] A first determining unit, configured to determine, for each light source, an influence value of the light source on the entity according to the illumination intensity of the light source and the distance between the light source and the entity;

[0039] A second acquisition unit is used to acquire a depth map of the entity photographed by the depth camera at the main light source and a rendering image of the virtual camera; wherein the main light source is the light source with the greatest impact on the entity among all light sources;

[0040] A first generating unit, configured to generate a scene depth map under the main light source according to the rendered image;

[0041] The second generating unit is used to generate a first shadow map according to the depth map of the entity and the scene depth map under the main light source.

[0042] Optionally, the first generating unit includes:

[0043] A coordinate conversion unit, for each fragment in the rendered image, performs coordinate system conversion on the world coordinates of the fragment to obtain the texture coordinates of the depth map of the fragment;

[0044] The first generating subunit is used to generate a scene depth map under a main light source according to the texture coordinates of the depth maps of all fragments.

[0045] Optionally, the coordinate conversion unit includes:

[0046] A first coordinate conversion subunit is used for converting the world coordinates of each fragment in the rendered image into coordinates in the light source view coordinate system of the main light source to obtain the light source view coordinates of the fragment;

[0047] A second coordinate conversion subunit, configured to convert the light source view coordinates of the fragment into light source clipping coordinates to obtain the light source clipping coordinates of the fragment;

[0048] A third coordinate conversion subunit, used for performing standardized coordinate conversion on the light source clipping coordinates of the fragment to obtain the standardized coordinates of the fragment;

[0049] The fourth coordinate conversion subunit is used to convert the standardized coordinates of the fragment into depth coordinates to obtain the texture coordinates of the depth map of the fragment.

[0050] Optionally, the second generating unit includes:

[0051] A second determination unit is used to determine, for each group of fragments, a shadow pixel output result of the group of fragments according to the depth value of the second fragment and the depth value of the first fragment; wherein the fragments at the same position in the depth map of the entity and the scene depth map under the main light source are a group of fragments; the group of fragments includes the first fragment and the second fragment; the depth value of the first fragment is the depth value of the fragment in the depth map of the entity; the depth value of the second fragment is the depth value of the fragment in the scene depth map under the main light source;

[0052] The drawing unit is used to draw a first shadow map according to the output results of all shadow pixels.

[0053] Optionally, the shadow map generating device further includes:

[0054] A third generating unit is used to generate, for each secondary light source, a scene depth map under the secondary light source according to the rendered image;

[0055] a fourth generating unit, configured to generate a second shadow map according to the depth map of the entity and the scene depth map under the secondary light source;

[0056] An optimization analysis unit, configured to perform optimization analysis based on the first shadow map and the second shadow map to obtain an optimization analysis result;

[0057] A correction unit is used to correct the pixels in the first shadow map using the pixels in the second shadow map to obtain a third shadow map if the optimization analysis result indicates that the first shadow map needs to be optimized.

[0058] Optionally, the optimization analysis unit includes:

[0059] an overlap rate calculation unit, configured to take a quotient of a union number of pixels in the first shadow map and the second shadow map and a number of all pixels in the second shadow map as an overlap rate between the first shadow map and the second shadow map;

[0060] an optimization analysis subunit, configured to determine that the first shadow map needs to be optimized if the overlap rate between the first shadow map and the second shadow map is greater than or equal to an overlap rate threshold;

[0061] The optimization analysis subunit is further configured to determine that the first shadow map does not need to be optimized if the overlap rate between the first shadow map and the second shadow map is less than an overlap rate threshold.

[0062] Optionally, the correction unit includes:

[0063] A first judgment unit is configured to, if the optimization analysis result indicates that the first shadow map needs to be optimized, judge, for each group of pixels, whether the first pixel is smaller than the second pixel; wherein pixels at the same position in the first shadow map and the second shadow map are a group of pixels; the first pixel is a pixel in the first shadow map; and the second pixel is a pixel in the second shadow map;

[0064] a first correction subunit, configured to correct the first pixel according to the second pixel, the exposure parameter, the influence value of the main light source, and the influence value of the secondary light source to obtain a third pixel if the first judgment unit determines that the first pixel is smaller than the second pixel;

[0065] A second judging unit, used to judge whether the second pixel is smaller than the first pixel;

[0066] a second correction subunit, configured to correct the third pixel according to the first pixel, the exposure parameter, the influence value of the main light source, and the influence value of the secondary light source to obtain a fourth pixel if the second judgment unit determines that the second pixel is smaller than the first pixel;

[0067] The fifth generating unit is used to generate a third shadow map according to all the uncorrected third pixels and all the fourth pixels.

[0068] A third aspect of the present application provides an electronic device, including:

[0069] one or more processors;

[0070] a storage device having one or more programs stored thereon;

[0071] When the one or more programs are executed by the one or more processors, the one or more processors implement the shadow map generation method as described in any one of the first aspects.

[0072] A fourth aspect of the present application provides a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for generating a shadow map as described in any one of the first aspects is implemented.

[0073] As can be seen from the above scheme, the present application provides a method, device, electronic device and computer storage medium for generating a shadow map. After obtaining the light source information of each light source, for each light source, according to the illumination intensity of the light source and the distance between the light source and the entity, the influence value of the light source on the entity is determined, and the light source with the largest influence value on the entity among all the light sources is taken as the main light source. Then, the depth map of the entity taken by the depth camera at the main light source and the rendering image of the virtual camera are obtained. After that, according to the rendering image, the scene depth map under the main light source is generated; finally, according to the depth map of the entity and the scene depth map under the main light source, the first shadow map is generated. In the actual application process of the present application, after obtaining the first shadow map, the first shadow map is applied to the output screen of the virtual camera, so that the output virtual scene is more naturally combined with the entity and the shooting effect is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0075] Figure 1 A flowchart of a method for generating a shadow map provided in an embodiment of the present application;

[0076] Figure 2 A flowchart of generating a scene depth map under a main light source provided by another embodiment of the present application;

[0077] Figure 3 A flowchart of a method for performing coordinate system conversion on the world coordinates of a fragment provided by another embodiment of the present application;

[0078] Figure 4 A flowchart of a method for generating a first shadow map provided by another embodiment of the present application;

[0079] Figure 5 A flowchart of a method for generating a shadow map provided in another embodiment of the present application;

[0080] Figure 6 A flowchart of a method for generating a third shadow map provided by another embodiment of the present application;

[0081] Figure 7 A schematic diagram of a shadow map generation device provided by another embodiment of the present application;

[0082] Figure 8A schematic diagram of a shadow map generation device provided by another embodiment of the present application;

[0083] Fig. 9 A schematic diagram of an electronic device for implementing a method for generating a shadow map provided in another embodiment of the present application. DETAILED DESCRIPTION

[0084] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0085] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0086] It should be noted that the information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0087] It should be noted that the concepts such as "first" and "second" mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0088] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0089] The present application embodiment provides a method for generating a shadow map, such as Figure 1 As shown, the specific steps include:

[0090] S101. Obtain light source information of each light source.

[0091] Among them, the light source is divided into real light source and virtual light source; the light source information includes the light intensity and the distance between the light source and the entity.

[0092] It should be noted that an entity can be, but is not limited to, an object, a person, etc., and is not limited here.

[0093] In the specific implementation process of the present application, if the light source is a real light source, the depth camera can be bound to the position of the light source according to but not limited to determining the actual light source position, and the distance between the light source and the entity can be obtained through the depth camera, which is not limited here. The distance between the virtual light source and the entity can be obtained directly by the Unreal Engine. The intensity of the virtual light source will adjust the light intensity according to the scene requirements when creating the light in the Unreal Engine. The light intensity of the virtual light source is known, and the real light source can be connected to the actual light group through the light controller. Its light intensity is also controlled in the Unreal Engine according to the scene and entity lighting requirements. The light intensity of the real light source is also known and can be directly obtained through statistics in the background.

[0094] S102. For each light source, determine the influence value of the light source on the entity according to the illumination intensity of the light source and the distance between the light source and the entity.

[0095] In the specific practical application process of this application, the light source can be calculated using the following formula Impact value on the entity :

[0096] ;in, For light source Light intensity; For light source The distance from the entity.

[0097] S103: Obtain a depth map of an entity captured by a depth camera at a main light source and a rendering image of a virtual camera.

[0098] Among them, the main light source is the light source that has the greatest impact on the entity among all light sources.

[0099] S104: Generate a scene depth map under the main light source according to the rendered image.

[0100] Optionally, in another embodiment of the present application, an implementation of step S104 is as follows: Figure 2 As shown, including:

[0101] S201 . For each fragment in the rendered image, perform coordinate system conversion on the world coordinates of the fragment to obtain texture coordinates of the depth map of the fragment.

[0102] Optionally, in another embodiment of the present application, an implementation of step S201 is as follows: Figure 3 As shown, including:

[0103] S301 . For each fragment in the rendered image, convert the world coordinates of the fragment into coordinates in the light source view coordinate system of the main light source to obtain the light source view coordinates of the fragment.

[0104] In the specific implementation process of the present application, the light view matrix (lightViewMatrix) can be used, but is not limited to, to convert the world coordinates of the fragment into the coordinates of the light view coordinate system of the main light source to obtain the light view coordinates of the fragment.

[0105] Specifically, the light source view coordinates of the fragment can be obtained by using but not limited to the following formula:

[0106] vlightView=lightViewMatrix·vworld; vworld is the world coordinate of the fragment, and vlightView is the light source view coordinate of the fragment.

[0107] lightViewMatrix= ; Among them, Rx, Ry, Rz are the components of the right vector, Ux, Uy, Uz are the components of the up vector, and Dx, Dy, Dz are the components of the direction vector.

[0108] Plight is the light source position, Tlight is the light source target point position, Ulight is the upward direction of the light source, D is the light source Z-axis direction vector, R is the light source X-axis direction vector, and U is the light source Y-axis direction vector. Then, D= ; R = ; U=D·R.

[0109] It should be noted that if the main light source is a virtual light source, step S302 can be directly executed. If the main light source is a real light source, the coordinates of the real light source need to be converted into coordinates in the virtual scene before executing step S302.

[0110] In the specific implementation process of the present application, the method of converting the coordinates of the real light source into the coordinates in the virtual scene can be but is not limited to the following steps:

[0111] (1) The coordinate system of the real space can be located by methods such as, but not limited to, infrared positioning, light field positioning, etc., which are not limited here.

[0112] (2) Establishing the relationship between the real and virtual coordinate systems, for example, through scaling (unifying the units of the real space and the virtual space), rotation transformation (making the axis directions of the real and virtual coordinate systems the same), translation transformation (making the origin positions of the real and virtual coordinate systems the same), etc., without limitation here.

[0113] (3) The conversion formula from real coordinates to virtual coordinates is:

[0114] Pv=R·1 / scale·Pr+T;

[0115] Pr is the real coordinate, Pv is the virtual coordinate, R is the rotation matrix, scale is the unit scale factor, and T is the translation vector.

[0116] S302 , convert the light source view coordinates of the fragment into light source clipping coordinates to obtain the light source clipping coordinates of the fragment.

[0117] In the specific implementation process of the present application, the light source view coordinates of the fragment may be converted to the light source clipping coordinates of the fragment by using, but not limited to, the projection matrix (lightProjectionMatrix) of the light source.

[0118] Specifically, the light source clipping coordinates of the fragment can be obtained by using but not limited to the following formula:

[0119] vlightClip=lightProjectionMatrix·vlightView; where vlightClip is the light source clipping coordinate system, and lightProjectionMatrix is ​​the projection matrix of the light source, taking the light as a parallel light source as an example.

[0120] lightProjectionMatrix= ;

[0121] Among them, left and right define the left and right boundaries of the view frustum on the X axis, left is the left boundary value of the view frustum, and right is the right boundary value of the view frustum; top and bottom define the upper and lower boundaries of the view frustum on the Y axis, top is the upper boundary value of the view frustum, and bottom is the lower boundary value of the view frustum; near and far define the near clipping plane and far clipping plane of the view frustum on the Z axis (depth direction), near is the plane closest to the camera (usually a positive value or a positive value close to 0), and far is the plane farthest from the camera.

[0122] S303 , performing a standardized coordinate conversion on the light source clipping coordinates of the fragment to obtain the standardized coordinates of the fragment.

[0123] The standardized coordinate transformation may be, but is not limited to, NDC (Normalized Device Coordinates) coordinate transformation.

[0124] Specifically, the standardized coordinates of the fragment can be obtained by using but not limited to the following formula:

[0125] vlightNDC= ;

[0126] Among them, W is the inverse vector of the depth value z in the light source space. Assume that the point in the light source space is Vlight, its coordinates are [x, y, z, 1], x is the x-axis of the light source space, aligned with the right direction of the light source perspective, and represents the horizontal vector of the light source observation direction; y is the y-axis of the light source space, aligned with the upper direction of the light source perspective, and represents the vertical upward vector of the light source observation direction; z is the z-axis of the light source space, aligned with the observation direction of the light source (negative direction, similar to the zzz-axis of the camera), and represents the line of sight direction vector of the light source (from the light source to the scene); 1 is the fourth dimension of the homogeneous coordinate, which is used to describe the representation of points in three-dimensional space in the homogeneous coordinate system. The main purpose is to facilitate matrix calculation and projection transformation. When w=1, it means that this is a point and can be directly restored to three-dimensional space. Vlight (x, y, z, 1) represents the coordinate point of the point in the light source space. After multiplying it by the projection matrix lightProjectionMatrix of the light source, the clipping coordinate Vclip can be obtained, that is, Vclip=lightProjectionMatrix·Vlight, and its coordinates are [Xclip, Yclip, Zclip, W], where Xclip represents the horizontal vector of the observation direction of the light source after clipping, Yclip represents the vertical upward vector of the observation direction of the light source after clipping, and Zclip represents the sight direction vector of the light source after clipping; W=−z, that is, W is usually the inverse vector of the depth value z in the light source space.

[0127] S304 , converting the normalized coordinates of the fragment into depth coordinates to obtain texture coordinates of the depth map of the fragment.

[0128] Specifically, the texture coordinates vshadowMap of the depth map of the fragment can be obtained by using but not limited to the following formula:

[0129] vshadowMap=vlightNDC×0.5+0.5.

[0130] S202: Generate a scene depth map under the main light source according to the texture coordinates of the depth maps of all fragments.

[0131] S105 . Generate a first shadow map according to the depth map of the entity and the scene depth map under the main light source.

[0132] Optionally, in another embodiment of the present application, an implementation of step S105 is as follows: Figure 4 As shown, including:

[0133] S401 . For each group of fragments, determine a shadow pixel output result of the group of fragments according to a depth value of a second fragment and a depth value of a first fragment.

[0134] Among them, the fragments at the same position in the depth map of the entity and the scene depth map under the main light source are a group of fragments; a group of fragments includes a first fragment and a second fragment; the depth value of the first fragment is the depth value of the fragment in the depth map of the entity; the depth value of the second fragment is the depth value of the fragment in the scene depth map under the main light source.

[0135] Specifically, if the depth value of the second fragment in a group of fragments is greater than the depth value of the first fragment, it means that the light is blocked and the fragment is in the shadow, and the shadow pixel is output in the rendered image, otherwise it is not output.

[0136] S402: Draw a first shadow map according to the output results of all shadow pixels.

[0137] Continuing with the above example, according to the output results of all shadow pixels, the shadow map of the real object in the virtual camera is drawn, and then the picture with the shadow of the real object can be output.

[0138] It can be seen from the above scheme that the present application provides a method for generating a shadow map. After obtaining the light source information of each light source, for each light source, the influence value of the light source on the entity is determined according to the illumination intensity of the light source and the distance between the light source and the entity. The light source with the largest influence value on the entity among all the light sources is taken as the main light source. Then, the depth map of the entity taken by the depth camera at the main light source and the rendered image of the virtual camera are obtained. After that, the scene depth map under the main light source is generated according to the rendered image; finally, the first shadow map is generated according to the depth map of the entity and the scene depth map under the main light source. In the actual application process of the present application, after obtaining the first shadow map, the first shadow map is applied to the output screen of the virtual camera, so that the output virtual scene is more naturally combined with the entity and the shooting effect is better.

[0139] Since in actual application scenarios, the secondary light source will also affect the shadow, therefore, in another embodiment of the present application, the shadow will be optimized in combination with the secondary light source, such as Figure 5 As shown, the embodiment of the present application provides a method for generating a shadow map, which specifically includes:

[0140] S501: Acquire light source information of each light source.

[0141] Among them, the light source is divided into real light source and virtual light source; the light source information includes the light intensity and the distance between the light source and the entity.

[0142] S502: For each light source, determine the influence value of the light source on the entity according to the illumination intensity of the light source and the distance between the light source and the entity.

[0143] S503: Obtain a depth map of an entity captured by a depth camera at a main light source and a rendering image of a virtual camera.

[0144] Among them, the main light source is the light source that has the greatest impact on the entity among all light sources.

[0145] S504: Generate a scene depth map under the main light source according to the rendered image.

[0146] S505 : Generate a first shadow map according to the depth map of the entity and the scene depth map under the main light source.

[0147] The specific implementation of steps S501 to S505 may refer to steps S101 to S105 and will not be described in detail here.

[0148] S506 : For each secondary light source, generate a scene depth map under the secondary light source according to the rendered image.

[0149] It should be noted that the implementation method of generating the scene depth map under the secondary light source according to the rendered image can refer to the implementation method of generating the scene depth map under the primary light source according to the rendered image in step S104, which will not be repeated here.

[0150] S507 . Generate a second shadow map according to the depth map of the entity and the scene depth map under the secondary light source.

[0151] It should be noted that the implementation method of generating the second shadow map based on the depth map of the entity and the scene depth map under the secondary light source can refer to the implementation method of generating the first shadow map based on the depth map of the entity and the scene depth map under the main light source in step S105, which will not be repeated here.

[0152] S508 , performing optimization analysis according to the first shadow map and the second shadow map to obtain optimization analysis results.

[0153] In the specific implementation process of the present application, the overlap ratio between the first shadow map and the second shadow map may be used for optimization analysis but is not limited thereto, and is not limited here.

[0154] Specifically, the quotient of the union number of pixels in the first shadow map and the second shadow map and the number of all pixels in the second shadow map is taken as the overlap rate between the first shadow map and the second shadow map.

[0155] Among them, if the overlap rate of the first shadow map and the second shadow map is greater than or equal to the overlap rate threshold, it is determined that the first shadow map needs to be optimized; if the overlap rate of the first shadow map and the second shadow map is less than the overlap rate threshold, it is determined that the first shadow map does not need to be optimized.

[0156] It should be noted that the overlap rate threshold is set in advance by technicians, experts, etc. and is not limited here.

[0157] It is understandable that when calculating the impact of multiple light sources on an object, the computing performance will be greatly reduced. Since the secondary light source itself is not as strong as the primary light source, if they are basically not in the same direction (that is, the overlap rate of the first shadow map and the second shadow map is less than the overlap rate threshold), there is no need to render it, otherwise the shadow will appear messy and increase performance pressure.

[0158] S509: If the optimization analysis result indicates that the first shadow map needs to be optimized, the pixels in the first shadow map are corrected using the pixels in the second shadow map to obtain a third shadow map.

[0159] Optionally, in another embodiment of the present application, an implementation of step S509 is as follows: Figure 6 As shown, including:

[0160] S601: If the optimization analysis result indicates that the first shadow map needs to be optimized, for each group of pixels, determine whether the first pixel is smaller than the second pixel.

[0161] The pixels at the same position in the first shadow map and the second shadow map are a group of pixels; the first pixel is a pixel in the first shadow map; and the second pixel is a pixel in the second shadow map.

[0162] It should be noted that in actual application scenarios, the projection generated by the secondary light source will affect the edge of the projection generated by the main light source. This application determines whether the projection generated by the secondary light source will affect the edge of the projection generated by the main light source by judging whether the first pixel is smaller than the second pixel.

[0163] Specifically, if it is determined that the first pixel is smaller than the second pixel, step S602 is executed; if it is determined that the first pixel is not smaller than the second pixel, no processing is performed. If it is determined that the first pixel is greater than or equal to the second pixel, it means that the projection generated by the secondary light source i has a complementary effect on the edge of the projection generated by the main light.

[0164] S602: Correct the first pixel according to the second pixel, the exposure parameter, the influence value of the main light source, and the influence value of the secondary light source to obtain a third pixel.

[0165] In the specific implementation process of the present application, the first pixel may be corrected using but not limited to the following calculation formula to obtain the third pixel:

[0166] S0'=S0+W·S ii · ; Where S0' is the third pixel, W is the exposure parameter used to control the intensity of the secondary light source, P0 is the impact value of the main light source, P i is the influence value of the secondary light source i, S0 is the first pixel, S iiis the second pixel.

[0167] S603: Determine whether the second pixel is smaller than the first pixel.

[0168] It should be noted that in actual application scenarios, the secondary light source itself affects the edge of the projection produced by the primary light source. This application determines whether the secondary light source itself affects the edge of the projection produced by the primary light source by judging whether the second pixel is smaller than the first pixel.

[0169] Specifically, if it is determined that the second pixel is smaller than the first pixel, step S604 is executed, and if it is determined that the second pixel is not smaller than the first pixel, no processing is performed. If it is determined that the second pixel is greater than or equal to the first pixel, it means that the secondary light source i itself does not have any lighting effect on the edge of the projection of the main light.

[0170] S604 , correct the third pixel according to the first pixel, the exposure parameter, the influence value of the main light source, and the influence value of the secondary light source to obtain a fourth pixel.

[0171] In the specific implementation process of the present application, the third pixel may be corrected using but not limited to the following calculation formula to obtain the fourth pixel:

[0172] S=S0'-W·S ii '· ; Where S0' is the third pixel, W is the exposure parameter used to control the intensity of the secondary light source, P0 is the impact value of the main light source, P i is the influence value of the secondary light source i, S is the fourth pixel, S ii ' is the first pixel.

[0173] S605 . Generate a third shadow map according to all uncorrected third pixels and all fourth pixels.

[0174] It can be seen from the above scheme that the present application provides a method for generating a shadow map. After generating the first shadow map, the pixels in the first shadow map are corrected in combination with the pixels in the second shadow map generated by each secondary light source to obtain multiple third shadow maps. Finally, all the third shadow maps are left and right with the output picture of the virtual camera to obtain the entity dynamic shadow after optimizing the virtual and real lights at the same time, so that the output virtual scene and the entity are combined more naturally and the shooting effect is better.

[0175] The embodiment of the present application provides a device for generating a shadow map, such as Figure 7 As shown, specifically including:

[0176] The first acquisition unit 701 is used to acquire light source information of each light source.

[0177] Among them, the light source is divided into real light source and virtual light source; the light source information includes light intensity and the distance between the light source and the entity.

[0178] The first determining unit 702 is configured to determine, for each light source, an influence value of the light source on the entity according to the illumination intensity of the light source and the distance between the light source and the entity.

[0179] The second acquisition unit 703 is used to acquire a depth map of an entity captured by a depth camera at a main light source and a rendering image of a virtual camera.

[0180] Among them, the main light source is the light source that has the greatest impact on the entity among all light sources.

[0181] The first generating unit 704 is used to generate a scene depth map under a main light source according to the rendered image.

[0182] Optionally, in another embodiment of the present application, an implementation of the first generating unit 704 includes:

[0183] The coordinate conversion unit is used to convert the world coordinates of each fragment in the rendered image into a coordinate system to obtain the texture coordinates of the depth map of the fragment.

[0184] The first generating subunit is used to generate a scene depth map under a main light source according to the texture coordinates of the depth maps of all fragments.

[0185] For the specific working process of the units disclosed in the above embodiments of the present application, please refer to the corresponding method embodiments, such as Figure 2 As shown, no further description is given here.

[0186] Optionally, in another embodiment of the present application, an implementation of the coordinate conversion unit includes:

[0187] The first coordinate conversion subunit is used to convert the world coordinates of each fragment in the rendered image into coordinates in the light source view coordinate system of the main light source to obtain the light source view coordinates of the fragment.

[0188] The second coordinate conversion subunit is used to convert the light source view coordinates of the fragment into light source clipping coordinates to obtain the light source clipping coordinates of the fragment.

[0189] The third coordinate conversion subunit is used to perform standardized coordinate conversion on the light source clipping coordinates of the fragment to obtain the standardized coordinates of the fragment.

[0190] The fourth coordinate conversion subunit is used to convert the normalized coordinates of the fragment into depth coordinates to obtain the texture coordinates of the depth map of the fragment.

[0191] For the specific working process of the units disclosed in the above embodiments of the present application, please refer to the corresponding method embodiments, such as Figure 3 As shown, no further description is given here.

[0192] The second generating unit 705 is used to generate a first shadow map according to the depth map of the entity and the scene depth map under the main light source.

[0193] For the specific working process of the units disclosed in the above embodiments of the present application, please refer to the corresponding method embodiments, such as Figure 1 As shown, no further description is given here.

[0194] Optionally, in another embodiment of the present application, an implementation of the second generating unit 705 includes:

[0195] The second determining unit is used to determine, for each group of fragments, a shadow pixel output result of the group of fragments according to the depth value of the second fragment and the depth value of the first fragment.

[0196] Among them, the fragments at the same position in the depth map of the entity and the scene depth map under the main light source are a group of fragments; a group of fragments includes a first fragment and a second fragment; the depth value of the first fragment is the depth value of the fragment in the depth map of the entity; the depth value of the second fragment is the depth value of the fragment in the scene depth map under the main light source.

[0197] The drawing unit is used to draw a first shadow map according to the output results of all shadow pixels.

[0198] For the specific working process of the units disclosed in the above embodiments of the present application, please refer to the corresponding method embodiments, such as Figure 4 As shown, no further description is given here.

[0199] As can be seen from the above scheme, the present application provides a shadow map generation device. After the first acquisition unit 701 acquires the light source information of each light source, the first determination unit 702 determines the impact value of the light source on the entity for each light source according to the light intensity of the light source and the distance between the light source and the entity, and takes the light source with the largest impact value on the entity among all the light sources as the main light source. Then, the second acquisition unit 703 acquires the depth map of the entity photographed by the depth camera at the main light source and the rendered image of the virtual camera. After that, the first generation unit 704 generates the scene depth map under the main light source according to the rendered image; finally, the second generation unit 705 generates the first shadow map according to the depth map of the entity and the scene depth map under the main light source. In the actual application process of the present application, after obtaining the first shadow map, the first shadow map is applied to the output screen of the virtual camera, so that the output virtual scene is more naturally combined with the entity and the shooting effect is better.

[0200] The embodiment of the present application provides a device for generating a shadow map, such as Figure 8 As shown, specifically including:

[0201] The first acquisition unit 801 is used to acquire light source information of each light source.

[0202] Among them, the light source is divided into real light source and virtual light source; the light source information includes the light intensity and the distance between the light source and the entity.

[0203] The first determining unit 802 is used to determine, for each light source, an influence value of the light source on the entity according to the illumination intensity of the light source and the distance between the light source and the entity.

[0204] The second acquisition unit 803 is used to acquire a depth map of an entity captured by a depth camera at a main light source and a rendering image of a virtual camera.

[0205] Among them, the main light source is the light source that has the greatest impact on the entity among all light sources.

[0206] The first generating unit 804 is used to generate a scene depth map under a main light source according to the rendered image.

[0207] The second generating unit 805 is used to generate a first shadow map according to the depth map of the entity and the scene depth map under the main light source.

[0208] The third generating unit 806 is configured to generate, for each secondary light source, a scene depth map under the secondary light source according to the rendered image.

[0209] The fourth generating unit 807 is used to generate a second shadow map according to the depth map of the entity and the scene depth map under the secondary light source.

[0210] The optimization analysis unit 808 is used to perform optimization analysis according to the first shadow map and the second shadow map to obtain an optimization analysis result.

[0211] Optionally, in another embodiment of the present application, an implementation of the optimization analysis unit 808 includes:

[0212] The overlap rate calculation unit is used to take the quotient of the union number of pixels in the first shadow map and the second shadow map and the number of all pixels in the second shadow map as the overlap rate between the first shadow map and the second shadow map.

[0213] The optimization analysis subunit is used to determine that the first shadow map needs to be optimized if the overlap rate between the first shadow map and the second shadow map is greater than or equal to an overlap rate threshold.

[0214] The optimization analysis subunit is further configured to determine that the first shadow map does not need to be optimized if the overlap rate between the first shadow map and the second shadow map is less than an overlap rate threshold.

[0215] The specific working process of the units disclosed in the above embodiments of the present application can be found in the corresponding method embodiments, which will not be repeated here.

[0216] The correction unit 809 is configured to correct the pixels in the first shadow map using the pixels in the second shadow map to obtain a third shadow map if the optimization analysis result indicates that the first shadow map needs to be optimized.

[0217] For the specific working process of the units disclosed in the above embodiments of the present application, please refer to the corresponding method embodiments, such as Figure 5 As shown, no further description is given here.

[0218] Optionally, in another embodiment of the present application, an implementation of the correction unit 809 includes:

[0219] The first judgment unit is configured to judge, for each group of pixels, whether the first pixel is smaller than the second pixel if the optimization analysis result indicates that the first shadow map needs to be optimized.

[0220] The pixels at the same position in the first shadow map and the second shadow map are a group of pixels; the first pixel is a pixel in the first shadow map; and the second pixel is a pixel in the second shadow map.

[0221] The first correction subunit is used to correct the first pixel according to the second pixel, the exposure parameter, the influence value of the main light source and the influence value of the secondary light source to obtain a third pixel if the first judgment unit determines that the first pixel is smaller than the second pixel.

[0222] The second judging unit is used to judge whether the second pixel is smaller than the first pixel.

[0223] The second correction subunit is used to correct the third pixel according to the first pixel, the exposure parameter, the influence value of the main light source and the influence value of the secondary light source to obtain a fourth pixel if the second judgment unit determines that the second pixel is smaller than the first pixel.

[0224] The fifth generating unit is used to generate a third shadow map according to all the uncorrected third pixels and all the fourth pixels.

[0225] For the specific working process of the units disclosed in the above embodiments of the present application, please refer to the corresponding method embodiments, such as Figure 6 As shown, no further description is given here.

[0226] As can be seen from the above scheme, the present application provides a shadow map generation device. After generating the first shadow map, the third generation unit 806 generates a scene depth map under the secondary light source for each secondary light source according to the rendered image. The fourth generation unit 807 generates a second shadow map according to the depth map of the entity and the scene depth map under the secondary light source. The optimization analysis unit 808 performs optimization analysis based on the first shadow map and the second shadow map to obtain an optimization analysis result. If the optimization analysis result shows that the first shadow map needs to be optimized, the correction unit 809 uses the pixels in the second shadow map to correct the pixels in the first shadow map to obtain a third shadow map. By combining the pixels in the second shadow map generated by each secondary light source to correct the pixels in the first shadow map, a plurality of third shadow maps are obtained, and finally all the third shadow maps are left and right with the output picture of the virtual camera to obtain the entity dynamic shadow after optimizing the virtual and real lights at the same time, so that the output virtual scene is more naturally combined with the entity and the shooting effect is better.

[0227] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0228] Another embodiment of the present application provides an electronic device, such as Fig. 9 As shown, including:

[0229] One or more processors 901.

[0230] The storage device 902 stores one or more programs.

[0231] When the one or more programs are executed by the one or more processors 901, the one or more processors 901 implement the shadow map generation method as described in any one of the above embodiments.

[0232] Another embodiment of the present application provides a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, a method for generating a shadow map as described in any one of the above embodiments is implemented.

[0233] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0234] It should be noted that the computer-readable medium mentioned above in the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0235] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0236] Another embodiment of the present application provides a computer program product, which, when executed, is used to execute the above-mentioned method for generating a shadow map.

[0237] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, the above-mentioned functions defined in the method of the embodiment of the present application are executed.

[0238] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in this application is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing this application.

[0239] Although several specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present application. Certain features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination.

[0240] The above description is only a preferred embodiment of the present application and an explanation of the technical principles used. Those skilled in the art should understand that the scope of application involved in the present application is not limited to the technical solution formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above application concept. For example, the above features are replaced with (but not limited to) technical features with similar functions applied in the present application.

Claims

1. A method for generating a shadow map, characterized in that: include: Obtaining light source information of each light source; wherein the light source is divided into a real light source and a virtual light source; the light source information includes light intensity and a distance between the light source and the entity; For each light source, determine the impact value of the light source on the entity according to the illumination intensity of the light source and the distance between the light source and the entity; Obtain a depth map of the entity captured by a depth camera at the main light source and a rendering image of the virtual camera; wherein the main light source is the light source with the greatest impact on the entity among all light sources; Generate a scene depth map under the main light source according to the rendered image; A first shadow map is generated according to the depth map of the entity and the depth map of the scene under the main light source.

2. The method for generating a shadow map according to claim 1, characterized in that: The step of generating a scene depth map under the main light source according to the rendered image of the virtual camera includes: For each fragment in the rendered image, performing coordinate system conversion on the world coordinates of the fragment to obtain texture coordinates of the depth map of the fragment; Generate the scene depth map under the main light source based on the texture coordinates of the depth map of all fragments.

3. The method for generating a shadow map according to claim 2, characterized in that: For each fragment in the rendered image, performing coordinate system conversion on the world coordinates of the fragment to obtain the texture coordinates of the depth map of the fragment includes: For each fragment in the rendered image, convert the world coordinates of the fragment into coordinates in the light source view coordinate system of the main light source to obtain the light source view coordinates of the fragment; Convert the light source view coordinates of the fragment into light source clipping coordinates to obtain the light source clipping coordinates of the fragment; Performing standardized coordinate conversion on the light source clipping coordinates of the fragment to obtain the standardized coordinates of the fragment; The normalized coordinates of the fragment are converted into depth coordinates to obtain texture coordinates of the depth map of the fragment.

4. The method for generating a shadow map according to claim 1, characterized in that: The generating a first shadow map according to the depth map of the entity and the scene depth map under the main light source comprises: For each group of fragments, determine the shadow pixel output result of the group of fragments according to the depth value of the second fragment and the depth value of the first fragment; wherein the fragments at the same position in the depth map of the entity and the scene depth map under the main light source are a group of fragments; the group of fragments includes the first fragment and the second fragment; the depth value of the first fragment is the depth value of the fragment in the depth map of the entity; the depth value of the second fragment is the depth value of the fragment in the scene depth map under the main light source; According to the output results of all shadow pixels, the first shadow map is drawn.

5. The method for generating a shadow map according to claim 1, characterized in that: After generating the first shadow map according to the depth map of the entity and the scene depth map under the main light source, the method further includes: For each secondary light source, generating a scene depth map under the secondary light source according to the rendered image; wherein the secondary light source is a light source other than the primary light source among all the light sources; generating a second shadow map according to the depth map of the entity and the depth map of the scene under the secondary light source; Performing optimization analysis based on the first shadow map and the second shadow map to obtain optimization analysis results; If the optimization analysis result indicates that the first shadow map needs to be optimized, the pixels in the first shadow map are corrected using the pixels in the second shadow map to obtain a third shadow map.

6. The method for generating a shadow map according to claim 5, characterized in that: The performing optimization analysis according to the first shadow map and the second shadow map to obtain the optimization analysis result includes: Taking the quotient of the union number of pixels in the first shadow map and the second shadow map and the number of all pixels in the second shadow map as the overlap ratio between the first shadow map and the second shadow map; If the overlap rate between the first shadow map and the second shadow map is greater than or equal to the overlap rate threshold, determining that the first shadow map needs to be optimized; If the overlap ratio between the first shadow map and the second shadow map is less than the overlap ratio threshold, it is determined that the first shadow map does not need to be optimized.

7. The method for generating a shadow map according to claim 5, characterized in that: If the optimization analysis result indicates that the first shadow map needs to be optimized, the pixels in the first shadow map are corrected using the pixels in the second shadow map to obtain a third shadow map, including: If the optimization analysis result indicates that the first shadow map needs to be optimized, for each group of pixels, determine whether the first pixel is smaller than the second pixel; wherein pixels at the same position in the first shadow map and the second shadow map are a group of pixels; the first pixel is a pixel in the first shadow map; and the second pixel is a pixel in the second shadow map; If it is determined that the first pixel is smaller than the second pixel, the first pixel is corrected according to the second pixel, the exposure parameter, the influence value of the main light source, and the influence value of the secondary light source to obtain a third pixel; Determining whether the second pixel is smaller than the first pixel; If it is determined that the second pixel is smaller than the first pixel, the third pixel is corrected according to the first pixel, the exposure parameter, the influence value of the main light source, and the influence value of the secondary light source to obtain a fourth pixel; A third shadow map is generated based on all the uncorrected third pixels and all the fourth pixels.

8. A shadow map generation device, characterized in that: include: A first acquisition unit is used to acquire light source information of each light source; wherein the light source is divided into a real light source and a virtual light source; the light source information includes light intensity and a distance between the light source and the entity; A first determining unit, configured to determine, for each light source, an influence value of the light source on the entity according to the illumination intensity of the light source and the distance between the light source and the entity; A second acquisition unit is used to acquire a depth map of the entity photographed by the depth camera at the main light source and a rendering image of the virtual camera; wherein the main light source is the light source with the greatest impact on the entity among all light sources; A first generating unit, configured to generate a scene depth map under the main light source according to the rendered image; The second generating unit is used to generate a first shadow map according to the depth map of the entity and the scene depth map under the main light source.

9. An electronic device, characterized in that: include: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the shadow map generation method according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that: A computer program is stored thereon, wherein when the computer program is executed by a processor, the method for generating a shadow map as claimed in any one of claims 1 to 7 is implemented.

Images