Virtual object shadow processing method and device, electronic device, and storage medium

Abstract

The present disclosure provides a virtual object shadow processing method, a virtual object shadow processing device, an electronic device and a computer-readable storage medium, which belong to the field of computer technology. The method comprises: obtaining an image to be processed including a first object and a second object; the second object is a virtual object; determining a virtual object model corresponding to the second object; in a three-dimensional space, using a virtual camera to obtain a projection image of the virtual object model along a target direction under a virtual light source; merging the projection image with the image to be processed to generate a target image with a shadow effect of the second object. The present disclosure can conveniently and efficiently generate a target image with a virtual object shadow effect.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a virtual object shadow processing method, a virtual object shadow processing device, an electronic device, and a computer-readable storage medium. Background Art

[0002] In the fields of games, animations, or film production involving virtual objects, there may be some interactive processes between objects or characters. In order to ensure the rationality and simulation of the interactive scene, the shadow problem caused by the occlusion of other surrounding objects by the object or character is often considered. Therefore, it is necessary to add corresponding shadows to the objects or characters in the scene. The existing technology usually adopts the method of manually drawing shadows in specific shadow areas. However, this method not only requires high manpower and time costs, but also requires the drawing personnel to master technical capabilities such as perspective and proportion, and the accuracy of shadow drawing is difficult to guarantee.

[0003] Therefore, how to conveniently and accurately draw corresponding shadows for virtual objects is a problem that needs to be solved urgently in the prior art.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The present disclosure provides a virtual object shadow processing method, a virtual object shadow processing device, an electronic device and a computer-readable storage medium, thereby overcoming the problem that the prior art cannot accurately and conveniently generate the shadow of a virtual object, at least to a certain extent.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a virtual object shadow processing method is provided, comprising: obtaining an image to be processed including a first object and a second object; the second object being a virtual object; determining a virtual object model corresponding to the second object; in three-dimensional space, obtaining a projection image of the virtual object model along a target direction under a virtual light source using a virtual camera; and merging the projection image with the image to be processed to generate a target image with a shadow effect of the second object.

[0008] According to one aspect of the present disclosure, a virtual object shadow processing device is provided, including: a to-be-processed image acquisition module, used to acquire an to-be-processed image including a first object and a second object; the second object is a virtual object; a virtual object model determination module, used to determine a virtual object model corresponding to the second object; a projection image determination module, used to acquire, in a three-dimensional space, a projection image of the virtual object model along a target direction under a virtual light source using a virtual camera; and a target image generation module, used to merge the projection image with the to-be-processed image to generate a target image with a second object shadow effect.

[0009] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute any one of the virtual object shadow processing methods described above by executing the executable instructions.

[0010] According to one aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the virtual object shadow processing method described in any one of the above is implemented.

[0011] The exemplary embodiments of the present disclosure have the following beneficial effects:

[0012] Obtain an image to be processed including a first object and a second object; the second object is a virtual object; add a virtual object model of the second object in a three-dimensional space; use a virtual camera to obtain a projection image of the virtual object model along a target direction under a virtual light source; merge the projection image with the image to be processed to generate a target image with a shadow effect of the second object. On the one hand, this exemplary embodiment proposes a new shadow processing method, which can use a virtual camera, a virtual light source and a virtual object model in a three-dimensional space to obtain a projection image of the virtual object model, and generate a target image with a shadow effect of the second object according to the projection image and the image to be processed. This exemplary embodiment does not require human participation in shadow drawing. The projection image can be obtained by constructing a virtual scene, and the shadow production process is more convenient and efficient. Compared with the method of drawing shadows in an image in the prior art, this exemplary embodiment uses a virtual camera to obtain a projection image of the virtual object model under a virtual light source. Considering that the virtual light source can better simulate the light in the scene, the projection image determined based on this method is also more accurate, which can improve the shadow display effect of the second object in the target image, and the shadow has higher rationality and simulation.

[0013] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0015] Figure 1 A flowchart of a virtual object shadow processing method in this exemplary embodiment is schematically shown;

[0016] Figure 2 A schematic diagram schematically illustrates a first image in this exemplary embodiment;

[0017] Figure 3 A schematic diagram schematically illustrates a second image in this exemplary embodiment;

[0018] Figure 4 A schematic diagram schematically shows an image to be processed in this exemplary embodiment;

[0019] Figure 5 A schematic diagram schematically illustrates a screen obtained by observing the projection of a virtual object model at the position of a virtual camera in this exemplary embodiment;

[0020] Figure 6 A schematic diagram schematically illustrates a virtual light source and a virtual camera in this exemplary embodiment;

[0021] Figure 7 A schematic diagram schematically illustrates configuring a virtual light source and a virtual camera in a three-dimensional space and creating a virtual projection surface in this exemplary embodiment;

[0022] Figure 8 A schematic diagram schematically illustrates adjusting any one or more of a virtual light source, a virtual camera, and a virtual projection surface in a three-dimensional space in this exemplary embodiment;

[0023] Fig. 9 A schematic diagram of a projection obtained in an adjustment state in this exemplary embodiment is schematically shown;

[0024] Fig.10 A schematic diagram of a projection obtained in another adjustment state in this exemplary embodiment is schematically shown;

[0025] Fig.11 A schematic diagram of a position state of an object after a change in this exemplary embodiment is schematically shown;

[0026] Fig.12 A schematic diagram schematically shows the position states of a virtual projection surface and a virtual object model after an object changes in this exemplary embodiment;

[0027] Fig.13 A schematic diagram schematically shows a relationship between a dynamic virtual object model and a position change of a virtual projection surface in this exemplary embodiment;

[0028] Fig.14 A schematic diagram schematically illustrates a projected image in this exemplary embodiment;

[0029] Fig.15 A schematic diagram schematically shows a display comparison of a projection image after blurring in this exemplary embodiment;

[0030] Fig.16 A schematic diagram of an architecture flow chart of a virtual object shadow processing method in this exemplary embodiment is shown;

[0031] Fig.17 A block diagram schematically shows a structure of a virtual object shadow processing device in this exemplary embodiment;

[0032] Fig.18 An electronic device for implementing the above method in this exemplary embodiment is schematically shown. DETAILED DESCRIPTION

[0033] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0034] An exemplary embodiment of the present disclosure first provides a virtual object shadow processing method.

[0035] The following is combined with Figure 1 This exemplary embodiment is further described as follows. Figure 1 As shown, the virtual object shadow processing method may include the following steps S110 to S140:

[0036] Step S110, acquiring an image to be processed including a first object and a second object; the second object is a virtual object.

[0037] Among them, the image to be processed refers to the image to which shadows need to be added, and the first object and the second object refer to the objects included in the image to be processed, which can be an image captured by a camera, a key frame captured from a video frame, an image directly obtained from a data source, or an image that can be generated by image synthesis, etc.

[0038] In this exemplary embodiment, the first object refers to the shadow recipient, the second object refers to the shadow generating subject, the second object interacting with the first object will generate a shadow belonging to the second object on the first object, and the second object may be a virtual object. For example, the image to be processed may include a real person character and a virtual animal, then the first object is a real person character, and the second object is a virtual animal. In this case, the shadow of the virtual animal can be added to the real person character. The first object may be a virtual object or a physical object, and this is not specifically limited in the present disclosure.

[0039] In an exemplary embodiment, the above step S110, obtaining the image to be processed including the first object and the second object, may include:

[0040] Acquire a first image including a first object and a third object, and a second image including a second object; wherein the second object is a virtual object, and the third object is a physical object corresponding to the second object;

[0041] The first image is combined with the second image to generate an image to be processed.

[0042] This exemplary embodiment can obtain a first image and a second image, and synthesize the first image and the second image to obtain an image to be processed. The first image includes a first object and a third object, the second image includes a second object, and the second object can be a virtual object. For example, the second image can be an image including a virtual object model obtained by a technician, and the third object can be a physical object corresponding to the second object. The first object can be a virtual object or a physical object. For example, Figure 2 A schematic diagram of a first image is shown, which includes a person A, i.e., a first object, and a prop B, i.e., a third object. Figure 2 In the example, both character A and prop B are entity objects. Figure 3 A schematic diagram of a second image is shown, which includes a virtual object C corresponding to the prop B, that is, the second object. The virtual object C may be a pre-established virtual object model.

[0043] In this exemplary embodiment, the second object is a virtual object, and the third object is a physical object corresponding to the second object. There is a corresponding relationship between the second object and the third object, and the corresponding relationship may be a relationship of similar or identical appearance, for example, the third object is a wooden sculpture of a "dog", and the second object is a virtual model of a "dog". The corresponding relationship may also be simply a relationship in which the second object is a virtual object of the third object, while the appearance, size, shape and other information of the second object and the third object are different, for example, the third object is a virtual model of a "dog", and the second model is a "pillow", and so on.

[0044] In practical applications, such as in filming or special effects production scenes, actors are often required to interact with fixed props, and then the animation effects of virtual objects corresponding to the fixed props are produced in post-production, and finally the interaction effect between the actors and the virtual objects is obtained. In this case, the shadows produced by the fixed props on the actors may not match the shadows produced by the virtual objects on the actors. Therefore, the shadow image of the virtual object can be produced by this exemplary embodiment to obtain the target image. Among them, the actors can be regarded as the first object, the fixed props can be regarded as the third object, and the virtual object can be regarded as the second object.

[0045] By merging the first image and the second image, an image to be processed including the first object and the virtual object can be obtained. Figure 4 As shown, at this point, the synthesis of the virtual object and the first object is completed, and the step of making the shadow of the virtual object can be further performed. It should be noted that the size, shape and other proportions of the virtual object in the image to be processed may be different from those of the third object in the first image. For example, the virtual object that is usually presented to interact with the actor will be larger than the physical prop used by the actor to replace the virtual object during filming. This disclosure does not make specific limitations on this.

[0046] This exemplary embodiment may use preset tools or software to implement image processing or shadow production processes. For example, the first image and the second image may be imported into nuke (a compositing software). The second image may be a virtual object rendered by DCC (digital content creation) software, such as a sequence frame in exr format rendered according to a lens.

[0047] Step S120: determining a virtual object model corresponding to the second object.

[0048] This exemplary embodiment can simulate and construct a three-dimensional space for generating the shadow of the virtual object, which can be specifically implemented using preset software. Then, a virtual object model corresponding to the second object can be determined in the three-dimensional space. For example, a pre-constructed virtual object model of the second object can be imported into the software, or a virtual object model corresponding to the second object can be added or created in the software.

[0049] The virtual object model corresponding to the second object refers to a three-dimensional model constructed based on the second object, which can be a static model or a dynamic model. When the virtual object model is a dynamic model, it can be regarded as adding an animation sequence about the second object in the three-dimensional space, that is, the virtual object model of the second object can be presented in the form of a dynamic animation in the three-dimensional space.

[0050] Step S130: In three-dimensional space, a virtual camera is used to obtain a projection image of the virtual object model along a target direction under a virtual light source.

[0051] The virtual camera refers to a camera configured in a three-dimensional space for capturing the projection of a virtual object model, and the virtual light source refers to a light source configured in a three-dimensional space for simulating ambient lighting. In this exemplary embodiment, the light emitted by the virtual light source may be directional, similar to a spotlight. The target direction refers to the direction in which the virtual camera acquires a projection image of the virtual object model. The projection image may be captured from the target direction, which may be the direction from the virtual light source to the virtual object model.

[0052] In this exemplary embodiment, the positions of the virtual camera and the virtual light source can be determined in the three-dimensional space based on experience or setting conditions, and the virtual camera and the virtual light source can be configured at the corresponding positions. For example, the light source effect that is the same as or matches the original light source layout in the image to be processed can be determined, or the placement position of the corresponding virtual light source can be determined based on the preset reasonable light source effect. At this time, the three-dimensional space can include a virtual object model, a virtual camera, and a virtual light source, and then the virtual camera can be used to obtain the projection image of the virtual object model along the target direction under the virtual light source. When obtaining the projection image, the shape of the obtained projection image can also be adjusted by adjusting the position of the virtual camera or the virtual light source.

[0053] Step S140: merging the projected image with the image to be processed to generate a target image with a shadow effect of the second object.

[0054] Finally, the generated projection image is added to the image to be processed, so that the target image can be obtained. The generated projection image includes the projection of the second object, so the target image is also an image with the shadow effect of the second object.

[0055] Before merging the projected image with the image to be processed, the projected image may be processed, for example, by blurring or material processing the projected image, so as to generate a more reasonable and accurate projected image to be merged with the image to be processed.

[0056] Based on the above description, in this exemplary embodiment, an image to be processed including a first object and a second object is obtained; the second object is a virtual object; a virtual object model of the second object is added in a three-dimensional space; a projection image of the virtual object model under a virtual light source along a target direction is obtained by using a virtual camera; the projection image is merged with the image to be processed to generate a target image with a shadow effect of the second object. On the one hand, this exemplary embodiment proposes a new shadow processing method, which can obtain a projection image of the virtual object model by using a virtual camera, a virtual light source and a virtual object model in a three-dimensional space, and generate a target image with a shadow effect of the second object according to the projection image and the image to be processed. This exemplary embodiment does not require human participation in shadow drawing, and a projection image can be obtained by constructing a virtual scene, and the shadow production process is more convenient and efficient; compared with the method of drawing shadows in an image in the prior art, this exemplary embodiment uses a virtual camera to obtain a projection image of a virtual object model under a virtual light source. Considering that the virtual light source can better simulate the light in the scene, the projection image determined based on this method is also more accurate, which can improve the shadow display effect of the second object in the target image, and the shadow has higher rationality and simulation.

[0057] In an exemplary embodiment, the target direction is the direction from the virtual light source to the virtual object model; the above step S130, in the three-dimensional space, using the virtual camera to obtain the projection image of the virtual object model along the target direction under the virtual light source, may include:

[0058] configuring a virtual light source and a virtual camera in a three-dimensional space, and creating a virtual projection surface for receiving the projection of the virtual object model;

[0059] A virtual camera is used to shoot the projection of the virtual object model on the virtual projection surface along the target direction under the illumination of the virtual light source to obtain a projection image of the virtual object model.

[0060] Among them, configuring the virtual light source and the virtual camera in the three-dimensional space can be to first determine the positions of the virtual light source and the virtual camera in the three-dimensional space, and then configure the virtual light source and the virtual camera at the corresponding positions. The virtual light source is used to illuminate the virtual object model. The position of the virtual camera can be regarded as a viewing angle position for observing the projection of the virtual object model under lighting. The projection image captured by the virtual camera is equivalent to the picture obtained by observing the projection of the virtual object model at the position of the virtual camera. The projection image can be, for example Figure 5 shown.

[0061] In this exemplary embodiment, the position of the virtual light source and the position of the virtual camera can be determined separately. The position of the virtual light source can be a position that matches the lighting effects in the image to be processed. For example, a position that matches the lighting effects of the image to be processed can be imported in three-dimensional space, or a virtual light source can be created in three-dimensional space and adjusted to a target position that matches the lighting effects of the image to be processed, etc.

[0062] The position of the virtual camera may be determined independently or according to the position of the virtual light source. In an exemplary embodiment, the positions of the virtual light source and the virtual camera are the same, or the virtual light source and the virtual camera change synchronously.

[0063] like Figure 6 As shown, in this exemplary embodiment, the virtual light source D and the virtual camera F can be in a binding relationship, and the virtual light source D and the virtual camera F can be in the same position, that is, the virtual light source is set at the position of the virtual camera, or the virtual camera is configured at the position of the virtual light source, so that the virtual light source and the virtual camera can change synchronously. For example, when the position of the virtual light source is moved, the virtual camera will move together, and its displacement or posture is the same as that of the virtual light source, that is, it can be regarded as that no matter how the virtual light source moves, the projection image of the virtual object model can be captured from the perspective corresponding to the position of the virtual light source.

[0064] In this exemplary embodiment, a virtual camera can be first imported into the three-dimensional space, such as by using the nuke software, and then a virtual camera can be created in the three-dimensional space, and the axis of the virtual camera can be connected to the virtual light source to bind the virtual light source to the position of the virtual camera in preparation for projection.

[0065] The virtual projection surface refers to a virtual plane for receiving the projection of the virtual object model. The virtual projection surface can be regarded as the first object, and the projection of the virtual object model on the first object can be determined according to the projection of the virtual object model on the virtual projection surface.

[0066] In this exemplary embodiment, the position of the virtual projection plane may be related to the first object. For example, the position of the virtual projection plane in the three-dimensional scene may be determined based on the position of the first object. The virtual projection plane will also be displaced accordingly if the first object is displaced. The position of the virtual projection plane may also be related to the second object or the relative posture of the first object and the second object. For example, in this exemplary embodiment, the position or distance of the first object relative to the second object may be first determined in the scene corresponding to the image to be processed, and then, based on the relative information, the position or distance of the virtual projection plane relative to the virtual object model may be determined in the three-dimensional scene to create a virtual projection plane corresponding to the virtual object model. When the virtual object model is displaced, the virtual projection plane will also be displaced, but the relative information between the virtual object model and the virtual projection plane may remain unchanged, or the technician may fine-tune the virtual projection plane after the displacement change on the basis of keeping the relative information unchanged to ensure the accuracy of its position.

[0067] After creating a virtual light source, a virtual camera, and a virtual projection surface in a three-dimensional space, the projection effect of the virtual object model under lighting conditions can be simulated in a three-dimensional scene. Specifically, a preset renderer can be created in the three-dimensional space of the software program with the help of preset software, such as Nuke software. For example, a Scanlinerender renderer is created to connect the virtual object model with the virtual camera to capture the projection image, and a card, i.e., a virtual projection surface, is created in the three-dimensional space. The virtual projection surface is placed in the three-dimensional space according to the display status of the main lens, and then the projection of the virtual light source to the virtual object model in the direction of the virtual projection surface is observed from the perspective of the virtual light source to obtain the projection image, for example Figure 7 As shown in FIG. 1 , a virtual light source D and a virtual camera E are configured in a three-dimensional space, and a virtual projection surface J is created for receiving the projection of the virtual object model F. Then, the virtual camera can be used to shoot the projection of the virtual object model on the virtual projection surface along the target direction under the illumination of the virtual light source to obtain the following image: Figure 5 Projected image of the virtual object model shown.

[0068] In an exemplary embodiment, the above-mentioned using a virtual camera to shoot the projection of the virtual object model on the virtual projection surface along the target direction under the illumination of the virtual light source to obtain the projection image of the virtual object model may include:

[0069] In three-dimensional space, the position and posture of any one or more of the virtual light source, virtual camera, and virtual projection surface are adjusted according to preset shooting conditions, and the virtual camera is used to shoot the projection of the virtual object model on the virtual projection surface along the target direction under the illumination of the virtual light source to obtain the projection image of the virtual object model.

[0070] This exemplary embodiment can adjust the position and posture of any one or more of the virtual light source, virtual camera, and virtual projection surface according to preset shooting conditions to obtain a projection image of the virtual object model. Adjusting the position and posture of any one or more of the virtual light source, virtual camera, and virtual projection surface can be adjusting the position or posture, for example, the virtual projection surface can be moved forward and backward, left and right, up and down, and can also be rotated. Each adjustment can be made to only one of them, or multiple ones can be adjusted, for example, the position of the virtual projection surface and the virtual camera can be adjusted at the same time. It should be noted that after the virtual camera and the virtual light source are bound, adjusting the virtual camera can be regarded as adjusting the virtual light source.

[0071] The preset shooting conditions may be judgment conditions for determining whether the projection of the virtual object model on the virtual projection surface is appropriate. The preset shooting conditions may be manually adjusted and judged. For example, when adjusting the posture of any one or more of the virtual light source, the virtual camera, and the virtual projection surface, according to the judgment of the adjuster, when it is determined that the projection of the virtual object model is appropriate, the virtual camera is used to shoot the corresponding projection image. The preset shooting conditions may also be set in advance according to the configuration information, such as conditions determined according to information such as the expected reasonable projection size, direction, offset direction, radial distortion degree, etc. When the preset shooting conditions are met during the process of adjusting one or more of the virtual light source, the virtual camera, and the virtual projection surface, the virtual camera may be used to shoot the projection of the virtual object model on the virtual projection surface along the target direction under the illumination of the virtual light source to obtain the projection image of the virtual object model. Figure 8 A schematic diagram of adjusting the posture of any one or more of a virtual light source, a virtual camera, and a virtual projection surface in a three-dimensional space is shown, wherein multiple adjustment parameters are given as examples, such as Translate, Rotate, Scale, and specific adjustment values ​​on the x, y, and z axes under each adjustment parameter, wherein the adjustment value is determined according to actual adjustment needs, and the present disclosure does not make specific limitations on this. Fig. 9 A schematic diagram of the corresponding projection obtained under an adjustment state is shown. Fig.10 A schematic diagram of the projection obtained under another adjustment state is shown, from which it can be seen that the position and other forms of the projection can be changed by adjusting the virtual light source or the virtual camera.

[0072] In an exemplary embodiment, the determining of the virtual object model corresponding to the second object may include:

[0073] determining animation information of a virtual object model corresponding to the second object;

[0074] The virtual object shadow processing method may further include the following steps:

[0075] determining first position change information of the virtual object model from the animation information;

[0076] Determine second position change information of the virtual projection surface according to the first position change information; the first position change information is associated with the second position change information;

[0077] According to the second position change information, the virtual projection surface is controlled to change its position in the three-dimensional space.

[0078] Considering that in actual scenes, the first object and the second object may undergo displacement changes in the scene, for example Figure 2 In the scene, character A moves with prop B and changes position. After the change, Fig.11 As shown, the position state of the character A' and the virtual object B' becomes. In this state, if a correct and reasonable virtual projection surface needs to be obtained, it is necessary to adjust synchronously or create a virtual projection surface of the virtual object model in this state, such as Fig.12 shown.

[0079] In this exemplary embodiment, the virtual object model corresponding to the second object may have animation effects. Therefore, when determining the virtual object model corresponding to the second object, this exemplary embodiment may directly determine the animation information of the virtual object model. The animation information may reflect changes in the displacement, posture or other states of the virtual object model. It may be a key frame sequence of the animation of the virtual object model. Each key frame may include state information of the virtual object model at a certain moment, such as position or posture information.

[0080] Then, the first position change information of the virtual object model can be determined from the animation information, and the first position change information refers to the position change of the virtual object model, such as the displacement change of up and down, left and right, front and back, etc. Further, according to the first position change information, the second position change information of the virtual projection surface is determined, and the second position change information refers to the position change of the virtual projection surface. According to the second position change information, the virtual projection surface can be controlled to move in three-dimensional space. In this exemplary embodiment, the first position change information and the second position change information have an associated relationship, and the associated relationship can be the same position change direction, the same position change amount, or the relative distance between the virtual object model and the virtual projection surface remains unchanged.

[0081] For example, in the preset software, the virtual projection plane can be associated with the position information of the virtual object model. The virtual object model carries animation information. After the association, the virtual projection plane also obtains the displacement information associated with the animation information of the virtual object model. That is, the virtual projection plane will move in the same way as the virtual object model, and the relative distance between the virtual object model and the virtual projection plane can remain unchanged. Fig.13A schematic diagram of a dynamic position change relationship between a virtual object model and a virtual projection surface is shown. The virtual object model is a model with an animation effect. The virtual object model F1 is the virtual object model before the position change, and the virtual object model F2 is the virtual object model after the position change. This exemplary embodiment can determine first position change information based on the animation information of the virtual object model, and determine second position change information of the virtual projection surface associated with the first position change information, so that the virtual projection surface can change with the change of the virtual object model, such as Fig.13 As shown, the virtual projection plane J1 is the virtual projection plane before the position change, and the virtual projection plane J2 is the virtual projection plane after the position change. Regardless of before or after the position change, the relative distance between the virtual object model and the virtual projection plane remains unchanged.

[0082] It should be noted that in actual applications, the first object may remain stationary, but the second object may move, causing the distance between the first object and the second object to change. Even if the relative distance between the first object and the second object remains unchanged, the first object may also have its own changes, such as rotating direction while the actor maintains the relative distance from the prop unchanged. Therefore, this exemplary embodiment can first make the virtual projection surface follow the virtual object model to change in the above manner, and then adjust the posture of the virtual projection surface separately after the change is completed, such as rotating the virtual projection surface to adapt to the state of the actor's rotation direction, or moving the virtual projection surface back to its original position to adapt to the state where the actor's position has not changed, etc.

[0083] In an exemplary embodiment, after using a virtual camera to acquire a projection image of a virtual object model along a target direction under a virtual light source, the virtual object shadow processing method may further include:

[0084] Post-processing the projection image to obtain a processed projection image; wherein the post-processing includes material processing and / or blur processing;

[0085] The projected image is combined with the image to be processed to generate a target image with a shadow effect of the second object, including:

[0086] The post-processed projection image is combined with the image to be processed to generate a target image with a shadow effect of the second object.

[0087] In order to ensure that the projected image can be used correctly and effectively, the present exemplary embodiment may perform post-processing on the obtained projected image, for example, it may perform material processing, blur processing, or material processing and blur processing. The material processing and blur processing may be performed sequentially or separately and then merged. The present disclosure does not make specific limitations on this. After completing the post-processing, the projected image may be merged with the image to be processed, for example, the projected image may be superimposed on the image to be processed, so as to reasonably add the projection of the virtual object model to the image to be processed to obtain the target image.

[0088] In this exemplary embodiment, material processing of the projection image may be performed by creating a material sticker of a specific color, such as a white material sticker, and a projection material, such as project3D, in a preset software, and then connecting the projection image with the projection material, and adding a mergemat material node to mix and superimpose the material map with the projection material to obtain a projected shadow image displayed against a background of a specific color, and finally, assigning the shadow image to the virtual projection surface through an applymaterial node, connecting the main camera, and rendering a projection image with a shadow effect through a scanlinerender node, such as Fig.14 shown.

[0089] The blurring of the projected image may be blurring of the projection in the projected image to create a soft shadow effect and improve the realistic effect of the shadow display. The blurring may include a variety of methods, such as Gaussian blurring, mean blurring, median blurring, radial blurring, etc., which are not specifically limited in the present disclosure. Fig.15 FIG. 4 shows a schematic diagram of display comparison of a projection image after blurring. Fig.15 (a) is the effect of adding no projection image to the image to be processed. Fig.15 (b) The effect of adding an unblurred projection image to the image to be processed. Fig.15 (c) The effect of adding a blurred projected image to the image to be processed.

[0090] It should be noted that post-processing of the projection image can be performed after the projection image is obtained on the virtual projection surface, or it can be performed during the projection process of the virtual projection surface. For example, after configuring a virtual light source and a virtual camera in three-dimensional space, the virtual light source illuminates the virtual object model, and the projection is observed through the virtual camera. At this time, the projection can be post-processed to finally obtain a post-processed projection image.

[0091] Fig.16The architecture flow chart of a virtual object shadow processing method in this exemplary embodiment is shown, which may specifically include the following steps: adding a first image 1602 in the preset software; the first image includes a first object and a third object, and the first object and the third object are physical objects; adding a second image 1604 in the preset software; the second image includes a second object, and the second object is a virtual object of the third object; determining an image to be processed 1606 according to the first image and the second image; the image to be processed includes the first object and the second object; determining a virtual object model 1608 corresponding to the second object in three-dimensional space, and the virtual object model may be a model file with animation information, which may be regarded as a model with animation; configuring a virtual light source 1610 and a virtual camera 1612 in three-dimensional space, and the virtual camera is connected to the virtual light source through an axis so that the positions of the virtual camera and the virtual light in three-dimensional space coincide. ; In three-dimensional space, create a scannerender renderer to connect the virtual object model and the virtual camera to obtain the projection of the virtual object model along the target direction under the virtual light source from the virtual camera position 1614; add a blur effect to the projection 1616; obtain the material map of the projection 1618 and the projection material 1620; mix and overlay the material map and the projection material 1622 to ensure that the projected shadow can be displayed under the preset color background; assign the obtained projection material to the virtual projection surface 1624 through the applymaterial node for material application 1626, and connect the main camera 1628 to render the projection image 1630 through the scannerender while matching the main camera state; finally, merge the projection image with the image to be processed to generate a target image 1632 with the shadow effect of the second object.

[0092] The main camera refers to the real-time camera used to observe the global scene. The main camera information can be obtained by inverting the real-time picture. The difference between the main camera and the virtual camera is that the virtual camera is used to observe the projection from the virtual light source position, while the main camera can observe all objects in the scene. For example, Figure 2 The camera corresponding to the image to be processed is shown, or as observed Figure 7 The observation camera of the overall scene shown, etc., in this exemplary embodiment, the shadow effect of the target image finally obtained is related to both the virtual camera and the main camera.

[0093] The exemplary embodiment of the present disclosure also provides a virtual object shadow processing device. Fig.17The device 1700 may include a to-be-processed image acquisition module 1710, used to acquire an to-be-processed image including a first object and a second object; the second object is a virtual object; a virtual object model determination module 1720, used to determine a virtual object model corresponding to the second object; a projection image determination module 1730, used to acquire a projection image of the virtual object model along a target direction under a virtual light source in a three-dimensional space by using a virtual camera; and a target image generation module 1740, used to merge the projection image with the to-be-processed image to generate a target image with a shadow effect of the second object.

[0094] In an exemplary embodiment, the target direction is the direction from the virtual light source to the virtual object model; the projection image determination module 1730 includes: a projection surface creation unit, which is used to configure the virtual light source and the virtual camera in the three-dimensional space, and create a virtual projection surface for receiving the projection of the virtual object model; a projection image acquisition unit, which is used to use the virtual camera to shoot the projection of the virtual object model on the virtual projection surface along the target direction under the illumination of the virtual light source, so as to obtain the projection image of the virtual object model.

[0095] In an exemplary embodiment, the projection image acquisition unit includes: an adjustment subunit, which is used to adjust the posture of any one or more of the virtual light source, the virtual camera, and the virtual projection surface in the three-dimensional space according to preset shooting conditions, and use the virtual camera to shoot the projection of the virtual object model on the virtual projection surface along the target direction under the illumination of the virtual light source to obtain the projection image of the virtual object model.

[0096] In an exemplary embodiment, the virtual light source and the virtual camera are at the same position, or the virtual light source and the virtual camera are in a synchronously changing relationship.

[0097] In an exemplary embodiment, the virtual object model determination module 1720 includes: an animation information determination unit, used to determine the animation information of the virtual object model corresponding to the second object; the virtual object shadow processing device also includes: a first position change determination unit, used to determine the first position change information of the virtual object model from the animation information; a second position change determination unit, used to determine the second position change information of the virtual projection surface according to the first position change information; the first position change information is associated with the second position change information; a position change control unit, used to control the virtual projection surface to change its position in the three-dimensional space according to the second position change information.

[0098] In an exemplary embodiment, after using a virtual camera to obtain a projection image of the virtual object model along the target direction under a virtual light source, the virtual object shadow processing device also includes: a post-processing unit, used to post-process the projection image to obtain a processed projection image; wherein the post-processing includes material processing and / or blur processing; a target image generation module 1740, including: an image merging unit, used to merge the post-processed projection image with the image to be processed to generate a target image with a second object shadow effect.

[0099] In an exemplary embodiment, the module 1710 for acquiring an image to be processed includes: an image acquisition unit, used to acquire a first image including a first object and a third object, and a second image including a second object; wherein the second object is a virtual object, and the third object is a physical object corresponding to the second object; and an image merging unit, used to merge the first image with the second image to generate the image to be processed.

[0100] The specific details of each part of the above-mentioned device have been described in detail in the implementation method of the method part. The undisclosed details can be found in the implementation method of the method part, so they will not be repeated here.

[0101] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the exemplary embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0102] The exemplary embodiments of the present disclosure also provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the virtual object shadow processing method is implemented.

[0103] In one embodiment, the computer program product may be a tangible product containing a computer program, such as a computer-readable storage medium storing a computer program. The readable storage medium may be a storage medium based on electrical, magnetic, optical, electromagnetic, infrared, or other signals, including but not limited to: random access memory (RAM), read-only memory (ROM), magnetic tape, floppy disk, flash memory (Flash), mechanical hard disk (HDD), solid-state drive (SSD), and the like. Exemplarily, the computer program product may be implemented as a non-volatile storage medium storing a computer program, such as a read-only memory, a NAND flash memory, and the like.

[0104] In one embodiment, the computer program product may be an intangible product including a computer program. Exemplarily, the computer program product may be implemented as a virtual digital product, such as a digital file storing an executable file, an installation package, etc. of the computer program.

[0105] The code of the computer program can be written in one or more programming languages. Programming languages ​​such as C language, Java, C++, etc. The program code can be executed entirely on the user computing device, or partially on the user computing device, or as a separate software package, or partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, such as a local area network (LAN), a wide area network (WAN), etc., or can be connected to an external computing device (e.g., an Internet connection provided by an operator).

[0106] The computer program may be carried or transmitted via electrical, magnetic, optical, electromagnetic, infrared, or other signals. The electronic device may convert the signal carrying the computer program into a digital signal, thereby running the computer program. When the computer program is run on an electronic device, its code is used to enable the electronic device to execute (more specifically, the processor of the electronic device may execute) the method steps of various exemplary embodiments of the present disclosure, such as the above-mentioned virtual object shadow processing method.

[0107] The exemplary embodiment of the present disclosure also provides an electronic device. The electronic device may include a processor and a memory. The memory stores executable instructions of the processor, such as program code. The processor executes the method in the exemplary embodiment by executing the executable instructions. In addition, the electronic device may also include a display for displaying a graphical user interface.

[0108] Reference below Fig.15 , the electronic device is exemplified in the form of a general-purpose computing device. It should be understood that Fig.15 The electronic device 1500 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0109] like Fig.15 As shown, the electronic device 1500 may include: a processor 1510 , a memory 1520 , a bus 1530 , an I / O (input / output) interface 1540 , a network adapter 1550 , and a display 1560 .

[0110] The memory 1520 may include a volatile memory, such as a RAM 1521, a cache unit 1522, and may also include a non-volatile memory, such as a ROM 1523. The memory 1520 may also include one or more program modules 1524, such program modules 1524 include but are not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof may include the implementation of a network environment. For example, the program module 1524 may include each module in the above-mentioned device.

[0111] Processor 1510 may include one or more processing units. For example, processor 1510 may include an AP (Application Processor), a modem processor, a GPU (Graphics Processing Unit), an ISP (Image Signal Processor), a controller, an encoder, a decoder, a DSP (Digital Signal Processor), a baseband processor and / or an NPU (Neural-Network Processing Unit), etc.

[0112] The processor 1510 may be used to execute executable instructions stored in the memory 1520, such as executing the above-mentioned virtual object shadow processing method.

[0113] The bus 1530 is used to realize the connection between different components of the electronic device 1500, and may include a data bus, an address bus, and a control bus.

[0114] The electronic device 1500 can communicate with one or more external devices 1600 (eg, a keyboard, a mouse, an external controller, etc.) through the I / O interface 1540 .

[0115] The electronic device 1500 can communicate with one or more networks through the network adapter 1550. For example, the network adapter 1550 can provide mobile communication solutions such as 3G / 4G / 5G, or provide wireless communication solutions such as wireless LAN, Bluetooth, near field communication, etc. The network adapter 1550 can communicate with other modules of the electronic device 1500 through the bus 1530.

[0116] The electronic device 1500 may display a graphical user interface through the display 1560 , such as displaying a game editing scene.

[0117] although Fig.15Not shown, other hardware and / or software modules may also be provided in the electronic device 1500, including but not limited to: a display, a microcode, a device driver, a redundant processor, an external disk drive array, a RAID system, a tape drive, and a data backup storage system.

[0118] As can be seen from the above, the technical solution of the present disclosure can be implemented as a method, an apparatus, a system, a computer program product, a storage medium, an electronic device, etc. Those skilled in the art can understand that various aspects of the present disclosure can be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software, such as being respectively referred to as a "circuit", "module" or "system".

[0119] It should be understood that the present disclosure is not limited to the specific method steps or structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. Those skilled in the art will easily think of other embodiments based on the specific embodiments provided by the present disclosure. Therefore, the specific embodiments provided by the present disclosure are only exemplary, and the scope and spirit of the present disclosure are indicated by the claims, and any variations, uses or adaptive changes of the present disclosure should be covered, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the technical field that are not disclosed in the present disclosure.

[0120] It will be appreciated by those skilled in the art that various aspects of the present disclosure may be implemented as a system, method or program product. Therefore, various aspects of the present disclosure may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which may be collectively referred to herein as a "circuit", "module" or "system". Those skilled in the art will readily think of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any variations, uses or adaptive changes of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and implementation are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0121] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A virtual object shadow processing method, characterized in that: include: Acquire an image to be processed including a first object and a second object; The second object is a virtual object; determining a virtual object model corresponding to the second object; In three-dimensional space, a virtual camera is used to obtain a projection image of the virtual object model along a target direction under a virtual light source; The projected image is combined with the image to be processed to generate a target image with a shadow effect of a second object.

2. The method according to claim 1, characterized in that: The target direction is the direction from the virtual light source to the virtual object model; the step of acquiring a projection image of the virtual object model along the target direction under the virtual light source by using a virtual camera in the three-dimensional space includes: configuring a virtual light source and a virtual camera in the three-dimensional space, and creating a virtual projection surface for receiving the projection of the virtual object model; The virtual camera is used to shoot a projection of the virtual object model on the virtual projection plane along the target direction under the illumination of the virtual light source, so as to obtain a projection image of the virtual object model.

3. The method according to claim 2, characterized in that The step of using the virtual camera to shoot the projection of the virtual object model on the virtual projection surface along the target direction under the illumination of the virtual light source to obtain a projection image of the virtual object model includes: In the three-dimensional space, the posture of any one or more of the virtual light source, the virtual camera, and the virtual projection plane is adjusted according to preset shooting conditions, and the virtual camera is used to shoot the projection of the virtual object model on the virtual projection plane along the target direction under the illumination of the virtual light source to obtain a projection image of the virtual object model.

4. The method according to claim 3, characterized in that The virtual light source and the virtual camera have the same position, or the virtual light source and the virtual camera change synchronously.

5. The method according to claim 2, characterized in that: The determining of the virtual object model corresponding to the second object comprises: determining animation information of a virtual object model corresponding to the second object; The method further comprises: determining first position change information of the virtual object model from the animation information; Determine second position change information of the virtual projection surface according to the first position change information; the first position change information is associated with the second position change information; The virtual projection plane is controlled to change its position in the three-dimensional space according to the second position change information.

6. The method according to claim 1, characterized in that After acquiring a projection image of the virtual object model along a target direction under a virtual light source by using a virtual camera, the method further includes: Post-processing the projection image to obtain a processed projection image; wherein the post-processing includes material processing and / or blur processing; The step of merging the projected image with the image to be processed to generate a target image having a shadow effect of a second object includes: The post-processed projection image is combined with the image to be processed to generate a target image with a shadow effect of the second object.

7. The method according to claim 1, characterized in that The acquiring of the image to be processed including the first object and the second object comprises: Acquire a first image including a first object and a third object, and a second image including a second object; wherein the second object is a virtual object, and the third object is a physical object corresponding to the second object; The first image is combined with the second image to generate the image to be processed.

8. A virtual object shadow processing device, characterized in that: include: A module for acquiring an image to be processed, used for acquiring an image to be processed including a first object and a second object; The second object is a virtual object; A virtual object model determining module, configured to determine a virtual object model corresponding to the second object; A projection image determination module, used to obtain a projection image of the virtual object model along a target direction under a virtual light source in a three-dimensional space by using a virtual camera; The target image generation module is used to merge the projection image with the image to be processed to generate a target image with a shadow effect of a second object.

9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to execute the virtual object shadow processing method described in any one of claims 1-7 by executing the executable instructions.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the virtual object shadow processing method according to any one of claims 1 to 7 is implemented.

Images