A special effect rendering method, device, equipment and storage medium

Abstract

The present disclosure provides a special effect rendering method, device, equipment and storage medium, the method comprises: acquiring a target image, the target image comprises a target object; acquiring a 3D object model, the 3D object model comprises a target area; based on the depth information of the 3D object model and the color information of the region of the 3D object model except the target area, the 3D object model is rendered on the target image, wherein the target object is displayed in the target area.

Description

Technical Field

[0001] This disclosure relates to the field of special effects rendering technology, and in particular to a special effects rendering method, apparatus, device and storage medium. Background Technology

[0002] Currently, when rendering special effects on images, the common practice is to apply special effects or rendering materials directly to the image. For example, during photo taking or short video shooting, sticker effects or beauty effects are often added to faces in photos or short videos. However, this method of rendering special effects is relatively simple and lacks variety. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this disclosure provides a method, apparatus, device and storage medium for special effects rendering.

[0004] According to a first aspect of the present disclosure, a special effects rendering method is provided, comprising: acquiring a target image, the target image including a target object; acquiring a 3D object model, the 3D object model including a target region; and rendering the 3D object model onto the target image based on depth information of the 3D object model and color information of a region on the 3D object model other than the target region, wherein the target object is displayed within the target region.

[0005] In some embodiments, rendering the 3D object model onto the target image based on the depth information of the 3D object model and the color information of the regions on the 3D object model other than the target region includes: assigning the depth information of the target region to the pixels on the target image corresponding to the target region, and assigning the depth information and color information of the regions on the 3D object model other than the target region to the pixels on the target image corresponding to the regions on the 3D object model other than the target region.

[0006] Based on this, by assigning the depth information of the target region on the 3D object model, as well as the depth and color information of the regions on the 3D object model other than the target region, to the corresponding pixels on the target image, the 3D object model can be rendered onto the target image without obscuring the target object. This results in a rendering where a portion of the target image and the 3D object model are integrated into a single image, achieving a fusion of virtual and reality.

[0007] In some embodiments, the 3D object model is located in a pre-acquired 3D scene model. The method further includes: rendering the 3D scene model onto the target image based on the depth and color information of the area outside the covered area in the 3D scene model, wherein the covered area is the area covered by the target area from the perspective of the rendering camera. Based on this, the result of special effects rendering can include both the 3D object model and the 3D scene model, making the rendering results more diverse and engaging.

[0008] In some embodiments, the 3D object model includes multiple sub-object models, and at least one sub-object model of the 3D object model moves along a motion path corresponding to the sub-object model; the sub-object model includes a portion of the 3D object model; and / or,

[0009] The 3D scene model includes multiple sub-scene models, and at least one sub-scene model of the 3D scene model moves along the motion path corresponding to the sub-scene model; the sub-scene model includes a portion of the 3D scene model.

[0010] Based on this, in the rendered image, the 3D scene model includes at least one sub-scene model that can move along the path corresponding to the sub-scene model; and / or the 3D object model includes at least one sub-object model that can move along the path corresponding to the sub-object model. This results in the rendered image including dynamically moving sub-object models and / or sub-scene models.

[0011] In some embodiments, the sub-object model moves along the motion path corresponding to the sub-object model from a starting position on the motion path corresponding to the sub-object model, and the method further includes:

[0012] Obtain the display range of the target image, and render the target sub-object model whose motion exceeds the display range at the starting position of the motion path of the target sub-object model.

[0013] By re-rendering the target sub-object model that has been moved out of the display range at the starting position of the corresponding motion path of the target sub-object model and maintaining the motion, the motion of the target sub-object model can be made to appear as a cyclical motion.

[0014] In some embodiments, the sub-scene model moves along the motion path corresponding to the sub-scene model from a starting position on the motion path corresponding to the sub-scene model, and the method further includes:

[0015] Obtain the display range of the target image, and render the target sub-scene model whose motion exceeds the display range at the starting position on the motion path of the target sub-scene model.

[0016] By re-rendering the target sub-scene model that has been moved out of the display range at the starting position of the corresponding motion path of the target sub-scene model and maintaining the motion, the motion of the target sub-scene model can be made to appear as a cyclical motion.

[0017] In some embodiments, before rendering the 3D object model and / or the 3D scene model, the method further includes:

[0018] Obtain the orientation of the target object; adjust the orientation of the 3D object model and / or the 3D scene model based on the orientation of the target object.

[0019] The orientation of the 3D object model and / or the 3D scene model can be adjusted according to the orientation of the target object, so that the orientation of the 3D object model and / or the 3D scene model in the rendering result can be different based on the orientation of the target object.

[0020] In some embodiments, before rendering the 3D object model and / or the 3D scene model, the method further includes:

[0021] Obtain the orientation of the target object;

[0022] The angle parameters of the rendering camera are adjusted based on the orientation of the target object; wherein, the orientation of the 3D object model and / or 3D scene model displayed on the target image is different under different angle parameters.

[0023] The orientation of 3D object models and / or 3D scene models in the rendering result can be changed by altering the angle parameters of the rendering camera. The angle parameters of the rendering camera can be changed according to the orientation of the target object, so that the orientation of 3D object models and / or 3D scene models in the rendering result changes with the change of the angle parameters of the rendering camera.

[0024] In some embodiments, when the target image includes multiple target objects, obtaining the 3D object model includes: obtaining multiple 3D object models, each of which corresponds to a target object, and each 3D object model includes a target region.

[0025] Based on this, when there are multiple target objects in the target image, multiple 3D object models can be obtained, so that the rendering result can contain multiple target objects and multiple 3D object models, with the multiple 3D object models corresponding to the multiple target objects and displayed on the rendered result image.

[0026] Furthermore, obtaining the 3D object model includes:

[0027] The target image is identified to obtain the identification result;

[0028] Based on the recognition results, the 3D object model corresponding to each target object is determined from the multiple 3D object models.

[0029] The system can identify target images and determine the 3D object model corresponding to the target object based on the identification results. This allows the system to determine the 3D object model corresponding to each target object, so that the target object is displayed in the rendering result in combination with the corresponding 3D object model.

[0030] Furthermore, in some embodiments, the recognition result includes the attribute information of the target object, and the 3D object model matches the attribute information of the corresponding target object.

[0031] Based on this, a 3D object model matching the attribute information of the target object can be determined, so that the 3D object model in the rendering result can be displayed corresponding to the target object with attribute information matching the 3D object model.

[0032] In some embodiments, attribute information of the target object can also be obtained;

[0033] Based on the attribute information of the target object, determine the attribute information of the 3D scene model;

[0034] The step of rendering the 3D scene model onto the target image based on the depth and color information of the area outside the coverage area in the 3D scene model includes:

[0035] The 3D scene model is rendered based on the depth and color information of the area outside the coverage area in the 3D scene model, as well as the attribute information of the 3D scene model.

[0036] The attribute information of a 3D scene model can be determined based on the attribute information of the target object, so as to render a 3D scene model with different attribute information for target objects with different attribute information in the target image.

[0037] According to a second aspect of the present disclosure, a special effects rendering apparatus is provided, comprising: a first acquisition module for acquiring a target image, the target image including a target object; a second acquisition module for acquiring a 3D object model, the 3D object model including a target region; and a rendering module for rendering the 3D object model onto the target image based on depth information of the 3D object model and color information of a region on the 3D object model other than the target region, wherein the target object is displayed within the target region.

[0038] According to a third aspect of the present disclosure, a special effects rendering device is provided, comprising: a processor; and a memory for storing processor-executable instructions, the processor being configured to implement the method described in any of the above embodiments.

[0039] According to a fourth aspect of the present disclosure, a computer-readable medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps of the method described in any of the above embodiments.

[0040] The technical solution provided by the embodiments of this disclosure involves acquiring a target image including a target object and acquiring a virtual 3D object model on the target image. Since the rendered image includes both the target object and the 3D object model, and the 3D object model cannot obscure the target object, the 3D object model includes a target area for displaying the target object. Then, based on the depth information of the 3D object model and the color information of the areas on the 3D object model other than the target area, the 3D object model is rendered onto the target image. By rendering the 3D object model onto the target image in the above manner, the real target object and the virtual 3D object model are integrated, enhancing the fun and immersion, and giving the user a more immersive experience.

[0041] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this disclosure, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0043] Figure 1 This is a flowchart of a special effects rendering method provided in an exemplary embodiment.

[0044] Figure 2 This is a rendering of the acquired target image provided in an exemplary embodiment.

[0045] Figure 3 This is an exemplary embodiment of the rendering of a partial 3D object model and a partial 3D scene model.

[0046] Figure 4 This is an exemplary embodiment of rendering a 3D object model and a 3D scene model onto a target image.

[0047] Figure 5 This is an exemplary embodiment of a diagram showing a 3D scene model divided into multiple sub-scene models, with the sub-scene models in motion.

[0048] Figure 6This is an exemplary embodiment of the rendering result showing the motion of a target sub-scene model.

[0049] Figure 7 This is an exemplary embodiment of the diagram showing the effect of a target object facing different directions.

[0050] Figure 8 This is an exemplary embodiment showing the effect of adjusting a 3D object model and a 3D scene model according to the orientation of a target object.

[0051] Figure 9 This is an exemplary embodiment of a rendering diagram showing the effect of adjusting the viewpoint of the rendering camera based on the orientation of the target object.

[0052] Figure 10 This is a flowchart of a special effects rendering method provided in an exemplary embodiment.

[0053] Figure 11 This is a schematic diagram of the structure of a special effects rendering device provided in an exemplary embodiment.

[0054] Figure 12 This is a schematic diagram of the structure of a special effects rendering device provided in an exemplary embodiment. Detailed Implementation

[0055] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0056] The terminology used in this disclosure is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. The singular forms “a,” “the,” and “the” as used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0057] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this disclosure, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0058] Currently, when rendering special effects on images, the general approach is to apply effects or rendering materials to the target object within the image. Taking a human face as an example, this can involve rendering beauty effects, adding decorative stickers such as hats and earrings, and enlarging, shrinking, or distorting the face. However, these methods are relatively simplistic and lack variety.

[0059] To address the aforementioned issues, this disclosure provides a special effects rendering method applicable to processing real-time captured images or videos, as well as pre-captured and stored images or videos. For example, in scenarios involving real-time image or video capture, during capture, the target object (such as a user's face or body part) can be displayed in a pre-made virtual 3D model, thereby enhancing the user experience, strengthening the integration of the target object with the 3D model, and giving the user a more immersive feeling.

[0060] Of course, the above application scenarios are for illustrative purposes only and should not be construed as limiting the embodiments of this disclosure.

[0061] To enable those skilled in the art to better understand the technical solutions in the embodiments of this disclosure, the technical solutions in the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this solution, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art should fall within the protection scope of the embodiments of this disclosure.

[0062] See Figure 1 , Figure 1 This is a flowchart of a special effects rendering method provided in this disclosure embodiment, including the following steps:

[0063] S102, Obtain a target image, wherein the target image includes a target object;

[0064] S104, Obtain a 3D object model, wherein the 3D object model includes a target region;

[0065] S106, based on the depth information of the 3D object model and the color information of the area on the 3D object model other than the target area, the 3D object model is rendered onto the target image, wherein the target object is displayed within the target area.

[0066] In S102, the target image can be acquired during real-time shooting via a camera, or it can be acquired from pre-captured videos or images. Since the captured or stored video frames may or may not include the target object, it can be determined whether each captured or stored image includes the target object, and the image containing the target object is identified as the target image. This can be determined by performing target detection on the image, or by using pre-annotated information. Images that do not include the target object are not processed with special effects rendering. The target object can be a body part such as a face, hand, or foot; a living being such as a person, animal, or plant; or a non-living being such as a building. The type of target object can be changed according to the needs of the actual shooting scene; this embodiment does not limit the type of target object.

[0067] In S104, a pre-made virtual 3D object model can be obtained, which includes a target area. In this embodiment, the 3D object model can be an animal model, such as a pig model, a monkey model, etc., or a mirror model, a painting model, etc. Different pre-made 3D object models can be obtained according to the desired effect of the actual scene. The target area can be the area occupied by a part of the model in the 3D object model. For example, the target area included in the 3D object model can be a body part of the 3D object model. For instance, when the 3D object model is a pig model, the target area can be the pig model's face, hands, or belly, etc. The target area included in the 3D object model can also be a specified range of area within the 3D object model. For instance, when the 3D object model is a painting model, the target area can be a rectangular area in the middle of the painting, or the range of an object or person in the painting. Of course, the target area in the 3D object model can be set according to the desired effect of the actual scene. The embodiments disclosed herein do not limit the 3D object model or the target region within the 3D object model.

[0068] In S106, the 3D object model can be rendered onto the target image based on the depth information of the 3D object model and the color information of the areas on the 3D object model other than the target area. In some embodiments, the depth information of the target area can be assigned to the pixels on the target image corresponding to the target area, and the depth and color information of the areas of the 3D object model other than the target area can be assigned to the pixels on the target image corresponding to the areas of the 3D object model other than the target area. In this way, the 3D object model can be rendered onto the target image, and in the rendering result, since only the depth information of the 3D object model corresponding to the target area is assigned to the pixels on the target image, the rendered 3D object model will not obscure the target object in the target image. Therefore, the final effect is to integrate the real-world target object into the virtual 3D object model.

[0069] In some embodiments, when rendering a 3D object model onto a target image, each pixel in the target image corresponds to a point in the 3D object model. For a set P1 of points in the non-target region of the 3D object model, the depth information and color information of each point in P1 are determined, and then the depth information and color information of each point in P1 are assigned to the corresponding pixels in the target image. For a set P2 of points in the target region of the 3D object model, the depth information of each point in P2 is determined, and the depth information of each point in P2 is assigned to the corresponding pixels in the target image (which may be pixels that make up the target object). Based on the above method, the 3D object model is rendered onto the target image. Of course, in actual implementation, the determined depth information and / or color information of each point in P1 can also be transformed, and the transformed depth information and / or color information of each point in P1 can be assigned to the corresponding pixels in the target image; the depth information of the points in P2 can also be transformed as described above. This embodiment of the present disclosure does not limit this.

[0070] In some cases, to further enhance the richness of the special effects rendering results, in addition to acquiring the 3D object model, a pre-made virtual 3D scene model can also be acquired in advance. The 3D object model resides within the 3D scene model. The 3D object model is used to render the area surrounding the target object, and the target object is ultimately displayed at its target area location within the 3D object model. The 3D scene model is used to render the area of ​​the target image outside the area rendered by the 3D object model, i.e., the background area of ​​the target image.

[0071] For example, a way to render a 3D scene model is to render the 3D scene model onto a target image based on the depth and color information of the area outside the covered area in the 3D scene model, where the covered area is the area covered by the target area from the perspective of the rendering camera.

[0072] When rendering 3D scene models and / or 3D object models, there is a virtual rendering camera that can capture a portion of the 3D object model and / or 3D scene model. The viewpoint from which the rendering camera captures that portion of the model is called the rendering camera viewpoint.

[0073] When rendering a 3D scene model, there can be overlap between the 3D scene model and the target area. This overlap can cause the 3D scene model to occlude the target object that was originally intended to be displayed in the target area. Therefore, to prevent the target object from being occluded by the 3D scene model, during the rendering of the 3D scene model, only the depth and color information of the area outside the occluded area can be rendered. Consequently, in the rendered target image, the pixels in the target area are not assigned the color information of the corresponding pixels in the 3D scene model. Therefore, the target area still displays the target object in the rendering result.

[0074] From the perspective of the rendering camera, the target area will cover a portion of the covered area in the 3D scene model. In one embodiment, before rendering the 3D scene model, the covered area can be removed from the 3D scene model, and then the 3D scene model is rendered based on the depth and color information of the area outside the covered area. In another embodiment, before rendering the 3D scene model, the covered area is not removed; instead, the 3D scene model is rendered based on its depth information and the color information of the area outside the covered area. That is, the covered area in the 3D scene model is rendered only with depth information, while the area outside the covered area is rendered with both depth and color information. Through either of these methods, the final target image can be displayed as follows: the target object is displayed in a 3D object model, and the 3D object model is displayed in a 3D scene model. The position of the 3D object model within the 3D scene model can be set according to actual needs, and this embodiment does not impose any limitations.

[0075] In some embodiments, when rendering a 3D scene model onto a target image, each pixel in the target image corresponds to a point in the 3D scene model. For a set Q1 of points in the non-covered area of ​​the 3D scene model, the depth and color information of each point in Q1 are determined, and then the depth and color information of each point in Q1 are assigned to the corresponding pixels in the target image. For a set Q2 of points in the covered area of ​​the 3D scene model, the depth information of each point in Q2 is determined, and the depth information of each point in Q2 is assigned to the corresponding pixels in the target image. Based on the above method, the 3D scene model is rendered onto the target image. Of course, in actual implementation, the determined depth and / or color information of each point in Q1 can also be transformed, and the transformed depth and / or color information of each point in Q1 can be assigned to the corresponding pixels in the target image; the depth information of the points in Q2 can also be transformed as described above. This embodiment of the present disclosure does not limit this aspect.

[0076] In one specific embodiment Figure 2 The target image to be acquired is a girl's face. Figure 3 This involves obtaining a portion of a 3D object model and a portion of a 3D scene model. The 3D object model, as shown in Figure 301, is the model of the small animal with a human face. The target area within the 3D object model, as shown in Figure 302, is the area containing the human face model. The 3D scene model, as shown in Figure 303, includes all other models except the 3D object model. Figure 3 Rendering 3D object models and 3D scene models to Figure 2 The effect diagram in the target image is as follows Figure 4 As shown, the target object ( Figure 2 The girl's face remains displayed in the rendered target image, while other areas of the original target image are covered by the effects of the 3D object model and 3D scene model. It is evident that through the above-mentioned special effects rendering method, the target object can be integrated into the pre-made 3D object model and 3D scene model, giving the user a more immersive experience and enhancing the fun and sense of immersion. In another embodiment, it is also possible to obtain... Figure 3 The 3D object model contained therein is rendered to... Figure 2 The effect in the target image is that the target object is displayed in a pre-made 3D object model. This will not be elaborated further in this disclosure.

[0077] After integrating the target object into the 3D object model and / or 3D scene model, to further enhance the interest of the 3D model, in some embodiments of this disclosure, motion animations of the 3D object model and / or 3D scene model can be pre-set. During 3D modeling, the 3D object model and 3D scene model can be divided into blocks according to actual needs. For example, the 3D object model can be divided into multiple sub-object models, each of which can be a portion of the 3D object model; and / or the 3D scene model can be divided into multiple sub-scene models, each of which can be a portion of the 3D scene model. Motion animations are then set for one or more sub-object models and / or one or more sub-scene models along their respective motion paths. After rendering the model, each sub-object model and / or sub-scene model moves along its corresponding motion path; thus, the effect of integrating the target object into the moving 3D object model and / or 3D scene model can be achieved. Of course, depending on the actual application requirements, you can set only the motion animation of the 3D object model and not the motion animation of the 3D scene model; or you can set only the motion animation of the 3D scene model and not the motion animation of the 3D object model; this disclosure does not limit this.

[0078] The motion path of the target sub-object model includes a starting position, which can be set according to actual needs. When the target sub-object model moves beyond the display range of the target image, it can be re-rendered at the starting position of its motion path, or the motion can be maintained.

[0079] The target sub-object model's motion path can have multiple starting positions. If the target sub-object model moves beyond the display area of ​​the target image, when re-rendering its position, it can randomly select any of these starting positions, or it can select one of them according to pre-set rules. Furthermore, the target sub-object model's motion speed and / or motion path can be set according to actual needs. The target sub-object model's motion speed and / or motion path can also be changed according to pre-set rules; moreover, the starting position on the target sub-object model's motion path can also change as the motion path changes. Based on this, the target sub-object model can perform a certain cyclical motion when displayed on the target image, enhancing its visual appeal.

[0080] The motion path of the target sub-scene model includes a starting position, which can be set according to actual needs. When the target sub-scene model moves beyond the display range of the target image, it can be re-rendered at the starting position of its motion path. Furthermore, the motion of the target sub-scene model can be maintained.

[0081] Multiple starting positions can exist along the motion path of the target sub-scene model. If the target sub-scene model moves beyond the display range of the target image and needs to be re-rendered, any one of the multiple starting positions can be randomly selected, or one of the multiple starting positions can be selected according to pre-set rules. Furthermore, the motion speed and / or motion path of the target sub-scene model can be set according to actual needs. The motion speed and / or motion path of the target sub-scene model can also be changed according to pre-set rules. Moreover, the starting position on the motion path of the target sub-scene model can also change as the motion path changes. Based on this, the target sub-scene model can perform a certain cyclical motion when displayed on the target image to enhance its appeal. This disclosure is merely an example of several implementation methods and combinations thereof, and is not intended to limit the scope of the disclosure.

[0082] During motion, the target sub-object model and / or target sub-scene model may exceed the display range of the target image. This could mean that the entire block of the target sub-object model and / or target sub-scene model exceeds the display range of the target image, or only a portion of the target sub-object model and / or target sub-scene model exceeds the display range of the target image. The specific setting depends on the actual needs, and this disclosure does not limit it. In a specific embodiment, see [reference needed]. Figure 5 ,like Figure 5 As shown in (5-1), the ground scene is pre-divided into 5 sub-scene models, labeled from left to right as block 501, block 502, block 504, and block 505 respectively. This is the initial state after rendering the ground scene onto the target image. Let block 501 in this ground scene be the target sub-scene model, and its current position be its starting position. The movement direction of the target sub-scene model 501 is from left to right, and its movement path is the entire displayed ground scene. The display range of the target image is the range displayed by the 5 sub-scene models in this ground scene. After a period of movement, the ground scene appears as follows... Figure 5As shown in Figure (5-2), the position of the target sub-scene model 501 at this time is as described. After a period of motion, the position of the target sub-scene model 501 is as shown. Figure 5 As shown in (5-3), the target sub-scene model 501 has moved beyond the display range of the ground scene, that is, the target sub-scene model 501 has moved beyond the display range of the target image. At this time, for the target sub-scene model 501 that has moved beyond the display range of the target image, the target sub-scene model 501 is quickly moved back to the initial position on the movement path of the target sub-scene model 501. The process of quick movement is as follows: Figure 5 Middle (5-3) to Figure 5 Middle (5-4) to Figure 5 As shown in (5-1) (the position of the target sub-scene model 501 is as follows) Figure 5 Middle (5-3) Figure 5 Middle (5-4) and Figure 5 As shown in (5-1), the target sub-scene model 501 is finally re-rendered at the starting position on the motion path of the target sub-scene model 501, that is, as shown in (5-1). Figure 5 The location of the target sub-scene model 501 in (5-1) achieves the effect of the entire scene appearing as an animation in a certain cyclical motion. Of course, the above block division method is not limited to the block division method of the sub-scene model. For example, depending on the actual situation and actual needs, any one or more parameters such as the shape, size, number, and movement mode of the blocks can be different from the situation shown in the figure.

[0083] In one embodiment based on rendered 3D object models and 3D scene models, the 3D object model includes multiple sub-object models, the 3D scene model includes multiple sub-scene models, and a certain number of sub-object models and / or sub-scene models move along their respective motion paths. See also Figure 6 , Figure 6 This refers to three frames from a video that renders a 3D object model and a 3D scene model. In this embodiment of the disclosure, the target image can be any target image frame, and this is not limited. For example... Figure 6 As shown in (6-1), there is a sun and a cloud above the 3D scene model. The sun and cloud are set as the target sub-scene model 601. After a period of movement, the target sub-scene model 601 displays the following position: Figure 6 As shown in (6-2). After the target sub-scene model 601 moves for a period of time, its displayed position is as follows. Figure 6 As shown in (6-3), the entire target sub-scene model is about to move out of the display range of the target image; when the entire target sub-scene model 601 moves out of the display range of the target image, the target sub-scene model 601 is quickly moved back. Figure 6The initial position of the target sub-scene model 601 in (6-1) is maintained, and the movement of the target sub-scene model 601 is maintained, so that the displayed scene is an infinite loop animation effect, which is highly interesting. Of course, this disclosure is only for illustrative purposes and is not intended to limit the scope of the work.

[0084] In the acquired target images, the target object may have different orientations in multiple acquired images. The orientation of the target object can be the direction it is facing. For example, if the target object is a face, the orientation of the face is the orientation of the target object; if the target object is a hand, the orientation can be the direction of the palm or the direction of the fingers, etc., and can be preset according to the specific scenario. In a specific embodiment of this disclosure, such as... Figure 2 , Figure 7 Middle (7-1) Figure 7 As shown in Figure (7-2), the girls' faces are facing different directions in these three images. To make the rendering more realistic and interesting, the orientation of the target object can be obtained before rendering the 3D object model and 3D scene model. The orientation of the 3D object model and / or 3D scene model can then be adjusted based on the target object's orientation. Specifically, the orientation of the target object can be determined first, and then the orientation of the 3D object model and / or 3D scene model can be determined accordingly. For example, the orientation of the 3D object model can be adjusted so that the orientation of the target object in the rendered result matches the orientation of the 3D object model. For instance, if the target object is facing left, the 3D object model can be adjusted so that its orientation in the rendered result also faces left.

[0085] The method for obtaining the orientation of the target object can be to extract features from the target image and calculate the orientation of the target object based on an orientation recognition algorithm; or it can be to pre-train a neural network for determining the orientation of the target object and use the neural network to determine the orientation of the target object. This embodiment of the present disclosure does not limit the method.

[0086] In the embodiments disclosed herein, such as Figure 4 , Figure 8 Middle (8-1) Figure 8 As shown in (8-2), these are the renderings of the three images mentioned above. Figure 2 , Figure 7 Middle (7-1) Figure 7The effect diagram of (7-2) is shown. Before rendering the 3D object model and the 3D scene model, the orientation of the 3D object model and the 3D scene model is adjusted according to the orientation of the girl's face, so that the orientation of the girl's face matches the orientation of the 3D object model and the 3D scene model, making the final rendered effect more realistic and interesting. In one embodiment, the transform matrix of the 3D object model and the 3D scene model can be calculated by using the solve PnP algorithm (robust pose estimation algorithm) to detect key points on the face and corresponding points on the standard 3D head, and then the orientation of the 3D object model and the 3D scene model can be adjusted. Of course, according to actual needs, only the orientation of the 3D object model can be adjusted without adjusting the orientation of the 3D scene model; or only the orientation of the 3D scene model can be adjusted without adjusting the orientation of the 3D object model. This embodiment of the disclosure is not limited here. In this embodiment, the target object is a girl's face. In a real-world scenario, the target object could also be a person's hand, in which case the orientation of the target object would be the direction of the fingers. The target object could also be a car, in which case the orientation of the target object would be the direction the front of the car would be, and so on. This disclosure does not impose any limitations on this. The angle parameter of the 3D object model's orientation can be adjusted according to actual needs. It can be adjusted so that the orientation of the 3D object model is the same as the orientation of the target object, or so that the orientation of the 3D object model is always perpendicular to the orientation of the target object, or so that the orientation of the 3D object model is at any angle to the orientation of the target object, and so on. When adjusting the orientation of the 3D scene model, it can be adjusted so that the orientation of the 3D scene model is the same as the orientation of the target object, or so that the orientation of the 3D scene model is always perpendicular to the orientation of the target object, or so that the orientation of the 3D scene model is at any angle to the orientation of the target object, and so on. This disclosure does not impose any limitations on this embodiment.

[0087] In an example of adjusting a 3D object model based on the orientation of the target object, see [link to example]. Figure 9 In diagram (9-1), the arrows indicate the viewpoint of the rendering camera. The leopard model on the right is a 3D object model. Assuming the target object is facing right, the leopard model also faces right. When the target object faces left, the rendered effect is as follows. Figure 9 As shown in (9-2), the leopard model's orientation is also adjusted to face left. In this embodiment, the rendering camera's perspective remains unchanged; this embodiment is merely an example and not intended to be limiting.

[0088] By adjusting the orientation of the 3D object model and / or 3D scene model based on the orientation of the target object, the display of the 3D object model and / or 3D scene model in the rendering result can be diversified, thus increasing the interest.

[0089] Before rendering 3D object models and 3D scene models, in addition to adjusting the orientation of the 3D object models and 3D scene models based on the orientation of the target object, the angle parameters of the rendering camera can also be adjusted based on the orientation of the target object. The angle parameters of the rendering camera are the angle of the rendering camera relative to the 3D object model and / or 3D scene model when rendering the 3D object model and / or 3D scene model. Specifically, the orientation of the target object can be determined first, and the rendering angle of the rendering camera can be determined based on the orientation of the target object. For example, the rendering angle of the rendering camera can be adjusted so that the orientation of the target object in the rendered result is consistent with the orientation of the 3D object model. For example, if the target object is facing left, the rendering angle of the rendering camera can be adjusted so that the orientation of the 3D object model in the rendered result is also to the left.

[0090] In one specific embodiment Figure 2 , Figure 7 Middle (7-1) Figure 7 In step (7-2), the girl's face is facing different directions. Based on the direction of the girl's face, instead of adjusting the orientation of the 3D object model and the 3D scene model, the angle parameters of the rendering camera are adjusted. The final rendered effect is as follows: Figure 4 , Figure 8 Middle (8-1) Figure 8 As shown in (8-2), adjusting the angle parameters of the rendering camera according to the direction of the girl's face can make the rendering effect more realistic and interesting. The angle parameters of the rendering camera can be adjusted according to actual needs. They can be adjusted so that the angle of the rendering camera is the same as or opposite to the direction of the target object; they can also be adjusted so that the angle of the rendering camera is at any angle to the direction of the target object, etc. This embodiment does not limit the scope of the invention.

[0091] In an example of adjusting the rendering camera's angle parameters based on the target object's orientation, see [link to example]. Figure 9 In diagram (9-1), the arrows indicate the viewpoint of the rendering camera. The leopard model on the right is a 3D object model. Assuming the target object is facing right, the rendering camera's viewpoint is also facing right. When the target object is facing left, adjust the rendering camera's angle parameters, such as... Figure 9 As shown in (9-3), the rendering camera's viewpoint is pointing to the left. In this embodiment, the orientation of the 3D object model remains unchanged. This embodiment is merely illustrative and not intended to be limiting.

[0092] By adjusting the angle parameters of the rendering camera based on the orientation of the target object, the display of 3D object models and / or 3D scene models in the rendering results can be diversified, increasing the interest.

[0093] Of course, based on the needs of practical applications, the orientation of the 3D object model and / or the 3D scene model, as well as the angle parameters of the rendering camera, can also be adjusted based on the orientation of the target object. The specific adjustment method is not limited in this embodiment.

[0094] In actual shooting processes or selected images, there may be situations where multiple target objects appear simultaneously in the target image. When a target image includes multiple target objects, the method for rendering 3D object models can also include: acquiring multiple 3D object models, each corresponding one-to-one with a target object, where each 3D object model includes a target region. Alternatively, if the presence of multiple target objects in the target image is detected beforehand, an equal number of 3D object models including target regions can be acquired, and each target object can be displayed in the target region of a separate 3D object model. This enriches the rendering methods when a target image contains multiple target objects, enhancing its appeal.

[0095] Based on this method, multiple 3D object models are obtained, and each target object is displayed in a 3D object model corresponding to it. The final rendered effect is that the target image includes multiple 3D object models, and each 3D object model includes one target object. Depending on the requirements of the actual application scenario, the 3D object models corresponding to different target objects can be the same 3D object model or different 3D object models. For example, if the target image includes three target objects, in one case, three identical pig models (3D object models) are obtained, with each target object corresponding to one pig model, and then special effects rendering is performed. In another case, three different models are obtained, such as one pig model, one puppy model, and one sheep model, with each target object corresponding to one of the 3D object models. The specific correspondence method is not limited here. Furthermore, in yet another case, two of the three obtained 3D object models are pig models and one is a sheep model, with each target object corresponding to one of the 3D object models. The specific correspondence method is not limited here.

[0096] In some embodiments, the acquired target image can be first identified to determine whether it contains multiple target objects; if so, further identification is performed, and based on the identification results, the 3D object model corresponding to each target object is determined from multiple 3D object models, and then matched and displayed one by one. Alternatively, features of the target image can be identified, and the 3D object model corresponding to those features can be matched and displayed with the target image.

[0097] In some specific embodiments, in a selfie scenario involving multiple users, where the target object is a face, if two faces appear simultaneously in the target image, two 3D object models can be obtained. Each of the two faces corresponds one-to-one with a different 3D object model, and both 3D object models include the target area. The final rendered effect is that the target image contains two 3D object models, and the target areas in each of the two 3D object models display the two faces. This method allows multiple target objects to be integrated into multiple 3D object models, enabling multiple target objects in the target image to simultaneously experience the fun and immersive effect of the special effects rendering in this embodiment.

[0098] In some embodiments, the target object is a face, and the 3D object model includes a virtual face model. Face recognition can also be performed on the faces in the target image to obtain the face recognition result. Based on the face recognition result, the corresponding 3D object model is determined. In a specific embodiment, if the target image includes two faces, two different 3D object models are obtained, such as a pig model and a sheep model. Feature information of these two faces can be obtained, such as eye information, nose information, mouth information, etc. Based on the feature information, face recognition is performed to determine the identity of each face, and the corresponding 3D object model is determined based on the identity of each face. For example, if the two faces are face A and face B, face recognition is performed based on the feature information of the faces to determine the identities of face A and face B; and the 3D object model corresponding to face A is determined to be a pig model, and the 3D object model corresponding to face B is determined to be a sheep model. Based on the above method, each different face can be determined to correspond to a different 3D object model, thereby enhancing the fun and realism of special effects rendering. For example, when users A and B are taking photos, the terminal device uses facial recognition to determine that user A's face corresponds to a pig model and user B's face corresponds to a sheep model. Even if user A's face leaves the camera's field of view for a period of time, meaning user A's face is not displayed in the target image for a period of time, when user A's face reappears in the target image, the terminal device can still obtain the pig model corresponding to user A's face through facial recognition and display user A's face in the pig model. Of course, the number of faces in the above embodiments is not limited. The correspondence between faces and 3D object models can be defined according to the actual scenario, for example, a larger face corresponds to a pig model, etc.; this disclosure does not limit this aspect.

[0099] In the above embodiments, the target object can also be, for example, a flower, in which case the attribute information can be the type of flower, the color of the flower, etc.; the target object can also be, for example, a building, in which case the attribute information can be the height of the building, the color of the building, etc. This disclosure does not limit the scope of the embodiments.

[0100] In order to further enhance the fun of special effects rendering, during the actual shooting process of the user or in the image selected by the user, the attribute information of the target object can also be identified, and the 3D object model is matched with the attribute information of the corresponding target object.

[0101] The process can begin by detecting the target object's attribute information. For a human, this includes attributes such as age and gender; for a flower, attributes include color and flower type. Then, a matching 3D object model is determined based on this attribute information for rendering. In some specific implementations, when the target object is a face, user attributes such as gender and age can be calculated based on information from various facial feature points. For example, a matching 3D object model can be obtained based on the user's gender; for instance, a male face might have a short-haired, muscular 3D object model, while a female face might have a long-haired, slender 3D object model. In another example, the user's age attribute can be calculated based on facial feature points, and a matching 3D object model can be obtained; for example, a child's face might have a cute animal 3D object model, while an adult's face might have a fierce wild animal 3D object model, and so on. Of course, besides the examples of facial attribute information listed above, there are many other facial attribute information, such as whether or not glasses are worn. In addition to facial attribute information, other attribute information is also included. For example, when the target object is a hand, hand attributes such as hand size and degree of aging are included; when the target object is a tree, tree attributes could be tree type, tree height, etc. For the attribute examples listed above, a 3D object model matching the aforementioned attributes can be pre-created based on the target object's attribute information. The above are all examples and are intended to illustrate the point; this disclosure is not intended to be limiting.

[0102] In a specific embodiment, when the 3D object model includes a virtual face model, it is possible not only to obtain a 3D object model that matches the attribute information of the target object based on the attribute information of the target object, but also to determine the attributes of the 3D scene model to be rendered based on the attribute information of the target object. Specifically, rendering the 3D scene model further includes: obtaining the attribute information of the target object; determining the attribute information of the 3D scene model based on the attribute information of the target object; and rendering the 3D scene model based on the depth and color information of the area outside the coverage area in the 3D scene and the attribute information of the 3D scene model.

[0103] The process can begin by detecting the attribute information of the target object. Based on this, the attribute information of the 3D scene model that matches the target object's attribute information is determined, thus identifying the 3D scene model to be rendered. In a specific embodiment, for example, if the target object is a face, user attribute information such as gender and age can be calculated based on the information of various feature points on the face. For example, based on the user's age information, the attribute information of the 3D scene model that matches the user's age information is determined. If the target object is a child's face, then the attribute of the 3D scene model is determined to be a child's attribute, and a 3D scene model including child elements, such as cute cartoon characters, is rendered. If the target object is an adult's face, then the attribute of the 3D scene model is determined to be an adult's attribute, and a 3D scene model including adult elements, such as fierce wild animals, is rendered. If the attribute information of the 3D scene model is determined based on the user's gender attribute, then a 3D scene model including female elements can be determined based on the user's female attribute, and a 3D scene model including male elements can be determined based on the user's male attribute. Here, the female attribute could be that the 3D scene model's color is pink, etc., and the male attribute could be that the 3D scene model's color is blue, etc. This embodiment is merely illustrative and not intended to be limiting.

[0104] In the above embodiments, the target object can also be, for example, a flower, in which case the attribute information can be the type of flower, the color of the flower, etc.; the target object can also be, for example, a building, in which case the attribute information can be the height of the building, the color of the building, etc. This disclosure does not limit the scope of the embodiments.

[0105] In some embodiments, the attribute information of the target object may be determined first, and a first 3D scene model with corresponding attribute information may be determined based on the attribute information of the target object. When rendering the first 3D scene model, each point on the target image corresponds to a point on the first 3D scene model. For a set M1 of points in the non-target region of the first 3D scene model, the depth information and color information of each point in M1 are determined, and then the depth information and color information of each point in M1 are assigned to the corresponding pixels on the target image. For a set M2 of points in the covered region of the first 3D scene model, the depth information of each point in M2 is determined, and the depth information of each point in M2 is assigned to the corresponding pixels on the target image. Based on the above method, the first 3D scene model is rendered on the target image.

[0106] In some embodiments, the attribute information of the target object can be determined first, and the attribute information of the 3D scene model can be determined based on the attribute information of the target object. When rendering the 3D scene model, each point on the target image corresponds to a point on the 3D scene model. For a set N1 of points in the non-target region of the 3D scene model, the depth information and color information of each point in N1 are determined, and the color information is changed according to a certain preset rule based on the aforementioned attribute information. The depth information and the changed color information of each point in N1 are assigned to the corresponding pixels on the target image. For a set N2 of points in the covered region of the 3D scene model, the depth information of each point in N2 is determined, and the depth information of each point in N2 is assigned to the corresponding pixels on the target image. Based on the above method, the 3D scene model is rendered on the target image.

[0107] The above embodiments are all illustrative examples of embodiments of this disclosure.

[0108] See Figure 10 , Figure 10 This is a detailed flowchart of one embodiment of the present disclosure.

[0109] According to S1002, a target image is acquired, which includes a target object. The target image can be acquired during real-time shooting by the user through a camera, or it can be acquired from a pre-shot video or image. The target object can be a body part such as a face, hand, or foot, or it can be a small animal, plant, or building, etc. The user can modify the type of target object according to the actual shooting scene. The number of target objects can be one or more. According to S1004, a pre-made virtual 3D object model and a 3D scene model are acquired. The 3D object model includes a target area. The 3D object model is located in the 3D scene model. The acquired 3D object model can be one 3D object model for one target object, or an equal number of 3D object models for multiple target objects. The aforementioned equal number of 3D object models can be equal numbers of identical 3D object models or equal numbers of different 3D object models, which is not limited in this disclosure. According to S1006, based on the depth information of the 3D scene model and the color information of the area on the 3D object model other than the target area, and based on the depth and color information of the area outside the covered area in the 3D scene model, the 3D object model and the 3D scene model are rendered onto the target image. The target object is displayed within the target area, and the covered area is the area covered by the target area from the rendering camera's perspective. Specifically, for rendering the 3D scene model, the covered area in the 3D scene model covered by the target area can be removed before rendering, and then the 3D scene model is rendered based on the depth and color information of the area outside the covered area. Alternatively, the covered area in the 3D scene model can be rendered without removing the covered area before rendering, and instead, the 3D scene model is rendered based on its depth information and the color information of the area outside the covered area. After rendering, the effect displayed on the target image is: the target object is displayed in the target area of ​​the pre-made 3D object model, and the 3D object model is located within the 3D scene model. That is, the target object is integrated into the virtual 3D object model and 3D scene model, so that users feel more immersed and thus improve the user experience.

[0110] Of course, all the above embodiments are illustrative examples, and the algorithms described are also illustrative only, and are not intended to limit the embodiments of this disclosure. One or more algorithms may be used in the embodiments of this disclosure to calculate the attribute information and feature information of the target object; these embodiments will not be elaborated upon here.

[0111] The above description is merely a specific implementation of the embodiments of this disclosure and is not intended to limit the scope of this solution. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this solution, and these improvements and modifications should also be considered within the scope of protection of this solution.

[0112] See Figure 11 This disclosure also provides an apparatus for special effects rendering, the apparatus comprising:

[0113] The first acquisition module 1102 is used to acquire a target image, wherein the target image includes a target object;

[0114] The second acquisition module 1104 is used to acquire a 3D object model, wherein the 3D object model includes a target region;

[0115] The rendering module 1106 is used to render the 3D object model onto the target image based on the depth information of the 3D object model and the color information of the area on the 3D object model other than the target area, wherein the target object is displayed in the target area.

[0116] In some embodiments, the target image acquired by the first acquisition module 1102 can be a target image captured by the user in real time through a camera, or a target image acquired from a pre-captured video or image. Before acquiring the target image, it is first determined whether the image includes a target object. If the target object is included, a target image is acquired. For images that do not include a target object, no special effects rendering is required. The target object can be a body part such as a face, hand, or foot, or a small animal, or a building, etc. The user can modify the type of target object according to the actual shooting scene.

[0117] In some embodiments, the 3D object model acquired by the second acquisition module 1104 is a pre-made virtual 3D object model, which includes a target area. The 3D object model type can be a small animal, such as a pig or a monkey, or it can be a mirror, a painting, etc., and different types of pre-made 3D object models can be acquired according to the desired effect of the actual scene. The target object included in the 3D object model can be a body part of the 3D object model; for example, when the 3D object model is a pig, the target area can be the pig's face, hands, or feet, etc. The target object included in the 3D object model can also be a specified area within the 3D object model; for example, when the 3D object model is a painting, the target area can be a rectangular area in the center of the painting, or it can be the area encompassed by a person or object in the painting. Of course, the target area in the 3D object model can be set according to the desired effect of the actual scene.

[0118] In some embodiments, the second acquisition module 1104 may also acquire a pre-made virtual 3D scene model, wherein the 3D object model is located in the 3D scene model.

[0119] In some embodiments, when the rendering module 1106 renders a 3D object model onto a target image, it may assign depth information of the target region to pixels in the target image corresponding to the target region, and assign depth and color information of regions in the 3D object model other than the target region to pixels in the target image corresponding to those regions. The final effect is to integrate a part of the real world into a virtual 3D object model, thus integrating the target object into the 3D object model.

[0120] In some embodiments, the rendering module 1106 also renders both the 3D object model and the 3D scene model onto the target image. The 3D object model is located within the 3D scene model. Based on the rendering method of the 3D object model in the previous embodiment, the method further includes: from the perspective of the rendering camera, the target region will cover a portion of the covered area in the 3D scene model. In one embodiment, before rendering the 3D scene model, the covered area in the 3D scene model is removed, and then the 3D scene model is rendered based on the depth and color information of the area outside the covered area, so that the final target image displays the effect where the target object is displayed within the 3D object model, and the 3D object model is displayed within the 3D scene model. In another embodiment, before rendering the 3D scene model, the covered area in the 3D scene model is not removed; instead, the 3D scene model is rendered based on the depth information of the 3D scene model and the color information of the area outside the covered area, so that the final target image displays the effect where the target object is displayed within the 3D object model, and the 3D object model is displayed within the 3D scene model.

[0121] In some embodiments, the apparatus further includes an animation setting module for pre-setting motion animations for the 3D object model and the 3D scene model. In a specific embodiment, during 3D modeling, the animation setting module first divides the 3D object model and / or 3D scene model into blocks according to requirements, for example, dividing the 3D object model into multiple sub-object models and the 3D scene model into multiple sub-scene models; after rendering the model, the 3D sub-object models and / or 3D sub-scene models move along their respective corresponding motion paths; thus achieving the effect of integrating the target object into the moving 3D object model and / or 3D scene model. Of course, depending on the actual application requirements, only the motion animation of the 3D object model can be set, without setting the motion animation of the 3D scene model; or only the motion animation of the 3D scene model can be set, without setting the motion animation of the 3D object model; to meet the user's personalized needs. Since during the movement of each block, there may be situations where the target 3D sub-object model and / or 3D sub-scene model moves beyond the display range of the target image. In one specific embodiment, the animation setting module further includes a movement subunit, which is used to quickly move the target 3D sub-object model and / or target 3D sub-scene model back to the initial position for a target 3D sub-object model and / or target 3D sub-scene model whose motion exceeds the display range of the target image, and to maintain the motion of the target 3D sub-object model and / or target 3D sub-scene model.

[0122] In some embodiments, the first acquisition module 1102 can also be used to acquire the orientation of the target object. In this embodiment, the apparatus further includes a first adjustment module for adjusting the orientation of the 3D model and / or the 3D scene model based on the orientation of the target object. In this embodiment, the apparatus further includes a second adjustment module for adjusting the angle parameters of the rendering camera based on the orientation of the target object.

[0123] In some embodiments of this disclosure, the apparatus further includes a corresponding module for acquiring multiple 3D object models when the target image includes multiple target objects, wherein each 3D object model corresponds one-to-one with a target object, and each 3D object model includes a target region.

[0124] In some embodiments of this disclosure, the apparatus further includes a recognition module for recognizing the target image and obtaining a recognition result; and then, based on the recognition result, determining the 3D object model corresponding to each target object from multiple 3D object models.

[0125] In some embodiments, the identification module can obtain the attribute information of the target object and obtain a 3D object model that matches the attribute information of the target object.

[0126] In some embodiments, the attribute information of the 3D scene model can be determined based on the attribute information of the target object obtained by the recognition module, and the 3D scene model can be rendered based on the depth and color information of the area outside the aforementioned coverage area in the 3D scene model, as well as the attribute information of the 3D scene model.

[0127] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0128] Embodiments of the special effects rendering apparatus disclosed herein can be applied to computer devices, such as servers or terminal devices. In one embodiment, the computer device further includes a camera; wherein the camera is used to acquire a target image captured by the user in real time. The apparatus embodiments can be implemented in software, or in hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by its processor reading the corresponding computer program instructions from non-volatile memory into memory and executing them. From a hardware perspective, such as... Figure 12 The diagram shown is a hardware structure diagram of a computer device containing the special effects rendering apparatus according to an embodiment of this disclosure. Except for... Figure 12 In addition to the processor 1210, memory 1230, network interface 1220, and non-volatile memory 1240 shown, the server or electronic device where the device 1231 is located in the embodiment may also include other hardware depending on the actual function of the computer device, which will not be described in detail here.

[0129] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described in any of the foregoing embodiments. Computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0130] The foregoing has described specific embodiments of this disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result.

[0131] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention applied herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not claimed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0132] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

[0133] The above description is merely a preferred embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A special effects rendering method, characterized in that, The method includes: Acquire a target image, wherein the target image includes a target object; A 3D object model is obtained, the 3D object model including a target region; the 3D object model includes multiple sub-object models, at least one of the sub-object models moving along a motion path corresponding to the sub-object model; the sub-object model includes a portion of the 3D object model; and / or the 3D scene model includes multiple sub-scene models, at least one of the sub-scene models moving along a motion path corresponding to the sub-scene model; the sub-scene model includes a portion of the 3D scene model. The depth information of the target region is assigned to the pixels on the target image corresponding to the target region, and the depth information and color information of the regions in the 3D object model other than the target region are assigned to the pixels on the target image corresponding to the regions in the 3D object model other than the target region, wherein the target object is displayed in the target region; The 3D object model is located in a pre-acquired 3D scene model, and the method further includes: The attributes of the 3D scene model are determined based on the attribute information of the target object; Based on the depth and color information of the area outside the coverage area in the 3D scene model and the attribute information of the 3D scene model, the 3D scene model is rendered so that the rendered 3D scene model includes elements corresponding to the attribute information of the target object. The sub-object model moves along the motion path corresponding to the sub-object model from its starting position, and the sub-scene model moves along the motion path corresponding to the sub-scene model from its starting position; the method further includes: Obtain the display range of the target image, render the target sub-object model whose motion exceeds the display range at the starting position on the motion path of the target sub-object model, and render the target sub-scene model whose motion exceeds the display range at the starting position on the motion path of the target sub-scene model; Before rendering the 3D object model and / or 3D scene model, the method further includes: Obtain the orientation of the target object; Adjust the orientation of the 3D object model and / or the 3D scene model based on the orientation of the target object; The angle parameters of the rendering camera are adjusted based on the orientation of the target object; wherein, the orientation of the 3D object model and / or 3D scene model displayed on the target image is different under different angle parameters.

2. The method according to claim 1, characterized in that, When the target image includes multiple target objects, obtaining the 3D object model includes: Obtain multiple 3D object models, each of which corresponds to the target object, and each 3D object model includes a target region.

3. The method according to claim 2, characterized in that, The process of obtaining the 3D object model includes: The target image is identified to obtain the identification result; Based on the recognition results, the 3D object model corresponding to each target object is determined from the multiple 3D object models.

4. The method according to claim 3, characterized in that, The recognition result includes the attribute information of the target object, and the 3D object model matches the attribute information of the corresponding target object.

5. A special effects rendering device, characterized in that, The device includes: The first acquisition module is used to acquire a target image, wherein the target image includes a target object; The second acquisition module is used to acquire a 3D object model, wherein the 3D object model includes a target region; the 3D object model includes multiple sub-object models, and at least one sub-object model of the 3D object model moves along a motion path corresponding to the sub-object model; the sub-object model includes a portion of the 3D object model; and / or the 3D scene model includes multiple sub-scene models, and at least one sub-scene model of the 3D scene model moves along a motion path corresponding to the sub-scene model; the sub-scene model includes a portion of the 3D scene model. The first rendering module is used to assign the depth information of the target region to the pixels on the target image corresponding to the target region, and to assign the depth information and color information of the regions in the 3D object model other than the target region to the pixels on the target image corresponding to the regions in the 3D object model other than the target region, wherein the target object is displayed in the target region; The 3D object model is located in a pre-acquired 3D scene model, and the device further includes: The second rendering module is used to determine the attributes of the 3D scene model based on the attribute information of the target object; and to render the 3D scene model based on the depth and color information of the area outside the coverage area in the 3D scene model and the attribute information of the 3D scene model, so that the rendered 3D scene model includes elements corresponding to the attribute information of the target object. The sub-object model moves along the motion path corresponding to the sub-object model from its starting position, and the sub-scene model moves along the motion path corresponding to the sub-scene model from its starting position; the device further includes: The third rendering module is used to obtain the display range of the target image, render the target sub-object model whose motion exceeds the display range at the starting position of the motion path of the target sub-object model, and render the target sub-scene model whose motion exceeds the display range at the starting position of the motion path of the target sub-scene model. The device further includes: The third acquisition module is used to acquire the orientation of the target object; The first adjustment module is used to adjust the orientation of the 3D object model and / or the 3D scene model based on the orientation of the target object; The second adjustment module is used to adjust the angle parameters of the rendering camera based on the orientation of the target object; wherein, the orientation of the 3D object model and / or 3D scene model displayed on the target image is different under different angle parameters.

6. A special effects rendering device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, When the processor executes the program, it implements the method as described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program performs the steps of the method described in any one of claims 1 to 4.

Images