A method, apparatus and electronic device for obtaining a pose-transformed image

Abstract

The present disclosure relates to a method and device for obtaining a pose transformation image, and an electronic device, and particularly relates to the technical field of images. The method comprises: obtaining an initial model of a target object; determining a tracking model of a local object in a real-time image according to a tracking parameter of the local object in the real-time image and the initial model, the local object being at least part of a feature of the target object; determining a pose transformation model according to the tracking model, color data of the local object in the real-time image, and a target pose parameter; and obtaining a pose transformation image of the local object based on the pose transformation model. The embodiments of the present disclosure are used to realize a variety of image special effects.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of image technology, and in particular, to a method and device for obtaining a pose transformation image and an electronic device. BACKGROUND

[0002] In some special effects, the original picture information of a portrait is obtained based on a camera of a mobile phone, and special effects are produced based on the portrait information obtained by the camera of the mobile phone. Some special effect props are set based on the size and position of the portrait information obtained by the camera of the mobile phone, so that the implementation boundary of the special effects is limited, and more diverse image special effects cannot be achieved. SUMMARY

[0003] To solve the above technical problems or at least partially solve the above technical problems, the present disclosure provides a method and device for obtaining a pose transformation image and an electronic device, which can achieve more diverse image special effects.

[0004] To achieve the above purpose, the technical scheme provided by the embodiments of the present disclosure is as follows:

[0005] In a first aspect, a method for obtaining a pose transformation image is provided, comprising:

[0006] obtaining an initial model of a target object;

[0007] determining a tracking model of a local object in a real-time image according to a tracking parameter of the local object in the real-time image and the initial model, the local object being at least part of a feature of the target object;

[0008] determining a pose transformation model according to the tracking model, color data of the local object in the real-time image, and a target pose parameter;

[0009] obtaining a pose transformation image of the local object based on the pose transformation model.

[0010] As an optional implementation manner of the embodiments of the present disclosure, the determination of the pose transformation model according to the tracking model, the color data of the local object in the real-time image, and the target pose parameter comprises:

[0011] mapping the color data of the local object in the real-time image to the local object in the tracking model to obtain a coloring model;

[0012] determining the pose transformation model according to the coloring model and the target pose parameter.

[0013] As an optional implementation of an embodiment of the present disclosure, the mapping of the color data of the local object of the real-time image to the local object in the tracking model to obtain a colored model comprises:

[0014] Obtaining original vertex coordinates in the local object of the tracking model;

[0015] Calculating first image coordinates according to the original vertex coordinates and an MVP matrix;

[0016] Sampling color data from the local object of the real-time image according to the first image coordinates;

[0017] Mapping the color data to the original vertex coordinates to obtain the colored model.

[0018] As an optional implementation of an embodiment of the present disclosure, the determining of the pose transformation model according to the colored model and the target pose parameter comprises:

[0019] Obtaining original vertex coordinates in the local object of the colored model;

[0020] Determining transformed vertex coordinates in the local object according to the target pose parameter and the original vertex coordinates to obtain the pose transformation model.

[0021] As an optional implementation of an embodiment of the present disclosure, the obtaining of the pose transformation image of the local object based on the pose transformation model comprises:

[0022] Calculating second image coordinates according to the transformed vertex coordinates in the pose transformation model and an MVP matrix;

[0023] Mapping color data of the transformed vertex coordinates in the pose transformation model to the second image coordinates to obtain the pose transformation image of the local object.

[0024] As an optional implementation of an embodiment of the present disclosure, the obtaining of the initial model of the target object comprises:

[0025] Creating an object model of a target object;

[0026] Creating a sleeve model of the target object;

[0027] Combining the object model and the sleeve model to obtain the initial model of the target object.

[0028] As an optional implementation of an embodiment of the present disclosure, before the determining of the pose transformation model according to the tracking model, color data of a local object in a real-time image, and a target pose parameter, the method further comprises:

[0029] obtaining an initial pose parameter and a terminal pose parameter;

[0030] determining at least one pose parameter according to the initial pose parameter and the terminal pose parameter;

[0031] obtaining the target pose parameter from the at least one pose parameter, the target pose parameter being any one of the at least one pose parameter.

[0032] As an optional implementation of an embodiment of the present disclosure, before the determining at least one pose parameter according to the initial pose parameter and the terminal pose parameter, the method further comprises:

[0033] obtaining at least one of a pose change duration and a cycle number, wherein the pose change duration is a duration from the initial pose parameter to the terminal pose parameter, and the cycle number is an execution number from the initial pose parameter to the terminal pose parameter;

[0034] the determining at least one pose parameter according to the initial pose parameter and the terminal pose parameter comprises:

[0035] determining the at least one pose parameter according to at least one of the pose change duration and the cycle number, the initial pose parameter and the terminal parameter.

[0036] In a second aspect, a device for obtaining a pose transformation image is provided, comprising:

[0037] a model obtaining module configured to obtain an initial model of a target object;

[0038] a tracking module configured to determine a tracking model of a local object in a real-time image according to a tracking parameter of the local object in the real-time image and the initial model;

[0039] a transformation module configured to determine a pose transformation model according to the tracking model, color data of the local object in the real-time image, and a target pose parameter;

[0040] an image obtaining module configured to obtain a pose transformation image of the local object based on the pose transformation model.

[0041] As an optional implementation of an embodiment of the present disclosure, the transformation module is specifically configured to:

[0042] map the color data of the local object of the real-time image to the local object in the tracking model to obtain a colored model;

[0043] determine the pose transformation model according to the colored model and the target pose parameter.

[0044] As an optional implementation of an embodiment of the present disclosure, the transformation module is specifically configured to:

[0045] obtain original vertex coordinates in the local object of the tracking model;

[0046] calculate first image coordinates according to the original vertex coordinates and an MVP matrix;

[0047] sample color data from the local object of the real-time image according to the first image coordinates;

[0048] map the color data to the original vertex coordinates to obtain the shading model.

[0049] As an optional implementation of an embodiment of the present disclosure, the transformation module is specifically configured to:

[0050] obtain original vertex coordinates in the local object of the shading model;

[0051] determine transformed vertex coordinates in the local object according to the target pose parameter and the original vertex coordinates to obtain the pose transformation model.

[0052] As an optional implementation of an embodiment of the present disclosure, the image acquisition module is specifically configured to:

[0053] calculate second image coordinates according to the transformed vertex coordinates in the pose transformation model and an MVP matrix;

[0054] map color data of the transformed vertex coordinates in the pose transformation model to the second image coordinates to obtain a pose transformation image of the local object.

[0055] As an optional implementation of an embodiment of the present disclosure, the model acquisition module is specifically configured to:

[0056] create an object model of a target object;

[0057] create a sleeve model of the target object;

[0058] combine the object model and the sleeve model to obtain an initial model of the target object.

[0059] As an optional implementation of an embodiment of the present disclosure, the device further comprises:

[0060] The parameter acquisition module is further configured to:

[0061] obtain initial pose parameters and terminal pose parameters;

[0062] determine at least one pose parameter according to the initial pose parameter and the terminal pose parameter;

[0063] obtain the target pose parameter from the at least one pose parameter, the target pose parameter being any one of the at least one pose parameter.

[0064] As an optional implementation of the embodiment of the present disclosure, the parameter obtaining module is further configured to:

[0065] obtain at least one of a pose change duration and a cycle number, wherein the pose change duration is a duration from the initial pose parameter to the terminal pose parameter, and the cycle number is an execution number from the initial pose parameter to the terminal pose parameter;

[0066] The parameter obtaining module is specifically configured to:

[0067] determine the at least one pose parameter according to at least one of the pose change duration and the cycle number, the initial pose parameter and the terminal parameter.

[0068] In a third aspect, an electronic device is provided, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor, and when the computer program is executed by the processor, the method for obtaining a pose transformation image according to the first aspect or any optional implementation thereof is implemented.

[0069] In a fourth aspect, a computer readable storage medium is provided, which includes a computer program stored thereon, and when the computer program is executed by a processor, the method for obtaining a pose transformation image according to the first aspect or any optional implementation thereof is implemented.

[0070] In a fifth aspect, a computer program product is provided, and the computer program product includes: when the computer program product is executed on a computer, the computer is caused to implement the method for obtaining a pose transformation image according to the first aspect or any optional implementation thereof.

[0071] Compared with the prior art, the technical scheme provided by the embodiments of the present disclosure has the following advantages: an initial model of a target object can be obtained; a tracking model of a local object in a real-time image is determined according to a tracking parameter of the local object in the real-time image and the initial model, the local object being at least part of a feature of the target object; a pose transformation model is determined according to the tracking model, color data of the local object in the real-time image, and a target pose parameter; and a pose transformation image of the local object is obtained based on the pose transformation model. Through the scheme, the tracking of the local object is performed, the tracking model of the local object in the real-time image is determined according to the tracking parameter of the local object in the real-time image and the initial model, the color data of the local object in the real-time image is sampled, the pose transformation of the local object in the real-time image is performed through the target pose parameter and the tracking model, the pose transformation model is determined, and finally the pose transformation image of the local object is obtained based on the pose transformation model. The local object in the pose transformation image obtained in this way is not limited by the pose of the local object in the previous real-time image, and the size, orientation, position, and the like of the local object can be set according to the target pose parameter. Therefore, the boundary of the special effect is not limited by the local object in the real-time image, so that more diverse image special effects can be realized. BRIEF DESCRIPTION OF DRAWINGS

[0072] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present disclosure and, together with the description, further serve to explain the principles of the present disclosure.

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced hereinafter. Obviously, those skilled in the art can obtain other drawings according to these drawings without any creative effort.

[0074] Figure 1 A user uses a mobile phone to take a selfie, and a special effect image displayed in the mobile phone at different distances is shown in the schematic diagram provided by the embodiments of the present disclosure;

[0075] Figure 2 A flowchart of a method for obtaining a pose transformation image provided by the embodiments of the present disclosure is shown in the schematic diagram;

[0076] Figure 3 A schematic diagram of the association between model changes and pose parameter settings provided by the embodiments of the present disclosure is shown in the schematic diagram;

[0077] Figure 4 A schematic diagram of a coloring model provided by the embodiments of the present disclosure is shown in the schematic diagram;

[0078] Figure 5A An exemplary effect schematic diagram after the method for obtaining a pose transformation image provided by the embodiments of the present disclosure is processed is shown in the schematic diagram;

[0079] Figure 5B Another exemplary effect schematic diagram after processing of the method for acquiring pose transformation images provided by the embodiment of the present disclosure is provided.

[0080] Figure 5C Another exemplary effect schematic diagram after processing of the method for acquiring pose transformation images provided by the embodiment of the present disclosure is provided.

[0081] Figure 6 A structure block diagram of a device for acquiring pose transformation images provided by the embodiment of the present disclosure is provided.

[0082] Figure 7 A structure schematic diagram of an electronic device provided by the embodiment of the present disclosure is provided. DETAILED DESCRIPTION

[0083] In order to enable a person skilled in the art to more clearly understand the above-mentioned purposes, features and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0084] In the following description, many specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure can also be implemented in other ways different from those described herein; obviously, the embodiments in the description are only some of the embodiments of the present disclosure, not all the embodiments.

[0085] At present, in some special effects play, the portrait information of the original picture is acquired based on the mobile phone camera, and special effects are made based on the portrait information acquired by the mobile phone camera. Some special effect props are set based on the size and position of the portrait information acquired by the mobile phone camera, so that the implementation boundary of the special effect is limited, and more diverse image special effects cannot be realized.

[0086] As shown in Figure 1 , a special effect image displayed in a mobile phone at different distances when a user takes a selfie with the mobile phone is provided by the embodiment of the present disclosure. Generally, when a user takes a selfie with a mobile phone, the distance between the user's face and the screen is between 20 cm and 40 cm, as shown in Figure 1 (a), when the distance between the user's face and the screen is 20 cm, the image of the user's face 11 in the screen of the mobile phone occupies a large proportion, and only part of the hairstyle special effect prop 12 can be leaked out in the screen, which cannot reflect the original design intention of the hairstyle special effect prop, as shown in Figure 1 (b), when the distance between the user's face and the screen is 40 cm, the hairstyle special effect prop 12 in the screen of the mobile phone can leak out most of the screen, but still cannot be completely displayed, and still does not fully meet the original design intention of the hairstyle special effect prop, and the implementation boundary of the special effect is very limited.

[0087] To solve the above problems, the embodiment of the disclosure provides a method and device for obtaining a pose transformation image, and an electronic device. The pose transformation image obtained is not limited by the pose of the local object in the previous real-time image, and the size, orientation, position, etc. of the local object can be set according to the target pose parameter. Thus, the implementation boundary of the special effect is not limited by the local object in the real-time image, so that more diverse image special effects can be achieved.

[0088] The method for obtaining a pose transformation image provided by the embodiment of the disclosure can be implemented by an electronic device or a device for obtaining a pose transformation image. The device for obtaining a pose transformation image can be a functional module or a functional entity for implementing the method for obtaining a pose transformation image in the electronic device.

[0089] The electronic device can be a server, a computer, a tablet computer, a mobile phone, a notebook computer, a palm computer, a vehicle-mounted terminal, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a personal computer (PC), etc. The embodiment of the disclosure does not make a specific limitation.

[0090] As shown in FIG. 1, it is a flowchart of a method for obtaining a pose transformation image provided by the embodiment of the disclosure. The method includes the following steps: Figure 2

[0091] 201, obtaining an initial model of a target object.

[0092] In the embodiment of the disclosure, the target object can be any object. The target object includes an object, an animal body, a human body, a head, a face, etc.

[0093] In some embodiments, obtaining the initial model of the target object can be to create an object model of the target object. For example, assuming that the target object is a head, the initial model of the target object can be a head model.

[0094] In some embodiments, obtaining the initial model of the target object can include creating an object model of the target object, creating a sleeve model of the target object, and then combining the object model and the sleeve model to obtain the initial model of the target object.

[0095] ​Exemplarily, assuming that the target object is a head, the object model of the target object can be a head model, and the cover model can be a head cover model. The head model and the head cover model are combined to obtain a head model including a head cover, i.e., an initial model of the target object. The head cover model is also referred to as a follow head model, which is a model expected to move together with the user's face in a certain effect.

[0096] In the embodiments of the present disclosure, the initial model of the target object can be a point-level animation (also referred to as a vertex animation / deformation animation) model. When creating the initial model of the target object, a BlendShape or a pose deformation related function in a 3D modeling software can be used.

[0097] Exemplarily, when creating a 3D model using Blend Shapes, interpolation operations can be performed between two adjacent meshes to fuse one shape into another shape. In this way, a plurality of shape models can be made, and animation effects can be achieved through the plurality of shape models.

[0098] Further, if there is a cover model matched with the object model of the target object, the object model of the target object and the cover model can be pre-positioned in the above modeling software to combine a model for subsequent processing. Subsequently, the model can be animated by the same method, and the animation can present various types of changes in the motion path, size, and dimensions of the model.

[0099] 202. Obtain tracking parameters of the local object in the real-time image.

[0100] The local object is at least part of a feature of the target object, and the real-time image can be an image collected in real time by a camera.

[0101] Exemplarily, in a scenario in which the user takes a selfie by using a mobile phone, the real-time image can be an image including a face picture of the user collected in real time when the user takes a selfie by using the mobile phone.

[0102] Exemplarily, the target object is a human body, and the local object can be an arm, a head, or a face. The target object is a head, and the local object is a face, eyes, or the like.

[0103] Exemplarily, assuming that the target object is a head and the local object is a face, after obtaining the head model, the tracking parameters of the face in the real-time image can be obtained, where the tracking parameters of the face include position information, scaling information, or rotation information of the face.

[0104] In some embodiments, the way to obtain the tracking parameters of the face can be: first, import the head model including the point-level animation described above into the special effect editor for parsing and create an entity to carry the rendering of the model. In order to enable the entity to track the face in real time, a face tracking component needs to be added to the entity. The face tracking component can return the position, scaling, rotation and other information of the face, and apply these parameter information to the entity carrying the model to realize the function of tracking the face.

[0105] 203. Determine a tracking model of the local object in the real-time image according to the tracking parameters of the local object in the real-time image and the initial model.

[0106] In some embodiments, according to the head model of the obtained point-level animation and the tracking parameters of the face in the real-time image, a tracking model of the face in the head model based on the tracking parameters can be determined.

[0107] 204. Obtain a target pose parameter.

[0108] The target pose parameter involved in the embodiments of the present disclosure refers to any pose parameter set for the established model, and the pose of the model can be transformed based on the target pose parameter.

[0109] In some embodiments, the way to obtain the target pose parameter can include: first, obtain an initial pose parameter and a terminal pose parameter, then determine at least one pose parameter according to the initial pose parameter and the terminal pose parameter, and obtain the target pose parameter from the at least one pose parameter. The target pose parameter is any one of the at least one pose parameter.

[0110] In some embodiments, the initial pose parameter and the terminal pose parameter can be the same or different.

[0111] In the process of obtaining the target pose parameter, the face deformation component corresponding to the model can be obtained in the external script first, and then the pose parameter of the face deformation component is set to control any value in the range of its pose parameter to play the animation corresponding to the state. If a series of continuous different pose parameters are set in a time period, the transformation effect of the corresponding model is to play a specified animation; if a pose parameter is set all the time, the model will always remain in the animation state of the specified frame.

[0112] For example, Figure 3As shown, a schematic diagram of the association between model changes and pose parameter settings provided by the embodiment of the present disclosure is shown, assuming that the pose parameter indicates translation in the X-axis direction. Among them, the initial pose parameter is 0, indicating that the model is at point A in the X-axis direction; the terminal pose parameter is 1, indicating that the model is at point B in the X-axis direction; the target pose parameter can be a parameter greater than or equal to 0 and less than or equal to 1, indicating the pose parameter of the model in any state between points A and B. For example, when the pose parameter is 0.5, the model is at point C in the X-axis direction.

[0113] In some embodiments, the manner of obtaining the target pose parameter can include: obtaining at least one of a pose change duration and a cycle number, determining at least one pose parameter according to the at least one of the pose change duration and the cycle number, the initial pose parameter and the terminal parameter, and obtaining the target pose parameter from the at least one pose parameter.

[0114] Among them, the pose change duration is the duration from the initial pose parameter to the terminal pose parameter, and the cycle number is the execution number from the initial pose parameter to the terminal pose parameter.

[0115] Further, in addition to setting the pose parameter range, the above-mentioned pose change duration can also be set, so that when processing the model transformation, a specified animation duration can be controlled to play to control the speed of the model transformation. Further, the above-mentioned cycle number can also be set, so that the specified animation can be controlled to play for how many times, so that better animation effects can be achieved.

[0116] 205. Determine the pose transformation model according to the tracking model, the color data of the local object in the real-time image, and the target pose parameter.

[0117] In some embodiments, 205 can be implemented by the following steps 205a and 205b:

[0118] 205a. Map the color data of the local object in the real-time image to the local object in the tracking model to obtain a colored model.

[0119] In the process of obtaining the colored model, the vertex coordinates of the tracking model need to be multiplied by the MVP matrix to obtain the corresponding vertex coordinates on the screen for transmission to the subsequent fragment shader. After that, the fragment shader will normalize the vertex coordinates transmitted from the vertex shader to obtain the correct image coordinates, which correspond to the screen position of the local object in the real-time image. In this way, the color data of the local object in the screen can be mapped to the tracking model.

[0120] Exemplarily, the color data of the face image acquired in the real-time image can be mapped to the face region in the tracking model (as a head model) to obtain a colored model. The color data refers to RGB data, i.e., data of three color channels of red (R), green (G), and blue (B).

[0121] In some embodiments, mapping the color data of the local object in the real-time image to the local object in the tracking model to obtain the colored model can be achieved through the following steps (1) to (4):

[0122] (1) Obtain the original vertex coordinates in the local object of the tracking model.

[0123] In the embodiments of the present disclosure, the original vertex coordinates refer to the vertex coordinates of the triangular patches on the tracking model.

[0124] (2) Calculate the first image coordinates according to the original vertex coordinates and the MVP matrix.

[0125] In the embodiments of the present disclosure, the original vertex coordinates can be multiplied by the MVP matrix to calculate the first image coordinates.

[0126] (3) Sample the color data from the local object in the real-time image according to the first image coordinates.

[0127] (4) Map the color data to the original vertex coordinates to obtain the colored model.

[0128] Exemplarily, as shown in Figure 4 FIG. 1 is a schematic diagram of obtaining a colored model provided by an embodiment of the present disclosure. According to the calculated image coordinates, a sampling point 411 can be determined from the face of the real-time image 41, the RGB data of the sampling point 411 can be acquired, and the sampled RGB data can be mapped to an original vertex coordinate 421 on the tracking model 42. In this way, the RGB of all parts of the face of the real-time image 41 can be sampled and mapped to the face on the tracking model 42, so that the colored model can be obtained.

[0129] 205b, determining a pose transformation model according to the colored model and target pose parameters.

[0130] In some embodiments, the manner of determining the pose transformation model according to the colored model and the target pose parameters can be: first obtaining the original vertex coordinates in the local object of the colored model, and then determining the transformation vertex coordinates in the local object according to the target pose parameters and the original vertex coordinates to obtain the pose transformation model.

[0131] The target pose parameters can be set for the original vertex coordinates in the colored model. By transforming the original vertex coordinates in the colored model, the transformed pose transformation model can be obtained.

[0132] The target pose parameter can include a transformation parameter for each original vertex coordinate in the coloring model, and each original vertex coordinate in the coloring model can be transformed by the transformation parameter of each original vertex coordinate to obtain a transformed pose transformation model.

[0133] 206. Obtain a pose transformation image of the local object based on the pose transformation model.

[0134] In some embodiments, the manner of obtaining the pose transformation image of the local object based on the pose transformation model can be: first calculating a second image coordinate by the transformation vertex coordinate in the pose transformation model and the MVP matrix, and then mapping the color data of the transformation vertex coordinate in the pose transformation model to the second image coordinate to obtain the pose transformation image of the local object.

[0135] In the embodiments of the present disclosure, an initial model of a target object can be obtained; a tracking model of a local object in a real-time image can be determined according to a tracking parameter of the local object in the real-time image and the initial model, the local object being at least part of a feature of the target object; a pose transformation model can be determined according to the tracking model, color data of the local object in the real-time image, and a target pose parameter; and a pose transformation image of the local object can be obtained based on the pose transformation model. Through the scheme, the tracking of the local object can be performed, the tracking model of the local object in the real-time image can be determined according to the tracking parameter of the local object in the real-time image and the initial model, the color data of the local object in the real-time image can be sampled, the local object in the real-time image can be pose-transformed by the target pose parameter and the tracking model to determine the pose transformation model, and finally the pose transformation image of the local object can be obtained based on the pose transformation model. The local object in the pose transformation image obtained in this way is not limited by the pose of the local object in the previous real-time image, and the size, orientation, position, etc. of the local object can be set according to the target pose parameter. Therefore, the boundary of the special effect is not limited by the local object in the real-time image, and thus more diverse image special effects can be achieved.

[0136] It should be noted that the target pose parameter can be any one of the set pose parameter range, and in the case of determining multiple different pose parameters, the 3D model established and the real-time image obtained can be processed. In this way, the animation of the local object effect transformed according to different pose parameters can be obtained, the boundary of the special effect is not limited by the local object in the real-time image, and thus more diverse image special effects can be achieved.

[0137] For example, Figure 5AThe image shown is an exemplary effect diagram of a pose transformation image processed according to the method for obtaining pose transformation images provided in this disclosure. First, target pose parameters need to be obtained. These target pose parameters are used for scaling transformation. After processing the image using the method provided in this disclosure in conjunction with a real-time image, and adding hairstyle effects props, the result is as shown below. Figure 5A The processed image shown in the image shows that... Figure 5A The image uses a real-time image of the face, but the face size is scaled down. By combining the target pose parameters with the hairstyle effect prop, the face size can be scaled to a suitable size so that the hairstyle effect prop can be fully displayed after it is added. This way, it is not limited by the size of the face in the real-time image and achieves the image effect that meets the original design intention of the hairstyle effect prop.

[0138] For example, such as Figure 5B The image shown is an exemplary schematic diagram illustrating another possible effect after processing by the method for obtaining a pose transformation image according to an embodiment of this disclosure. First, multiple pose parameters (exemplarily three pose parameters) can be obtained. These three pose parameters can be used for rotation and scaling transformations. Following the method provided in this disclosure, using these three pose parameters and processing them in conjunction with a real-time image, and then adding a hairstyle effect prop, the following effect can be obtained: Figure 5B The three processed images shown in the figure demonstrate that... Figure 5B The three images used real-time images of the face, but the face size was reduced by scaling. The three images were also rotated to different degrees based on different pose parameters. When these three images are played continuously, the processed animation can be presented. The animation can show the hairstyle effects and props rotating with the face, without being limited by the size, position, and orientation of the face in the real-time image, thus achieving more diverse image effects.

[0139] For example, such as Figure 5C The image shown is an exemplary effect diagram of a processed image obtained according to the method for obtaining pose transformation images provided in this disclosure. First, target pose parameters need to be obtained. These parameters are used for scaling transformations and specifying the corresponding number of faces. After processing the image using the method provided in this disclosure in conjunction with a real-time image, and adding hairstyle effects, the result is as follows: Figure 5C The processed image shown in the image shows that... Figure 5C The image uses real-time facial images, but the size of the faces is reduced by scaling, and three corresponding faces are generated. This allows for more diverse image effects without being limited by the size and number of faces in the real-time image.

[0140] It should be noted that the above three schematic diagrams are only exemplary descriptions, after the image is processed by the method of the embodiment of the present disclosure, the local object in the original image can be transformed based on any attitude parameter to obtain more local objects with different transformation effects, and various special effect props can be added based on the transformed local objects. Since the transformed local object is no longer limited by the size, position, orientation, etc. of the local object in the real-time image, more diverse image special effects can be achieved.

[0141] As shown in Figure 6 The embodiment of the present disclosure provides a structural block diagram of a device for obtaining an attitude transformation image, which comprises:

[0142] A model obtaining module 601 is configured to obtain an initial model of a target object.

[0143] A tracking module 602 is configured to determine a tracking model of a local object in a real-time image according to a tracking parameter of the local object in the real-time image and the initial model.

[0144] A transformation module 603 is configured to determine an attitude transformation model according to the tracking model, color data of the local object in the real-time image, and a target attitude parameter.

[0145] An image obtaining module 604 is configured to obtain an attitude transformation image of the local object based on the attitude transformation model.

[0146] As an optional implementation of the embodiment of the present disclosure, the transformation module 603 is specifically configured to:

[0147] Map the color data of the local object of the real-time image to the local object in the tracking model to obtain a colored model.

[0148] Determine the attitude transformation model according to the colored model and the target attitude parameter.

[0149] As an optional implementation of the embodiment of the present disclosure, the transformation module 603 is specifically configured to:

[0150] Obtain original vertex coordinates in the local object of the tracking model.

[0151] Calculate first image coordinates according to the original vertex coordinates and an MVP matrix.

[0152] Sample color data from the local object of the real-time image according to the first image coordinates.

[0153] Map the color data to the original vertex coordinates to obtain the colored model.

[0154] As an optional implementation of the embodiment of the present disclosure, the transformation module 603 is specifically configured to:

[0155] obtain original vertex coordinates in the local object of the coloring model;

[0156] determine transformed vertex coordinates in the local object according to the target pose parameter and the original vertex coordinates, to obtain the pose transformation model.

[0157] As an optional implementation of the embodiment of the present disclosure, the image acquisition module 604 is specifically configured to:

[0158] calculate second image coordinates by taking the transformed vertex coordinates in the pose transformation model and an MVP matrix;

[0159] map color data of the transformed vertex coordinates in the pose transformation model to the second image coordinates, to obtain a pose transformation image of the local object.

[0160] As an optional implementation of the embodiment of the present disclosure, the model acquisition module 601 is specifically configured to:

[0161] create an object model of a target object;

[0162] create a sleeve model of the target object;

[0163] combine the object model and the sleeve model, to obtain an initial model of the target object.

[0164] As an optional implementation of the embodiment of the present disclosure, the device further comprises:

[0165] The parameter acquisition module 605 is further configured to:

[0166] obtain an initial pose parameter and a terminal pose parameter;

[0167] determine at least one pose parameter according to the initial pose parameter and the terminal pose parameter;

[0168] obtain the target pose parameter from the at least one pose parameter, the target pose parameter being any one of the at least one pose parameter.

[0169] As an optional implementation of the embodiment of the present disclosure, the parameter acquisition module 605 is further configured to:

[0170] acquire at least one of a posture change duration and a cycle number, wherein the posture change duration is a duration from the initial posture parameter to the terminal posture parameter, and the cycle number is an execution number from the initial posture parameter to the terminal posture parameter.

[0171] The parameter acquisition module 605 is specifically configured to:

[0172] determine the at least one posture parameter according to at least one of the posture change duration and the cycle number, the initial posture parameter, and the terminal parameter.

[0173] As shown in Figure 7 The electronic device includes a processor 701, a memory 702, and a computer program stored in the memory 702 and executable on the processor 701, and the computer program is executed by the processor 701 to implement each process of the method of acquiring a posture transformation image in the method embodiment. The same technical effects can be achieved, and to avoid repetition, details are not repeated here.

[0174] The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement each process of the method of acquiring a posture transformation image in the method embodiment, and the same technical effects can be achieved. To avoid repetition, details are not repeated here.

[0175] The computer readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0176] The computer program product stores a computer program, and the computer program is executed by the processor to implement each process of the method of acquiring a posture transformation image in the method embodiment, and the same technical effects can be achieved. To avoid repetition, details are not repeated here.

[0177] Those skilled in the art should understand that the embodiments of the present disclosure can be provided as a method, a system, or a computer program product. Therefore, the present disclosure can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure can take the form of a computer program product implemented on one or more computer usable storage media having computer usable program code contained therein.

[0178] In this disclosure, the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, or the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0179] In this disclosure, the memory can include non-persistent memory in computer-readable media including, for example, random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), and / or non-volatile memory, e.g., read only memory (ROM), such as flash memory. The memory is an example of computer-readable media.

[0180] In this disclosure, computer-readable media include non-transitory computer- readable media and transitory computer-readable media. Transitory computer-readable media, such as modulated data signals and carrier waves, include memory of the media, such as those of the processor, and volatile memory, such as those of the RAM. Non-transitory computer-readable media, on the other hand, include, for example, media designed to store information permanently, such as those of the ROM, and / or removable and non-removable storage devices, such as those of the flash memory, and / or the like. Examples of computer-readable media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic disks storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information that can be accessed by a computing device. According to the definition herein, computer-readable media do not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0181] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes the element.

[0182] The above are merely specific embodiments of this disclosure, enabling those skilled in the art to understand or implement this disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to these embodiments, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for acquiring pose transformation images, characterized in that, include: Obtain the initial model of the target object; Based on the tracking parameters of local objects in the real-time image and the initial model, a tracking model for the local objects in the real-time image is determined, wherein the local objects are at least some features of the target object; Based on the tracking model, the color data of local objects in the real-time image, and the target pose parameters, the pose transformation model is determined; Based on the posture transformation model, the posture transformation image of the local object is obtained.

2. The method according to claim 1, characterized in that, The step of determining the pose transformation model based on the tracking model, the color data of local objects in the real-time image, and the target pose parameters includes: The color data of the local objects in the real-time image are mapped to the local objects in the tracking model to obtain the color model; The pose transformation model is determined based on the coloring model and the target pose parameters.

3. The method according to claim 2, characterized in that, The step of mapping the color data of the local objects in the real-time image to the local objects in the tracking model to obtain a colorization model includes: Obtain the original vertex coordinates in the local object of the tracking model; Calculate the first image coordinates based on the original vertex coordinates and the MVP matrix; Color data is sampled from the local object of the real-time image based on the first image coordinates; The color data is mapped to the original vertex coordinates to obtain the shading model.

4. The method according to claim 2, characterized in that, Determining the pose transformation model based on the shading model and the target pose parameters includes: Obtain the original vertex coordinates in the local object of the shaded model; Based on the target pose parameters and the original vertex coordinates, the coordinates of the transformed vertices in the local object are determined to obtain the pose transformation model.

5. The method according to claim 4, characterized in that, The step of obtaining the pose transformation image of the local object based on the pose transformation model includes: The second image coordinates are calculated by combining the coordinates of the transformed vertex in the pose transformation model with the MVP matrix; The color data of the transformation vertex coordinates in the pose transformation model are mapped to the second image coordinates to obtain the pose transformation image of the local object.

6. The method according to claim 1, characterized in that, The process of obtaining the initial model of the target object includes: Create an object model for the target object; Create a nested model of the target object; The object model is combined with the nested model to obtain the initial model of the target object.

7. The method according to any one of claims 1 to 6, characterized in that, Before determining the pose transformation model based on the tracking model, the color data of local objects in the real-time image, and the target pose parameters, the method further includes: Obtain the initial attitude parameters and the final attitude parameters; Determine at least one attitude parameter based on the initial attitude parameters and the final attitude parameters; The target attitude parameter is obtained from the at least one attitude parameter, wherein the target attitude parameter is any one of the at least one attitude parameters.

8. The method according to claim 7, characterized in that, Before determining at least one attitude parameter based on the initial attitude parameters and the final attitude parameters, the method further includes: Obtain at least one of attitude change duration and loop count, wherein the attitude change duration is the duration from the initial attitude parameter to the final attitude parameter, and the loop count is the number of times the attitude change from the initial attitude parameter to the final attitude parameter is executed; The step of determining at least one attitude parameter based on the initial attitude parameters and the final attitude parameters includes: The at least one attitude parameter is determined based on at least one of the attitude change duration and the number of cycles, the initial attitude parameter, and the final attitude parameter.

9. An apparatus for acquiring pose transformation images, characterized in that, include: The model acquisition module is used to acquire the initial model of the target object; The tracking module is used to determine the tracking model of the local object in the real-time image based on the tracking parameters of the local object in the real-time image and the initial model, wherein the local object is at least a part of the features of the target object; The transformation module is used to determine the attitude transformation model based on the tracking model, the color data of local objects in the real-time image, and the target attitude parameters. The image acquisition module is used to acquire the pose transformation image of the local object based on the pose transformation model.

10. An electronic device, characterized in that, include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the method for acquiring a pose transformation image as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, include: A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the method for acquiring a pose transformation image as described in any one of claims 1 to 8.

Images