Photographing method and apparatus therefor

Abstract

The application discloses a photographing method and device, and belongs to the technical field of camera shooting. The method comprises the following steps: in the case that an original image is collected, determining a first image of a target body part in the original image and first coordinate information of the first image; reconstructing first 3D body data according to the first image; performing spatial posture transformation processing on the first 3D body data according to target clone special effect parameters to obtain second 3D body data in different postures; rendering the second 3D body data to obtain a second image and mask data; and fusing the second image and the original image into a target image according to the target clone special effect parameters, the first coordinate information and the mask data.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of camera shooting, and particularly relates to a shooting method and a device thereof. BACKGROUND

[0002] The "shadow double" special effect refers to a shooting effect in which the same person appears in multiple positions in a photo. At present, the "shadow double" special effect is favored by more and more photography enthusiasts.

[0003] The existing "shadow double" special effect can be realized in two ways. One way requires the photographer to use a panoramic shooting mode to cooperate with different walking positions of the person, which requires high shooting skills of the photographer and is not suitable for ordinary people to shoot. The other way is to use a portrait cutout fusion algorithm to fuse multiple portraits to one image, which not only requires shooting multiple images, but also requires the photographer to have professional cutout and puzzle skills, and the algorithm is complex, the processing time is long, and the portrait fusion is prone to abnormal situations. SUMMARY

[0004] The embodiment of the application aims to provide a shooting method, which can solve the problem that the existing way of shooting an image with a double effect is complicated.

[0005] In a first aspect, the embodiment of the application provides a setting method, which comprises the following steps:

[0006] In the case of collecting an original image, determining a first image of a target body part in the original image and first coordinate information of the first image;

[0007] Reconstructing first 3D body data according to the first image;

[0008] Performing spatial pose transformation processing on the first 3D body data according to target double effect parameters to obtain second 3D body data in different poses;

[0009] Rendering the second 3D body data to obtain a second image and mask data;

[0010] Fusing the second image and the original image into a target image according to the target double effect parameters, the first coordinate information and the mask data.

[0011] In a second aspect, the embodiment of the application provides a shooting device, which comprises:

[0012] An acquisition module, configured to, in the case of collecting an original image, determine a first image of a target body part in the original image and first coordinate information of the first image;

[0013] reconstructing a first 3D body data according to the first image;

[0014] a spatial transformation module configured to perform spatial pose transformation on the first 3D body data according to the target avatar special effect parameter to obtain second 3D body data in different poses;

[0015] a rendering module configured to render the second 3D body data to obtain a second image and mask data;

[0016] a fusion module configured to fuse the second image and the original image into a target image according to the target avatar special effect parameter, the first coordinate information and the mask data.

[0017] In a third aspect, an electronic device is provided, which includes a processor and a memory. The memory stores programs or instructions executable on the processor. When the programs or instructions are executed by the processor, the steps of the method according to the first aspect are implemented.

[0018] In a fourth aspect, a readable storage medium is provided, which stores programs or instructions. When the programs or instructions are executed by a processor, the steps of the method according to the first aspect are implemented.

[0019] In a fifth aspect, a chip is provided, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is configured to execute programs or instructions to implement the method according to the first aspect.

[0020] In a sixth aspect, a computer program product is provided, which is stored in a storage medium. The program product is executed by at least one processor to implement the method according to the first aspect.

[0021] In the embodiments of the present application, when an original image is collected, a first image of a target body part in the original image and first coordinate information of the first image are determined, a first 3D body data is reconstructed according to the first image, spatial pose transformation is performed on the first 3D body data according to an avatar special effect parameter to obtain second 3D body data in different poses, the second 3D body data is rendered to obtain a second image and mask data, and the second image and the original image are fused into a target image according to the avatar special effect parameter, the first coordinate information and the mask data. The user only needs to control the camera to take pictures, and the target image with different poses can be automatically generated based on the collected original image, which saves the steps of walking and taking pictures multiple times, and thus effectively solves the problem that the existing way of taking an image with an avatar special effect is complicated. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a step flow chart of a photographing method provided by an embodiment of the present application.

[0023] Figure 2 is a process schematic diagram of performing a body doubling special effect photographing on a single face in an embodiment of the present application.

[0024] Figure 3 is a structural schematic diagram of a photographing device provided by an embodiment of the present application.

[0025] Figure 4 is a structural schematic diagram of an electronic device provided by an embodiment of the present application.

[0026] Figure 5 is a hardware structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0027] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0028] The terms "first", "second", and the like in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are usually a category, and are not limited to the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the front and rear associated objects are in an "or" relationship.

[0029] The setting method provided by the embodiments of the present application will be described in detail below in combination with the drawings, specific embodiments and application scenarios.

[0030] Please refer to Figure 1 , which shows a step flow chart of the setting method provided by an embodiment of the present application, wherein the method can include steps 101-105.

[0031] In actual application, the method can be applied to an electronic device with a camera function, which can be a terminal device such as a mobile phone, a tablet computer, a camera, a smart watch, etc. configured with one or more cameras.

[0032] Step 101, in the case of collecting an original image, determining a first image of a target body part in the original image and first coordinate information of the first image.

[0033] In step 101, the original image can be a camera shot image or a preview image; the target body part is a body part that needs to realize a clone effect, which can be a face, a hand, or a whole body, and the target body part can be one or more; the first coordinate information is the position information of the image body image in the original image.

[0034] In step 101, that is, in the case of detecting a shot image or a preview image, the first image of the target body part is automatically obtained, and the position information of the first image of the target body part in the shot image or the preview image is determined.

[0035] Step 102, reconstructing a first 3D body data according to the first image.

[0036] In step 102, a 3D single-frame reconstruction algorithm is used to perform real-time 3D reconstruction on the first image of the target body part obtained from the original image, to obtain 3D data corresponding to the target body part, that is, the first 3D body data. Optionally, the data includes point cloud data, texture information, and a pose state matrix.

[0037] Step 103, performing spatial pose transformation processing on the first 3D body data according to a target clone effect parameter to obtain second 3D body data in different poses.

[0038] In step 103, the target clone effect parameter is a parameter representing the pose and position of each clone, which can be a fixed value; the target clone effect parameter can also be set according to a first input of a user when the first input is received, and the first input is an input of the user setting the clone effect parameter; in this step, the first 3D body data obtained in step 102 is processed according to the target clone effect parameter to obtain a plurality of second 3D body data, which can present all action poses of each clone in the clone effect according to the same texture. That is, for each action pose of a clone in the clone effect, there is a second 3D body data in the plurality of second 3D body data that can present the action pose of the clone.

[0039] Step 104, rendering the second 3D body data to obtain a second image and mask data.

[0040] In step 104, the second 3D body data of different postures obtained by transformation in step 103 are respectively rendered in real time to obtain the second images corresponding to each avatar in the avatar special effect and the mask data corresponding to each second image.

[0041] In step 105, the second images and the original images are fused into target images according to the target avatar special effect parameters, the first coordinate information and the mask data.

[0042] In step 105, the specific position of the first image of the target body part in the original image is anchored according to the avatar special effect parameters and the first coordinate information of the first image, then each second image is superimposed on the corresponding position of the original image, and then the fusion is performed according to the mask data and the fusion edge is smoothed. In this way, the second images of the target body part in multiple different postures are fused into the original image, and the "shadow avatar" special effect of the target body part is realized.

[0043] Alternatively, each second image is superimposed on a different position of the original image according to the avatar special effect parameters and the first coordinate information of the first image, and then the fusion is performed according to the mask data, so as to ensure that each second image and the first image do not overlap with each other, and the avatar effect can be presented more completely.

[0044] In the photographing method, the user only needs to control the camera to photograph, and the target images with multiple different postures can be automatically generated based on the original images collected, thereby saving the steps of walking, multiple photographing, and effectively solving the problem that the existing photographing method of the image with the avatar special effect is complicated.

[0045] Alternatively, in an embodiment, the user can pre-set whether to start the avatar special effect photographing mode, and in the case of starting the avatar special effect photographing mode, the target body part that needs to present the avatar effect is selected, for example, the face is selected as the target body part, and the target body part can be one or more. When the camera is started to photograph, if it is detected that the avatar special effect photographing mode has been started and the target body part has been selected, the photographing method provided in the present application is executed.

[0046] Alternatively, in a specific embodiment, in the case of setting one target body part, if the first image of the target body part is obtained in the original image, the subsequent steps of realizing the avatar special effect are executed, otherwise the original image is directly fed back to the display end. In this specific embodiment, the target images with multiple different postures are automatically generated only in the case that the first image of the target body part exists in the original image collected, and the avatar special effect of the first image of the single target body part can be realized.

[0047] Optionally, in an embodiment, the photographing method provided by the present application, the target body part is a human face; and the step of obtaining the first image of the target body part in the original image and the first coordinate information of the first image comprises steps 111-113.

[0048] Step 111: performing human face image detection on the original image.

[0049] In step 111, a human face detection technique is used to identify whether a human face image exists in the original image.

[0050] Step 112: in the case where a human face image is detected, determining second coordinate information of a rectangular region in which the human face image is located.

[0051] In step 112, in the case where a human face image is detected in the original image, a rectangular frame is used to frame an image region in the original image in which the human face image is located; when the rectangular frame can frame the human face image, coordinate information of the rectangular frame is determined as the second coordinate information.

[0052] Optionally, in an embodiment, in the case where each side of the rectangular region is parallel to the horizontal and vertical axes of the pixel coordinate system of the original image, the second coordinate information comprises coordinate information of two opposite vertices of the rectangular region. In this embodiment, because the two opposite vertices can be used to determine the rectangular region when the direction of each side of the rectangular region is determined, in the case where each side of the rectangular region is parallel to the horizontal and vertical axes of the pixel coordinate system of the original image, only the coordinate information of the top-left vertex and the bottom-right vertex or the coordinate information of the top-right vertex and the bottom-left vertex of the rectangular region need to be obtained, so that the range of the rectangular region in which the human face image is located can be determined.

[0053] Step 113: according to the second coordinate information, cutting out the human face image from the original image, and determining the second coordinate information as the first coordinate information.

[0054] In step 113, because the second coordinate information defines the range of the human face image in the original image, according to the second coordinate information, the human face image can be cut out from the original image, and the second coordinate information can be determined as the first coordinate information, i.e., the rectangular region is directly determined as the region in which the human face image is located.

[0055] For example, the top-left vertex and the bottom-right vertex of the rectangular region are coordinate points P1 (X1, Y1) and P2 (X2, Y2) in the pixel coordinate system of the original image. According to the vertex coordinate information P1 and P2 of the rectangular region, an image with an X coordinate range between X2 and X1 and a Y coordinate range between Y2 and Y1 can be cut out from the original image as the face image, which is output to the 3D reconstruction algorithm for real-time 3D face reconstruction. At the same time, the P1 and P2 coordinate position information is saved for later face image fusion.

[0056] Optionally, in an embodiment, the first 3D body data includes first point cloud data, texture information, and a first pose matrix in the photographing method provided by the embodiment of the application.

[0057] In step 301, a second pose matrix is determined according to the target avatar effect parameter and the first pose matrix.

[0058] In 301, the avatar effect parameter determines the poses of each avatar corresponding to the target body in the avatar effect, and the poses corresponding to each avatar can be represented by a second pose matrix, which is transformed from the original first pose matrix of the first image. Therefore, a plurality of second pose matrices can be determined according to the avatar effect parameter and the first pose matrix, so that each pose matrix corresponds to the pose of the corresponding avatar in the avatar effect.

[0059] Optionally, in an embodiment, the avatar effect parameter includes a rotation matrix offset, the first pose matrix includes a first rotation matrix, and the second pose matrix includes a second rotation matrix. In step 301, the second rotation matrix is determined according to the first rotation matrix and the rotation matrix offset.

[0060] In this embodiment of the application, the second rotation matrix is determined by the sum of the first rotation matrix and the rotation matrix offset that meets the requirements of the avatar effect. Then, the second pose matrix is directly determined according to the second rotation matrix. At this time, the translation matrix of the second pose matrix is a zero matrix. Subsequent spatial transformation processing of the first point cloud data based on the second pose matrix can obtain second point cloud data in the 3D space, which is in different orientations but has the same position.

[0061] Optionally, in another embodiment, the avatar effect parameter includes a rotation matrix offset and a translation matrix offset, the first pose matrix includes a first rotation matrix and a first translation matrix, and the second pose matrix includes a second rotation matrix and a second translation matrix, and the step 301 includes: determining the second rotation matrix according to the first rotation matrix and the rotation matrix offset; and determining the second translation matrix according to the first translation matrix and the translation matrix offset.

[0062] In this embodiment, the second rotation matrix is the sum of the first rotation matrix and the rotation matrix offset satisfying the avatar effect requirement, the second translation matrix is the sum of the first translation matrix and the translation matrix offset satisfying the avatar effect requirement, and the second pose matrix is the combination of the second rotation matrix and the second translation matrix. Subsequent spatial transformation processing of the first point cloud data based on the second pose matrix can obtain second point cloud data in different orientations and different positions in the 3D space.

[0063] The step 302 includes: determining second point cloud data corresponding to the second pose matrix according to the first point cloud data.

[0064] In the step 302, for each second pose matrix, the rotation matrix thereof is multiplied by the first point cloud data, and the translation matrix thereof is added, so that the second point cloud data corresponding to the second pose matrix can be obtained, and the second point cloud data can present the avatar pose corresponding to the second pose matrix.

[0065] In actual applications, different offset amounts are added to the first rotation matrix R, so that the original first 3D point cloud data Points is rotated to an incorrect direction, and a group of 3D point cloud data Points* in different orientations can be obtained. The translation transformation is performed on the Points*, and a group of second 3D point cloud data Points* in different orientations around the original first 3D point cloud data Points can be obtained.

[0066] The step 303 includes: superimposing the texture information on the second point cloud data to obtain the second 3D body data.

[0067] In the step 303, the texture information generated by the first image data of the target body part during 3D reconstruction is superimposed on the second point cloud data in different poses, so that a plurality of groups of 3D body data with the same texture and different poses, i.e., the second 3D body data, can be obtained.

[0068] In this embodiment, according to the pose special effect requirement, the first 3D body data obtained by reconstruction is used for spatial pose transformation processing to obtain a plurality of second 3D face data with the same texture and different poses, and the second 3D face data is fused into the original image, so that the body double special effect image of the target body can be obtained.

[0069] Optionally, in an embodiment, the first 3D body data includes a first pose matrix; the first pose matrix includes a first translation matrix, and the body double special effect parameter includes a translation matrix offset; and the step 105 includes steps 501-503.

[0070] In step 501, the second image is size-transformed into a third image according to the first image, the second 3D body data corresponding to the second image, and the first 3D body data.

[0071] In this step, because the second 3D body data and the first 3D body data reflect the distance relationship between the first image of the target body part and the third image corresponding to the body double, and the object imaging conforms to the imaging logical relationship of large near and small far, the size relationship between the third image and the first image can be determined according to the second 3D body data and the first 3D body data, and then the size of the third image can be determined in combination with the size of the first image. According to the size, the second image only needs to be size-transformed, and the obtained image is the above-mentioned third image.

[0072] In step 502, a translation amount of the third image is determined according to the translation matrix offset, the first translation matrix, and the actual size of the third image.

[0073] In this step, the translation matrix offset determines the displacement amount of each body double of the target body relative to the first 3D body data in the three-dimensional space, and the imaging sizes of each body double need to conform to the imaging logical relationship of large near and small far. When the imaging sizes are different, the translation amount of the same translation matrix offset on the two-dimensional plane is also different, so the translation amount of each body double needs to be determined in combination with the actual size of the corresponding two-dimensional image.

[0074] Specifically, the above-mentioned translation amount includes an x-axis direction translation amount ΔP x and a y-axis direction translation amount ΔP y :

[0075]

[0076]

[0077] wherein T x represents the component of the first translation matrix in the x-axis direction, T x represents the component of the translation matrix offset in the y-axis direction, represents a component of the translation matrix bias in the x-axis direction, represents a component of the translation matrix bias in the y-axis direction, W represents the width of the third image, and H represents the height of the third image.

[0078] Step 503: fusing the third image and the original image into a target image according to the translation, the first coordinate information, and the mask data.

[0079] In this step, because the first coordinate information can anchor the specific position of the first image of the original target body part on the original image, the mask data determines the part that needs to be shielded and hidden in each third image, and the translation corresponding to each third image determines the displacement of the third image relative to the first image of the target body part on the original image. Therefore, by taking the above first coordinate information as a reference, the third image is controlled to be translated from the first image of the target body part to the corresponding position according to the translation corresponding to the third image, and then the third image is fused with the original image according to the above mask data, so that the target image presenting the corresponding body effect of the body effect parameter can be obtained.

[0080] In this embodiment, before the rendered second image is fused with the original image, the size of the second image is first transformed, the translation of the second image is calculated according to the translation matrix bias, and then the transformed second image is translated according to the above translation and fused with the original image, so that the fused image not only presents the required body effect, but also conforms to the normal imaging logic of far smaller near larger.

[0081] Optionally, in an embodiment, the above step 501 includes steps 511-513.

[0082] Step 511: determining the original size of the first image, the second average depth of the second 3D body data, and the first average depth of the first 3D body data.

[0083] In this step, the original size of the first image is determined according to the first coordinate information of the first image, which can include the original width and the original height; the first average depth is calculated according to the first point cloud data in the first 3D body data; and the second average depth is calculated according to the second point cloud data in the second 3D body data.

[0084] Step 512: calculating the target size of the third image according to the original size, the first average depth, and the second average depth.

[0085] In this step, because the deeper the depth is, the farther the corresponding avatar is from the camera, and the smaller the size is, and the original image size of the target body part is a constant value, the image size of the corresponding avatar can be determined as the target size of the third image in combination with the proportional relationship between the first average depth and the second average depth.

[0086] Specifically, the width and height of the third image are calculated by formula (3) and formula (4) as W and H respectively, and then:

[0087]

[0088]

[0089] wherein, W * represents the width of the first image, H * represents the height of the first image, Z * represents the first average depth, and Z represents the second average depth.

[0090] Step 513, transforming the second image according to the target size to obtain the third image.

[0091] In this step, the second image obtained by rendering is transformed to the target size to obtain the third image.

[0092] In this embodiment, the target size of the third image is determined based on the original size of the first image, the second average depth of the second 3D body data, and the first average depth of the first 3D body data, and then the second image is transformed to the target size to obtain each third image that conforms to the imaging logic of far small and near large, so that the final avatar special effect also conforms to the normal imaging logic, and the avatar special effect is more realistic and natural.

[0093] Please refer to Figure 2 , Figure 2 Fig. 1 shows a process diagram of performing avatar special effect shooting on a single face in an embodiment of the present application.

[0094] As Figure 2 shown, in step 201, after starting the shooting function, a single frame of original image captured by the camera is acquired;

[0095] In step 202, the single frame of original image is captured for face detection to determine the number of detected face rectangular frames.

[0096] In step 203, the original image is judged according to the number of face rectangular frames; if the number of face rectangular frames is not 1, it is determined that the original image does not conform to the single face scene, and the original image is directly returned to the display end for display; if the number of face rectangular frames is 1, it is determined that the original image conforms to the single face scene, and thus step 204 is entered;

[0097] In step 204, according to the left top point P1 (X1, Y1) and the right bottom point P2 (X2, Y2) of the face rectangular frame, a face image with a width of W=X2-X1 and a height of H=Y2-Y1 can be cut out on the original image, and the P1 and P2 coordinate position information is saved as face position information;

[0098] In step 205, the face image cut out in step 203 is output to a 3D single frame face reconstruction algorithm for real-time 3D face reconstruction, and 3D face data is obtained, including point cloud data, texture information and pose state matrix R / T, wherein R is a rotation matrix and T is a translation matrix;

[0099] In step 206, the 3D face data reconstructed in step 204 is processed by spatial pose transformation according to the split body special effect parameters, and a plurality of 3D face data with the same texture and different poses is obtained;

[0100] In step 207, the different pose 3D face data obtained by transformation is rendered in real time, and face images at different positions in the same pixel coordinate system and mask data corresponding to the face images are obtained;

[0101] In step 208, according to the imaging logical relationship of near large and far small, the plurality of face images with different poses rendered in step 207 are size transformed, the translation amount of the face images after size transformation is determined according to the split body special effect parameters, the specific position of the original face image on the original image is anchored based on the above face position information, the different pose face images after size transformation are fused to the original image based on the above translation amount, and the original image is fed back to the display end;

[0102] In step 209, the display end displays the received image, and presents the single face "shadow split body" special effect.

[0103] The setting method provided in the embodiment of the application can be executed by a shooting device. In the embodiment of the application, the shooting method executed by the shooting device is taken as an example to illustrate the shooting device provided in the embodiment of the application.

[0104] Please refer to Figure 3 , which shows a structural schematic diagram of a shooting device provided in the embodiment of the application, as shown in Figure 3 , the device comprises:

[0105] The acquisition module 31 is configured to, in a case where an original image is collected, determine a first image of a target body part in the original image and first coordinate information of the first image;

[0106] The reconstruction module 32 is configured to reconstruct first 3D body data according to the first image;

[0107] The spatial transformation module 33 is configured to perform spatial pose transformation processing on the first 3D body data according to a target avatar special effect parameter, to obtain second 3D body data in different poses;

[0108] The rendering module 34 is configured to perform rendering on the second 3D body data, to obtain a second image and mask data;

[0109] The fusion module 35 is configured to fuse the second image and the original image into a target image according to the target avatar special effect parameter, the first coordinate information and the mask data.

[0110] Optionally, in the apparatus, the first 3D body data includes first point cloud data, texture information and a first pose matrix;

[0111] The spatial transformation module 33 includes:

[0112] The first determination unit is configured to determine a second pose matrix according to the target avatar special effect parameter and the first pose matrix;

[0113] The second determination unit is configured to determine second point cloud data corresponding to the second pose matrix according to the first point cloud data;

[0114] The superposition unit is configured to superimpose the texture information onto the second point cloud data, to obtain the second 3D body data.

[0115] Optionally, in the apparatus, the avatar special effect parameter includes a rotation matrix offset amount; the first pose matrix includes a first rotation matrix, and the second pose matrix includes a second rotation matrix;

[0116] The first determination unit is specifically configured to determine the second rotation matrix according to the first rotation matrix and the rotation matrix offset amount.

[0117] Optionally, in the apparatus, the avatar special effect parameter includes a rotation matrix offset amount and a translation matrix offset amount; the first pose matrix includes a first rotation matrix and a first translation matrix, and the second pose matrix includes a second rotation matrix and a second translation matrix;

[0118] The first determining unit is specifically configured to determine the second rotation matrix according to the first rotation matrix and the rotation matrix offset.

[0119] The second translation matrix is determined according to the first translation matrix and the translation matrix offset.

[0120] Optionally, in the device, the first 3D body data includes a first pose matrix; the first pose matrix includes a first translation matrix, and the avatar special effect parameter includes a translation matrix offset.

[0121] The fusion module 35 includes:

[0122] The size transformation unit is configured to perform size transformation on the second image to obtain a third image according to the first image, the second 3D body data and the first 3D body data.

[0123] The third determining unit is configured to determine a translation amount of the third image according to the translation matrix offset, the first translation matrix and an actual size of the third image.

[0124] The fusion unit is configured to fuse the third image and the original image into a target image according to the translation amount, the first coordinate information and the mask data.

[0125] Optionally, in the device, the size transformation unit includes:

[0126] The determining subunit is configured to determine an original size of the first image, a second average depth of the second 3D body data and a first average depth of the first 3D body data.

[0127] The calculating subunit is configured to calculate a target size of the third image according to the original size, the first average depth and the second average depth.

[0128] The transforming subunit is configured to transform the second image according to the target size to obtain the third image.

[0129] Optionally, in the device, the target body part is a face.

[0130] The acquisition module 31 includes:

[0131] The detecting unit is configured to perform face image detection on the original image.

[0132] The third determining unit is configured to determine second coordinate information of a rectangular region in which the face image is located, in a case where the face image is detected.

[0133] An image cutting unit is configured to cut the face image from the original image according to the second coordinate information, and determine the second coordinate information as the first coordinate information.

[0134] In summary, the photographing device provided by the embodiments of the present application can automatically generate target images with different poses based on the collected original image, only by controlling the camera to take a picture, and thus the steps of walking and taking pictures multiple times are saved, and thus the problem that the existing way of taking a picture with a split image special effect is complicated is effectively solved.

[0135] The photographing device in the embodiments of the present application can be an electronic device or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices other than a terminal. For example, the electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a mobile Internet device (MID), an augmented reality (AR) / virtual reality (VR) device, a robot, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), and can also be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, and the embodiments of the present application are not limited in this regard.

[0136] The photographing device in the embodiments of the present application can be a device with an operating system. The operating system can be an Android operating system, an iOS operating system, or other possible operating systems, and the embodiments of the present application are not limited in this regard.

[0137] The photographing device provided by the embodiments of the present application can implement Figures 1 to 2 The processes implemented by the method embodiments are not repeated here to avoid repetition.

[0138] Optionally, as shown in Figure 4 The embodiments of the present application also provide an electronic device 400, which includes a processor 401 and a memory 402. The memory 402 stores programs or instructions that can be run on the processor 401. When the programs or instructions are executed by the processor 401, the steps of the above setting method embodiments are implemented, and the same technical effects are achieved. To avoid repetition, the steps are not repeated here.

[0139] It should be noted that the electronic device in the embodiments of the present application includes the mobile electronic device and the non-mobile electronic device described above.

[0140] Figure 5 A hardware structure schematic diagram of an electronic device according to an embodiment of the present application.

[0141] The electronic device 600 includes, but is not limited to, a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609, and a processor 610, etc.

[0142] Those skilled in the art can understand that the electronic device 600 can also include a power supply (such as a battery) for powering each component, and the power supply can be logically connected to the processor 610 through a power management system, so as to realize the functions of managing charging, discharging, and power consumption management through the power management system. Figure 5 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the figure, or combine certain components, or different component arrangements, which are not described here.

[0143] The user input unit 607 is configured to receive a user input operation on a clone special effect parameter.

[0144] The processor 610 is configured to, in a case where an original image is collected, determine a first image of a target body part in the original image and first coordinate information of the first image, reconstruct first 3D body data according to the first image, perform spatial pose transformation processing on the first 3D body data according to a target clone special effect parameter to obtain second 3D body data in different poses, perform rendering on the second 3D body data to obtain a second image and mask data, and fuse the second image and the original image into a target image according to the target clone special effect parameter, the first coordinate information, and the mask data.

[0145] The display unit 606 is configured to display the target image.

[0146] The electronic device provided in the embodiments of the present application can automatically generate a target image with different poses based on the collected original image when the user only needs to control the camera to take a picture, thereby saving the steps of walking, multiple shooting, and effectively solving the problem that the existing technology is relatively cumbersome to take a picture with a clone special effect image.

[0147] Optionally, the first 3D body data includes first point cloud data, texture information, and a first pose matrix.

[0148] The processor 610 is specifically configured to determine a second pose matrix according to the target clone special effect parameter and the first pose matrix; determine second point cloud data corresponding to the second pose matrix according to the first point cloud data; and superimpose the texture information to the second point cloud data to obtain the second 3D body data.

[0149] Optionally, the clone special effect parameter includes a rotation matrix offset; the first pose matrix includes a first rotation matrix, and the second pose matrix includes a second rotation matrix.

[0150] The processor 610 is specifically configured to determine the second rotation matrix according to the first rotation matrix and the rotation matrix offset.

[0151] Optionally, the clone special effect parameter includes a rotation matrix offset and a translation matrix offset; the first pose matrix includes a first rotation matrix and a first translation matrix, and the second pose matrix includes a second rotation matrix and a second translation matrix.

[0152] The processor 610 is specifically configured to determine the second rotation matrix according to the first rotation matrix and the rotation matrix offset, and determine the second translation matrix according to the first translation matrix and the translation matrix offset.

[0153] Optionally, the first 3D body data includes a first pose matrix; the first pose matrix includes a first translation matrix, and the clone special effect parameter includes a translation matrix offset.

[0154] The processor 610 is specifically configured to perform size transformation on the second image to obtain a third image according to the first image, the second 3D body data and the first 3D body data; determine a translation amount of the third image according to the translation matrix offset, the first translation matrix and an actual size of the third image; and fuse the third image and the original image into a target image according to the translation amount, the first coordinate information and the mask data.

[0155] Optionally, the processor 610 is specifically configured to determine an original size of the first image, a second average depth of the second 3D body data and a first average depth of the first 3D body data; calculate a target size of the third image according to the original size, the first average depth and the second average depth; and transform the second image according to the target size to obtain the third image.

[0156] Optionally, the target body part is a face.

[0157] The processor 610 is specifically configured to perform face image detection on the original image; in a case where a face image is detected, determine second coordinate information of a rectangular region where the face image is located; according to the second coordinate information, cut the face image from the original image, and determine the second coordinate information as the first coordinate information.

[0158] It should be understood that in the embodiments of the present application, the input unit 604 can include a graphics processor (Graphics Processing Unit, GPU) 6041 and a microphone 6042. The graphics processor 6041 processes image data of a still picture or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 606 can include a display panel 6061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 607 includes at least one of a touch panel 6071 and other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 can include a touch detection device and a touch controller. The other input devices 6072 can include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), a trackball, a mouse, an operation lever, and the like, which will not be described here.

[0159] The memory 609 can be used to store software programs and various data. The memory 609 can mainly include a first storage area storing programs or instructions and a second storage area storing data, wherein the first storage area can store an operating system, application programs or instructions required by at least one function (such as a sound playing function, an image playing function, etc.), and the like. In addition, the memory 609 can include a volatile memory or a non-volatile memory, or the memory 609 can include both a volatile memory and a non-volatile memory. The non-volatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM), a Static RAM (SRAM), a Dynamic RAM (DRAM), a Synchronous DRAM (SDRAM), a Double Data Rate SDRAM (DDR SDRAM), an Enhanced SDRAM (ESDRAM), a Synch link DRAM (SLDRAM), and a Direct Rambus RAM (DRRAM). The memory 609 in the embodiments of the present application includes but is not limited to these and any other suitable types of memories.

[0160] The processor 610 can include one or more processing units; optionally, the processor 610 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and an application program, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 610.

[0161] The embodiments of the present application also provide a readable storage medium, the readable storage medium stores programs or instructions, the programs or instructions are executed by a processor to realize the processes of the above-mentioned photographing method embodiments, and the same technical effects can be achieved. To avoid repetition, details are not described here.

[0162] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer readable only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0163] The embodiment of the present application further provides a chip, which comprises a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running programs or instructions to realize the processes of the above-mentioned photographing method embodiments and achieve the same technical effects. To avoid repetition, details are not described herein.

[0164] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0165] The embodiment of the present application provides a computer program product, which is stored in a storage medium, and is executed by at least one processor to realize the processes of the above-mentioned photographing method embodiments and achieve the same technical effects. To avoid repetition, details are not described herein.

[0166] It should be noted that in this document, the term "comprising" or "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to the order of performing the functions as shown or discussed, but can also include performing the functions in a substantially simultaneous manner or in a reverse order, for example, the described method can be performed in an order different from that described, and various steps can also be added, omitted or combined. In addition, the features described with reference to some examples can be combined in other examples.

[0167] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned example methods can be realized by means of software and a necessary general hardware platform, and of course, can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product in essence or in the form of a part that contributes to the prior art, which is stored in a storage medium (such as a ROM / RAM, a magnetic disk, or an optical disk) and includes a plurality of instructions for causing a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application.

[0168] The embodiments of the present application are described above in combination with the drawings, but the present application is not limited to the above-mentioned specific embodiments, and the above-mentioned specific embodiments are only illustrative and not restrictive. Those skilled in the art can make many forms under the inspiration of the present application without departing from the scope of the present application and the scope protected by the claims.

Claims

1. A photographing method characterized by comprising: Comprising: In the case of collecting the original image, determining a first image of a target body part in the original image and first coordinate information of the first image; wherein the target body part is obtained by receiving a selection operation of a target body part requiring to present a body effect in the case of starting a body effect shooting mode; the target body part includes part of the body; Reconstructing first 3D body data according to the first image; the first 3D body data includes a first pose matrix; the first pose matrix includes a first translation matrix; According to the target body effect parameter, the first 3D body data is subjected to spatial pose transformation processing to obtain second 3D body data of different poses; wherein the target body effect parameter is a parameter representing the pose and position of each body; the target body effect parameter includes a translation matrix bias; Rendering the second 3D body data to obtain a second image and mask data; According to the target body effect parameter, the first coordinate information and the mask data, the second image and the original image are fused into a target image; According to the target body effect parameter, the first coordinate information and the mask data, the second image and the original image are fused into a target image, comprising: Determining the original size of the first image, the second average depth of the second 3D body data and the first average depth of the first 3D body data; According to the original size, the first average depth and the second average depth, calculating the target size of the third image; According to the target size, the second image is transformed to obtain the third image; According to the translation matrix bias, the first translation matrix and the actual size of the third image, the translation amount of the third image is determined; According to the translation amount, the first coordinate information and the mask data, the third image and the original image are fused into a target image.

2. The photographing method according to claim 1, wherein The first 3D body data further includes first point cloud data and texture information; According to the target body effect parameter, the first 3D body data is subjected to spatial pose transformation processing to obtain second 3D body data of different poses, comprising: According to the target body effect parameter and the first pose matrix, a second pose matrix is determined; According to the first point cloud data, second point cloud data corresponding to the second pose matrix is determined; The texture information is superimposed on the second point cloud data to obtain the second 3D body data.

3. The photographing method according to claim 2, wherein The target body effect parameter includes a rotation matrix bias and a translation matrix bias; the first pose matrix includes a first rotation matrix and the first translation matrix, and the second pose matrix includes a second rotation matrix and a second translation matrix; According to the target body effect parameter and the first pose matrix, a second pose matrix is determined, comprising: According to the first rotation matrix and the rotation matrix bias, the second rotation matrix is determined; According to the first translation matrix and the translation matrix bias, the second translation matrix is determined.

4. An imaging device, characterized by comprising: Comprising: The acquisition module is configured to, in a case where an original image is collected, determine a first image of a target body part in the original image and first coordinate information of the first image, wherein the target body part is obtained by receiving a selection operation of a target body part requiring a body double effect in a case where a body double effect shooting mode is started, and the target body part includes a part of a body; The reconstruction module is configured to reconstruct first 3D body data according to the first image, wherein the first 3D body data includes a first pose matrix, and the first pose matrix includes a first translation matrix; The spatial transformation module is configured to perform spatial pose transformation processing on the first 3D body data according to target body double effect parameters to obtain second 3D body data in different poses, wherein the target body double effect parameters represent poses and positions of each body double, and the target body double effect parameters include a rotation matrix offset and a translation matrix offset; The rendering module is configured to render the second 3D body data to obtain a second image and mask data; The fusion module is configured to fuse the second image and the original image into a target image according to the target body double effect parameters, the first coordinate information, and the mask data. The fusion module includes: The size transformation unit is configured to perform size transformation on the second image into a third image according to the first image, the second 3D body data, and the first 3D body data. The third determination unit is configured to determine a translation amount of the third image according to the translation matrix offset, the first translation matrix, and an actual size of the third image. The fusion unit is configured to fuse the third image and the original image into a target image according to the translation amount, the first coordinate information, and the mask data. The size transformation unit includes: The determination subunit is configured to determine an original size of the first image, a second average depth of the second 3D body data, and a first average depth of the first 3D body data. The calculation subunit is configured to calculate a target size of the third image according to the original size, the first average depth, and the second average depth. The transformation subunit is configured to transform the second image according to the target size to obtain the third image.

5. The apparatus of claim 4, wherein, The first 3D body data includes first point cloud data, texture information, and a first pose matrix. The spatial transformation module includes: The first determination unit is configured to determine a second pose matrix according to the target body double effect parameters and the first pose matrix. The second determination unit is configured to determine second point cloud data corresponding to the second pose matrix according to the first point cloud data. The superposition unit is configured to superimpose the texture information on the second point cloud data to obtain the second 3D body data.

6. The apparatus of claim 5, wherein, The target body double effect parameters include a rotation matrix offset and a translation matrix offset, the first pose matrix includes a first rotation matrix and the first translation matrix, and the second pose matrix includes a second rotation matrix and a second translation matrix. The first determining unit is specifically configured to determine the second rotation matrix according to the first rotation matrix and the rotation matrix offset. The second translation matrix is determined according to the first translation matrix and the translation matrix offset.

Images