Image processing method and device, electronic equipment and computer readable medium

By obtaining human bone information and depth information, building a three-dimensional grid model and adjusting the bone area, the problem of inadequate posture and expression in mobile terminal photography is solved, and posture and expression adjustment in image processing is realized, which improves shooting efficiency and quality .

CN118941748APending Publication Date: 2024-11-12NANJING OPPO SOFTWARE TECH CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310530884.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-11
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

When taking photos on mobile terminals, users are prone to being unable to observe their own postures, resulting in ugly postures and inadequate expressions, which leads to repeated adjustments and shooting.

Method used

By obtaining the human bone information and depth information in the pending image, a three-dimensional mesh model is constructed based on the skin operation, and the bone area is adjusted to change the body characteristics, thereby generating the adjusted image.

Benefits of technology

It is realized that the human posture and expression in the captured image are adjusted through image processing methods without real-time adjustment of posture and expression, thereby improving shooting efficiency and quality.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941748A_ABST
    Figure CN118941748A_ABST
Patent Text Reader

Abstract

The invention discloses an image processing method and device, electronic equipment and a computer readable medium, and relates to the technical field of computers, and the method comprises the steps: obtaining a to-be-processed first image; acquiring skeleton information and depth information corresponding to a human body image in the first image; based on the depth information and the skeleton information, obtaining a specified three-dimensional grid model corresponding to the human body image through skin operation; adjusting a target skeleton area in the specified three-dimensional mesh model based on parameter information so as to change body features of a body part corresponding to the target skeleton area in the specified three-dimensional mesh model; and obtaining a second image corresponding to the adjusted specified three-dimensional grid model. Therefore, by determining the skeleton area of the specified three-dimensional mesh model, the body features of the corresponding body parts can be changed, and then the body features are converted into the image, so that the body parts in the to-be-processed image can be changed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and more specifically, to an image processing method, device, electronic device and computer-readable medium. Background Art

[0002] As mobile terminal photography technology matures and related hardware costs continue to decrease, the demand for mobile terminal users to use mobile terminals to take photos is also growing. Compared with traditional cameras, mobile terminals have a higher penetration rate and richer communication and storage functions, which also puts higher requirements on mobile terminal photography software systems. Although its photography functions are becoming more and more abundant, there are still many problems in taking photos, such as stiff postures, inadequate movements / expressions, etc. Summary of the invention

[0003] The present application proposes an image processing method, device, electronic device and computer-readable medium to improve the above-mentioned defects.

[0004] In a first aspect, an embodiment of the present application provides an image processing method, which is applied to an electronic device, and the method includes: obtaining a first image to be processed; obtaining bone information and depth information corresponding to a human body image in the first image; based on the depth information and the bone information, obtaining a specified three-dimensional mesh model corresponding to the human body image through a skinning operation; adjusting a target bone area in the specified three-dimensional mesh model based on parameter information to change the physical features of a body part corresponding to the target bone area in the specified three-dimensional mesh model; and obtaining a second image corresponding to the adjusted specified three-dimensional mesh model.

[0005] In a second aspect, an embodiment of the present application further provides an image processing device, which is applied to an electronic device, and the device includes: a first acquisition unit, a second acquisition unit, a binding unit, an adjustment unit, and a conversion unit. The first acquisition unit is used to acquire a first image to be processed. The second acquisition unit is used to acquire the bone information and depth information corresponding to the human body image in the first image. The skinning unit is used to obtain a specified three-dimensional mesh model corresponding to the human body image through a skinning operation based on the depth information and the bone information. The adjustment unit is used to adjust the target bone area in the specified three-dimensional mesh model based on parameter information to change the physical features of the body part corresponding to the target bone area in the specified three-dimensional mesh model. The conversion unit is used to acquire a second image corresponding to the adjusted specified three-dimensional mesh model.

[0006] In a third aspect, an embodiment of the present application also provides an electronic device, comprising: one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the above method.

[0007] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the readable storage medium stores a program code executable by a processor, and when the program code is executed by the processor, the processor executes the above method.

[0008] The image processing method, device, electronic device and computer-readable medium provided by the present application, after obtaining the first image to be processed, obtain the bone information and depth information corresponding to the human body image in the first image, based on the depth information and the bone information, based on the skinning operation, obtain the specified three-dimensional mesh model corresponding to the human body image, adjust the target bone area in the specified three-dimensional mesh model based on the parameter information, because after the skinning operation, the specified three-dimensional mesh model image is bound to the bone information and the human body parts of the specified three-dimensional mesh model image can be driven by the skeleton, so adjusting the target bone area will change the body features of the body parts corresponding to the target bone area, and then, obtain the second image corresponding to the adjusted specified three-dimensional mesh model. Therefore, by determining the bone area of ​​the specified three-dimensional mesh model, the body features of the corresponding body parts can be changed, and then converted into an image, it is possible to achieve the change of the human body parts in the image to be processed.

[0009] Other features and advantages of the embodiments of the present application will be described in the subsequent description, and partly become apparent from the description, or can be understood by practicing the embodiments of the present application. The purposes and other advantages of the embodiments of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A schematic diagram of a human body posture provided by an embodiment of the present application is shown;

[0012] Figure 2 A schematic diagram of an image processing method provided by an embodiment of the present application is shown;

[0013] Figure 3 A schematic diagram of a human body image provided by an embodiment of the present application is shown;

[0014] Figure 4 A schematic diagram showing human skeleton information provided by an embodiment of the present application is shown;

[0015] Figure 5 A schematic diagram showing a graphics processing flow provided by an embodiment of the present application is shown;

[0016] Figure 6 A schematic diagram showing the relationship between 2D coordinates and 3D coordinates provided in an embodiment of the present application is shown;

[0017] Figure 7 A schematic diagram of a skinning operation provided by an embodiment of the present application is shown;

[0018] Figure 8 A schematic diagram of a UV map provided in an embodiment of the present application is shown;

[0019] Fig. 9 A schematic diagram of an image processing method provided by another embodiment of the present application is shown;

[0020] Fig.10 A schematic diagram of a specified three-dimensional grid model provided by an embodiment of the present application is shown;

[0021] Fig.11 A schematic diagram of a skeleton drive for a specified three-dimensional mesh model provided by an embodiment of the present application is shown;

[0022] Fig.12 A schematic diagram of a skeleton drive for a specified three-dimensional mesh model provided by another embodiment of the present application is shown;

[0023] Fig.13 A schematic diagram showing a first image and a second image provided by an embodiment of the present application is shown;

[0024] Fig.14 A schematic diagram of a designated selection interface provided by an embodiment of the present application is shown;

[0025] Fig.15 A schematic diagram of posture adjustment provided by an embodiment of the present application is shown;

[0026] Fig.16 A schematic diagram showing an image processing method provided by another embodiment of the present application is shown;

[0027] Fig.17 A schematic diagram of a skeleton drive of a specified three-dimensional mesh model provided by another embodiment of the present application is shown;

[0028] Fig.18A schematic diagram of a skeleton drive of a specified three-dimensional mesh model provided by another embodiment of the present application is shown;

[0029] Fig.19 A module block diagram of an image processing device provided by an embodiment of the present application is shown;

[0030] Fig. 20 A schematic diagram of an electronic device provided by an embodiment of the present application is shown;

[0031] Fig.21 A schematic diagram of a computer-readable storage medium provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0032] In order to make those skilled in the art better understand the present application scheme, the technical scheme in the present application embodiment will be clearly and completely described below in conjunction with the drawings in the present application embodiment. Obviously, the described embodiment is only a part of the present application embodiment, rather than all the embodiments. The components of the present application embodiment usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application for protection, but merely represents the selected embodiment of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0033] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0034] As mobile terminal photography technology matures and related hardware costs continue to decrease, the demand for mobile terminal users to use mobile terminals to take photos is also growing. Compared with traditional cameras, mobile terminals have a higher penetration rate and richer communication and storage functions, which also puts higher requirements on mobile terminal photography software systems.

[0035] However, the inventors found in their research that Figure 1As shown in the figure, although users can show a variety of postures when taking photos, for ordinary people who take photos, the person being photographed cannot observe his or her own posture and often worries that the posture is ugly. This is even more true for people who are not good at posing, which leads to people having to repeatedly adjust their posture and retake the photo. Not only that, there will also be problems with inadequate expressions. Therefore, when taking photos, users usually encounter problems such as inadequate posture and inadequate expressions.

[0036] In order to solve the above-mentioned defects, the present application proposes an image processing method, device, electronic device and computer-readable medium, so that when it is necessary to adjust the posture or facial expression of a human body in the image, the adjustment of the posture or facial expression of the human body can be completed by driving the human skeleton.

[0037] See also Figure 2 , Figure 2 An image processing method provided in an embodiment of the present application is shown, and the method includes: S201 to S205.

[0038] S201: Acquire a first image to be processed.

[0039] As an implementation mode, the first image is an image captured by an image acquisition device of an electronic device. Of course, it can also be an image obtained by the electronic device from other devices, for example, an image downloaded from a server via the Internet, and there is no limitation on this. In an embodiment of the present application, in order to solve the aforementioned problems such as inappropriate posture and / or inappropriate expression when taking photos, an image captured by an image acquisition device of the electronic device can be used as the first image. Specifically, the electronic device includes an image acquisition device. For example, the image acquisition device can be a front camera and / or a rear camera of the electronic device.

[0040] In addition, it should be noted that in order to facilitate the subsequent adjustment of the person in the image through the human skeleton, the first image refers to an image containing a human body, for example, it can be an image captured by a user through an image capture device of an electronic device when taking a photo of a person.

[0041] S202: Obtain skeleton information and depth information corresponding to the human body image in the first image.

[0042] It should be noted that the first image can be a color image, and the human body image is obtained by segmenting the first image, that is, the first image is subjected to portrait segmentation to obtain an image containing only the portrait, thereby obtaining the human body image. Portrait segmentation is to segment the pixels belonging to the portrait in the input picture containing the portrait, and the input is a color picture containing the portrait; the output is a binary picture, and the pixels belonging to the portrait are represented by white, and the pixels not belonging to the portrait are represented by black. Among them, the algorithm for portrait segmentation applied to the embodiment of the present application can be Convolutional Neural Networks (CNN), Fully Convolutional Networks (FCN), etc., which are not limited here. Figure 3 As shown, Figure 3 An example diagram of a first image is shown. After portrait segmentation is performed on the first image 310 , a human body image 320 including a human body region 321 is obtained.

[0043] As an implementation method, in order to obtain the depth information of a human body image, when the first image is obtained, the depth image corresponding to the first image can be obtained at the same time. Exemplarily, the image acquisition device of the electronic device has the function of acquiring the depth image. When the image acquisition device of the electronic device acquires the first image, it also acquires the depth image of the shooting scene corresponding to the first image. The depth image can correspond to each pixel of the first image, that is, the depth value of each pixel in the first image can be determined in the depth image. Then, after the human body image in the first image is obtained in the aforementioned manner, the position of each pixel in the human body image in the first image can be determined, so that the depth information corresponding to the human body image, that is, the depth value of each pixel in the human body image, can be determined from the depth image.

[0044] As another implementation, the electronic device may perform depth estimation on the first image by monocular image depth estimation, thereby obtaining depth information of the image content in the first image. Optionally, the electronic device may obtain the depth information of the image content in the first image by inputting the first image into a pre-trained depth estimation model.

[0045] Exemplarily, the depth estimation model can be a Monodepth2 model, which uses an unsupervised learning method based on monocular images to complete the task of depth estimation. When training the Monodepth2 model, two sets of networks are trained simultaneously in the convolutional neural network according to the principle of the SFM model: a deep network and a pose network. The input of the training network is a continuous multi-sequential image of a video sequence. The deep network inputs the target view, and the pose network inputs the target view and the previous sequential view. The deep network is processed by the convolutional neural network to output the corresponding depth image. The pose network calculates the change in the camera motion posture. The reprojected image is jointly constructed based on the output results of the two sets of networks, and the reprojection error is calculated and introduced into the loss function. The model parameters are updated through the loss function to backpropagate and optimize the training network model.

[0046] As an implementation, the human body posture of the human body image in the first image is analyzed by a skeleton detection algorithm, so as to determine the skeleton information corresponding to the human body image. Exemplarily, the skeleton detection algorithm can be to analyze the human body image, detect the skeleton key points in the human body image, and then obtain the information such as bones and joints through the skeleton key points, so as to obtain the skeleton information of the human body image in the first image. Figure 4 As shown, the human body image in the first image is detected to obtain various joints and bones in the human body image, and then the bone information is obtained. Since the human skeleton is detected through the first image, each key bone point of the human skeleton can correspond to each pixel point in the human body image. Therefore, the bone information includes the position of each key bone point on the human body image, the degree of freedom of the joint point, the constraint, and the inverse kinematic parameters (Inverse Kinematic, IK), forward kinematic parameters (Forward Kinematic, FK), etc. Among them, the way to detect the bone information can be through the K3M sequential iteration algorithm or the deep learning algorithm, which is not limited here.

[0047] As another embodiment, the skeletal information corresponding to the human body image in the first image may not be obtained by detecting the various key skeletal points of the human body image, but may be pre-set skeletal information. Exemplarily, multiple skeletal information may be pre-set, each skeletal information corresponds to a human feature, and the human feature may be information such as a person's gender and race. After acquiring the human body image of the first image, the first image is analyzed to obtain the human body feature corresponding to the first image, and a skeletal information is determined from the pre-set multiple skeletal information based on the human body feature as the skeletal information corresponding to the human body image in the first image. It should be noted that the skeletal information determined in this way does not match the posture or position of the human body image in the first image. However, the skeletal information can be adjusted later by binding the skeletal information to the human body image. By using this general skeletal information method, the amount of calculation required to detect the human body image to obtain skeletal information can be reduced.

[0048] S203: Based on the depth information and the skeleton information, a specified three-dimensional mesh model corresponding to the human body image is obtained through a skinning operation.

[0049] Please note that Figure 5 In the embodiment of the present application, the specified three-dimensional mesh model is a 3D model. Relative to the specified three-dimensional mesh model, the first image is a 2D image. Therefore, the process of obtaining the specified three-dimensional mesh model corresponding to the human body image can also be regarded as a process of converting 2D to 3D. The process of converting 2D to 3D can be divided into the establishment of a portrait 3D mesh model and the portrait skeleton binding operation.

[0050] As an implementation method, the implementation method of establishing a portrait 3D mesh model can be, based on the human body image and the depth information and the camera parameters acquired by the image acquisition device, obtaining an initial three-dimensional mesh model (i.e., 3D Mesh) corresponding to the human body image, which can also be called a first three-dimensional mesh model, binding the skeleton information to the first three-dimensional mesh model to obtain a second three-dimensional mesh model; performing a skinning operation on the second three-dimensional mesh model to obtain the specified three-dimensional mesh model.

[0051] For example, see Figure 6 , Figure 6 A schematic diagram of a pinhole imaging model is shown, where f is the focal length of the camera, the coordinate system where the physical imaging plane 510 is located is the pixel coordinate system O'x'y', and the coordinate system where the camera plane 520 is located is the camera coordinate system Oxyz. The coordinate relationship corresponding to the pinhole imaging model is:

[0052]

[0053] In the above formula (1), [u, v] is the pixel coordinate on the 2D image, that is, the coordinate of each pixel point in the pixel coordinate system, Z is the depth value corresponding to each pixel point in the depth map, and [X, Y, Z] is the 3D coordinate corresponding to the pixel point. x ,f y ,c x ,c y ] is the camera internal parameter. x ,c y ] represents the pixel coordinates of the origin of the center of the screen, [f x ,f y ] is the projection of the focal length in the X-axis / Y-axis direction, in pixels. Usually, the intrinsic parameters of the camera are fixed parameter values ​​after the camera leaves the factory and will not change during use. In an embodiment of the present application, the pixel coordinates of each pixel point of the human body image in the first image and the intrinsic parameters of the image acquisition device of the electronic device used to acquire the first image, i.e., the camera parameters, are substituted into the above formula (1), so as to obtain the three-dimensional coordinates corresponding to each pixel point of the human body image in the first image, i.e., the 3D coordinates, and then, based on the three-dimensional coordinates, the first three-dimensional mesh model corresponding to the human body image is obtained. Among them, Mesh is composed of a series of planar 2D polygons, which are defined by vertices. Vertices are points that store XYZ coordinate values ​​in 3D space, i.e., the three-dimensional coordinates corresponding to each pixel point of the human body image in the first image obtained above.

[0054] It should be noted that the process of binding the skeleton information to the first three-dimensional mesh model can be regarded as the process of aligning the positions of the skeleton information and the first three-dimensional mesh model, that is, each key skeleton point (for example, joint point) in the skeleton information corresponds to each position of the human body in the first three-dimensional mesh model, and the length of the skeleton also matches the length of each part of the human body in the first three-dimensional mesh model. Based on the above content, it can be determined that the method of determining the skeleton information can include at least two methods, one is to detect the human body image to obtain the skeleton information, and the other is to determine the skeleton information that matches the human body features of the human body image in the first image from the preset skeleton information. Regardless of which method is used to obtain the skeleton information, the key skeleton points and the position, size and angle of each skeleton in the skeleton information can be changed in order to be able to align with the human body in the first three-dimensional mesh model, for example, the skeleton is elongated, the skeleton is rotated or moved, etc., so that each part of the human body in the first three-dimensional mesh model can be correctly aligned with the skeleton information.

[0055] In an embodiment of the present application, the first three-dimensional mesh model that is successfully bound to the skeleton information is named as the second three-dimensional mesh model, and then a skinning operation is performed on the second three-dimensional mesh model to obtain the specified three-dimensional mesh model. The skinning operation refers to binding the vertices of the mesh (i.e., the second three-dimensional mesh model) to the corresponding skeleton to obtain the specified three-dimensional mesh model. When the skeleton moves, the specified three-dimensional mesh model will move to the corresponding position according to the bound skeleton. As an implementation method, after the skeleton information is bound to the first three-dimensional mesh model to obtain the second three-dimensional mesh model, each vertex of the second three-dimensional mesh model can correspond to each skeleton region position in the skeleton information. When the skinning operation is performed, the skeleton region corresponding to each vertex can be determined. That is, based on the skeleton region corresponding to each of the aforementioned vertices in position, the skeleton region corresponding to each vertex when the skinning operation is performed can be determined. For example, in the second three-dimensional mesh model, the vertices in the left shoulder region correspond to the positions of the left shoulder bones. When the skinning operation is performed on the second three-dimensional mesh model to obtain the specified three-dimensional mesh model, the skeleton region corresponding to the vertices in the left shoulder region is the left shoulder skeleton. Figure 7 As shown in the figure, the area in the circle represents the vertex area of ​​the specified 3D mesh model that can be affected by the left shoulder bone. In addition, the weight of each vertex affected by the bone that can bring the vertex movement in this area is different. The larger the weight, the greater the range of movement that can be driven, such as Figure 7 In the circular area, the left shoulder bone has a greater influence weight on each vertices in the vertex area from the left armpit to the left shoulder surface, and the left shoulder bone has a smaller influence weight on each vertices in the vertex area on the left anterior chest surface. That is to say, when the left shoulder bone moves a certain amplitude, each vertices in the vertex area from the left armpit to the left shoulder surface are moved with a greater amplitude driven by the left shoulder bone, and each vertices in the vertex area on the left anterior chest surface are moved with a smaller amplitude driven by the left shoulder bone.

[0056] That is to say, the aforementioned change from the first three-dimensional mesh model to the second three-dimensional mesh model can be regarded as aligning the position, size, etc. of the skeleton with the 3D human body, and the skinning operation from the second three-dimensional mesh model to the specified three-dimensional mesh model can be regarded as the process of matching the skeleton with the 3D human body, which is the prerequisite for realizing skeleton-driven human body movement.

[0057] S204: Adjusting the target bone region in the specified three-dimensional mesh model based on the parameter information to change the body features of the body part corresponding to the target bone region in the specified three-dimensional mesh model.

[0058] Through the aforementioned skinning operation, the skeleton in the specified three-dimensional model can already drive the body part bound to the skeleton, so after determining the part that needs to be adjusted, determine the target skeleton area corresponding to the part, and then adjust the target skeleton area in the specified three-dimensional mesh model based on the parameter information, thereby changing the body features of the body part corresponding to the target skeleton area in the specified three-dimensional mesh model. Exemplarily, the body features may include features such as posture and size, which will be specifically described through subsequent embodiments.

[0059] S205: Acquire the adjusted second image corresponding to the specified three-dimensional mesh model.

[0060] It is understandable that if Figure 5 As shown, the process of obtaining the second image through the adjusted designated three-dimensional mesh model can be regarded as a process of converting 3D to 2D. As an implementation method, the aforementioned process of converting 2D to 3D can convert a two-dimensional image into a three-dimensional mesh model. Conversely, by the aforementioned first correspondence between 2D coordinates and 3D coordinates, each three-dimensional coordinate point of the adjusted designated three-dimensional mesh model is converted into a two-dimensional coordinate point to obtain a two-dimensional image, i.e., the second image. As an implementation method, in the above-mentioned process of converting 3D to 2D, there is a UV mapping operation. Specifically, the aforementioned first three-dimensional mesh model can be an uncolored mesh model. Of course, the first three-dimensional mesh model can also be a mesh model that has been subjected to a UV mapping operation through the RGB information of the human body image, i.e., it has been colored. Whether the first three-dimensional mesh model is colored does not affect the subsequent generation of the second image.

[0061] Assuming that the first three-dimensional mesh model is not colored, then after adjusting the target bone region in the specified three-dimensional mesh model based on the parameter information, a UV mapping operation is performed on the adjusted specified three-dimensional mesh model using the RGB information of the human body image in the first image to obtain a colored specified three-dimensional mesh model, which is named as the three-dimensional mesh model to be processed, such as Figure 8 As shown, Figure 8 The coloring process of the three-dimensional mesh model, that is, the process of UV mapping operation, that is, in the process of adjusting the target bone area in the specified three-dimensional mesh model based on the parameter information, the uncolored specified three-dimensional mesh model is adjusted, that is, Figure 8 (a) shows the adjusted uncolored specified 3D mesh model, Figure 8 (b) is the RGB information of the human body image, that is, the UV texture. Then, using Figure 8 (b) UV texture pair Figure 8 (a) Perform the mapping operation and obtain Figure 8 (c) The specified three-dimensional mesh model after coloring is the three-dimensional mesh model to be processed.

[0062] Then, a three-dimensional to two-dimensional mapping operation is performed on the three-dimensional mesh model to be processed to obtain an adjusted human body image, and then the adjusted human body image is fused with the background image other than the human body image in the first image to obtain a second image. Specifically, the RGB information of the second image obtained after mapping matches the RGB information of the specified three-dimensional mesh model after coloring. Specifically, the physical features of the body part corresponding to the target skeletal area can be named as the target body part. In the second image obtained by the above method, the physical features of the target body part have been changed compared to the first image.

[0063] Exemplarily, the 3D to 2D mapping operation can be, by image_vertices = mesh.transform.to_image (transformed_vertices, h, w), converting each 3D coordinate point of the specified 3D mesh model after adjustment into a 2D coordinate point, and then executing the following code to obtain the second image, i.e., image = mesh.render.render_colors (image_vertices, bfm.triangles, colors, h, w). Among them, colors is the color RGB information after UV mapping, and h and w represent the size of the second image, which can be kept consistent with the size of the first image, of course, it can also be different, which is not limited here.

[0064] Therefore, after obtaining the first image to be processed, the embodiment of the present application obtains the skeleton information and depth information corresponding to the human body image in the first image, obtains the specified three-dimensional mesh model corresponding to the human body image based on the depth information and the skeleton information, and based on the skinning operation, adjusts the target skeleton area in the specified three-dimensional mesh model based on the parameter information. Since after the skinning operation, the specified three-dimensional mesh model image is bound to the skeleton information and the human body parts of the specified three-dimensional mesh model image can be driven by the skeleton, adjusting the target skeleton area will change the body features of the body parts corresponding to the target skeleton area, and then obtain the second image corresponding to the adjusted specified three-dimensional mesh model. Therefore, by determining the skeleton area of ​​the specified three-dimensional mesh model, the body features of the corresponding body parts can be changed, and then converted into an image, it is possible to achieve the change of the human body parts in the image to be processed.

[0065] See also Fig. 9 , Fig. 9 An image processing method provided in an embodiment of the present application is shown, and the method includes: S901 to S905.

[0066] S901: Acquire a first image to be processed.

[0067] S902: Obtain skeleton information and depth information corresponding to the human body image in the first image.

[0068] S903: Based on the depth information and the skeleton information, a specified three-dimensional mesh model corresponding to the human body image is obtained through a skinning operation.

[0069] It should be noted that the specific implementation of the aforementioned S901 to S903 can refer to the aforementioned embodiment and will not be repeated here.

[0070] S904: driving the target joint point region to drive the first skeleton region to move based on the rotation parameter, so as to change the human body posture of the specified three-dimensional mesh model.

[0071] In the embodiment of the present application, the operation of S904 can be called a skeleton driving operation, the aforementioned target skeleton region includes a target joint region and a first skeleton region connected to the target joint, and the aforementioned parameter information includes a rotation parameter. The rotation of the joint region can drive the movement or rotation of the skeleton region connected to the joint, such as Fig.10 As shown, the joint point or the bone area can rotate in the three-dimensional coordinate system O-XYZ corresponding to the three-dimensional mesh model, that is, it can move in the three planes of XOY plane, YOZ and XOZ. For the joint point, it can rotate in three dimensions in three dimensions, that is, rotate toward the three axes of X, Y, and Z. Then the aforementioned rotation parameters may include the rotation angle and the rotation direction. Based on the structural characteristics of the joint point and the bones connected to the joint point, it can be known that when the joint point rotates in different directions, it can drive the movement and rotation of the bone area connected to the joint point. Of course, it can also move and rotate at the same time. In addition, limited by the structural characteristics of the human skeleton, not every joint point has the function of rotating on the three axes of X, Y, and Z at the same time.

[0072] As an implementation method, the rotation parameter includes a rotation angle and a rotation direction, and the implementation method of S904 is to drive the target joint point area to rotate along the rotation direction based on the rotation angle to drive the first skeleton area to rotate. When the target joint point area can rotate in different directions, the target joint point area can drive the first skeleton area to rotate in different directions, thereby realizing the rotation of the first skeleton area with the target joint point area as the rotation center, or the rotation of the first skeleton area with the central axis passing through the target joint point.

[0073] When the specified three-dimensional mesh model is obtained, the direction in which each joint point in the model can rotate and the bone area that can be driven have been set accordingly. Fig.11 As shown, Fig.11The human skeleton shown includes a first joint point 1101 and a second joint point 1102, and a bone region 1103 connected between the first joint point 1101 and the second joint point 1102. For example, the target joint point region may be the first joint point 1101, and the first bone region may be the bone region 1103. That is, the target joint point region is the joint point at the elbow, and the first bone region is the forearm bone connected to the elbow joint point. Fig.11 In the corresponding embodiment, the forearm is controlled to rotate relative to the elbow joint point by the rotation of the elbow joint point to achieve the hand-raising gesture. Fig.11 As shown, the first joint point 1101 rotates along the first rotation direction F1, and Fig.10 In comparison, the rotation angle can be nearly 90°, for example, an angle between 85° and 95°, and the first rotation direction F1 is parallel to the XOY plane, that is, it rotates from the X-axis toward the Y-axis. In other words, the first joint point 1101 rotates along the first rotation direction F1, thereby driving the skeletal area 1103 to move, thereby driving the body part corresponding to the skeletal area 1103 (that is, the forearm) to move, and the human body posture is changed to a hand-raising posture.

[0074] See also Fig.12 ,exist Fig.12 In the example, the target joint point region is the second joint point 1102, and the first bone region may be the bone region 1104. That is, the target joint point region is the joint point at the wrist, and the first bone region is the hand bone connected to the wrist joint point. Fig.12 In a corresponding embodiment, the direction of the hand is changed by rotating the wrist joint. Fig.12 As shown, the second joint point 1102 rotates along the second rotation direction F2, and Fig.11 In comparison, the rotation angle can be nearly 90°, for example, an angle between 85° and 95°, and the second rotation direction F2 is parallel to the XOZ plane, that is, the second joint point 1102 rotates along the second rotation direction F2, thereby driving the skeletal area 1104 to rotate, thereby driving the body part corresponding to the skeletal area 1104 (i.e., the palm) to rotate, and the palm faces outward or inward.

[0075] S905: Acquire the adjusted second image corresponding to the specified three-dimensional mesh model.

[0076] Based on the above-mentioned adjustment method of the target skeleton area, the posture of the person in the photo containing the human body image collected by the electronic device can be adjusted, such as Fig.13 As shown, Fig.13 (a) is an image captured by an image capture device of the electronic device, i.e., a first image, Fig.13(b) is the image after the above-mentioned skeleton driving operation, that is, the second image. Fig.13 As can be seen from (a) and (b), by driving certain joints of the limbs in the specified 3D mesh model, the postures of the two human bodies in the image are changed to show a chasing posture. Fig.13 (a) and Fig.13 (b) It can be seen that after the human body posture in the first image is changed, the changed human body posture is fused with the background in the first image to obtain a second image including a new human body image and a background image. Therefore, even if the user's posture is not satisfactory when taking a picture, the human body posture in the image can be adjusted by the method provided in the embodiment of the present application, without having to repeatedly correct the posture and take pictures during shooting.

[0077] It should be noted that before performing the operation of adjusting the target bone region in the specified three-dimensional mesh model based on the parameter information, it is necessary to first determine the target bone region and the parameter information. Therefore, the implementation method of adjusting the target bone region in the specified three-dimensional mesh model based on the parameter information includes obtaining the target bone region to be adjusted and the parameter information corresponding to the target bone region; adjusting the target bone region based on the parameter information.

[0078] As an embodiment, the target bone region to be adjusted and the parameter information corresponding to the target bone region can be input by the user. For example, the user can input the target bone region and the parameter information corresponding to the target bone region through an application that can operate the specified three-dimensional mesh model. That is, the user opens the specified three-dimensional mesh model in the application and selects at least one joint point and / or bone region in the specified three-dimensional mesh model, so as to determine the target bone region. Then, the user can input parameters in a targeted manner as the parameter information corresponding to the target bone region. Of course, the user can also select the desired posture in the application. The application automatically determines the target bone region and the parameter information corresponding to the target bone region according to the desired posture selected by the user. Specifically, the parameters corresponding to the desired posture can be used as the target parameters, and the parameters corresponding to the current posture of the user in the specified three-dimensional mesh model can be used as the initial parameters. The difference between the target parameters and the initial parameters is used as the parameter information corresponding to the target bone region, and the difference includes at least one of the position difference and the angle difference between the target parameters and the initial parameters.

[0079] Of course, the user can also select the desired posture in the display interface corresponding to the first image, such as Fig.14As shown, a designated selection interface is displayed in the display interface of the first image, and the user can select a desired posture from at least one selected posture displayed in the designated selection interface, and then the electronic device can determine the target bone area corresponding to the desired posture and the parameter information corresponding to the target bone area, and the determined target bone area and parameter information can adjust the current posture in the first image to the desired posture. It should be noted that the difference between posture and posture is that posture includes position in addition to posture.

[0080] As another implementation, the scene information of the first image can also be analyzed, and the above-mentioned target bone region and parameter information corresponding to the target bone region can be determined based on the scene information. The scene information can include at least one of the scene types represented by the initial position of the human body image in the first image and the background image in the first image, wherein the initial position includes the position relationship and initial posture of the human body to be adjusted in the first image, etc. The human body to be adjusted can be all the human bodies in the first image, or the human body of the image acquisition device closest to the electronic device in the first image, or the human body with specific characteristics in the first image, wherein the specific characteristics can be at least one of specific character attributes and action attributes, wherein the character attributes refer to parameters related to the character identity such as height, gender, race, clothing color, hairstyle, hair color, etc., and the action attributes include types such as motion state and non-motion state. Exemplarily, since the first image is an image captured by an image capture device of an electronic device, usually when capturing the first image, the human body in the first image will first enter the image capture range of the image capture device, and the image capture device can obtain the image of the human body at this time. For example, when the camera application of the electronic device is shooting, the image captured by the image capture device is displayed in the preview interface of the camera. Therefore, the image data in the preview interface within a predetermined time period before the moment of capturing the first image can determine the motion attribute of each human body in the first image, that is, whether it is in a moving state or a non-moving state. For example, the motion attribute of each human body in the first image is determined by continuous multiple frames of images.

[0081] If the scene information can include the initial posture of the human body image in the first image, the initial posture can characterize the positional relationship between the various human bodies to be adjusted in the first image and the posture of each human body to be adjusted. Then, the initial posture is compared with multiple preset postures acquired in advance to determine the preset posture with the highest matching degree as the target posture. Since the posture of the human body is often a fixed posture under the target posture, by determining the difference between the target posture and the current initial posture, the part to be adjusted of the human body to be adjusted in the first image and the changes in body features corresponding to the part to be adjusted can be determined. With the above Fig.13 For example, assuming the initial posture is Fig.13The target pose is Fig.13 By comparing the target posture and the initial posture, it can be determined that at least the part to be adjusted includes Fig.13 The difference between the target posture and the initial posture of the limbs of the left human body and the right human body in (a) can determine the body feature changes corresponding to the part to be adjusted, and the body feature changes include at least one of the position and angle to be moved corresponding to the part to be adjusted, and then the part to be adjusted and the body feature changes corresponding to the part to be adjusted are determined to the target bone area and parameter information in the specified three-dimensional mesh model through the obtained specified three-dimensional mesh model.

[0082] It is understandable that through Fig.13 It can be seen that the positional relationship between each human body to be adjusted in the initial posture and the target posture is relatively close, and the postures of the two are also relatively close. This is because the difference between the initial posture and the target posture should not be too large, otherwise adjusting the initial posture to the target posture may make the user feel abrupt. Therefore, the preset posture with the highest matching degree determined by the aforementioned comparison operation is usually not much different from the initial posture in the first image, and may be more in line with the user's expectations. For example, the user may expect a certain posture, but did not pose correctly when shooting, but the basic posture may meet the expected posture, so that the embodiment of the present application can play a role in correcting the posture.

[0083] It should be noted that if the number of human bodies to be adjusted in the first image is more than 2 (that is, greater than or equal to 2), the initial posture includes the positional relationship between the human bodies to be adjusted and the posture of each human body to be adjusted. Then, the above-mentioned implementation method of comparing the initial posture with the multiple preset postures acquired in advance may include, from the multiple preset postures acquired in advance, determining the number of human bodies corresponding to each preset posture, taking the preset posture with more than 2 human bodies as the first posture to be selected, and then, matching the positional relationship between the human bodies to be adjusted in the first image with the positional relationship between the human bodies in the first posture to be selected to obtain a position matching result, matching the posture of the human bodies to be adjusted in the first image with the posture of each human body in the first posture to be selected to obtain a position and posture matching result, wherein the positional relationship may include orientation, etc. For example, user A is on the left side of user B, and the posture matching process will be matched according to the position relationship. For example, first, the first candidate posture with a position matching degree higher than the first threshold is determined as the candidate posture through the position matching result, and then, according to the position relationship between the various human bodies to be adjusted in the first image, the posture of each human body to be adjusted is matched with the posture of the corresponding human body in the candidate posture to obtain the posture matching result of each human body. For example, the position relationship between the various human bodies to be adjusted in the first image is that user A is on the left side of user B. Based on this position relationship, the matching object corresponding to each human body to be adjusted is determined from the alternative postures. For example, the human body on the left side of all the human bodies in the alternative postures is determined as the matching object of user A, and the human body on the right side of all the human bodies in the alternative postures is determined as the matching object of user B, and each corresponds to the matching. Then, based on the posture matching result of each human body, the posture matching result between the first image and the alternative posture is obtained. For example, the average value of the posture matching result of each human body is used as the posture matching result between the first image and the alternative posture. The specific details are not limited here. Then, the alternative posture corresponding to the highest posture matching result is used as the target posture corresponding to the first image.

[0084] If the number of human bodies to be adjusted in the first image is 1, the initial posture includes the posture of the human body to be adjusted, and the number of human bodies corresponding to each preset posture is determined from multiple preset postures acquired in advance, and the preset posture with 1 human body is taken as the second posture to be selected, and the posture matching result between the posture of the human body to be adjusted in the first image and the posture of the human body in the second posture to be selected is obtained, and then, the second posture to be selected corresponding to the highest posture matching result is taken as the target posture corresponding to the first image.

[0085] It should be noted that the scene information of the first image may include the scene type represented by the background image in the first image, and the implementation method of determining the above-mentioned target bone region and the parameter information corresponding to the target bone region based on the scene information is to determine the target scene type of the first image based on the background image in the first image, and find the target preset posture corresponding to the target scene type in the second corresponding relationship between the pre-set scene type and the preset posture as the target posture corresponding to the first image. For example, if the scene type of the first image is a basketball scene, then the preset posture corresponding to the basketball scene is found from the second corresponding relationship as the target posture corresponding to the first image. Then, according to the difference between the target posture and the initial posture of the person in the first image, the body feature change corresponding to the part to be adjusted can be determined, and the body feature change includes at least one of the position to be moved and the angle to be adjusted corresponding to the part to be adjusted, and then the part to be adjusted and the body feature change corresponding to the part to be adjusted are determined to the target bone region and parameter information in the specified three-dimensional mesh model through the obtained specified three-dimensional mesh model.

[0086] In addition, the scene information mentioned above may include the initial posture of the human body image in the first image and the scene type of the first image, that is, after acquiring the first image, the initial posture and scene type corresponding to the first image may be determined, and the target posture corresponding to the first image may be determined in combination with the scene type and the initial posture. Exemplarily, the aforementioned first corresponding relationship is pre-set, and all preset postures corresponding to the scene type of the first image are found in the first corresponding relationship as the first posture, and then the posture matching the initial posture is determined from the multiple first postures as the target posture, wherein the process of matching the first posture and the initial posture may refer to the aforementioned embodiment, and will not be repeated here. Exemplarily, assuming that the scene type of the first image is outdoor sports, all preset postures corresponding to the outdoor sports type are found in the first corresponding relationship as the first posture, and then the target posture matching the initial posture is determined from the first posture. Compared with obtaining the target posture by matching only the initial posture, the calculation amount and complexity can be reduced because the matching range is narrowed.

[0087] It should be noted that the above-mentioned implementation method of determining the target bone region and parameter information can be applied to any embodiment of the present application.

[0088] Therefore, the image processing method provided in the embodiment of the present application can adjust the posture of the human body in the first image, and unlike the processing method of the photo editing software directly on the first image, the embodiment of the present application changes the posture of the human body in the first image by driving the bones in the three-dimensional mesh model corresponding to the human body in the first image, which is more in line with the user's movement habits and the human body's skeletal structure, making the changed human body posture more realistic. Among them, the adjustment of the posture is not only the adjustment of a certain body part, but also the adjustment of the posture of the entire human body, such as Fig.15 As shown, firstly, the 3D Mesh reconstruction of the portrait is performed, that is, the first 3D mesh model is established, then the skeleton is bound, and then the skinning operation is performed, the Mesh area affected by the skeleton, that is, the target skeleton area, is set, and then the skeleton drives the Mesh grid to be changed, and finally UV mapping is performed to complete the Color image post-processing, so as to adjust the entire posture of the human body, as shown in FIG. Fig.15 As shown, the human body posture is changed.

[0089] See also Fig.16 , Fig.16 An image processing method provided in an embodiment of the present application is shown, and the method includes: S1601 to S1605.

[0090] S1601: Acquire a first image to be processed.

[0091] S1602: Obtain skeleton information and depth information corresponding to the human body image in the first image.

[0092] S1603: Based on the depth information and the skeleton information, a specified three-dimensional mesh model corresponding to the human body image is obtained through a skinning operation.

[0093] It should be noted that the implementation of the aforementioned S1601 to S1603 can refer to the aforementioned embodiments and will not be repeated here.

[0094] S1604: Adjust the second skeletal region in the specified three-dimensional mesh model based on the parameter information to adjust the size or shape of the human body part bound to the second skeletal region in the specified three-dimensional mesh model.

[0095] The image processing method provided in the embodiment of the present application can not only adjust the human body posture of the human body image in the first image, but also adjust the size and / or shape of a certain human body part in the human body image.

[0096] As an embodiment, the target bone region includes a second bone region, the parameter information includes a size parameter, and the size of the second bone region is changed based on the size parameter to adjust the size of the human body part bound to the second bone region in the specified three-dimensional mesh model. Since the specified three-dimensional mesh model is bound to the bone information of the human body image in the first image and is skinned, the change in the size of a certain bone will cause the size of the body part bound to the bone to also be changed. The size of the second bone region can be a three-dimensional size, that is, the size of the second bone region can be changed in the direction of the X, Y, and Z axes.

[0097] like Fig.17 As shown, Fig.17 (a) is the first image, Fig.17 (b) is the size adjustment of the second bone region in the specified three-dimensional mesh model corresponding to the human body image of the first image. It can be seen that the second bone region is the nasal bone region. By increasing the size of the nasal bone region in the X-axis direction, we can obtain Fig.17 The second image shown in (c) is that the width of the nose tip corresponding to the nasal bone area is widened by increasing the size of the nasal bone area in the X-axis direction.

[0098] like Fig.18 As shown, the second bone region is the mandibular region, Fig.18 (a) is the first image, Fig.18 (b) adjusting the size of the mandibular region in the specified three-dimensional mesh model corresponding to the human body image of the first image, combined with Fig.17 It can be seen that for Fig.18 (b) The Z-axis direction of the mandibular region can be adjusted, such as Fig.18 As shown in (c), the mandibular region is elongated, so that the chin corresponding to the mandibular region can be lengthened.

[0099] As another implementation, the shape of the human body part corresponding to a certain bone region can also be adjusted by rotating the bone region. Exemplarily, the target bone region includes a second bone region, the parameter information includes a rotation parameter, and the second bone region is controlled to rotate based on the rotation parameter to adjust the shape of the human body part bound to the second bone region in the specified three-dimensional mesh model. For example, the second bone region is a nose bridge region, and the nose bridge region is controlled to rotate toward the left or right side, thereby changing the shape of the human nose.

[0100] S1605: Acquire a second image corresponding to the adjusted specified three-dimensional mesh model.

[0101] Therefore, based on the aforementioned embodiments, not only can the posture of the human body be changed through skeleton driving, but the shape and / or size of certain parts of the human body can also be changed through the skeleton area.

[0102] See also Fig.19 , which shows a structural block diagram of an image processing device 1900 provided in an embodiment of the present application. The device may include: a first acquisition unit 1901, a second acquisition unit 1902, a binding unit 1903, an adjustment unit 1904 and a conversion unit 1905.

[0103] The first acquisition unit 1901 is used to acquire a first image to be processed.

[0104] The second acquisition unit 1902 is used to acquire the skeleton information and depth information corresponding to the human body image in the first image.

[0105] Furthermore, the second acquisition unit 1902 is also used to acquire a depth image corresponding to the first image; segment the first image to obtain the human body image; obtain depth information corresponding to the human body image from the depth image; and detect the human body image to obtain the skeleton information.

[0106] The skinning unit 1903 is used to obtain a specified three-dimensional mesh model corresponding to the human body image through a skinning operation based on the depth information and the skeleton information.

[0107] Furthermore, the first image is an image captured by an image acquisition device of the electronic device, and the skinning unit 1903 is also used to obtain a first three-dimensional mesh model corresponding to the human body image based on the human body image and the depth information and the camera parameters captured by the image acquisition device; bind the skeleton information to the first three-dimensional mesh model to obtain a second three-dimensional mesh model; and perform a skinning operation on the second three-dimensional mesh model to obtain the specified three-dimensional mesh model.

[0108] The adjusting unit 1904 is configured to adjust the target bone region in the specified three-dimensional mesh model based on the parameter information to change the body feature of the body part corresponding to the target bone region in the specified three-dimensional mesh model.

[0109] Furthermore, the target skeleton area includes a target joint point area and a first skeleton area connected to the target joint point, the parameter information includes a rotation parameter, and the adjustment unit 1904 is also used to drive the target joint point area to drive the first skeleton area to move based on the rotation parameter to change the human body posture of the specified three-dimensional mesh model.

[0110] Furthermore, the rotation parameter includes a rotation angle and a rotation direction, and the adjustment unit 1904 is further configured to drive the target joint point region to rotate along the rotation direction based on the rotation angle, so as to drive the first skeleton region to rotate.

[0111] Furthermore, the target bone region includes a second bone region, the parameter information includes a size parameter, and the adjustment unit 1904 is also used to change the size of the second bone region based on the size parameter to adjust the size of the human body part bound to the second bone region in the specified three-dimensional mesh model.

[0112] Furthermore, the target bone region includes a second bone region, the parameter information includes a rotation parameter, and the adjustment unit 1904 is also used to control the rotation of the second bone region based on the rotation parameter to adjust the shape of the human body part bound to the second bone region in the specified three-dimensional mesh model.

[0113] Furthermore, the adjustment unit 1904 is further configured to obtain a target bone region to be adjusted and parameter information corresponding to the target bone region; and adjust the target bone region based on the parameter information.

[0114] Furthermore, the adjustment unit 1904 is also used to identify scene information of the first image; based on the scene information, determine the part to be adjusted and the body feature changes corresponding to the part to be adjusted; based on the part to be adjusted, determine the target bone area in the specified three-dimensional mesh model; and determine parameter information based on the body feature changes.

[0115] The conversion unit 1905 is used to obtain the adjusted second image corresponding to the specified three-dimensional mesh model.

[0116] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0117] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0118] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0119] Please refer to Fig. 20, which shows a structural block diagram of an electronic device provided in an embodiment of the present application. The electronic device 100 may be an electronic device capable of running applications, such as a smart phone, a tablet computer, an e-book, etc. The electronic device 100 in the present application may include one or more of the following components: a processor 110, a memory 120, and one or more applications, wherein the one or more applications may be stored in the memory 120 and configured to be executed by one or more processors 110, and the one or more programs are configured to execute the method described in the aforementioned method embodiment.

[0120] The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes various functions and processes data of the electronic device 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Optionally, the processor 110 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 110 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 110, but may be implemented separately through a communication chip.

[0121] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). The memory 120 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device 100 during use.

[0122] Please refer to Fig.21, which shows a structural block diagram of a computer-readable medium provided in an embodiment of the present application. The computer-readable medium 2100 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0123] The computer readable medium 2100 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable medium 2100 includes a non-transitory computer-readable storage medium. The computer readable medium 2100 has storage space for program code 2110 that performs any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 2110 can be compressed, for example, in an appropriate form.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image processing method, characterized in that: Applied to electronic equipment, the method comprises: Acquire a first image to be processed; Acquire skeleton information and depth information corresponding to the human body image in the first image; Based on the depth information and the skeleton information, a specified three-dimensional mesh model corresponding to the human body image is obtained through a skinning operation; Adjusting the target bone region in the specified three-dimensional mesh model based on the parameter information to change the body characteristics of the body part corresponding to the target bone region in the specified three-dimensional mesh model; Acquire a second image corresponding to the adjusted specified three-dimensional mesh model.

2. The method according to claim 1, characterized in that The target bone region includes a target joint point region and a first bone region connected to the target joint point, the parameter information includes a rotation parameter, and adjusting the target bone region in the specified three-dimensional mesh model based on the parameter information includes: The target joint point region is driven to move the first skeleton region based on the rotation parameter to change the human body posture of the specified three-dimensional mesh model.

3. The method according to claim 2, characterized in that The rotation parameter includes a rotation angle and a rotation direction, and the driving the target joint point region to drive the first skeleton region to move based on the rotation parameter includes: The target joint point region is driven to rotate along the rotation direction based on the rotation angle, so as to drive the first skeleton region to rotate.

4. The method according to claim 1, characterized in that: The target bone region includes a second bone region, the parameter information includes a size parameter, and adjusting the target bone region in the specified three-dimensional mesh model based on the parameter information includes: The size of the second skeletal region is changed based on the size parameter to adjust the size of the human body part bound to the second skeletal region in the specified three-dimensional mesh model.

5. The method according to claim 1, characterized in that The target bone region includes a second bone region, the parameter information includes a rotation parameter, and adjusting the target bone region in the specified three-dimensional mesh model based on the parameter information includes: The second skeletal region is controlled to rotate based on the rotation parameter to adjust the shape of the human body part bound to the second skeletal region in the specified three-dimensional mesh model.

6. The method according to any one of claims 1 to 5, characterized in that: The first image is an image acquired by the image acquisition device of the electronic device, and the step of obtaining a specified three-dimensional mesh model corresponding to the human body image through a skinning operation based on the depth information and the skeleton information includes: Based on the human body image and the depth information and the camera parameters acquired by the image acquisition device, a first three-dimensional mesh model corresponding to the human body image is obtained; Binding the skeleton information to the first three-dimensional mesh model to obtain a second three-dimensional mesh model; A skinning operation is performed on the second three-dimensional mesh model to obtain the specified three-dimensional mesh model.

7. The method according to any one of claims 1 to 5, characterized in that: The acquiring of skeleton information and depth information corresponding to the human body image in the first image includes: Acquire a depth image corresponding to the first image; Segmenting the first image to obtain the human body image; Obtaining depth information corresponding to the human body image from the depth image; The human body image is detected to obtain the skeleton information.

8. The method according to any one of claims 1 to 5, characterized in that: The step of adjusting the target bone region in the specified three-dimensional mesh model based on the parameter information includes: Acquire a target bone region to be adjusted and parameter information corresponding to the target bone region; The target bone region is adjusted based on the parameter information.

9. The method according to claim 8, characterized in that The obtaining of the target bone region to be adjusted and parameter information corresponding to the target bone region includes: identifying scene information of the first image; Based on the scene information, determining the part to be adjusted and the change of body features corresponding to the part to be adjusted; Determine a target bone region in the specified three-dimensional mesh model based on the part to be adjusted; Parameter information is determined based on the changes in the body characteristics.

10. An image processing device, characterized in that: Applied to electronic equipment, the device comprises: A first acquisition unit, used for acquiring a first image to be processed; A second acquisition unit, used to acquire skeleton information and depth information corresponding to the human body image in the first image; A skinning unit, configured to obtain a specified three-dimensional mesh model corresponding to the human body image through a skinning operation based on the depth information and the skeleton information; an adjusting unit, configured to adjust the target bone region in the specified three-dimensional mesh model based on the parameter information, so as to change the body feature of the body part corresponding to the target bone region in the specified three-dimensional mesh model; A conversion unit is used to obtain an adjusted second image corresponding to the specified three-dimensional grid model.

11. An electronic device, characterized in that: include: one or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to execute the method according to any one of claims 1-9.

12. A computer readable medium, characterized in that The computer-readable medium stores a program code executable by a processor, and when the program code is executed by the processor, the processor executes the method according to any one of claims 1 to 9.