Image processing method and apparatus

Abstract

The application discloses an image processing method and device, and belongs to the technical field of image processing. The method comprises the following steps: performing human body skeleton feature point extraction on a portrait image in a target image to obtain human body skeleton feature point information corresponding to the portrait image; constructing a three-dimensional portrait image corresponding to the portrait image based on the human body skeleton feature point information; and superimposedly displaying the three-dimensional portrait image and the target image.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to an image processing method and device. BACKGROUND

[0002] With the rapid development of image technology, more and more users can conveniently download or take portrait pictures.

[0003] In the related art, when a user views the portrait picture, the user often views the picture by printing the picture or displays the picture on a screen, but the portrait picture viewed in this way is not vivid enough. SUMMARY

[0004] Embodiments of the present application provide an image processing method and device, which can solve the problem that a portrait picture is not vivid enough.

[0005] In a first aspect, an image processing method is provided, and the method comprises the following steps:

[0006] extracting human body skeleton feature points of a portrait image in a target image to obtain human body skeleton feature point information corresponding to the portrait image;

[0007] constructing a three-dimensional portrait image corresponding to the portrait image based on the human body skeleton feature point information;

[0008] superimposedly displaying the three-dimensional portrait image and the target image.

[0009] In a second aspect, an image processing device is provided, and the device comprises the following modules:

[0010] an extraction module configured to extract human body skeleton feature points of a portrait image in a target image to obtain human body skeleton feature point information corresponding to the portrait image;

[0011] a construction module configured to construct a three-dimensional portrait image corresponding to the portrait image based on the human body skeleton feature point information;

[0012] a display module configured to superimposedly display the three-dimensional portrait image and the target image.

[0013] In a third aspect, an electronic device is provided, which comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and the program or instruction is executed by the processor to implement the steps of the method according to the first aspect.

[0014] In a fourth aspect, an embodiment of the present application provides a readable storage medium, the readable storage medium storing a program or instructions, the program or instructions being executed by a processor to implement the steps of the method according to the first aspect.

[0015] In a fifth aspect, an embodiment of the present application provides a chip, the chip comprising a processor and a communication interface, the communication interface being coupled to the processor, and the processor being configured to execute a program or instructions to implement the method according to the first aspect.

[0016] In the embodiment of the present application, the human body skeleton feature points of the portrait image in the target image are extracted, and the three-dimensional portrait image corresponding to the portrait image is constructed based on the human body skeleton feature point information, so that a three-dimensional portrait image more consistent with the characteristics of the portrait image can be constructed, and the three-dimensional portrait image constructed based on the human body skeleton feature point information can better fit the target image for display. In the embodiment of the present application, the three-dimensional portrait image corresponding to the portrait image in the target image is constructed and superimposed and displayed, so that a more lifelike and more dynamic portrait image display effect is realized. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 A flowchart of an image processing method is provided for the embodiment of the present application.

[0018] Figure 2 A human body skeleton action feature point information diagram corresponding to the action control instruction is provided for the embodiment of the present application.

[0019] Figure 3 A human body skeleton feature point extraction diagram is provided for the embodiment of the present application.

[0020] Figure 4 A structure diagram of an image processing device is provided for the embodiment of the present application.

[0021] Figure 5 A structure diagram of an electronic device is provided for the embodiment of the present application.

[0022] Figure 6 A hardware structure diagram of an electronic device is provided for the embodiment of the present application. DETAILED DESCRIPTION

[0023] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art belong to the scope of protection of the present application.

[0024] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally a class, not limited to the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the objects before and after are in an "or" relationship.

[0025] The image processing method and device, electronic device and storage medium provided by the embodiments of the present application will be described in detail below in conjunction with the drawings, specific embodiments and application scenarios.

[0026] Figure 1 The flowchart of the image processing method provided by the embodiments of the present application is shown as Figure 1 indicated, including:

[0027] Step 110, performing human body skeleton feature point extraction on the portrait image in the target image to obtain human body skeleton feature point information corresponding to the portrait image;

[0028] Specifically, the target image described in the embodiments of the present application is an image containing a portrait image, specifically an image of a target object, or a certain video frame in a video of a target object, which can be specifically photographed by a user through an electronic device corresponding thereto, or photographed by a merchant through a specific electronic device. The electronic device can be a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook or a personal digital assistant (PDA), etc. The non-mobile electronic device can be a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine or a self-service machine, etc. The embodiments of the present application are not limited specifically.

[0029] It can be understood that the target object described in the embodiments of the present application can be specifically a promotional image of a merchant, for example, a human-shaped stand for promoting products, a printed photo or a portrait image in a display screen, or a specific person.

[0030] Ideally, the target image described in the embodiments of the present application should be taken in the front view direction of the target object. If the angle of shooting is tilted, the image is adjusted in angle to obtain the image in the front view, i.e. the target image.

[0031] The adjustment method can be based on the position point information of the target object, and perspective correction is performed based on the position information to obtain the target image without depth difference.

[0032] Alternatively, the adjustment method can also be to correct the target object according to a depth correction model to obtain the target image without depth difference. The depth correction model is a model with angle correction function, and specifically can be a model pre-trained by a deep learning network.

[0033] The portrait image described in the embodiments of the present application is obtained by analyzing the target image based on a portrait recognition method, i.e. after the portrait part in the target image is extracted, the portrait image in the target image is obtained. The specific portrait recognition method can be a portrait recognition neural network or other portrait recognition methods, which are not limited in the embodiments of the present application.

[0034] The human body skeletal feature point described in the embodiments of the present application can represent the position of the human body feature point in the image, and can also effectively represent the human body action and posture.

[0035] The extraction of the human body skeletal feature point described in the embodiments of the present application can be realized by a human body feature point recognition model capable of recognizing human body skeletal feature points. After the portrait image is input into the human body feature point recognition model, the corresponding human body skeletal feature point information can be obtained.

[0036] Since the portrait image in the target object has corresponding action and posture, the human body skeletal feature point information of the portrait image can effectively help to generate a portrait three-dimensional image with the same action and posture as the portrait.

[0037] Step 120, constructing a portrait three-dimensional image corresponding to the portrait image based on the human body skeletal feature point information;

[0038] Specifically, after the human body skeletal feature point information is determined, the embodiments of the present application also acquire the portrait three-dimensional material corresponding to the portrait image, perform portrait three-dimensional image rendering based on the human body skeletal feature point information, and finally obtain a portrait three-dimensional image with the same action and posture as the portrait image.

[0039] Step 130, superimposedly displaying the portrait three-dimensional image and the target image.

[0040] Specifically, in the embodiments of the present application, the display can be on the electronic device of the user, or on the electronic device of the merchant for publicity with the target object.

[0041] In the embodiments of the present application, the human body skeletal feature point information of the portrait image is identified, and the portrait three-dimensional image is constructed based on the human body skeletal feature point information of the portrait image, that is, the portrait three-dimensional image and the portrait image have the same human body skeletal feature point information.

[0042] The portrait image is in the target image, so when the portrait three-dimensional image and the target image are superimposed and displayed, the feature points in the human body skeletal feature point information of the portrait three-dimensional image can be adjusted to the positions of the feature points in the human body skeletal feature point information of the portrait image, so that the portrait three-dimensional image is displayed on the portrait image, that is, the superimposed display of the portrait three-dimensional image and the target image is completed.

[0043] Optionally, after the three-dimensional coordinate system of the portrait three-dimensional image is identified, other background images in the target image except the portrait image can be further obtained, and the line features and the depth of field features in the other background images are analyzed to construct a target three-dimensional coordinate system corresponding to the target image.

[0044] After the origin of the portrait coordinate system corresponding to the portrait three-dimensional image is aligned with the target three-dimensional coordinate system, the portrait three-dimensional image can be superimposed and displayed in the target image, and after the alignment of the coordinate systems is completed, the display position of the portrait three-dimensional image can be further adjusted.

[0045] In the embodiments of the present application, the human body skeletal feature points of the portrait image in the target image are extracted, and the portrait three-dimensional image corresponding to the portrait image is constructed based on the human body skeletal feature point information, so that a portrait three-dimensional image more consistent with the features of the portrait image can be constructed, and the portrait three-dimensional image constructed based on the human body skeletal feature point information can be better displayed on the target image. In the embodiments of the present application, the portrait three-dimensional image corresponding to the portrait image in the target image is constructed and superimposed and displayed, so that a more lifelike and more dynamic portrait image display effect is realized.

[0046] Optionally, the human body skeletal feature points of the portrait image in the target image are extracted, and the first human body skeletal feature point information corresponding to the portrait image is obtained, including:

[0047] The portrait image is analyzed based on the human feature point recognition model to obtain initial human body skeletal feature point information;

[0048] The initial human body skeletal feature point information is corrected based on the reference human body skeletal feature point information to obtain the human body skeletal feature point information corresponding to the portrait image.

[0049] Specifically, the human feature point recognition model described in the embodiments of the present application can be a model capable of realizing the human feature point recognition function of a portrait image, which can be a neural network model pre-trained, such as a high-resolution network model or a convolutional neural network model.

[0050] The human feature point recognition model described in the embodiments of the present application can be obtained according to a portrait sample image carrying human skeleton feature point labels.

[0051] After inputting the portrait image into the human feature point recognition model in the embodiments of the present application, initial human skeleton feature point information recognized by the model can be obtained. However, the image content of the portrait image is often complex, and the recognition result of the model can also have certain errors, so the initial human skeleton feature point information can not be accurate enough. Therefore, the embodiments of the present application introduce reference human skeleton feature point information to correct it.

[0052] The reference human skeleton feature point information described in the embodiments of the present application can be human skeleton feature point information generated by pre-referencing a medical human skeleton model, which can be used as a standard for human skeleton feature point information.

[0053] In the process of correcting the initial human skeleton feature point information based on the reference human skeleton feature point information, the number of human feature points in the reference human skeleton feature point information can be used as a basis to adjust the initial human skeleton key points.

[0054] In the embodiments of the present application, the human feature point recognition model can effectively recognize the initial human skeleton feature point information corresponding to the portrait image, and the reference human skeleton feature point information can effectively improve the accuracy of human skeleton feature point information recognition by correcting the initial human skeleton feature point information.

[0055] Optionally, the constructing the portrait three-dimensional image corresponding to the portrait image based on the human skeleton feature point information comprises:

[0056] performing face recognition on the portrait image to obtain human face image information corresponding to the portrait image;

[0057] constructing an initial portrait three-dimensional image based on the human face image information corresponding to the portrait three-dimensional material and the human skeleton feature point information;

[0058] performing face rendering on the initial portrait three-dimensional image based on the human face image information to obtain the portrait three-dimensional image.

[0059] The face recognition of the portrait image is specifically implemented by a face recognition algorithm or a face recognition model, and the embodiments of the present application do not limit this.

[0060] In the embodiments of the present application, after face recognition of the portrait image, a face region image corresponding to the portrait image is obtained, which can be used as face image information corresponding to the portrait image.

[0061] Optionally, since the portrait image is often of a well-known person, face recognition can be performed on the face region image to confirm the name information corresponding to the face image, and then the face image corresponding to the name information is obtained from the Internet, and the obtained face image is used as the face image information corresponding to the portrait image.

[0062] In the embodiments of the present application, the process of obtaining the face image from the Internet can specifically be selecting the image according to the recommendation of a search engine or the click rate of the face image corresponding to the name information.

[0063] It can be understood that since different people have different body proportions, for example, some people may have a relatively robust physique, and some people have a relatively small physique, in order to better construct a portrait three-dimensional image close to a real person, the embodiments of the present application need to obtain different portrait three-dimensional materials according to different people.

[0064] In the embodiments of the present application, since the number of brand endorsers is often limited, the portrait three-dimensional materials corresponding to each endorser can be pre-stored, and a set of general portrait three-dimensional materials is also stored for use when the corresponding endorser is not matched.

[0065] In the embodiments of the present application, in the case where the face image information is a face region image corresponding to the portrait image, the face image information can be matched with the face image of each endorser, and the corresponding portrait three-dimensional material is obtained according to the matching result.

[0066] In the case where the face image information is name information corresponding to the face image, the corresponding portrait three-dimensional material can also be obtained based on the name information corresponding to the face image information.

[0067] In the case where the face image information recognition fails, that is, the corresponding portrait three-dimensional material is not found, the general portrait three-dimensional material is used at this time.

[0068] The portrait three-dimensional material can specifically include pre-recorded skeleton animation material, costume animation material, hairstyle animation material, etc., and each portrait three-dimensional material corresponds to human body skeleton feature point information, that is, the position of each portrait three-dimensional material rendering is associated with the human body skeleton feature point information.

[0069] In the embodiments of the present application, the face region animation material, the skeleton animation material, the costume animation material, the hairstyle animation material, etc. can be added to the corresponding human skeleton feature point information, so as to construct the initial portrait three-dimensional image.

[0070] However, whether the rendering of the face part is realistic or not has a decisive significance for the final effect of the portrait three-dimensional image. Therefore, in the embodiments of the present application, after the initial portrait three-dimensional image is constructed, the face region is further modeled and rendered, that is, the face image information can be fitted to the face region of the initial portrait three-dimensional image, so as to obtain the face texture of the portrait three-dimensional image after the face fitting, which is more detailed.

[0071] In the embodiments of the present application, the face recognition is performed on the portrait image, so as to determine the matching portrait three-dimensional material according to the face recognition result, and then generate the initial portrait three-dimensional image which is more consistent with the characteristics of the corresponding character of the portrait image. Finally, the face rendering is performed on the initial portrait three-dimensional image by the face image information, and finally the portrait three-dimensional image with detailed face texture is obtained. The character three-dimensional image is closer to the real character, more vivid and lifelike, and can bring better empathy and experience to the user.

[0072] Optionally, after the portrait three-dimensional image and the target image are superimposed and displayed, the method further includes:

[0073] An action control instruction is acquired.

[0074] Human skeleton action feature point information corresponding to the action control instruction is determined.

[0075] The human skeleton feature point information corresponding to the portrait three-dimensional image is processed based on the human skeleton action feature point information, so as to obtain the portrait three-dimensional image after action adjustment.

[0076] Specifically, the action control instruction described in the embodiments of the present application can be a pre-set action instruction, such as a hand waving, jumping, hugging, giving flowers or dancing action instruction, and the pre-set action control instruction corresponds to one or more human skeleton action feature point information.

[0077] The action control instruction described in the embodiments of the present application can also be an action control instruction generated based on the captured user action by the camera or the motion capture sensor, which recognizes the human skeleton action feature point information corresponding to each action of the user. That is, the portrait three-dimensional image can follow the user in real time and synchronize the action at this time. The action capture sensor or the camera can be installed at the target object, which can communicate with the display device to synchronize the action control instruction. The action capture sensor or the camera can also be set in the electronic device of the user.

[0078] In the embodiment of the present application, since the human body skeleton feature point information can represent human action and posture, it can be understood that the human body skeleton feature points corresponding to the portrait three-dimensional image can be controlled, so as to control various actions. Specifically, different human body skeleton action feature point coordinates can be set, and a series of continuous human body skeleton feature point changes can constitute a group of actions.

[0079] Therefore, in the embodiment of the present application, each action control instruction corresponds to a group of human body skeleton action feature point information, which can include one or more human body skeleton action feature point coordinates. Figure 2 The human body skeleton action feature point information corresponding to the action control instruction provided in the embodiment of the present application is shown in FIG. 2. As shown in FIG. 2, the dance action can correspond to three human body skeleton action feature point information. After adjusting the human body skeleton feature points to the positions corresponding to the human body skeleton feature point information, the dance action of the portrait three-dimensional image can be controlled. Figure 2

[0080] When the user clicks or touches the screen, an interactive action can be prompted, for example, a display interface for selecting an action control instruction is provided, and in the interface, the user can select an action that the portrait three-dimensional image is expected to perform.

[0081] When the preset action control instruction is executed, the key point coordinates in the human body skeleton feature point information corresponding to the portrait three-dimensional image are adjusted according to the key point coordinates in the human body skeleton action feature point information corresponding to the action control instruction, so as to control the portrait three-dimensional image to perform the action.

[0082] When the action control instruction generated by capturing the user action is executed, the coordinates of the feature points in the human body skeleton feature point information corresponding to the portrait three-dimensional image are adjusted to the coordinates of the action feature points in the human body skeleton action feature point information corresponding to the user action, so as to realize the action tracking of the portrait three-dimensional image to the display real user, for example, when the user stretches out his hand, the portrait three-dimensional image also makes the action of stretching out his hand.

[0083] In the embodiment of the present application, after the human body skeleton action feature point information corresponding to the action control instruction is obtained, the coordinates of each feature point in the human body skeleton feature point information corresponding to the portrait three-dimensional image are adjusted to the coordinates of the action feature points in the human body skeleton action feature point information, so as to control the portrait three-dimensional image to make various actions, so that the portrait three-dimensional image is more lively, and the portrait three-dimensional image can also track and imitate the action of the user, which can bring stronger interaction and enhance the interest.

[0084] ​Optionally, the initial human body skeleton feature point information includes M initial human body skeleton feature points, the reference human body skeleton feature point information includes N reference human body skeleton feature points, and the initial human body skeleton feature point information is corrected based on the reference human body skeleton feature point information to obtain human body skeleton feature point information corresponding to the portrait image, including:

[0085] In a case where M is greater than N, in the initial human body skeleton feature point information, the initial human body skeleton feature points that do not match the reference human body skeleton feature points are removed to obtain the human body skeleton feature point information corresponding to the portrait image.

[0086] Or, in a case where M is less than or equal to N, the initial human body skeleton feature point information is taken as the human body skeleton feature point information corresponding to the portrait image, where M and N are positive integers.

[0087] Specifically, in a case where the number M of initial human body feature points in the initial human body skeleton feature point information is less than or equal to the number N of reference human body feature points in the reference human body skeleton feature point information, it is indicated that the recognition result has a relatively high reliability, and the initial human body skeleton feature point information is directly taken as the human body skeleton feature point information corresponding to the portrait image without modification.

[0088] In a case where the number M of initial human body feature points in the initial human body skeleton feature point information is greater than the number N of reference human body feature points in the reference human body skeleton feature point information, it is indicated that there are misrecognized human body feature points in the initial human body skeleton feature point information, and the human body feature points that do not correspond to the reference human body skeleton feature point information in the initial human body skeleton feature point information are found and merged with adjacent human body feature points or directly deleted to finally obtain the human body skeleton feature point information corresponding to the portrait image.

[0089] In the embodiment of the present application, the initial human body skeleton feature point information extracted by the algorithm model is further corrected by taking the reference human body skeleton feature point information with standard human body skeleton feature point information as a standard, so as to ensure the accuracy of the human body skeleton feature point information corresponding to the portrait image.

[0090] Optionally, the scheme of the present application is further described by taking a human figure card as an example in the embodiment of the present application, that is, the target image in the embodiment of the present application is an image of a portrait card, and the portrait image is an image of a human figure card region.

[0091] In the embodiment of the present application, the human figure card region image in the target image is subjected to cutout processing to obtain a portrait image in the target image, Figure 3 The human body skeleton feature point extraction schematic diagram provided in the embodiment of the present application is as follows, Figure 3As shown, the human body skeleton feature point extraction is performed on the portrait image in the target image to obtain human body skeleton feature point information corresponding to the human figure region image, the portrait three-dimensional image is constructed based on the human body skeleton feature point information, and then the portrait three-dimensional image is superimposed and displayed with the target image.

[0092] As an optional embodiment, a rendering background can also be added in the target image described in the embodiments of the present application, and the portrait three-dimensional image is superimposed and displayed with the rendering background.

[0093] In the embodiments of the present application, the human body skeleton feature point extraction is performed on the portrait image in the target image, and the portrait three-dimensional image corresponding to the portrait image is constructed based on the human body skeleton feature point information, so that a portrait three-dimensional image more in line with the characteristics of the portrait image can be constructed, and the portrait three-dimensional image constructed based on the human body skeleton feature point information can better fit the target image for display. The embodiments of the present application construct the corresponding portrait three-dimensional image based on the portrait image in the target image, and superimpose and display, so as to realize a more lively and dynamic portrait image display effect.

[0094] It should be noted that the image processing method provided in the embodiments of the present application can be executed by an image processing device or a control module in the image processing device for executing the image processing method. The image processing device provided in the embodiments of the present application is described by taking the image processing device executing the image processing method as an example.

[0095] Figure 4 The structure schematic diagram of the image processing device provided in the embodiments of the present application is shown in Figure 4 The structure schematic diagram of the image processing device provided in the embodiments of the present application is shown in

[0096] Optionally, the extraction module is specifically configured to:

[0097] The portrait image is analyzed based on the human feature point recognition model to obtain initial human body skeleton feature point information;

[0098] The initial human body skeleton feature point information is corrected based on the reference human body skeleton feature point information to obtain human body skeleton feature point information corresponding to the portrait image.

[0099] Optionally, the construction module is specifically configured to:

[0100] performing face recognition on the portrait image to obtain face image information corresponding to the portrait image;

[0101] constructing an initial portrait three-dimensional image based on the portrait three-dimensional material corresponding to the face image information and the human skeleton feature point information;

[0102] performing face rendering on the initial portrait three-dimensional image based on the face image information to obtain the portrait three-dimensional image.

[0103] Optionally, the device further comprises:

[0104] an acquisition module configured to acquire an action control instruction;

[0105] a determination module configured to determine human skeleton action feature point information corresponding to the action control instruction;

[0106] an adjustment module configured to process human skeleton feature point information corresponding to the portrait three-dimensional image based on the human skeleton action feature point information to obtain a portrait three-dimensional image after action adjustment.

[0107] Optionally, the initial human skeleton feature point information includes M initial human skeleton feature points, and the reference human skeleton feature point information includes N reference human skeleton feature points, and the extraction module is specifically configured to:

[0108] in a case where M is greater than N, remove, from the initial human skeleton feature point information, initial human skeleton feature points that do not match the reference human skeleton feature points to obtain human skeleton feature point information corresponding to the portrait image;

[0109] or, in a case where M is less than or equal to N, take the initial human skeleton feature point information as the human skeleton feature point information corresponding to the portrait image, where M and N are positive integers.

[0110] In the embodiments of the present application, human skeleton feature points are extracted from a portrait image in a target image, and a portrait three-dimensional image corresponding to the portrait image is constructed based on the human skeleton feature point information, so that a portrait three-dimensional image that is more consistent with the features of the portrait image can be constructed, and the portrait three-dimensional image constructed based on the human skeleton feature point information can better fit the target image for display. The embodiments of the present application construct a portrait three-dimensional image corresponding to the portrait image in the target image and superimpose display, so as to achieve a more lifelike and more dynamic portrait image display effect.

[0111] The image processing device in this application embodiment can be a device, or a component, integrated circuit, or chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0112] The image processing device in this application embodiment can be a device with an operating system. The operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit the specific operating system.

[0113] The image processing apparatus provided in this application embodiment can achieve... Figures 1 to 3 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0114] Optionally, Figure 5 This is a schematic diagram of the electronic device structure provided in the embodiments of this application, such as... Figure 5 As shown, this application embodiment also provides an electronic device 500, including a processor 501, a memory 502, and a program or instructions stored in the memory 502 and executable on the processor 501. When the program or instructions are executed by the processor 501, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0115] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0116] Figure 6 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0117] The electronic device 600 includes, but is not limited to, components such as: radio frequency unit 601, network module 602, audio output unit 603, input unit 604, sensor 605, display unit 606, user input unit 607, interface unit 608, memory 609, and processor 610.

[0118] Those skilled in the art can understand that the electronic device 600 can also include a power supply (such as a battery) for supplying power to each component, and the power supply can be logically connected to the processor 610 through a power management system, so that the power management system can realize functions such as management of charging, discharging, and power consumption management. Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and the electronic device can include more or fewer components than the figure, or combine certain components, or different component arrangements, which are not described here.

[0119] The processor 610 is configured to perform human body skeleton feature point extraction on a portrait image in a target image to obtain human body skeleton feature point information corresponding to the portrait image.

[0120] The processor 610 is configured to construct a portrait three-dimensional image corresponding to the portrait image based on the human body skeleton feature point information.

[0121] The display unit 606 is configured to superimely display the portrait three-dimensional image and the target image.

[0122] Optionally, the processor 610 is configured to analyze the portrait image based on a human feature point recognition model to obtain initial human body skeleton feature point information.

[0123] The processor 610 is configured to correct the initial human body skeleton feature point information based on the reference human body skeleton feature point information to obtain the human body skeleton feature point information corresponding to the portrait image.

[0124] Optionally, the processor 610 is configured to perform face recognition on the portrait image to obtain face image information corresponding to the portrait image.

[0125] The processor 610 is configured to construct an initial portrait three-dimensional image based on the human body skeleton feature point information and portrait three-dimensional material corresponding to the face image information.

[0126] The processor 610 is configured to perform face rendering on the initial portrait three-dimensional image based on the face image information to obtain the portrait three-dimensional image.

[0127] Optionally, the user input unit 607 is configured to obtain a motion control instruction.

[0128] The processor 610 is configured to determine human body skeleton motion feature point information corresponding to the motion control instruction.

[0129] The processor 610 is configured to process the human body skeleton feature point information corresponding to the portrait three-dimensional image based on the human body skeleton motion feature point information to obtain a portrait three-dimensional image after motion adjustment.

[0130] Optionally, the processor 610 is configured to, in the case that M is greater than N, remove, from the initial human body skeletal feature point information, initial human body skeletal feature points that do not match the reference human body skeletal feature points, to obtain human body skeletal feature point information corresponding to the portrait image.

[0131] Or, in the case that M is less than or equal to N, the initial human body skeletal feature point information is taken as the human body skeletal feature point information corresponding to the portrait image, where M and N are both positive integers.

[0132] In the embodiments of the present application, the human body skeletal feature points of the portrait image in the target image are extracted, and a portrait three-dimensional image corresponding to the portrait image is constructed based on the human body skeletal feature point information, so that a portrait three-dimensional image that is more in line with the characteristics of the portrait image can be constructed, and the portrait three-dimensional image constructed based on the human body skeletal feature point information can better fit the target image for display. In the embodiments of the present application, a portrait three-dimensional image corresponding to the portrait image in the target image is constructed and superimposed and displayed, so that a more lifelike and more dynamic portrait image display effect is achieved.

[0133] It should be understood that, in the embodiments of the present application, the input unit 604 can include a graphics processing unit (GPU) 6041 and a microphone 6042. The graphics processing unit 6041 processes image data of a still picture or a video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 606 can include a display panel 6061, which can be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 607 includes a touch panel 6071 and other input devices 6072. The touch panel 6071 is also called a touch screen. The touch panel 6071 can include a touch detection device and a touch controller. The other input devices 6072 can include, but are not limited to, a physical keyboard, function keys (such as volume control keys, on-off keys, etc.), a trackball, a mouse, a joystick, etc., which will not be described here. The memory 609 can be used to store software programs and various data, including but not limited to application programs and an operating system. The processor 610 can integrate an application processor and a modem processor, where the application processor mainly processes the operating system, user interface and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 610.

[0134] The embodiments of the present application also provide a readable storage medium having a program or instructions stored thereon, which, when executed by a processor, implements various processes of the above-mentioned image processing method embodiments and achieves the same technical effects. To avoid repetition, details will not be described here.

[0135] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0136] The chip provided in the embodiments of the present application includes a processor and a communication interface. The communication interface is coupled with the processor. The processor is used to run programs or instructions, to realize the processes of the above image processing method embodiments, and to achieve the same technical effects. To avoid repetition, details are not described herein.

[0137] It should be understood that the chip mentioned in the embodiments of the present application can also be referred to as a system-level chip, a system chip, a chip system, or a system-on-chip, etc.

[0138] It should be noted that in this document, the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such a process, method, article, or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article, or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, but can also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method can be performed in an order different from that described, and various steps can also be added, omitted, or combined. In addition, the features described with reference to certain examples can be combined in other examples.

[0139] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions for making a terminal (which can be a mobile phone, computer, server, or network equipment, etc.) execute the methods described in various embodiments of the present application.

[0140] The embodiments of the present application are described above with reference to the accompanying drawings, but the present application is not limited to the specific embodiments described above, and the specific embodiments described above are merely illustrative, but not restrictive, and a person of ordinary skill in the art can make many forms under the inspiration of the present application without departing from the purpose of the present application and the scope protected by the claims.

Claims

1. An image processing method, characterized by, The method comprises the following steps: extracting human body skeleton feature points of a portrait image in a target image to obtain human body skeleton feature point information corresponding to the portrait image; constructing a three-dimensional portrait image corresponding to the portrait image based on the human body skeleton feature point information; superimposedly displaying the three-dimensional portrait image and the target image; wherein, the step of constructing the three-dimensional portrait image corresponding to the portrait image based on the human body skeleton feature point information comprises: performing face recognition on the portrait image to obtain face image information corresponding to the portrait image; constructing an initial three-dimensional portrait image based on the face image information corresponding three-dimensional portrait material and the human body skeleton feature point information; performing face rendering on the initial three-dimensional portrait image based on the face image information to obtain the three-dimensional portrait image; wherein, the step of performing face recognition on the portrait image to obtain face image information corresponding to the portrait image comprises: performing face recognition on the portrait image to confirm name information corresponding to the portrait image, obtaining a face image corresponding to the name information from the Internet, and taking the obtained face image as the face image information corresponding to the portrait image; wherein, the face image corresponding to the name information is selected according to user click rate.

2. The image processing method of claim 1, wherein, The step of extracting human body skeleton feature points of a portrait image in a target image to obtain first human body skeleton feature point information corresponding to the portrait image comprises: analyzing the portrait image based on a human body feature point recognition model to obtain initial human body skeleton feature point information; correcting the initial human body skeleton feature point information based on reference human body skeleton feature point information to obtain human body skeleton feature point information corresponding to the portrait image.

3. The image processing method of claim 1, wherein, After the step of superimposedly displaying the three-dimensional portrait image and the target image, the method further comprises: obtaining an action control instruction; determining human body skeleton action feature point information corresponding to the action control instruction; processing human body skeleton feature point information corresponding to the three-dimensional portrait image based on the human body skeleton action feature point information to obtain a three-dimensional portrait image after action adjustment.

4. The image processing method of claim 2, wherein, The initial human body skeleton feature point information comprises M initial human body skeleton feature points, the reference human body skeleton feature point information comprises N reference human body skeleton feature points, and the step of correcting the initial human body skeleton feature point information based on the reference human body skeleton feature point information to obtain human body skeleton feature point information corresponding to the portrait image comprises: in the case that M is greater than N, removing initial human body skeleton feature points that do not match the reference human body skeleton feature points from the initial human body skeleton feature point information to obtain human body skeleton feature point information corresponding to the portrait image; or, in the case that M is less than or equal to N, taking the initial human body skeleton feature point information as the human body skeleton feature point information corresponding to the portrait image, wherein M and N are both positive integers.

5. An image processing apparatus characterized by comprising: The method comprises the following steps: extracting human body skeleton feature points of a portrait image in a target image to obtain human body skeleton feature point information corresponding to the portrait image; The constructing module is configured to construct a portrait three-dimensional image corresponding to the portrait image based on the human body skeleton feature point information. The display module is configured to superimposedly display the portrait three-dimensional image and the target image. The constructing module is specifically configured to: perform face recognition on the portrait image to obtain face image information corresponding to the portrait image; construct an initial portrait three-dimensional image based on the face image information corresponding to the portrait three-dimensional material and the human body skeleton feature point information; perform face rendering on the initial portrait three-dimensional image based on the face image information to obtain the portrait three-dimensional image. The face recognition on the portrait image to obtain the face image information corresponding to the portrait image includes: perform face recognition on the portrait image to confirm person name information corresponding to the portrait image, obtain a face image corresponding to the person name information from the Internet, and take the obtained face image as the face image information corresponding to the portrait image. The face image corresponding to the person name information is selected according to a user click rate.

6. The image processing apparatus according to claim 5, characterized by The extracting module is specifically configured to: analyze the portrait image based on a human feature point recognition model to obtain initial human body skeleton feature point information; correct the initial human body skeleton feature point information based on reference human body skeleton feature point information to obtain the human body skeleton feature point information corresponding to the portrait image.

7. The image processing apparatus according to claim 5, characterized by, The device further includes: an obtaining module configured to obtain an action control instruction; a determining module configured to determine human body skeleton action feature point information corresponding to the action control instruction; an adjusting module configured to process human body skeleton feature point information corresponding to the portrait three-dimensional image based on the human body skeleton action feature point information to obtain a portrait three-dimensional image after action adjustment.

8. The image processing apparatus according to claim 6, characterized by, The initial human body skeleton feature point information includes M initial human body skeleton feature points, the reference human body skeleton feature point information includes N reference human body skeleton feature points, and the extracting module is specifically configured to: remove, from the initial human body skeleton feature point information, initial human body skeleton feature points that do not match the reference human body skeleton feature points to obtain the human body skeleton feature point information corresponding to the portrait image, when M is greater than N; or, take the initial human body skeleton feature point information as the human body skeleton feature point information corresponding to the portrait image, when M is less than or equal to N, where M and N are positive integers.

Images