Image processing methods, apparatuses, electronic devices and storage media

Abstract

This disclosure provides an image processing method, apparatus, electronic device, and storage medium, relating to the field of artificial intelligence technology, and particularly to the field of image desensitization. The specific implementation scheme is as follows: Detecting the face of a target object in a target object image to obtain facial pose information; determining a corresponding three-dimensional facial template from a facial template library based on the facial pose information; projecting the three-dimensional facial template onto the target object image to obtain a two-dimensional facial template that at least covers the face of the target object; scaling the two-dimensional facial template based on facial key points in the two-dimensional facial template and facial key points in the target object image; and applying Gaussian blur to the target object image based on the coverage area of ​​the scaled two-dimensional facial template in the target object image.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more particularly to the field of image desensitization. Specifically, this disclosure relates to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of digital media and the widespread use of social media platforms, the generation and sharing of digital images have become increasingly frequent. However, this also brings the risk of privacy breaches, especially for images containing facial information, such as human or animal faces. Therefore, it is necessary to develop effective technical means to anonymize these images or videos to protect privacy information. Summary of the Invention

[0003] This disclosure provides an image processing method, apparatus, electronic device, and storage medium.

[0004] According to one aspect of this disclosure, an image processing method is provided, comprising:

[0005] The face of the target object in the target object image is detected to obtain facial pose information;

[0006] Based on the facial pose information, a corresponding three-dimensional facial template is determined from the facial template library;

[0007] The three-dimensional face template is projected onto the target object image to obtain a two-dimensional face template that at least covers the face of the target object;

[0008] Based on the facial key points in the two-dimensional face template and the facial key points in the target object image, the two-dimensional face template is stretched and adjusted.

[0009] Based on the coverage area of ​​the two-dimensional face template in the target object image after scaling adjustment, the target object image is subjected to Gaussian blur.

[0010] According to another aspect of this disclosure, an image processing apparatus is provided, comprising:

[0011] The pose detection module is used to detect the face of the target object in the target object image and obtain facial pose information;

[0012] The template determination module is used to determine the corresponding three-dimensional face template in the face template library based on the face pose information.

[0013] A template projection module is used to project the three-dimensional face template onto the target object image to obtain a two-dimensional face template that at least covers the face of the target object;

[0014] The template scaling module is used to scale the two-dimensional face template based on the facial key points in the two-dimensional face template and the facial key points in the target object image.

[0015] An image blurring module is used to perform Gaussian blurring on the target object image based on the coverage area of ​​the two-dimensional face template after scaling adjustment in the target object image.

[0016] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0017] At least one central processing unit; and

[0018] A memory communicatively connected to the at least one central processing unit; wherein,

[0019] The memory stores instructions that can be executed by the at least one central processing unit (CPU) to enable the at least one CPU to perform any image processing method according to the embodiments of this disclosure.

[0020] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the image processing methods in the embodiments of this disclosure.

[0021] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the image processing methods according to embodiments of this disclosure.

[0022] According to the technology disclosed herein, a three-dimensional face template corresponding to the facial pose of the target object is projected onto the target object image to obtain a two-dimensional face template that at least covers the face of the target object. Then, using facial key points in the two-dimensional face template and facial key points in the target object image, the two-dimensional face template is scaled and adjusted to determine the coverage area of ​​the scaled and adjusted two-dimensional face template in the target object image. This coverage area in the target object image is then Gaussian blurred to achieve sufficient desensitization of the facial region of the target object in the target object image, protecting the privacy information of the target object. Moreover, the desensitization technology of this disclosure is simple and fast.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0024] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0025] Figure 1 This is a scene diagram of an image processing method according to an embodiment of the present disclosure;

[0026] Figure 2 This is a flowchart of an image processing method according to an embodiment of the present disclosure;

[0027] Figure 3 This is a schematic diagram of a projected image according to an embodiment of the present disclosure;

[0028] Figure 4A This is a schematic diagram of an image before scaling according to an embodiment of the present disclosure;

[0029] Figure 4B This is a schematic diagram of a scaled image according to an embodiment of the present disclosure;

[0030] Figure 5 This is a schematic diagram of the telescoping reference direction according to an embodiment of the present disclosure;

[0031] Figure 6 This is a structural block diagram of an image processing apparatus according to an embodiment of the present disclosure;

[0032] Figure 7 This is a structural block diagram of an image processing apparatus according to another embodiment of the present disclosure;

[0033] Figure 8 This is a block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0034] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0035] To facilitate understanding of the trajectory planning method provided in the embodiments of this disclosure, the related technologies of the embodiments of this disclosure are described below. The following related technologies are optional solutions and can be arbitrarily combined with the technical solutions of the embodiments of this disclosure, and they all fall within the protection scope of the embodiments of this disclosure.

[0036] Figure 1 This is a scene diagram illustrating an image processing method according to an embodiment of this disclosure. Figure 1As shown, this scenario includes a server 11 and multiple clients 12. Clients can be mobile smartphones, desktop computers, wearable electronic devices, etc. The server 11 can receive images or videos uploaded by users from multiple clients 12. The server 11 stores these images or videos in a database. The server 11 can retrieve images or videos from the database and anonymize any content in these images or videos that involves privacy information. In this way, the anonymized data can be used elsewhere while ensuring user privacy and security. Of course, users can also directly anonymize images or videos in the client 12 using an application, and then upload the anonymized data to the server 11 or pass it to other clients 12.

[0037] It should be noted that all images used in this disclosure are obtained with the user's authorization, and their storage and application comply with relevant laws and regulations and do not violate public order and good morals.

[0038] Figure 2 This is a flowchart of an image processing method according to an embodiment of the present disclosure. The method can be applied to an electronic device. This electronic device may be, for example, a terminal, a server, or other processing device. The terminal may be a desktop computer, mobile device, PDA (Personal Digital Assistant), handheld device, computing device, in-vehicle device, wearable device, or other user equipment (UE). In some implementations, the electronic device can implement the image processing method of the present disclosure by having a central processing unit call computer-readable instructions stored in memory.

[0039] like Figure 1 As shown, the method may include the following steps:

[0040] S210, Detect the face of the target object in the target object image to obtain facial pose information;

[0041] S220, Based on facial pose information, determine the corresponding 3D facial template in the facial template library;

[0042] S230, Project the three-dimensional face template onto the target object image to obtain a two-dimensional face template that at least covers the face of the target object;

[0043] S240, Based on the facial key points in the two-dimensional face template and the facial key points in the target object image, the two-dimensional face template is stretched and adjusted;

[0044] S250, based on the coverage area of ​​the two-dimensional face template after scaling and adjustment in the target object image, performs Gaussian blur on the target object image.

[0045] Understandably, the target audience can be elementary school students, middle school students, men or women, or animals such as cats, dogs, and rabbits.

[0046] Understandably, the target object image disclosed herein is obtained with the user's authorization, and the face of the target object in the target object image is desensitized during use, thereby protecting the privacy information of the target object.

[0047] Understandably, software or algorithms such as Hopenet or Head Pose Estimation can be used to perform pose detection on the face of the target object in the target object image to obtain the face pose information of the target object.

[0048] Understandably, facial pose information can include the specific values ​​of the target object's face in the three angles of yaw, pitch, and roll.

[0049] Understandably, the face template library stores multiple 3D face templates, which can be grouped by category. Each category group can include multiple 3D face templates in different poses. 3D face templates in different poses within the same category have different yaw, pitch, and roll angles.

[0050] For example, in step S220 above, based on the category information of the target object, a corresponding 3D face set is obtained from the face template library. This set includes multiple 3D face templates with different poses corresponding to that category. Based on the face pose information, a corresponding 3D face template is determined from the 3D face set. The face pose of this 3D face template matches the face pose information of the target object. Matching can be understood as the face poses being similar or identical.

[0051] For example, in step S230 above, each point in the three-dimensional face template is projected onto the target object image.

[0052] For example, in step S230 above, the three-dimensional face template is divided into multiple regions, and one or more points of each region are selected and projected onto the corresponding region in the target object image.

[0053] For example, in step S240 above, a scaling direction is determined based on the facial key points in the two-dimensional face template and the facial key points in the target object image. The two-dimensional face template is then scaled and adjusted according to this scaling direction so that the facial key points in the two-dimensional face template are aligned with the facial key points in the target object image.

[0054] Understandably, if the positions of facial key points in a 2D face template are close to or the same as the positions of corresponding facial key points in the target object image, it can be considered as alignment.

[0055] Understandably, Gaussian blurring is applied to the area covered by the two-dimensional face template in the target object image to achieve face desensitization of the target object image.

[0056] In some technologies, face desensitization schemes for target objects in images typically involve blurring the face outline or key points within the face outline. Alternatively, face swapping can be used to replace the image within the face outline.

[0057] However, this method can contaminate non-facial areas of the target image or result in insufficient facial desensitization. Furthermore, desensitization using face swapping requires more complex algorithms, leading to lower efficiency.

[0058] Therefore, in this embodiment, a three-dimensional face template corresponding to the facial pose of the target object is projected onto the target object image to obtain a two-dimensional face template that at least covers the face of the target object. Then, using facial key points in the two-dimensional face template and facial key points in the target object image, the two-dimensional face template is scaled to determine the coverage area of ​​the scaled two-dimensional face template in the target object image. Gaussian blur is then applied to this coverage area in the target object image, achieving sufficient desensitization of the facial region of the target object in the target object image and protecting the privacy information of the target object. Moreover, the desensitization steps are simple and fast.

[0059] In one embodiment, the step of projecting a three-dimensional face template onto a target object image to obtain a two-dimensional face template that at least covers the face of the target object may include: determining a scaling factor based on the positional information of key eye points in the three-dimensional face template and the positional information of key eye points in the target object image; scaling the three-dimensional face template based on the scaling factor; and projecting the scaled three-dimensional face template onto the target object image to obtain a two-dimensional face template that at least covers the face of the target object.

[0060] Understandably, the scaling factor is determined based on the eye key points in the 3D face template and the eye key points in the target object image.

[0061] Understandably, the average distance is determined based on the distance between each eye key point in the 3D face template and the corresponding eye key point in the target object image, and the scaling factor is determined using this average distance.

[0062] Understandably, the target eye key point is determined among multiple eye key points in the 3D face template, the distance between the target eye key point and the corresponding eye key point in the target object image is calculated, and then the scaling factor is determined using this distance.

[0063] Understandably, a 3D face template is three-dimensional, and its corresponding 3D coordinate system has three axes. The 3D face template is scaled along each coordinate axis according to a scaling factor.

[0064] For example, a projection reference is determined, and according to this projection reference, each pixel in the scaled 3D face template is projected onto the target object image, at least covering the 2D face template of the target object's face. For example, the projection reference could be one that allows a target facial key point in the 3D face template to be projected onto the position of the corresponding facial key point in the target object image.

[0065] For example, the scaled 3D face template is divided into multiple regions. According to a projection reference, a pixel from each region is projected onto the target object image, resulting in multiple projection points. Any three adjacent projection points are then stitched together to form a triangular facet, thus combining to obtain the 2D face template. For instance, this projection reference allows a pixel from a target region within the multiple regions to be projected onto a specified region in the target object image. This pixel can be the center point of the region or any pixel within the region.

[0066] According to the above implementation method, by utilizing the positional information of the key eye points in the 3D face template and the positional information of the key eye points in the target object image, the scaling factor can be accurately determined. Then, using the scaling factor, the 3D face template is scaled so that the size of the scaled 3D face template is similar to or the same as the size of the target object image. Finally, the 3D face template is projected onto the target object image to obtain a 2D face template that can fit the face of the target object in the target object image.

[0067] In one embodiment, the step of projecting the scaled three-dimensional face template onto the target object image to obtain a two-dimensional face template that at least covers the face of the target object may include: dividing the scaled three-dimensional face template into multiple first regions; dividing the target object in the target object image into multiple second regions; determining a one-to-one correspondence between the multiple first regions and the multiple second regions; based on the one-to-one correspondence, projecting the reference points in each of the first regions of the three-dimensional face template onto the corresponding second regions in the target object image to obtain projection points in each of the second regions; and stitching together any three adjacent projection points in the target object image to form a triangular facet to obtain a two-dimensional face template.

[0068] Understandably, the orthographic projection areas of each first region onto the target object image may or may not be the same.

[0069] Understandably, the areas of the various second regions may be the same or different.

[0070] Understandably, distances are calculated between multiple first regions and multiple second regions, and the closest first regions and second regions are paired to form a one-to-one correspondence.

[0071] Understandably, the reference point can be any pixel in the first region, or it can be the center point of the first region.

[0072] Understandably, by stitching together any three adjacent projection points in the target object image to form triangular patches, and then connecting these triangular patches, a two-dimensional face template can be obtained. Furthermore, this two-dimensional face template fits the face of the target object in the target object image.

[0073] For example, such as Figure 3 As shown, during projection, only one pixel is projected within a defined region. For example, if there is already a projected point at position (50, 50) in the target object image, other projected points within a region centered at (50, 50) with a radius of R can be discarded. Here, R can be 1 / 40 or 1 / 80 of the length of the target object's face in the target object image.

[0074] It should be noted that, Figure 3 The facial images shown are not real captured images, but digitally generated images with blurred facial features; they are used for illustrative purposes only.

[0075] According to the above implementation method, following the rule of projecting only one point per region, the 3D face template is projected onto the target object image, and any three adjacent projection points on the target object image are stitched together to form a triangular patch, thus obtaining a 2D face template. In this way, it is not necessary to project all pixels of the 3D face template onto the target object image; only one pixel within a specified range is projected, which improves projection efficiency and consequently increases the efficiency of generating the 2D face template.

[0076] In one implementation, determining a scaling factor based on the positional information of key eye points in a 3D face template and the positional information of key eye points in a target object image includes: determining a first distance between a first outer corner point and a first inner corner point of the eye based on the positional information of a first outer corner point and a first inner corner point of the eye in the 3D face template; determining a second distance between a second outer corner point and a second inner corner point of the eye based on the positional information of a second outer corner point and a second inner corner point of the eye in the target object image; and determining a scaling factor based on the ratio of the second distance to the first distance.

[0077] Understandably, the outer corner point and inner corner point of the first eye in a 3D face template refer to the inner and outer corner points of the same eye.

[0078] Understandably, the second outer corner point and the second inner corner point of the eye in the target object image refer to the inner and outer corner points of the same eye. Furthermore, this eye is on the same side as the eye indicated in the 3D face template, for example, both being the left eye or both being the right eye.

[0079] For example, the coordinates of the outer corner and inner corner of the first eye in the 3D face template are (270, 360, 590) and (530, 360, 590), respectively. The coordinates of the second outer corner and inner corner of the second eye in the target object image are (120, 170) and (190, 170), respectively. Thus, the first distance is 260, the second distance is 70, and the scaling factor is 70 / 260 = 7 / 26.

[0080] According to the above implementation method, a scaling factor is obtained by using the ratio of the width of the eyes in the 3D face template to the width of the eyes in the target object image. The 3D face template is then scaled according to this scaling factor so that the scaled 3D face template can match the face size of the target object in the target object image.

[0081] In one implementation, scaling a three-dimensional face template based on a scaling factor includes: determining a scaling direction based on the coordinate axes of the three-dimensional coordinate system corresponding to the three-dimensional face template; and scaling the three-dimensional face template based on the scaling factor and the scaling direction.

[0082] For example, the scaling direction may include the directions of the three coordinate axes.

[0083] For example, the 3D face template is scaled along the scaling direction corresponding to each coordinate axis according to a scaling factor. The 3D face template is scaled to a multiple of the scaling factor.

[0084] like Figure 4A and Figure 4B As shown, when the scaling factor is 1 / 2, it will be from Figure 4AThe image in the image is reduced to Figure 4B The image in the image.

[0085] According to the above implementation method, the three-dimensional face template is scaled along the coordinate axis direction of the three-dimensional face template according to the scaling factor.

[0086] In one implementation, the two-dimensional face template is scaled and adjusted based on facial key points in the two-dimensional face template and facial key points in the target object image. This includes: determining the midpoint between the two eyes and the midpoint between the two corners of the mouth in the two-dimensional face template; determining the scaling direction based on the line connecting the midpoints of the eyes and the midpoints of the mouths and the perpendicular bisector perpendicular to the line connecting the two eyes and the mouths; and scaling and adjusting the two-dimensional face template based on the scaling direction so that the facial key points in the two-dimensional face template are aligned with the corresponding facial key points in the target object image.

[0087] For example, such as Figure 5 As shown, it illustrates the line connecting the midpoint of the eye and the midpoint of the mouth on the face, as well as the perpendicular bisector of that line, which serves as the reference direction for stretching or contracting. You can first stretch or contract along the direction of the line connecting the eyes, and then stretch or contract along the direction of the perpendicular bisector.

[0088] According to the above implementation method, stretching or scaling along the line connecting the midpoint between the two eyes and the midpoint of the corner of the mouth, or along the perpendicular bisector of the line, can quickly align the facial key points in the two-dimensional face template with the facial key points on the target object image.

[0089] Figure 6 This is a structural block diagram of an image processing apparatus according to an embodiment of the present disclosure.

[0090] like Figure 6 As shown, the image processing apparatus may include:

[0091] The pose detection module 610 is used to detect the face of the target object in the target object image and obtain facial pose information;

[0092] The template determination module 620 is used to determine the corresponding three-dimensional face template in the face template library based on the face pose information.

[0093] Template projection module 630 is used to project the three-dimensional face template into the target object image to obtain a two-dimensional face template that at least covers the face of the target object;

[0094] The template scaling module 640 is used to scale the two-dimensional face template based on the facial key points in the two-dimensional face template and the facial key points in the target object image.

[0095] The image blurring module 650 is used to perform Gaussian blurring on the target object image based on the coverage area of ​​the two-dimensional face template after scaling adjustment in the target object image.

[0096] Figure 7 This is a structural block diagram of an image processing apparatus according to another embodiment of the present disclosure.

[0097] like Figure 6 and Figure 7 As shown, Figure 7 The posture detection module 710, template determination module 720, template projection module 730, template scaling module 740, and image blurring module 750 are respectively connected to... Figure 6 The posture detection module 610, template determination module 620, template projection module 630, template scaling module 640, and image blurring module 650 have the same structure and function, and will not be described in detail here.

[0098] In one embodiment, the template projection module 730 includes:

[0099] The scaling factor determination unit 731 is used to determine the scaling factor based on the position information of the key eye points in the three-dimensional face template and the position information of the key eye points in the target object image.

[0100] Template scaling unit 732 is used to scale the three-dimensional face template based on the scaling factor;

[0101] Template projection unit 733 is used to project the scaled three-dimensional face template onto the target object image to obtain a two-dimensional face template that at least covers the face of the target object.

[0102] In one embodiment, the template projection unit 733 is specifically used for:

[0103] The scaled-up 3D face template is divided into multiple first regions;

[0104] The target object in the target object image is divided into multiple second regions;

[0105] Determine a one-to-one correspondence between the plurality of first regions and the plurality of second regions;

[0106] Based on the one-to-one correspondence, the reference points in each of the first regions of the three-dimensional face template are projected onto the corresponding second regions in the target object image to obtain the projection points in each of the second regions.

[0107] The two-dimensional face template is obtained by stitching together any three adjacent projection points in the target object image to form a triangular face.

[0108] In one embodiment, the scaling factor determining unit 731 is specifically used for:

[0109] Based on the position information of the first outer corner point and the first inner corner point of the eye in the three-dimensional face template, the first distance between the first outer corner point and the first inner corner point of the eye is determined;

[0110] Based on the position information of the second outer corner point and the second inner corner point of the eye in the target object image, a second distance between the second outer corner point and the second inner corner point of the eye is determined;

[0111] The scaling factor is determined based on the ratio of the second distance to the first distance.

[0112] In one embodiment, the template scaling unit 732 is specifically used for:

[0113] The scaling direction is determined based on the coordinate axes of the three-dimensional coordinate system corresponding to the three-dimensional face template;

[0114] The 3D face template is scaled based on the scaling factor and the scaling direction.

[0115] In one embodiment, the template telescoping module 740 includes:

[0116] The midpoint determination unit 741 is used to determine the midpoint of the eyes between the two eyes and the midpoint of the mouth between the two corners of the mouth in the two-dimensional face template.

[0117] The extension / retraction direction determination unit 742 is used to determine the extension / retraction direction based on the line connecting the midpoint of the eye and the midpoint of the mouth, and the perpendicular bisector perpendicular to the line connecting the eye and the mouth.

[0118] The scaling adjustment unit 743 is used to scale the two-dimensional face template based on the scaling direction so that the facial key points in the two-dimensional face template are aligned with the corresponding facial key points in the target object image.

[0119] The specific functions and examples of each module and submodule of the apparatus in this disclosure can be found in the relevant descriptions of the corresponding steps in the above method embodiments, and will not be repeated here.

[0120] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0121] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product. The electronic device may be a server, and the server may further include any of the power supply control devices described in the embodiments of this disclosure.

[0122] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0123] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 may also store various programs and data required for the operation of the electronic device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0124] Multiple components in electronic device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows electronic device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0125] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as an image processing method. For example, in some embodiments, an image processing method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of an image processing method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform an image processing method by any other suitable means (e.g., by means of firmware).

[0126] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0128] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0130] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0131] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0132] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0133] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An image processing method, comprising: The face of the target object in the target object image is detected to obtain facial pose information; Based on the facial pose information, a corresponding three-dimensional facial template is determined from the facial template library; Projecting the three-dimensional face template onto the target object image to obtain a two-dimensional face template that at least covers the face of the target object includes: based on the one-to-one correspondence between each first region in the three-dimensional face module and each second region of the target object in the target object image, projecting reference points in each first region of the three-dimensional face template onto the corresponding second regions in the target object image to obtain projection points in each second region; and stitching together any three adjacent projection points in the target object image to form a triangular facet to obtain the two-dimensional face template, wherein only one pixel is projected in each region. Based on the facial key points in the two-dimensional face template and the facial key points in the target object image, the two-dimensional face template is stretched and adjusted. Based on the coverage area of ​​the two-dimensional face template in the target object image after scaling adjustment, the target object image is subjected to Gaussian blur.

2. The method according to claim 1, wherein, The step of projecting the three-dimensional face template onto the target object image to obtain a two-dimensional face template that at least covers the face of the target object further includes: Based on the positional information of the key eye points in the 3D face template and the positional information of the key eye points in the target object image, the scaling factor is determined; The 3D face template is scaled based on the scaling factor.

3. The method according to claim 2, wherein, Also includes: The scaled-up 3D face template is divided into multiple first regions; The target object in the target object image is divided into multiple second regions; Determine the one-to-one correspondence between the plurality of first regions and the plurality of second regions.

4. The method according to claim 2, wherein, The determination of the scaling factor based on the positional information of the key eye points in the 3D face template and the positional information of the key eye points in the target object image includes: Based on the position information of the first outer corner point and the first inner corner point of the eye in the three-dimensional face template, the first distance between the first outer corner point and the first inner corner point of the eye is determined; Based on the position information of the second outer corner point and the second inner corner point of the eye in the target object image, a second distance between the second outer corner point and the second inner corner point of the eye is determined; The scaling factor is determined based on the ratio of the second distance to the first distance.

5. The method according to claim 4, wherein, The scaling of the 3D face template based on the scaling factor includes: The scaling direction is determined based on the coordinate axes of the three-dimensional coordinate system corresponding to the three-dimensional face template; The 3D face template is scaled based on the scaling factor and the scaling direction.

6. The method according to any one of claims 1-5, wherein, The step of scaling the two-dimensional face template based on facial key points in the two-dimensional face template and facial key points in the target object image includes: In the two-dimensional face template, the midpoint between the two eyes and the midpoint between the two corners of the mouth are determined; The direction of extension and retraction is determined based on the line connecting the midpoint of the eye and the midpoint of the mouth, and the perpendicular bisector perpendicular to the line connecting them. Based on the stretching direction, the two-dimensional face template is stretched and adjusted so that the facial key points in the two-dimensional face template are aligned with the corresponding facial key points in the target object image.

7. An image processing apparatus, comprising: The pose detection module is used to detect the face of the target object in the target object image and obtain facial pose information; The template determination module is used to determine the corresponding three-dimensional face template in the face template library based on the face pose information. A template projection module is used to project the three-dimensional face template onto the target object image to obtain a two-dimensional face template that at least covers the face of the target object. The module includes a template projection unit, which, based on a one-to-one correspondence between each first region in the three-dimensional face module and each second region of the target object in the target object image, projects reference points in each of the first regions of the three-dimensional face template onto the corresponding second regions in the target object image to obtain projection points in each of the second regions. Any three adjacent projection points in the target object image are then stitched together to form a triangular facet to obtain the two-dimensional face template, wherein only one pixel is projected within each region. The template scaling module is used to scale the two-dimensional face template based on the facial key points in the two-dimensional face template and the facial key points in the target object image. An image blurring module is used to perform Gaussian blurring on the target object image based on the coverage area of ​​the two-dimensional face template after scaling adjustment in the target object image.

8. The apparatus according to claim 7, wherein, The template projection module further includes: The scaling factor determination unit is used to determine the scaling factor based on the position information of the key eye points in the three-dimensional face template and the position information of the key eye points in the target object image. The template scaling unit is used to scale the three-dimensional face template based on the scaling factor.

9. The apparatus according to claim 8, wherein, The template projection unit is also used for: The scaled-up 3D face template is divided into multiple first regions; The target object in the target object image is divided into multiple second regions; Determine the one-to-one correspondence between the plurality of first regions and the plurality of second regions.

10. The apparatus according to claim 8, wherein, The scaling factor determination unit is specifically used for: Based on the position information of the first outer corner point and the first inner corner point of the eye in the three-dimensional face template, the first distance between the first outer corner point and the first inner corner point of the eye is determined; Based on the position information of the second outer corner point and the second inner corner point of the eye in the target object image, a second distance between the second outer corner point and the second inner corner point of the eye is determined; The scaling factor is determined based on the ratio of the second distance to the first distance.

11. The apparatus according to claim 10, wherein, The template scaling unit is specifically used for: The scaling direction is determined based on the coordinate axes of the three-dimensional coordinate system corresponding to the three-dimensional face template; The 3D face template is scaled based on the scaling factor and the scaling direction.

12. The apparatus according to any one of claims 7-11, wherein, The template scaling module includes: The midpoint determination unit is used to determine the midpoint of the eyes between the two eyes and the midpoint of the mouth between the two corners of the mouth in the two-dimensional face template. The telescopic direction determination unit is used to determine the telescopic direction based on the line connecting the midpoint of the eye and the midpoint of the mouth, and the perpendicular bisector perpendicular to the line connecting the eye and the mouth. The scaling adjustment unit is used to scale the two-dimensional face template based on the scaling direction so that the facial key points in the two-dimensional face template are aligned with the corresponding facial key points in the target object image.

13. An electronic device, comprising: At least one central processing unit; as well as A memory communicatively connected to the at least one central processing unit; wherein, The memory stores instructions executable by the at least one central processing unit (CPU) to enable the at least one CPU to perform the method of any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-6.

15. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-6.

Images