Facial image replacement method and system based on AI

Through the AI-based facial image replacement method, using AI edge algorithm and multiple iterations to find the mean, the problems of insufficient authenticity of the replacement effect, poor cross-domain adaptability and low computing efficiency in the prior art are solved, efficient and natural facial image replacement is achieved, and highly controllable replacement results are provided.

CN119494893BActive Publication Date: 2025-03-25ZHEJIANG HUAYE NETWORK TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202510073575.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-01-17
Publication Date
2025-03-25
Estimated Expiration
2045-01-17

AI Technical Summary

Technical Problem

In the case of lighting changes, complex backgrounds or dynamic expressions, the existing facial image replacement technology lacks authenticity, poor cross-domain adaptability, low computing efficiency, and lacks an effective privacy protection mechanism.

Method used

A facial image replacement method based on AI is proposed. By establishing a set of facial images of avatars, the most similar target facial images are automatically analyzed and selected, and iterative replacement is used to use AI edge algorithms, and the replacement effect is improved through head center calibration and size calibration.

Benefits of technology

It realizes efficient and natural facial image replacement in a variety of complex scenarios, improves the authenticity and adaptability of the replacement effect, optimizes the efficiency of computing resources, and provides highly controllable replacement results.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119494893B_ABST
    Figure CN119494893B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of image processing technology, and specifically provides an AI-based facial image replacement method and system, which aims to achieve efficient and accurate facial image replacement. The method includes the following steps: S1, establishing a target facial image set based on video frames, and ensuring the consistency of images in the set through midpoint correction and size calibration; S2, selecting the most similar facial images from the target set to form a target set; S3, using an AI edge algorithm to perform iterative replacement, and generating a natural replacement effect by optimizing the edge contour difference between the target and the image to be replaced; S4, outputting the replaced image and restoring the original video. The present invention also provides a system in which each module works together to achieve accurate replacement of facial images. The present invention improves the authenticity, stability and efficiency of the replacement effect in complex scenes, and is widely applicable to video special effects production, virtual image generation and other visual processing application fields.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to an AI-based facial image replacement method and system. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, breakthroughs have been made in the field of computer vision. Facial image processing technology, as an important branch of computer vision, has been widely used in entertainment, education, security and other fields. Facial image replacement technology is an important application. It can realize multiple functions such as virtual image generation, video special effects production and identity simulation by accurately capturing, analyzing and replacing the target facial features.

[0003] Existing facial image replacement technologies mainly rely on traditional image processing algorithms or deep learning-based generative adversarial networks (GANs). Traditional methods usually require a lot of manual participation, including key point annotation, feature point alignment, and texture synthesis. Although they can complete facial replacement to a certain extent, they are inefficient and the replacement effect is still insufficient in terms of authenticity and naturalness. Deep learning-based technologies, through end-to-end training, can more efficiently extract facial features and generate high-quality replacement images. However, such methods still face the following challenges:

[0004] 1. The replacement effect is not realistic enough: In the case of lighting changes, complex backgrounds or dynamic expressions, the generated face replacement images are prone to unnatural transitions, inaccurate feature alignment, and artifacts, which affect the overall visual effect.

[0005] 2. Poor cross-domain adaptability: When the resolution, skin color or expression of the target and source images are very different, the replacement effect of the existing methods will be significantly reduced, and it is difficult to obtain consistent performance in a variety of scenarios.

[0006] 3. Computational efficiency limitations: Complex network models and algorithms are highly dependent on hardware resources, resulting in limited real-time processing capabilities, which makes it difficult to meet the real-time and efficiency requirements of certain scenarios.

[0007] 4. Privacy protection issues: Facial image replacement involves sensitive user data. The existing technology lacks an effective privacy protection mechanism, which may lead to the risk of data leakage or abuse.

[0008] Based on the above problems, combined with the latest research results of deep learning and computer vision, an efficient, robust and privacy-preserving facial image replacement method and system is developed, which has important research significance and practical value for improving the authenticity and wide application of the replacement effect. Summary of the invention

[0009] In view of the above problems, the object of the present invention is to propose: a facial image replacement method based on AI, the method comprising the following steps:

[0010] S1. Establish a set of avatar facial images: determine the frames to be replaced based on each frame of the video, crop and round the avatar to obtain a target facial image set ;

[0011] S2. Target facial image replacement: From the avatar facial image collection Automatically analyze and select the most similar multiple targets to obtain a target set;

[0012] S3, iterative replacement: using AI edge algorithm, in the target set Substitute all the facial images to be replaced in , replace the target facial image multiple times and calculate the average until all the facial images to be replaced Replacement completed;

[0013] S4, output: output all the images and restore the original video.

[0014] Furthermore, when the head image is cropped and rounded in step S1, each target facial image as a set element The following conditions are met:

[0015] S11. Head center calibration: locate facial contours If the center origin is not in the center of the whole map Position, then the horizontal and vertical coordinates are corrected according to the following formula to make the facial contour The center coincides with the center of the entire image:

[0016]

[0017] in, and Represents the image width and image height respectively, Represents the horizontal and vertical ratio of the actual pixel coordinates, x is equal to the column number divided by the image width W, and y is equal to the row number divided by the image height H; Represents the horizontal and vertical ratio of the pixel correction coordinates; Represents the scalar coefficient used for weighted interpolation calculation; Represents facial contour The median value of the horizontal and vertical ratio; Indicates the median value of the entire image.

[0018] Furthermore, when the head image is cropped and rounded in step S1, each target facial image as a set element The following conditions are also met:

[0019] S12. Head size calibration: Calculate facial contours The minimum bounding rectangle of ; When the minimum enclosing rectangle The side length satisfies When the minimum enclosing rectangle is The side length satisfies When , the length and width are scaled in geometric proportion so that the minimum bounding rectangle covers the image and a corrected image is generated;

[0020] in, Represents the minimum enclosing rectangle Horizontal width and vertical width The larger value between Indicates the image width and image height The smaller value in .

[0021] Furthermore, step S2 specifically includes:

[0022] S21. Extraction of facial images to be replaced: Determine the facial set to be replaced All facial images to be replaced in , the set to be replaced The following boundary conditions are met:

[0023]

[0024] in, and Respectively represent the median horizontal and vertical ratios of the face to be replaced, and Respectively represent The median horizontal and vertical ratios of the target facial images; Indicates the aspect ratio of the facial image to be replaced; Respectively represent the width and height ratios of the reference facial image; The aspect ratio set of the face image to be replaced and the aspect ratio set of the target face image must have an intersection;

[0025] S22, target facial image extraction: according to the target facial image and the facial image to be replaced, from the target facial image set Automatically analyze the target face image and construct the target set in the order of the difference in the aspect ratio of the minimum bounding rectangle between the target face image and the face image to be replaced from small to large .

[0026] Furthermore, the AI ​​edge algorithm in step S3 specifically includes:

[0027] Extract the edge contours of the target facial image and the facial image to be replaced and perform displacement transformation so that the transformed edge information satisfies the minimum difference formula:

[0028]

[0029] in, , , and Respectively represent the horizontal ratio value, vertical ratio value, center point horizontal coordinate, and center point vertical coordinate of the edge contour of the target facial image; , , and ,as well as , , and Respectively represent the horizontal ratio value, vertical ratio value, center point horizontal coordinate, and center point vertical coordinate of the edge contour of the target facial image before and after optimization; Indicates finding the minimum value.

[0030] Furthermore, the averaging in step S3 means that after all facial images are replaced in the AI ​​edge algorithm, the replaced facial images are binarized, the original pixels are represented by two different colors and compared, all information is counted, and the average value of all pixels at the same position is calculated as the mean of the pixel.

[0031] Furthermore, in step S12, the facial image is scaled based on the maximum edge value.

[0032] The present invention also provides an AI-based facial image replacement system for implementing the AI-based facial image replacement method, comprising:

[0033] Correction end: used to perform midpoint correction and size correction on the acquired image;

[0034] Replacement end: used to replace facial images in consecutive frames using AI edge computing according to replacement requirements;

[0035] Iteration end: used to return the frame to the replacement end for further replacement when the replaced image does not meet the output requirements;

[0036] Output end: used to output the replaced image when the replaced image meets the output requirements.

[0037] Furthermore, satisfying the output requirement means that a mean deviation between an edge contour of the replaced facial image and an edge contour of the original facial image is less than a preset pixel threshold.

[0038] Furthermore, the preset pixel threshold is 3 pixels.

[0039] Beneficial Effects

[0040] The present invention provides an AI-based facial image replacement method, which can achieve efficient and natural facial image replacement in a variety of complex scenarios and has the following beneficial effects:

[0041] 1. Improve the authenticity and naturalness of the replacement effect: Through precise cropping, calibration and iterative replacement of the head face image set, this method can significantly reduce artifacts and unnatural transitions when dealing with lighting changes, complex backgrounds and dynamic expressions, making the replaced image more realistic and natural. The AI ​​edge algorithm is used to optimize the target facial images involved in the replacement process, further improving the accuracy and visual consistency of the replacement.

[0042] 2. Enhanced adaptability to diverse scenarios: The head center calibration and size calibration in step S1 can make the target facial image maintain high consistency in different resolutions, skin colors and expressions. The multi-target automatic analysis and selection mechanism in step S2 ensures that the most similar replacement target can be selected in diverse scenarios to meet different application requirements.

[0043] 3. Improve computing efficiency: By introducing AI edge algorithms and multiple iterative averaging processing methods, the present invention significantly optimizes the efficiency of computing resources while achieving high-quality face replacement. Compared with traditional algorithms, AI edge algorithms can quickly complete target alignment and replacement, greatly shorten processing time, and meet the requirements of real-time processing.

[0044] 4. Improve replacement accuracy and stability: The iterative replacement mechanism in step S3 ensures the stability of the replacement process. Through multiple replacements and averaging, it eliminates the accidental errors that may be introduced by a single replacement, further improving the accuracy and consistency of the replacement effect.

[0045] 5. Provide highly controllable replacement results: Through the multiple constraints introduced in steps S1 and S2, such as the facial center position, the aspect ratio of the minimum enclosing rectangle, the minimum difference in edge contours, etc., the controllability of the replacement process is ensured. Users can adjust the algorithm parameters according to specific needs to obtain more refined replacement results. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the method flow of the present invention. DETAILED DESCRIPTION

[0047] In order to deepen the understanding of the present invention, the present invention will be further described in detail below in conjunction with examples. The examples are only used to explain the present invention and do not constitute a limitation on the protection scope of the present invention.

[0048] Embodiment 1

[0049] according to Figure 1 As shown, this embodiment provides an AI-based facial image replacement method, the method comprising the following steps:

[0050] S1. Establish a set of avatar facial images: determine the frames to be replaced based on each frame of the video, crop and round the avatar to obtain a target facial image set ;

[0051] S2. Target facial image replacement: From the avatar facial image collection Automatically analyze and select the most similar multiple targets to obtain a target set;

[0052] S3, iterative replacement: using AI edge algorithm, in the target set Substitute all the facial images to be replaced in , replace the target facial image multiple times and calculate the average until all the facial images to be replaced Replacement completed;

[0053] S4, output: output all the images and restore the original video.

[0054] Furthermore, when the head image is cropped and rounded in step S1, each target facial image as a set element The following conditions are met:

[0055] S11. Head center calibration: locate facial contours If the center origin is not in the center of the whole map Position, then the horizontal and vertical coordinates are corrected according to the following formula to make the facial contour The center coincides with the center of the entire image:

[0056]

[0057] in, and Represents the image width and image height respectively, Represents the horizontal and vertical ratio of the actual pixel coordinates, x is equal to the column number divided by the image width W, and y is equal to the row number divided by the image height H; Represents the horizontal and vertical ratio of the pixel correction coordinates; Represents the scalar coefficient used for weighted interpolation calculation; Represents facial contour The median value of the horizontal and vertical ratio; Indicates the median value of the entire image.

[0058] Furthermore, when the head image is cropped and rounded in step S1, each target facial image as a set element The following conditions are also met:

[0059] S12. Head size calibration: Calculate facial contours The minimum bounding rectangle of ; When the minimum enclosing rectangle The side length satisfies When the minimum enclosing rectangle is The side length satisfies When , the length and width are scaled in geometric proportion so that the minimum bounding rectangle covers the image and a corrected image is generated;

[0060] in, Represents the minimum enclosing rectangle Horizontal width and vertical width The larger value between Indicates the image width and image height The smaller value in .

[0061] Furthermore, step S2 specifically includes:

[0062] S21. Extraction of facial images to be replaced: Determine the facial set to be replaced All facial images to be replaced in , the set to be replaced The following boundary conditions are met:

[0063]

[0064] in, and Respectively represent the median horizontal and vertical ratios of the face to be replaced, and Respectively represent The median horizontal and vertical ratios of the target facial images; Indicates the aspect ratio of the facial image to be replaced; Respectively represent the width and height ratios of the reference facial image; The aspect ratio set of the face image to be replaced and the aspect ratio set of the target face image must have an intersection;

[0065] S22, target facial image extraction: according to the target facial image and the facial image to be replaced, from the target facial image set Automatically analyze the target face image and construct the target set in the order of the difference in the aspect ratio of the minimum bounding rectangle between the target face image and the face image to be replaced from small to large .

[0066] Furthermore, the AI ​​edge algorithm in step S3 specifically includes:

[0067] Extract the edge contours of the target facial image and the facial image to be replaced and perform displacement transformation so that the transformed edge information satisfies the minimum difference formula:

[0068]

[0069] in, , , and Respectively represent the horizontal ratio value, vertical ratio value, center point horizontal coordinate, and center point vertical coordinate of the edge contour of the target facial image; , , and ,as well as , , and Respectively represent the horizontal ratio value, vertical ratio value, center point horizontal coordinate, and center point vertical coordinate of the edge contour of the target facial image before and after optimization; Indicates finding the minimum value.

[0070] Furthermore, the averaging in step S3 means that after all facial images are replaced in the AI ​​edge algorithm, the replaced facial images are binarized, the original pixels are represented by two different colors and compared, all information is counted, and the average value of all pixels at the same position is calculated as the mean of the pixel.

[0071] Furthermore, in step S12, the facial image is scaled based on the maximum edge value. Specific embodiment 2

[0073] A specific embodiment is given below, taking a video with a resolution of 640×480 and a length of about 120 frames as an example, to demonstrate how to apply the steps (S1~S4) in this solution to perform AI-based facial image replacement.

[0074] This example includes specific numerical examples of extracting facial images from videos, center calibration and size calibration, examples of matching the target facial image with the facial image to be replaced, and the process of iterative replacement using AI edge algorithms and finally outputting a video.

[0075] 1. Implementation Environment and Original Data Assumptions

[0076] Video resolution: 640×480

[0077] Video frame rate: 30 fps

[0078] Video length: 120 frames

[0079] Face detection model: Assume that a common deep learning face detection algorithm (such as MTCNN, RetinaFace, etc.) is used to detect facial key points and facial contours.

[0080] There are several face frames in the video that need to be replaced. The faces in these frames will be extracted and processed. In this example, it is assumed that we finally extract 10 "target face images" for reference. And 5 "face images to be replaced" ;

[0081] 2. S1: Building a collection of avatar facial images

[0082] 1.1 Extract the frame to be replaced and crop it

[0083] Run a face detection algorithm on each frame of the video.

[0084] Crop the detected face and keep only the head (including the edge area of ​​the face).

[0085] The face regions obtained by cropping are integrated into an initial image set.

[0086] Assume that after detection and rough cropping, we have obtained several face screenshots, and after screening, we get 10 relatively clear and positive target face images. , these images will be used as "references" or "templates".

[0087] 1.2 Center calibration when cropping and rounding the image (S11)

[0088] Original image size: 640×480

[0089] The coordinates of the center point of the whole map:

[0090] Assume that in a target face image The center of the detected facial contour The corresponding actual pixel coordinates are (360, 216). Converted into proportional coordinates (decimals between 0 and 1) with respect to width and height, they can be expressed as:

[0091]

[0092] Detected face center ratio: (0.5625, 0.45);

[0093] Full image center ratio: (0.5, 0.5);

[0094] The difference between the center of the facial contour and the center of the whole image: (0.5625−0.5, 0.45−0.5)=(0.0625, −0.05);

[0095] In order to make the center of the face closer to the center of the whole image, this scheme uses a weighted interpolation coefficient Perform displacement calibration, the formula is as follows:

[0096]

[0097] After substituting the values: and ;

[0098] Then the correction amount satisfies: ;

[0099] The original center (0.5625) satisfies the following after weighting: ;

[0100] So ultimately; .

[0101] The same is true for the y direction. If the original center ratio is 0.45, then: ;

[0102] The correction amount satisfies: ;

[0103] After weighting, it satisfies: ;

[0104] The coordinates of the final corrected facial center in the proportional coordinate system are .

[0105] To map it back to actual pixels, we can multiply it by the corresponding width and height to get , and can be further rounded off according to the actual scenario.

[0106] 1.3 Size calibration when cropping and rounding the avatar (S12)

[0107] After the center position is corrected, the head size needs to be further calibrated. For example, the detected facial contour The minimum bounding rectangle of The actual width and height are:

[0108] width Pixels, Height Pixel.

[0109] Since the overall size of the video frame in this example is 640×480, it satisfies: ; The maximum side of the minimum enclosing rectangle is ;

[0110] Judgment: If satisfied , it means that the width or height is exactly the same as the shortest side of the video and no scaling is required. , it means that the face is still relatively "small" after being cropped, and it can be enlarged proportionally to align it with the smallest edge.

[0111] In this example, , so it is necessary to enlarge the minimum enclosing rectangle to 480 in equal proportion; enlargement ratio: ; After adjustment, the width and height of the rectangle become 480 and 432 respectively, which can ensure that in subsequent replacement, the face area is more closely matched with the overall proportion of the video.

[0112] After completing the above center calibration and size calibration, the target facial image that meets the requirements of the solution of the present invention is obtained. The same process is performed for other face frames in the video to form the final target face image set. .

[0113] 3. S2: Target facial image replacement

[0114] 2.1 Extraction of facial image to be replaced (S21)

[0115] Suppose we select 5 “face images to be replaced” from the video again, recorded as These images also experienced and Similar cropping, center calibration and size calibration are performed to ensure comparability in coordinate system, aspect ratio, etc. during subsequent comparison.

[0116] According to the requirements of the solution, the facial image to be replaced needs to meet the following exemplary boundary conditions:

[0117] 1. Center lateral difference ≤ 0.1:

[0118] If the target face image The median value of the horizontal ratio is 0.45, then the facial image to be replaced The median value of the horizontal ratio must satisfy .

[0119] 2. Center longitudinal difference ≤ 0.1:

[0120] Similar to the judgment of the horizontal difference, ensure that the vertical center positions are roughly corresponding.

[0121] 3. The aspect ratio sets have intersections:

[0122] like The aspect ratio is between 1.1 and 1.2. It should also fall within or partially fall within 1.1-1.2 to ensure that there is no excessive stretching or distortion during matching.

[0123] In practical applications, factors such as facial angle and expression are usually considered. In this example, only the aspect ratio and center coordinates are listed.

[0124] 2.2 Target facial image extraction (S22)

[0125] Confirmed which After meeting the conditions, you need to be in the target face image collection Automatically analyze and select the most similar images as candidate targets to form a target set ;For example:

[0126] Compare the "minimum bounding rectangle" aspect ratio:

[0127] Calculate one by one and The differences are sorted in order from small to large.

[0128] Select the first k reference images with the highest similarity: for example, you can select images with a difference less than 0.05 to enter the target set , so we can get the target set , where each image is closer to the parameters of the width, height, and center position of the face to be replaced.

[0129] IV. S3: Iterative Replacement

[0130] 3.1 AI edge algorithm is replaced multiple times

[0131] For the target set obtained The corresponding facial image to be replaced , execute the AI ​​edge algorithm; the steps are as follows:

[0132] 1. Extract edge contours: Perform edge detection on each facial image (such as Canny operator, Sobel operator, etc.) to obtain the binary boundaries of the facial contour and main features.

[0133] 2. Displacement transformation:

[0134] By translating, scaling, and slightly rotating, the differences between the target facial image and the facial image to be replaced in terms of edge contour position, size, and center point coordinates are reduced.

[0135] Take a target face image The edge information is used as a reference to calculate For example: , Initially 450, after adjustment 0, making the difference Minimum;

[0136] Center coordinates ,Will By matching it with the translation transformation, the difference Also as small as possible.

[0137] Multiple iterations and averaging: Example of multiple iterations and replacements:

[0138] The iteration number n=1,2,3... represents the replacement round; each round compares and fine-tunes the edge coordinates and color fusion of the facial image; when the difference or the number of iterations reaches a threshold, the iteration ends.

[0139] 3.2 Binarization comparison and “average” of replacement results

[0140] After all facial images are replaced, each pixel position is binary compared: for example, red is used to represent the original pixel point, and green is used to represent the replaced pixel point. If overlapped, it may be displayed as a mixed color such as yellow. The results of multiple iterations at the same pixel position are averaged: assuming that the grayscale values ​​of a pixel in the five iteration results are 0, 50, 100, 150, and 200 respectively, the final pixel average can be taken as ; This way you can get a smoother and more natural blending effect.

[0141] 5. S4: Output and video recovery

[0142] All replaced frame images are resynthesized into a video in the frame order of the original video to ensure that the frame rate and resolution are consistent with the original video.

[0143] Save the output: You will get a new video with the face replacement completed. If necessary, you can do further post-processing, such as color correction, facial key point modification, etc.

[0144] Through this specific embodiment, we demonstrate how to use this solution to perform face detection, center position and size calibration, select the most similar target facial image, iterate replacement, and finally output the complete process of the video on a sample video with a resolution of 640×480.

[0145] In S1, the face center and size of each image are determined, and necessary enlargement or displacement calibration is performed;

[0146] In S2, the most similar target faces are screened and obtained by combining boundary conditions to form a target set;

[0147] In S3, multiple replacements and fusions are performed through AI edge algorithms, and statistics and averages are calculated at the pixel level;

[0148] In S4, all processed frames are integrated back into the video stream for output.

[0149] This method is suitable for scenarios where multiple faces in a video need to be replaced or dynamically fused. It relies on AI face detection and edge alignment algorithms to achieve high-precision and natural face-changing effects. Specific embodiment three

[0151] An AI-based facial image replacement system is used to implement the AI-based facial image replacement method, including: a correction end: used to perform midpoint correction and size correction on the acquired image; a replacement end: used to replace the facial images in consecutive frames using AI edge calculation according to the replacement requirements; an iteration end: used to return the frame to the replacement end for continuing replacement when the replaced image does not meet the output requirements; an output end: used to output the replaced image when the replaced image meets the output requirements.

[0152] Meeting the output requirement means that the average deviation between the edge contour of the replaced facial image and the edge contour of the original facial image is less than a preset pixel threshold, which is 3 pixels of the preset pixel threshold.

[0153] The above shows and describes the basic principles, main features and advantages of the present invention. It should be understood by those skilled in the art that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A facial image replacement method based on AI, characterized in that: The method comprises the following steps: S1. Establish a set of avatar facial images: determine the frames to be replaced based on each frame of the video, crop and round the avatar to obtain a target facial image set ; S2. Target facial image replacement: From the avatar facial image collection Automatically analyze and select the most similar multiple targets to obtain a target set; S3, iterative replacement: using AI edge algorithm, in the target set Substitute all the facial images to be replaced in , replace the target facial image multiple times and calculate the average until all the facial images to be replaced Replacement completed; S4, output: output all the images and restore the original video; In step S3, the AI ​​edge algorithm specifically includes: Extract the edge contours of the target facial image and the facial image to be replaced and perform displacement transformation so that the transformed edge information satisfies the minimum difference formula: ; in, , , and Respectively represent the horizontal ratio value, vertical ratio value, center point horizontal coordinate, and center point vertical coordinate of the edge contour of the target facial image; , , and ,as well as , , and Respectively represent the horizontal ratio value, vertical ratio value, center point horizontal coordinate, and center point vertical coordinate of the edge contour of the target facial image before and after optimization; Indicates seeking the minimum value; The averaging in step S3 means that after all facial images are replaced in the AI ​​edge algorithm, the replaced facial images are binarized, the original pixels are represented by two different colors and compared, all information is counted, and the average value of all pixels at the same position is calculated as the mean of the pixel.

2. The AI-based facial image replacement method according to claim 1, characterized in that: When the head image is cropped and rounded in step S1, each target facial image as a set element The following conditions are met: S11. Head center calibration: locate facial contours If the center origin is not in the center of the whole map Position, then the horizontal and vertical coordinates are corrected according to the following formula to make the facial contour The center coincides with the center of the entire image: ; in, and Represents the image width and image height respectively, Represents the horizontal and vertical ratio of the actual pixel coordinates, x is equal to the column number divided by the image width W, and y is equal to the row number divided by the image height H; Represents the horizontal and vertical ratio of the pixel correction coordinates; Represents the scalar coefficient used for weighted interpolation calculation; Represents facial contour The median value of the horizontal and vertical ratio; Indicates the median value of the entire image.

3. The AI-based facial image replacement method according to claim 1, characterized in that: When the head image is cropped and rounded in step S1, each target facial image as a set element The following conditions are also met: S12. Head size calibration: Calculate facial contours The minimum bounding rectangle of ; When the minimum enclosing rectangle The side length satisfies When the minimum enclosing rectangle is The side length satisfies When , the length and width are scaled in geometric proportion so that the minimum bounding rectangle covers the image and a corrected image is generated; in, Represents the minimum enclosing rectangle Horizontal width and vertical width The larger value between Indicates the image width and image height The smaller value in .

4. The AI-based facial image replacement method according to claim 1, characterized in that: Step S2 specifically includes: S21. Extraction of facial images to be replaced: Determine the facial set to be replaced All facial images to be replaced in , the set to be replaced The following boundary conditions are met: ; in, and Respectively represent the median horizontal and vertical ratios of the face to be replaced, and Respectively represent The median horizontal and vertical ratios of the target facial images; Indicates the aspect ratio of the facial image to be replaced; Respectively represent the width and height ratios of the reference facial image; The aspect ratio set of the face image to be replaced and the aspect ratio set of the target face image must have an intersection; S22, target facial image extraction: according to the target facial image and the facial image to be replaced, from the target facial image set Automatically analyze the target face image and construct the target set in the order of the difference in the aspect ratio of the minimum bounding rectangle between the target face image and the face image to be replaced from small to large .

5. The AI-based facial image replacement method according to claim 3, characterized in that: In step S12, the facial image is scaled based on the maximum edge value.

6. An AI-based facial image replacement system, used to implement an AI-based facial image replacement method as claimed in any one of claims 1 to 5, characterized in that: include: Correction end: used to perform midpoint correction and size correction on the acquired image; Replacement end: used to replace facial images in consecutive frames using AI edge computing according to replacement requirements; Iteration end: used to return the frame to the replacement end for further replacement when the replaced image does not meet the output requirements; Output end: used to output the replaced image when the replaced image meets the output requirements.

7. The AI-based facial image replacement system according to claim 6, characterized in that: Meeting the output requirement means that a mean deviation between an edge contour of the replaced facial image and an edge contour of the original facial image is less than a preset pixel threshold.

8. The AI-based facial image replacement system according to claim 7, characterized in that: The three pixels of the preset pixel threshold.

Citation Information

Patent Citations

  • Video-based 3D human face expression cartoon driving method

    CN101739712A

  • High-definition face replacement video generation method and system

    CN112446364A