Face change recognition method and device, electronic equipment and storage medium
By performing color space conversion and feature extraction and fusion on the initial facial image, combined with the pre-trained classifier model, the problems of low accuracy and high cost of face change recognition in the prior art are solved, and efficient face change recognition is achieved.
Patent Information
- Application Number
- CN202411492492.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-24
- Publication Date
- 2025-02-25
AI Technical Summary
In the prior art, when performing face change recognition, the recognition accuracy is low and the cost is high.
By performing color space conversion on the initial facial image, LBP and LPQ feature vectors are extracted, and feature fusion is performed, and a pre-trained classifier model is input for face swap recognition.
While ensuring the accuracy of face change recognition results, it effectively reduces the cost of face change recognition.
Smart Images
Figure CN119516623A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of computer technology, and in particular to a face-changing recognition method, device, electronic device and storage medium. Background Art
[0002] In recent years, AI face-changing technology based on generative adversarial networks has made significant progress, making it easier to generate realistic face-changing images. Deepfake technology seamlessly transfers one person's facial expressions and features to another person's face by training a deep neural network. The generated images or videos are often difficult to identify their authenticity with the naked eye. As AI face-changing technology gradually matures and is widely used in entertainment, advertising and other fields, it has gradually aroused people's concerns about privacy and security. Therefore, face-changing recognition of facial images is required in some business scenarios.
[0003] In the related art, when performing face-changing recognition, the recognition accuracy is low and the cost is high. Summary of the invention
[0004] The present disclosure aims to solve one of the technical problems in the related art at least to some extent.
[0005] To this end, the first purpose of the present invention is to propose a face-changing recognition method, thereby effectively reducing the cost of face-changing recognition while ensuring the accuracy of the face-changing recognition results.
[0006] The second objective of the present disclosure is to provide a face-changing recognition device.
[0007] A third objective of the present disclosure is to provide an electronic device.
[0008] A fourth objective of the present disclosure is to provide a computer-readable storage medium.
[0009] A fifth object of the present disclosure is to provide a computer program product.
[0010] To achieve the above-mentioned purpose, the first embodiment of the present disclosure proposes a face-changing recognition method, comprising:
[0011] Performing color space conversion on the initial facial image to obtain a target facial image, wherein the target facial image includes a plurality of candidate pixels, each of the candidate pixels having a corresponding pixel value;
[0012] Determine a target pixel from the plurality of candidate pixels according to the pixel value, wherein the plurality of target pixels jointly constitute a reference facial region;
[0013] Performing feature extraction on the reference facial region to obtain an LBP feature vector and an LPQ feature vector;
[0014] Performing feature fusion on the LBP feature vector and the LPQ feature vector to obtain a target feature vector;
[0015] The target feature vector is input into a pre-trained classifier model to obtain a face-changing recognition result, wherein the face-changing recognition result is used to indicate whether the initial facial image has been subjected to face-changing processing.
[0016] To achieve the above-mentioned purpose, the second embodiment of the present disclosure proposes a face-changing recognition device, comprising:
[0017] A processing module, configured to perform color space conversion on the initial facial image to obtain a target facial image, wherein the target facial image includes a plurality of candidate pixels, each of the candidate pixels having a corresponding pixel value;
[0018] a determination module, configured to determine a target pixel from the plurality of candidate pixels according to the pixel value, wherein the plurality of target pixels jointly constitute a reference facial region;
[0019] A feature extraction module, used for extracting features from the reference facial region to obtain an LBP feature vector and an LPQ feature vector;
[0020] A feature fusion module, used for performing feature fusion on the LBP feature vector and the LPQ feature vector to obtain a target feature vector;
[0021] The recognition module is used to input the target feature vector into a pre-trained classifier model to obtain a face-changing recognition result, wherein the face-changing recognition result is used to indicate whether the initial facial image has been processed by face-changing.
[0022] To achieve the above-mentioned purpose, the third aspect embodiment of the present disclosure proposes an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so as to implement a face-changing recognition method proposed in the first aspect embodiment of the present disclosure.
[0023] To achieve the above-mentioned purpose, the fourth aspect embodiment of the present disclosure proposes a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed by the processor, they are used to implement the face-changing recognition method proposed in the first aspect embodiment of the present disclosure.
[0024] To achieve the above-mentioned purpose, the fifth aspect of the present disclosure proposes a computer program product, including a computer program, which, when executed by a processor, implements the face-changing recognition method proposed in the first aspect of the present disclosure.
[0025] The face-changing recognition method, device, electronic device, non-transient computer-readable storage medium storing computer instructions, and computer program product provided by the present disclosure obtain a target face image by performing color space conversion on an initial face image, wherein the target face image includes multiple candidate pixels, each candidate pixel having a corresponding pixel value; determining a target pixel from multiple candidate pixels according to the pixel value, wherein multiple target pixels jointly constitute a reference face area; extracting features from the reference face area to obtain an LBP feature vector and an LPQ feature vector; performing feature fusion on the LBP feature vector and the LPQ feature vector to obtain a target feature vector; inputting the target feature vector into a pre-trained classifier model to obtain a face-changing recognition result, wherein the face-changing recognition result is used to indicate whether the initial face image has been processed by face-changing. Thus, the face-changing recognition cost can be effectively reduced while ensuring the accuracy of the face-changing recognition result.
[0026] Additional aspects and advantages of the present disclosure will be given in part in the following description and in part will be obvious from the following description or learned through practice of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] The above and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0028] Figure 1 A flowchart of a face-changing recognition method provided by an embodiment of the present disclosure;
[0029] Figure 2 A flowchart of another face-changing recognition method provided by an embodiment of the present disclosure;
[0030] Figure 3 is a schematic diagram of the face-changing recognition process proposed in the present disclosure;
[0031] Figure 4 A structural schematic diagram of a face-changing recognition device provided by an embodiment of the present disclosure;
[0032] Figure 5 It is a schematic structural diagram of an electronic device shown in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION
[0033] Embodiments of the present disclosure are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.
[0034] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this disclosure are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. The acquisition, transmission, storage, use and processing of data in the technical solution of this disclosure are in compliance with the relevant provisions of national laws and regulations.
[0035] It should be noted that in the embodiments of the present disclosure, certain software, components, models and other existing solutions in the industry may be mentioned, which should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present disclosure, but it does not mean that the applicant has or will necessarily use the solution.
[0036] Figure 1 A flowchart of a face-changing recognition method provided in an embodiment of the present disclosure.
[0037] This embodiment takes the face-changing recognition method being configured in a face-changing recognition device as an example. In this embodiment, the face-changing recognition method can be configured in the face-changing recognition device. The face-changing recognition device can be set in a server or in an electronic device, and there is no limitation to this.
[0038] This embodiment takes the face-changing recognition method configured in an electronic device as an example. The electronic device is a hardware device such as a smart phone, a tablet computer, a personal digital assistant, an e-book, etc. having various operating systems.
[0039] It should be noted that the execution subject of the embodiment of the present disclosure may be, for example, a central processing unit (CPU) in a server or electronic device in terms of hardware, or may be, for example, a related background service in a server or electronic device in terms of software, without limitation.
[0040] like Figure 1 As shown, the face-changing recognition method includes:
[0041] S101: Performing color space conversion on an initial facial image to obtain a target facial image, wherein the target facial image includes a plurality of candidate pixels, each of which has a corresponding pixel value.
[0042] The initial facial image refers to an unprocessed image containing facial information.
[0043] The target facial image refers to the facial image obtained by converting the initial facial image through color space conversion.
[0044] In the embodiments of the present disclosure, when performing color space conversion on the initial facial image to obtain the target facial image, different color space conversion methods can be flexibly adopted in combination with the color mode of the initial facial image and the requirements of the image color mode in the subsequent feature extraction process, without any restriction.
[0045] The candidate pixels refer to the pixels that make up the target facial image. The pixel value can be used to describe the color and brightness information of the pixel. The specific form of the pixel value is determined by the color mode of the target facial image.
[0046] It is understandable that the initial facial image obtained in daily life is usually in RGB color mode (or any other color mode), and the RGB image may not be suitable for directly participating in the subsequent face-changing recognition step. Therefore, in the embodiment of the present disclosure, the initial facial image can be converted into a color space to obtain a target facial image to ensure the practicality of the obtained target facial image for the face-changing recognition operation process.
[0047] S102: Determine a target pixel from a plurality of candidate pixels according to the pixel value, wherein the plurality of target pixels jointly constitute a reference facial region.
[0048] The target pixel refers to the pixel at the face position in the target face image.
[0049] The reference facial region refers to an image that may belong to the facial region and is jointly constituted by multiple target pixels.
[0050] In the embodiments of the present disclosure, when determining a target pixel from multiple candidate pixels based on pixel values, the pixel values may be input into a pre-trained machine learning model to determine the target pixel from multiple candidate pixels. Alternatively, the target pixel may be determined from multiple candidate pixels based on pixel values based on a third-party device, and there is no limitation to this.
[0051] It is understandable that, in addition to facial image information, the target facial image may also contain image information of other objects. In addition to possibly interfering with the subsequent feature extraction process, this other object image information may also increase additional computing costs. Therefore, in the embodiments of the present disclosure, the target pixel can be determined from multiple candidate pixels based on the pixel value, and then the reference facial area can be determined, thereby providing a reliable and practical execution object for the subsequent feature extraction process.
[0052] S103: Extract features from the reference facial region to obtain an LBP feature vector and an LPQ feature vector.
[0053] Among them, LBP (Local Binary Pattern) feature vector and LPQ (Local Phase Quantization) feature vector are both effective methods for describing image features in image processing and computer vision.
[0054] In the disclosed embodiment, when extracting features from the reference facial region to obtain an LBP feature vector, a 3x3 neighborhood may be selected for calculation, and the central pixel value is compared with the surrounding 8 pixel values. If the surrounding pixel value is greater than or equal to the central pixel, the corresponding binary bit is 1, otherwise it is 0. Finally, an 8-bit binary number is formed, which is converted into a decimal system as the LBP value.
[0055] In the disclosed embodiment, when extracting features from a reference facial region to obtain an LPQ feature vector, the image may be subjected to local window processing, and then the frequency components of each local window may be calculated using a two-dimensional fast Fourier transform (FFT), and then the phase information may be extracted from the FFT and quantized to form an LPQ feature.
[0056] In the disclosed embodiment, when feature extraction is performed on the reference facial area to obtain the LBP feature vector and the LPQ feature vector, reliable reference information can be provided for subsequent face-changing recognition, thereby effectively improving the accuracy of face-changing recognition.
[0057] S104: Perform feature fusion on the LBP feature vector and the LPQ feature vector to obtain a target feature vector.
[0058] The target feature vector refers to the fused feature vector obtained after the LBP feature vector and the LPQ feature vector are fused.
[0059] In the disclosed embodiment, when performing feature fusion on the LBP feature vector and the LPQ feature vector, the corresponding feature fusion method can be flexibly selected based on the specific application scenario and the feature representation required for the subsequent face-changing recognition process, such as: weighted average, splicing, linear change, nonlinear fusion, feature fusion, etc., without limitation.
[0060] In the disclosed embodiment, when the LBP feature vector and the LPQ feature vector are feature fused to obtain a target feature vector, the obtained target feature vector can comprehensively indicate the feature information of the image in different dimensions, thereby providing reliable data support for subsequent face-changing recognition.
[0061] S105: Input the target feature vector into a pre-trained classifier model to obtain a face-changing recognition result, wherein the face-changing recognition result is used to indicate whether the initial facial image has been processed by face-changing.
[0062] Among them, the classifier model can refer to a pre-trained machine learning classifier for face-changing recognition, such as an SVM classifier, or it can also refer to any other device or equipment that can achieve similar functions, without limitation.
[0063] It is understandable that the classifier model usually does not require a large amount of labeled data for training, thereby effectively reducing the model training cost of the face-changing recognition process.
[0064] That is to say, in the embodiment of the present disclosure, after the LBP feature vector and the LPQ feature vector are feature fused to obtain the target feature vector, the target feature vector can be input into the pre-trained classifier model to obtain the face-changing recognition result, thereby determining whether the initial facial image has undergone face-changing processing.
[0065] In this embodiment, the target facial image is obtained by performing color space conversion on the initial facial image, wherein the target facial image includes multiple candidate pixels, each candidate pixel has a corresponding pixel value; the target pixel is determined from the multiple candidate pixels according to the pixel value, wherein the multiple target pixels jointly constitute a reference facial region; the reference facial region is subjected to feature extraction to obtain an LBP feature vector and an LPQ feature vector; the LBP feature vector and the LPQ feature vector are subjected to feature fusion to obtain a target feature vector; the target feature vector is input into a pre-trained classifier model to obtain a face-changing recognition result, wherein the face-changing recognition result is used to indicate whether the initial facial image has been subjected to face-changing processing. Thus, the face-changing recognition cost can be effectively reduced while ensuring the accuracy of the face-changing recognition result.
[0066] Figure 2 A flowchart of another face-changing recognition method provided by an embodiment of the present disclosure.
[0067] like Figure 2 As shown, the face-changing recognition method includes:
[0068] S201: Perform YCbCr color space conversion on an initial facial image to obtain a target facial image, wherein the target facial image includes a plurality of candidate pixels, each candidate pixel has a corresponding pixel value, and the pixel value includes: a blue difference component and a red difference component.
[0069] The YCbCr color space conversion of the initial facial image refers to converting the initial facial image into the YCbCr color space. At this time, the pixel value of the target facial image includes three components:
[0070] Y (brightness): represents the brightness information of the image, and the range is usually 0 to 255. The higher the Y value, the brighter the pixel.
[0071] Cb (blue difference): represents the difference between blue and brightness. It is used to encode the chrominance information of the image, and the range is also 0 to 255. The change in value determines the intensity of blue.
[0072] Cr (Red Difference): Indicates the difference between red and brightness, ranging from 0 to 255. It determines the intensity of red.
[0073] That is, in the embodiment of the present disclosure, the initial facial image can be converted into a YCbCr color space to obtain a target facial image, thereby ensuring the practicability and applicability of the obtained target facial image for the subsequent feature extraction process.
[0074] S202: Determine a blue difference component value range and a red difference component value range of the facial skin color in the initial facial image.
[0075] Among them, the blue difference component value range and the red difference component value range refer to the value range of the blue difference component and the red difference component corresponding to the user's facial skin color.
[0076] In the embodiment of the present disclosure, when determining the blue difference component value range and the red difference component value range of the facial skin color in the initial facial image, a large number of sample facial images can be obtained, and then the blue difference component value and the red difference component value corresponding to the sample facial images are determined, and then the blue difference component value range and the red difference component value range of the above-mentioned facial skin color are determined.
[0077] Optionally, in some embodiments, when determining the blue difference component value range and the red difference component value range of the facial skin color in the initial facial image, the facial skin color type of the initial facial image may be determined; and the blue difference component value range and the red difference component value range may be determined based on the facial skin color type. Thus, the appropriate blue difference component value range and the red difference component value range may be selected quickly and accurately in combination with the facial skin color type to ensure the adaptability of the obtained blue difference component value range and the red difference component value range to the initial facial image.
[0078] The facial skin color type may refer to the facial skin color type in the initial facial image, such as yellow skin, white skin, black skin, etc., and there is no limitation on this.
[0079] In the embodiment of the present disclosure, when determining the blue difference component value range and the red difference component value range according to the facial skin color type, the blue difference component value range and the red difference component value range corresponding to the above-mentioned facial skin color type can be determined from a preset relationship table.
[0080] Optionally, in some embodiments, when determining the blue difference component value range and the red difference component value range according to the facial skin color type, the following steps may be performed: determining the blue difference component reference value range and the red difference component reference value range according to the facial skin color type; determining the illumination features of the initial facial image; and adjusting the blue difference component reference value range and the red difference component reference value range based on the illumination features to obtain the blue difference component value range and the red difference component value range. Thus, the blue difference component reference value range and the red difference component reference value range may be effectively adjusted in combination with the illumination features of the initial facial image to ensure the robustness of the obtained blue difference component value range and the red difference component value range.
[0081] The reference value range of the blue difference component and the reference value range of the red difference component refer to the component value ranges preliminarily determined based on the facial skin color type. The illumination feature can be used to describe the relevant features of the illumination when the above-mentioned initial facial image is taken, for example, it can include illumination intensity, illumination type, etc., without limitation.
[0082] It is understandable that the skin color of the initial facial image, in addition to being related to the facial skin color type, may also be affected by the light when taking the photo. Therefore, the blue difference component reference value range and the red difference component reference value range can be adjusted based on the lighting characteristics of the initial facial image.
[0083] Optionally, in the embodiment of the present disclosure, after determining the reference value range of the blue difference component and the reference value range of the red difference component according to the facial skin color type, the reference value range of the blue difference component and the reference value range of the red difference component may also be adjusted in combination with any relevant features that may affect the facial skin color, such as the age, gender, and residential area of the user corresponding to the initial facial image, to obtain the blue difference component value range and the red difference component value range, and there is no limitation on this.
[0084] S203: If the blue difference component of the candidate pixel belongs to the blue difference component value range, and the red difference component belongs to the red difference component value range, then determine the candidate pixel as a target pixel, wherein a plurality of target pixels jointly constitute a reference facial region.
[0085] That is, in the disclosed embodiment, the pixel value includes: a blue difference component and a red difference component. After the initial facial image is converted into the YCbCr color space to obtain the target facial image, the blue difference component value range and the red difference component value range of the facial skin color in the initial facial image can be determined; if the blue difference component of the candidate pixel belongs to the blue difference component value range, and the red difference component belongs to the red difference component value range, then the candidate pixel is determined to be the target pixel. In this way, the target pixel can be quickly and accurately screened from multiple candidate pixels, thereby improving the reliability of the obtained target pixel.
[0086] S204: Extract features from the reference facial region to obtain an LBP feature vector and an LPQ feature vector.
[0087] For the description of S204, please refer to the above embodiment, which will not be repeated here.
[0088] S205: concatenate the LBP feature vector and the LPQ feature vector to obtain a high-dimensional feature vector as a target feature vector.
[0089] That is, in the disclosed embodiment, after extracting features from the reference facial region to obtain the LBP feature vector and the LPQ feature vector, the LBP feature vector and the LPQ feature vector can be concatenated to obtain a high-dimensional feature vector as the target feature vector. Thus, the obtained target feature vector can effectively improve the indication integrity of the LBP feature vector and the LPQ feature vector, thereby ensuring the practicability of the obtained target feature vector in the face-changing recognition process.
[0090] S206: Input the target feature vector into a pre-trained classifier model to obtain a face-changing recognition result, wherein the face-changing recognition result is used to indicate whether the initial facial image has been processed by face-changing.
[0091] For the description of S206 , please refer to the above embodiment, which will not be repeated here.
[0092] In this embodiment, the target facial image is obtained by performing YCbCr color space conversion on the initial facial image. Thus, the practicability and applicability of the obtained target facial image for the subsequent feature extraction process can be guaranteed. By determining the blue difference component value range and the red difference component value range of the facial skin color in the initial facial image; if the blue difference component of the candidate pixel belongs to the blue difference component value range, and the red difference component belongs to the red difference component value range, the candidate pixel is determined to be the target pixel. Thus, the target pixel can be quickly and accurately screened from multiple candidate pixels, thereby improving the reliability of the obtained target pixel. By splicing the LBP feature vector and the LPQ feature vector, a high-dimensional feature vector is obtained as the target feature vector. Thus, the indication integrity of the obtained target feature vector for the LBP feature vector and the LPQ feature vector can be effectively improved, thereby ensuring the practicability of the obtained target feature vector in the face-changing recognition process.
[0093] Optionally, the present disclosure also proposes a face-swapping recognition method, wherein the classifier model can be trained based on the following method: constructing an initial machine learning classifier; obtaining a sample data set according to the facial skin color type, wherein the sample data set includes a plurality of positive sample facial images and a plurality of negative sample facial images, the positive sample facial images have not been subjected to face-swapping processing, and the negative sample facial images have been subjected to face-swapping processing; training the initial machine learning classifier based on the sample data set to obtain a classifier model. In this way, the training cost of the classifier model can be reduced, and the practicability of the obtained classifier model can be ensured.
[0094] For example, in the embodiment of the present disclosure, the classifier model can be trained based on the following method:
[0095] 1. Data Collection:
[0096] Positive facial images: Collect facial images that have not been processed by face replacement.
[0097] Negative facial images: Collect facial images that have been manipulated (e.g., images generated using deepfake technology).
[0098] 2. Data Preprocessing
[0099] Image cleaning: Remove blurry, unclear or noisy images.
[0100] Labeling: Labeling each image as “real” or “fake”.
[0101] Normalization: Scale the images to the same size and normalize the pixel values.
[0102] 3. Feature extraction
[0103] Use deep learning models such as convolutional neural networks (CNN) to automatically extract image features, or use traditional methods (such as SIFT, HOG) to manually extract features.
[0104] 4. Model selection
[0105] Choose a suitable classifier model, for example: CNN (such as ResNet, Inception, etc.), support vector machine (SVM), random forest, XGBoost, etc.
[0106] 5. Model Training
[0107] Split the dataset into a training set and a validation set (usually 70% training, 30% validation).
[0108] Use the cross entropy loss function to train the model and adjust the hyperparameters (learning rate, batch size, etc.).
[0109] Perform multiple rounds of training and monitor metrics such as precision and recall.
[0110] 6. Model Evaluation
[0111] Evaluate model performance on the validation set, view confusion matrix, ROC curve, etc. Perform hyperparameter tuning to improve model performance.
[0112] 7. Testing and Application
[0113] Evaluate the performance of the final model on the test set. Deploy the model to detect whether a face image has been manipulated in real time.
[0114] 8. Keep updating
[0115] The dataset and models are updated regularly to adapt to emerging face-changing technologies.
[0116] Based on the above embodiments, the present disclosure can perform face-changing recognition as follows: Figure 3 As shown, Figure 3 The following is a schematic diagram of the face-changing recognition process proposed in the present disclosure, which includes the following steps:
[0117] 1. Image preprocessing
[0118] Input an undetected RGB image and convert the input RGB image to YCbCr color space.
[0119] Apply the transformation formula to each pixel:
[0120] Y=0.299R+0.587G+0.114B
[0121] Cb=128-0.168736R-0.331264G+0.5B
[0122] Cr=128+0.5R-0.418688G-0.081312B
[0123] Detect face regions in YCbCr color space using a predefined skin color range.
[0124] The facial area is detected according to the following ranges: 100≤Cb≤120; 140≤Cr≤175.
[0125] Mark the detected suspected facial area (i.e. the reference facial area mentioned above), traverse each pixel of the image, and determine whether the Cb and Cr values of the pixel are within the predefined skin color range. If the Cb and Cr values of the pixel are within the skin color range, the pixel is marked as 1, otherwise, the pixel is marked as 0. Through the above steps, a binary mask image of the same size as the original image is generated, where 1 represents the suspected facial area and 0 represents the non-suspected facial area. Morphological operations and connected domain analysis are performed on the generated binary mask image to reduce noise and isolated pixels.
[0126] 2. Texture feature extraction
[0127] Receive the YCbCr image output after image preprocessing and the marked suspected facial area. Apply the LBP algorithm to the suspected facial area. For each pixel, calculate the difference between it and the neighboring pixels. Binarize the difference to generate an 8-bit binary pattern. Convert the binary pattern to decimal to get the LBP value. Calculate the histogram of all LBP values, count the frequency of each value, and form the LBP feature vector.
[0128] The mathematical formula of LBP is as follows:
[0129]
[0130] Where (x c ,y c ) is the center pixel, i c is the gray value, i p is the gray value of adjacent pixels, s is a function symbol
[0131]
[0132] Apply the LPQ algorithm to the suspected facial area and divide the image into local blocks. Perform a short-time Fourier transform (STFT) on each block. Quantify the phase information to obtain the LPQ value, calculate the histogram of all LPQ values, and count the frequency of each value to form the LPQ feature vector.
[0133] The LPQ descriptor is used to process blurry images using a short-time Fourier transform:
[0134]
[0135] Where ω(m, n) is the window function, f(x, y) is the input image, μ m and μ n is the frequency component.
[0136] 3. Feature combination and classification module
[0137] Receive the LBP and LPQ feature vectors output by the texture feature extraction module, and concatenate the LBP feature vector and the LPQ feature vector to form a high-dimensional feature vector. Input the combined high-dimensional feature vector into the trained SVM classifier. The SVM classifier uses the kernel function to calculate the distance between the input feature vector and the support vector. According to the trained model parameters, calculate the decision function value:
[0138]
[0139] Among them, α i is the Lagrange multiplier, y i is the label of the support vector, K(x i +x) is the kernel function and b is the bias term.
[0140] Based on the output of the decision function, determine whether the input image is an AI face-changing image.
[0141] 4. Output the classification results, indicating whether the input image is an AI face-swapped image.
[0142] Figure 4 A schematic diagram of the structure of a face-changing recognition device provided in an embodiment of the present disclosure.
[0143] like Figure 4 As shown, the face recognition device 40 includes:
[0144] The processing module 401 is used to perform color space conversion on the initial facial image to obtain a target facial image, wherein the target facial image includes a plurality of candidate pixels, each candidate pixel having a corresponding pixel value;
[0145] A determination module 402, configured to determine a target pixel from a plurality of candidate pixels according to the pixel value, wherein the plurality of target pixels jointly constitute a reference facial region;
[0146] A feature extraction module 403 is used to extract features from the reference facial region to obtain an LBP feature vector and an LPQ feature vector;
[0147] A feature fusion module 404 is used to perform feature fusion on the LBP feature vector and the LPQ feature vector to obtain a target feature vector;
[0148] The recognition module 405 is used to input the target feature vector into the pre-trained classifier model to obtain a face-changing recognition result, wherein the face-changing recognition result is used to indicate whether the initial facial image has been processed by face-changing.
[0149] It should be noted that the above explanation of the face-changing recognition method is also applicable to the face-changing recognition device of this embodiment and will not be repeated here.
[0150] In this embodiment, the target facial image is obtained by performing color space conversion on the initial facial image, wherein the target facial image includes multiple candidate pixels, each candidate pixel has a corresponding pixel value; the target pixel is determined from the multiple candidate pixels according to the pixel value, wherein the multiple target pixels jointly constitute a reference facial region; the reference facial region is subjected to feature extraction to obtain an LBP feature vector and an LPQ feature vector; the LBP feature vector and the LPQ feature vector are subjected to feature fusion to obtain a target feature vector; the target feature vector is input into a pre-trained classifier model to obtain a face-changing recognition result, wherein the face-changing recognition result is used to indicate whether the initial facial image has been subjected to face-changing processing. Thus, the face-changing recognition cost can be effectively reduced while ensuring the accuracy of the face-changing recognition result.
[0151] As an example, Figure 5 is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of the present disclosure, such as Figure 5 As shown, the electronic device 500 may further include:
[0152] The memory 510 and the processor 520, a bus 530 connecting different components (including the memory 510 and the processor 520), the memory 510 stores a computer program, and when the processor 520 executes the program, the face-changing recognition method of the embodiment of the present disclosure is implemented.
[0153] Bus 530 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.
[0154] The electronic device 500 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the electronic device 500, including volatile and non-volatile media, removable and non-removable media.
[0155] The memory 510 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 540 and / or cache memory 550. The electronic device 500 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 560 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 530 via one or more data medium interfaces. The memory 510 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.
[0156] A program / utility 580 having a set (at least one) of program modules 570 may be stored, for example, in the memory 510, such program modules 570 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 570 generally perform the functions and / or methods of the embodiments described in the present disclosure.
[0157] The electronic device 500 may also communicate with one or more external devices 590 (e.g., keyboard, pointing device, display 591, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 500, and / or communicate with any device that enables the electronic device 500 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed via an input / output (I / O) interface 592. In addition, the electronic device 500 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) via a network adapter 593. As shown, the network adapter 593 communicates with other modules of the electronic device 500 via a bus 530. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.
[0158] The processor 520 executes various functional applications and data processing by running the programs stored in the memory 510 .
[0159] It should be noted that the implementation process and technical principles of the electronic device of this embodiment refer to the aforementioned explanation of the face-changing recognition method of the embodiment of the present disclosure, and will not be repeated here.
[0160] In order to implement the above embodiments, the present disclosure also proposes an electronic device, including: a processor, and a memory communicatively connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method provided by the above embodiments.
[0161] In order to implement the above embodiments, the present disclosure further proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the methods provided by the above embodiments.
[0162] In order to implement the above embodiments, the present disclosure also proposes a computer program product, including a computer program, which implements the methods provided by the above embodiments when executed by a processor.
[0163] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this disclosure shall comply with the relevant laws and regulations and shall not violate public order and good morals.
[0164] It should be noted that personal information from users should be collected for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection / sharing should be carried out after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign the agreement / authorization including authorization of relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others who have access to personal information data comply with its privacy policy and procedures.
[0165] The present disclosure anticipates providing implementation schemes for users to selectively block the use or access of personal information data. That is, the present disclosure anticipates providing hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, risks can be minimized by limiting data collection and deleting the data. In addition, when applicable, such personal information is de-identified to protect the privacy of the user.
[0166] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.
[0167] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, "plurality" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.
[0168] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present disclosure belong.
[0169] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.
[0170] It should be understood that the various parts of the present disclosure can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.
[0171] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.
[0172] In addition, each functional unit in each embodiment of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.
[0173] The storage medium mentioned above may be a read-only memory, a disk or an optical disk, etc. Although the embodiments of the present disclosure have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations of the present disclosure. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present disclosure.
Claims
1. A face-changing recognition method, characterized in that: The following steps are involved: Performing color space conversion on the initial facial image to obtain a target facial image, wherein the target facial image includes a plurality of candidate pixels, each of the candidate pixels having a corresponding pixel value; Determine a target pixel from the plurality of candidate pixels according to the pixel value, wherein the plurality of target pixels jointly constitute a reference facial region; Performing feature extraction on the reference facial region to obtain an LBP feature vector and an LPQ feature vector; Performing feature fusion on the LBP feature vector and the LPQ feature vector to obtain a target feature vector; The target feature vector is input into a pre-trained classifier model to obtain a face-changing recognition result, wherein the face-changing recognition result is used to indicate whether the initial facial image has been subjected to face-changing processing.
2. The method according to claim 1, characterized in that The step of performing color space conversion on the initial facial image to obtain a target facial image includes: The initial facial image is converted into a YCbCr color space to obtain the target facial image.
3. The method according to claim 2, characterized in that The pixel value includes: a blue difference component and a red difference component; Wherein, determining the target pixel from the plurality of candidate pixels according to the pixel value comprises: Determine a blue difference component value range and a red difference component value range of facial skin color in the initial facial image; If the blue difference component of the candidate pixel belongs to the blue difference component value range, and the red difference component belongs to the red difference component value range, the candidate pixel is determined to be the target pixel.
4. The method according to claim 3, characterized in that: The step of determining a blue difference component value range and a red difference component value range of the facial skin color in the initial facial image includes: Determining a facial skin color type of the initial facial image; The blue difference component value range and the red difference component value range are determined according to the facial skin color type.
5. The method according to claim 4, characterized in that The step of determining the blue difference component value range and the red difference component value range according to the facial skin color type includes: Determining a reference value range of a blue difference component and a reference value range of a red difference component according to the facial skin color type; determining lighting characteristics of the initial facial image; The blue difference component reference value range and the red difference component reference value range are adjusted based on the illumination characteristics to obtain the blue difference component value range and the red difference component value range.
6. The method according to claim 1, characterized in that The performing feature fusion on the LBP feature vector and the LPQ feature vector to obtain a target feature vector includes: The LBP feature vector and the LPQ feature vector are concatenated to obtain a high-dimensional feature vector as the target feature vector.
7. The method according to claim 4, characterized in that The method further comprises: Build an initial machine learning classifier; According to the facial skin color type, a sample data set is obtained, wherein the sample data set includes a plurality of positive sample facial images and a plurality of negative sample facial images, the positive sample facial images have not been subjected to face-swapping processing, and the negative sample facial images have been subjected to face-swapping processing; The initial machine learning classifier is trained based on the sample data set to obtain the classifier model.
8. A face recognition device, characterized in that: include: A processing module, configured to perform color space conversion on the initial facial image to obtain a target facial image, wherein the target facial image includes a plurality of candidate pixels, each of the candidate pixels having a corresponding pixel value; a determination module, configured to determine a target pixel from the plurality of candidate pixels according to the pixel value, wherein the plurality of target pixels jointly constitute a reference facial region; A feature extraction module, used for extracting features from the reference facial region to obtain an LBP feature vector and an LPQ feature vector; A feature fusion module, used for performing feature fusion on the LBP feature vector and the LPQ feature vector to obtain a target feature vector; The recognition module is used to input the target feature vector into a pre-trained classifier model to obtain a face-changing recognition result, wherein the face-changing recognition result is used to indicate whether the initial facial image has been processed by face-changing.
9. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.
11. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 7 when being executed by a processor.