Video frame face image processing method and system based on fuzzy factors

Abstract

The invention discloses a fuzzy factor-based video frame face image processing method and system. The method comprises the following operation methods: acquiring continuous frames of face images to be detected; preprocessing the to-be-detected face image to obtain a to-be-detected face grayscale image; obtaining a gray value of each pixel point in the to-be-detected face gray image; calculating a clear region image of the to-be-detected face grayscale image according to the grayscale value of each pixel point in the to-be-detected face grayscale image; matching key features in sample face images in a face database by using the key features in the clear area image to obtain original key features of the original face image; and performing verification identification detection on the face image to be detected based on the original key features of the original face image and the key features in the clear area image.

Description

Technical Field

[0001] The present invention relates to video images, and in particular to a video frame face image processing method and system based on fuzzy factors. Background Art

[0002] With the rapid development of face recognition technology, face image processing in video frames has been widely used in many fields such as security monitoring. However, in practical applications, due to the complexity of the video capture environment, such as lighting changes, motion blur and occlusion, the quality of face images greatly affects the accuracy of subsequent processing.

[0003] Traditional face detection and recognition methods mostly rely on static image processing and fail to fully utilize the dynamic information in video sequences. Such methods often show shortcomings when faced with complex scenes.

[0004] In recent years, the advancement of deep learning technology has brought new opportunities for face image processing. Recognizing face images in continuous video frames can significantly improve the accuracy of face detection and recognition. However, existing methods still need to solve some feature ambiguity problems in video frame processing, such as dynamic changes and blurring of facial features. Summary of the invention

[0005] The purpose of the present invention is to provide a video frame face image processing method and system based on fuzzy factors, which solves the above technical problems pointed out in the prior art.

[0006] The present invention provides a video frame face image processing method based on fuzzy factor, comprising the following steps:

[0007] Acquire continuous frames of face images to be detected; preprocess the face images to be detected to obtain grayscale images of the face to be detected; and obtain the grayscale value of each pixel in the grayscale image of the face to be detected;

[0008] Calculating a clear area image of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected;

[0009] Using the key features in the clear area image to match the key features in the sample face image in the face database, to obtain the original key features of the original face image;

[0010] The key features in the clear area image are nose key features, eye key features, eyebrow key features, mouth key features and face key features;

[0011] The face image to be detected is verified and identified based on the original key features of the original face image and the key features in the clear area image.

[0012] Preferably, a clear area image of the grayscale image of the face to be detected is calculated according to the grayscale value of each pixel in the grayscale image of the face to be detected, and the key features in the clear area image are matched with the key features in the sample face images in the face database to obtain the original key features of the original face image. The specific operation steps are as follows:

[0013] Calculating a fuzzy factor of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected;

[0014] A clear area image is acquired based on the blur factor.

[0015] Preferably, the fuzzy factor of the grayscale image of the face to be detected is calculated according to the grayscale value of each pixel in the grayscale image of the face to be detected, and the specific operation steps are as follows:

[0016] Performing candidate frame selection on the grayscale image of the face to be detected to obtain multiple candidate frames of the face to be detected;

[0017] The fuzzy factor is obtained by the grayscale value range of each pixel point inside the candidate frame of the face to be detected and the distance between adjacent pixels;

[0018] The fuzzy factor is the fuzzy pixel in the face image to be detected;

[0019] The internal fuzzy factor is calculated for each candidate face frame to be detected; the calculation formula of the fuzzy factor is:

[0020]

[0021] Where W n It represents the fuzzy factor of the nth candidate face frame to be detected in the grayscale image of the face to be detected, r represents the grayscale value of the pixel; Vr represents the number of pixels with grayscale value r, and V represents the total number of pixels inside the candidate face frame to be detected.

[0022] Preferably, the clear area image is obtained based on the fuzzy factor, and the specific operation steps are as follows:

[0023] Preset a fuzzy factor neighborhood distance r; calculate the distance of each fuzzy factor inside each candidate frame of the face to be detected, and determine whether the distance between each fuzzy factor is less than or equal to the fuzzy factor neighborhood distance r;

[0024] If so, the fuzzy factor density is obtained;

[0025] If not, the discrete fuzzy factors in the fuzzy factors inside the candidate frame of the face to be detected are discarded;

[0026] Preset a fuzzy factor density threshold; determine whether the fuzzy factor density of each candidate face frame to be detected is greater than or equal to the preset fuzzy factor density threshold;

[0027] If yes, retain the candidate frames of the face to be detected; merge the retained candidate frames of the face to be detected to obtain multiple blurred area images;

[0028] If not, the candidate frame of the face to be detected that is smaller than the preset fuzzy factor density threshold is extracted to obtain a clear area image.

[0029] Preferably, the fuzzy factor is obtained by the grayscale value range of each pixel point inside the candidate frame of the face to be detected and the distance between adjacent pixels, including the following steps:

[0030] Preset the gray value range threshold q and adjacent distance threshold w;

[0031] Determine whether the grayscale value of the pixel point inside the candidate frame of the face to be detected is within the grayscale value range threshold q, and at the same time determine whether the distance between each pixel point is less than the adjacent distance threshold w;

[0032] If not, it is determined that the candidate frame of the face to be detected has a fuzzy factor;

[0033] If so, it is determined that the candidate frame of the face to be detected does not have a fuzzy factor.

[0034] Preferably, the key features in the clear area image are matched with the key features in the sample face image in the face database to obtain the original key features of the original face image. The specific operation steps are as follows:

[0035] Extracting local key features corresponding to the key features in the clear area image, and calculating the edge strength of the local key features;

[0036] The clear area image is divided into groups according to the edge strength of the local key features corresponding to the key features, and corresponding sharpening processing is performed on different divided groups to obtain an optimized clear area image;

[0037] Performing a key point extraction operation on the key features of the optimized clear area image to obtain a first target key point; and determining the facial coordinates of the first target key point, and connecting the extracted facial coordinates of multiple first target key points to form a second target key point contour line;

[0038] Extracting original key features from sample face images in a face database, performing key point extraction operations on the original key features to obtain original key points; connecting the original key points of the sample face images in the face database to form a key point contour line;

[0039] Matching the second target key point contour line formed by the first target key point inside the optimized clear area image with the key point contour line formed by the original key point of the sample face image in the face database to calculate the key point contour line shape similarity;

[0040] According to the key point contour shape similarity, a sample face image corresponding to the highest key point contour shape similarity is selected as the original face image; and key features inside the original face image are obtained.

[0041] Preferably, the clear area image is divided into groups according to the edge strength of the local key features corresponding to the key features, and corresponding sharpening processing is performed on different divided groups to obtain an optimized clear area image. The specific operation steps are as follows:

[0042] Acquire all pixel points within the key features in each of the clear area images, and calculate the edge strength of each pixel point within the key features;

[0043] Obtain the total number Z of all pixels in the key feature of the current clear area image, divide the total number of pixels into N groups, each group contains Z / N consecutive adjacent pixels, and calculate the average edge strength P of adjacent consecutive adjacent pixels in each group;

[0044] According to the average edge intensity threshold value P', the key features are screened and divided into a group formed by continuous adjacent pixels with high edge intensity and a group formed by continuous adjacent pixels with low edge intensity, and then the continuous adjacent pixels with high edge intensity are sharpened and optimized to obtain the optimized local key features; the local key features of the continuous adjacent pixels with low edge intensity are retained without processing; finally, all the key features in the clear area image are optimized as described above, and naturally the final optimized overall clear area image is obtained.

[0045] Preferably, verification and recognition detection is performed on the face image to be detected based on the original key features of the original face image and the key features in the clear area image, and the specific operation steps are as follows:

[0046] Cutting the original key features inside the original face image of the coordinates corresponding to the key features of the blurred area image in the face image to be detected to obtain multiple original key features of the original face images with the same blurred area image positions;

[0047] The original key features of the original face image at the same position are replaced and merged with the multiple fuzzy area images to obtain the key features of the new face image to be detected;

[0048] Preset a matching degree threshold z; calculate the matching degree between the key features of the new face image to be detected and the original key features of the original face image, and determine whether the matching degree is greater than the matching degree threshold z;

[0049] If yes, then the original key features of the original face image are determined to be the key features of the face image to be detected;

[0050] If not, the face database is manually updated for the face image to be detected.

[0051] Accordingly, the present invention also proposes a video frame face image processing system based on fuzzy factors, comprising: an acquisition module; a detection module; a recognition module;

[0052] The acquisition module is used to acquire continuous frames of face images to be detected; preprocess the face images to be detected to obtain grayscale images of the face to be detected; and obtain the grayscale value of each pixel in the grayscale image of the face to be detected;

[0053] The detection module is used to calculate the clear area image of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected, and use the key features in the clear area image to match the key features in the sample face image in the face database to obtain the original key features of the original face image;

[0054] The recognition module is used to perform verification and recognition detection on the face image to be detected based on the original key features of the original face image and the key features in the clear area image.

[0055] Compared with the prior art, the embodiments of the present invention have at least the following technical advantages:

[0056] By analyzing the above-mentioned video frame face image processing method based on fuzzy factors provided by the present invention, it can be known that in specific applications, by selecting candidate frames for the grayscale image of the face to be detected, the amount of calculation of the overall fuzzy factor in the grayscale image of the face to be detected can be reduced, thereby quickly performing detection; when the grayscale values ​​of the pixels inside the multiple candidate frames of the face to be detected obtained by the candidate frames are large or the distance between the pixels is far, the image of the face to be detected will be blurred, thereby causing the appearance of fuzzy factors; by calculating the distance between fuzzy factors, the fuzzy factor density in each candidate frame of the face to be detected can be obtained, and the discrete fuzzy factors with a long distance can be removed by the fuzzy factor density, thereby increasing the speed of face detection;

[0057] Furthermore, a fuzzy factor density threshold is set to select candidate frames of faces to be detected with high fuzzy factor density and merge them as fuzzy area images; and candidate frames of faces to be detected with low fuzzy factor density are selected as clear area images; in this way, it is possible to distinguish which are fuzzy areas and clear areas in the face image to be detected;

[0058] Furthermore, the edge strength of the pixel points in the clear image area is calculated, that is, the clear area image is divided into groups according to the edge strength of the local key features corresponding to the key features, and corresponding sharpening processing is performed on different divided groups to obtain an optimized clear area image; the first target key points are extracted from the key features in the optimized clear area image, and the first target key points are connected to obtain the second target key point contour line; and at the same time, the original key features are extracted from the sample face images in the face database, and the original key points are extracted and connected to obtain the key point contour line, so that the second target key point contour line and the key point contour line are matched for similarity, so as to obtain the original face image with the highest similarity, so that the original face image with the highest similarity corresponding to the face image to be detected can be obtained most quickly, and the face features to be detected are determined by the original key features of the original face image, so that the information of the face image to be detected can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0060] Figure 1 The overall process method of a video frame face image processing method based on fuzzy factors provided in the first embodiment of the present invention;

[0061] Figure 2 A flowchart of obtaining a clear area image of a video frame face image processing method based on a fuzzy factor provided in Embodiment 1 of the present invention;

[0062] Figure 3 A schematic diagram of a clear area image and a blurred area image of a video frame face image processing method based on a fuzzy factor provided in the first embodiment of the present invention;

[0063] Figure 4 A flowchart of key point contour shape similarity matching of a video frame face image processing method based on fuzzy factors provided in the first embodiment of the present invention;

[0064] Figure 5A flowchart of an optimized clear area image of a video frame face image processing method based on a fuzzy factor provided in Embodiment 1 of the present invention;

[0065] Figure 6 A flowchart of a video frame face image processing system based on fuzzy factors provided in Embodiment 2 of the present invention;

[0066] Marking: acquisition module 10; detection module 20; verification module 30. DETAILED DESCRIPTION

[0067] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0068] The present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings.

[0069] Embodiment 1

[0070] like Figure 1 As shown, the present invention proposes a video frame face image processing method based on fuzzy factor, comprising the following steps:

[0071] S10: Acquire continuous frames of face images to be detected; preprocess the face images to be detected to obtain grayscale images of the face to be detected; and obtain the grayscale value of each pixel in the grayscale image of the face to be detected;

[0072] It should be noted that the roadside camera randomly shoots pedestrians on the roadside to form a continuous frame image of the video, and the face image to be detected is obtained by shooting the pedestrians on the roadside through the continuous frame image of the video (that is, the pedestrians on the roadside shot by the camera can be a single pedestrian or multiple pedestrians, so the face in the face image to be detected can also be a single face or multiple faces); the face image to be detected is preprocessed, and the preprocessing includes denoising, filtering, and grayscale conversion to obtain a grayscale image of the face to be detected; then the grayscale value of each pixel in the grayscale image of the face to be detected is obtained, so as to prepare for the subsequent detection of the face;

[0073] S20: Calculating a clear area image of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected;

[0074] S30: using the key features in the clear area image to match the key features in the sample face image in the face database to obtain the original key features of the original face image;

[0075] It should be noted that when roadside cameras shoot pedestrians on the roadside, the continuous frames of the video may not be clear due to the sunlight or the dark night. In addition, the roadside cameras may also have a blurring effect when shooting walking pedestrians, resulting in fewer clear parts of the face and more blurred areas (that is, the above-mentioned sunlight, the dark night and the blurring of the roadside cameras will cause the acquired face image to be blurred (that is, when pedestrians are walking, their bodies and heads may shake, and when they shake, the roadside cameras will shoot relatively blurred continuous frames of video)).

[0076] Therefore, the grayscale value of the grayscale image of the face to be detected is calculated to calculate the clear area image of the facial features, and then the clear area image is compared with the sample face key features of the sample face image in the database to calculate the similarity of the face key features; thereby determining the original key features of the original face image corresponding to the blurred area of ​​the face image to be detected (the key features include nose key features, eye key features, eyebrow key features, mouth key features and face key features);

[0077] S40: Verify and identify the face image to be detected based on the original key features of the original face image and the key features in the clear area image (the above S20 is the first recognition operation, identifying the identity of the original face in the face database that can be recognized by the clear area image. In S30, a new face image to be detected is constructed using the key features on the original face image corresponding to the known fuzzy area image and the clear area image. Then, a secondary recognition is performed based on the new face image to be detected and the original face image to verify whether the recognition result of the above S30 is accurate. After obtaining a definite detection result, the identity information of the person corresponding to the face image to be detected is also recognized. In addition, if the verification fails, the cloud platform feedback personnel perform manual operation of face recognition).

[0078] It should be noted that the above-mentioned embodiment of the present application calculates the similarity between the clear area image and each sample face image in the face database, and then obtains a sample face image with the highest similarity, and the sample face image is determined to be the original face image. Then, according to the position of the fuzzy area image in the face image to be detected, the area image at the corresponding position on the original face image is found, key features are collected for it, and then the key features are replaced with the position of the above-mentioned fuzzy area image, so as to obtain a new face image to be detected (that is, the key features at the corresponding position of the fuzzy area image of the face image to be detected in the original face image, and the key features at the corresponding position of the original face image and the clear area image in the face image to be detected jointly constitute a new face image to be detected), and then the new face image to be detected is used to perform re-identification and verification with the above-mentioned original face image to determine whether the new face image to be detected is completely consistent with the original face image or exceeds the matching degree threshold z. After obtaining a definite judgment result, it is detected that the identity of the person corresponding to the face image to be detected is the identity of the person corresponding to the original face image in the face database.

[0079] Specifically, Figure 2 As shown, in step S20, the clear area image of the grayscale image of the face to be detected is calculated according to the grayscale value of each pixel in the grayscale image of the face to be detected. The specific operation steps are as follows:

[0080] Calculating a fuzzy factor of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected;

[0081] Then a clear area image is acquired based on the blur factor.

[0082] The fuzzy factor of the grayscale image of the face to be detected is calculated according to the grayscale value of each pixel in the grayscale image of the face to be detected; then a clear area image is obtained based on the fuzzy factor, and the specific operation steps are as follows:

[0083] S21: Select candidate frames for the grayscale image of the face to be detected to obtain multiple candidate frames for the face to be detected;

[0084] S22: Obtain a fuzzy factor according to the grayscale value range of each pixel point inside the candidate frame of the face to be detected and the distance between adjacent pixels (that is, the distance between each adjacent pixel point can reflect the distribution of the pixel points inside each candidate frame of the face to be detected);

[0085] The fuzzy factor is the fuzzy pixel in the face image to be detected;

[0086] Preferably, the fuzzy factor is obtained by the grayscale value range of each pixel point inside the candidate frame of the face to be detected and the distance between adjacent pixels, including the following steps:

[0087] S221: Preset the gray value range threshold q and the adjacent distance threshold w (ie, the adjacent distance threshold w is the maximum distance threshold between pixels);

[0088] S222: Determine whether the grayscale value of the pixel point inside the candidate frame of the face to be detected is within the grayscale value range threshold q, and determine whether the distance between each pixel point is less than the adjacent distance threshold w;

[0089] If not (i.e., the grayscale value of the pixel points inside the candidate frame of the face to be detected does not have a grayscale value range threshold q or the distance between each pixel point is greater than or equal to the adjacent distance threshold w, or the grayscale value of the pixel points inside the candidate frame of the face to be detected does not have a grayscale value range threshold q and the distance between each pixel point is greater than or equal to the adjacent distance threshold w), it is determined that the candidate frame of the face to be detected has a fuzzy factor (i.e., if not, it indicates that the grayscale value range of the pixel points inside the candidate frame of the face to be detected is large, and the distribution distance of the pixel points inside the candidate frame of the face to be detected is large);

[0090] If so (i.e., the grayscale values ​​of the pixels inside the candidate frame of the face to be detected have a grayscale value range threshold q and the distance between each pixel is less than the adjacent distance threshold w), then it is determined that the candidate frame of the face to be detected does not have a fuzzy factor (i.e., if so, it indicates that the grayscale value range of the pixels inside the candidate frame of the face to be detected is small, and the distribution distance of the pixels inside the candidate frame of the face to be detected is small);

[0091] It should be noted that the YOLO algorithm is used to generate a candidate frame of size 10*10, and the candidate frame is selected and divided for the grayscale image of the face to be detected, so as to obtain multiple segmented candidate frames of the face to be detected, thereby reducing the pressure of calculating the fuzzy factor for the entire grayscale image of the face to be detected, thereby achieving fast detection;

[0092] When the video frame image is shot, the blurred picture will increase the gray value of the pixel points in the face image to be detected, and due to the blur, the distance between the pixels will also be distributed more (that is, the distribution of the pixels is larger), so a blur factor will appear; the blur factor will cause the face image to be detected to be blurred, and the more the blur factor, the higher the blur degree of the face image to be detected;

[0093] S23: Calculate the internal fuzzy factor of each candidate face frame to be detected; the calculation formula of the fuzzy factor is:

[0094]

[0095] Where W n represents the fuzzy factor of the nth candidate face frame to be detected in the grayscale image of the face to be detected, r represents the grayscale value of the pixel; Vr represents the number of pixels with grayscale value r, and V represents the total number of pixels inside the candidate face frame to be detected;

[0096] S24: Preset a fuzzy factor neighborhood distance r; calculate the distance of each fuzzy factor inside each candidate frame of the face to be detected, and determine whether the distance between each fuzzy factor is less than or equal to the fuzzy factor neighborhood distance r;

[0097] If yes, then the fuzzy factor density is obtained (that is, when calculating the distance between each fuzzy factor, the fuzzy factors are counted at the same time until the distance between all the fuzzy factors is calculated, and the total number of fuzzy factors is obtained, thereby calculating the candidate frame of the face to be detected to obtain the fuzzy factor density);

[0098] If not, the discrete fuzzy factors in the fuzzy factors inside the candidate frame of the face to be detected are discarded;

[0099] It should be noted that by setting the fuzzy factor neighborhood distance, the fuzzy factor inside each candidate frame of the face to be detected is calculated, and the distance distribution between the fuzzy factors can be obtained according to the fuzzy factor neighborhood distance, and the density of the fuzzy factor can be understood through the distance distribution; and at the same time, the discrete fuzzy factors that are not calculated by the neighborhood distance are discarded. This is because when the face image to be detected is blurred, the whole face or a large area of ​​the face image to be detected (i.e., 5 key features such as nose, eyes, eyebrows, mouth, and face, 3 or 4 of which are blurred) is blurred; therefore, the fuzzy factor neighborhood distance can not only understand the blur degree and area of ​​the face image to be detected, but also remove the discrete fuzzy factors, so as to quickly identify the face image to be detected;

[0100] S25: Preset a fuzzy factor density threshold; determine whether the fuzzy factor density of each candidate face frame to be detected is greater than or equal to the preset fuzzy factor density threshold;

[0101] If yes, retain the candidate frames of the face to be detected; merge the retained candidate frames of the face to be detected to obtain multiple blurred area images;

[0102] If not, extract the candidate frames of the face to be detected that are smaller than the preset fuzzy factor density threshold to obtain a clear area image (that is, the extracted part is the clear area image, that is, the part with higher clarity. If the fuzzy factor density of all the candidate frames of the face to be detected is low, the original face image can also be detected from the face to be detected image);

[0103] It should be noted that by setting the fuzzy factor neighborhood distance, the fuzzy factor inside each candidate face frame to be detected is calculated, the number of fuzzy factors inside each candidate face frame to be detected is determined, and the candidate face frame to be detected with a smaller number of fuzzy factors is extracted for subsequent detection; because when detecting face recognition, fewer fuzzy factors result in less face occlusion (i.e., less blur) during recognition, such as the key features of the nose, eyes, eyebrows, mouth, and face;

[0104] like Figure 3 As shown, when the fuzzy factor appears, although the face image to be detected is less blurred, the clarity of the less blurred area can be calculated, thereby obtaining the part of the clear area image;

[0105] Specifically, Figure 3 As shown, in step 30, the key features in the clear area image are matched with the key features in the sample face image in the face database to obtain the original key features of the original face image. The specific operation steps are as follows:

[0106] Calculating facial similarity based on the clear area image and sample facial images in a facial database, obtaining a sample facial image with the highest similarity to the clear area image and using it as an original facial image;

[0107] The original key features of the original face image corresponding to the blurred area image are determined based on the original face image.

[0108] In the specific implementation, the face similarity is calculated based on the clear area image and the sample face images in the face database, and the sample face image with the highest similarity to the clear area image is obtained as the original face image; based on the original face image, the original key features of the original face image corresponding to the fuzzy area image are determined. The specific operation steps are as follows:

[0109] S31: extracting local key features corresponding to the key features in the clear area image, and calculating the edge strength of the local key features;

[0110] The clear area image is divided into groups according to the edge strength of the local key features corresponding to the key features, and corresponding sharpening processing is performed on different divided groups to obtain an optimized clear area image;

[0111] It should be noted that in step S24, it has been explained that the clear area image has a lower fuzzy factor. In order to make the clear image area clearer, the key features of the clear area image are subjected to more subtle edge strength recognition of local key features, and then different forms of edge optimization processing are performed on this basis. A more detailed sharpening processing is performed on the group with a higher average edge strength to make the clear area image clearer; however, no processing is performed on the group with a lower average edge strength.

[0112] S32: performing a key point extraction operation on the key features of the optimized clear area image to obtain a first target key point; and determining the facial coordinates of the first target key point, connecting the extracted facial coordinates of multiple first target key points to form a second target key point contour line (the first target key point is a contour point, the second target key point contour line is a connecting line formed by connecting the first target key points, and further the key point contour line shape similarity refers to the contour shape similarity);

[0113] Extracting original key features from sample face images in a face database, performing key point extraction operations on the original key features to obtain original key points; connecting the original key points of the sample face images in the face database to form a key point contour line;

[0114] S33: matching the second target key point contour line formed by the first target key point inside the optimized clear area image with the key point contour line formed by the original key point of the sample face image in the face database to calculate the key point contour line shape similarity;

[0115] According to the key point contour shape similarity, a sample face image corresponding to the highest key point contour shape similarity is selected as the original face image; and key features inside the original face image are obtained.

[0116] It should be noted that the key features of the clear area inside the optimized clear area image are determined, and the parts of the key features are (such as nose, eyes, eyebrows, mouth, face), and then the key points of the key features are identified by the face recognition algorithm. The key points are the contour points on the eye contour, nose contour, mouth contour, eyebrow contour, and facial contour. Common key points are 68 points or 98 points; the image in the face database is determined to have the same position as the key features of the clear area inside the optimized clear area image (that is, if the optimized clear area image is the eye, then the eye of the image in the face database is found to extract the original key points) to extract the original key points, and the second target key point contour line formed by the first target key point inside the optimized clear area image is matched with the key point contour line formed by the original key point of the sample face image in the face database, and the original face image of the image in the face database with the highest similarity is extracted, and the original key features of the original face image are the key features of the face image to be detected;

[0117] If the key feature inside the optimized clear area image is a single part such as the nose, then it is necessary to determine the shape, size, height and other features (i.e., the features include the nose tip, nose wing, nose bridge, nasolabial groove, nasal root, etc.), and also determine whether the face image to be detected is a frontal image or a side image (i.e., the side image can better reflect the characteristics of the nose, and the shape and height of the nose are more prominent), and extract key points (i.e., a total of 9 key points of the nose), and select the original face image with the highest similarity by comparing the key points of each feature;

[0118] The nose key point is the part with fewer key points in the face image, so if there are other key features in the optimized clear area image (that is, in addition to the nose, the other four key features of the eyes, eyebrows, mouth, and face), it is more likely to accurately identify and match the original face image;

[0119] Specifically, Figure 4 As shown, in step S31, the clear area image is divided into groups according to the edge strength of the local key features corresponding to the key features, and corresponding sharpening processing is performed for different division groups to obtain an optimized clear area image. The specific operation steps are as follows:

[0120] S311: Acquire all pixel points in the key features in each of the clear area images, and calculate the edge strength of each pixel point in the key features (note that at this time, only the pixel points in the key features are identified, rather than the subsequent first target pixel points. The edge strength used to describe the clear area image is calculated by each pixel point in the key features);

[0121] It should be noted that, through the obtained clear area image, it is possible to directly determine whether the key feature inside is the nose, the mouth or other features, and determine the edge strength of each pixel point based on the obtained key feature (that is, the face image to be detected is composed of pixels, and when the key feature is, for example, the mouth, it is also displayed through pixels, so the edge strength of each pixel point in the mouth can be calculated); the edge strength can be calculated by calculating the gradient of the pixel point through the Sobel operator to determine the edge strength of the key feature;

[0122] S312: Obtain the total number Z of all pixels in the key feature of the current clear area image, divide the total number of pixels into N groups, each group contains Z / N consecutive adjacent pixels, and calculate the average edge strength P of the consecutive adjacent pixels in each group;

[0123] S313: According to the average edge intensity threshold P'; the key features are screened and divided into a group formed by continuous adjacent pixels with high edge intensity (i.e., the group with average edge intensity P greater than or equal to the average edge intensity threshold P' is defined as the current group) and a group formed by continuous adjacent pixels with low edge intensity (i.e., the group with average edge intensity P less than the average edge intensity threshold P' is defined as the current group), and then the continuous adjacent pixels with high edge intensity are sharpened and optimized to obtain the optimized local key features; the local key features of the continuous adjacent pixels with low edge intensity are retained without processing; finally, all the key features in the clear area image are optimized as described above, and the final optimized overall clear area image is naturally obtained.

[0124] It should be noted that the pixel points on the local key features formed by the group of continuous adjacent pixel points with high edge strength (i.e., the continuous adjacent pixel points with high edge strength) are subjected to sharpening optimization processing; for the pixel points on the local key features formed by the group of continuous adjacent pixel points with low edge strength (i.e., the continuous adjacent pixel points with high edge strength), it is recommended not to perform any processing, that is, not to perform sharpening optimization processing; the corresponding local key features after optimization processing or without optimization processing are merged to obtain the current key feature;

[0125] Finally, all key features (multiple key features) in the clear area image are identified and optimized or not optimized accordingly, and the final optimized overall clear area image is naturally obtained.

[0126] Explanation: The clear area image may include multiple key features. Then, as far as the key features identified in the current clear area image are concerned, there are also different performances of high edge strength and low edge strength of the edge formed by continuous adjacent pixels. Therefore, it is necessary to identify the division, that is, to identify a section of continuous adjacent pixels with high edge strength and a section of continuous adjacent pixels with low edge strength of the key features, and then perform different optimization processes on the key features with different high edge strengths, and finally obtain the optimized key features. Finally, all the key features in the clear area image are optimized as above, and naturally the final optimized overall clear area image is obtained.

[0127] Explanation: The algorithm analyzes each area in the image and calculates the average edge strength of different groups of pixels divided by a key feature of the area. For continuous pixels with high edge strength (such as the outline of the nose wing), the image enhancement and sharpening processing will be further strengthened during the processing and calculation, because these local key features (i.e. a group of continuous adjacent pixels) need to be enhanced to highlight the details, that is, the clear area image is optimized (the specific optimization method is sharpening processing). For local key features with low edge strength (such as: the outline of the nasolabial groove or the outline of the root of the nose), the sharpening processing will not be strengthened during the processing and calculation to avoid excessive enhancement and increase of noise.

[0128] The above edge strength P includes two aspects, one is the pixel contrast, and the other is the color saturation of the pixel;

[0129] It should be noted that, by calculating the average edge strength of the key features, the edge strength of the pixel points of the key features can be determined and corresponding enhanced optimization processing can be performed, and the local key features composed of a group of continuous adjacent pixel points with high edge strength can be further sharpened (because studies have found that the pixels on the contour of the nose wing are often obvious, with high contrast and strong color saturation, which are often the most representative edges of the nose shape, while the contour of the root of the nose is often not obvious, and the contour of the root of the nose itself has little influence on the shape of the nose), while the local key features composed of a group of continuous adjacent pixel points with low edge strength are not processed.

[0130] Specifically, based on the original key features of the original face image and the key features in the clear area image, the face image to be detected is verified and recognized, and the specific operation steps are as follows:

[0131] Cutting the original key features in the original face image of the coordinates corresponding to the key features of the blurred area image in the face image to be detected to obtain the key features of the original face image with the same position of multiple blurred area images;

[0132] The original key features of the original face image at the same position are replaced and merged with the multiple fuzzy area images to obtain the key features of the new face image to be detected;

[0133] Preset a matching degree threshold z; calculate the matching degree between the key features of the new face image to be detected and the original key features of the original face image, and determine whether the matching degree is greater than the matching degree threshold z;

[0134] If yes, the original face image is determined to be the face image to be detected, and the original key features of the original face image are the key features of the face image to be detected;

[0135] If not, manually update the face database for the face image to be detected;

[0136] It should be noted that the key feature position inside the original face image is cropped (i.e., the position of the key feature of the clear area image optimized in step S251 is cropped), and multiple fuzzy areas are replaced and merged, and the matching degree is calculated between the key features of the new face image to be detected and the original key features of the original face image (i.e., the key points of the key features of the new face image to be detected are extracted and matched with the original key points of the original key features of the original face image, and the key points are matched with the original key points), and it is determined whether there is a mismatch in the face, so that the original face image can be verified, and thus it can also be determined that the original key features of the original face image are the key features of the face image to be detected;

[0137] The new face image to be detected is matched with the original face image. If the matching degree is high, it means that the original face image is the face image to be detected; if the matching degree is low, it means that the face image to be detected and the original face image do not match.

[0138] Embodiment 2

[0139] like Figure 5 As shown, accordingly, the present invention also proposes a video continuous frame image processing system, comprising: an acquisition module 10; a detection module 20; a verification module 30;

[0140] The acquisition module 10 is used to acquire continuous frames of face images to be detected; preprocess the face images to be detected to obtain grayscale images of the face to be detected; and obtain the grayscale value of each pixel in the grayscale image of the face to be detected;

[0141] The detection module 20 is used to calculate the clear area image of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected, and use the key features in the clear area image to match the key features in the sample face image in the face database to obtain the original key features of the original face image;

[0142] The verification module 30 is used to verify and identify the face image to be detected based on the original key features of the original face image and the key features in the clear area image.

[0143] In summary, the video frame face image processing method and system based on fuzzy factors proposed in the example of the present invention can reduce the amount of calculation of the overall fuzzy factor in the grayscale image of the face to be detected by selecting the candidate frame of the grayscale image of the face to be detected, so as to quickly perform the detection; when the grayscale values ​​of the pixels inside the multiple candidate frames of the face to be detected obtained by the candidate frame are large or the distance between the pixels is far, the appearance of the face image to be detected will be blurred, thereby causing the appearance of fuzzy factors; by calculating the distance between the fuzzy factors, the fuzzy factor density in each candidate frame of the face to be detected can be obtained, and the discrete fuzzy factors with a long distance are removed by the fuzzy factor density, thereby increasing the speed of face detection;

[0144] Furthermore, a fuzzy factor density threshold is set to select candidate frames of faces to be detected with high fuzzy factor density and merge them as fuzzy area images; and candidate frames of faces to be detected with low fuzzy factor density are selected as clear area images; in this way, it is possible to distinguish which are fuzzy areas and clear areas in the face image to be detected;

[0145] Furthermore, the edge strength of the pixel points in the clear image area is calculated, and the key features are sharpened in more detail according to the strength of the edge strength, so as to improve the clarity of the clear area and reduce the fuzziness of the local key features; then the first target key points are extracted from the key features in the optimized clear area image, and the first target key points are connected to obtain the second target key point contour line; and at the same time, the original key features are extracted from the sample face images in the face database, and the original key points are extracted and connected to obtain the key point contour line, so as to match the second target key point contour line with the key point contour line for similarity, so as to obtain the original face image with the highest similarity, so as to obtain the original face image with the highest similarity corresponding to the face image to be detected in the fastest way, and the face features to be detected are determined by the original key features of the original face image, so as to obtain the information of the face image to be detected.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. A person skilled in the art may modify the technical solutions described in the above embodiments, or replace part or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video frame face image processing method based on fuzzy factor, characterized in that: The following methods are included: Acquire continuous frames of face images to be detected; preprocess the face images to be detected to obtain grayscale images of the face to be detected; and obtain the grayscale value of each pixel in the grayscale image of the face to be detected; Calculating a clear area image of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected; Using the key features in the clear area image to match the key features in the sample face image in the face database, to obtain the original key features of the original face image; The key features in the clear area image are nose key features, eye key features, eyebrow key features, mouth key features and face key features; The face image to be detected is verified and identified based on the original key features of the original face image and the key features in the clear area image.

2. According to claim 1, a video frame face image processing method based on fuzzy factor is characterized in that: Calculate the clear area image of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected, and the specific operation steps are as follows: Calculating a fuzzy factor of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected; Then a clear area image is acquired based on the blur factor.

3. According to claim 2, a video frame face image processing method based on fuzzy factor is characterized in that: The fuzzy factor of the grayscale image of the face to be detected is calculated according to the grayscale value of each pixel in the grayscale image of the face to be detected. The specific operation steps are as follows: Performing candidate frame selection on the grayscale image of the face to be detected to obtain multiple candidate frames of the face to be detected; The fuzzy factor is obtained by the grayscale value range of each pixel point inside the candidate frame of the face to be detected and the distance between adjacent pixels; The fuzzy factor is the fuzzy pixel in the face image to be detected; The internal fuzzy factor is calculated for each candidate face frame to be detected; the calculation formula of the fuzzy factor is: Where W n It represents the fuzzy factor of the nth candidate face frame to be detected in the grayscale image of the face to be detected, r represents the grayscale value of the pixel; Vr represents the number of pixels with grayscale value r, and V represents the total number of pixels inside the candidate face frame to be detected.

4. According to claim 3, a video frame face image processing method based on fuzzy factor is characterized in that: The clear area image is obtained based on the fuzzy factor, and the specific operation steps are as follows: Preset a fuzzy factor neighborhood distance r; calculate the distance of each fuzzy factor inside each candidate frame of the face to be detected, and determine whether the distance between each fuzzy factor is less than or equal to the fuzzy factor neighborhood distance r; If so, the fuzzy factor density is obtained; If not, the discrete fuzzy factors in the fuzzy factors inside the candidate frame of the face to be detected are discarded; Preset a fuzzy factor density threshold; determine whether the fuzzy factor density of each candidate face frame to be detected is greater than or equal to the preset fuzzy factor density threshold; If yes, retain the candidate frames of the face to be detected; merge the retained candidate frames of the face to be detected to obtain multiple blurred area images; If not, the candidate frame of the face to be detected that is smaller than the preset fuzzy factor density threshold is extracted to obtain a clear area image.

5. The method for processing face images of video frames based on fuzzy factors according to claim 4, characterized in that: The fuzzy factor is obtained by the grayscale value range of each pixel point inside the candidate frame of the face to be detected and the distance between adjacent pixels, including the following steps: Preset the gray value range threshold q and adjacent distance threshold w; Determine whether the grayscale value of the pixel point inside the candidate frame of the face to be detected is within the grayscale value range threshold q, and at the same time determine whether the distance between each pixel point is less than the adjacent distance threshold w; If not, it is determined that the candidate frame of the face to be detected has a fuzzy factor; If so, it is determined that the candidate frame of the face to be detected does not have a fuzzy factor.

6. The method for processing face images of video frames based on fuzzy factors according to claim 5, characterized in that: The key features in the clear area image are matched with the key features in the sample face image in the face database to obtain the original key features of the original face image. The specific operation steps are as follows: Extracting local key features corresponding to the key features in the clear area image, and calculating the edge strength of the local key features; The clear area image is divided into groups according to the edge strength of the local key features corresponding to the key features, and corresponding sharpening processing is performed on different divided groups to obtain an optimized clear area image; Performing a key point extraction operation on the key features of the optimized clear area image to obtain a first target key point; and determining the facial coordinates of the first target key points, and connecting the extracted facial coordinates of the plurality of first target key points to form a second target key point contour line; Extracting original key features from sample face images in a face database, and performing a key point extraction operation on the original key features to obtain original key points; A key point contour line formed by connecting the original key points of the sample face images in the face database; Matching the second target key point contour line formed by the first target key point inside the optimized clear area image with the key point contour line formed by the original key point of the sample face image in the face database to calculate the key point contour line shape similarity; According to the key point contour shape similarity, a sample face image corresponding to the highest key point contour shape similarity is selected as the original face image; and key features inside the original face image are obtained.

7. The method for processing face images of video frames based on fuzzy factors according to claim 6, characterized in that: The clear area image is divided into groups according to the edge strength of the local key features corresponding to the key features, and corresponding sharpening processing is performed on different division groups to obtain an optimized clear area image. The specific operation steps are as follows: Acquire all pixel points within the key features in each of the clear area images, and calculate the edge strength of each pixel point within the key features; Obtain the total number Z of all pixels in the key feature of the current clear area image, divide the total number of pixels into N groups, each group contains Z / N consecutive adjacent pixels, and calculate the average edge strength P of adjacent consecutive adjacent pixels in each group; According to the average edge intensity threshold value P', the key features are screened and divided into a group formed by continuous adjacent pixels with high edge intensity and a group formed by continuous adjacent pixels with low edge intensity, and then the continuous adjacent pixels with high edge intensity are sharpened and optimized to obtain the optimized local key features; the local key features of the continuous adjacent pixels with low edge intensity are retained without processing; finally, all the key features in the clear area image are optimized as described above, and naturally the final optimized overall clear area image is obtained.

8. The method for processing face images of video frames based on fuzzy factors according to claim 7, characterized in that: Based on the original key features of the original face image and the key features in the clear area image, the face image to be detected is verified and recognized, and the specific operation steps are as follows: Cutting the original key features inside the original face image of the coordinates corresponding to the key features of the blurred area image in the face image to be detected to obtain multiple original key features of the original face images with the same blurred area image positions; The original key features of the original face image at the same position are replaced and merged with the multiple fuzzy area images to obtain the key features of the new face image to be detected; Preset matching degree threshold z; Calculating the matching degree between the key features of the new face image to be detected and the original key features of the original face image, and determining whether the matching degree is greater than a matching degree threshold z; If yes, then the original key features of the original face image are determined to be the key features of the face image to be detected; If not, the face database is manually updated for the face image to be detected.

9. A video continuous frame image processing system, characterized in that: include: Get module; Detection module; Identify the module; The acquisition module is used to acquire continuous frames of face images to be detected; pre-process the face images to be detected to obtain grayscale images of the face to be detected; Obtaining the grayscale value of each pixel in the grayscale image of the face to be detected; The detection module is used to calculate the clear area image of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected, and use the key features in the clear area image to match the key features in the sample face image in the face database to obtain the original key features of the original face image; The recognition module is used to perform verification and recognition detection on the face image to be detected based on the original key features of the original face image and the key features in the clear area image.

Images