A video frame face image processing method and system based on fuzzy factor

Abstract

The present invention discloses a video frame face image processing method and system based on fuzzy factors, which includes the following operation methods: obtaining continuous frames of face images to be detected; preprocessing the face images to be detected to obtain a grayscale image of the face to be detected; obtaining the grayscale value of each pixel point in the grayscale image of the face to be detected; calculating a clear area image of the grayscale image of the face to be detected according to the grayscale value of each pixel point in the grayscale image of the face to be detected; using key features in the clear area image to match key features in sample face images in a face database to obtain original key features of the original face image; and verifying and identifying the face image to be detected based on the original key features of the original face image and the key features in the clear area image.

Description

Technical Field

[0001] The present invention relates to video images, and in particular to a video frame face image processing method and system based on fuzzy factors. Background Art

[0002] With the rapid development of face recognition technology, facial image processing in video frames has been widely used in various fields such as security monitoring. However, in practical applications, due to the complexity of the video capture environment, such as lighting changes, motion blur, and occlusion, the quality of facial images greatly affects the accuracy of subsequent processing.

[0003] Traditional face detection and recognition methods mostly rely on static image processing and fail to fully utilize the dynamic information in video sequences. Such methods often show shortcomings when faced with complex scenes.

[0004] Recent advances in deep learning technology have brought new opportunities to facial image processing. Recognizing faces in consecutive video frames can significantly improve the accuracy of face detection and recognition. However, existing methods still need to address some feature ambiguity issues in video frame processing, such as blurring caused by dynamic changes in facial features. Summary of the Invention

[0005] The purpose of the present invention is to provide a video frame face image processing method and system based on fuzzy factors, which solves the above-mentioned technical problems pointed out in the prior art.

[0006] The present invention provides a video frame face image processing method based on fuzzy factors, comprising the following steps:

[0007] Acquire continuous frames of face images to be detected; preprocess the face images to be detected to obtain a grayscale image of the face to be detected; and obtain the grayscale value of each pixel in the grayscale image of the face to be detected;

[0008] Calculating a clear area image of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected;

[0009] Matching key features in sample face images in a face database using key features in the clear area image to obtain original key features of the original face image;

[0010] The key features in the clear area image are nose key features, eye key features, eyebrow key features, mouth key features and face key features;

[0011] The face image to be detected is verified and identified based on the original key features of the original face image and the key features in the clear area image.

[0012] Preferably, a clear region image of the grayscale image of the face to be detected is calculated based on the grayscale value of each pixel in the grayscale image of the face to be detected, and key features in the clear region image are matched with key features in sample face images in a face database to obtain original key features of the original face image. The specific operation steps are as follows:

[0013] Calculating a fuzzy factor of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected;

[0014] A clear area image is acquired based on the blur factor.

[0015] Preferably, the fuzzy factor of the grayscale image of the face to be detected is calculated according to the grayscale value of each pixel in the grayscale image of the face to be detected, and the specific operation steps are as follows:

[0016] Performing candidate frame selection on the grayscale image of the face to be detected to obtain multiple candidate frames of the face to be detected;

[0017] The fuzzy factor is obtained by the grayscale value range of each pixel point inside the candidate frame of the face to be detected and the distance between adjacent pixels;

[0018] The fuzzy factor is the fuzzy pixel in the face image to be detected;

[0019] The internal fuzzy factor of each candidate face frame to be detected is calculated; the calculation formula of the fuzzy factor is:

[0020]

[0021] Where W n It represents the fuzzy factor of the nth candidate face frame to be detected in the grayscale image to be detected, r represents the grayscale value of the pixel; Vr represents the number of pixels with grayscale value r, and V represents the total number of pixels inside the candidate face frame to be detected.

[0022] Preferably, the clear area image is obtained based on the fuzzy factor, and the specific operation steps are as follows:

[0023] Preset the fuzzy factor neighborhood distance r; calculate the distance of each fuzzy factor within each candidate frame of the face to be detected, and determine whether the distance between each fuzzy factor is less than or equal to the fuzzy factor neighborhood distance r;

[0024] If so, the fuzzy factor density is obtained;

[0025] If not, the discrete fuzzy factors in the fuzzy factors inside the candidate frame of the face to be detected are discarded;

[0026] Preset a fuzzy factor density threshold; determine whether the fuzzy factor density of each candidate face frame to be detected is greater than or equal to the preset fuzzy factor density threshold;

[0027] If so, retain the candidate frames of the face to be detected; merge the retained candidate frames of the face to be detected to obtain multiple blurred area images;

[0028] If not, the candidate frame of the face to be detected that is smaller than the preset fuzzy factor density threshold is extracted to obtain a clear area image.

[0029] Preferably, the fuzzy factor is obtained by the grayscale value range of each pixel point inside the candidate frame of the face to be detected and the distance between adjacent pixels, including the following steps:

[0030] Preset grayscale value range threshold q and adjacent distance threshold w;

[0031] Determine whether the grayscale value of the pixel point inside the candidate frame of the face to be detected is within the grayscale value range threshold q, and at the same time determine whether the distance between each pixel point is less than the adjacent distance threshold w;

[0032] If not, it is determined that the candidate frame of the face to be detected has a fuzzy factor;

[0033] If so, it is determined that the candidate frame of the face to be detected does not have a fuzzy factor.

[0034] Preferably, the key features in the clear area image are matched with the key features in the sample face images in the face database to obtain the original key features of the original face image. The specific operation steps are as follows:

[0035] Extracting local key features corresponding to the key features in the clear area image, and calculating the edge strength of the local key features;

[0036] Dividing the clear area image into groups based on the edge strength of local key features corresponding to the key features, and performing corresponding sharpening processing on different divided groups to obtain an optimized clear area image;

[0037] Performing a key point extraction operation on the key features of the optimized clear area image to obtain a first target key point; determining the facial coordinates of the first target key point, and connecting the extracted facial coordinates of multiple first target key points to form a second target key point contour line;

[0038] Extracting original key features from sample face images in a face database, performing a key point extraction operation on the original key features to obtain original key points; connecting the original key points of the sample face images in the face database to form a key point contour line;

[0039] Matching the second target key point contour line formed by the first target key point inside the optimized clear area image with the key point contour line formed by the original key point of the sample face image in the face database to calculate the key point contour line shape similarity;

[0040] According to the key point contour shape similarity, a sample face image corresponding to the highest key point contour shape similarity is selected as the original face image; and key features inside the original face image are obtained.

[0041] Preferably, the clear area image is divided into groups based on the edge strength of local key features corresponding to the key features, and corresponding sharpening processing is performed on different divided groups to obtain an optimized clear area image. The specific operation steps are as follows:

[0042] Acquire all pixel points within the key features in each of the clear area images, and calculate the edge strength of each pixel point within the key features;

[0043] Obtaining the total number Z of all pixels within the key feature of the current clear area image, dividing the total number of pixels into N groups, with each group containing Z / N consecutive adjacent pixels, and calculating the average edge strength P of the adjacent consecutive adjacent pixels in each group;

[0044] According to the average edge intensity threshold P', the key features are screened and divided into a group formed by continuous adjacent pixels with high edge intensity and a group formed by continuous adjacent pixels with low edge intensity, and then the continuous adjacent pixels with high edge intensity are sharpened and optimized to obtain the optimized local key features; the local key features of the continuous adjacent pixels with low edge intensity are retained without processing; finally, all the key features in the clear area image are optimized as described above, and the final optimized overall clear area image is naturally obtained.

[0045] Preferably, verification and recognition detection of the face image to be detected is performed based on the original key features of the original face image and the key features in the clear area image, and the specific operation steps are as follows:

[0046] Cutting the original key features of the original face image at the coordinates corresponding to the key features of the blurred area image in the face image to be detected to obtain multiple original key features of the original face image at the same blurred area image position;

[0047] Replace and merge the original key features of the original face image at the same position with the multiple fuzzy area images to obtain the key features of a new face image to be detected;

[0048] Preset a matching degree threshold z; calculate the matching degree between the key features of the new face image to be detected and the original key features of the original face image, and determine whether the matching degree is greater than the matching degree threshold z;

[0049] If so, the original key features of the original face image are determined to be the key features of the face image to be detected;

[0050] If not, the face database is manually updated for the face image to be detected.

[0051] Accordingly, the present invention also proposes a video frame face image processing system based on fuzzy factors, comprising: an acquisition module; a detection module; a recognition module;

[0052] The acquisition module is used to acquire continuous frames of face images to be detected; preprocess the face images to be detected to obtain a grayscale image of the face to be detected; and obtain the grayscale value of each pixel in the grayscale image of the face to be detected;

[0053] The detection module is used to calculate a clear area image of the grayscale image of the face to be detected based on the grayscale value of each pixel in the grayscale image of the face to be detected, and use the key features in the clear area image to match the key features in the sample face images in the face database to obtain the original key features of the original face image;

[0054] The recognition module is used to perform verification and recognition detection on the face image to be detected based on the original key features of the original face image and the key features in the clear area image.

[0055] Compared with the prior art, the embodiments of the present invention have at least the following technical advantages:

[0056] By analyzing the above-mentioned video frame face image processing method based on fuzzy factors provided by the present invention, it can be known that in specific applications, by selecting candidate frames for the grayscale image of the face to be detected, the amount of calculation of the overall fuzzy factor in the grayscale image of the face to be detected can be reduced, thereby quickly performing detection; when the grayscale values of the pixels inside the multiple candidate frames of the face to be detected obtained by the candidate frames are large or the distances between the pixels are far, the image of the face to be detected will appear blurred, thereby causing the appearance of fuzzy factors; by calculating the distances between the fuzzy factors, the fuzzy factor density within each candidate frame of the face to be detected can be obtained, and the discrete fuzzy factors with a long distance can be removed by using the fuzzy factor density, thereby increasing the speed of face detection;

[0057] Furthermore, a fuzzy factor density threshold is set to select candidate frames of the face to be detected with a high fuzzy factor density and merge them as a fuzzy area image; and candidate frames of the face to be detected with a low fuzzy factor density are selected as a clear area image. In this way, the fuzzy area and the clear area in the face image to be detected can be distinguished;

[0058] Furthermore, the edge strength of the pixel points in the clear image area is calculated, that is, the clear area image is divided into groups according to the edge strength of the local key features corresponding to the key features, and corresponding sharpening processing is performed on different divided groups to obtain an optimized clear area image; the first target key points are extracted from the key features in the optimized clear area image, and the first target key points are connected to obtain the second target key point contour line; and at the same time, the original key features are extracted from the sample face images in the face database, and the original key points are extracted and connected to obtain the key point contour line, so that the second target key point contour line and the key point contour line are matched for similarity, so as to obtain the original face image with the highest similarity, so that the original face image with the highest similarity corresponding to the face image to be detected can be obtained most quickly, and the face features to be detected are determined by the original key features of the original face image, so that the information of the face image to be detected can be obtained. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0060] Figure 1 The overall process method of a video frame face image processing method based on fuzzy factors provided in the first embodiment of the present invention;

[0061] Figure 2 A flowchart of obtaining a clear area image of a video frame face image processing method based on a fuzzy factor provided in Example 1 of the present invention;

[0062] Figure 3 A schematic diagram of a clear area image and a blurred area image of a video frame face image processing method based on a fuzzy factor provided in the first embodiment of the present invention;

[0063] Figure 4 A flowchart of key point contour shape similarity matching of a video frame face image processing method based on fuzzy factors provided in the first embodiment of the present invention;

[0064] Figure 5A flowchart of an optimized clear area image of a video frame face image processing method based on a fuzzy factor provided in the first embodiment of the present invention;

[0065] Figure 6 A flowchart of a video frame face image processing system based on fuzzy factors provided in the second embodiment of the present invention;

[0066] Mark: acquisition module 10; detection module 20; verification module 30. DETAILED DESCRIPTION

[0067] The technical solution of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0068] The present invention will be further described in detail below through specific embodiments in conjunction with the accompanying drawings.

[0069] Example 1

[0070] like Figure 1 As shown, the present invention proposes a video frame face image processing method based on fuzzy factor, which includes the following steps:

[0071] S10: Acquire continuous frames of face images to be detected; preprocess the face images to be detected to obtain a grayscale image of the face to be detected; and obtain the grayscale value of each pixel in the grayscale image of the face to be detected;

[0072] It should be noted that a roadside camera randomly shoots pedestrians on the roadside to form continuous frame images of video, and the face image to be detected is obtained by shooting the pedestrians in the continuous frame images of the video (that is, the pedestrians on the roadside shot by the camera can be a single pedestrian or multiple pedestrians, so the face in the face image to be detected can also be a single face or multiple faces); the face image to be detected is preprocessed, and the preprocessing includes denoising, filtering, and grayscale conversion to obtain a grayscale image of the face to be detected; then the grayscale value of each pixel in the grayscale image of the face to be detected is obtained, thereby preparing for subsequent face detection;

[0073] S20: Calculating a clear area image of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected;

[0074] S30: using the key features in the clear area image to match the key features in the sample face image in the face database to obtain the original key features of the original face image;

[0075] It should be noted that when roadside cameras are filming pedestrians on the roadside, the continuous frames of the video may not be clear due to the sunlight or the darkness of the night. In addition, the roadside cameras may also have a blurring effect when filming pedestrians walking, resulting in fewer clear parts of the face and more blurred areas (that is, the sunlight, the darkness of the night, and the blurring of the roadside cameras will cause the acquired face images to be blurred (that is, when pedestrians walk, their bodies and heads may shake, and when they shake, the roadside cameras will capture relatively blurred continuous frames of video)).

[0076] Therefore, the grayscale value of the grayscale image of the face to be detected is calculated to calculate the clear area image of the facial features, and then the clear area image is compared with the sample face key features of the sample face images in the database to calculate the similarity of the face key features; thereby determining the original key features of the original face image corresponding to the blurred area of the face image to be detected (the key features include nose key features, eye key features, eyebrow key features, mouth key features and facial key features);

[0077] S40: Verify and identify the face image to be detected based on the original key features of the original face image and the key features in the clear area image (the above S20 is the first recognition operation, identifying the identity of the original face in the face database that can be recognized by the clear area image. In S30, a new face image to be detected is constructed using the key features on the original face image corresponding to the known fuzzy area image and the clear area image. Then, a secondary recognition is performed based on the new face image to be detected and the original face image to verify whether the recognition result of the above S30 is accurate. After obtaining a definite detection result, the identity information of the person corresponding to the face image to be detected is also identified. In addition, if the verification fails, the cloud platform feedback personnel perform manual operation of face recognition).

[0078] It should be noted that the above-mentioned embodiment of the present application calculates the similarity between the clear area image and each sample face image in the face database, and then obtains a sample face image with the highest similarity. The sample face image is determined to be the original face image, and then according to the position of the fuzzy area image in the face image to be detected, the area image at the corresponding position on the original face image is found, and the key features are collected. Then, the key features are replaced with the position of the above-mentioned fuzzy area image, thereby obtaining a new face image to be detected (that is, the key features at the position corresponding to the fuzzy area image of the face image to be detected in the original face image, and the key features at the corresponding position of the original face image and the clear area image in the face image to be detected jointly constitute a new face image to be detected). Then, this new face image to be detected is used to perform re-identification and verification with the above-mentioned original face image to determine whether the new face image to be detected is completely consistent with the original face image or exceeds the matching threshold z. After obtaining a definite judgment result, it is detected that the identity of the person corresponding to the face image to be detected is the identity of the person corresponding to the original face image in the face database.

[0079] Specifically, if Figure 2 As shown, in step S20, the clear area image of the grayscale image of the face to be detected is calculated according to the grayscale value of each pixel in the grayscale image of the face to be detected. The specific operation steps are as follows:

[0080] Calculating a fuzzy factor of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected;

[0081] Then, a clear area image is acquired based on the blur factor.

[0082] Calculate the fuzzy factor of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected; then obtain a clear area image based on the fuzzy factor, and the specific operation steps are as follows:

[0083] S21: Select candidate frames for the grayscale image of the face to be detected to obtain multiple candidate frames for the face to be detected;

[0084] S22: Obtain a fuzzy factor based on the grayscale value range of each pixel within the candidate face frame to be detected and the distance between adjacent pixels (i.e., the distance between each adjacent pixel can reflect the distribution of pixels within each candidate face frame to be detected);

[0085] The fuzzy factor is the fuzzy pixel in the face image to be detected;

[0086] Preferably, the fuzzy factor is obtained by the grayscale value range of each pixel point inside the candidate frame of the face to be detected and the distance between adjacent pixels, including the following steps:

[0087] S221: Preset the grayscale value range threshold q and the adjacent distance threshold w (ie, the adjacent distance threshold w is the maximum distance threshold between pixels);

[0088] S222: Determine whether the grayscale value of the pixel point inside the candidate frame of the face to be detected is within the grayscale value range threshold q, and determine whether the distance between each pixel point is less than the adjacent distance threshold w;

[0089] If not (i.e., the grayscale value of the pixel point inside the candidate frame of the to-be-detected face does not exist in the grayscale value range threshold q or the distance between each pixel point is greater than or equal to the adjacent distance threshold w, or the grayscale value of the pixel point inside the candidate frame of the to-be-detected face does not exist in the grayscale value range threshold q and the distance between each pixel point is greater than or equal to the adjacent distance threshold w), then it is determined that the candidate frame of the to-be-detected face has a fuzzy factor (i.e., if not, it indicates that the grayscale value range of the pixel points inside the candidate frame of the to-be-detected face is large, and the distribution distance of the pixel points inside the candidate frame of the to-be-detected face is large);

[0090] If so (i.e., the grayscale values of the pixels inside the candidate frame of the face to be detected satisfy the grayscale value range threshold q and the distance between each pixel is less than the adjacent distance threshold w), then it is determined that the candidate frame of the face to be detected does not have a fuzzy factor (i.e., if so, it indicates that the grayscale value range of the pixels inside the candidate frame of the face to be detected is small, and the distribution distance of the pixels inside the candidate frame of the face to be detected is small);

[0091] It should be noted that the YOLO algorithm generates a candidate frame of size 10*10, and the candidate frame is divided into frames on the grayscale image of the face to be detected to obtain multiple segmented candidate frames of the face to be detected, thereby reducing the pressure of calculating the fuzzy factor of the entire grayscale image of the face to be detected, thereby achieving fast detection;

[0092] When the video frame image is shot, the blurred image will increase the grayscale value of the pixels in the face image to be detected, and due to the blur, the distance between the pixels will also be distributed more widely (that is, the distribution of the pixels is large), so a fuzzy factor will appear; the fuzzy factor will cause the face image to be detected to be blurred. The larger the fuzzy factor, the higher the degree of blur of the face image to be detected.

[0093] S23: Calculate the internal fuzzy factor of each candidate face frame to be detected; the calculation formula of the fuzzy factor is:

[0094]

[0095] Where W n represents the fuzzy factor of the nth candidate face frame to be detected in the grayscale image to be detected, r represents the grayscale value of the pixel; Vr represents the number of pixels with grayscale value r, and V represents the total number of pixels inside the candidate face frame to be detected;

[0096] S24: Preset the fuzzy factor neighborhood distance r; calculate the distance of each fuzzy factor within each candidate frame of the face to be detected, and determine whether the distance between each fuzzy factor is less than or equal to the fuzzy factor neighborhood distance r;

[0097] If yes, then the fuzzy factor density is obtained (i.e., when calculating the distance between each fuzzy factor, the fuzzy factors are counted at the same time until the distance between all fuzzy factors is calculated, and the total number of fuzzy factors is obtained, thereby calculating the candidate frame of the face to be detected to obtain the fuzzy factor density);

[0098] If not, the discrete fuzzy factors in the fuzzy factors inside the candidate frame of the face to be detected are discarded;

[0099] It should be noted that, by setting the fuzzy factor neighborhood distance, the fuzzy factor inside each candidate frame of the face to be detected is calculated, and the distance distribution between the fuzzy factors can be obtained according to the fuzzy factor neighborhood distance, and the density of the fuzzy factors can be understood through the distance distribution; and at the same time, the discrete fuzzy factors that are not calculated by the neighborhood distance are discarded. This is because when the face image to be detected is blurred, the whole face or a large area of the face image to be detected (i.e., 5 key features such as the nose, eyes, eyebrows, mouth, and face, 3 or 4 of which are blurred) are blurred; therefore, the fuzzy factor neighborhood distance can not only be used to understand the blur degree and area of the face image to be detected, but also to remove discrete fuzzy factors, thereby quickly identifying the face image to be detected;

[0100] S25: Preset a fuzzy factor density threshold; determine whether the fuzzy factor density of each candidate face frame to be detected is greater than or equal to the preset fuzzy factor density threshold;

[0101] If so, retain the candidate frames of the face to be detected; merge the retained candidate frames of the face to be detected to obtain multiple blurred area images;

[0102] If not, extract the candidate frames of the face to be detected that are smaller than the preset fuzzy factor density threshold to obtain a clear area image (that is, the extracted part is the clear area image, that is, the part with higher clarity. If the fuzzy factor density of all the candidate frames of the face to be detected is low, the original face image can also be detected in the face to be detected image);

[0103] It should be noted that by setting the fuzzy factor neighborhood distance, the fuzzy factor inside each candidate face frame to be detected is calculated, the number of fuzzy factors inside each candidate face frame to be detected is determined, and the candidate face frame to be detected with a smaller number of fuzzy factors is extracted for subsequent detection; because when detecting face recognition, fewer fuzzy factors result in less face occlusion (i.e., less blur) during recognition, such as the key features of the nose, eyes, eyebrows, mouth, and face;

[0104] like Figure 3 As shown in FIG, when the fuzzy factor appears, although the face image to be detected is less blurred, the clarity of the less blurred area can be calculated, thereby obtaining the part of the clear area image;

[0105] Specifically, if Figure 3 As shown, in step 30, the key features in the clear area image are matched with the key features in the sample face image in the face database to obtain the original key features of the original face image. The specific operation steps are as follows:

[0106] Calculating facial similarity based on the clear area image and sample facial images in a face database, obtaining a sample facial image having the highest similarity to the clear area image and using it as the original facial image;

[0107] The original key features of the original facial image corresponding to the blurred area image are determined based on the original facial image.

[0108] During specific execution, facial similarity is calculated based on the clear area image and sample facial images in the face database, and the sample facial image with the highest similarity to the clear area image is obtained and used as the original facial image; based on the original facial image, the original key features of the original facial image corresponding to the blurred area image are determined. The specific operation steps are as follows:

[0109] S31: extracting local key features corresponding to the key features in the clear area image, and calculating the edge strength of the local key features;

[0110] Dividing the clear area image into groups based on the edge strength of local key features corresponding to the key features, and performing corresponding sharpening processing on different divided groups to obtain an optimized clear area image;

[0111] It should be noted that in step S24, it has been explained that the clear area image has a low fuzzy factor. In order to make the clear image area clearer, the key features of the clear area image are subjected to more subtle local key feature edge strength recognition, and then different forms of edge optimization processing are performed based on this. A more detailed sharpening processing is performed on the group with a higher average edge strength to make the clear area image clearer; however, no processing is performed on the group with a lower average edge strength.

[0112] S32: performing a key point extraction operation on the key features of the optimized clear area image to obtain a first target key point; determining the facial coordinates of the first target key point, and connecting the extracted facial coordinates of multiple first target key points to form a second target key point contour line (the first target key point is a contour point, and the second target key point contour line is a connecting line formed by connecting the first target key points. Further, the key point contour line shape similarity refers to the contour shape similarity);

[0113] Extracting original key features from sample face images in a face database, performing a key point extraction operation on the original key features to obtain original key points; connecting the original key points of the sample face images in the face database to form a key point contour line;

[0114] S33: matching the second target key point contour line formed by the first target key point inside the optimized clear area image with the key point contour line formed by the original key points of the sample face image in the face database to calculate the key point contour line shape similarity;

[0115] According to the key point contour shape similarity, a sample face image corresponding to the highest key point contour shape similarity is selected as the original face image; and key features inside the original face image are obtained.

[0116] It should be noted that the key features of the clear area inside the optimized clear area image are determined, and what part of the key features is (such as nose, eyes, eyebrows, mouth, face), and then the key points of the key features are identified by the face recognition algorithm. The key points are the contour points on the eye contour, nose contour, mouth contour, eyebrow contour, and facial contour. Common key points are 68 points or 98 points; the image in the face database is determined to have the same position as the key features of the clear area inside the optimized clear area image (that is, if the optimized clear area image is the eye, then the eye of the image in the face database is found to extract the original key points) to extract the original key points, and the second target key point contour line formed by the first target key point inside the optimized clear area image is matched with the key point contour line formed by the original key points of the sample face image in the face database, and the original face image of the image in the face database with the highest similarity is extracted, and the original key features of the original face image are the key features of the face image to be detected;

[0117] If the key feature within the optimized clear area image is a single part such as the nose, then it is necessary to determine the shape, size, height and other features (i.e., features include the nose tip, nose wings, nose bridge, nasolabial groove, nasal root, etc.), and also determine whether the face image to be detected is a frontal image or a side image (i.e., the side image can better reflect the characteristics of the nose, and the shape and height of the nose are more prominent), and extract key points (i.e., a total of 9 key points for the nose). By comparing the key points of each feature, the original face image with the highest similarity is selected;

[0118] The nose is a part of the face image with relatively few key points. Therefore, if the optimized clear area image contains other key features (i.e., the other four key features of the eyes, eyebrows, mouth, and face besides the nose), it will be more likely to be accurately recognized and matched with the original face image.

[0119] Specifically, if Figure 4 As shown, in step S31, the clear area image is divided into groups based on the edge strength of the local key features corresponding to the key features, and corresponding sharpening processing is performed on different division groups to obtain an optimized clear area image. The specific operation steps are as follows:

[0120] S311: Acquire all pixels within the key features in each of the clear area images, and calculate the edge strength of each pixel within the key features (note that only the pixels within the key features are identified at this time, not the subsequent first target pixel. The edge strength used to describe the clear area image is calculated by calculating each pixel within the key features);

[0121] It should be noted that the clear area image obtained can be used to directly determine whether the key feature inside is the nose, mouth or other features, and the edge strength of each pixel point can be determined based on the obtained key features (that is, the face image to be detected is composed of pixels, and when the key feature is, for example, the mouth, it is also displayed through pixels, so the edge strength of each pixel point in the mouth can be calculated); the edge strength can be calculated by calculating the gradient of the pixel point using the Sobel operator to determine the edge strength of the key feature;

[0122] S312: Obtain the total number Z of all pixels within the key feature of the current clear area image, divide the total number of pixels into N groups, with each group containing Z / N consecutive adjacent pixels, and calculate the average edge strength P of the consecutive adjacent pixels in each group;

[0123] S313: According to the average edge intensity threshold P', the key features are screened and divided into a group formed by continuous adjacent pixel points with high edge intensity (i.e., the group with average edge intensity P greater than or equal to the average edge intensity threshold P' is defined as the current group) and a group formed by continuous adjacent pixel points with low edge intensity (i.e., the group with average edge intensity P less than the average edge intensity threshold P' is defined as the current group), and then the continuous adjacent pixel points with high edge intensity are sharpened and optimized to obtain the optimized local key features; the local key features of the continuous adjacent pixel points with low edge intensity are retained without processing; finally, all the key features in the clear area image are optimized as described above, and the final optimized overall clear area image is naturally obtained.

[0124] It should be noted that the pixel points on the local key features formed by the group of continuous adjacent pixel points with high edge intensity (i.e., the continuous adjacent pixel points with high edge intensity) are subjected to sharpening optimization processing; the pixel points on the local key features formed by the group of continuous adjacent pixel points with low edge intensity (i.e., the continuous adjacent pixel points with high edge intensity) are recommended not to be subjected to any processing, i.e., not to be subjected to sharpening optimization processing; the corresponding local key features after or without optimization processing are merged to obtain the current key feature;

[0125] Finally, all key features (multiple key features) in the clear area image are identified and optimized or not optimized accordingly, and the final optimized overall clear area image is naturally obtained.

[0126] Explanation: A clear area image may include multiple key features. As far as the key features identified in the current clear area image are concerned, there are also different performances of edges formed by continuous adjacent pixels with high edge strength and edges with low edge strength. Therefore, it is necessary to divide and identify them, that is, to identify a section of continuous adjacent pixels with high edge strength and a section of continuous adjacent pixels with low edge strength as key features, and then perform different optimization processes on key features with different high edge strengths, and finally obtain the optimized key features. Finally, all key features in the clear area image are optimized as above, and naturally the final optimized overall clear area image is obtained.

[0127] Explanation: The algorithm analyzes each area in the image and calculates the average edge strength of different groups of pixels divided by a key feature in the area. For continuous pixels with high edge strength (such as the outline of the nose wing), the image enhancement and sharpening processing will be further strengthened during the processing calculation, because these local key features (i.e. a group of continuous adjacent pixels) need to be enhanced to highlight the details, that is, the image of the clear area is optimized (the specific optimization method is sharpening processing). For local key features with low edge strength (such as the outline of the nasolabial groove or the outline of the root of the nose), the sharpening processing is not strengthened during the processing calculation to avoid excessive enhancement and increased noise.

[0128] The above edge strength P includes two aspects: one is the pixel contrast, and the other is the color saturation of the pixel;

[0129] It should be noted that, by calculating the average edge strength of the key features, the edge strength of the pixel points of the key features can be determined and corresponding enhanced optimization processing can be performed. The local key features composed of a group of continuous adjacent pixel points with high edge strength are further sharpened (because studies have found that the pixels on the contour of the nose wing are often obvious, with high contrast and strong color saturation, which are often the most representative edges of the nose shape. However, the contour of the root of the nose is often not obvious, and the contour of the root of the nose itself has little influence on the shape of the nose), while the local key features composed of a group of continuous adjacent pixel points with low edge strength are not processed.

[0130] Specifically, based on the original key features of the original face image and the key features in the clear area image, the face image to be detected is verified and recognized, and the specific operation steps are as follows:

[0131] Cutting the original key features of the original face image at the coordinates corresponding to the key features of the blurred area image in the face image to be detected to obtain multiple key features of the original face image at the same blurred area image position;

[0132] Replace and merge the original key features of the original face image at the same position with the multiple fuzzy area images to obtain the key features of a new face image to be detected;

[0133] Preset a matching degree threshold z; calculate the matching degree between the key features of the new face image to be detected and the original key features of the original face image, and determine whether the matching degree is greater than the matching degree threshold z;

[0134] If so, the original face image is determined to be the face image to be detected, and the original key features of the original face image are the key features of the face image to be detected;

[0135] If not, manually update the face database for the face image to be detected;

[0136] It should be noted that the key features of the original facial image are cropped (i.e., the positions of the key features of the clear area image optimized in step S251 are cropped), and multiple fuzzy areas are replaced and merged, and the matching degree of the key features of the new facial image to be detected and the original key features of the original facial image is calculated (i.e., the key points of the key features of the new facial image to be detected are extracted and matched with the original key points of the original key features of the original facial image, and the key points are matched with the original key points), and it is determined whether there are any face mismatches. In this way, the original facial image can be verified, and it can also be determined that the original key features of the original facial image are the key features of the facial image to be detected;

[0137] The new face image to be detected is matched with the original face image. If the matching degree is high, it means that the original face image is the face image to be detected; if the matching degree is low, it means that the face image to be detected and the original face image do not match.

[0138] Example 2

[0139] like Figure 5 As shown, accordingly, the present invention also proposes a video continuous frame image processing system, including: an acquisition module 10; a detection module 20; a verification module 30;

[0140] The acquisition module 10 is used to acquire continuous frames of face images to be detected; preprocess the face images to be detected to obtain a grayscale image of the face to be detected; and obtain the grayscale value of each pixel in the grayscale image of the face to be detected;

[0141] The detection module 20 is used to calculate a clear area image of the grayscale image of the face to be detected based on the grayscale value of each pixel in the grayscale image of the face to be detected, and use the key features in the clear area image to match the key features in the sample face images in the face database to obtain the original key features of the original face image;

[0142] The verification module 30 is used to perform verification and recognition detection on the face image to be detected based on the original key features of the original face image and the key features in the clear area image.

[0143] In summary, the video frame face image processing method and system based on fuzzy factors proposed in the examples of the present invention can reduce the amount of calculation of the overall fuzzy factor in the grayscale image of the face to be detected by selecting a candidate frame for the grayscale image of the face to be detected, thereby quickly performing detection; when the grayscale values of the pixels within the multiple candidate frames of the face to be detected obtained by the candidate frame are large or the distance between the pixels is far, the image of the face to be detected will be blurred, resulting in the appearance of fuzzy factors; by calculating the distance between the fuzzy factors, the fuzzy factor density within each candidate frame of the face to be detected can be obtained, and the discrete fuzzy factors with a long distance are removed according to the fuzzy factor density, thereby increasing the speed of face detection;

[0144] Furthermore, a fuzzy factor density threshold is set to select candidate frames of the face to be detected with a high fuzzy factor density and merge them as a fuzzy area image; and candidate frames of the face to be detected with a low fuzzy factor density are selected as a clear area image. In this way, the fuzzy area and the clear area in the face image to be detected can be distinguished;

[0145] Furthermore, the edge strength of the pixels in the clear image area is calculated, and the key features are sharpened in more detail according to the strength of the edge strength, thereby improving the clarity of the clear area and reducing the fuzziness of the local key features; then the first target key points are extracted from the key features in the optimized clear area image, and the first target key points are connected to obtain the second target key point contour line; and at the same time, the original key features are extracted from the sample face images in the face database, and the original key points are extracted and connected to obtain the key point contour line, so that the second target key point contour line and the key point contour line are matched for similarity, thereby obtaining the original face image with the highest similarity, so that the original face image with the highest similarity corresponding to the face image to be detected can be obtained most quickly, and the face features to be detected are determined by the original key features of the original face image, so that the information of the face image to be detected can be obtained.

[0146] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. A person skilled in the art may modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A video frame face image processing method based on fuzzy factor, characterized in that: The following methods are included: Acquire continuous frames of face images to be detected; preprocess the face images to be detected to obtain a grayscale image of the face to be detected; and obtain the grayscale value of each pixel in the grayscale image of the face to be detected; Calculating a clear area image of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected; Matching key features in sample face images in a face database using key features in the clear area image to obtain original key features of the original face image; The key features in the clear area image are the nose key features, the eye key features, the eyebrow key features, and the mouth key features; Performing verification and recognition detection on the face image to be detected based on the original key features of the original face image and the key features in the clear area image; Calculate the clear area image of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected, and the specific operation steps are as follows: Calculating a fuzzy factor of the grayscale image of the face to be detected according to the grayscale value of each pixel in the grayscale image of the face to be detected; Then, a clear area image is acquired based on the blur factor; The fuzzy factor of the grayscale image of the face to be detected is calculated according to the grayscale value of each pixel in the grayscale image of the face to be detected. The specific operation steps are as follows: Performing candidate frame selection on the grayscale image of the face to be detected to obtain multiple candidate frames of the face to be detected; The fuzzy factor is obtained by the grayscale value range of each pixel point inside the candidate frame of the face to be detected and the distance between adjacent pixels; The fuzzy factor is the fuzzy pixel in the face image to be detected; The internal fuzzy factor of each candidate face frame to be detected is calculated; the calculation formula of the fuzzy factor is: Where W n It represents the fuzzy factor of the nth candidate face frame to be detected in the grayscale image to be detected, r represents the grayscale value of the pixel; Vr represents the number of pixels with grayscale value r, and V represents the total number of pixels inside the candidate face frame to be detected.

2. The video frame face image processing method based on fuzzy factor according to claim 1, characterized in that: The clear area image is obtained based on the fuzzy factor. The specific steps are as follows: Preset the fuzzy factor neighborhood distance r; calculate the distance of each fuzzy factor within each candidate frame of the face to be detected, and determine whether the distance between each fuzzy factor is less than or equal to the fuzzy factor neighborhood distance r; If so, the fuzzy factor density is obtained; If not, the discrete fuzzy factors in the fuzzy factors inside the candidate frame of the face to be detected are discarded; Preset a fuzzy factor density threshold; determine whether the fuzzy factor density of each candidate face frame to be detected is greater than or equal to the preset fuzzy factor density threshold; If so, retain the candidate frames of the face to be detected; merge the retained candidate frames of the face to be detected to obtain multiple blurred area images; If not, the candidate frame of the face to be detected that is smaller than the preset fuzzy factor density threshold is extracted to obtain a clear area image.

3. The video frame face image processing method based on fuzzy factor according to claim 2, characterized in that: The fuzzy factor is obtained by the grayscale value range of each pixel point inside the candidate frame of the face to be detected and the distance between adjacent pixels, including the following steps: Preset grayscale value range threshold q and adjacent distance threshold w; Determine whether the grayscale value of the pixel point inside the candidate frame of the face to be detected is within the grayscale value range threshold q, and at the same time determine whether the distance between each pixel point is less than the adjacent distance threshold w; If not, it is determined that the candidate frame of the face to be detected has a fuzzy factor; If so, it is determined that the candidate frame of the face to be detected does not have a fuzzy factor.

4. The video frame face image processing method based on fuzzy factor according to claim 3, characterized in that: The key features in the clear area image are matched with the key features in the sample face image in the face database to obtain the original key features of the original face image. The specific operation steps are as follows: Extracting local key features corresponding to the key features in the clear area image, and calculating the edge strength of the local key features; Dividing the clear area image into groups based on the edge strength of local key features corresponding to the key features, and performing corresponding sharpening processing on different divided groups to obtain an optimized clear area image; Performing a key point extraction operation on the key features of the optimized clear area image to obtain a first target key point; and determining the facial coordinates of the first target key points, and connecting the extracted facial coordinates of the plurality of first target key points to form a second target key point contour line; Extracting original key features from sample face images in a face database, and performing a key point extraction operation on the original key features to obtain original key points; A key point contour line formed by connecting the original key points of the sample face images in the face database; Matching the second target key point contour line formed by the first target key point inside the optimized clear area image with the key point contour line formed by the original key point of the sample face image in the face database to calculate the key point contour line shape similarity; According to the key point contour shape similarity, a sample face image corresponding to the highest key point contour shape similarity is selected as the original face image; and key features inside the original face image are obtained.

5. The video frame face image processing method based on fuzzy factor according to claim 4, characterized in that: The clear area image is divided into groups based on the edge strength of the local key features corresponding to the key features, and corresponding sharpening processing is performed on different division groups to obtain an optimized clear area image. The specific operation steps are as follows: Acquire all pixel points within the key features in each of the clear area images, and calculate the edge strength of each pixel point within the key features; Obtaining the total number Z of all pixels within the key feature of the current clear area image, dividing the total number of pixels into N groups, with each group containing Z / N consecutive adjacent pixels, and calculating the average edge strength P of the adjacent consecutive adjacent pixels in each group; According to the average edge intensity threshold P', the key features are screened and divided into a group formed by continuous adjacent pixels with high edge intensity and a group formed by continuous adjacent pixels with low edge intensity, and then the continuous adjacent pixels with high edge intensity are sharpened and optimized to obtain the optimized local key features; the local key features of the continuous adjacent pixels with low edge intensity are retained without processing; finally, all the key features in the clear area image are optimized as described above, and the final optimized overall clear area image is naturally obtained.

6. The video frame face image processing method based on fuzzy factor according to claim 5, characterized in that: The face image to be detected is verified and recognized based on the original key features of the original face image and the key features in the clear area image. The specific operation steps are as follows: Cutting the original key features of the original face image at the coordinates corresponding to the key features of the blurred area image in the face image to be detected to obtain multiple original key features of the original face image at the same blurred area image position; Replace and merge the original key features of the original face image at the same position with the multiple fuzzy area images to obtain the key features of a new face image to be detected; Preset matching threshold z; Calculating a matching degree between the key features of the new face image to be detected and the original key features of the original face image, and determining whether the matching degree is greater than a matching degree threshold z; If so, the original key features of the original face image are determined to be the key features of the face image to be detected; If not, the face database is manually updated for the face image to be detected.

Images