Face recognition method and device, equipment, storage medium and product

Abstract

The embodiment of the invention provides a face recognition method and device, equipment, a storage medium and a product, and relates to the field of biological recognition. The method comprises the following steps: acquiring a to-be-identified video, and extracting the to-be-identified video according to frames to obtain a plurality of to-be-identified images containing to-be-identified main bodies; performing feature extraction on the plurality of to-be-recognized images to obtain face features of to-be-recognized subjects corresponding to the plurality of to-be-recognized images; and fusing the face features of the to-be-recognized subjects corresponding to the plurality of to-be-recognized images to obtain comprehensive features of the to-be-recognized subjects, and comparing the comprehensive features with a preset feature library to obtain a face recognition result. According to the method provided by the invention, the feature extraction method for the face recognition scene, which has universal usability and higher recognition degree, is provided, the comprehensive features obtained through feature fusion are recognized, a single frame of image is prevented from being interfered, and the accuracy of face recognition is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of biometric identification, and in particular to a face recognition method, device, equipment, storage medium and product. BACKGROUND

[0002] Face recognition is a biometric identification technology based on face feature information of a person, and is mainly used for identity verification and identity confirmation services, and is mostly achieved by analyzing and identifying face features in an image.

[0003] Specifically, face recognition is mostly achieved by collecting an image containing a face through a camera or a camera head, detecting a face in the collected single image, and then performing face recognition on the detected face. This solution is limited by the recognition distance, angle, light, background, etc. of face recognition, resulting in insufficient accuracy of face recognition. SUMMARY

[0004] The present application provides a face recognition method, device, equipment, storage medium and product to solve the technical problem of insufficient accuracy of face recognition.

[0005] In a first aspect, the present application provides a face recognition method, comprising:

[0006] obtaining a to-be-identified video, performing frame-by-frame extraction on the to-be-identified video to obtain a plurality of to-be-identified images containing a to-be-identified subject; the to-be-identified video contains the to-be-identified subject;

[0007] performing feature extraction on the plurality of to-be-identified images to obtain face features of the to-be-identified subject corresponding to the plurality of to-be-identified images;

[0008] fusing the face features of the to-be-identified subject corresponding to the plurality of to-be-identified images to obtain comprehensive features of the to-be-identified subject, comparing the comprehensive features with a preset feature library to obtain a face recognition result.

[0009] In a second aspect, the present application provides a face recognition device, comprising:

[0010] an image acquisition module configured to obtain a to-be-identified video, perform frame-by-frame extraction on the to-be-identified video to obtain a plurality of to-be-identified images containing a to-be-identified subject; the to-be-identified video contains the to-be-identified subject;

[0011] a feature extraction module configured to perform feature extraction on the plurality of to-be-identified images to obtain face features of the to-be-identified subject corresponding to the plurality of to-be-identified images;

[0012] a feature comparison module configured to fuse the face features of the to-be-identified subject corresponding to the plurality of to-be-identified images to obtain comprehensive features of the to-be-identified subject, compare the comprehensive features with a preset feature library to obtain a face recognition result.

[0013] In a third aspect, the present application provides an electronic device, comprising: a processor, and a memory connected to the processor in communication;

[0014] The memory stores computer-executable instructions.

[0015] The processor executes the computer-executable instructions stored in the memory to implement the method of any one of the first aspect.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method of any one of the first aspect.

[0017] In a fifth aspect, the present application provides a computer program product, comprising a computer program, wherein the computer program is executed by a processor to implement the method of any one of the first aspect.

[0018] The face recognition method provided by the present application extracts the to-be-recognized video containing the to-be-recognized subject frame by frame to obtain a to-be-recognized image set, extracts features from each frame of the to-be-recognized image set, fuses the feature values corresponding to each image in the image set to obtain comprehensive features, and determines the result by comparing the comprehensive features, thereby providing a feature extraction method for a face recognition scene, which has universal usability and higher recognition degree. The comprehensive features obtained by feature fusion are used for recognition, which avoids interference of a single frame of image and improves the accuracy of face recognition. BRIEF DESCRIPTION OF DRAWINGS

[0019] The accompanying drawings, which are incorporated into and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0020] Figure 1 Flowchart of the face recognition method provided by the embodiment of the present application Figure 1 ;

[0021] Figure 2 Flowchart of the video processing method provided by the present application for a face recognition, feature comparison, and the like scene

[0022] Figure 3 Flowchart of the face recognition method provided by the embodiment of the present application Figure 2 ;

[0023] Figure 4 Flowchart of the image enhancement provided by the present application

[0024] Figure 5 Structural diagram of the face recognition device provided by the embodiment of the present application

[0025] Figure 6 A structural schematic diagram of an electronic device is provided for an embodiment of the present application.

[0026] The specific embodiments of the present application have been shown through the above-described drawings, and will be described in more detail hereinafter. These drawings and the written description are not intended to restrict the scope of the present application in any way, but to illustrate the concept of the present application for those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0027] The exemplary embodiments will be described in detail herein with reference to the attached drawings. The same or similar components are denoted by the same or similar reference numerals throughout the drawings and the written description, unless otherwise indicated. The embodiments described in the following exemplary embodiments are not meant to represent all implementations consistent with the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present application as detailed in the appended claims.

[0028] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of related data comply with relevant laws, regulations and standards of relevant countries and regions, necessary security measures are taken, public order and good customs are not violated, and appropriate operation portals are provided for users to choose authorization or refusal.

[0029] And the present application involves big data analysis of user information (including but not limited to personal biological characteristics, identity data, consumption data, asset data, electronic terminal operation data, etc.), and uses artificial intelligence technology for automatic decision-making, and makes technical solutions based on automatic decision-making results that have a significant impact on personal rights and interests, provides appropriate operation portals for users to choose to agree or refuse automatic decision-making results; if the user chooses to refuse, the expert decision-making process is entered.

[0030] It should be noted that the face recognition method, device, equipment, storage medium and product provided by the present application can be used in the field of biometric recognition, and can also be used in any field other than biometric recognition. The application field of the face recognition method, device, equipment, storage medium and product in the present application is not limited.

[0031] Specifically, face recognition usually needs to collect images of a subject to be recognized, and the image collection has certain requirements for the environment, for example, the user needs to face the image collection device to take a front-facing photo. In addition, it may also involve requirements for light, background, etc., and the requirements for the image collection environment are high. However, in actual scenarios, for example, when a user is conducting business at a financial institution, face recognition may be involved for identity authentication, and when the user is using face payment or face access, and when the user is queuing for payment or queuing for access, the subject to be recognized in the collected image usually does not meet the specific angle distance, or the background of the image is cluttered, multiple faces appear, etc., which may result in failure to accurately recognize the user's face, and thus may result in payment failure or using someone else's account.

[0032] The face recognition method, device, equipment, storage medium and product provided by the present application can be applied to face login or face recognition, face payment or face access, and other payment-related application scenarios. A video containing features of a subject to be recognized is obtained through image collection. Since the video stream contains subjects to be recognized at multiple angles, it is not limited to collecting a single front-facing photo, and thus the face features obtained through collection are more comprehensive, which facilitates more accurate face recognition in the subsequent process. The image set in the video is processed frame by frame, face features at multiple angles are extracted from the image set, and the integrated face features are obtained through fusion for face feature recognition, aiming to solve the above technical problems of the prior art.

[0033] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present application will be described below with reference to the accompanying drawings.

[0034] Figure 1 Flowchart of the face recognition method provided by the embodiments of the present application Figure 1 As shown in Figure 1 , the method comprises:

[0035] S101, acquiring a video to be recognized, and performing frame extraction on the video to be recognized to obtain multiple images to be recognized containing a subject to be recognized.

[0036] The subject to be recognized is contained in the video to be recognized. In actual applications, the user and the environment in the video to be recognized in the present embodiment do not need to meet specific requirements. For example, the video to be recognized can be unclear or non-frontal, i.e., it is not limited to being collected in the front of the subject to be recognized, and there is no excessive requirement for the background in which the subject to be recognized is located, and there is no need to meet specific requirements for the distance, specific angle, and background in the video.

[0037] Optionally, the video to be identified is frame-extracted to obtain a set of image frames containing the subject to be identified, specifically including: frame-extracting the video to be identified to obtain a set of initial image frames; determining whether the image frames in the set of initial image frames contain the subject to be identified, and when the image frames contain the subject to be identified, determining the image frames as the image to be identified, until the set of initial image frames is traversed.

[0038] In an example, a video to be identified is obtained by image acquisition equipment, and the video to be identified is frame-extracted to obtain a set of initial image frames corresponding to the video to be identified. From the set of initial image frames, frame images containing the subject to be identified are extracted and determined as the image to be identified, until all initial image frames in the set of initial image frames are traversed, to obtain a set of images to be identified containing the subject to be identified.

[0039] By separating different frame images in the video, the image frames containing the subject to be identified are screened, other image frames not containing the subject to be identified are excluded, the number of images to be processed in subsequent feature extraction is reduced, the efficiency of feature recognition is further improved, only the frame images containing the subject to be identified are retained, and the accuracy of feature recognition is improved.

[0040] S102, feature extraction is performed on the plurality of images to be identified to obtain face features of the subject to be identified corresponding to the plurality of images to be identified.

[0041] In an example, face feature extraction is performed on each image to be identified containing the subject to be identified to obtain face features of the subject to be identified corresponding to each image to be identified, and a set of face features of the subject to be identified corresponding to all images to be identified is obtained. The method of face feature extraction can calculate or learn a set of low-dimensional and discriminative face feature vectors (i.e., feature vectors or embedding vectors) from the preprocessed images to be identified, and the method of face feature extraction is not limited in this example.

[0042] S103, the face features of the subject to be identified corresponding to the plurality of images to be identified are fused to obtain a comprehensive feature of the subject to be identified, and the comprehensive feature is compared with a preset feature library to obtain a face recognition result.

[0043] In an example, a complete face feature value is obtained by fusing face features of the to-be-identified subject corresponding to all to-be-identified images, that is, a comprehensive feature of the to-be-identified subject. The comprehensive feature is compared with a preset face feature library to determine whether the comprehensive feature exists in the preset face feature library. When the comprehensive feature exists in the preset face feature library, it is determined that the matching is successful. When the comprehensive feature does not exist in the preset face feature library, it is determined that the matching fails, and a face recognition result of matching success or matching failure is obtained. The preset face feature library can include comprehensive features corresponding to different to-be-identified subjects. The comparison between the comprehensive feature and the preset feature library can be performed by calculating the similarity between the comprehensive feature and each comprehensive feature vector in the preset feature library. According to the similarity score and a preset strategy, it is determined whether the comprehensive feature already exists in the preset feature library. The manner of comparing the comprehensive feature with the preset feature library in this example is not limited.

[0044] In an implementation scenario, Figure 2 A flowchart of a video processing method for face recognition, feature comparison and the like provided in the present application is shown in FIG. 1. Figure 2 As shown in FIG. 1, the video processing method for face recognition, feature comparison and the like can include the following steps:

[0045] 1) Extracting a frame image set from a frame part containing a to-be-identified subject in a video, wherein the video is a to-be-identified video, and the frame image is a to-be-identified image.

[0046] 2) Performing image enhancement preprocessing on the frame image set.

[0047] 3) Performing feature extraction processing on the preprocessed frame image to obtain a feature set. The feature set is a set of face features of the to-be-identified subject corresponding to multiple to-be-identified images.

[0048] 4) Extracting a complete face feature value from the face feature set. The complete face feature value is a comprehensive feature of the to-be-identified subject obtained by fusing face features of the to-be-identified subject corresponding to multiple to-be-identified images.

[0049] 5) Comparing the extracted complete face feature value with a preset feature value library to obtain a comparison result. The comparison result is a face recognition result.

[0050] Specifically, the specific manner of image enhancement preprocessing is not limited. In some examples, image enhancement can be achieved by a color constancy model (Retinex) algorithm, such as a single-scale Retinex algorithm and a multi-scale Retinex algorithm.

[0051] Specifically, the image of an object observed by an observer is determined by the reflection of light The reflection of the object is obtained by the reflection of the light It is determined by itself. That is, the image is composed of two parts, one is the light illumination component , and the other is the object reflection component . The image can be expressed by Mathematical formula. Based on the Retinex theory, the basic idea of the single-scale Retinex algorithm is to apply a Gaussian filter to the Retinex algorithm. In order to facilitate the filtering operation of the image, the logarithm of the image is taken first, and then the logarithm is converted to a real number after the operation is completed. The center-surround function is used to estimate the value of the illumination component by convolving with the captured image The specific calculation process of the single-scale Retinex algorithm is: In the single-scale Retinex algorithm, each pixel point in the input image is filtered with its field and the filter core to predict the illumination part in the image, that is, to calculate the weighted average value of each pixel point in the image and each pixel point in the filter core. By removing the properties in the image, the properties in the image are retained to realize image enhancement.

[0052] On the basis of the single-scale Retinex algorithm, the multi-scale Retinex algorithm realizes image enhancement by obtaining multiple different center-surround scale c values to calculate the image respectively, and the processed image can both retain the detail information of the image and retain the color information of the image. Its specific expression is:

[0053]

[0054] Where needs to meet: . The Gaussian filter function under each scale is represented by

[0055]

[0056] Wherein, The value of should also meet: . Wherein, K represents the number of scales selected in the algorithm, when K = 1, it is equivalent to the single-scale Retinex algorithm. The weight under each scale is represented by The normalization factor of the filter function under different Gaussian surround scales is represented by Different Gaussian surround scales are represented by

[0057] The characteristics of the to-be-recognized subject are recognized by using multiple frames in the video, and the face features are extracted by combining multiple frames in the video, so that the user can perform face recognition through a video instead of extracting feature values from a specific angle, thereby improving the efficiency of feature extraction and the accuracy of feature comparison. The bilateral filter takes into account the spatial information and pixel value information characteristics of the image, facilitating feature extraction after frame image enhancement, and improving the effectiveness and accuracy of feature extraction.

[0058] The scheme of the embodiment can be used in common face swiping login, face swiping access and other scenarios requiring face recognition, thereby improving the accuracy of face recognition, reducing the need for the user to perform face recognition multiple times or adjust to a specific distance and angle for face recognition, reducing the time required for the user to perform face swiping login or face swiping access, and improving the user experience. It is particularly suitable for scenarios where face recognition is compared with a single front view, such as comparison with an ID photo, access to company or community access control.

[0059] The face recognition method provided by the embodiment extracts frames from a to-be-recognized video containing a to-be-recognized subject to obtain a to-be-recognized image set, extracts features from each frame of the to-be-recognized image set, and fuses feature values corresponding to each image in the image set to obtain comprehensive features. The comprehensive features are used for feature comparison to determine the result, thereby providing a feature extraction method for face recognition scenarios with universal usability and higher recognition degree. The comprehensive features obtained by feature fusion are used for recognition, which avoids interference of a single frame image and improves the accuracy of face recognition.

[0060] However, for the single-scale Retinex algorithm, the effect of image enhancement is greatly related to the value of the center surrounding scale c. When the standard deviation c takes a smaller value, the algorithm is more accurate in details related information of the image, but color distortion occurs. When the standard deviation c takes a larger value, the algorithm is more conducive to preserving color information of the image, but less details related information is preserved. The accuracy of image enhancement is greatly related to the value of the center surrounding scale c, and cannot simultaneously consider multiple attributes of the image. After image enhancement processing, other information of the image is distorted, and the accuracy of OCR text information extraction is limited and not high in applicability.

[0061] The values of the scale number k and the center surrounding scale selected by the multi-scale Retinex algorithm have uncertainty, and different experiments need to be performed on different parameters to adjust appropriate parameters. At the same time, the Gaussian filter used by the multi-scale Retinex algorithm cannot simultaneously consider the spatial information and pixel value information of the image, and can only adjust the center surrounding scale and The value of the proportion controls the weight of spatial information and pixel value information, thereby controlling the focus of the Retinex calculation. Therefore, it is impossible to simultaneously consider both spatial and pixel value information in a single calculation, resulting in limited image enhancement effects and lacking universal applicability. Different application scenarios require different adjustments to the center-surround scale for image enhancement. and The value of is adjusted differently.

[0062] Considering the situation in the above example, Figure 3 Flowchart of the face recognition method provided in this application Figure 2 ,like Figure 3 As shown, in this embodiment... Figure 1 Based on the embodiments, an image enhancement algorithm is provided, which can be applied to the face recognition method of this embodiment. In conjunction with this image enhancement algorithm, the face recognition method of this embodiment includes:

[0063] S301. Obtain the video to be identified, extract the video frame by frame to obtain multiple images containing the subject to be identified.

[0064] S302. For each image to be identified, perform separation calculations on the image to be identified to obtain the logarithmic domain image under each color component.

[0065] Optionally, the image to be recognized is separated and calculated to obtain the logarithmic domain image under each color component. Specifically, this includes: separating the image to be recognized according to different color components to obtain the initial image under each color component; converting each pixel in the initial image under each color component from real number type to logarithmic type to obtain the logarithmic domain image under each color component.

[0066] In one example, the image to be recognized The RGB (Red, Green, Blue) color model is converted to the HSV (Hue, Saturation, Value) color model to obtain initial images for the hue, saturation, and value components. The pixels in the initial image for the hue component are converted from real to logarithmic types to obtain the logarithmic domain image for the hue component; the pixels in the initial image for the saturation component are converted from real to logarithmic types to obtain the logarithmic domain image for the saturation component; and the pixels in the initial image for the value component are converted from real to logarithmic types to obtain the logarithmic domain image for the value.

[0067] The color component separation is performed on the to-be-identified image, and the to-be-identified image is converted into a logarithmic domain image, so that the color components are independent of each other, the color distortion is avoided in the feature extraction, and the accuracy of image enhancement is further improved.

[0068] In S303, the adaptive weight, the spatial domain correction function and the value domain correction function, and the spatial domain center wrap scale and the value domain center wrap scale of each color component are calculated by using the adaptive filter function.

[0069] Optionally, the adaptive weight of the to-be-identified image under each color component is calculated by using the adaptive filter function, including: for each color component, the pixel value of a current pixel point and the pixel value of a neighborhood pixel point of the to-be-identified image under the color component are obtained based on different preset scale parameters; and the product of the pixel value of the current pixel point and the pixel value of the neighborhood pixel point is calculated, and the adaptive weight under the color component is obtained by combining the initial weight corresponding to each scale of the current pixel point.

[0070] In an example, the adaptive weight is calculated by using the following formula:

[0071]

[0072] wherein, the pixel value of the current pixel point is, the pixel value of the neighborhood pixel point is, the initial weight corresponding to each scale of the current pixel point is, the adaptive weight corresponding to the current pixel point under the color component is. The adaptive weights under the H, S and V color components are respectively.

[0073] The adaptive weight corresponding to each color component is calculated by using the pixel value and the initial weight corresponding to each scale of each pixel point under different color components, the adaptive weight corresponding to each color component is determined, the adaptive weight is used to adjust the correction function, and the proportion of the pixel point calculation is controlled according to the feature of the pixel point in the image.

[0074] Optionally, the spatial domain center wrap scale and the value domain center wrap scale under each color component are calculated, including: for each color component, the minimum value of the adaptive Gaussian filter and the adaptive median filter corresponding to the spatial domain of the to-be-identified image under the color component is determined as the spatial domain center wrap scale according to the adaptive Gaussian filter and the adaptive median filter corresponding to the spatial domain of the to-be-identified image under the color component; and the minimum value of the adaptive Gaussian filter and the adaptive median filter corresponding to the value domain of the to-be-identified image under the color component is determined as the value domain center wrap scale according to the adaptive Gaussian filter and the adaptive median filter corresponding to the value domain of the to-be-identified image under the color component.

[0075] In an example, the spatial domain center-surround scale of each color component is calculated by an adaptive filtering function and the value domain center-surround scale may include:

[0076] For each color component, the adaptive Gaussian filter and the adaptive median filter corresponding to the spatial domain of the image to be identified are calculated respectively through the spatial domain or the value domain of the image to be identified, the minimum value of the adaptive Gaussian filter and the adaptive median filter corresponding to the spatial domain of the image to be identified is taken, and the minimum value is determined as the spatial domain center-surround scale of the image to be identified under the color component; the adaptive Gaussian filter and the adaptive median filter corresponding to the value domain of the image to be identified are calculated, and the minimum value of the adaptive Gaussian filter and the adaptive median filter corresponding to the value domain of the image to be identified is taken, and the minimum value is determined as the value domain center-surround scale of the image to be identified under the color component.

[0077] For example, the processing flow of the adaptive Gaussian filter can include: dynamically adjusting the scale parameter of the Gaussian kernel according to the noise intensity of the local region of the image to be identified : the region with noise greater than a preset first noise threshold uses a first scale parameter of the Gaussian kernel to enhance the smoothing effect, and the region with noise less than the preset first noise threshold uses a second scale parameter of the Gaussian kernel to retain details. The first scale parameter of the Gaussian kernel is greater than the second scale parameter of the Gaussian kernel. The processing flow of the adaptive median filter can include: dynamically adjusting the window size of the filter according to the noise density of the local region of the image to be identified: the region with more noise than a preset second noise threshold uses a first window to enhance the denoising effect, and the region with less noise than the preset second noise threshold uses a second window to retain details. The first window is greater than the second window. By taking the minimum value of the adaptive Gaussian filter and the adaptive median filter, the center-surround scale corresponding to the spatial domain and the value domain is determined, and the center-surround scale of the spatial domain and the value domain is adaptively adjusted for the image, which realizes adaptive adjustment of the spatial domain and the value domain, avoids loss of details or excessive smoothing caused by fixed spatial domain and value domain, and further improves the enhancement effect of image enhancement, so that the accuracy of feature extraction is further improved.

[0078] By taking the minimum value of the adaptive Gaussian filter and the adaptive median filter, the center-surround scale corresponding to the spatial domain and the value domain is determined, and the center-surround scale of the spatial domain and the value domain is adaptively adjusted for the image, which realizes adaptive adjustment of the spatial domain and the value domain, avoids loss of details or excessive smoothing caused by fixed spatial domain and value domain, and further improves the enhancement effect of image enhancement, so that the accuracy of feature extraction is further improved.

[0079] ​​Optionally, the spatial domain correction function and the value domain correction function of the to-be-identified image under each color component are calculated, including: for each color component, obtaining the spatial domain correction function under the color component according to the distance between each pixel point and the neighborhood pixel point of the to-be-identified image under the color component and the spatial domain center wrap-around scale; and obtaining the value domain correction function under the color component according to the absolute value of the difference between the pixel value of each pixel point and the pixel value of the neighborhood pixel point of the to-be-identified image under the color component, in combination with the value domain center wrap-around scale.

[0080] In an example, for each color component, the spatial domain correction function under each color component can be calculated by the following formula and the value domain correction function

[0081]

[0082]

[0083]

[0084] wherein, is the current pixel point, is the pixel value of the current pixel point, is the neighborhood pixel point, is the pixel value of the neighborhood pixel point, is the spatial domain correction function of the current pixel point under the color component, is the spatial domain center wrap-around scale of the current pixel point under the color component, is the value domain correction function of the current pixel point under the color component, is the value domain center wrap-around scale of the current pixel point under the color component.

[0085] For example, the calculation rule of the spatial domain includes: local statistical analysis: by analyzing the statistical characteristics (such as variance, mean, gradient, etc.) of the local area of the to-be-identified image, the noise intensity, texture complexity, etc. of the area are obtained, and the local statistical result is obtained; dynamically adjusting the filter parameters: according to the local statistical result, the size, weight or shape of the filter is dynamically adjusted to optimize the filtering effect; different images in different domains, different image calculations, multi-scale processing is used: different filters are applied under different scales to adapt to different scale features (such as edges, textures, etc.) in the to-be-identified image.

[0086] The calculation rule of the value domain includes: using a histogram-based value domain filter: by analyzing the histogram of the image, the value domain range of the noise pixels of the to-be-identified image is identified, and the noise pixels are filtered and the value domain correction is added.

[0087] ​The spatial domain correction function and the value domain correction function under each color component are calculated through the relationship between each pixel point and the neighborhood pixel points, the bilateral filtering is adjusted, the spatial domain center wrap scale and the value domain center wrap scale are combined, and the proportion of pixel point calculation is controlled according to the characteristics of the current pixel point, so that different enhancement is realized for different pixel points, the image enhancement effect is improved, and the accuracy of feature extraction of a single frame image is improved.

[0088] In S304, for each color component, the log domain image under the color component, the adaptive weight, the spatial domain correction function and the value domain correction function, and the spatial domain center wrap scale and the value domain center wrap scale are convolved to obtain the illumination component under each color component.

[0089] Optionally, for each color component, the log domain image under the color component, the adaptive weight, the spatial domain correction function and the value domain correction function, and the spatial domain center wrap scale and the value domain center wrap scale are convolved to obtain the illumination component under each color component, specifically including: for each color component, the spatial domain correction function, the value domain correction function, the spatial domain center wrap scale and the value domain center wrap scale under the color component are convolved based on the image bilateral filter to obtain a processing result; and the processing result, the adaptive weight and the log domain image under the color component are convolved to obtain the illumination component under the color component.

[0090] In an example, for each color component, the processing result under the color component is calculated by the following formula:

[0091]

[0092] wherein, is the processing result under the color component. The processing results under the hue component, the saturation component and the brightness component are calculated respectively 、 and .

[0093] The processing result, the adaptive weight and the log domain image under the color component are convolved to obtain the illumination component under the color component, which can be calculated by the following formula:

[0094]

[0095] wherein, is the log domain image of the image to be identified under the color component, is the adaptive weight of the image to be identified under the color component, is the illumination component under the color component.

[0096] The bilateral filtering function under different components is calculated by the spatial domain center surrounding scale and the value domain center surrounding scale of each pixel point and the neighborhood pixel points under different color components, combined with the spatial domain correction function and the value domain correction function, and the illumination component under different color components is obtained combined with the adaptive weight, which realizes the consideration of the spatial information and the pixel value information of the image at the same time, and makes the feature extraction of the single frame image more accurate.

[0097] S305, according to the illumination component under each color component, the enhanced to-be-recognized image is obtained by merging.

[0098] Optionally, according to the illumination component under each color component, the enhanced to-be-recognized image is obtained by merging, specifically including: the illumination components under each color component are merged to obtain an initial result image; each pixel point in the initial result image is converted from the logarithmic type to the real type to obtain the corresponding inverse light image of the to-be-recognized image; the inverse light image is linearly stretched to obtain the enhanced to-be-recognized image.

[0099] In an example, the illumination components under the hue component, the saturation component and the brightness component are merged to obtain an initial result image ; each pixel point in the initial result image is converted from the logarithmic type to the real type to obtain the corresponding inverse light image of the to-be-recognized image ; since the range of the pixel value of each pixel point in the inverse light image is not between 0-255, the inverse light image is linearly stretched and converted into the corresponding format to obtain the enhanced to-be-recognized image.

[0100] By merging the illumination components under different color components and converting each pixel point in the merged image into a real number, the inverse light image is obtained, the inverse light image is stretched to obtain the enhanced image, the enhanced image is more saturated and natural, the structural information is more abundant, and the visual effect is effectively improved.

[0101] Figure 4 In an implementation scenario, Figure 4 a flowchart of an image enhancement provided by the present application is as followsAs shown, the image enhancement process can include: converting the input to-be-recognized image from an RGB color model to an HSV color model; converting each pixel point in the to-be-recognized image under the HSV color model from a real number type to a logarithmic type to obtain a logarithmic domain image S corresponding to the to-be-recognized image under the HSV model; obtaining a corresponding adaptive correction function according to an adaptive function D corresponding to the to-be-recognized image; inputting the adaptive standard deviation and the adaptive correction function into a bilateral filter of a fusion adaptive factor, and performing convolution operation on the logarithmic domain image S to obtain an illumination component L; obtaining a reflection component r by performing operation on the logarithmic domain image S and the illumination component L; and converting each pixel point in the reflection component r from a real number type to a logarithmic type to obtain a reflection component R.

[0102] Correspondingly, S306, feature extraction is performed on the plurality of enhanced to-be-recognized images to obtain face features of the to-be-recognized subject corresponding to the plurality of enhanced to-be-recognized images.

[0103] S307, the face features of the to-be-recognized subject corresponding to the plurality of enhanced to-be-recognized images are fused to obtain comprehensive features of the to-be-recognized subject, and the comprehensive features are compared with a preset feature library to obtain a face recognition result.

[0104] The face recognition method provided in this embodiment, through adaptive weights under each color component, spatial domain center wrap scale, value domain wrap scale and corresponding correction function, combined with logarithmic domain image convolution calculation to obtain an illumination component, and then to obtain an enhanced image, through adaptive filtering calculation to obtain bilateral adaptive weights, spatial domain center wrap scale, value domain wrap scale and corresponding correction function, while taking into account the spatial information and pixel value information of the image, the proportion of pixel point calculation can be controlled according to the characteristics of the current pixel point, realizing different enhancement for the characteristics of different pixel points in a single frame image. Through image enhancement, the limitations of recognition distance, angle, light, background, etc. of the existing face recognition are overcome, and the method is suitable for enhancing the subject in the image and weakening the background of the image, so that the feature extraction of a single frame image is more accurate.

[0105] Figure 5 The structure schematic diagram of the face recognition device provided in this application is shown in Figure 5 As shown, the face recognition device 50 provided in this embodiment includes:

[0106] The image acquisition module 501 is configured to acquire a to-be-recognized video, perform frame extraction on the to-be-recognized video, and obtain a plurality of to-be-recognized images containing a to-be-recognized subject; and the to-be-recognized video contains the to-be-recognized subject.

[0107] The feature extraction module 502 is configured to perform feature extraction on the plurality of to-be-recognized images to obtain face features of the to-be-recognized subject corresponding to the plurality of to-be-recognized images.

[0108] The feature comparison module 503 is configured to fuse face features of a plurality of to-be-identified subjects corresponding to a plurality of to-be-identified images, to obtain comprehensive features of the to-be-identified subjects, and to compare the comprehensive features with a preset feature library to obtain a face recognition result.

[0109] In a possible implementation, the face recognition apparatus 50 is further configured to: for each to-be-identified image, perform separate calculation on the to-be-identified image to obtain a log domain image under each color component; calculate, by using an adaptive filter function, an adaptive weight, a spatial domain correction function and a value domain correction function under each color component, and a spatial domain center wrap scale and a value domain center wrap scale; and for each color component, perform convolution processing on the log domain image under the color component, the adaptive weight, the spatial domain correction function and the value domain correction function, and the spatial domain center wrap scale and the value domain center wrap scale to obtain an illumination component under the color component; and combine the illumination components under the color components to obtain an enhanced to-be-identified image.

[0110] In a possible implementation, the face recognition apparatus 50 is further configured to: separate the to-be-identified image according to different color components to obtain initial images under the color components; and convert each pixel point in the initial images under the color components from a real number type to a logarithm type to obtain log domain images under the color components.

[0111] In a possible implementation, the face recognition apparatus 50 is further configured to: combine the illumination components under the color components to obtain an initial result image; convert each pixel point in the initial result image from the logarithm type to the real number type to obtain a reflection image corresponding to the to-be-identified image; and perform linear stretching on the reflection image to obtain the enhanced to-be-identified image.

[0112] In a possible implementation, the face recognition apparatus 50 is further configured to: for each color component, perform convolution operation based on an image bilateral filter according to the spatial domain correction function, the value domain correction function, the spatial domain center wrap scale and the value domain center wrap scale under the color component to obtain a processing result; and perform convolution on the processing result, the adaptive weight under the color component and the log domain image to obtain the illumination component under the color component.

[0113] In a possible implementation, the face recognition apparatus 50 is further configured to: for each color component, obtain pixel values of a current pixel point and pixel values of neighbor pixel points of the to-be-identified image under the color component based on different preset scale parameters; calculate a product of the pixel values of the current pixel point and the pixel values of the neighbor pixel points, and combine initial weights corresponding to the current pixel point at different scales to obtain the adaptive weight under the color component.

[0114] In one possible implementation, the face recognition device 50 is further specifically used to: for each color component, determine the minimum value of the adaptive Gaussian filtering and adaptive median filtering corresponding to the spatial domain of the image to be recognized under that color component as the spatial domain center-surrounding scale; and determine the minimum value of the adaptive Gaussian filtering and adaptive median filtering corresponding to the value domain of the image to be recognized under that color component as the value domain center-surrounding scale.

[0115] In one possible implementation, the face recognition device 50 is further specifically used to: for each color component, obtain a spatial domain correction function for that color component based on the distance between each pixel of the image to be recognized and its neighboring pixels and the spatial domain center-surrounding scale; and obtain a value domain correction function for that color component based on the absolute value of the difference between the pixel value of each pixel of the image to be recognized and the pixel value of its neighboring pixels, combined with the value domain center-surrounding scale.

[0116] In one possible implementation, the image acquisition module 501 is specifically used to: extract the video stream to be identified frame by frame to obtain a set of initial image frames; determine whether the initial image frames in the set of initial image frames contain the subject to be identified; if the initial image frame contains the subject to be identified, determine that the initial image frame is the image to be identified, until the set of initial image frames has been traversed.

[0117] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 600 may include a memory 601 and a processor 602. Optionally, the electronic device may also include a transceiver 603, wherein the memory 601 and the processor 602 communicate; for example, the memory 601, the processor 602 and the transceiver 603 may communicate via a communication bus 604, the memory 601 is used to store a computer program, and the processor 602 executes the computer program to implement the method of the above embodiments.

[0118] Optionally, the processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method embodiments disclosed in the present application can be directly embodied as the execution of hardware processor, or the combination of hardware and software modules in the processor.

[0119] The embodiments of the present application further provide a computer readable storage medium, and the computer readable storage medium stores computer execution instructions. The computer execution instructions are executed by the processor to implement the method in any of the method embodiments.

[0120] The embodiments of the present application further provide a computer program product, and the computer program product includes a computer program. The computer program is executed by the processor to implement the method in any of the method embodiments.

[0121] All or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The foregoing program can be stored in a readable memory. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the foregoing memory (storage medium) includes a read-only memory (ROM), a RAM, a flash memory, a hard disk, a solid state disk, a magnetic tape, a floppy disk, an optical disc, and any combination thereof.

[0122] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams according to the method, device (system), and computer program product of the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to produce a machine, so that the instructions executed by the computer or other programmable data processing devices produce a device implemented in the flowcharts and / or block diagrams. Figure 1 The device for implementing the functions specified in one flow or multiple flows and / or one block or multiple blocks. Figure 1 The device for implementing the functions specified in one flow or multiple flows and / or one block or multiple blocks.

[0123] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flow Figure 1 The functions of a flow or multiple flows and / or a block or multiple blocks in accordance with the embodiments of the present application can be implemented as hardware, software, firmware or any combination thereof. Figure 1

[0124] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow Figure 1 The functions of a flow or multiple flows and / or a block or multiple blocks in accordance with the embodiments of the present application can be implemented as hardware, software, firmware or any combination thereof. Figure 1

[0125] Obviously, persons having ordinary skill in the art can make various modifications and variations to the embodiments of the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the embodiments of the present application belong to the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

[0126] In the present application, the term "comprising" and its variants can refer to non-limiting inclusion; the term "or" and its variants can refer to "and / or". In the present application, the terms "first", "second" and the like are used to distinguish similar objects, not necessarily to describe a specific order or sequence. In the present application, "multiple" refers to two or more. "And / or", which describes the relationship between the associated objects, means that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects.​​

Claims

1. A face recognition method, characterized by, The method comprises: acquiring a to-be-identified video, performing frame extraction on the to-be-identified video to obtain a plurality of to-be-identified images containing a to-be-identified subject; the to-be-identified video contains the to-be-identified subject; performing feature extraction on the plurality of to-be-identified images to obtain face features of the to-be-identified subject corresponding to the plurality of to-be-identified images; fusing the face features of the to-be-identified subject corresponding to the plurality of to-be-identified images to obtain comprehensive features of the to-be-identified subject, and comparing the comprehensive features with a preset feature library to obtain a face recognition result.

2. The method of claim 1, wherein, Before the feature extraction on the plurality of to-be-identified images, the method further comprises: for each to-be-identified image, performing separation calculation on the to-be-identified image to obtain a log domain image under each color component; calculating, through an adaptive filter function, an adaptive weight, a spatial domain correction function and a value domain correction function under each color component, and a spatial domain center wrap scale and a value domain center wrap scale; for each color component, performing convolution processing on the log domain image, the adaptive weight, the spatial domain correction function and the value domain correction function, and the spatial domain center wrap scale and the value domain center wrap scale under the color component to obtain an illumination component under the color component; merging the illumination components under the color components to obtain an enhanced to-be-identified image.

3. The method of claim 2, wherein, The separation calculation on the to-be-identified image to obtain the log domain image under each color component specifically comprises: separating the to-be-identified image according to different color components to obtain an initial image under each color component; converting each pixel point in the initial image under each color component from a real number type to a logarithm type to obtain a log domain image under each color component.

4. The method of claim 3, wherein, The merging of the illumination components under the color components to obtain the enhanced to-be-identified image specifically comprises: merging the illumination components under the color components to obtain an initial result image; converting each pixel point in the initial result image from a logarithm type to a real number type to obtain a reflection image corresponding to the to-be-identified image; performing linear stretching on the reflection image to obtain the enhanced to-be-identified image.

5. The method of claim 2, wherein, The convolution processing on the log domain image, the adaptive weight, the spatial domain correction function and the value domain correction function, and the spatial domain center wrap scale and the value domain center wrap scale under the color component to obtain the illumination component under the color component specifically comprises: for each color component, performing convolution operation based on an image bilateral filter according to the spatial domain correction function, the value domain correction function, the spatial domain center wrap scale and the value domain center wrap scale under the color component to obtain a processing result; performing convolution on the processing result, the adaptive weight and the log domain image under the color component to obtain the illumination component under the color component.

6. The method of claim 2, wherein, The calculation of the adaptive weight of the to-be-identified image under each color component through the adaptive filter function comprises: for each color component, based on different preset scale parameters, acquiring a pixel value of a current pixel point and a pixel value of a neighborhood pixel point of the to-be-identified image under the color component; The product of the pixel value of the current pixel point and the pixel value of the neighborhood pixel point is calculated, and the adaptive weight under the color component is obtained by combining the initial weight corresponding to each scale of the current pixel point.

7. The method of claim 2, wherein, The calculation of the spatial domain center wrap scale and the value domain center wrap scale under each color component comprises: For each color component, the minimum value of the adaptive Gaussian filter and the adaptive median filter corresponding to the spatial domain of the to-be-identified image under the color component is determined according to the adaptive Gaussian filter and the adaptive median filter corresponding to the spatial domain of the to-be-identified image under the color component. The minimum value of the adaptive Gaussian filter and the adaptive median filter corresponding to the value domain of the to-be-identified image under the color component is determined according to the adaptive Gaussian filter and the adaptive median filter corresponding to the value domain of the to-be-identified image under the color component.

8. The method of claim 7, wherein, The calculation of the spatial domain correction function and the value domain correction function of the to-be-identified image under each color component comprises: For each color component, the spatial domain correction function under the color component is obtained according to the distance between each pixel point and the neighborhood pixel point of the to-be-identified image under the color component and the spatial domain center wrap scale. The value domain correction function under the color component is obtained according to the absolute value of the difference between the pixel value of each pixel point and the pixel value of the neighborhood pixel point of the to-be-identified image under the color component and the value domain center wrap scale.

9. The method of claim 1, wherein, The to-be-identified video is extracted frame by frame to obtain a set of image frames containing a to-be-identified subject, and specifically comprises: The to-be-identified video stream is extracted frame by frame to obtain a set of initial image frames; It is judged whether the initial image frame in the set of initial image frames contains the to-be-identified subject, and when the initial image frame contains the to-be-identified subject, the initial image frame is determined as the to-be-identified image, until the set of initial image frames is traversed.

10. A face recognition apparatus, characterized by comprising: It comprises: An image acquisition module is configured to acquire a to-be-identified video, extract the to-be-identified video frame by frame, and obtain a plurality of to-be-identified images containing a to-be-identified subject; the to-be-identified video contains the to-be-identified subject; A feature extraction module is configured to perform feature extraction on the plurality of to-be-identified images to obtain facial features of the to-be-identified subject corresponding to the plurality of to-be-identified images; A feature comparison module is configured to fuse the facial features of the to-be-identified subject corresponding to the plurality of to-be-identified images to obtain comprehensive features of the to-be-identified subject, and compare the comprehensive features with a preset feature library to obtain a face recognition result.

11. An electronic device, comprising: It comprises: A processor and a memory in communication connection with the processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory to implement the method of any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method of any one of claims 1 to 9.

13. A computer program product, characterised in that, It comprises a computer program which, when executed by a processor, implements the method of any one of claims 1 to 9.

Images