Face recognition method, electronic device and storage medium

Abstract

The present invention discloses a face recognition method, an electronic device and a storage medium, the method comprising extracting multiple frames of face images containing a target face from a video stream; extracting face features from the multiple frames of face images respectively to obtain a first face feature; enhancing the first face feature and fusing the enhanced first face feature to obtain a second face feature; comparing the second face feature with a pre-stored third face feature to determine a face recognition result. The technical solution provided by the embodiment of the present invention performs face recognition based on face features extracted from multiple frames of face images, which can overcome the problem that the traditional method performs face recognition based only on the features of a single image, resulting in the recognition result being greatly affected by noise interference. The embodiment of the present invention also performs feature enhancement and fusion on the first face feature extracted from the multiple frames of face images to further improve the success rate and reliability of face recognition.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a face recognition method, electronic equipment and storage medium. Background Art

[0002] At present, face recognition is widely used in security monitoring, criminal arrest, crowd statistics analysis and other application scenarios. However, face recognition is easily interfered by various external noises in actual application. For example: face deflection; large side face; motion blur and out-of-focus blur; face obstruction (such as masks, sunglasses); low light intensity and contrast; artificial blocks generated by the encoding and decoding process of video transmission, etc. Due to the interference of noise, the accuracy of face recognition is greatly reduced, which limits the application and development of face recognition technology. Summary of the invention

[0003] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the claims.

[0004] The embodiments of the present invention provide a face recognition method, an electronic device and a storage medium, which can reduce the influence of noise interference on the face recognition accuracy, thereby improving the success rate of face recognition.

[0005] On the one hand, an embodiment of the present invention provides a face recognition method, comprising:

[0006] Extract multiple frames of face images containing the target face from the video stream;

[0007] Extracting facial features from the plurality of frames of facial images respectively to obtain a first facial feature;

[0008] Performing feature enhancement on the first facial feature, and fusing the enhanced first facial feature to obtain a second facial feature;

[0009] The second facial feature is compared with a pre-stored third facial feature to determine a facial recognition result.

[0010] On the other hand, an embodiment of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the face recognition method described above when executing the program.

[0011] On the other hand, an embodiment of the present invention provides a computer-readable storage medium storing a computer program, which implements the steps of the face recognition method described above when executed by a processor.

[0012] The embodiment of the present invention includes: extracting multiple frames of face images containing a target face from a video stream; extracting face features from the multiple frames of face images respectively to obtain a first face feature; enhancing the first face feature and fusing the enhanced first face feature to obtain a second face feature; comparing the second face feature with a pre-stored third face feature to determine a face recognition result. The technical solution provided by the embodiment of the present invention performs face recognition based on the face features extracted from multiple frames of face images, so that the face feature samples are richer and more diverse, feature complementarity is achieved, and more information is available for face recognition, thereby overcoming the problem that the traditional method performs face recognition based only on the features of a single image, which makes the recognition result greatly affected by noise interference. The embodiment of the present invention also performs feature enhancement and fusion on the first face feature extracted from the multiple frames of face images to achieve compensation for the face features, further improving the success rate and reliability of face recognition.

[0013] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0015] Figure 1 is a flow chart of a face recognition method provided by an embodiment of the present invention;

[0016] Figure 2 yes Figure 1 Sub-flow chart of step S100 in;

[0017] Figure 3 yes Figure 2 Sub-flow chart of step S110 in ;

[0018] Figure 4 yes Figure 2 Step S130 in the sub-flow chart;

[0019] Figure 5 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0021] It should be understood that in the description of the embodiments of the present invention, the meaning of multiple (or multiple) is more than two, greater than, less than, and exceeding are understood to exclude the number itself, and above, below, and within are understood to include the number itself. If there is a description of "first", "second", etc., it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0022] Figure 1 FIG. 1 is a flow chart of a face recognition method provided by an embodiment of the present invention. Figure 1 As shown, the method includes but is not limited to the following steps S100 to S400.

[0023] Step S100: extract multiple frames of face images containing the target face from the video stream.

[0024] In a specific implementation, the video can be collected by a front-end camera, and then the video stream output by the camera is subsequently processed to obtain multiple frames of facial images containing the target face. In step S100 of the embodiment of the present invention, multiple frames of facial images containing the target face are extracted from the video stream, which can be done by Figure 2 Steps S110 to S130 are implemented as shown.

[0025] Step S110: extracting multiple frames of first face images containing the target face from the video stream.

[0026] Exemplarily, step S110 may be performed as follows: Figure 3 Steps S111 and S112 shown are implemented.

[0027] Step S111, performing face detection on the video stream to obtain facial position information of the target face in the current frame of the video stream.

[0028] For example, a face detection network such as a multi-tasks cascade neural network (MTCNN) or RetinaFace can be used to obtain the position information of the target face in the current frame of the video. The position information may be information such as the position information of key points of the face and the boundary information of the face.

[0029] Step S112, performing face trajectory tracking according to the face position information, and extracting multiple frames of first face images containing the target face from the video stream.

[0030] For example, the position of the target face in the next frame of video can be predicted based on the position information of the target face in the current frame of video obtained during face detection, so as to achieve face trajectory tracking. By tracking the target face trajectory, the image of the target face can be captured from multiple frames of video stream, so as to obtain a series of face trajectory images containing the target face, and the series of face trajectory images are used as the first face images of multiple frames.

[0031] In some embodiments, the facial key point position information may specifically include a plurality of contour point position information, wherein the plurality of contour point position information may include left eye position information, right eye position information, nose position information, left mouth corner position information, and right mouth corner position information.

[0032] Correspondingly, such as Figure 3 As shown, step S110 may further include step S113, calibrating the angle of the target face in the first face image according to the position information of the plurality of contour points.

[0033] Exemplarily, since the target object may be active in the captured video stream, among the multiple frames of the first face image composed of a series of face track images obtained by face track tracking, there may be some images in which the angle of the target face is tilted. In this way, the above-mentioned contour point position information can be used to calibrate the target face in the first face image. Specifically, the above-mentioned multiple contour point position information can be input into the face calibration algorithm, and the face calibration algorithm can be used to perform tilt correction on the target face in the first face image.

[0034] Step S120 , performing face quality analysis processing on the multiple frames of first face images respectively to obtain face priori information of each frame of the first face image.

[0035] The embodiment of the present invention adopts a lightweight face quality evaluation algorithm to perform face quality analysis processing on each frame of the first face image, and obtains face prior information corresponding to each frame of the first face image.

[0036] Specifically, face quality evaluation in multiple dimensions may be performed on each frame of the first face image, and the face prior information thus obtained includes multiple different types of indicator parameters.

[0037] Exemplarily, the index parameters may include three types of index parameters: blur degree parameter, deflection angle parameter and resolution parameter. In specific implementation, the characteristic modulus of the first face image can be obtained by using a lightweight face feature extraction model, and the blur degree parameter is determined based on the obtained characteristic modulus. Generally speaking, the larger the characteristic modulus, the lower the blur degree; the first face image can be binarized using local feature binarization LBP, and a face symmetry index can be output, and the deflection angle parameter can be determined based on the face symmetry index. For example, when the symmetry index is 1, it represents a frontal face angle, and the deflection angle is 0; the interpupillary distance is determined using the left eye position information and the right eye position information obtained during face detection, and the resolution parameter is determined based on the interpupillary distance. Generally speaking, the larger the interpupillary distance, the higher the resolution, and the smaller the interpupillary distance, the lower the resolution.

[0038] It should be understood that the embodiments of the present invention are not limited to the above three indicator parameters, and may also include other different types of indicator parameters, or replace any one or more of the above three indicator parameters with other different types of indicator parameters, which is not limited in the embodiments of the present invention.

[0039] The embodiment of the present invention uses multiple different types of index parameters to perform multi-dimensional quality evaluation on each frame of the first face image, so as to reflect the strength of the facial detail features in the first face image in different dimensions. In the aforementioned example, a lightweight face quality scoring method is used to examine the face quality in three dimensions: blur degree, face deflection angle, and resolution. The obtained face prior information is used, on the one hand, to subsequently select high-quality images for face feature extraction, to ensure that the extracted features have good richness and feature diversity; on the other hand, it can act on the subsequent feature enhancement link to improve the generalization of face features.

[0040] Step S130, selecting a plurality of frames of second face images from the plurality of frames of first face images according to the face prior information of each frame of the first face image. Specifically, step S130 may include the following steps: Figure 4 Steps S131 to S133 are shown.

[0041] Step S131, linearly weighting multiple indicator parameters to obtain a global quality score, and obtaining a first preset number of preliminary selected images from multiple frames of first facial images according to the global quality score.

[0042] Exemplarily, the global quality score can be obtained by performing a linear weighted calculation on the multiple indicator parameters included in the face prior information in step S120. The global quality score can be used to perform a comprehensive quality assessment on each frame of the first face image, and each frame of the first face image can be ranked according to the global quality score, and the first face image with a high ranking can be selected as the preliminary image. In addition, the number of preliminary images to be obtained can be determined by presetting a first preset number value.

[0043] As an example, the first preset number can be a percentage value, such as setting the first preset number to 30%. When 100 frames of first face images are extracted from the video stream through step S110, and the face prior information of each frame of the first face image is obtained according to the method provided in step S120, a linear weighted calculation is performed on multiple indicator parameters contained in the face prior information to obtain a global quality score of each frame of the first face image. Then, according to the global quality score of each frame of the first face image, the 100 frames of first face images are ranked from high to low according to the score, and the top 30 first face images are taken as preliminary images.

[0044] Step S132, arranging and combining the first preset number of preliminary selection images to obtain a plurality of preliminary selection image combinations, wherein each preliminary selection image combination includes a second preset number of preliminary selection images.

[0045] Continuing with the above example, the second preset number can be set according to the number of second face images to be finally obtained, for example, the second preset number is set to 3. In this way, the 30 frames of preliminary selected images can be arranged and combined to obtain C3 3 0 preliminary image combinations, each of which contains 3 frames of preliminary images.

[0046] Step S133, obtaining an image distinction degree parameter of each preliminary selected image combination according to a plurality of index parameters, and selecting a final selected image combination from the plurality of preliminary selected image combinations according to the image distinction degree parameter.

[0047] Among them, the image differentiation degree parameter is used to characterize the differences in facial detail features between multiple frames of preliminary images contained in the preliminary image combination. Generally speaking, under the premise of ensuring the quality of the face, the greater the degree of differentiation of facial detail features, the more usable information the image combination contains, and the facial features extracted in this way show strong generalization and are more suitable for face recognition systems in open scenarios.

[0048] The image distinction degree parameter of the preliminary image combination can be determined by calculating the cumulative distances between the images in the combination in multiple dimensions to determine the image distinction degree parameter of the current preliminary image combination.

[0049] Continuing with the above example, assuming that the current preliminary image combination is T1, and combination T1 contains three images numbered P1, P2 and P3, calculate the distances between P1 and P2, P1 and P3, and P2 and P3 in each dimension, for example: the distances between P1 and P2 in the three dimensions of blur degree, face deflection angle, and resolution are calculated to be S1(P1P2), S2(P1P2), and S3(P1P2), respectively; the distances between P1 and P3 in the three dimensions of blur degree, face deflection angle, and resolution are calculated to be S1(P1P3), S2(P1P3), and S3(P1P3), respectively; the distances between P2 and P3 in the three dimensions of blur degree, face deflection angle, and resolution are calculated to be S1(P2P3), S2(P2P3), and S3(P2P3), respectively. Then the image distinction degree parameters of the preliminary image combination T1 are:

[0050] S1=S1(P1P2)+S2(P1P2)+S3(P1P2)+S1(P1P3)+S2(P1P3)+S3(P1P3)+S1(P2P3)+S2(P2P3)+S3(P2P3).

[0051] According to the calculated image distinction degree parameters of each preliminary selected image combination, the preliminary selected image combination with the largest image distinction degree parameter is selected as the final selected image combination.

[0052] Step S134: using the image included in the final selected image combination as the second face image.

[0053] When the final image combination is selected, the images included in the final image combination are used as the second face image. For example, if combination T1 is selected as the final image combination, images P1, P2 and P3 included in combination T1 are used as the second face image.

[0054] Step S200, extracting facial features from multiple frames of facial images to obtain first facial features.

[0055] Exemplarily, the facial image in step S200 may be a plurality of frames of second facial images obtained in step S134.

[0056] Exemplarily, a neural network can be used to extract facial features from multiple frames of facial images to obtain a first facial feature. The extracted first facial feature includes a multi-dimensional facial vector. The neural network can use a facial feature extraction algorithm such as Resnet152 to output a set of 256-dimensional deep facial features. These features represent the original facial image information encoding without feature enhancement.

[0057] Step S300, enhancing the first facial feature, and fusing the enhanced first facial feature to obtain a second facial feature.

[0058] Exemplarily, a deep convolutional neural network is used to perform a dot multiplication operation on the first facial feature and the facial prior information to obtain an enhanced first facial feature, wherein the facial prior information is obtained by performing facial quality analysis on the facial image in the aforementioned step S120.

[0059] Continuing with the previous example, the 256-dimensional deep facial features extracted from the second facial image by the Resnet152 algorithm and the facial prior information corresponding to the second facial image (blur degree parameter, facial deflection angle parameter, resolution parameter) are input into the deep convolutional neural network. The deep convolutional neural network performs a dot multiplication operation on the deep facial features and the facial prior information to enhance the facial features using the facial prior information output by the face quality evaluation algorithm.

[0060] Different from traditional image-level enhancement methods, such as image deblurring and super-resolution, the embodiment of the present invention adopts a feature-level enhancement method. Compared with the image-level enhancement method, the advantage of adopting feature-level enhancement is that the processing object is a set of multi-dimensional face vectors, the amount of calculation is small, and thus the processing efficiency can be greatly improved.

[0061] Exemplarily, the deep convolutional neural network used to achieve feature enhancement can be a series connection of two fully connected layers, trained on a facial feature extraction dataset, to obtain a feature enhancement module for compensating the original features. The three quality indicators output by the face quality scoring module reflect the strength of the face image in three dimensions: blur, deflection angle, and resolution. These indicators control the feature enhancement module to enhance the original features through a dot product operation.

[0062] After the enhancement process of the first facial feature is completed, the enhanced first facial feature is fused to obtain the second facial feature.

[0063] Specifically, the enhanced first facial features may be fused through an average pooling operation to obtain the second facial features.

[0064] Step S400, comparing the second facial feature with the pre-stored third facial feature to determine a facial recognition result.

[0065] Specifically, the Euclidean algorithm may be used to compare the second facial feature with the pre-stored third facial feature to determine the facial recognition result.

[0066] The face recognition method provided by the embodiment of the present invention is further exemplified below in conjunction with specific application scenarios.

[0067] Scenario 1: Smart city nighttime face monitoring scenario

[0068] In the national implementation of the construction of smart city informatization, intelligent security monitoring systems play an important role. Traditional face recognition monitoring systems perform well on sunny days with good lighting conditions. However, at night, due to the complexity of night scenes, low brightness, aging of fill light equipment, poor angle configuration, temperature, rain and snow and many other reasons, traditional monitoring systems often experience a significant drop in recognition accuracy. The construction of urban nighttime monitoring systems is of great significance for the monitoring of wanted fugitives and idle people in society. Based on this background, this example illustrates a face recognition monitoring system in a city nighttime control scenario. When the face recognition method provided by an embodiment of the present invention is applied to the system, it can specifically include the following method steps:

[0069] Step S501, collect the facial image sets of fugitives, idle people in society, and key monitored persons. These facial images are usually frontal and high-definition pictures, so no additional image processing is required. These facial images are encoded using a facial feature extraction algorithm and stored in the database to form a base database set.

[0070] Step S502, obtain the monitoring video collected by the monitoring equipment in the monitoring area at night, and the monitoring area can be a fixed area such as a community, a street, etc. The monitoring video can be transmitted in an online video stream or saved locally offline. The video stream information will be transmitted to the back-end data processing module for video image analysis.

[0071] Step S503: perform face detection and trajectory tracking on the video information collected by each monitoring device to obtain a group of face trajectory images containing the target face.

[0072] Step S504, a lightweight face quality evaluation algorithm is used to score each face image in the trajectory in three dimensions: blur, deflection angle, and resolution. At the same time, the global quality score result of the three indicators is also output, and its score is the linear weighted score of the three indicators, and the weighting coefficient is obtained by the regression method. The quality of the face image is given by the global quality score, and the three indicators reflect the strength of the face detail features in different dimensions.

[0073] Step S505 , based on the global quality score and the indicators of the three dimensions, multiple images with relatively high quality and large degree of distinguishing facial detail features are selected from the face track images as a face candidate set.

[0074] Step S506, extract facial features from the images in the face candidate set, enhance the extracted facial features using a feature-level enhancement method, perform an average pooling operation on the enhanced facial features, merge all facial features in the face candidate set, and output the facial features that are ultimately used for subsequent comparison and matching.

[0075] Step S507, compare the facial features output in step 506 with the facial features stored in the database, calculate the Euclidean distance between the two, and when the distance is less than a certain threshold, it is considered that the captured face matches the ID of the fugitive or social idler stored in the database. Send a signal to the terminal device and display the recognition result on the display device.

[0076] Scenario 2: Monitoring and analyzing the activity trajectories of people in a crowded environment

[0077] In the intelligent upgrading and transformation of large public places, the use of personnel activity trajectory information to count personnel residence time, personnel density, and crowd flow has high economic value and social significance. For example, by counting personnel activity trajectories and analyzing personnel flow in the subway transfer center, personnel evacuation channels can be reasonably deployed to improve transfer efficiency. For another example, in large shopping malls, analyzing personnel residence time and crowd flow has important reference value for rationally arranging exhibition area locations and arranging commodity sales areas. When connecting personnel activity trajectories in series, due to the limited acquisition range of a single video acquisition device, facial images often come from multiple acquisition devices. In order to obtain the correct facial trajectory path, facial features must have strong generalization to match the correct trajectory path. However, in crowded scenes, faces are often prone to occlusion, side faces, and motion blur, and these noises seriously hinder the stability of facial features. Based on this background, this example illustrates a personnel activity trajectory monitoring and analysis system in crowded scenes. When the face recognition method provided by the embodiment of the present invention is applied to the system, it can specifically include the following method steps:

[0078] Step S601, obtain the surveillance video of each surveillance device in a public place within a period of time, where the place can be a shopping mall, subway transfer center, airport and other public areas. Give each surveillance video collected by the surveillance device an ID number, such as ID1, ID2, ..., IDN. The video data collected by these surveillance devices are transmitted to the background for video image analysis.

[0079] Step S602: using face detection and face tracking methods, processing the video stream collected by the monitoring device of each ID to obtain a face track image set corresponding to the ID.

[0080] Step S603: Use a lightweight face quality evaluation algorithm to score each face image in the face track image set corresponding to the ID in terms of blur, deflection angle, and resolution. At the same time, the global quality score result combining the three indicators is also output.

[0081] Step S604: Generate a face candidate set corresponding to the ID based on the global quality score and the indicators of the three dimensions.

[0082] Step S605, extracting, enhancing and fusing facial features of the images included in the face candidate set corresponding to the ID, and outputting facial features corresponding to the ID;

[0083] Step S606, each ID contains a certain number of facial features, which represent the number of people captured by this monitoring device during this period of time. The Euclidean distance between the facial features of ID1, ID2, ..., IDN is calculated. When the distance is less than a certain threshold, the identity is considered to be matched successfully, and the trajectory information of the person under different ID monitoring devices is associated;

[0084] Step S607, save the personnel trajectory in the form of a timeline in the database, or display it on the interface for the operator to review or use.

[0085] The solution provided by the embodiment of the present invention is to address the problem that the traditional solution is easily interfered by various noises in an open monitoring scenario. A large number of optimizations are performed on the data processing unit, which greatly improves the overall performance of the face recognition monitoring system. Specifically, it is embodied in:

[0086] The recognition accuracy of the system is greatly improved: In typical monitoring scenarios, the single face image captured by the face detection algorithm is often interfered by noise, and various types of missing features often exist in the face details. For example, in a certain monitoring area, when the distance between the object and the acquisition device is large, the acquired face image is often affected by out-of-focus blur, and when the distance between the two is small, the face image often produces motion blur. The face recognition method proposed in the present invention uses multiple face images of the same object with a motion trajectory for feature fusion extraction, which effectively avoids the information loss that may occur in the traditional use of a single face image. At the same time, the prior information of the face image in multiple dimensions is used to enhance the face features, and the face features finally obtained are highly generalized. At the same time, the face image is enhanced by feature enhancement, and for some highly incomplete face images, such as yin-yang faces, large deflection angles, scarf and mask occlusion, etc., considerable feature generalization can also be maintained.

[0087] Greatly improve the operating efficiency of the system: Under the premise of ensuring high recognition accuracy of the system, the face recognition method proposed in the present invention is based on the principle of lightweight design, and uses a lightweight deep neural convolutional network in the face quality evaluation, face feature fusion and face feature enhancement modules. For example, the face quality evaluation algorithm performs calculations in three dimensions: blur, deflection angle and resolution, thereby avoiding excessive occupation of system performance and being able to better meet the real-time requirements of the face recognition monitoring system.

[0088] Figure 5 FIG. 7 shows an electronic device 70 provided by an embodiment of the present invention. Figure 5 As shown, the electronic device 70 includes but is not limited to:

[0089] A memory 72, used for storing programs;

[0090] The processor 71 is used to execute the program stored in the memory 72. When the processor 71 executes the program stored in the memory 72, the processor 71 is used to execute the above-mentioned face recognition method.

[0091] The processor 71 and the memory 72 may be connected via a bus or other means.

[0092] The memory 72 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs and non-transitory computer executable programs, such as the face recognition method described in the embodiment of the present invention. The processor 71 implements the above-mentioned face recognition method by running the non-transitory software program and instructions stored in the memory 72.

[0093] The memory 72 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data for executing the above-mentioned face recognition method. In addition, the memory 72 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 72 may optionally include a memory remotely disposed relative to the processor 71, and these remote memories may be connected to the processor 71 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0094] The non-transient software program and instructions required to implement the above-mentioned face recognition method are stored in the memory 72. When executed by one or more processors 71, the above-mentioned face recognition method is executed, for example, Figure 1 The method steps S100 to S400 described in Figure 2 The method steps S110 to S130 described in Figure 3 The method steps S111 to S113 described in Figure 4 Method steps S131 to S134 described in .

[0095] An embodiment of the present invention further provides a storage medium storing computer executable instructions, wherein the computer executable instructions are used to execute the above-mentioned face recognition method.

[0096] In one embodiment, the storage medium stores computer executable instructions, and the computer executable instructions are executed by one or more control processors 71, for example, by a processor 71 in the electronic device 70, so that the one or more processors 71 can execute the above-mentioned face recognition method, for example, execute Figure 1 The method steps S100 to S400 described in Figure 2 The method steps S110 to S130 described in Figure 3 The method steps S111 to S113 described in Figure 4 Method steps S131 to S134 described in .

[0097] The above described embodiments are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0098] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0099] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above implementation. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A face recognition method, characterized in that: include: Extracting multiple frames of first face images containing the target face from the video stream; Performing face quality analysis on the plurality of frames of the first face images respectively to obtain face prior information of each frame of the first face image, wherein the face prior information includes a plurality of different types of indicator parameters; Linearly weighting the plurality of index parameters to obtain a global quality score, and acquiring a first preset number of preliminary selected images from the plurality of frames of the first facial images according to the global quality score; Arrange and combine the first preset number of the preliminary selected images to obtain a plurality of preliminary selected image combinations, wherein each preliminary selected image combination includes a second preset number of the preliminary selected images; Acquire an image distinction degree parameter of each of the preliminary selected image combinations according to the plurality of index parameters, and select a final selected image combination from the plurality of preliminary selected image combinations according to the image distinction degree parameter; Using the initially selected image included in the finally selected image combination as the second face image; Extracting facial features from the plurality of frames of the second facial images to obtain first facial features; Performing feature enhancement on the first facial feature, and fusing the enhanced first facial feature to obtain a second facial feature; The second facial feature is compared with a pre-stored third facial feature to determine a facial recognition result.

2. The face recognition method according to claim 1, characterized in that: The index parameters include a blur degree parameter, a deflection angle parameter and a resolution parameter.

3. The face recognition method according to claim 1, characterized in that: The step of extracting multiple frames of first face images containing the target face from the video stream includes: Performing face detection on the video stream to obtain face position information of the target face in the current frame of the video stream; The face trajectory is tracked according to the face position information, and a plurality of frames of first face images containing the target face are extracted from the video stream.

4. The face recognition method according to claim 3, characterized in that: The facial position information includes a plurality of contour point position information; The step of extracting a plurality of frames of first face images containing the target face from the video stream further comprises: The angle of the target face in the first face image is calibrated according to the position information of the plurality of contour points.

5. The face recognition method according to claim 1, characterized in that: The extracting facial features from the plurality of frames of the second facial images to obtain the first facial features includes: A neural network is used to extract facial features from multiple frames of the second facial images to obtain the first facial features; wherein the extracted first facial features include multi-dimensional facial vectors.

6. The face recognition method according to claim 5, characterized in that: The step of enhancing the first facial feature includes: A deep convolutional neural network is used to perform a dot multiplication operation on the first facial feature and the facial prior information to obtain an enhanced first facial feature; wherein the facial prior information is obtained by performing facial quality analysis processing on the second facial image.

7. The face recognition method according to claim 1 or 5, characterized in that: The step of fusing the enhanced first facial features to obtain the second facial features includes: The enhanced first facial features are fused through an average pooling operation to obtain second facial features.

8. The face recognition method according to claim 1, characterized in that: Comparing the second facial feature with a pre-stored third facial feature to determine a facial recognition result includes: The second facial feature is compared with the pre-stored third facial feature using the Euclidean algorithm to determine a facial recognition result.

9. An electronic device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the steps of the method according to any one of claims 1 to 8 are implemented when the processor executes the program.

10. A computer-readable storage medium, characterized in that: A computer program is stored, and when the program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images