Facial recognition method and facial recognition device

Abstract

The invention discloses a face recognition method and a face recognition device, and belongs to the technical field of face recognition. The face recognition method comprises the following steps: acquiring a first face image of a user under infrared illumination and a second face image of the user under visible illumination; on the basis of the first face image and the second face image, when it is determined that the user is an unregistered user, registering the user on the basis of a pre-registration condition, the first face image and the second face image; and on the basis of the first face image and the second face image, under the condition that the user is determined to be a registered user, determining that the face recognition of the user is passed. According to the face recognition method, the user does not need to fill basic information and provide the third face image on site, the registration mode is convenient and flexible, the on-site registration efficiency is improved, and the method can be suitable for the registration process of various application scenes.

Description

Technical Field

[0001] The present application belongs to the field of facial recognition technology, and in particular, relates to a facial recognition method and a facial recognition device. Background Art

[0002] Dual-mode facial recognition is widely used in fields such as identity authentication, secure payment, and access control management. The application scenarios of dual-mode facial recognition mainly include two processes, namely user registration and user identification. In related technologies, users register through the dual-mode facial recognition registration terminal installed in the application scenario. However, the above registration method is less convenient and flexible, and the registration efficiency is low, which is not conducive to the promotion of application scenarios. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a facial recognition method and a facial recognition device, which do not require the user to fill in basic information and provide a third facial image on site, and the registration method is convenient and flexible, improves the efficiency of on-site registration, and can be applied to the registration process of various application scenarios.

[0004] In a first aspect, the present application provides a facial recognition method, which is applied to a dual-mode facial recognition terminal; the method comprises:

[0005] Acquire a first facial image of a user under infrared light and a second facial image of the user under visible light;

[0006] In the case where it is determined that the user is an unregistered user based on the first facial image and the second facial image, registering the user based on a pre-registration situation, the first facial image, and the second facial image; the pre-registration situation is used to indicate whether the user has provided pre-registration information in advance through pre-registration; the pre-registration information is obtained through pre-registration based on basic information corresponding to the user and a third facial image of the user under visible light;

[0007] In a case where it is determined that the user is a registered user based on the first facial image and the second facial image, it is determined that the user's facial recognition has passed.

[0008] According to the facial recognition method of the present application, whether the user is an unregistered user is determined by obtaining the first facial image and the second facial image. When the user is determined to be an unregistered user, the user is registered based on the pre-registration information, the first facial image and the second facial image obtained by the user through pre-registration, and the user is subsequently facially recognized based on the information after registration, thereby improving the accuracy of facial recognition and ensuring the security of application scenarios. During the registration process, the user is not required to fill in basic information and provide a third facial image on site. The registration method is convenient and flexible, thereby improving the efficiency of on-site registration and being applicable to the registration process of various application scenarios.

[0009] According to the facial recognition method of the present application, registering the user based on the pre-registration situation, the first facial image and the second facial image includes:

[0010] When it is determined that the pre-registration information is provided and the similarity between the third facial image and the second facial image in the pre-registration information is greater than a first similarity threshold, the user is registered based on the dual-mode feature value corresponding to the user obtained from the first facial image and the second facial image and the pre-registration information.

[0011] According to the facial recognition method of the present application, registering the user based on the pre-registration situation, the first facial image and the second facial image includes:

[0012] If it is determined that the pre-registration information is not provided, outputting prompt information; the prompt information is used to prompt the user to pre-register or register;

[0013] Registering the user based on the acquired pre-registration information corresponding to the user, the first facial image, and the second facial image;

[0014] or;

[0015] The user is registered based on the acquired basic information corresponding to the user, the first facial image, and the second facial image.

[0016] According to the facial recognition method of the present application, the determining that the pre-registration information does not exist includes:

[0017] Calculating the similarity between the second facial image and each historical visible light facial image in the registration information database; the historical visible light facial image is a visible light facial image corresponding to the pre-registered user;

[0018] When the similarities between the second facial image and each of the historical visible light facial images are not greater than a first similarity threshold, it is determined that the pre-registration information does not exist.

[0019] According to the facial recognition method of the present application, determining that the user is an unregistered user based on the first facial image and the second facial image includes:

[0020] Extracting features from the first facial image and the second facial image respectively to obtain a first feature value and a second feature value;

[0021] fusing the first eigenvalue and the second eigenvalue to obtain a dual-mode eigenvalue;

[0022] When the similarities between the dual-mode feature value and each historical dual-mode feature value in the registration information database are not greater than a second similarity threshold, the user is determined to be the unregistered user; and the historical dual-mode feature value is the dual-mode feature value corresponding to the registered user.

[0023] In a second aspect, the present application provides a facial recognition method, which is applied to a user terminal; the method comprises:

[0024] Acquire basic information corresponding to the user and a third facial image of the user under visible light;

[0025] performing face detection on the third facial image;

[0026] If the detection result is qualified, pre-registration is performed based on the basic information and the third facial image to obtain pre-registration information corresponding to the user; the pre-registration information is used to store and register the user;

[0027] When the detection result is unqualified, a new third facial image is acquired.

[0028] According to the facial recognition method of the present application, the pre-registration is performed based on the basic information and the third facial image to obtain the pre-registration information corresponding to the user, including:

[0029] Performing feature extraction on the third facial image to obtain a third feature value;

[0030] Based on the basic information and the third characteristic value, pre-registration is performed to obtain pre-registration information corresponding to the user.

[0031] In a third aspect, the present application provides a facial recognition device, which is applied to a dual-mode facial recognition terminal; the device comprises:

[0032] A first processing module, used to obtain a first facial image of a user under infrared light and a second facial image of the user under visible light;

[0033] a second processing module, configured to register the user based on a pre-registration situation, the first facial image, and the second facial image, if the user is determined to be an unregistered user based on the first facial image and the second facial image; the pre-registration situation is used to indicate whether the user has provided pre-registration information in advance through pre-registration; the pre-registration information is obtained by pre-registration based on basic information corresponding to the user and a third facial image of the user under visible light;

[0034] The third processing module is used to determine whether the user's facial recognition passes when it is determined that the user is a registered user based on the first facial image and the second facial image.

[0035] According to the facial recognition device of the present application, whether the user is an unregistered user is determined by obtaining the first facial image and the second facial image. When the user is determined to be an unregistered user, the user is registered based on the pre-registration information, the first facial image and the second facial image obtained by the user through pre-registration, and the user is subsequently facially recognized based on the information after registration, thereby improving the accuracy of facial recognition and ensuring the security of application scenarios. During the registration process, the user is not required to fill in basic information and provide a third facial image on site. The registration method is convenient and flexible, thereby improving the efficiency of on-site registration and being applicable to the registration process of various application scenarios.

[0036] In a fourth aspect, the present application provides a facial recognition device, applied to a user terminal; the device comprises:

[0037] a fourth processing module, configured to obtain basic information corresponding to the user and a third facial image of the user under visible light;

[0038] a fifth processing module, configured to perform face detection on the third facial image;

[0039] A sixth processing module, configured to perform pre-registration based on the basic information and the third facial image to obtain pre-registration information corresponding to the user when the detection result is qualified; the pre-registration information is used to store and register the user;

[0040] The seventh processing module is used to obtain a new third facial image when the detection result is unqualified.

[0041] In a fifth aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the facial recognition method as described in the first aspect above is implemented.

[0042] In a sixth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the facial recognition method as described in the first aspect above is implemented.

[0043] In a seventh aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the facial recognition method as described in the first aspect above.

[0044] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0045] By obtaining the first facial image and the second facial image, it is determined whether the user is an unregistered user. When it is determined that the user is an unregistered user, the user is registered based on the pre-registration information, the first facial image and the second facial image obtained by the user through pre-registration, and the user is subsequently recognized by facial recognition based on the information after registration, thereby improving the accuracy of facial recognition and ensuring the security of application scenarios. During the registration process, the user is not required to fill in basic information and provide a third facial image on site. The registration method is convenient and flexible, which improves the efficiency of on-site registration and can be applied to the registration process of various application scenarios.

[0046] Furthermore, by determining that the similarity between the third facial image and the second facial image is greater than the first similarity threshold, it is determined that the user has pre-registered, and quick registration is performed based on the pre-registration information obtained by pre-registration and the dual-mode feature values ​​corresponding to the first facial image and the second facial image. The user does not need to fill in information on site in the application scenario, thereby improving registration efficiency, and there is no need to store the acquired images, thereby reducing the amount of stored data and improving the user experience.

[0047] Furthermore, by performing feature extraction on the first facial image and the second facial image, a first eigenvalue and a second eigenvalue representing key features of the first facial image and the second facial image are obtained, and the first eigenvalue and the second eigenvalue are fused to obtain a dual-mode eigenvalue representing richer features. The user is determined to be an unregistered user through the dual-mode eigenvalue, and there is no need to match the first eigenvalue and the second eigenvalue separately, thereby improving recognition efficiency and recognition accuracy.

[0048] Furthermore, basic information corresponding to the user and a third facial image corresponding to the user are obtained through the user terminal, and the filling of the basic information of the user is separated from the acquisition of the visible light facial image and the infrared light facial image, thereby reducing the dependence of the registration process on dual-mode facial recognition, and performing facial detection on the third facial image, effectively removing the third facial image with poor image quality, and obtaining a third facial image that meets the subsequent registration and facial recognition requirements, thereby improving the availability of the obtained data, reducing the occurrence of invalid pre-registration, and improving user experience.

[0049] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0051] Figure 1 This is one of the flowcharts of the facial recognition method provided in the embodiment of the present application;

[0052] Figure 2 This is the second flow chart of the facial recognition method provided in the embodiment of the present application;

[0053] Figure 3 This is the third flow chart of the facial recognition method provided in the embodiment of the present application;

[0054] Figure 4 This is the fourth flow chart of the facial recognition method provided in the embodiment of the present application;

[0055] Figure 5 This is one of the structural schematic diagrams of the facial recognition device provided in the embodiment of the present application;

[0056] Figure 6 This is the fifth flow chart of the facial recognition method provided in the embodiment of the present application;

[0057] Figure 7 This is the second structural diagram of the facial recognition device provided in the embodiment of the present application;

[0058] Figure 8 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0060] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0061] The facial recognition method, facial recognition device, electronic device and readable storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0062] Among them, the facial recognition method can be applied to a terminal, and can be specifically executed by hardware or software in the terminal.

[0063] The terminal includes but is not limited to portable communication devices such as mobile phones or tablet computers. It should also be understood that in some embodiments, the terminal may not be a portable communication device, but a desktop computer.

[0064] In the following various embodiments, a terminal including a display and a touch-sensitive surface is described. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, a mouse and a joystick.

[0065] The facial recognition method provided in the embodiment of the present application may be executed by an electronic device or a functional module or functional entity in the electronic device that can implement the facial recognition method. The electronic devices mentioned in the embodiment of the present application include but are not limited to mobile phones, tablet computers, computers, cameras, and wearable devices. The facial recognition method provided in the embodiment of the present application is described below using the electronic device as an example of the execution subject.

[0066] like Figure 1 As shown, the facial recognition method is applied to a dual-mode facial recognition terminal, and the method includes: step 110, step 120 and step 130.

[0067] Step 110: Acquire a first facial image of the user under infrared light and a second facial image of the user under visible light;

[0068] In this step, the user is the user whose face is being recognized at the current acquisition moment.

[0069] Infrared light is light that is located between visible light and microwaves in the electromagnetic spectrum. It is beyond the perception range of the human eye and has thermal effects and penetration.

[0070] The first facial image is collected under infrared light, has a strong ability to resist environmental interference, and can still well reflect information such as the user's temperature under low visibility conditions such as fog, haze and night.

[0071] The facial features in the first facial image are difficult to forge, and can enhance the recognition capability against disguise attacks such as photos and videos.

[0072] Visible light is the light in the electromagnetic spectrum that can be perceived by the human eye and is commonly used in production and life, such as natural light and artificial light.

[0073] The second facial image is an image collected under visible light and can reflect user information such as the user's outline and texture.

[0074] It should be noted that the collection of the second facial image has higher requirements on the environment. For example, in the case of insufficient lighting conditions or bad weather conditions, the quality of the second facial image will be affected. The first facial image has lower requirements on the environment and high-quality images can still be collected even in bad environments.

[0075] The first facial image and the second facial image may be used together for subsequent facial recognition of the user to determine whether the user is a registered user.

[0076] In some embodiments, acquiring a first facial image of a user under infrared light and a second facial image of the user under visible light may further include:

[0077] Acquire an initial first facial image of the user under infrared light and an initial second facial image of the user under visible light;

[0078] Performing face detection on the initial first face image and the initial second face image respectively;

[0079] When it is determined that the detection result corresponding to the initial first facial image is qualified and the detection result corresponding to the initial second facial image is qualified, determining the initial first facial image as the first facial image and determining the initial second facial image as the second facial image;

[0080] When it is determined that at least one of the detection results corresponding to the initial first facial image and the initial second facial image is unqualified, the first facial image of the user under infrared light and the second facial image of the user under visible light are reacquired.

[0081] In this embodiment, the initial first facial image is an image captured under infrared light, and it is not determined whether the image quality can be used for subsequent facial recognition.

[0082] The initial second facial image is an image captured under visible light, and it is not determined whether the image quality can be used for subsequent facial recognition.

[0083] Face detection is used to detect image quality.

[0084] Facial detection may include: facial integrity detection, clarity detection, lighting detection, posture detection, occlusion detection, resolution detection and other image quality related detection items.

[0085] In the actual implementation process, after the initial first facial image and the initial second facial image are acquired, face detection can be performed on the initial first facial image and the initial second facial image respectively by using a face detection algorithm.

[0086] When it is determined that the facial integrity detection, clarity detection, lighting detection, posture detection, occlusion detection and resolution detection of the initial first facial image and the initial second facial image are all qualified, the initial first facial image is determined as the first facial image, and the initial second facial image is determined as the second facial image.

[0087] When it is determined that one or more of the detection items such as facial integrity detection, clarity detection, lighting detection, posture detection, occlusion detection and resolution detection of the initial first facial image and the initial second facial image are unqualified, the initial first facial image and the initial second facial image are re-captured, and facial detection is performed on the newly acquired initial first facial image and the initial second facial image until the detection result corresponding to the initial first facial image is qualified and the detection result corresponding to the initial second facial image is qualified.

[0088] According to the facial recognition method provided by the embodiment of the present application, facial detection is performed on the initial first facial image and the initial second facial image to determine whether the collected initial first facial image and the initial second facial image can be used for subsequent facial recognition, thereby improving the usability of the acquired initial first facial image and the initial second facial image, reducing invalid facial recognition, and improving the user experience.

[0089] By combining the first facial image and the second facial image to perform facial recognition on the user, the advantages of the first facial image and the second facial image can be fully utilized to improve the precision and accuracy of facial recognition of the user, and it can be applicable to complex and changing environments to provide more reliable facial recognition services.

[0090] During the actual implementation process, the first facial image and the second facial image can be acquired through a dual-mode facial recognition terminal equipped with visible light and infrared cameras.

[0091] Dual-mode facial recognition is a facial recognition method that uses both an infrared light facial image and a visible light facial image of the user to identify the user and authenticate the user.

[0092] The dual-mode facial recognition terminal can collect the infrared light facial image and the visible light facial image of the current user, and obtain the infrared light facial image and the visible light facial image of the registered user.

[0093] The first facial image may also be acquired through an infrared light facial image sensor, and the second facial image may also be acquired through a visible light facial image sensor.

[0094] Of course, in the actual implementation process, the first facial image of the user under infrared light and the second facial image of the user under visible light can also be obtained by any other feasible method, and this application is not limited to this.

[0095] Step 120: if it is determined that the user is an unregistered user based on the first facial image and the second facial image, register the user based on the pre-registration status, the first facial image and the second facial image;

[0096] In this step, the unregistered user is a user whose data representing the user identity is not stored in the registration information database in the relevant application scenario.

[0097] Among them, relevant application scenarios may include: shopping malls and supermarkets, transportation, university management, and financial payment scenarios.

[0098] The registration information database is a database that stores user pre-registration information and registration information.

[0099] In some embodiments, in different application scenarios, registration information databases may be established to store the user's pre-registration information and registration information, respectively, to improve the efficiency of subsequent facial recognition based on the data in the registration information database.

[0100] The registration information is the information obtained after the user registers.

[0101] The registration information may include: a first facial image, a second facial image, and basic information of the user.

[0102] The pre-registration status is used to indicate whether the user has provided pre-registration information in advance through pre-registration.

[0103] The pre-registration status may include: pre-registration has been performed and pre-registration has not been performed.

[0104] If the pre-registration has been performed, the pre-registration information corresponding to the user can be queried in the registration information database.

[0105] If pre-registration is not performed, the pre-registration information corresponding to the user cannot be queried in the registration information database.

[0106] The pre-registration information is obtained by pre-registration based on the basic information corresponding to the user and the third facial image of the user under visible light.

[0107] Basic information is information that can identify or describe the user's identity, attributes or characteristics.

[0108] Basic information may include: the user's name, age, gender, contact information, home address and other information.

[0109] The third facial image is a facial image provided by the user under visible light. In the actual implementation process, when the user is determined to be an unregistered user through the first facial image and the second facial image, the user can be registered based on the actual situation of whether there is pre-registration information corresponding to the user in the registration information database, the first facial image and the second facial image.

[0110] The following describes the registration process when the pre-registration information corresponding to the user exists in the registration information database and the registration process when the pre-registration information corresponding to the user does not exist in the registration information database.

[0111] First, the registration process of the user's corresponding pre-registration information in the registration information database

[0112] When the pre-registration information corresponding to the user can be obtained, the pre-registration information, the first facial image and the second facial image are stored in a registration information database to register the user.

[0113] Second, the registration process where the user's corresponding pre-registration information does not exist in the registration information database

[0114] In the case where the pre-registration information corresponding to the user cannot be obtained, the user is registered based on the data provided by the user's subsequent operations.

[0115] The user's subsequent operations may include: providing basic information, a first facial image and a second facial image through a dual-mode registration device provided at the dual-mode facial recognition site, or providing pre-registration information through a user terminal.

[0116] like Figure 2 As shown, in the actual execution process, the basic information and the third facial image corresponding to the user can be obtained through the user terminal.

[0117] A registration interface for entering the user's corresponding basic information and the third facial image can be provided to the user. The user uploads the user's corresponding basic information and the third facial image through the user terminal to obtain the user's corresponding basic information and the user's third facial image under visible light.

[0118] User terminals are terminals that allow users to interact with the network to achieve information transmission and resource access.

[0119] User terminals may include: personal computers, mobile devices, smart TVs, etc.

[0120] During the actual implementation process, after the user fills in all the information or all the required information and before submitting the information in the registration interface, when the user clicks the Get Verification Code button, a verification code will be sent to the phone number provided by the user. The user will fill in the received verification code into the registration interface and submit the filled-in information and the verification code together.

[0121] After the user submits the registration page, the submitted verification code is verified. If the submitted verification code is consistent with the verification code stored on the server and sent to the user's mobile terminal, it is determined that the user has pre-registered and the pre-registration is successful, and the corresponding pre-registration information of the user is obtained.

[0122] After obtaining the basic information and the third facial image of the user, the basic information and the third facial image of the user may be bound to obtain the pre-registration information corresponding to the user.

[0123] If the verification code submitted is inconsistent with the verification code stored on the server and sent to the user's mobile terminal, the user will be prompted that the verification code is wrong. If all required information includes ID card information, the name, age, ID card number and other information entered can be checked for consistency to ensure the accuracy of the basic information provided by the user.

[0124] For example, when the name and ID number do not match, a prompt message is output to indicate that the name or ID number is wrong.

[0125] For another example, when the number of digits in the ID card number provided by the user is incorrect or the ID card number does not exist, a prompt message is output to remind the user that the ID card number is incorrect.

[0126] The pre-registration information may be stored in a registration information database for subsequent further registration based on the pre-registration information.

[0127] It should be noted that users do not need to be present at the dual-mode facial recognition site, and can pre-register through a user terminal equipped with a visible light facial image sensor. The user terminal equipped with a visible light facial image sensor can receive the basic information corresponding to the user and collect a third facial image corresponding to the user.

[0128] Step 130: When it is determined that the user is a registered user based on the first facial image and the second facial image, determine that the user facial recognition passes.

[0129] In this step, the registered user is a user whose data representing the user identity has been stored in the registration information database in the relevant application scenario.

[0130] During the actual execution process, the acquired first facial image and second facial image can be matched with the infrared light facial images (i.e., the first facial image) and visible light facial images (i.e., the second facial image) of different users stored in the registration information database. When the first facial image and the second facial image of the user match the infrared light facial image and the visible light facial image of a certain user stored in the registration information database, the user is determined to be a registered user, and it can be determined that the user's facial recognition has passed.

[0131] During the research and development process, the inventor discovered that in the related technology, users register through a dual-mode face recognition registration terminal installed in the application scenario. However, the above registration method is less convenient and flexible, which is not conducive to the promotion of application scenarios.

[0132] The present application obtains a first facial image of the user under infrared light and a second facial image under visible light to obtain multiple images for facial recognition of the user. Facial recognition is performed through multiple images, thereby improving the accuracy of facial recognition and ensuring the security of application scenarios. When the user is determined to be an unregistered user through the first facial image and the second facial image, the user is registered through pre-registration information obtained by pre-registration before the user is in the facial recognition scene and the first facial image and the second facial image obtained on the spot in the facial recognition scene. The user does not need to fill in basic information and provide a third facial image on site. The registration method is convenient and flexible, improves the efficiency of on-site registration, can collect user data in stages, reduces dependence on registration equipment, and can be applicable to the registration process of various application scenarios. After the user is registered, it can be determined whether the user's facial recognition has passed based on the first facial image and the second facial image of the user, thereby improving the facial recognition speed and recognition efficiency.

[0133] According to the facial recognition method provided by the embodiment of the present application, whether the user is an unregistered user is determined by obtaining the first facial image and the second facial image. When the user is determined to be an unregistered user, the user is registered based on the pre-registration information, the first facial image and the second facial image obtained by the user through pre-registration, and the user is subsequently subjected to facial recognition based on the information after registration, thereby improving the accuracy of facial recognition and ensuring the security of application scenarios. During the registration process, the user is not required to fill in basic information and provide a third facial image on site. The registration method is convenient and flexible, thereby improving the efficiency of on-site registration and being applicable to the registration process of various application scenarios.

[0134] like Figure 3 As shown, in some embodiments, based on the first facial image and the second facial image, determining that the user is an unregistered user may include:

[0135] Extracting features from the first facial image and the second facial image respectively to obtain a first feature value and a second feature value;

[0136] The first eigenvalue and the second eigenvalue are fused to obtain a dual-mode eigenvalue;

[0137] When the similarities between the dual-mode feature value and each historical dual-mode feature value in the registration information database are not greater than the second similarity threshold, it is determined that the user is an unregistered user.

[0138] In this embodiment, the first feature value is a value representing a key feature of the first facial image.

[0139] The key features of the first facial image may include: color features, texture features, shape features, frequency domain features, statistical features, deep learning features and other features.

[0140] The second feature value is a value representing a key feature of the second facial image.

[0141] The second facial image key features may include: temperature features, texture features, shape features, statistical features, frequency domain features and the like.

[0142] During the actual implementation process, features of the first facial image and the second facial image can be extracted respectively through models such as a deep learning model, a neural network model, and an artificial intelligence model to obtain a first eigenvalue and a second eigenvalue.

[0143] A bimodal eigenvalue is an eigenvalue that includes information in a first eigenvalue and information in a second eigenvalue.

[0144] In the actual implementation process, the first eigenvalue and the second eigenvalue can be fused by a feature fusion algorithm or feature concatenation to obtain a dual-mode eigenvalue.

[0145] Of course, in the actual implementation process, the first eigenvalue and the second eigenvalue may be fused by any other feasible method to obtain a dual-mode eigenvalue, which is not limited in this application.

[0146] After obtaining the bimodal eigenvalues, the bimodal eigenvalues ​​can be normalized to alleviate the differences in dimensions and value ranges between subsequent different features, so that different features and indicators are comparable.

[0147] The historical dual-mode feature value is the dual-mode feature value corresponding to the registered user.

[0148] The historical dual-mode feature values ​​have been stored in the registration information database before the current collection time.

[0149] The second similarity threshold is a preset value for determining whether the user has registered.

[0150] The specific value of the second similarity threshold can be customized by the user or determined based on actual conditions, and is not limited in this application.

[0151] In the actual implementation process, after the dual-mode feature value is obtained, the dual-mode feature value can be compared one by one with each historical dual-mode feature value in the registration information database.

[0152] When the similarity between the bi-modal feature value and any historical bi-modal feature value is greater than a second similarity threshold, it is determined that the user is a registered user.

[0153] When the similarities between the bi-modal feature value and all historical bi-modal feature values ​​are not greater than the second similarity threshold, it is determined that the user is an unregistered user.

[0154] According to the facial recognition method provided by the embodiment of the present application, by performing feature extraction on the first facial image and the second facial image, a first eigenvalue and a second eigenvalue characterizing key features of the first facial image and the second facial image are obtained, and the first eigenvalue and the second eigenvalue are fused to obtain a dual-mode eigenvalue characterizing richer features. The user is determined to be an unregistered user through the dual-mode eigenvalue, and there is no need to match the first eigenvalue and the second eigenvalue separately, thereby improving recognition efficiency and recognition accuracy.

[0155] In some embodiments, step 120 may further include:

[0156] If it is determined that the pre-registration information is not provided, output a prompt message;

[0157] Registering the user based on the acquired pre-registration information corresponding to the user, the first facial image, and the second facial image;

[0158] or;

[0159] The user is registered based on the acquired basic information corresponding to the user, the first facial image, and the second facial image.

[0160] In this embodiment, the absence of pre-registration information means that in the registration information database, there is no pre-registration information corresponding to the user who performs facial recognition.

[0161] In the actual implementation process, a question and answer pop-up window may be provided to the user to allow the user to enter whether he or she has pre-registered. If the user chooses not to pre-register, it is determined that there is no pre-registration information for the user.

[0162] The prompt information is used to prompt the user to pre-register or register.

[0163] The prompt message may be: the current user has not pre-registered, please pre-register first; or the user has not registered, please register through the registration device at position A.

[0164] Of course, in the actual implementation process, the prompt information can also be any other information that can prompt the user to pre-register or register, and this application does not limit it.

[0165] After the user pre-registers based on the prompt information, the pre-registration information corresponding to the user can be obtained. At this time, the user can be registered based on the pre-registration information corresponding to the user, the first facial image and the second facial image. The specific registration process has been described in detail above and will not be repeated here to avoid repetition.

[0166] If the user does not pre-register based on the prompt information, the user's facial recognition will fail.

[0167] When a user registers through an on-site registration device, the basic information, first facial image, and second facial image corresponding to the user can be obtained through the registration device, and the basic information, first facial image, and second facial image corresponding to the user obtained through the registration device are stored in a registration information database to register the user.

[0168] According to the facial recognition method provided in the embodiment of the present application, by outputting a prompt message to prompt the user to pre-register when it is determined that there is no pre-registration information of the user undergoing facial recognition, the user is registered based on the pre-registration information, the first facial image and the second facial image corresponding to the user, effectively prompting the user to complete the registration process and improving the user experience.

[0169] In some embodiments, determining that pre-registration information is not provided may include:

[0170] Calculating the similarity between the second facial image and each historical visible light facial image in the registration information database;

[0171] When the similarities between the second facial image and each historical visible light facial image are not greater than the first similarity threshold, it is determined that no pre-registration information is provided.

[0172] In this embodiment, the historical visible light facial image is a visible light facial image corresponding to a user who has been pre-registered.

[0173] During the actual implementation process, when the user is determined to be an unregistered user based on the first facial image and the second facial image, the similarity between the second facial image and each historical visible light facial image can be calculated to determine whether the user's pre-registration information is stored in the registration information database.

[0174] When the similarities between the second facial image and each historical visible light facial image are not greater than the first similarity threshold, it can be determined that the user has not pre-registered, thereby determining that there is no pre-registration information in the registration information database.

[0175] When the similarity between the second facial image and any of the historical visible light facial images is greater than the first similarity threshold, it can be determined that the user has pre-registered and that pre-registration information exists in the registration information database.

[0176] According to the facial recognition method provided in the embodiment of the present application, by calculating the similarity between the second facial image and each historical visible light facial image, and comparing the calculated similarity with the first similarity threshold, when the calculated similarities are not greater than the first similarity threshold, it is determined that pre-registration information is not provided, thereby enhancing the automated processing capability of the dual-mode facial recognition terminal, reducing manual intervention, and providing more objective and consistent comparison results.

[0177] In some embodiments, step 120 may further include:

[0178] When it is determined that pre-registration information is provided and the similarity between the third facial image and the second facial image in the pre-registration information is greater than a first similarity threshold, the user is registered based on the dual-mode feature value corresponding to the user obtained from the first facial image and the second facial image and the pre-registration information.

[0179] In this embodiment, the first similarity threshold is a preset value for determining whether the user has pre-registered.

[0180] The specific value of the first similarity threshold can be customized by the user or determined based on actual conditions, and is not limited in this application.

[0181] During the actual implementation process, when the user is determined to be an unregistered user based on the first facial image and the second facial image, it is possible to further determine whether the user has already pre-registered based on the similarity between the third facial image and the second facial image in the pre-registration information, thereby registering the user based on the actual situation of whether the user has pre-registered.

[0182] When the similarity between the third facial image and the second facial image is greater than the first similarity threshold, it is represented that an image similar to the second facial image can be retrieved from the pre-registration database, so that it can be determined that the user has pre-registered.

[0183] In an actual implementation process, the similarity between the third facial image and the second facial image may be determined by using the third eigenvalue corresponding to the third facial image and the second eigenvalue corresponding to the second facial image.

[0184] In the actual implementation process, it is also possible to determine whether the user has pre-registered based on the size relationship between the second difference between the third eigenvalue corresponding to the third facial image and the second eigenvalue corresponding to the second facial image and the difference threshold.

[0185] When the second difference between the third eigenvalue and the second eigenvalue is not greater than the difference threshold, it indicates that a eigenvalue close to the second eigenvalue can be retrieved from the pre-registration database, so that it can be determined that the user has pre-registered.

[0186] After determining that the user has pre-registered, the dual-mode feature values ​​and pre-registration information corresponding to the first facial image and the second facial image collected at the application scene can be stored in the registration information database to register the user.

[0187] In some embodiments, the registration information database may be divided into a plurality of sub-databases.

[0188] In this embodiment, the multiple sub-databases may include: a pre-registration information database and a registered information database.

[0189] The pre-registration information database may store pre-registration information obtained by pre-registering a user.

[0190] The registered information database can store the registration information obtained by registering the user.

[0191] It should be noted that the pre-registration information includes basic information corresponding to the user and information related to the third facial image corresponding to the user.

[0192] The registration information includes the user's basic information and the user's corresponding dual-mode feature value.

[0193] According to the facial recognition method provided in the embodiment of the present application, by dividing the registration information database into multiple sub-databases, it is possible to effectively manage the data generated at each registration stage of the user and improve the efficiency of subsequent data extraction.

[0194] like Figure 3 As shown, in the actual implementation process, the initial first facial image and the initial second facial image can be collected through the dual-mode facial recognition terminal. When the initial first facial image and the initial second facial image pass the facial detection, the initial first facial image and the initial second facial image are respectively determined as the first facial image and the second facial image for user registration or facial recognition.

[0195] In the process of registering or identifying a user based on the first facial image and the second facial image, it is necessary to extract features from the first facial image and the second facial image, and to fuse the obtained first feature value and the second feature value to obtain a dual-mode feature value, and to match the obtained dual-mode feature value with the historical dual-mode feature value in the registration information database.

[0196] When the similarity between the user's bi-modal feature value and any historical bi-modal feature value in the registration information database is greater than a second similarity threshold, it is determined that the user has been pre-registered, and the similarity between the third facial image and the second facial image in the pre-registration information corresponding to the user is further compared. When the similarity between the third facial image and the second facial image is greater than the first similarity threshold, the user is registered.

[0197] like Figure 4 As shown, after obtaining the first facial image and the second facial image through the dual-mode facial recognition terminal, facial detection can be performed on the first facial image and the second facial image respectively. When the detection result corresponding to the first facial image and the detection result corresponding to the second facial image are both qualified, feature extraction is performed on the first facial image and the second facial image respectively to obtain a first eigenvalue and a second eigenvalue, and the first eigenvalue and the second eigenvalue are fused to obtain a dual-mode eigenvalue, and the dual-mode eigenvalue is compared with the historical dual-mode eigenvalue in the registration information database. When there is a historical dual-mode eigenvalue with a smaller difference from the dual-mode eigenvalue in the historical dual-mode eigenvalue, the user passes facial recognition.

[0198] According to the facial recognition method provided in the embodiment of the present application, by determining that the similarity between the third facial image and the second facial image is greater than the first similarity threshold, it is determined that the user has pre-registered, and based on the pre-registration information obtained by the pre-registration and the dual-mode feature values ​​corresponding to the first facial image and the second facial image, quick registration is performed. The user does not need to fill in information on site in the application scenario, thereby improving registration efficiency, and there is no need to store the acquired images, thereby reducing the amount of stored data and improving the user experience.

[0199] The facial recognition method provided in the embodiment of the present application can be executed by a facial recognition device. In the embodiment of the present application, a facial recognition device executing the facial recognition method is taken as an example to illustrate the facial recognition device provided in the embodiment of the present application.

[0200] The embodiment of the present application also provides a facial recognition device.

[0201] like Figure 5 As shown, the facial recognition device is applied to a dual-mode facial recognition terminal, and the device includes: a first processing module 510 , a second processing module 520 and a third processing module 530 .

[0202] A first processing module 510, configured to obtain a first facial image of a user under infrared light and a second facial image of the user under visible light;

[0203] The second processing module 520 is used to register the user based on the pre-registration status, the first facial image and the second facial image when it is determined that the user is an unregistered user based on the first facial image and the second facial image; the pre-registration status is used to indicate whether the user has pre-registered information in advance through pre-registration; the pre-registration information is obtained by pre-registration based on the basic information corresponding to the user and the third facial image of the user under visible light;

[0204] The third processing module 530 is used to determine that the user face recognition passes when it is determined that the user is a registered user based on the first face image and the second face image.

[0205] According to the facial recognition device provided in the embodiment of the present application, whether the user is an unregistered user is determined by obtaining the first facial image and the second facial image. When the user is determined to be an unregistered user, the user is registered based on the pre-registration information, the first facial image and the second facial image obtained by the user through pre-registration, and the user is subsequently subjected to facial recognition based on the information after registration, thereby improving the accuracy of facial recognition and ensuring the security of application scenarios. During the registration process, the user is not required to fill in basic information and provide the third facial image on site. The registration method is convenient and flexible, thereby improving the efficiency of on-site registration and being applicable to the registration process of various application scenarios.

[0206] In some embodiments, the second processing module 520 may further include:

[0207] When it is determined that pre-registration information is provided and the similarity between the third facial image and the second facial image in the pre-registration information is greater than a first similarity threshold, the user is registered based on the dual-mode feature value corresponding to the user obtained from the first facial image and the second facial image and the pre-registration information.

[0208] In some embodiments, the second processing module 520 may further include:

[0209] If it is determined that the pre-registration information is not provided, output a prompt message; the prompt message is used to prompt the user to pre-register or register;

[0210] Registering the user based on the acquired pre-registration information corresponding to the user, the first facial image, and the second facial image;

[0211] or;

[0212] The user is registered based on the acquired basic information corresponding to the user, the first facial image, and the second facial image.

[0213] In some embodiments, the second processing module 520 may further include:

[0214] Calculating the similarity between the second facial image and each historical visible light facial image in the registration information database; the historical visible light facial image is a visible light facial image corresponding to the pre-registered user;

[0215] When the similarities between the second facial image and each historical visible light facial image are not greater than the first similarity threshold, it is determined that no pre-registration information is provided.

[0216] In some embodiments, the second processing module 520 may further include:

[0217] Extracting features from the first facial image and the second facial image respectively to obtain a first feature value and a second feature value;

[0218] The first eigenvalue and the second eigenvalue are fused to obtain a dual-mode eigenvalue;

[0219] When the similarities between the dual-mode feature value and each historical dual-mode feature value in the registration information database are not greater than the second similarity threshold, the user is determined to be an unregistered user; and the historical dual-mode feature value is the dual-mode feature value corresponding to the registered user.

[0220] The present application also provides a facial recognition method.

[0221] like Figure 6As shown, the facial recognition method is applied to a user terminal, and the method includes: step 610, step 630 and step 640.

[0222] Step 610: Obtain basic information corresponding to the user and a third facial image of the user under visible light;

[0223] In this step, during the actual execution process, the user may upload basic information corresponding to the user and a third facial image of the user under visible light through the user terminal.

[0224] Step 620: performing face detection on the third facial image;

[0225] In this step, face detection is used to detect image quality.

[0226] Face detection is used to obtain high-quality third face images.

[0227] Facial detection may include: facial integrity detection, clarity detection, lighting detection, posture detection, occlusion detection, resolution detection and other image quality related detection items.

[0228] After the third facial image provided by the user is acquired, the quality of the third facial image may be detected by using a facial detection algorithm or an image recognition method.

[0229] Of course, in the actual implementation process, face detection can also be performed by any other feasible method, which is not limited in this application.

[0230] Step 630: If the detection result is qualified, pre-registration is performed based on the basic information and the third facial image to obtain pre-registration information corresponding to the user; the pre-registration information is used to store and register the user;

[0231] In this step, the qualified detection result means that the detection results corresponding to each detection included in the face detection are all qualified.

[0232] When the facial integrity detection, clarity detection, lighting detection, posture detection, occlusion detection and resolution detection items of the third facial image are all qualified, the basic information and the third facial image are bound, and the bound basic information and the third facial image are stored in the registration information database to pre-register the user in advance and obtain the pre-registration information corresponding to the user.

[0233] Step 640: When the detection result is unqualified, obtain a new third facial image.

[0234] In this step, the detection result is unqualified if one or more of the detection results corresponding to the various detections included in the face detection are unqualified.

[0235] If one or more of the detection items such as facial integrity detection, clarity detection, lighting detection, posture detection, occlusion detection and resolution detection of the third facial image fail to meet the requirements, it indicates that the image quality provided by the user is low, which affects the subsequent registration process. The user can be prompted to provide a new third facial image to obtain a new third facial image.

[0236] According to the facial recognition method provided in the embodiment of the present application, basic information corresponding to the user and a third facial image corresponding to the user are obtained through a user terminal, and the filling of the basic information of the user is separated from the acquisition of the visible light facial image and the infrared light facial image, thereby reducing the dependence of the registration process on dual-mode facial recognition, and performing facial detection on the third facial image, effectively removing the third facial image with poor image quality, and obtaining a third facial image that meets the subsequent registration and facial recognition requirements, thereby improving the availability of the acquired data, reducing the occurrence of invalid pre-registration, and improving the user experience.

[0237] In some embodiments, pre-registration is performed based on the basic information and the third facial image to obtain pre-registration information corresponding to the user, which may also include:

[0238] Performing feature extraction on the third facial image to obtain a third feature value;

[0239] Based on the basic information and the third characteristic value, pre-registration is performed to obtain pre-registration information corresponding to the user.

[0240] In this embodiment, the third feature value is a value representing a key feature of the third facial image.

[0241] During the actual implementation process, features of the third facial image can be extracted through models such as deep learning models, neural network models, and artificial intelligence models to obtain a third eigenvalue.

[0242] After the third characteristic value is obtained, pre-registration is performed based on the basic information and the third characteristic value to obtain pre-registration information corresponding to the user, and the pre-registration information can be stored in a registration information database.

[0243] In some embodiments, pre-registration is performed based on the basic information and the third characteristic value to obtain pre-registration information corresponding to the user, which may include:

[0244] Comparing the third eigenvalue with the eigenvalues ​​corresponding to each historical visible light facial image in the registration information database to obtain a first difference;

[0245] When the first difference is greater than the difference threshold, pre-registration is performed based on the basic information and the third characteristic value to obtain pre-registration information corresponding to the user.

[0246] In this embodiment, the first difference degree is the degree of difference between the third eigenvalue and the eigenvalue corresponding to the historical visible light facial image.

[0247] It should be noted that, when the first difference is large, it indicates that the similarity between the third facial image and the historical visible light facial image is low.

[0248] When the first difference is small, it indicates that the similarity between the third facial image and the historical visible light facial image is high.

[0249] The difference threshold is a preset value for determining whether any two feature values ​​represent the facial features of the same user.

[0250] The difference threshold can be user-defined or determined based on actual conditions, and this application does not limit it.

[0251] like Figure 2 As shown, in the actual implementation process, the user can provide basic information and a third facial image through the user terminal. After obtaining the third facial image provided by the user, facial detection is performed on the third facial image. When the detection result corresponding to the third facial image is qualified, feature extraction is performed on the third facial image to obtain a third feature value, and the third feature value is compared with the feature values ​​corresponding to each historical visible light facial image in the registration information database.

[0252] When the first difference between the third eigenvalue and the eigenvalues ​​corresponding to each historical visible light facial image exceeds the difference threshold, it indicates that the similarity between the third facial image and the historical visible light facial images is low, and it is determined that the user has not pre-registered. The third eigenvalue can be bound to the basic information corresponding to the user, and the user can be pre-registered to obtain pre-registration information, which can be stored in a pre-registration information database.

[0253] According to the facial recognition method provided in the embodiment of the present application, by performing feature extraction on the third facial image, a third feature value is obtained, and the facial features of the user are effectively obtained, so that pre-registration is performed based on the facial features that distinguish the user from other users and the basic information corresponding to the user, and pre-registration information is obtained, thereby improving the representativeness and distinctiveness of the pre-registration information, and there is no need to store the third facial image, thereby reducing the amount of data that needs to be stored and improving the utilization rate of the registration information data volume.

[0254] The facial recognition method provided in the embodiment of the present application can be executed by a facial recognition device. In the embodiment of the present application, a facial recognition device executing the facial recognition method is taken as an example to illustrate the facial recognition device provided in the embodiment of the present application.

[0255] The embodiment of the present application also provides a facial recognition device.

[0256] like Figure 7 As shown, the facial recognition device is applied to a user terminal, and the device includes: a fourth processing module 710 , a fifth processing module 720 , a sixth processing module 730 and a seventh processing module 740 .

[0257] The fourth processing module 710 is used to obtain basic information corresponding to the user and a third facial image of the user under visible light;

[0258] A fifth processing module 720, configured to perform face detection on the third facial image;

[0259] The sixth processing module 730 is used to perform pre-registration based on the basic information and the third facial image to obtain pre-registration information corresponding to the user when the detection result is qualified; the pre-registration information is used to store and register the user;

[0260] The seventh processing module 740 is used to obtain a new third facial image when the detection result is unqualified.

[0261] In some embodiments, the sixth processing module 730 may also be used to: extract features from the third facial image to obtain a third feature value;

[0262] Based on the basic information and the third characteristic value, pre-registration is performed to obtain the pre-registration information corresponding to the user. The facial recognition device in the embodiment of the present application can be an electronic device, or a component in an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a car electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc. It can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0263] The facial recognition device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an IOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0264] The facial recognition device provided in the embodiment of the present application can achieve Figures 1 to 4 and Figure 6 To avoid repetition, the various processes implemented by the method embodiment are not described here.

[0265] In some embodiments, Figure 8 As shown, an embodiment of the present application further provides an electronic device 800, including a processor 801, a memory 802, and a computer program stored in the memory 802 and executable on the processor 801. When the program is executed by the processor 801, each process of the above-mentioned facial recognition method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0266] The embodiment of the present application also provides a dual-mode facial recognition system.

[0267] In this embodiment, the dual-mode facial recognition system includes: an infrared light image sensor and a visible light image sensor.

[0268] In an embodiment, the infrared light image sensor is used to capture a first facial image of a user.

[0269] The visible light image sensor is used to capture a second facial image of the user.

[0270] The first facial image and the second facial image may be used to perform facial recognition on the user.

[0271] According to the dual-mode facial recognition system provided by the embodiment of the present application, whether the user is an unregistered user is determined by obtaining the first facial image and the second facial image. When the user is determined to be an unregistered user, the user is registered based on the pre-registration information, the first facial image and the second facial image obtained by the user through pre-registration, and the user is subsequently subjected to facial recognition based on the information after registration, thereby improving the accuracy of facial recognition and ensuring the security of application scenarios. During the registration process, the user is not required to fill in basic information and provide a third facial image on site, and the registration method is convenient and flexible, thereby improving the efficiency of on-site registration and being applicable to the registration process of various application scenarios.

[0272] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0273] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0274] An embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned facial recognition method when executed by a processor.

[0275] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0276] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned facial recognition method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0277] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0278] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0279] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0280] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

[0281] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0282] For example, the facial recognition method can be used in financial institutions, public transportation, and education scenarios. Financial institutions include but are not limited to banks, securities firms, or insurance institutions, public transportation includes but is not limited to subway stations, high-speed rail stations, and airports, and education scenarios include: examination rooms and teaching sign-in. In the above scenarios, all parties have been fully authorized before data is collected through facial recognition methods.

[0283] Of course, facial recognition methods can also be used in private scenarios, such as company attendance. When used in private scenarios, user authorization must be obtained before data is collected through facial recognition methods.

[0284] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0285] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.

Claims

1. A facial recognition method, characterized in that: Applied to a dual-mode facial recognition terminal; the method comprises: Acquire a first facial image of a user under infrared light and a second facial image of the user under visible light; In the case where it is determined that the user is an unregistered user based on the first facial image and the second facial image, registering the user based on a pre-registration situation, the first facial image, and the second facial image; the pre-registration situation is used to indicate whether the user has provided pre-registration information in advance through pre-registration; the pre-registration information is obtained through pre-registration based on basic information corresponding to the user and a third facial image of the user under visible light; In a case where it is determined that the user is a registered user based on the first facial image and the second facial image, it is determined that the user's facial recognition has passed.

2. The facial recognition method according to claim 1, characterized in that: The registering the user based on the pre-registration situation, the first facial image and the second facial image includes: When it is determined that the pre-registration information is provided and the similarity between the third facial image and the second facial image in the pre-registration information is greater than a first similarity threshold, the user is registered based on the dual-mode feature value corresponding to the user obtained from the first facial image and the second facial image and the pre-registration information.

3. The facial recognition method according to claim 1, characterized in that: The registering the user based on the pre-registration situation, the first facial image and the second facial image includes: If it is determined that the pre-registration information is not provided, outputting prompt information; the prompt information is used to prompt the user to pre-register or register; Registering the user based on the acquired pre-registration information corresponding to the user, the first facial image, and the second facial image; or; The user is registered based on the acquired basic information corresponding to the user, the first facial image, and the second facial image.

4. The facial recognition method according to claim 3, characterized in that: The determining that the pre-registration information is not provided includes: Calculating the similarity between the second facial image and each historical visible light facial image in the registration information database; the historical visible light facial image is a visible light facial image corresponding to the pre-registered user; When the similarities between the second facial image and each of the historical visible light facial images are not greater than a first similarity threshold, it is determined that the pre-registration information is not provided.

5. The facial recognition method according to any one of claims 1 to 4, characterized in that: The determining, based on the first facial image and the second facial image, that the user is an unregistered user includes: Extracting features from the first facial image and the second facial image respectively to obtain a first feature value and a second feature value; fusing the first eigenvalue and the second eigenvalue to obtain a dual-mode eigenvalue; When the similarities between the dual-mode feature value and each historical dual-mode feature value in the registration information database are not greater than a second similarity threshold, the user is determined to be the unregistered user; and the historical dual-mode feature value is the dual-mode feature value corresponding to the registered user.

6. A facial recognition method, characterized in that: Applied to a user terminal; the method comprises: Acquire basic information corresponding to the user and a third facial image of the user under visible light; performing face detection on the third facial image; If the detection result is qualified, pre-registration is performed based on the basic information and the third facial image to obtain pre-registration information corresponding to the user; the pre-registration information is used to store and register the user; When the detection result is unqualified, a new third facial image is acquired.

7. The facial recognition method according to claim 6, characterized in that: The pre-registration based on the basic information and the third facial image to obtain the pre-registration information corresponding to the user includes: Performing feature extraction on the third facial image to obtain a third feature value; Based on the basic information and the third characteristic value, pre-registration is performed to obtain pre-registration information corresponding to the user.

8. A facial recognition device, characterized in that: Applicable to a dual-mode facial recognition terminal; the device comprises: A first processing module, used to obtain a first facial image of a user under infrared light and a second facial image of the user under visible light; a second processing module, configured to register the user based on a pre-registration situation, the first facial image, and the second facial image, if the user is determined to be an unregistered user based on the first facial image and the second facial image; the pre-registration situation is used to indicate whether the user has provided pre-registration information in advance through pre-registration; the pre-registration information is obtained by pre-registration based on basic information corresponding to the user and a third facial image of the user under visible light; The third processing module is used to determine whether the user's facial recognition passes when it is determined that the user is a registered user based on the first facial image and the second facial image.

9. A facial recognition device, characterized in that: Applied to a user terminal; the device comprises: a fourth processing module, configured to obtain basic information corresponding to the user and a third facial image of the user under visible light; a fifth processing module, configured to perform face detection on the third facial image; A sixth processing module, configured to perform pre-registration based on the basic information and the third facial image to obtain pre-registration information corresponding to the user when the detection result is qualified; the pre-registration information is used to store and register the user; The seventh processing module is used to obtain a new third facial image when the detection result is unqualified.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the facial recognition method according to any one of claims 1 to 7 is implemented.

Images