Face recognition method and electronic device

Abstract

The application provides a face recognition method and an electronic device. The method can be applied to an electronic device comprising a multispectral camera and an infrared flash. When a face is registered, the electronic device emits infrared light through the infrared flash and captures multiple face images of the user at different angles through the multispectral camera. The electronic device can establish a multispectral face model based on the multiple face images captured by the multispectral camera. In a face recognition scenario such as face unlocking the device or face payment, the electronic device can turn on the infrared flash and the multispectral camera. The electronic device can perform face recognition according to the image captured by the multispectral camera and the multispectral face model. The above method can reduce the cost of face recognition and ensure the accuracy of face recognition.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to facial recognition methods and electronic devices. Background Technology

[0002] With the widespread use of mobile phones and other electronic devices, applications based on facial recognition technology, such as facial unlocking devices and facial payment, are becoming increasingly common. Improving the accuracy of facial recognition is beneficial for protecting users' privacy and personal property security on electronic devices. Summary of the Invention

[0003] This application provides a face recognition method and electronic device. The method uses an infrared flash and a multispectral camera for face recognition, which can reduce the cost of face recognition and improve the accuracy of face recognition.

[0004] Firstly, this application provides a face recognition method. The method is applied to an electronic device, which includes an infrared flash and a multispectral camera. The image sensor of the multispectral camera includes a filter array, which comprises multiple filter units arranged in an array. Each filter unit includes N rows * N columns of sub-units arranged in an array, where N is greater than 2. The N rows * N columns of sub-units include multiple infrared filter sub-units and multiple visible light filter sub-units. The infrared filter sub-units are used to filter infrared light, and the visible light filter sub-units are used to filter visible light. The multiple infrared filter sub-units are evenly spaced along the horizontal and vertical directions, with one or more visible light filter sub-units spaced between two infrared filter sub-units. The electronic device emits infrared light through the infrared flash; the electronic device captures a first image through the multispectral camera, the first image containing a second face; the electronic device performs anti-spoofing detection on the second face based on the first image, and detects whether the second face matches the first face stored in the electronic device; if the anti-spoofing detection passes and the second face matches the first face, the electronic device determines that face recognition is successful.

[0005] The first image may include one or more images containing a second face.

[0006] The image sensor for the multispectral camera can be referenced in the embodiments of this application. Figures 3A to 3D as well as Figure 4A and Figure 4B The diagram shown is shown in the image.

[0007] As can be seen, the uniform distribution of infrared filter sub-units in a multispectral camera avoids image distortion. Electronic devices can capture clear facial images using a multispectral camera, thereby improving the accuracy of facial recognition. Specifically, electronic devices use a multispectral camera and an infrared flash for facial recognition. Multispectral cameras and infrared flashes are inexpensive, thus reducing the cost of facial recognition. The images captured by the multispectral camera contain the spectral features of the face, enabling anti-spoofing detection of faces, improving the accuracy of facial recognition, and better protecting users' privacy and personal property security within electronic devices.

[0008] In conjunction with the first aspect, in some embodiments, the image sensor of the multispectral camera has 5 or more types of visible light filter photonic units.

[0009] In conjunction with the first aspect, in some embodiments, the image sensor includes multiple channels, with one channel corresponding to one sub-unit. The visible light filter sub-unit corresponding to the infrared filter sub-unit includes a first visible light filter sub-unit and a second visible light filter sub-unit. The first visible light filter sub-unit and the infrared filter sub-unit are arranged in a row or column. The second visible light filter sub-unit is located diagonally opposite the infrared filter sub-unit. The channel corresponding to the first visible light filter sub-unit is the first channel, and the channel corresponding to the second visible light filter sub-unit is the second channel. The sensitivity of the second channel is greater than that of the first channel.

[0010] It can be seen that in the visible light filter units adjacent to the infrared filter unit, the channel corresponding to the visible light filter unit located diagonally opposite the infrared filter unit has greater sensitivity. This prevents electrons from overflowing from the pixels corresponding to the second part of the visible light filter unit to adjacent pixels, thereby preventing interference with the pixels corresponding to the infrared filter unit and thus avoiding affecting the infrared channel imaging used for anti-counterfeiting detection.

[0011] In conjunction with the first aspect, in some embodiments, the filter unit includes 4*4 sub-units. In each filter unit, the number of infrared filter sub-units is 4, and the number of visible light filter sub-units is 6. Each type of visible light filter sub-unit is repeated twice. Alternatively, in each filter unit, the number of visible light filter sub-units is 8. Among the 8 types of visible light filter sub-units, 4 types of visible light filter sub-units are repeated twice. Different types of visible light filter sub-units are used to filter visible light of different spectral bands.

[0012] In conjunction with the first aspect, in some embodiments, the filtering unit includes an infrared filtering subunit for collecting infrared light greater than or equal to 820 nm and less than or equal to 870 nm, and / or an infrared filtering subunit for collecting infrared light greater than or equal to 920 nm and less than or equal to 970 nm.

[0013] In conjunction with the first aspect, in some embodiments, the electronic device further includes a screen, and before the electronic device captures the first image using a multispectral camera, the electronic device detects the ambient light brightness; if the ambient light brightness is lower than the first brightness, the electronic device controls the brightness of the screen to be greater than or equal to a second brightness.

[0014] It can be seen that in low-light environments, electronic devices can use screen illumination. This not only eliminates the need to add additional lighting devices to electronic devices, reducing the cost of facial recognition, but also improves the accuracy of facial recognition in low-light environments, thereby enhancing the experience of facial unlocking devices or facial payment in low-light environments.

[0015] In conjunction with the first aspect, in some embodiments, the image sensor includes multiple third channels and multiple fourth channels, one third channel corresponding to one infrared filter subunit, one fourth channel corresponding to one visible light filter subunit, and the sensitivity of the fourth channel being less than the first sensitivity; if the ambient light brightness is greater than the third brightness, the electronic device generates a first image based on the signals from the multiple third channels and multiple fourth channels.

[0016] It can be seen that in overly bright scenes, the visible light channel with high sensitivity is prone to overexposure, leading to information loss. Therefore, in overly bright scenes, using the visible light channel with low sensitivity instead of the high-sensitivity visible light channel can improve the accuracy of face recognition.

[0017] In conjunction with the first aspect, in some embodiments, the electronic device receives an operation for performing face registration; the electronic device emits infrared light through an infrared flash; the electronic device captures multiple images containing a first face through a multispectral camera, the multiple images including images of the first face from multiple angles; the electronic device determines first face information and first multispectral anti-counterfeiting information of the first face based on the multiple images, the first face information is used for face matching, and the first multispectral anti-counterfeiting information is used for anti-counterfeiting detection; the electronic device stores the first face information and the first multispectral anti-counterfeiting information.

[0018] The first face information and the first multispectral anti-counterfeiting information can belong to the multispectral face model of the first face.

[0019] As can be seen, electronic devices can use multispectral cameras and infrared flashes for facial recognition. Multispectral cameras and infrared flashes are inexpensive, which helps to reduce the cost of facial recognition while ensuring accuracy.

[0020] In conjunction with the first aspect, in some embodiments, the electronic device determines the second face information and the second multispectral anti-counterfeiting information of the second face based on the first image; the electronic device performs anti-counterfeiting detection on the second face based on the second multispectral anti-counterfeiting information and the first multispectral anti-counterfeiting information, and detects whether the second face matches the first face based on the second face information and the first face information; wherein, if the first multispectral anti-counterfeiting information matches the second multispectral anti-counterfeiting information, the anti-counterfeiting detection passes; if the first face information matches the second face information, the second face matches the first face.

[0021] As can be seen, electronic devices can use multispectral cameras and infrared flashes for facial recognition. Multispectral cameras and infrared flashes are inexpensive. The methods described above can reduce the cost of facial recognition. Images captured by multispectral cameras contain the spectral features of faces, enabling anti-spoofing detection of faces, improving the accuracy of facial recognition, and better protecting users' privacy and personal property security on electronic devices.

[0022] In conjunction with the first aspect, in some embodiments, the electronic device further includes an RGB camera, which emits infrared light through an infrared flash; the electronic device captures a second image through a multispectral camera and a third image through an RGB camera, both of which contain a fourth face; the electronic device performs anti-spoofing detection on the fourth face based on the second and third images, and detects whether the fourth face matches the third face stored in the electronic device; if the anti-spoofing detection passes and the fourth face matches the third face, the electronic device determines that the face recognition is successful.

[0023] The second image may include one or more multispectral images containing the fourth face. The third image may include one or more RGB images containing the fourth face. The second and third images may be captured simultaneously by an electronic device using a multispectral camera and an RGB camera.

[0024] As can be seen, electronic devices can perform facial recognition by combining images captured by multispectral and RGB cameras. Specifically, the electronic device can determine the depth information of a face based on the images captured by the multispectral and RGB cameras. In addition to using multispectral anti-counterfeiting information for detection, electronic devices can also use depth information for anti-counterfeiting purposes. This can improve the accuracy of facial recognition and better protect the privacy and personal property of users on electronic devices.

[0025] In conjunction with the first aspect, in some embodiments, the electronic device receives an operation for face registration; the electronic device emits infrared light via an infrared flash; the electronic device captures multiple multispectral images containing a third face via a multispectral camera and multiple RGB images containing a third face via an RGB camera, both the multispectral images and the multiple RGB images including images of the third face from multiple angles; the electronic device determines third face information of the third face based on the multiple multispectral images and / or the multiple RGB images, determines first depth information based on the multiple multispectral images and the multiple RGB images, and determines third multispectral anti-counterfeiting information of the third face based on the multiple multispectral images, the third face information being used for face matching, and the first depth information and the third multispectral anti-counterfeiting information being used for anti-counterfeiting detection; the electronic device stores the third face information, the first depth information, and the third multispectral anti-counterfeiting information.

[0026] The third face information, the first depth information, and the third multispectral anti-counterfeiting information can belong to the multispectral face model of the third face.

[0027] As can be seen, electronic devices can use images captured by multispectral cameras and RGB cameras to perform face modeling, resulting in a multispectral face model. This multispectral face model incorporates multispectral anti-spoofing information and depth information, which can improve the accuracy of face recognition and better protect users' privacy and personal property security within electronic devices.

[0028] In conjunction with the first aspect, in some embodiments, the electronic device determines the fourth face information of the fourth face based on the second image and / or the third image, determines the second depth information based on the second image and the third image, and determines the fourth multispectral anti-counterfeiting information of the fourth face based on the second image; the electronic device performs anti-counterfeiting detection on the fourth face based on the first depth information, the second depth information, the third multispectral anti-counterfeiting information, and the fourth multispectral anti-counterfeiting information, and detects whether the fourth face matches the third face based on the third face information and the fourth face information; wherein, if the first depth information and the second depth information match, and the third multispectral anti-counterfeiting information and the fourth multispectral anti-counterfeiting information match, then the anti-counterfeiting detection passes; if the fourth face information matches the third face information, then the fourth face matches the third face.

[0029] As can be seen, electronic devices can perform facial recognition by combining images captured by multispectral cameras and RGB cameras. The methods described above utilize the spectral features and depth information of the face, which can improve the accuracy of facial recognition and better protect users' privacy and personal property security within electronic devices.

[0030] Secondly, this application provides a face recognition method. This method is applied to an electronic device, which includes an infrared flash, a multispectral camera, and an RGB camera. The image sensor of the multispectral camera includes a filter array, which comprises multiple filter units arranged in an array. Each filter unit includes N rows * N columns of sub-units, where N is greater than 2. The N rows * N columns of sub-units include multiple infrared filter sub-units and multiple visible light filter sub-units. The infrared filter sub-units are used to filter infrared light, and the visible light filter sub-units are used to filter visible light. The multiple infrared filter sub-units are evenly spaced along the horizontal and vertical directions, with one or more visible light filter sub-units spaced between two infrared filter sub-units. The electronic device emits infrared light through the infrared flash; the electronic device captures a second image through the multispectral camera and a third image through the RGB camera. Both the second and third images contain a fourth face; the electronic device performs anti-spoofing detection on the fourth face based on the second and third images, and checks whether the fourth face matches the third face stored in the electronic device; if the anti-spoofing detection passes and the fourth face matches the third face, the electronic device determines that face recognition is successful.

[0031] The second image may include one or more multispectral images containing the fourth face. The third image may include one or more RGB images containing the fourth face. The second and third images may be captured simultaneously by an electronic device using a multispectral camera and an RGB camera.

[0032] As can be seen, electronic devices can perform facial recognition by combining images captured by multispectral and RGB cameras. Specifically, the electronic device can determine the depth information of a face based on the images captured by the multispectral and RGB cameras. In addition to using multispectral anti-counterfeiting information for detection, electronic devices can also use depth information for anti-counterfeiting purposes. This can improve the accuracy of facial recognition and better protect the privacy and personal property of users on electronic devices.

[0033] In conjunction with the second aspect, in some embodiments, the number of visible light filter photonic units in the image sensor of the multispectral camera is greater than or equal to 5.

[0034] In conjunction with the second aspect, in some embodiments, the image sensor includes multiple channels, with one channel corresponding to one sub-unit. The visible light filter sub-unit corresponding to the infrared filter sub-unit includes a first visible light filter sub-unit and a second visible light filter sub-unit. The first visible light filter sub-unit and the infrared filter sub-unit are arranged in a row or column. The second visible light filter sub-unit is located diagonally opposite the infrared filter sub-unit. The channel corresponding to the first visible light filter sub-unit is the first channel, and the channel corresponding to the second visible light filter sub-unit is the second channel. The sensitivity of the second channel is greater than that of the first channel.

[0035] It can be seen that in the visible light filter units adjacent to the infrared filter unit, the channel corresponding to the visible light filter unit located diagonally opposite the infrared filter unit has greater sensitivity. This prevents electrons from overflowing from the pixels corresponding to the second part of the visible light filter unit to adjacent pixels, thereby preventing interference with the pixels corresponding to the infrared filter unit and thus avoiding affecting the infrared channel imaging used for anti-counterfeiting detection.

[0036] In conjunction with the second aspect, in some embodiments, the filter unit includes 4*4 sub-units. In each filter unit, the number of infrared filter sub-units is 4, and the number of visible light filter sub-units is 6, with each type of visible light filter sub-unit being repeated twice. Alternatively, in each filter unit, the number of visible light filter sub-units is 8, with 4 types of visible light filter sub-units being repeated twice. Different types of visible light filter sub-units are used to filter visible light of different spectral bands.

[0037] In conjunction with the second aspect, in some embodiments, the filtering unit includes an infrared filtering subunit for collecting infrared light greater than or equal to 820 nm and less than or equal to 870 nm, and / or an infrared filtering subunit for collecting infrared light greater than or equal to 920 nm and less than or equal to 970 nm.

[0038] In conjunction with the second aspect, in some embodiments, the electronic device further includes a screen. Before the electronic device captures a second image via a multispectral camera and a third image via an RGB camera, the electronic device detects ambient light brightness. If the ambient light brightness is lower than a first brightness, the electronic device controls the brightness of the screen to be greater than or equal to a second brightness.

[0039] It can be seen that in low-light environments, electronic devices can use screen illumination. This not only eliminates the need to add additional lighting devices to electronic devices, reducing the cost of facial recognition, but also improves the accuracy of facial recognition in low-light environments, thereby enhancing the experience of facial unlocking devices or facial payment in low-light environments.

[0040] In conjunction with the second aspect, in some embodiments, the image sensor includes multiple third channels and multiple fourth channels, one third channel corresponding to one infrared filter subunit and one fourth channel corresponding to one visible light filter subunit, the sensitivity of the fourth channel being less than the first sensitivity; if the ambient light brightness is greater than the third brightness, the electronic device generates a second image based on the signals from the multiple third channels and multiple fourth channels.

[0041] It can be seen that in overly bright scenes, the visible light channel with high sensitivity is prone to overexposure, leading to information loss. Therefore, in overly bright scenes, using the visible light channel with low sensitivity instead of the high-sensitivity visible light channel can improve the accuracy of face recognition.

[0042] In conjunction with the second aspect, in some embodiments, the electronic device receives an operation for face registration; the electronic device emits infrared light through an infrared flash; the electronic device captures multiple multispectral images containing a third face through a multispectral camera and multiple RGB images containing a third face through an RGB camera, both the multispectral images and the multiple RGB images including images of the third face from multiple angles; the electronic device determines third face information of the third face based on the multiple multispectral images and / or the multiple RGB images, determines first depth information based on the multiple multispectral images and the multiple RGB images, and determines third multispectral anti-counterfeiting information of the third face based on the multiple multispectral images, the third face information is used for face matching, and the third depth information and the third multispectral anti-counterfeiting information are used for anti-counterfeiting detection; the electronic device stores the third face information, the first depth information, and the third multispectral anti-counterfeiting information.

[0043] The third face information, the first depth information, and the third multispectral anti-counterfeiting information can belong to the multispectral face model of the third face.

[0044] As can be seen, electronic devices can use images captured by multispectral cameras and RGB cameras to perform face modeling, resulting in a multispectral face model. This multispectral face model incorporates multispectral anti-spoofing information and depth information, which can improve the accuracy of face recognition and better protect users' privacy and personal property security within electronic devices.

[0045] In conjunction with the second aspect, in some embodiments, the electronic device determines the fourth face information of the fourth face based on the second image and / or the third image, determines the second depth information based on the second image and the third image, and determines the fourth multispectral anti-counterfeiting information of the fourth face based on the second image; the electronic device performs anti-counterfeiting detection on the fourth face based on the first depth information, the second depth information, the third multispectral anti-counterfeiting information, and the fourth multispectral anti-counterfeiting information, and detects whether the fourth face matches the third face based on the third face information and the fourth face information; wherein, if the first depth information and the second depth information match, and the third multispectral anti-counterfeiting information and the fourth multispectral anti-counterfeiting information match, then the anti-counterfeiting detection passes; if the fourth face information matches the third face information, then the fourth face matches the third face.

[0046] As can be seen, electronic devices can perform facial recognition by combining images captured by multispectral cameras and RGB cameras. The methods described above utilize the spectral features and depth information of the face, which can improve the accuracy of facial recognition and better protect users' privacy and personal property security within electronic devices.

[0047] Thirdly, this application provides an electronic device. The electronic device may include an infrared flash, a multispectral camera, a memory, and a processor. The infrared flash is used to emit infrared light. The multispectral camera is used to capture multispectral images. The memory is used to store a computer program. The processor is used to invoke the computer program to execute any possible implementation method as described in the first or second aspect.

[0048] Fourthly, this application provides a computer-readable storage medium storing instructions that, when executed by a processor, can implement any possible implementation method as described in the first or second aspect.

[0049] Fifthly, this application provides a computer program product that may contain computer instructions that, when executed on a processor, can implement any possible implementation method as described in the first or second aspect.

[0050] In a sixth aspect, this application provides a chip for use in an electronic device, the chip including one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform any possible implementation method as described in the first or second aspect.

[0051] It is understood that the electronic device provided in the third aspect, the computer-readable storage medium provided in the fourth aspect, the computer program product provided in the fifth aspect, and the chip provided in the sixth aspect are all used to execute the methods provided in the embodiments of this application. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the corresponding methods, and will not be repeated here. Attached Figure Description

[0052] Figure 1 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0053] Figure 2A and Figure 2B This is a schematic diagram of the camera module structure of the multispectral camera provided in the embodiments of this application;

[0054] Figure 2C This is a schematic diagram of the image sensor provided in an embodiment of this application;

[0055] Figure 2D This is a schematic diagram of the transmittance curve of the filter material provided in the embodiments of this application;

[0056] Figure 2E This is a schematic diagram of the lens transmittance curve provided in the embodiments of this application;

[0057] Figure 2F This is a schematic diagram of the transmittance curve of the infrared filter provided in the embodiments of this application;

[0058] Figure 2G This is a schematic diagram of the basis response curve provided in the embodiments of this application;

[0059] Figure 2H This is a schematic diagram of the channel response curve provided in the embodiments of this application;

[0060] Figures 3A to 3D This is a schematic diagram of the filter array and filter unit in the image sensor provided in the embodiments of this application;

[0061] Figure 4A and Figure 4B This is a schematic diagram of some other filter units provided in the embodiments of this application;

[0062] Figure 5 This application provides a face recognition method.

[0063] Figures 6A to 6F These are schematic diagrams of some facial recognition scenarios provided in the embodiments of this application;

[0064] Figure 7 This application provides a face recognition method.

[0065] Figure 8 This is another face recognition method provided in the embodiments of this application;

[0066] Figure 9 This is another face recognition method provided in the embodiments of this application. Detailed Implementation

[0067] The technical solutions of the embodiments of this application are described below with reference to the accompanying drawings. In the description of the embodiments of this application, the terminology used in the following embodiments is for the purpose of describing specific embodiments only and is not intended to limit the application. As used in the specification and appended claims of this application, the singular expressions "a," "the," "the," "the," and "this" are intended to also include expressions such as "one or more," unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, "at least one" and "one or more" refer to one or more (including two). The term "and / or" is used to describe the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0068] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes direct connections and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0069] In the embodiments of this application, the words "exemplarily" or "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of the words "exemplarily" or "for example" is intended to present the relevant concepts in a specific manner.

[0070] This application provides a face recognition method. This method can be applied to electronic devices including a multispectral camera, an infrared flash, and a screen. The infrared flash can emit infrared light in one or more spectral bands, such as infrared light with wavelengths in the 820-870 nanometer (nm) range, or infrared light with wavelengths in the 920-970 nm range. The multispectral camera can be used to acquire light signals in multiple spectral bands to generate a multispectral image. For example, the multispectral camera can acquire light signals in multiple visible spectral bands, as well as light signals in one or more near-infrared spectral bands. That is, the multispectral camera supports multiple spectral bands. In the image sensor of the multispectral camera, the number of visible light filter sub-units used for filtering visible light is greater than or equal to five. Different types of visible light filter sub-units can be used to filter visible light in different spectral bands.

[0071] During facial recognition, the electronic device can activate its infrared flash to emit infrared light. Optionally, when natural light is too low, the electronic device can use its screen for supplemental lighting. The electronic device can capture multiple facial images of the user from different angles using a multispectral camera and build a multispectral facial model based on these images. In facial recognition scenarios such as facial unlocking devices and facial payment, the electronic device can activate its infrared flash to emit infrared light. Optionally, when natural light is too low, the electronic device can use its screen for supplemental lighting. The electronic device can capture one or more facial images using a multispectral camera and perform facial recognition based on these images and the multispectral facial model.

[0072] As demonstrated by the methods described above, electronic devices can utilize infrared flashlights and multispectral cameras for face registration and recognition. Infrared flashlights and multispectral cameras are relatively inexpensive, and these methods can reduce the cost of existing payment-grade face recognition systems (such as systems using 3D cameras based on time-of-flight (TOP) or structured light). Multispectral images captured by multispectral cameras contain the spectral characteristics of light signals absorbed and reflected by the face across multiple spectral bands. Compared to RGB (red, green, blue) images, using multispectral images for face recognition can improve accuracy. Furthermore, electronic devices can use their screens for supplemental lighting when natural light is insufficient, ensuring that the multispectral camera can capture light signals across multiple spectral bands, further enhancing the accuracy of face recognition.

[0073] Figure 1 An exemplary schematic diagram of an electronic device provided in an embodiment of this application is shown.

[0074] like Figure 1 As shown, the electronic device 100 may include a multispectral camera, an RGB camera, an infrared flash, a screen, a processor, and a memory.

[0075] An RGB camera can be used to capture RGB images. An RGB camera can also be called a three-channel camera.

[0076] An infrared flash lamp can be a light-emitting diode (LED) lamp capable of emitting infrared light. This application embodiment does not limit the materials used to manufacture the infrared flash lamp. Electronic device 100 may include one or more infrared flash lamps.

[0077] In some embodiments, an infrared flash is adapted to a multispectral camera. The infrared flash emits infrared light within the target spectral band that the multispectral camera needs to capture. For example, if the multispectral camera needs to capture infrared light in the 820-870nm range, the infrared flash emits infrared light in that range. Alternatively, if the multispectral camera needs to capture infrared light in the 920-970nm range, the infrared flash emits infrared light in that range. Or, if the multispectral camera needs to simultaneously capture infrared light in both the 820-870nm and 920-970nm ranges, the electronic device 100 may include multiple infrared flashes, which simultaneously emit infrared light in both ranges so that the infrared photosensitive unit in the image sensor can sense light simultaneously. A spectral band can represent a wavelength range. A spectral band can also be called a wavelength band.

[0078] In some embodiments, both the multispectral camera and the RGB camera can be front-facing cameras of the electronic device 100. An infrared flash can be spaced apart from the multispectral camera to provide infrared light for the multispectral camera to capture images.

[0079] The screen can be used to display images, videos, etc. It can be a flexible or rigid display. For example, it can be an organic light-emitting diode (OLED) display, an active-matrix organic light-emitting diode (AMOLED) display, a mini organic light-emitting diode (MLED) display, a micro organic light-emitting diode (MOLED) display, a quantum dot light-emitting diode (QLED) display, or a liquid crystal display (LCD), etc.

[0080] The memory may store a computer program for enabling a processor to implement the face recognition method of this application through an interface or protocol. For example, the computer program stored in the memory may be used for: detecting natural light brightness and using screen illumination when the natural light brightness is below a brightness threshold; detecting whether a face image captured by a multispectral camera and / or an RGB camera contains a face; modeling and generating a multispectral face model based on the face image captured by the multispectral camera; performing face recognition using the multispectral face model; calculating depth information based on the face image captured by the multispectral camera and the face image captured by the RGB camera; and modeling and generating a multispectral face model based on the depth and the face image captured by the multispectral camera and the RGB camera, etc.

[0081] The processor can communicate with multispectral and RGB cameras. It can acquire and process image data from these cameras. The communication connection between the processor and the multispectral and RGB cameras can include data transmission via electrical connections such as wiring, or through coupling. It is understood that the camera module and the multispectral and RGB cameras can also communicate using other methods capable of data transmission.

[0082] In some embodiments, after the processor processes the image data, it can transfer the image to the memory so that the image can be retrieved from the memory and displayed on the screen whenever it is needed to view the image later.

[0083] It is understandable that the above Figure 1 The structure shown is merely an illustrative example of this application; the electronic device 100 may also include components that are more advanced than those described above. Figure 1 The diagram shows more or fewer structures. Electronic device 100 can be a mobile phone, tablet computer, laptop computer, smart screen, personal digital assistant (PDA), camera, personal computer, laptop computer, in-vehicle equipment, wearable device, augmented reality (AR) glasses, AR headset, virtual reality (VR) glasses, or VR headset, or any other device with camera functionality. This application does not limit the type of electronic device 100.

[0084] Figure 2A and Figure 2B An exemplary schematic diagram of the camera module structure of the multispectral camera provided in an embodiment of this application is shown.

[0085] like Figure 2AAs shown, the camera module structure of a multispectral camera may include: a lens, a lens bracket, an infrared filter, a multispectral image sensor, a circuit board, a board-to-board (BTB) circuit, and a flexible printed circuit board (FPC).

[0086] The lens is mounted on a lens holder. The head bracket is used to fix the lens. The lens can be used for focused imaging.

[0087] Infrared filters can be used to filter infrared light in the target spectrum.

[0088] In some embodiments, the infrared filter can filter out mid-infrared light (wavelength range of 3000-6000 nm) and far-infrared light (wavelength range of 6000-15000 nm), while retaining near-infrared light (wavelength range of 750-3000 nm). Alternatively, the infrared filter can also retain some bands of near-infrared light, for example, retaining near-infrared light with a wavelength range of 750-1000 nm, while filtering out infrared light in other bands. The embodiments of this application do not limit the infrared light filtered by the infrared filter.

[0089] Multispectral image sensors can be used to sense light information, such as visible light and near-infrared light.

[0090] The circuit board is the peripheral circuit for the operation of the multispectral image sensor.

[0091] FPC is a flexible wire circuit that connects the BTB to the circuit board. BTB is the port for connecting the camera module to electronic equipment.

[0092] It can be seen that, Figure 2A The camera module shown includes an infrared filter, which can filter out infrared light in the target spectrum and better restore image colors. Figure 2A The camera module shown can also be called a module structure with infrared cutoff.

[0093] In some embodiments, the multispectral camera does not include an infrared filter. A camera module without an infrared filter can also be referred to as a module structure without infrared cutoff.

[0094] like Figure 2B As shown, the camera module structure of a multispectral camera may include a lens, lens bracket, multispectral image sensor, circuit board, board-to-board (BTB), or flexible printed circuit board (FPC). For details, please refer to [reference needed]. Figure 2A The description shown.

[0095] Figure 2CA schematic diagram of an image sensor provided in an embodiment of this application is shown as an example.

[0096] Figure 2C The image sensor shown can be Figure 2A and Figure 2B The multispectral image sensor in the multispectral camera shown.

[0097] like Figure 2C As shown, an image sensor may include a circuit layer, a pixel layer, a filter array, and a microlens array stacked sequentially. The microlens array is used to focus light. The filter array is used to filter light, and may include various filter materials, each with a different transmittance. For example, the filter array may include at least four filter materials. The pixel layer receives light and converts the optical signal into an electrical signal. The circuit layer processes and transmits the electrical signal. In other words, visible light, infrared light, and other optical signals can pass sequentially through the microlens array and the filter array to the pixel layer, where they are converted into electrical signals, and the circuit layer processes these electrical signals.

[0098] For example, a microlens array may include multiple microlenses, a filter array may include multiple sub-units, and a pixel layer may include multiple pixels. An image sensor may have multiple channels, with one channel corresponding to one microlens, one sub-unit, and one pixel, and different channels corresponding to different microlenses, sub-units, and pixels.

[0099] In some embodiments, the microlens array can remove the microlens corresponding to a certain channel to avoid overexposure.

[0100] In some embodiments, the filter array can achieve different transmittances through, but not limited to, single-layer dyes, multi-layer dye stacking, coatings, gratings, microstructures, etc., to filter multiple spectral bands of light signals.

[0101] It should be noted that, Figure 2C The dashed box in the middle represents the channel of the image sensor. Different types of sub-units in the filter array are represented by different symbols; for example, T1, T2, T3, and T4 can represent different types of sub-units. Different types of sub-units can use different filter materials. Understandably, Figure 2C The symbol corresponding to the neutron unit is not limited in type and location; in other embodiments, there are other design types and locations.

[0102] In some embodiments, the images generated by the image sensor, after spectral sampling, can achieve an image resolution of 320P or higher. This can improve the accuracy of face recognition.

[0103] Figure 2DAn exemplary schematic diagram of the transmittance curve of a light-filtering material is shown. For example... Figure 2D As shown, filter material 1 and filter material 2 may have different transmittances for different wavelengths of light. Both filter material 1 and filter material 2 can transmit a portion of the visible spectrum and a portion of the near-infrared spectrum. The above transmittance curves of the filter materials are merely illustrative examples of this application and should not be construed as limiting the scope of this application.

[0104] From the above Figure 2A and Figure 2B As shown in the camera module structure, light first passes through the lens and undergoes attenuation. The degree of attenuation varies for different spectral bands, meaning the lens can have different transmittance for different wavelengths of light.

[0105] Figure 2E An example is shown as a schematic diagram of the lens transmittance curve. For example... Figure 2E As shown, light with a spectral range in the range of λ1-λ2 can pass through the lens. Light in the λ1-λ2 range can include visible light and near-infrared light. This application does not limit the values ​​of λ1 and λ2. The lens transmittance can vary with wavelength. The above lens transmittance curves are merely illustrative examples and should not be construed as limiting the scope of this application.

[0106] In a camera module that includes an infrared filter, light passes through the infrared filter after passing through the lens. Light in the target spectrum (such as mid-infrared light and far-infrared light) is filtered out.

[0107] Figure 2F A schematic diagram illustrating the transmittance curve of an infrared filter is provided. Figure 2F As shown, the infrared filter can retain light with spectral ranges in the ranges of λ3-λ4 and λ5-λ6, while filtering out light outside these ranges. This application does not limit the values ​​of λ3, λ4, λ5, and λ6. For example, light in the range of λ3-λ4 may include visible light. Light in the range of λ5-λ6 may include near-infrared light. The transmittance of the infrared filter can vary with wavelength. The above infrared filter transmittance curve is merely an illustrative example of this application and should not be construed as limiting the scope of this application.

[0108] Furthermore, the light passes through the filter array of the multispectral image sensor. Different filter materials in the filter array have corresponding transmittances of light, as detailed above. Figure 2D This section introduces the concept of optical signal conversion. Therefore, the optical signals received by pixel layers under different filter materials vary. The pixel layer can convert optical signals into electrical signals. The conversion efficiency of optical signals to electrical signals in different spectral bands can be characterized by the substrate response curve.

[0109] Figure 2GAn exemplary schematic diagram of the substrate response curve is shown. Figure 2G As shown, the conversion efficiency of the pixel layer in converting optical signals into electrical signals can vary with wavelength. The above substrate response curves are merely illustrative examples of this application and should not be construed as limiting the scope of this application.

[0110] Based on the aforementioned light propagation path, in a camera module containing an infrared filter, the response curve of each channel can be: Channel Response Curve (QE) = Lens Transmittance Curve * Infrared Filter Transmittance Curve * Transmittance Curve of the Filter Material Corresponding to the Channel * Substrate Response Curve. It can be understood that the response curve of a channel in a multispectral image sensor represents the efficiency with which a light signal is converted into an electrical signal from the lens to the corresponding pixel in the pixel layer, i.e., the light conversion efficiency of the camera module.

[0111] Based on the above light propagation path, in a camera module without an infrared filter, the response curve of each channel can be: Channel Response Curve (QE) = Lens Transmittance Curve * Transmittance Curve of the Filter Material Corresponding to the Channel * Substrate Response Curve.

[0112] Figure 2H An example diagram of the channel response curve is shown. (e.g.) Figure 2H As shown, the response curves of channel 1 and channel 2 are different. The peak value of the response curve of channel 1 corresponds to a wavelength of λ7, while the peak value of the response curve of channel 2 corresponds to a wavelength of λ8. This application embodiment does not limit the values ​​of λ7 and λ8. The above channel response curves are merely illustrative examples of this application and should not be construed as limiting the scope of this application.

[0113] Figures 3A to 3D A schematic diagram of the filter array and filter unit in the image sensor provided in the embodiments of this application is shown as an example.

[0114] like Figure 3A As shown, the filter array may include multiple filter units arranged in an array, and a filter unit may include N rows * N columns of sub-units arranged in an array, where N is greater than or equal to 2. Figure 3A The diagram uses N=4 as an example. A filter unit can also be called a minimum repeating unit.

[0115] It should be noted that all the filter units in the array are identical. Figure 3A The area outlined by the medium-thick solid line represents a filter unit. Figure 3A Different types of sub-units are represented by different symbols. Figure 3A This illustration only shows one arrangement of different types of sub-units. In other embodiments, the filter array may include more or fewer types of sub-units, and the different types of sub-units may be arranged in other ways.

[0116] A filter unit may include one or more infrared filter subunits (e.g., Tr) and multiple visible light filter subunits (e.g., T1, T2, T3, T4, T5, T6). The infrared filter subunits allow infrared light of the target spectral band to pass through. The visible light filter subunits allow visible light of the target spectral band to pass through.

[0117] In some embodiments, the multiple visible light filter sub-units in a filter unit can employ different filter materials. That is, a filter unit may include multiple types of visible light filter sub-units. For example, Figure 3A The filter materials corresponding to T1, T2, T3, T4, T5, and T6 shown can be different. Different types of visible light filter subunits can be used to filter visible light in different spectral bands.

[0118] In the image sensor of a multispectral camera, a filter unit includes five or more types of visible light filter sub-units.

[0119] In some embodiments, a filter unit may include multiple filter areas.

[0120] like Figure 3B As shown, a 4*4 filter unit may include filter area 311, filter area 312, filter area 313, and filter area 314. Each of filter areas 311 to 314 may include 2*2 sub-units. In some embodiments, the types and order of sub-units in filter area 311 are the same as those in filter area 314. The types and order of sub-units in filter area 312 are the same as those in filter area 313. Filter areas 311 and 314 may be referred to as first filter areas, and filter areas 312 and 313 may be referred to as second filter areas. That is, two first filter areas are arranged along one diagonal of the filter unit, and two second filter areas are arranged along the other diagonal of the filter unit. Because the filter units are evenly and regularly distributed, it is beneficial to achieve low-resolution output images and low-power output images. The image distribution is also uniform, and it is also beneficial to perform de-mosaic processing on the image in subsequent image processing.

[0121] Each filtering region within a single filtering unit may contain at least one infrared filtering subunit. For example, Figure 3B Each of the filter regions 311 to 314 shown can contain an infrared filter unit.

[0122] In some embodiments, in a filtering unit, multiple infrared filtering subunits are arranged at equal intervals along both the lateral and longitudinal directions, and one or more visible light filtering subunits may be spaced apart between two adjacent infrared filtering subunits. Figure 3BAs shown, in a 4*4 filter unit, adjacent infrared filter subunits are separated by one visible light filter subunit in the vertical direction and by one visible light filter subunit in the horizontal direction. The multiple infrared filter subunits are evenly spaced in both the horizontal and vertical directions, achieving a uniform distribution and helping to avoid imaging distortion.

[0123] As can be seen from the foregoing embodiments, the image sensor has multiple channels, and each channel corresponds one-to-one with a multiple sub-unit. For example... Figure 3C As shown, in the filter unit, the visible light filter unit adjacent to the infrared filter unit may include a first visible light filter unit (e.g., T1) and a second visible light filter unit (e.g., T2, T5). The first visible light filter unit is located diagonally opposite the infrared filter unit. The second visible light filter unit is arranged in a row or column with the infrared filter unit. The channel corresponding to the first visible light filter unit can be called the first channel. The channel corresponding to the second visible light filter unit can be called the second channel. The sensitivity of the second channel is greater than that of the first channel.

[0124] Sensitivity, also known as light conversion efficiency, represents the efficiency of light conversion. The sensitivity of a channel can be represented by the area of ​​its response curve integrated. Under the same energy of white light, a channel with higher sensitivity can capture more photons, resulting in a larger signal value. Since devices have upper and lower limits for their signal values, when the signal value exceeds the upper limit, the device can only output the upper limit, leading to signal distortion and unusable signals. Therefore, channels with high sensitivity are not suitable for color reproduction in bright areas. Channels with high sensitivity are suitable for dark areas because their large signal value makes them less susceptible to noise, giving them an advantage over other channels. Similarly, channels with low sensitivity capture less energy, resulting in a lower signal value, and are easily overwhelmed by noise in dark areas. Therefore, channels with low sensitivity are suitable for bright areas because their low signal value makes them less likely to reach their upper limit in bright areas, allowing for better color reproduction in bright areas.

[0125] It should be noted that the channel corresponding to the visible light filter unit can be called the visible light channel, and the channel corresponding to the infrared filter unit can be called the infrared channel.

[0126] In some embodiments, in the visible light filter subunits adjacent to the infrared filter subunits, the channel corresponding to the visible light filter subunit located diagonally opposite the infrared filter subunit has greater sensitivity.

[0127] It can be seen that the first part of the visible light filter photonic unit and the second part of the visible light filter photonic unit are arranged according to... Figure 3CThe arrangement shown can prevent electrons from overflowing from the pixels corresponding to the second visible light filter unit to adjacent pixels, thereby preventing interference with the pixels corresponding to the infrared filter unit and thus avoiding affecting the infrared channel imaging used for anti-counterfeiting detection.

[0128] In some embodiments, when N is an even number, a filter unit may include four filter regions. A filter region includes N / 2 rows * N / 2 columns of sub-units. The pixel layer in the image sensor may include multiple pixel regions. Multiple pixel regions are configured in a one-to-one correspondence with multiple filter regions. Among the four pixel regions corresponding to the four filter regions, they are sequentially divided into a first pixel region, a second pixel region, a third pixel region, and a fourth pixel region according to the reading order of the circuit layer. The channel corresponding to the first pixel region has a sensitivity of 1, and the channels corresponding to the other pixel regions in the pixel layer have a sensitivity of 2, where sensitivity 1 is greater than or equal to sensitivity 2. For example, as... Figure 3B As shown, the pixel area corresponding to filter area 311 is the first pixel area. The pixel area corresponding to filter area 312 is the second pixel area. The pixel area corresponding to filter area 313 is the third pixel area. The pixel area corresponding to filter area 314 is the fourth pixel area. The sensitivity of the channel corresponding to filter area 311 is greater than the sensitivity of the channels corresponding to filter areas 312 to 314. It is understandable that placing the channel with higher sensitivity at the beginning of the circuit layer readout order facilitates the traversal of the processing path of the main control chip in the electronic device, thereby improving the processing efficiency of the main control chip.

[0129] The following uses a 4*4 filter unit as an example to introduce the filter material arrangement and transmittance of the filter material provided in the embodiments of this application.

[0130] In some embodiments, such as Figure 3B and Figure 3C As shown, a filter unit can include six visible light filter materials and one infrared filter material. The filter unit contains 12 sub-units for visible light filtering, meaning each of the six visible light filter materials is reused twice; and four sub-units for infrared filtering, all using the same infrared filter material. This ensures a uniform distribution of different types of visible light filter sub-units within the filter unit, improving imaging performance.

[0131] For the above six visible light photosensitive units and one infrared photosensitive unit, their transmittance curves exhibit the following characteristics:

[0132] The visible light filter unit 1 (i.e., T1) satisfies the following: the peak wavelength of visible light transmittance is ≥510nm; the transmittance of light in the wavelength range of 400-470nm is ≤20%; and the transmittance of light in the wavelength range of 520-700nm is ≥90%.

[0133] The visible light filter unit 2 (i.e., T2) satisfies the following: the peak wavelength of visible light transmittance corresponds to the wavelength between 440nm and 510nm; transmittance ≥80% for light in the wavelength range of 415-525nm; transmittance ≤40% for light in the wavelength range of 560-570nm; transmittance ≤20% for light in the wavelength range of 570-730nm; and transmittance ≥95% for light in the wavelength range of 795-1000nm.

[0134] The visible light filter unit 3 (i.e., T3) satisfies the following: the peak wavelength of visible light transmittance is ≥600nm; the transmittance of light in the wavelength range of 400-570nm is ≤10%; and the transmittance of light in the wavelength range of 610-1000nm is ≥95%.

[0135] The visible light filter unit 4 (i.e., T4) satisfies the following: the peak wavelength of visible light transmittance corresponds to the wavelength between 440nm and 460nm; transmittance ≥60% for light in the wavelength range of 400-480nm; transmittance ≤40% for light in the wavelength range of 495-510nm; transmittance ≤10% for light in the wavelength range of 525-700nm; and transmittance ≥90% for light in the wavelength range of 830-1000nm.

[0136] The visible light filter unit 5 (i.e., T5) satisfies the following: the peak wavelength of visible light transmittance corresponds to the wavelength between 525nm and 560nm; transmittance ≤10% for light in the wavelength range of 400-465nm; transmittance ≥80% for light in the wavelength range of 535-560nm; transmittance ≤40% for light in the wavelength range of 650-750nm; and transmittance ≥95% for light in the wavelength range of 820-1000nm.

[0137] The visible light filter unit 6 (i.e., T6) satisfies the following: the peak wavelength of visible light transmittance corresponds to the wavelength between 515nm and 550nm; transmittance ≤10% for light in the wavelength range of 400-470nm; transmittance ≥80% for light in the wavelength range of 500-575nm; transmittance ≤20% for light in the wavelength range of 660-695nm; and transmittance ≥95% for light in the wavelength range of 880-1000nm.

[0138] The infrared filter subunit (Tr) satisfies the following: the peak infrared light transmittance corresponds to a wavelength ≥ 800 nm; the transmittance for light in the wavelength range of 400-765 nm is ≤ 10%; and the transmittance for light in the wavelength range of 850-1000 nm is ≥ 95%.

[0139] In some embodiments, the visible light filter subunit 1 is disposed adjacent to the infrared filter subunit, and located diagonally opposite the infrared filter subunit. The channel corresponding to the visible light filter subunit 1 has the highest sensitivity. Therefore, placing the visible light filter subunit 1 diagonally opposite the infrared filter subunit prevents electrons from overflowing from the pixels corresponding to the visible light filter subunit 1 to the pixels corresponding to the infrared filter subunit, thereby avoiding the influence of the infrared channel and improving the accuracy of face recognition.

[0140] In some embodiments, when the circuit layer reads the electrical signals of all channels, the order in which the circuit layer reads the electrical signals of the multiple channels corresponding to visible light filter units 1, 2, and 5 can precede the order in which the circuit layer reads the electrical signals of the multiple channels corresponding to visible light filter units 3, 4, and 6. Since the channels corresponding to visible light filter units 1, 2, and 5 have high sensitivity, placing them at the upper left corner (2x2) of the filter unit facilitates the traversal of the processing paths of the main control chip in the electronic device.

[0141] In some embodiments, such as Figure 3D As shown, a filter unit may include eight visible light filter materials and one infrared filter material. The filter unit contains 12 sub-units that are visible light filter sub-units. Four types of visible light filter sub-units (e.g., T1, T2, T3, T5) are repeated twice, while the other four types (e.g., T4, T6, T7, T8) are only used once. That is, of the eight visible light filter materials, four are used twice in one filter unit, and the other four are used once in one filter unit. The filter unit also contains four infrared filter sub-units, all of which use the same infrared filter material.

[0142] For the above 8 visible light photosensitive units and 1 infrared photosensitive unit, their transmittance curves exhibit the following characteristics:

[0143] The visible light filter unit 1 (i.e., T1) satisfies the following: the peak wavelength of visible light transmittance is ≥510nm; the transmittance of light in the wavelength range of 400-470nm is ≤20%; and the transmittance of light in the wavelength range of 520-700nm is ≥90%.

[0144] The visible light filter unit 2 (i.e., T2) satisfies the following: the peak wavelength of visible light transmittance corresponds to the wavelength between 440nm and 510nm; transmittance ≥80% for light in the wavelength range of 415-525nm; transmittance ≤40% for light in the wavelength range of 560-570nm; transmittance ≤20% for light in the wavelength range of 570-730nm; and transmittance ≥95% for light in the wavelength range of 795-1000nm.

[0145] The visible light filter unit 3 (i.e., T3) satisfies the following: the peak wavelength of visible light transmittance is ≥620nm; the transmittance of light in the wavelength range of 400-460nm is ≥60%; the transmittance of light in the wavelength range of 535-570nm is ≤10%; and the transmittance of light in the wavelength range of 635-1000nm is ≥95%.

[0146] The visible light filter unit 4 (i.e., T4) satisfies the following: the peak wavelength of visible light transmittance corresponds to the wavelength between 440nm and 460nm; transmittance ≥60% for light in the wavelength range of 400-480nm; transmittance ≤40% for light in the wavelength range of 495-510nm; transmittance ≤10% for light in the wavelength range of 525-700nm; and transmittance ≥90% for light in the wavelength range of 830-1000nm.

[0147] The visible light filter unit 5 (i.e., T5) satisfies the following: the peak wavelength of visible light transmittance corresponds to the wavelength between 525nm and 560nm; transmittance ≤10% for light in the wavelength range of 400-465nm; transmittance ≥80% for light in the wavelength range of 535-560nm; transmittance ≤40% for light in the wavelength range of 650-750nm; and transmittance ≥95% for light in the wavelength range of 820-1000nm.

[0148] The visible light filter unit 6 (i.e., T6) satisfies the following: the peak wavelength of visible light transmittance is ≥600nm; the transmittance of light in the wavelength range of 400-570nm is ≤10%; and the transmittance of light in the wavelength range of 610-1000nm is ≥95%.

[0149] The visible light filter unit 7 (i.e., T7) satisfies the following: the peak wavelength of visible light transmittance corresponds to the wavelength between 440nm and 465nm; the transmittance for light in the wavelength range of 400-485nm is ≥60%; the transmittance for light in the wavelength range of 540-775nm is ≤10%; and the transmittance for light in the wavelength range of 850-1000nm is ≥95%.

[0150] The visible light filter unit 8 (i.e., T8) satisfies the following: the peak wavelength of visible light transmittance corresponds to the wavelength between 515nm and 550nm; transmittance ≤10% for light in the wavelength range of 400-470nm; transmittance ≥80% for light in the wavelength range of 500-575nm; transmittance ≤20% for light in the wavelength range of 660-695nm; and transmittance ≥95% for light in the wavelength range of 880-1000nm.

[0151] The infrared filter subunit (Tr) satisfies the following: the peak infrared light transmittance corresponds to a wavelength ≥ 800 nm; the transmittance for light in the wavelength range of 400-765 nm is ≤ 10%; and the transmittance for light in the wavelength range of 850-1000 nm is ≥ 95%.

[0152] In some embodiments, the sensitivity of the corresponding channels of the visible light filter units 1, 2, 3, and 5 can be greater than the sensitivity of the corresponding channels of the visible light filter units 4, 6, 7, and 8. Repeating the more sensitive filter units 1, 2, 3, and 5 twice helps the camera module overcome the problem of low light intake and improves the accuracy of face recognition.

[0153] Figure 4A and Figure 4B Schematic diagrams of other filter units provided in embodiments of this application are shown as examples.

[0154] like Figure 4A As shown, a filter unit can include 3*3 sub-units. Specifically, a filter unit can include 8 visible light filter sub-units (e.g., T1-T8) and 1 infrared filter sub-unit (e.g., Tr). The 8 visible light filter sub-units can use various different visible light filtering materials. The infrared filter sub-unit can be located in the center of the filter unit, i.e., in the 2nd row and 2nd column.

[0155] like Figure 4B As shown, a filter unit may include 3*3 sub-units. Specifically, a filter unit may include 7 visible light filter sub-units (e.g., T1-T7) and 2 infrared filter sub-units (e.g., Tr). The 7 visible light filter sub-units may use various different visible light filter materials. The 2 infrared filter sub-units may use the same or different infrared filter materials. The 2 infrared filter sub-units may be located in the 1st row and 3rd column, and the 3rd row and 1st column, respectively, of the filter unit. This application embodiment does not limit the transmittance of the filter materials in the above-mentioned filter unit.

[0156] The above-described filter units are merely illustrative examples and should not be construed as limiting the scope of this application. The filter array of an image sensor may include filter units of other sizes or with other arrangements.

[0157] In some embodiments, the image sensor of a multispectral camera can detect light signals in six or more spectral bands, where at least one band is a near-infrared band and the others are visible light bands. For example, the near-infrared band may include 820-870 nm and / or 920-970 nm. That is, in the filter array of the image sensor, a filter unit may include at least six types of sub-units. These six types of sub-units may include at least one type of infrared filter sub-unit, and the others are visible light filter sub-units. By configuring the image sensor to detect light signals in six or more spectral bands, the multispectral camera can acquire more facial spectral features, thereby improving the accuracy of face recognition.

[0158] Figure 5 An exemplary embodiment of this application provides a face recognition method.

[0159] S511, An operation for face registration was detected.

[0160] In some embodiments, the electronic device can detect the act of registering a face for unlocking the electronic device, or the act of registering a face for facial payment, etc. This application embodiment does not limit the operation used for face registration.

[0161] S512, emits infrared light through an infrared flash lamp.

[0162] Depending on the operation used for facial recognition, the electronic device 100 can activate its infrared flash to emit infrared light in the target spectral band. This target spectral band of infrared light can be a spectral band supported by the multispectral camera of the electronic device 100.

[0163] In some embodiments, the electronic device 100 can detect the intensity of infrared light signals in natural light. If the intensity of the infrared light signal in the target spectral band of natural light is lower than a preset infrared light signal intensity threshold, the electronic device 100 can activate an infrared flash. If the intensity of the infrared light signal in the target spectral band of natural light is not lower than the preset infrared light signal intensity threshold, the electronic device 100 does not need to activate the infrared flash. This reduces the power consumption of the electronic device 100. This application embodiment does not limit the value of the aforementioned infrared light signal intensity threshold.

[0164] S513. Use screen fill light when the natural light brightness is lower than the brightness threshold.

[0165] In some embodiments, the electronic device 100 can detect natural light brightness. Natural light can also be referred to as ambient light. For example, the electronic device 100 may include an ambient light sensor. The electronic device 100 can detect natural light brightness through the ambient light sensor. Alternatively, the electronic device 100 can also detect natural light brightness through devices such as a multispectral camera. The embodiments of this application do not limit the method for detecting natural light brightness.

[0166] The electronic device 100 can determine whether the natural light brightness is lower than a preset brightness threshold. This embodiment does not limit the value of this brightness threshold. If the natural light brightness is lower than the brightness threshold, the electronic device 100 can maintain a screen brightness greater than or equal to brightness 1. This embodiment does not limit the value of brightness 1. That is, the electronic device 100 can use screen supplementary lighting when natural light is insufficient. Alternatively, if the natural light brightness is lower than the brightness threshold, the electronic device 100 can adjust the screen brightness according to a preset screen brightness adjustment strategy to achieve the purpose of using screen supplementary lighting. This embodiment does not limit the method of adjusting screen brightness when using screen supplementary lighting.

[0167] Understandably, in situations where natural light is insufficient, electronic device 100 can supplement light through its screen. This not only eliminates the need for additional lighting devices in electronic device 100, reducing its cost, but also ensures that the camera of electronic device 100 can capture clear faces even in low-light conditions, improving the accuracy of face recognition.

[0168] S514, prompts the user to turn their face.

[0169] In some embodiments, during the face registration process, the electronic device 100 may prompt the user to turn their face so that multiple facial images of the user from different angles can be captured. Registering the face based on multiple facial images from different angles can improve the success rate and accuracy of subsequent face recognition.

[0170] In addition, during the face registration process, the electronic device 100 can also activate the RGB camera and display the captured RGB image on the screen. This allows users to easily preview the face captured by the camera.

[0171] S515 captures multiple facial images of the user from different angles using a multispectral camera.

[0172] During the face registration process, the multispectral camera can continuously collect data and capture multiple facial images of the user from different angles. The multispectral camera can be referenced in the description of the foregoing embodiments.

[0173] In some embodiments, the electronic device 100 can detect a face region in an image captured by a multispectral camera. If the size of the face region is smaller than a preset size, the electronic device 100 can prompt the user to bring their face closer to the electronic device 100. This application embodiment does not limit the value of the preset size. For example, the preset size can be 100 pixels * 100 pixels. This can increase the face region in the image captured by the multispectral camera, more clearly capturing the spectral features of the face, thereby improving the accuracy of face registration.

[0174] S516. Given complete facial images, establish a multispectral facial model based on multiple facial images.

[0175] In some embodiments, the electronic device 100 can perform completeness detection on images captured by a multispectral camera in real time. This completeness detection can be used to determine whether the multispectral camera has captured the image required for face modeling, such as an image containing the user's frontal face, an image containing the user's left side of the face, an image containing the user's right side of the face, etc. This application does not limit the method of completeness detection.

[0176] Once it is determined that the facial images captured by the multispectral camera are complete, the electronic device 100 can establish a multispectral face model 1 based on the multiple facial images captured by the multispectral camera. For example, the face used for facial registration on the electronic device 100 is user 1's face 1. The multispectral face model 1 can be the result obtained by the electronic device 100 in modeling face 1. The multispectral face model 1 can include facial information of face 1 and multispectral anti-spoofing information.

[0177] The facial information of face 1 may include facial features such as the contour features and facial features of face 1, which can indicate the appearance of user 1's face. The facial information of face 1 can be used for face matching so that the electronic device 100 can determine whether the captured face is face 1. In some embodiments, the electronic device 100 can perform spectral sampling on the face image captured by the multispectral camera to obtain an image corresponding to the infrared spectral band. The electronic device 100 can extract the facial information of face 1 from the image corresponding to the infrared spectral band.

[0178] The multispectral anti-spoofing information of face 1 may include the spectral features of face 1, which can indicate the absorption rate of face 1 to light in different spectral bands. The multispectral anti-spoofing information of face 1 can be used for anti-spoofing detection so that electronic device 100 can determine whether the captured face is a forged face (e.g., a face forged using an image / video / mask). In some embodiments, the multispectral anti-spoofing information of face 1 may include both the infrared spectral features of face 1 and the visible light spectral features of face 1. Anti-spoofing detection can be used to detect whether the face image is an image obtained by a multispectral camera capturing a real person, thereby reducing the use of other people's images or videos, or forged face methods such as face masks, to pass face recognition, and improving the security of face recognition. The process of establishing the multispectral face model 1 can be a process of extracting face information and multispectral anti-spoofing information from multiple face images taken from multiple angles by a multispectral camera. This application embodiment does not limit the above-described face modeling method.

[0179] Understandably, face images captured by multispectral cameras include the spectral features of the face. Real faces and fake faces (such as faces in images, videos, or face masks) absorb and reflect different amounts of light. That is, real faces and fake faces have different spectral characteristics. Therefore, in multispectral face model 1 obtained by modeling faces based on face images captured by multispectral cameras, multispectral anti-spoofing information can be used for anti-spoofing detection of faces.

[0180] In some embodiments, steps S512 and S513 can be performed simultaneously so that the multispectral camera can simultaneously acquire visible light and infrared light signals. If the multispectral camera needs to acquire infrared light of multiple spectral bands, the electronic device 100 can control the infrared flash to simultaneously emit infrared light of multiple spectral bands. This can improve the efficiency of face recognition.

[0181] Alternatively, steps S512 and S513 may not be performed simultaneously. For example, in low natural light conditions, in response to the operation for face registration, the electronic device 100 may first execute step S512, whereby the multispectral camera can acquire infrared light signals and generate an image containing infrared light features. Further, the electronic device 100 may execute step S513, whereby the multispectral camera can acquire visible light signals and generate an image containing visible light features. The electronic device 100 can then perform face modeling based on the image containing infrared light features and the image containing visible light features. Optionally, if the multispectral camera needs to acquire infrared light of multiple spectral bands, the electronic device 100 can control the infrared flash to emit infrared light of multiple spectral bands in turn. While the infrared flash emits infrared light of multiple spectral bands in turn, the multispectral camera can acquire infrared light of different spectral bands at different time periods, generating images containing infrared light features of different spectral bands.

[0182] S517. Based on the modeling results of multispectral face model 1, prompt the user whether the face registration was successful or failed.

[0183] In some embodiments, the electronic device 100 can detect whether the multispectral face model 1 has been successfully modeled.

[0184] If the multispectral face model 1 is successfully created, the electronic device 100 can notify the user that the face registration is successful and save the multispectral face model 1. After successful face registration, the electronic device 100 can use the multispectral face model 1 to unlock the electronic device 100 and / or make face payments.

[0185] If the multispectral face model 1 fails to model, the electronic device 100 can prompt the user that the face registration has failed.

[0186] As can be seen from the above method, the electronic device 100 can use a multispectral camera and an infrared flash for face recognition, and use the screen for illumination when natural light is insufficient. Multispectral cameras and infrared flashes are low-cost, which helps to reduce the cost of face recognition while ensuring accuracy.

[0187] Figures 6A to 6F The illustration shows some scene diagrams of face registration provided in the embodiments of this application.

[0188] like Figure 6A As shown, the electronic device 100 can display a user interface 610. The user interface 610 may include a face registration control 611. The face registration control 611 can be used to trigger the electronic device 100 to begin face registration. In response to an operation on the face registration control 611, the electronic device 100 can display... Figure 6B The user interface 620 shown is shown.

[0189] In some embodiments, in response to an operation of the face registration control 611, the electronic device 100 may also activate an infrared flash to emit infrared light and adjust the screen brightness according to the natural light intensity, as described above. Figure 5 The steps S512 and S513 are shown. The electronic device 100 can also activate the multispectral camera to acquire image data.

[0190] like Figure 6B As shown, the user interface 620 may include a face registration prompt 621, an image preview box 622, and a face rotation prompt 623.

[0191] The face registration prompt 621 can be used to guide the face registration process. For example, the content of the face registration prompt 621 may include "Start face registration, please slowly turn your face as prompted".

[0192] The image preview frame 622 can be used to display images captured by the electronic device 100. The electronic device 100 can activate an RGB camera and display the image captured by the RGB camera in the image preview frame 622. The image preview frame 622 allows users to preview their own facial images and adjust their face to a suitable position relative to the camera of the electronic device 100. For example, if the user's face is positioned slightly to the left in the image preview frame 622, the user can move their face to the right or move the electronic device 100 to the left to center their face in the image preview frame 622. As another example, if the user's face appears too small in the image preview frame 622, the user can move their face closer to the electronic device 100 to allow their face to occupy more of the image preview frame 622.

[0193] The face rotation prompt 623 can be used to prompt the user to rotate their face. For example, the content of the face rotation prompt 623 may be "Please turn to the left," which can be used to instruct the user to turn their face to the left so that the electronic device 100 can capture face images from multiple angles. The electronic device 100 is not limited to prompting the user to rotate their face through the face rotation prompt 623; it can also prompt the user to rotate their face through voice broadcasting or other methods.

[0194] like Figure 6C As shown, the electronic device 100 can display the image captured by the RGB camera in real time in the image preview box 622. When the user turns their face to the left according to the face rotation prompt 623, the electronic device 100 can... Figure 6C The image preview box 622 shows an image of the user's left face captured by an RGB camera, along with a face rotation prompt 624. The face rotation prompt 624 can be found in [reference needed]. Figure 6B The face rotation prompt 623 is shown. The content of face rotation prompt 624 is "Please turn to the right", which can be used to prompt the user to turn their face to the right.

[0195] In some embodiments, the electronic device 100 can detect whether an image captured by a multispectral camera contains a face. If the image captured by the multispectral camera within a preset time period 1 does not contain a face, the electronic device 100 can prompt the user that face capture has failed. This application embodiment does not limit the value of the preset time period 1. For example, the electronic device 100 can display... Figure 6DThe user interface 630 is shown. The user interface 630 may include a prompt 631, a confirm control 632, and a cancel control 633. The prompt 631 can be used to inform the user that face capture has failed. The confirm control 632 can be used to attempt face capture again. The cancel control 633 can be used to cancel face capture. In some embodiments, the electronic device 100 can capture images using a multispectral camera within a preset time period 2. If the captured face image is still incomplete after the preset time period 2, the electronic device 100 can prompt the user that face capture has failed; see [reference needed] for details. Figure 6D The user interface 630 is described above. This application embodiment does not limit the value of the preset duration 2.

[0196] In some embodiments, when the completeness of the face image captured by the multispectral camera is detected, the electronic device 100 can display... Figure 6E The user interface 640 is shown. The user interface 640 may include a prompt 641. The prompt 641 can be used to inform the user that face capture is complete and that they need to wait for information to be entered.

[0197] After face capture is completed, the electronic device 100 can establish a multispectral face model 1 based on the face image captured by the multispectral camera, as detailed above. Figure 5 Step S516 is shown.

[0198] In some embodiments, when the multispectral face model 1 is successfully modeled, the electronic device 100 can display... Figure 6F The user interface 650 is shown. The user interface 650 may include a prompt 651. The prompt 651 can be used to notify the user that the face registration was successful.

[0199] The above illustrations of facial recognition scenarios are merely illustrative examples of this application and should not be construed as limiting the scope of this application.

[0200] Figure 7 An exemplary embodiment of the present application provides a face recognition method.

[0201] S711, An operation for performing face recognition was detected.

[0202] In the scenario of a face unlock device, the operation for performing face recognition may include triggering the device unlocking operation when the electronic device 100 is in a locked screen state. For example, in response to a tap on the screen, the electronic device 100 can light up the screen and begin face recognition to unlock the device. Alternatively, the electronic device 100 may have enabled the raise-to-wake function. When a raise-to-wake action is detected, the electronic device 100 can light up the screen and begin face recognition to unlock the device. The operations used to trigger the electronic device 100 to begin face recognition in the above face unlock device scenarios are merely illustrative examples of this application and should not be construed as limiting this application.

[0203] In facial recognition payment scenarios, the operations used for facial recognition can include making payments using a payment application. For example, when scanning a payment code or checking out online purchases using a payment application, the electronic device 100 can display the payment interface of the payment application. Upon entering the payment interface, the payment application can call the system's facial recognition interface to initiate facial recognition for payment via facial recognition.

[0204] Not limited to facial unlocking devices and facial payment scenarios, the above-mentioned operations for facial recognition can also be operations that trigger facial recognition in other facial application scenarios.

[0205] S712 emits infrared light via an infrared flash lamp.

[0206] S713, Use screen fill light when natural light brightness is below the brightness threshold.

[0207] Steps S712 and S713 can be referred to the foregoing. Figure 5 Steps S512 and S513 are shown.

[0208] In some embodiments, if the natural light brightness is lower than the brightness threshold, the electronic device 100 can keep the screen off during the face recognition process (i.e., when performing steps S711, S712, S714, and S715).

[0209] S714: Capture one or more facial images using a multispectral camera.

[0210] In some embodiments, the electronic device 100 can detect whether an image captured by a multispectral camera contains a human face. If the image captured by the multispectral camera does not contain a human face, the multispectral camera can continue to capture images in order to capture images containing a human face. Optionally, if the multispectral camera still fails to capture an image containing a human face after a preset time period, the electronic device 100 can stop face recognition and prompt the user that no face has been recognized.

[0211] Optionally, the electronic device 100 can also detect whether the size of the face region in the multispectral image meets a preset size. If the size of the face region is smaller than the preset size, the electronic device 100 can prompt the user to move their face closer to the electronic device 100. This can increase the face region in the image captured by the multispectral camera, more clearly capture the spectral features of the face, and thus improve the accuracy of face recognition.

[0212] In some embodiments, the electronic device 100 may also activate the RGB camera to capture images and display the images captured by the RGB camera on the screen so that the user can preview their face image during the face recognition process.

[0213] S715. Perform face recognition based on one or more face images or a multispectral face model 1.

[0214] As can be seen from the aforementioned structure of the multispectral camera, the face image captured by the multispectral camera can contain pixels corresponding to multiple channels, such as pixels corresponding to the infrared channel and pixels corresponding to multiple visible light channels of different spectral bands.

[0215] In some embodiments, the electronic device 100 can perform spectral sampling on a face image captured by a multispectral camera to obtain images corresponding to different spectral bands. For example, images corresponding to multiple different visible spectral bands and images corresponding to infrared spectral bands. The electronic device 100 can extract face information and multispectral anti-counterfeiting information from the images corresponding to different spectral bands, and then compare them with multispectral face model 1. For example, multispectral face model 1 includes face information and multispectral anti-counterfeiting information of face 1. The electronic device 100 captures a face image of face 2 through a multispectral camera. After performing spectral sampling on the face image of face 2, the electronic device 100 can extract face information and multispectral anti-counterfeiting information of face 2 from it. The face information of face 2 may include the contour features and facial features of face 2. The multispectral anti-counterfeiting information of face 2 may include the spectral features of face 2. In some embodiments, the electronic device 100 can extract the face information of face 2 from the image corresponding to the infrared spectral band, and obtain the multispectral anti-counterfeiting information of face 2 from the image corresponding to the infrared spectral band and the image corresponding to the visible spectral band. The multispectral anti-counterfeiting information of Face2 may include the spectral characteristics of Face2's infrared light and the spectral characteristics of its visible light.

[0216] The electronic device 100 can detect whether the multispectral anti-counterfeiting information of face 2 matches the multispectral anti-counterfeiting information of face 1, and detect whether the facial information of face 2 matches the facial information of face 1. If the multispectral anti-counterfeiting information of face 2 matches the multispectral anti-counterfeiting information of face 1, and the facial information of face 2 matches the facial information of face 1, then the electronic device 100 can determine that the face recognition is successful.

[0217] The matching of multispectral anti-counterfeiting information of face 2 with that of face 1 can include the fact that the multispectral anti-counterfeiting information of face 2 is the same as that of face 1, or the similarity is higher than a similarity threshold 1. Matching of multispectral anti-counterfeiting information of face 2 with that of face 1 indicates that the anti-counterfeiting detection has passed, meaning that the object captured by the multispectral camera is a real face. Matching of facial information of face 2 with that of face 1 can include the fact that the facial information of face 2 is the same as that of face 1, or the similarity is higher than a similarity threshold 2. Matching of facial information of face 2 with that of face 1 indicates that the face captured by electronic device 100 is face 1 that electronic device 100 previously recorded. This application embodiment does not limit the method for detecting whether the multispectral anti-counterfeiting information of face 2 matches that of face 1, nor the method for detecting whether the facial information of face 2 matches that of face 1.

[0218] In some embodiments, the electronic device 100 may first detect whether the multispectral anti-spoofing information of face 2 matches the multispectral anti-spoofing information of face 1. If the multispectral anti-spoofing information of face 2 matches the multispectral anti-spoofing information of face 1, the electronic device 100 may then detect whether the facial information of face 2 matches the facial information of face 1. If the multispectral anti-spoofing information of face 2 does not match the multispectral anti-spoofing information of face 1, the electronic device 100 may determine that the face recognition has failed without further detecting whether the facial information of face 2 matches the facial information of face 1. Alternatively, the electronic device 100 may first detect whether the facial information of face 2 matches the facial information of face 1. If the facial information of face 2 matches the facial information of face 1, the electronic device 100 may then detect whether the multispectral anti-spoofing information of face 2 matches the multispectral anti-spoofing information of face 1. If the facial information of face 2 does not match the facial information of face 1, the electronic device 100 may determine that the face recognition has failed without further detecting whether the multispectral anti-spoofing information of face 2 matches the multispectral anti-spoofing information of face 1.

[0219] In some embodiments, the electronic device 100 may store an algorithm model 1 for performing face recognition. Algorithm model 1 can detect whether face information matches and whether multispectral anti-counterfeiting information matches. Algorithm model 1 can be a model based on neural networks or deep learning. This application embodiment does not limit the implementation method of algorithm model 1. The input to algorithm model 1 may include a face image captured by a multispectral camera, and the output may include the face recognition result. The image input to algorithm model 1 may be an image corresponding to different spectral bands obtained by spectral sampling of the face image captured by the multispectral camera.

[0220] For example, electronic device 100 can input a face image containing face 2 captured by a multispectral camera into algorithm model 1, and determine whether face recognition is successful based on the output of algorithm model 1. Specifically, if the face information of face 2 matches the face information of face 1 and the multispectral anti-spoofing information of face 2 matches the multispectral anti-spoofing information of face 1, then algorithm model 1 can output a result indicating successful face recognition; otherwise, algorithm model 1 can output a result indicating failed face recognition.

[0221] In some embodiments, the visible light channel of the multispectral camera includes a high-sensitivity visible light channel and a low-sensitivity visible light channel. The electronic device 100 can select the channel for face recognition based on the natural light brightness. Specifically, if the natural light brightness is greater than 2, the electronic device 100 can extract pixels corresponding to the low-sensitivity visible light channel and pixels corresponding to the infrared channel to generate an image and perform face recognition. If the natural light brightness is less than or equal to 2, the electronic device 100 can use pixels corresponding to all channels to generate an image and perform face recognition.

[0222] Understandably, a natural light brightness greater than 2 indicates that the electronic device 100 is currently in a scene with excessively bright natural light. In excessively bright scenes, the visible light channel with high sensitivity is prone to overexposure, leading to information loss. Therefore, using a less sensitive visible light channel instead of a high-sensitivity visible light channel in excessively bright scenes can improve the accuracy of facial recognition.

[0223] Furthermore, in situations with insufficient natural light, i.e., in very dark scenes, the electronic device 100 can use screen illumination to enable the multispectral camera to capture sufficient light signals. This improves the accuracy of facial recognition in low-light environments.

[0224] If face recognition fails (i.e., the multispectral anti-counterfeiting information of face 2 does not match the multispectral anti-counterfeiting information of face 1, and / or, the facial information of face 2 does not match the facial information of face 1), the electronic device 100 can provide a prompt related to the face recognition failure based on the current face recognition scenario. For example, if the face recognition scenario is a face unlock device scenario, the electronic device 100 can prompt the user that face unlock has failed. If the face recognition scenario is a face payment scenario, the electronic device 100 can prompt the user that payment has failed.

[0225] If facial recognition is successful (i.e., the multispectral anti-counterfeiting information of face 2 matches the multispectral anti-counterfeiting information of face 1, and the facial information of face 2 matches the facial information of face 1), then the electronic device 100 can perform operations related to the successful facial recognition based on the current facial recognition scenario. For example, if the facial recognition scenario is for unlocking the device, the electronic device 100 can unlock the device and display the unlocked interface (e.g., the desktop of the electronic device 100, or the user interface of the APP in the electronic device 100). If the facial recognition scenario is for facial payment, then the electronic device 100 can complete the payment.

[0226] In some embodiments, step S713 is optional. In the scenario of a face unlock device, the electronic device 100 can use an infrared flash for illumination, without the need for screen illumination. For example, the electronic device 100 can activate the infrared flash and multispectral camera in a screen-off state to perform face recognition, and unlock the electronic device 100 upon successful face recognition. This can improve the speed of face unlocking.

[0227] As shown by the above method, the electronic device 100 can use a multispectral camera and an infrared flash for face recognition. Multispectral cameras and infrared flashes are low-cost. The above method can reduce the cost of face recognition. The images captured by the multispectral camera contain the spectral features of the face, enabling anti-spoofing detection of faces, improving the accuracy of face recognition, and better protecting the user's privacy and personal property security in the electronic device. Furthermore, in low-light environments, the electronic device 100 can use screen illumination, which not only eliminates the need for additional lighting devices in the electronic device 100, reducing the cost of face recognition, but also improves the accuracy of face recognition in low-light environments, thereby enhancing the experience of face unlocking devices or face payment in low-light conditions.

[0228] Figure 8 Another face recognition method provided by an embodiment of this application is illustrated by way of example.

[0229] S811, An operation for face registration was detected.

[0230] S812 emits infrared light through an infrared flash lamp.

[0231] S813, Use screen fill light when natural light brightness is below the brightness threshold.

[0232] S814 prompts the user to turn their face.

[0233] Steps S811 to S814 can be referred to the above. Figure 5 The steps S511 to S514 are shown.

[0234] The S815 captures multiple facial images of the user from different angles using a multispectral camera and an RGB camera.

[0235] During the face registration process, the electronic device 100 can collect data through a multispectral camera, capturing multiple facial images of the user from different angles. The electronic device 100 can also continuously collect data through an RGB camera, capturing multiple facial images of the user from different angles. Facial images captured by a multispectral camera can be called multispectral face images. Facial images captured by an RGB camera can be called RGB face images.

[0236] It should be noted that multispectral cameras and RGB cameras can capture images simultaneously.

[0237] S816. Given complete facial images, establish a multispectral facial model 2 based on the facial images captured by the multispectral camera and the facial images captured by the RGB camera.

[0238] In some embodiments, the electronic device 100 can perform completeness detection on images captured by a multispectral camera and / or images captured by an RGB camera in real time. Completeness detection can be referred to the foregoing. Figure 5 The following is a description of step S516.

[0239] When the face image captured by the multispectral camera and / or the image captured by the RGB camera are complete, the electronic device 100 can establish a face model 2 based on the multispectral face image captured by the multispectral camera and the RGB face image captured by the RGB camera. The electronic device 100 can calculate depth information based on the multispectral face image and the RGB face image. The depth information of a point can be determined by the disparity of that point between the multispectral camera and the RGB camera. The specific method for calculating depth information can refer to the method for calculating depth information using dual cameras; this embodiment does not limit this method. The electronic device 100 can also determine face information based on the multispectral face image and / or the RGB face image, and determine multispectral anti-spoofing information based on the multispectral face image. For example, the face used for face registration on the electronic device 100 is the face 3 of user 2. The multispectral face model 2 can be the result obtained by the electronic device 100 performing face modeling on face 3. The multispectral face model 2 can include the face information of face 3, multispectral anti-spoofing information, and depth information. The facial information of face 3 may include facial features such as contour features and facial features, indicating the appearance of user 2's face. The multispectral anti-spoofing information of face 3 may include the spectral features of face 3, indicating the absorption rate of face 3 to different spectral bands of light (e.g., visible light, infrared light). The depth information of face 3 indicates the 3D structure of face 3. The facial information of face 3 can be used for face matching so that electronic device 100 can determine whether the captured face is face 3. The multispectral anti-spoofing information and depth information of face 3 can be used for anti-spoofing detection so that electronic device 100 can determine whether the captured face is a forged face. Anti-spoofing detection can be used to detect whether the face image is an image obtained from a multispectral camera or an RGB camera capturing a real person, thereby reducing the use of other people's images or videos, or forged face methods such as face masks, to pass face recognition, improving the security of face recognition. The process of establishing the multispectral face model 2 can be a process of determining facial information, multispectral anti-spoofing information, and depth information based on multispectral face images and RGB face images. The embodiments of this application do not limit the above-described face modeling method.

[0240] In some embodiments, the electronic device 100 can perform spectral sampling on the multispectral face image to obtain an image corresponding to the infrared spectral band. The electronic device 100 can extract the facial information of face 3 from the image corresponding to the infrared spectral band. Alternatively, the electronic device 100 can combine the multispectral face image and the RGB face image to determine the facial information of face 3.

[0241] The multispectral anti-counterfeiting information of Face3 may include the spectral characteristics of Face3's infrared light and visible light.

[0242] Understandably, multispectral face images encompass the spectral features of a face. Real faces and fake faces (such as faces in images, videos, or face masks) absorb and reflect different amounts of light. That is, real faces and fake faces have different spectral characteristics. Furthermore, electronic device 100 can determine the depth information of a face based on multispectral and RGB face images. Facial depth information reflects the three-dimensional structure of a face. In scenarios where images or videos are used to forge faces, these images and videos are planar and lack three-dimensional depth information. Moreover, the three-dimensional structure of a face mask is usually difficult to completely replicate the three-dimensional structure of a real face. Therefore, based on the aforementioned spectral features and depth information of the face, anti-spoofing detection can be performed more accurately.

[0243] S817. Based on the modeling results of the multispectral face model 2, prompt the user whether the face registration was successful or failed.

[0244] Step S817 can be referred to the above. Figure 5 The step S517 is shown.

[0245] As shown by the above method, electronic device 100 can use images captured by a multispectral camera and an RGB camera to perform face modeling, obtaining a multispectral face model 2. Multispectral face model 2 contains multispectral anti-spoofing information and depth information, which can improve the accuracy of face recognition and better protect the user's privacy and personal property security within the electronic device.

[0246] Figure 9 Another face recognition method provided by an embodiment of this application is illustrated by way of example.

[0247] S911, An operation for performing face recognition was detected.

[0248] S912 emits infrared light through an infrared flash lamp.

[0249] S913. Use screen fill light when the natural light brightness is lower than the brightness threshold.

[0250] Steps S911 to S913 can be referred to the above. Figure 7 The steps S711 to S713 are shown.

[0251] S914. Capture one or more facial images using a multispectral camera and one or more facial images using an RGB camera.

[0252] Multispectral cameras and RGB cameras can capture images simultaneously. For capturing facial images with a multispectral camera, please refer to the aforementioned methods. Figure 7 Step S714 is shown.

[0253] In some embodiments, the electronic device 100 can detect whether the image captured by the RGB camera contains a human face. If the image captured by the RGB camera does not contain a human face, the RGB camera can continue to capture images in order to capture images containing a human face. Optionally, if the RGB camera still fails to capture an image containing a human face after a preset time period, the electronic device 100 can stop face recognition and prompt the user that no human face has been recognized.

[0254] S915 performs face recognition based on face images captured by a multispectral camera and an RGB camera, and a multispectral face model 2.

[0255] Electronic device 100 can determine facial information, multispectral anti-spoofing information, and depth information based on facial images captured by a multispectral camera and an RGB camera, and then compare them with a multispectral facial model 2. For example, multispectral facial model 2 includes facial information and multispectral anti-spoofing information of face 3. Electronic device 100 captures a multispectral facial image of face 4 using a multispectral camera and an RGB facial image of face 4 using an RGB camera. Electronic device 100 can determine facial information of face 4 based on the multispectral facial image and / or the RGB facial image of face 4, determine depth information of face 4 using the multispectral facial image and the RGB image of face 4, and determine multispectral anti-spoofing information of face 4 using the multispectral facial image of face 4. Facial information of face 4 may include contour features and facial features of face 4. In some embodiments, electronic device 100 can perform spectral sampling on the multispectral facial image of face 4 to obtain an infrared spectral image. Electronic device 100 can extract facial information of face 4 from the infrared spectral image. The multispectral anti-spoofing information of face 4 may include the spectral features of face 4 (e.g., the spectral features of infrared light and the spectral features of visible light). The depth information of face 4 can indicate the 3D structure of face 4. The electronic device 100 can detect whether the multispectral anti-spoofing information of face 4 matches the multispectral anti-spoofing information of face 3, whether the depth information of face 4 matches the depth information of face 3, and whether the facial information of face 4 matches the facial information of face 3. If the multispectral anti-spoofing information of face 4 matches the multispectral anti-spoofing information of face 3, the depth information of face 4 matches the depth information of face 3, and the facial information of face 4 matches the facial information of face 3, then the electronic device 100 can determine that the face recognition is successful.

[0256] The matching of multispectral anti-spoofing information of face 4 with multispectral anti-spoofing information of face 3 can include the fact that the multispectral anti-spoofing information of face 4 is the same as that of face 3, or the similarity is higher than the similarity threshold 1. The matching of depth information of face 4 with depth information of face 3 can include the fact that the depth information of face 4 is the same as that of face 3, or the similarity is higher than the similarity threshold 3. The matching of multispectral anti-spoofing information of face 4 with multispectral anti-spoofing information of face 3, and the matching of depth information of face 4 with depth information of face 3, can indicate that the anti-spoofing detection has passed, meaning that the object captured by the multispectral camera is a real face. The matching of facial information of face 4 with facial information of face 3 can include the fact that the facial information of face 4 is the same as that of face 3, or the similarity is higher than the similarity threshold 2. The matching of facial information of face 4 with facial information of face 3 can indicate that the face captured by electronic device 100 is face 3 that electronic device 100 previously recorded. This application does not limit the methods for detecting whether the multispectral anti-counterfeiting information of face 4 matches the multispectral anti-counterfeiting information of face 3, the methods for detecting whether the depth information of face 4 matches the depth information of face 3, or the methods for detecting whether the facial information of face 4 matches the facial information of face 3.

[0257] In some embodiments, the electronic device 100 can perform spectral sampling on face images captured by a multispectral camera to obtain images corresponding to different spectral bands. For example, images corresponding to multiple different visible spectral bands and images corresponding to infrared spectral bands. The electronic device 100 can perform face recognition on face images captured by an RGB camera and images corresponding to different spectral bands.

[0258] In some embodiments, the electronic device 100 may store an algorithm model 2 for performing face recognition. Algorithm model 2 can detect whether face information matches, whether multispectral anti-counterfeiting information matches, and whether depth information matches. Algorithm model 2 can be a model based on neural networks or deep learning. This application embodiment does not limit the implementation method of algorithm model 2. The input to algorithm model 2 may include face images captured by a multispectral camera and face images captured by RGB cameras. The face images captured by the multispectral camera input to algorithm model 2 can be images corresponding to different spectral bands obtained after spectral band sampling.

[0259] For example, electronic device 100 can input a multispectral face image containing face 4 captured by a multispectral camera and an RGB face image containing face 4 captured by an RGB camera into algorithm model 2, and determine whether face recognition is successful based on the output of algorithm model 2. Specifically, if the multispectral anti-spoofing information of face 4 matches the multispectral anti-spoofing information of face 3, the depth information of face 4 matches the depth information of face 3, and the facial information of face 4 matches the facial information of face 3, then algorithm model 2 can output a successful face recognition result; otherwise, algorithm model 2 can output a failed face recognition result.

[0260] In some embodiments, the visible light channel of the multispectral camera includes a high-sensitivity visible light channel and a low-sensitivity visible light channel. The electronic device 100 can select the channel for face recognition based on the natural light brightness. Specifically, if the natural light brightness is greater than 2, the electronic device 100 can extract pixels corresponding to the low-sensitivity visible light channel and pixels corresponding to the infrared channel to generate an image and perform face recognition. If the natural light brightness is less than or equal to 2, the electronic device 100 can use pixels corresponding to all channels to generate an image and perform face recognition.

[0261] Understandably, a natural light brightness greater than 2 indicates that the electronic device 100 is currently in a scene with excessively bright natural light. In excessively bright scenes, the visible light channel with high sensitivity is prone to overexposure, leading to information loss. Therefore, using a less sensitive visible light channel instead of a high-sensitivity visible light channel in excessively bright scenes can improve the accuracy of facial recognition.

[0262] Furthermore, in situations with insufficient natural light, i.e., in very dark scenes, the electronic device 100 can use screen illumination to enable the multispectral camera to capture sufficient light signals. This improves the accuracy of facial recognition in low-light environments.

[0263] If facial recognition fails (i.e., one or more of the following conditions are met: the multispectral anti-spoofing information of face 4 does not match the multispectral anti-spoofing information of face 3, the depth information of face 4 does not match the depth information of face 3, or the facial information of face 4 does not match the facial information of face 3), then the electronic device 100 can provide a prompt related to the facial recognition failure based on the current facial recognition scenario. For example, if the facial recognition scenario is a facial unlock device scenario, the electronic device 100 can prompt the user that facial unlocking has failed. If the facial recognition scenario is a facial payment scenario, the electronic device 100 can prompt the user that payment has failed.

[0264] If facial recognition is successful (i.e., the multispectral anti-spoofing information of face 4 matches the multispectral anti-spoofing information of face 3, the depth information of face 4 matches the depth information of face 3, and the facial information of face 4 matches the facial information of face 3), then the electronic device 100 can perform operations related to successful facial recognition based on the current facial recognition scenario. For example, if the facial recognition scenario is for unlocking the device, the electronic device 100 can unlock the device and display the unlocked interface (e.g., the desktop of the electronic device 100, or the user interface of the APP in the electronic device 100). If the facial recognition scenario is for facial payment, the electronic device 100 can complete the payment.

[0265] As described above, the electronic device 100 can perform facial recognition by combining images captured by a multispectral camera and an RGB camera. This method utilizes the spectral features and depth information of the face, which can improve the accuracy of facial recognition and better protect the user's privacy and personal property security within the electronic device. Furthermore, in low-light environments, the electronic device 100 can use screen illumination, eliminating the need for additional lighting components and reducing the cost of facial recognition. This also improves the accuracy of facial recognition in low-light conditions, thereby enhancing the experience of facial unlocking devices or facial payment in such environments.

[0266] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described method embodiments.

[0267] This application also provides a computer program product, including a computer program that, when run on a processor, can implement the steps in the various method embodiments described above.

[0268] This application also provides a chip system, which includes a processing circuit and an interface circuit. The interface circuit receives code instructions and transmits them to the processing circuit. The processing circuit executes the code instructions to enable the chip system to implement the steps of any method embodiment of this application. The chip system can be a single chip or a chip module composed of multiple chips.

[0269] It is understood that the user interfaces described in the embodiments of this application are merely example interfaces and do not constitute a limitation on the solution of this application. In other embodiments, the user interface may adopt different interface layouts, may include more or fewer controls, and may add or remove other functional options, as long as they are based on the same inventive concept provided in this application, they are all within the protection scope of this application.

[0270] It should be noted that, without causing contradictions or conflicts, any feature in any embodiment of this application, or any part of any feature, can be combined, and the combined technical solution is also within the scope of the embodiments of this application.

[0271] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A face recognition method, characterized in that, The method is applied to an electronic device, which includes an infrared flash and a multispectral camera. The image sensor of the multispectral camera includes a filter array, which includes multiple filter units arranged in an array. Each filter unit includes N rows * N columns of sub-units arranged in an array, where N is greater than 2. The N rows * N columns of sub-units include multiple infrared filter sub-units and multiple visible light filter sub-units. The infrared filter sub-units are used to filter infrared light, and the visible light filter sub-units are used to filter visible light. The multiple infrared filter sub-units are evenly spaced along the horizontal and vertical directions, and one or more visible light filter sub-units are spaced between two infrared filter sub-units. The method includes: The electronic device emits infrared light via the infrared flash lamp; The electronic device captures a first image using the multispectral camera, and the first image contains a second human face. The electronic device performs anti-counterfeiting detection on the second face based on the first image, and detects whether the second face matches the first face stored in the electronic device; If the anti-counterfeiting detection passes and the second face matches the first face, the electronic device determines that the face recognition is successful.

2. The method according to claim 1, characterized in that, The number of visible light filter sub-units in the image sensor of the multispectral camera is greater than or equal to 5.

3. The method according to claim 1 or 2, characterized in that, The image sensor includes multiple channels, with each channel corresponding to one sub-unit. The visible light filter sub-unit corresponding to the infrared filter sub-unit includes a first visible light filter sub-unit and a second visible light filter sub-unit. The first visible light filter sub-unit and the infrared filter sub-unit are arranged in a row or column. The second visible light filter sub-unit is located diagonally opposite the infrared filter sub-unit. The channel corresponding to the first visible light filter sub-unit is the first channel, and the channel corresponding to the second visible light filter sub-unit is the second channel. The sensitivity of the second channel is greater than that of the first channel.

4. The method according to any one of claims 1-3, characterized in that, The filtering unit includes 4*4 sub-units. In each filtering unit, there are 4 infrared filtering sub-units and 6 types of visible light filtering sub-units. Each type of visible light filtering sub-unit is repeated twice. Alternatively, in each filtering unit, there are 8 types of visible light filtering sub-units. Among the 8 types of visible light filtering sub-units, 4 types of visible light filtering sub-units are repeated twice. Different types of visible light filtering sub-units are used to filter visible light of different spectral bands.

5. The method according to any one of claims 1-4, characterized in that, The filtering unit includes an infrared filtering subunit for collecting infrared light greater than or equal to 820 nm and less than or equal to 870 nm, and / or an infrared filtering subunit for collecting infrared light greater than or equal to 920 nm and less than or equal to 970 nm.

6. The method according to any one of claims 1-5, characterized in that, The electronic device further includes a screen, and before the electronic device captures the first image via the multispectral camera, the method further includes: The electronic device detects ambient light intensity; If the ambient light brightness is lower than the first brightness, the electronic device controls the screen brightness to be greater than or equal to the second brightness.

7. The method according to any one of claims 1-6, characterized in that, The image sensor includes multiple third channels and multiple fourth channels. One third channel corresponds to one infrared filter subunit, and one fourth channel corresponds to one visible light filter subunit. The sensitivity of the fourth channel is less than that of the first channel. The electronic device captures a first image via the multispectral camera, including: If the ambient light intensity is greater than the third brightness, the electronic device generates the first image based on the signals from the plurality of third channels and the plurality of fourth channels.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: The electronic device receives an operation for facial recognition; The electronic device emits infrared light via the infrared flash lamp; The electronic device captures multiple images containing the first face using the multispectral camera, the multiple images including images of the first face from multiple angles; The electronic device determines the first face information and the first multispectral anti-counterfeiting information of the first face based on the multiple images. The first face information is used for face matching, and the first multispectral anti-counterfeiting information is used for anti-counterfeiting detection. The electronic device stores the first facial information and the first multispectral anti-counterfeiting information.

9. The method according to claim 8, characterized in that, The electronic device performs anti-spoofing detection on the second face based on the first image, and detects whether the second face matches the first face stored in the electronic device, including: The electronic device determines the second face information and the second multispectral anti-counterfeiting information of the second face based on the first image; The electronic device performs anti-counterfeiting detection on the second face based on the second multispectral anti-counterfeiting information and the first multispectral anti-counterfeiting information, and detects whether the second face matches the first face based on the second face information and the first face information; Specifically, if the first multispectral anti-counterfeiting information matches the second multispectral anti-counterfeiting information, the anti-counterfeiting detection passes; if the first face information matches the second face information, the second face matches the first face.

10. The method according to any one of claims 1-9, characterized in that, The electronic device also includes an RGB camera, and the method further includes: The electronic device emits infrared light via the infrared flash lamp; The electronic device captures a second image using the multispectral camera and a third image using the RGB camera, both of which contain a fourth face. The electronic device performs anti-counterfeiting detection on the fourth face based on the second image and the third image, and detects whether the fourth face matches the third face stored in the electronic device; If the anti-counterfeiting detection passes and the fourth face matches the third face, the electronic device determines that the face recognition is successful.

11. The method according to claim 10, characterized in that, The method further includes: The electronic device receives an operation for facial recognition; The electronic device emits infrared light via the infrared flash lamp; The electronic device captures multiple multispectral images containing the third face using the multispectral camera and multiple RGB images containing the third face using the RGB camera. Both the multiple multispectral images and the multiple RGB images include images of the third face from multiple angles. The electronic device determines the third face information of the third face based on the multiple multispectral images and / or the multiple RGB images, determines the first depth information based on the multiple multispectral images and the multiple RGB images, and determines the third multispectral anti-counterfeiting information of the third face based on the multiple multispectral images. The third face information is used for face matching, and the first depth information and the third multispectral anti-counterfeiting information are used for anti-counterfeiting detection. The electronic device stores the third facial information, the first depth information, and the third multispectral anti-counterfeiting information.

12. The method according to claim 11, characterized in that, The electronic device performs anti-spoofing detection on the fourth face based on the second image and the third image, and detects whether the fourth face matches the third face stored in the electronic device, including: The electronic device determines the fourth face information of the fourth face based on the second image and / or the third image, determines the second depth information based on the second image and the third image, and determines the fourth multispectral anti-counterfeiting information of the fourth face based on the second image. The electronic device performs anti-counterfeiting detection on the fourth face based on the first depth information, the second depth information, the third multispectral anti-counterfeiting information, and the fourth multispectral anti-counterfeiting information, and detects whether the fourth face matches the third face based on the third face information and the fourth face information; Specifically, if the first depth information and the second depth information match, and the third multispectral anti-counterfeiting information and the fourth multispectral anti-counterfeiting information match, then the anti-counterfeiting detection passes; if the fourth face information matches the third face information, then the fourth face matches the third face.

13. A face recognition method, characterized in that, The method is applied to an electronic device, which includes an infrared flash, a multispectral camera, and an RGB camera. The image sensor of the multispectral camera includes a filter array, which includes multiple filter units arranged in an array. Each filter unit includes N rows * N columns of sub-units arranged in an array, where N is greater than 2. The N rows * N columns of sub-units include multiple infrared filter sub-units and multiple visible light filter sub-units. The infrared filter sub-units are used to filter infrared light, and the visible light filter sub-units are used to filter visible light. The multiple infrared filter sub-units are evenly spaced along the horizontal and vertical directions, and one or more visible light filter sub-units are spaced between two infrared filter sub-units. The method includes: The electronic device emits infrared light via the infrared flash lamp; The electronic device captures a second image using the multispectral camera and a third image using the RGB camera, both of which contain a fourth face. The electronic device performs anti-counterfeiting detection on the fourth face based on the second image and the third image, and detects whether the fourth face matches the third face stored in the electronic device; If the anti-counterfeiting detection passes and the fourth face matches the third face, the electronic device determines that the face recognition is successful.

14. The method according to claim 13, characterized in that, The number of visible light filter sub-units in the image sensor of the multispectral camera is greater than or equal to 5.

15. The method according to claim 13 or 14, characterized in that, The image sensor includes multiple channels, with each channel corresponding to one sub-unit. The visible light filter sub-unit corresponding to the infrared filter sub-unit includes a first visible light filter sub-unit and a second visible light filter sub-unit. The first visible light filter sub-unit and the infrared filter sub-unit are arranged in a row or column. The second visible light filter sub-unit is located diagonally opposite the infrared filter sub-unit. The channel corresponding to the first visible light filter sub-unit is the first channel, and the channel corresponding to the second visible light filter sub-unit is the second channel. The sensitivity of the second channel is greater than that of the first channel.

16. The method according to any one of claims 13-15, characterized in that, The filtering unit comprises 4*4 sub-units. In each filtering unit, there are 4 infrared filtering sub-units and 6 types of visible light filtering sub-units. Each type of visible light filtering sub-unit is repeated twice. Alternatively, in each filtering unit, there are 8 types of visible light filtering sub-units. Among the 8 types of visible light filtering sub-units, 4 types are repeated twice. Different types of visible light filtering sub-units are used to filter visible light of different spectral bands.

17. The method according to any one of claims 13-16, characterized in that, The filtering unit includes an infrared filtering subunit for collecting infrared light greater than or equal to 820 nm and less than or equal to 870 nm, and / or an infrared filtering subunit for collecting infrared light greater than or equal to 920 nm and less than or equal to 970 nm.

18. The method according to any one of claims 13-17, characterized in that, The electronic device further includes a screen, and before the electronic device captures a second image via the multispectral camera and a third image via the RGB camera, the method further includes: The electronic device detects ambient light intensity; If the ambient light brightness is lower than the first brightness, the electronic device controls the screen brightness to be greater than or equal to the second brightness.

19. The method according to any one of claims 13-18, characterized in that, The image sensor includes multiple third channels and multiple fourth channels. One third channel corresponds to one infrared filter subunit, and one fourth channel corresponds to one visible light filter subunit. The sensitivity of the fourth channel is less than that of the first channel. The electronic device captures a second image via the multispectral camera, including: If the ambient light intensity is greater than the third intensity, the electronic device generates the second image based on the signals from the plurality of third channels and the plurality of fourth channels.

20. The method according to any one of claims 13-19, characterized in that, The method further includes: The electronic device receives an operation for facial recognition; The electronic device emits infrared light via the infrared flash lamp; The electronic device captures multiple multispectral images containing the third face using the multispectral camera and multiple RGB images containing the third face using the RGB camera. Both the multiple multispectral images and the multiple RGB images include images of the third face from multiple angles. The electronic device determines the third face information of the third face based on the multiple multispectral images and / or the multiple RGB images, determines the first depth information based on the multiple multispectral images and the multiple RGB images, and determines the third multispectral anti-counterfeiting information of the third face based on the multiple multispectral images. The third face information is used for face matching, and the third depth information and the third multispectral anti-counterfeiting information are used for anti-counterfeiting detection. The electronic device stores the third facial information, the first depth information, and the third multispectral anti-counterfeiting information.

21. The method according to claim 20, characterized in that, The electronic device performs anti-spoofing detection on the fourth face based on the second image and the third image, and detects whether the fourth face matches the third face stored in the electronic device, including: The electronic device determines the fourth face information of the fourth face based on the second image and / or the third image, determines the second depth information based on the second image and the third image, and determines the fourth multispectral anti-counterfeiting information of the fourth face based on the second image; The electronic device performs anti-counterfeiting detection on the fourth face based on the first depth information, the second depth information, the third multispectral anti-counterfeiting information, and the fourth multispectral anti-counterfeiting information, and detects whether the fourth face matches the third face based on the third face information and the fourth face information; Specifically, if the first depth information and the second depth information match, and the third multispectral anti-counterfeiting information and the fourth multispectral anti-counterfeiting information match, then the anti-counterfeiting detection passes; if the fourth face information matches the third face information, then the fourth face matches the third face.

22. An electronic device, characterized in that, The electronic device includes an infrared flash, a multispectral camera, a memory, and a processor, wherein the infrared flash is used to emit infrared light; the multispectral camera is used to capture multispectral images; the memory is used to store a computer program; and the processor executes the computer program to implement the method of any one of claims 1-21.

23. A computer-readable storage medium storing instructions, characterized in that, When the instructions are executed by the processor, they implement the method of any one of claims 1-21.

24. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed by a processor, implement the method of any one of claims 1-21.

Images