Face recognition method, device and equipment and storage medium

Abstract

The present specification discloses a face recognition method, device and equipment, and a storage medium, the method is executed by a terminal, the terminal comprises a structured light system, the structured light system comprises an infrared speckle projector and an infrared camera. The steps comprise: acquiring a first infrared face image of a target object when the infrared speckle projector is turned off, the first infrared face image is photographed by the infrared camera; determining an environment state in which the terminal is located based on the first infrared face image; starting the infrared speckle projector and adjusting the working parameters of the structured light system based on the environment state; and performing face recognition on the target object based on the structured light system with adjusted working parameters.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a face recognition method, apparatus, device, and storage medium. Background Technology

[0002] With the development of computer technology, facial recognition technology has been widely used in recent years. For example, facial recognition technology is used to unlock mobile phones, greatly improving the efficiency of unlocking. In addition, facial recognition systems are widely deployed on payment platforms, allowing users to quickly complete payments through facial recognition, greatly improving payment efficiency.

[0003] In related technologies, the success rate and accuracy of facial recognition are affected by the environment. How to improve the success rate and accuracy of facial recognition in different environments is an urgent issue that needs to be studied. Summary of the Invention

[0004] This specification provides a face recognition method, apparatus, device, and storage medium, which can improve the success rate and accuracy of face recognition in different environments. The technical solution is as follows:

[0005] On one hand, a face recognition method is provided, executed by a terminal, the terminal including a structured light system, the structured light system including an infrared speckle projector and an infrared camera, the method including:

[0006] With the infrared speckle projector off, a first infrared face image of the target object is acquired, the first infrared face image being captured by the infrared camera;

[0007] Based on the first infrared face image, the environmental state of the terminal is determined;

[0008] The infrared speckle projector is activated, and the operating parameters of the structured light system are adjusted based on the environmental conditions.

[0009] The structured light system, with its adjusted operating parameters, performs facial recognition on the target object.

[0010] On one hand, a face recognition device is provided for use in a terminal, the terminal including a structured light system, the structured light system including an infrared speckle projector and an infrared camera, the device including:

[0011] The image acquisition module is used to acquire a first infrared face image of the target object when the infrared speckle projector is turned off, wherein the first infrared face image is captured by the infrared camera;

[0012] An environment state determination module is used to determine the environmental state of the terminal based on the first infrared face image.

[0013] The operating parameter adjustment module is used to start the infrared speckle projector and adjust the operating parameters of the structured light system based on the environmental conditions.

[0014] A face recognition module is used to perform face recognition on the target object based on the structured light system after adjusting the working parameters.

[0015] In one possible implementation, the environment state determination module is configured to perform any of the following:

[0016] If the brightness value of the first infrared face image is greater than or equal to the brightness threshold, the environmental state of the terminal is determined to be the first environmental state, which indicates that there is a light source in the environment where the terminal is located that meets the target lighting conditions.

[0017] If a speckle pattern is present in the first infrared face image, the environmental state of the terminal is determined to be a second environmental state, which indicates that there is interference from other infrared speckle projectors in the environment where the terminal is located.

[0018] In one possible implementation, the structured light system further includes a visible light camera, and the operating parameter adjustment module is configured to perform at least one of the following when the environmental state of the terminal is the first environmental state:

[0019] Reduce the exposure time of the infrared camera;

[0020] Improve the signal-to-noise ratio of the infrared speckle projector;

[0021] The metering area of ​​the visible light camera is adjusted to the face area of ​​the target object.

[0022] In one possible implementation, the structured light system further includes a visible light camera, and the operating parameter adjustment module is configured to perform at least one of the following when the environmental state of the terminal is the second environmental state:

[0023] Adjust the operating time of the infrared speckle projector;

[0024] Adjust the operating frequency of the infrared speckle projector;

[0025] Improve the signal-to-noise ratio of the infrared speckle projector;

[0026] Reduce the exposure time of the infrared camera.

[0027] In one possible implementation, the device further includes a first adjustment module, configured to perform at least one of the following when the environmental state of the terminal is the first environmental state:

[0028] Increase the screen brightness of the terminal;

[0029] Increase the contrast of the current display interface of the terminal;

[0030] Increase the volume of the terminal.

[0031] In one possible implementation, the device further includes a second adjustment module, configured to perform at least one of the following when the environmental state of the terminal is the second environmental state:

[0032] Display a prompt message;

[0033] Play the prompt voice message;

[0034] Both the prompt message and the prompt voice are used to prompt the user to change the location of the terminal.

[0035] In one possible implementation, the structured light system further includes a visible light camera and a face recognition module, used to acquire a second infrared face image captured by the infrared camera and a first color face image captured by the visible light camera, wherein both the second infrared face image and the first color face image are acquired after the operating parameters of the structured light system are adjusted; and face recognition is performed on the target object based on the second infrared face image and the first color face image.

[0036] In one possible implementation, the face recognition module is configured to determine the quality score of the second infrared face image; control the infrared speckle projector to turn off when the quality score of the second infrared face image is less than or equal to a first quality score threshold; acquire a third infrared face image captured by the infrared camera; and perform face recognition on the target object based on the third infrared face image and the first color face image.

[0037] In one possible implementation, the face recognition module is further configured to reduce the weight of the second infrared face image in the face recognition process when the quality score of the second infrared face image is less than or equal to a first quality score threshold, wherein the weight is positively correlated with the degree of influence on the face recognition result; and to perform face recognition on the target object based on the second infrared face image and the first color face image.

[0038] In one possible implementation, the face recognition module is further configured to: determine the three-dimensional face structure of the target object based on the second infrared face image when the quality score of the second infrared face image is greater than the first quality score threshold; extract features from the three-dimensional face structure and the first color face image respectively to obtain the three-dimensional face features and two-dimensional face features of the target object; and perform face recognition on the target object based on the three-dimensional face features and the two-dimensional face features.

[0039] In one possible implementation, the face recognition module is further configured to determine the quality score of the first color face image; if the quality score of the first color face image is less than or equal to a second quality score threshold, perform image enhancement on the first color face image to obtain a second color face image; and perform face recognition on the target object based on the second infrared face image and the second color face image.

[0040] On one hand, a computer device is provided, the computer device including one or more processors and one or more memories, the one or more memories storing at least one computer program, the computer program being loaded and executed by the one or more processors to implement the face recognition method.

[0041] On one hand, a computer-readable storage medium is provided, wherein at least one computer program is stored in the computer-readable storage medium, the computer program being loaded and executed by a processor to implement the face recognition method.

[0042] On one hand, a computer program product or computer program is provided, which includes program code stored in a computer-readable storage medium. The processor of a computer device reads the program code from the computer-readable storage medium and executes the program code, causing the computer device to perform the aforementioned face recognition method.

[0043] The technical solution provided in the embodiments of this specification allows for the acquisition of a first infrared face image by an infrared camera within the structured light system when the infrared speckle projector in the structured light system is turned off. This first infrared face image is used to determine the environmental state of the terminal. After obtaining the environmental state, the infrared speckle projector is activated, and the operating parameters of the structured light system are adjusted based on the environmental state, enabling the structured light system to operate in a manner matching the environmental state. Based on the adjusted operating parameters, the structured light system performs face recognition on the target object, thereby improving the success rate and accuracy of face recognition under different environmental conditions. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a schematic diagram of the implementation environment of a face recognition method provided in the embodiments of this specification;

[0046] Figure 2 This is a flowchart of a face recognition method provided in the embodiments of this specification;

[0047] Figure 3 This is a flowchart of another face recognition method provided in the embodiments of this specification;

[0048] Figure 4 This is a schematic diagram of the structure of a terminal provided in an embodiment of this specification;

[0049] Figure 5 This is a schematic diagram of the structure of a face recognition device provided in the embodiments of this specification;

[0050] Figure 6 This is a schematic diagram of another terminal provided in the embodiments of this specification. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this specification clearer, the embodiments of this specification will be further described in detail below with reference to the accompanying drawings.

[0052] In this manual, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function. It should be understood that there is no logical or temporal dependency between "first," "second," and "nth," nor are there any restrictions on quantity or execution order.

[0053] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0054] Artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain better results.

[0055] Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence. Its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learning by demonstration.

[0056] Biometrics: Biometrics technology combines computers with high-tech methods such as optics, acoustics, biosensors, and biostatistics to identify individuals using their inherent physiological characteristics (such as fingerprints, facial images, and irises) and behavioral features (such as handwriting, voice, and gait).

[0057] Facial recognition: Facial recognition is a biometric technology that identifies individuals based on their facial features. It involves using cameras or webcams to capture images or video streams containing faces, automatically detecting and tracking faces within the images, and then performing facial recognition on the detected faces. This technology is also commonly referred to as portrait recognition or face recognition.

[0058] Structured light system: A structured light system is a system structure consisting of a projector and a camera. Specific light information is projected onto the surface of an object and its background by the projector, and then captured by the camera. Based on the changes in the light signal caused by the object, information such as the object's position and depth is calculated, thereby reconstructing the entire three-dimensional space. Typically, the camera is an infrared camera, and the projector is an infrared speckle projector.

[0059] Infrared camera: A camera that captures infrared light and uses it to create images.

[0060] Infrared speckle projector: A device that projects speckles in the form of infrared light.

[0061] Speckle: A random pattern projected by an infrared speckle projector; an infrared camera calculates distance by the offset of the speckle shape.

[0062] Normalization: Mapping sequences of values ​​with different ranges to the interval (0, 1) to facilitate data processing. In some cases, normalized values ​​can be directly expressed as probabilities.

[0063] Dropout is a method for optimizing deep artificial neural networks. During the learning process, it reduces the interdependence between nodes by randomly setting some weights or outputs of the hidden layers to zero, thereby regularizing the neural network and reducing its structural risks. For example, in model training, given a vector (1, 2, 3, 4), after inputting this vector into a dropout layer, the dropout layer can randomly convert one of the numbers in the vector (1, 2, 3, 4) to 0. For example, converting 2 to 0 would change the vector to (1, 0, 3, 4).

[0064] Embedded coding, mathematically speaking, represents a correspondence, that is, mapping data in space X to space Y using a function F. This function F is injective, and the mapping result preserves the structure. An injective function means that the mapped data uniquely corresponds to the original data, and preserving the structure means that the order of the original data remains the same. For example, if there are data X1 and X2 before mapping, after mapping we get Y1 corresponding to X1 and Y2 corresponding to X2. If the original data X1 > X2, then correspondingly, the mapped data Y1 > Y2. For words, this means mapping words to another space to facilitate subsequent machine learning and processing.

[0065] Attention weights represent the importance of a piece of data during training or prediction. Importance indicates the magnitude of the influence of input data on output data. Data with high importance corresponds to higher attention weights, while data with low importance corresponds to lower attention weights. The importance of data varies in different scenarios, and training the model to assign attention weights is essentially the process of determining data importance.

[0066] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the infrared face images and color face images involved in the embodiments of this specification were obtained under full authorization.

[0067] The implementation environment of the technical solutions provided in the embodiments of this specification will be described below.

[0068] Figure 1 This is a schematic diagram illustrating the implementation environment of a face recognition method provided in the embodiments of this specification. See also... Figure 1 The implementation environment may include terminal 110 and server 140.

[0069] Terminal 110 is connected to server 140 via a wireless or wired network. Optionally, terminal 110 may be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. Terminal 110 has an application that supports facial recognition installed and runs. Terminal 110 includes a structured light system, which includes an infrared speckle projector and an infrared camera. In some embodiments, the structured light system also includes a visible light camera.

[0070] Server 140 is a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. Server 140 provides background services for applications running on terminal 110.

[0071] Those skilled in the art will understand that the number of terminals described above can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more, in which case other terminals may also be included in the above implementation environment. This specification does not limit the number of terminals or the type of device in the embodiments.

[0072] After introducing the implementation environment of the embodiments of this specification, the application scenarios of the embodiments of this specification will be described below in conjunction with the above implementation environment. In the following description, the terminal is also the terminal 110 in the above implementation environment, and the server is also the server 120 in the above implementation environment. The technical solutions provided by the embodiments of this specification can be applied in various scenarios of applying face recognition systems, such as terminals that provide face unlocking functions, or payment applications that provide face payment functions, or payment devices that provide face payment functions, or vending machines with face payment functions, or access control devices with face recognition. The embodiments of this specification do not limit this application.

[0073] Taking the face recognition method provided in this embodiment of the specification applied to a terminal providing face unlock functionality as an example, the terminal includes a structured light system, which includes an infrared speckle projector and an infrared camera. The terminal implements face recognition functions through this structured light system. During face unlocking, the terminal turns off the infrared speckle projector in the structured light system. With the infrared speckle projector off, the terminal acquires a first infrared face image of the target object through the infrared camera in the structured light system. Based on the first infrared face image, the terminal determines the environmental state of the terminal. The terminal activates the infrared speckle projector and adjusts the operating parameters of the structured light system based on the environmental state to match the operating parameters with the environmental state. Based on the adjusted operating parameters of the structured light system, the terminal performs face recognition on the target object. The terminal determines whether to unlock based on the face recognition result.

[0074] In various payment applications that provide facial recognition payment functionality, after the terminal adjusts the working parameters of the structured light system based on the environmental conditions, it can use the structured light system with the adjusted working parameters to collect the facial information of the target object and send the facial information to the server for facial recognition, thereby determining whether to make payment based on the result of facial recognition.

[0075] In other scenarios, the face recognition method provided in the embodiments of this specification belongs to the same inventive concept as described above, and will not be repeated here.

[0076] After introducing the implementation environment and application scenarios of the embodiments in this specification, the face recognition method provided in the embodiments of this specification will be described below. See [link to documentation]. Figure 2 Taking the terminal as the executing entity as an example, the method includes the following steps.

[0077] 202. With the infrared speckle projector off, the terminal acquires a first infrared face image of the target object, which is captured by an infrared camera.

[0078] The terminal includes a structured light system comprising an infrared speckle projector and an infrared (IR) camera. The structured light system is used for face recognition. The infrared speckle projector projects speckles in the form of infrared light, and the infrared camera collects the infrared light and creates an image based on it. During face recognition using the structured light system, the infrared speckle projector projects speckles onto the target object's face, and the infrared camera captures an image of the target object's face. The resulting infrared face image carries the speckles of the target object's face. The shape of the speckles allows for the determination of the three-dimensional coordinates of multiple points on the target object's face, achieving three-dimensional reconstruction of the target object's face. Face recognition is then performed based on the result of the three-dimensional reconstruction. In some embodiments, when the infrared speckle projector is on, the infrared face image captured by the infrared camera carries speckle representations of three-dimensional information, and is therefore also referred to as a three-dimensional infrared image. When the infrared speckle projector is off, the infrared face image captured by the infrared camera does not carry speckles, and is therefore also referred to as a two-dimensional infrared image.

[0079] 204. The terminal determines the environmental state of the terminal based on the first infrared face image.

[0080] Among them, the environmental state can represent the state of the environment in which the terminal is located. Different environmental states will have different degrees of impact on the face recognition process based on the structured light system. Accordingly, since the first infrared face image is used to determine the environmental state, the first infrared face image can also be called the environmental state determination image.

[0081] 206. The terminal starts the infrared speckle projector and adjusts the operating parameters of the structured light system based on the environmental conditions.

[0082] Adjusting the operating parameters of the structured light system includes adjusting at least one of the operating parameters of the infrared speckle projector and the infrared camera. Adjusting the operating parameters of the structured light system enables it to operate in a state that matches the environmental conditions, thereby improving the imaging effect of the infrared camera and ultimately increasing the success rate and accuracy of face recognition.

[0083] 208. The terminal performs facial recognition on the target object based on the structured light system after adjusting the working parameters.

[0084] Among them, performing face recognition on the target object based on the structured light system with adjusted working parameters means acquiring the face image of the target object based on the structured light system with adjusted working parameters, and performing face recognition based on the information provided by the face image.

[0085] The technical solution provided in the embodiments of this specification allows for the acquisition of a first infrared face image by an infrared camera within the structured light system when the infrared speckle projector in the structured light system is turned off. This first infrared face image is used to determine the environmental state of the terminal. After obtaining the environmental state, the infrared speckle projector is activated, and the operating parameters of the structured light system are adjusted based on the environmental state, enabling the structured light system to operate in a manner matching the environmental state. Based on the adjusted operating parameters, the structured light system performs face recognition on the target object, thereby improving the success rate and accuracy of face recognition under different environmental conditions.

[0086] Steps 202-208 above are a simplified description of the face recognition method provided in the embodiments of this specification. The face recognition method provided in the embodiments of this specification will be explained more clearly below with some examples. See [link to documentation]. Figure 3 Taking the terminal as the executing entity as an example, the method includes the following steps.

[0087] 302. In response to the face recognition command, the terminal controls the infrared speckle projector to turn off.

[0088] The terminal includes a structured light system, which comprises an infrared speckle projector and an infrared (IR) camera. In some embodiments, the structured light system is mounted on the front of the terminal, and during user use, the structured light system can capture images of the user's face, for example, see [link to example]. Figure 4The structured light system 401 is mounted on the top of the front of the terminal 400. The structured light system 401 includes an infrared speckle projector 402 and an infrared camera 403. In some embodiments, the structured light system also includes a visible light camera 404. The structured light system is used for face recognition. The infrared speckle projector projects speckles in the form of infrared light, and the infrared camera collects infrared light and forms an image based on it. During face recognition based on the structured light system, the infrared speckle projector projects speckles onto the target object's face, and the infrared camera captures an image of the target object's face. The resulting infrared face image carries the speckles of the target object's face. The shape of the speckles allows for the determination of the three-dimensional coordinates of multiple points on the target object's face, achieving three-dimensional reconstruction of the target object's face. Face recognition is then performed based on the result of the three-dimensional reconstruction. The face recognition command instructs the terminal to perform face recognition.

[0089] In different scenarios, the facial recognition command is triggered in different ways, which will be explained in the following sections for different scenarios.

[0090] In the scenario of face unlock, the face recognition command is triggered by the unlock operation, which includes a swipe operation performed on the terminal in the locked state, clicking the unlock control displayed on the terminal in the locked state, and adjusting the angle between the terminal and the horizontal plane to the target angle, etc. The target angle is set by the technician according to the actual situation, and this specification embodiment does not limit it.

[0091] In the context of facial recognition payment, the facial recognition instruction is triggered by a payment operation, which includes clicking on the payment controls displayed on the terminal.

[0092] In a facial recognition attendance scenario, the facial recognition command is triggered by a facial recognition operation, which includes clicking on the facial recognition control displayed on the terminal. Alternatively, the facial recognition command may be automatically triggered by the terminal upon detecting a face.

[0093] It should be noted that, in addition to the above scenarios, the face recognition command can be triggered in other ways, and the embodiments in this specification do not limit this.

[0094] The implementation process of step 302 above will be explained below.

[0095] In one possible implementation, in response to a face recognition command, the terminal sends a shutdown command to the infrared speckle projector, which controls the infrared speckle projector to shut down. In response to the shutdown command, if the infrared speckle projector is on, it shuts down; if the infrared speckle projector is off, it remains off.

[0096] The purpose of shutting down the infrared speckle projector is to eliminate its influence on subsequent environmental condition determination and improve the accuracy of such determination.

[0097] 304. With the infrared speckle projector off, the terminal acquires a first infrared face image of the target object, which is captured by an infrared camera.

[0098] When the infrared speckle projector is on, the infrared face image captured by the infrared camera carries speckle representations of three-dimensional information, and is therefore also called a three-dimensional infrared image. When the infrared speckle projector is off, the infrared face image captured by the infrared camera does not carry speckle, and is therefore also called a two-dimensional infrared image. The speckle projected by the infrared speckle projector is a randomly generated pattern, and there are multiple speckles. The more speckles, the richer the three-dimensional information obtained.

[0099] The first infrared face image is used to determine the environmental state of the terminal. In some embodiments, the number of the first infrared face images is one or more, and this specification does not limit this. When the number of the first infrared face images is one, the terminal can subsequently determine the environmental state of the terminal based on one first infrared face image; when the number of the first infrared face images is multiple, the terminal can subsequently determine the environmental state of the terminal based on multiple first infrared face images.

[0100] In some embodiments, after step 302, the terminal may perform the following steps in addition to performing step 304.

[0101] In one possible implementation, the structured light system also includes a visible light camera, which, when the infrared speckle projector is off, allows the terminal to acquire a color face image of the target object, captured by the visible light camera.

[0102] The visible light camera is used to capture visible light and create images based on it; specifically, visible light refers to human-readable light. The color facial image is used to determine the environment in which the terminal is located.

[0103] In this implementation, in addition to capturing the first infrared face image through an infrared camera, the terminal can also capture the color face image through a visible light camera, thus providing a wider range of ways to determine the environmental state of the terminal.

[0104] 306. The terminal determines the environmental state of the terminal based on the first infrared face image.

[0105] Among them, the environmental state can represent the state of the environment in which the terminal is located. Different environmental states will have different degrees of impact on the face recognition process based on the structured light system. Accordingly, since the first infrared face image is used to determine the environmental state, the first infrared face image can also be called the environmental state determination image.

[0106] In one possible implementation, if the brightness value of the first infrared face image is greater than or equal to a brightness threshold, the terminal determines the environmental state in which the terminal is located as a first environmental state, which indicates that there is a light source in the environment in which the terminal is located that meets the target lighting conditions.

[0107] In this context, a light source meeting the target conditions refers to a light source with a luminous intensity greater than or equal to a luminous intensity threshold, meaning the light source has a high luminous intensity. This luminous intensity threshold is set by technicians based on actual conditions, and this specification does not limit this setting in the embodiments. This light source meeting the target illumination conditions may affect the face recognition process based on the structured light system. In some embodiments, the first environmental state is also referred to as an outdoor state or an indoor high-brightness state. The outdoor state indicates that the terminal is outdoors, and the light source meeting the target illumination conditions is the sun; the indoor high-brightness state indicates that the terminal is indoors, and there is a lighting device indoors with a luminous intensity greater than or equal to the luminous intensity threshold, and this lighting device is an infrared lighting device.

[0108] In this situation, the terminal can determine the environmental state of the terminal based on the brightness value of the first infrared face image, and the determination of the environmental state is highly efficient.

[0109] In some embodiments, the brightness value of the first infrared face image refers to the average brightness value of multiple pixels in the first infrared face image, or the average brightness value of multiple pixels in the face region of the first infrared face image. The above embodiments will be described below using these two cases as examples.

[0110] Taking the brightness value of the first infrared face image as the average brightness value of multiple pixels in the first infrared face image as an example, the terminal adds up the pixel values ​​of multiple pixels in the first infrared face image and divides the sum by the number of pixels to obtain the average brightness value of multiple pixels in the first infrared face image. The terminal compares the average brightness value with a brightness threshold. If the brightness value of the first infrared face image is greater than or equal to the brightness threshold, the terminal determines that the environmental state of the terminal is the first environmental state.

[0111] Taking the brightness value of the first infrared face image as the average brightness value of multiple pixels in the face region of the first infrared face image as an example, the terminal performs face detection on the first infrared face image to obtain the face region in the first infrared face image. The terminal adds up the pixel values ​​of multiple pixels in the face region and divides the sum by the number of pixels to obtain the average brightness value of multiple pixels in the face region. The terminal compares the average brightness value with a brightness threshold. If the brightness value of the first infrared face image is greater than or equal to the brightness threshold, the terminal determines that the environmental state of the terminal is the first environmental state.

[0112] The above explanation is based on the example of having only one infrared face image. When there are multiple infrared face images, the terminal can perform the following steps.

[0113] In one possible implementation, if the brightness value of the first infrared face image of the first target number in the plurality of first infrared face images is greater than or equal to a brightness threshold, the terminal determines the environmental state in which the terminal is located as a first environmental state, which indicates that there is a light source in the environment in which the terminal is located that meets the target lighting conditions.

[0114] The plurality of first infrared face images are acquired by the infrared camera at preset time intervals. Both the preset time interval and the number of first targets are set by technicians according to actual conditions, and this specification does not limit this. The method for determining the relationship between the brightness values ​​of the plurality of first infrared face images and a brightness value threshold belongs to the same inventive concept as the above-described embodiments. The implementation process is described in the relevant descriptions of the above embodiments and will not be repeated here.

[0115] In one possible implementation, if the brightness value of the first infrared face image is less than the brightness threshold, the terminal determines that the environmental state in which the terminal is located is a third environmental state, which indicates that there is no light source in the environment in which the terminal is located that meets the target lighting conditions.

[0116] In some embodiments, this third environmental state is also referred to as the normal lighting environment state.

[0117] In one possible implementation, if a speckle pattern is present in the first infrared face image, the terminal determines the environmental state in which the terminal is located as a second environmental state, which indicates that there is interference from other infrared speckle projectors in the environment in which the terminal is located.

[0118] Since the terminal's infrared speckle projector is turned off, the presence of speckle in the first infrared face image indicates the presence of other infrared speckle projectors near the terminal. The speckle projected by these other infrared speckle projectors constitutes interference. Accordingly, this second environmental state is also referred to as the interference state.

[0119] In this implementation, the terminal can determine whether there is interference based on speckle in the first infrared face image, and thus determine the environmental state in which the terminal is located, with high efficiency in determining the environmental state.

[0120] For example, the terminal performs speckle detection on the first infrared face image to determine whether speckle exists in the image. If a speckle pattern is found in the first infrared face image, the terminal determines its current environmental state as a second environmental state.

[0121] In this specification, speckle detection of the first infrared face image is performed using either a target detection method or a pattern matching method, and the embodiments therein are not limited to this method.

[0122] For example, the terminal inputs the first infrared face image into a speckle detection model, and uses the speckle detection model to perform speckle detection on the first infrared face image to determine whether speckle exists in the first infrared face image. The speckle detection model can be a target detection model of any structure, and this embodiment does not limit this. Alternatively, the terminal slides a preset pattern across the first infrared face image to determine whether there is a pattern on the first infrared face image whose similarity to the preset pattern is greater than or equal to a similarity threshold. If a pattern with similarity to the preset pattern greater than or equal to the similarity threshold exists on the first infrared face image, it indicates that speckle exists in the first infrared face image. If a speckle pattern exists in the first infrared face image, the terminal determines its current environmental state as a second environmental state.

[0123] The above explanation is based on the example of having only one infrared face image. When there are multiple infrared face images, the terminal can perform the following steps.

[0124] In one possible implementation, if a speckle pattern exists in the first infrared face images of the second target number among the plurality of first infrared face images, the terminal determines the environmental state in which the terminal is located as a second environmental state, which indicates that there is interference from other infrared speckle projectors in the environment in which the terminal is located.

[0125] The plurality of first infrared face images are acquired by the infrared camera at preset time intervals. Both the preset time interval and the number of second targets are set by technicians according to actual conditions, and this specification does not limit this. The method for determining whether speckle exists in the plurality of first infrared face images belongs to the same inventive concept as the above-described embodiments, and the implementation process is described in the relevant descriptions of the above embodiments, and will not be repeated here.

[0126] In one possible implementation, if there is no speckle pattern in the first infrared face image, the terminal determines that the environmental state in which the terminal is located is a fourth environmental state, which indicates that there is no interference from other infrared speckle projectors in the environment in which the terminal is located.

[0127] In some embodiments, this fourth environmental state is also referred to as an interference-free environmental state.

[0128] It should be noted that the above embodiments are all described using the brightness value or speckle of the first infrared face image as an example to determine the environmental state. In other possible embodiments, the terminal can comprehensively determine the environmental state by combining the brightness value and speckle, as follows.

[0129] In one possible implementation, if no speckle pattern is found in the first infrared face image, the terminal determines the brightness value of the first infrared face image. If the brightness value of the first infrared face image is greater than or equal to a brightness threshold, the terminal determines the environmental state in which the terminal is located as the first environmental state.

[0130] In one possible implementation, if the brightness value of the first infrared face image is less than a brightness threshold, the terminal determines whether speckle patterns exist in the first infrared face image. If speckle patterns exist in the first infrared face image, the terminal determines the environmental state in which it is located as the second environmental state.

[0131] 308. The terminal starts the infrared speckle projector.

[0132] In one possible implementation, the terminal sends an activation command to the infrared speckle projector, which controls the infrared speckle projector to turn on. In response to the activation command, the infrared speckle projector turns on.

[0133] The purpose of turning on the infrared speckle projector is to project speckles onto the face of the target object so that the infrared camera can collect the speckles for facial recognition.

[0134] Optionally, after step 306, in addition to performing step 308 as described above, the terminal can also perform any of the following steps.

[0135] In one possible implementation, when the terminal is in the first environmental state, the terminal performs at least one of the following: increases the screen brightness of the terminal, increases the contrast of the current display interface of the terminal, and increases the volume of the terminal.

[0136] The first environmental state indicates that the environment in which the terminal is located contains a light source with high luminous intensity. Increasing the terminal's screen brightness helps users see the content displayed on the screen more clearly, improving the user interaction experience. Increasing the contrast of the terminal's current display interface also helps users see the content displayed on the interface more clearly, improving the user experience. Increasing the terminal's volume helps users listen to audio played by the terminal when it is outdoors.

[0137] In one possible implementation, when the terminal is in the second environmental state, the terminal performs at least one of the following: displaying a prompt message and playing a prompt voice, both of which are used to prompt the change of the terminal's location.

[0138] The second environmental state indicates that there is interference in the environment in which the terminal is located. The user is prompted to adjust the position of the terminal to eliminate the interference by displaying a prompt message or playing a prompt voice.

[0139] 310. The terminal adjusts the operating parameters of the structured light system based on the environmental conditions.

[0140] Adjusting the operating parameters of the structured light system includes adjusting at least one of the operating parameters of the infrared speckle projector and the infrared camera. Adjusting the operating parameters of the structured light system enables it to operate in a state that matches the environmental conditions, thereby improving the imaging effect of the infrared camera and ultimately increasing the success rate and accuracy of face recognition.

[0141] In one possible implementation, when the terminal is in the first environmental state, the terminal performs at least one of the following: reducing the exposure time of the infrared camera and increasing the signal-to-noise ratio of the infrared speckle projector.

[0142] The first environmental state indicates that the environment in which the terminal is located contains a light source with high luminous intensity. Adjusting the operating parameters of the structured light system is to reduce the impact of this light source on face recognition. Reducing the exposure time of the infrared camera can reduce the exposure of the infrared camera, thereby improving the imaging effect of the infrared camera. Increasing the signal-to-noise ratio of the infrared speckle projector can improve the analog gain of the infrared speckle projector, thereby improving the quality of the speckle. Improving the signal-to-noise ratio can be achieved by increasing the emission power of the infrared speckle projector (e.g., increasing the current value).

[0143] In one possible implementation, when the terminal is in the second environmental state, the terminal performs at least one of the following: adjusting the operating time of the infrared speckle projector, adjusting the operating frequency of the infrared speckle projector, increasing the signal-to-noise ratio of the infrared speckle projector, and reducing the exposure time of the infrared camera.

[0144] The second environmental state indicates that interference exists in the environment where the terminal is located. Adjusting the operating parameters of the structured light system is to reduce the impact of this interference on face recognition. Adjusting the operating time of the infrared speckle projector can change the pattern of speckle projection, thereby reducing the probability of interference. In some embodiments, adjusting the operating time of the infrared speckle projector includes advancing or delaying the time when the infrared speckle projector projects the speckle. Adjusting the operating frequency of the infrared speckle projector can also change the pattern of speckle projection, thereby reducing the probability of interference. In some embodiments, adjusting the operating frequency of the infrared speckle projector includes increasing or decreasing the frequency at which the infrared speckle projector projects the speckle.

[0145] The above description uses an infrared speckle projector and an infrared camera as an example of a structured light system. In some embodiments, the structured light system also includes a visible light camera. Accordingly, the terminal adjusts the operating parameters of the structured light system in the following manner.

[0146] In one possible implementation, when the terminal is in the first environmental state, the terminal performs at least one of the following: reducing the exposure time of the infrared camera, increasing the signal-to-noise ratio of the infrared speckle projector, and adjusting the metering area of ​​the visible light camera to the face area of ​​the target object.

[0147] Metering refers to measuring the brightness of light reflected back from the subject, also known as reflective metering. Metering area is a concept in local metering; adjusting the metering area to the face area of ​​the target object allows for metering based on that face area, making the metering results more consistent with the initial environmental conditions. This improves the quality of the image captured by the visible light camera, thereby increasing the success rate and accuracy of face recognition.

[0148] In one possible implementation, when the terminal is in the second environmental state, the terminal performs at least one of the following: adjusting the operating time of the infrared speckle projector, increasing the signal-to-noise ratio of the infrared speckle projector, and reducing the exposure time of the infrared camera.

[0149] 312. The terminal performs facial recognition on the target object based on the structured light system after adjusting the working parameters.

[0150] Specifically, performing face recognition on the target object based on the structured light system with adjusted operating parameters means acquiring a face image of the target object using the structured light system with adjusted operating parameters, and performing face recognition based on the information provided by the face image. In some embodiments, the face image includes at least one of an infrared face image and a color face image.

[0151] In one possible implementation, the terminal acquires a second infrared face image captured by the infrared camera, which is acquired after adjusting the operating parameters of the structured light system. Based on this second infrared face image, the terminal performs face recognition on the target object.

[0152] In this implementation, the terminal can acquire a second infrared face image through the infrared camera and perform face recognition on the target object based on the second infrared face image. Since the second infrared face image is acquired after the working parameters of the structured light system are adjusted, the second infrared face image is more adapted to the environmental conditions of the terminal, and the success rate and accuracy of face recognition based on the second infrared face image are high.

[0153] For example, the terminal acquires a second infrared face image captured by the infrared camera. Based on this second infrared face image, the terminal determines the three-dimensional face structure of the target object. The terminal extracts features from this three-dimensional face structure to obtain the three-dimensional face features of the target object. Based on these three-dimensional face features, the terminal performs face recognition on the target object.

[0154] For example, the terminal acquires a second infrared face image captured by the infrared camera. Based on the shape of speckle in the second infrared face image, the terminal determines the three-dimensional face structure of the target object, which includes the three-dimensional coordinates of multiple key points on the target object's face. The terminal inputs this three-dimensional structural feature into a three-dimensional feature extraction model, and extracts features from the three-dimensional face structure through the model to obtain the three-dimensional face features of the target object. The three-dimensional feature extraction model can employ three-dimensional convolution, fully connected layers, or three-dimensional attention encoding for feature extraction; this embodiment does not limit the specific method used. The terminal determines the feature similarity between the three-dimensional face features and the standard three-dimensional face features of the target object. The standard three-dimensional face features are obtained based on infrared face images of the target object acquired in a normal environment. A normal environment refers to an environment without light sources that meet the target conditions and without interference. If the feature similarity is greater than or equal to a similarity threshold, the face recognition of the target object is deemed successful. If the feature similarity is less than the similarity threshold, the face recognition of the target object is deemed unsuccessful. The similarity threshold is set by technicians according to the actual situation, and the embodiments in this specification do not limit it.

[0155] In some embodiments, before determining the feature similarity between the 3D facial feature and the standard 3D facial feature of the target object, the terminal can also perform liveness attack detection on the target object based on the 3D facial feature. Liveness attack detection is used to determine whether the current face recognition is a liveness attack. For example, the terminal inputs the 3D facial feature into a liveness attack detection model, performs a fully connected and normalized operation on the 3D facial feature using the liveness attack detection, and obtains a liveness attack classification value corresponding to the 3D facial feature. If the liveness attack classification value is greater than or equal to a classification value threshold, the current face recognition is determined to be a liveness attack; if the liveness attack classification value is less than the classification value threshold, the current face recognition is determined not to be a liveness attack. The classification value threshold is set by a technician according to the actual situation, and this specification does not limit this. If the current face recognition is not a liveness attack, the terminal performs the subsequent step of determining the feature similarity between the 3D facial feature and the standard 3D facial feature of the target object; if the current face recognition is a liveness attack, the terminal terminates the face recognition process and directly determines that the face recognition has failed.

[0156] In one possible implementation, the structured light system further includes a visible light camera. The terminal acquires a second infrared face image captured by the infrared camera and a first color face image captured by the visible light camera. Both the second infrared face image and the first color face image are acquired after the operating parameters of the structured light system have been adjusted. Based on the second infrared face image and the first color face image, the terminal performs face recognition on the target object.

[0157] The visible light camera is used to collect visible light and perform imaging based on visible light. The color face images collected by the visible light camera can assist in the face recognition process.

[0158] In this implementation, the terminal can combine infrared and color face images for face recognition, resulting in a high accuracy rate.

[0159] The technical solutions provided in the examples above will be further illustrated with a few more examples.

[0160] Example 1: The structured light system also includes a visible light camera. The terminal acquires a second infrared face image captured by the infrared camera and a first color face image captured by the visible light camera. The terminal determines the quality score of the second infrared face image. If the quality score of the second infrared face image is less than or equal to a first quality score threshold, the terminal controls the infrared speckle projector to turn off. The terminal acquires a third infrared face image captured by the infrared camera. Based on the third infrared face image and the first color face image, the terminal performs face recognition on the target object.

[0161] The quality score of the second infrared face image is positively correlated with its overall quality; a higher quality score indicates higher quality, and a lower quality score indicates lower quality. The first quality score threshold is set by a technician based on actual conditions, and this specification does not limit this setting. In some embodiments, the quality of the second infrared face image refers to the quality of speckle in the image; for example, the speckle quality is represented by a speckle quality score. The difference between the second and third infrared face images is that the second image contains speckle, while the third image does not. Accordingly, the second infrared face image is also referred to as a 3D infrared face image, and the third infrared face image is also referred to as a 2D infrared face image.

[0162] In this implementation, before performing face recognition based on the second infrared face image, the quality score of the second infrared face image is first determined. If the quality score of the second infrared face image is low, that is, if the quality of the second infrared face image is poor, the terminal controls the infrared speckle projector to turn off and acquires a third infrared face image through an infrared camera. Finally, face recognition of the target object is achieved based on the third infrared face image and the first color face image.

[0163] For example, the structured light system also includes a visible light camera. The terminal acquires a second infrared face image captured by the infrared camera and a first color face image captured by the visible light camera. The terminal uses an Image Quality Assessment (IQA) algorithm to determine the quality score of the second infrared face image. This image quality assessment algorithm includes subjective evaluation and objective evaluation. This embodiment of the specification does not limit the type of image quality assessment algorithm. If the quality score of the second infrared face image is less than or equal to a first quality score threshold, the terminal controls the infrared speckle projector to turn off. The terminal acquires a third infrared face image captured by the infrared camera. The terminal inputs the third infrared face image and the first color face image into a two-dimensional feature extraction model. The two-dimensional feature extraction model performs feature extraction on the third infrared face image and the first color face image to obtain the first infrared face image features and the first color face image features. The terminal determines a first feature similarity between the first infrared face image features and the standard infrared face image features of the target object, and a second feature similarity between the first color face image features and the standard color face image features of the target object. If both the first feature similarity and the second feature similarity are greater than or equal to the similarity threshold, the face recognition of the target object is determined to be successful. If either the first feature similarity or the second feature similarity is less than the similarity threshold, the face recognition of the target object is determined to be unsuccessful. The standard infrared face image features and standard color face image features are both image features obtained based on infrared and color face images acquired in a normal environment.

[0164] Alternatively, after the terminal determines the first feature similarity and the second feature similarity, the terminal can also perform a weighted sum of the first feature similarity and the second feature similarity to obtain a first fused similarity. If the first fused similarity is greater than or equal to a similarity threshold, the face recognition of the target object is determined to have passed. If the first fused similarity is less than the similarity threshold, the face recognition of the target object is determined to have failed.

[0165] Example 2: The structured light system also includes a visible light camera. The terminal acquires a second infrared face image captured by the infrared camera and a first color face image captured by the visible light camera. The terminal determines the quality score of the second infrared face image. If the quality score of the second infrared face image is less than or equal to a first quality score threshold, the terminal reduces the weight of the second infrared face image in the face recognition process. This weight is positively correlated with the degree of influence on the face recognition result. Based on the second infrared face image and the first color face image, the terminal performs face recognition on the target object.

[0166] In this implementation, before performing face recognition based on the second infrared face image, the quality score of the second infrared face image is determined. If the quality score of the second infrared face image is low, that is, if the quality of the second infrared face image is poor, the terminal can reduce the weight of the second infrared face image in the face recognition process, that is, reduce the influence of the second infrared face image on the face recognition result, thereby improving the success rate of face recognition.

[0167] For example, the structured light system also includes a visible light camera. The terminal acquires a second infrared face image captured by the infrared camera and a first color face image captured by the visible light camera. The terminal uses an image quality evaluation algorithm to determine the quality score of the second infrared face image. If the quality score of the second infrared face image is less than or equal to a first quality score threshold, the terminal reduces the weight of the second infrared face image in the face recognition process to a target weight, which is positively correlated with the quality score of the second infrared face image. Based on the shape of speckle in the second infrared face image, the terminal determines the three-dimensional face structure of the target object, which includes the three-dimensional coordinates of multiple key points on the target object's face. The terminal inputs the three-dimensional face structure into a three-dimensional feature extraction model and extracts features from the three-dimensional face structure to obtain three-dimensional face features. The terminal inputs the first color face image into a two-dimensional feature extraction model and extracts features from the first color face image to obtain first color face image features, which are also two-dimensional face image features. The terminal determines a second feature similarity between the 3D facial feature and the standard 3D facial feature of the target object, and a third feature similarity between the second infrared facial image feature and the standard infrared facial image feature of the target object. Based on the target weight, the terminal performs a weighted fusion of the second and third feature similarities to obtain a second fused similarity. If the second fused similarity is greater than or equal to a similarity threshold, the facial recognition of the target object is deemed successful. If the second fused similarity is less than the similarity threshold, the facial recognition of the target object is deemed unsuccessful.

[0168] In some embodiments, before determining the second feature similarity between the 3D facial feature and the standard 3D facial feature of the target object, the terminal can also perform liveness attack detection on the target object based on the 3D facial feature. Liveness attack detection is used to determine whether the current face recognition is a liveness attack. For example, the terminal inputs the 3D facial feature into a liveness attack detection model, performs a fully connected and normalized operation on the 3D facial feature using the liveness attack detection, and obtains the liveness attack classification value corresponding to the 3D facial feature. If the liveness attack classification value is greater than or equal to the target classification value threshold, the current face recognition is determined to be a liveness attack; if the liveness attack classification value is less than the target classification value threshold, the current face recognition is determined not to be a liveness attack. The target classification value threshold is obtained by reducing a preset classification value threshold based on the target weight, that is, reducing the rejection threshold for liveness attack detection. If the face recognition is not a liveness attack, the terminal will proceed to determine the second feature similarity between the 3D face feature and the standard 3D face feature of the target object; if the face recognition is a liveness attack, the terminal will terminate the face recognition process and directly determine that the face recognition has failed.

[0169] Example 3: The structured light system also includes a visible light camera. The terminal acquires a second infrared face image captured by the infrared camera and a first color face image captured by the visible light camera. The terminal determines the quality score of the second infrared face image. If the quality score of the second infrared face image is greater than a first quality score threshold, the terminal determines the three-dimensional face structure of the target object based on the second infrared face image. The terminal performs feature extraction on the three-dimensional face structure and the first color face image respectively to obtain the three-dimensional face features and two-dimensional face features of the target object. The terminal performs face recognition on the target object based on the three-dimensional face features and the two-dimensional face features.

[0170] For example, the structured light system also includes a visible light camera. The terminal acquires a second infrared face image captured by the infrared camera and a first color face image captured by the visible light camera. The terminal uses an image quality evaluation algorithm to determine the quality score of the second infrared face image. If the quality score of the second infrared face image is greater than a first quality score threshold, the terminal determines the three-dimensional face structure of the target object based on the shape of speckle in the second infrared face image. This three-dimensional face structure includes the three-dimensional coordinates of multiple key points on the target object's face. The terminal inputs this three-dimensional face structure into a three-dimensional feature extraction model and extracts features from it to obtain three-dimensional face features. The terminal inputs the first color face image into a two-dimensional feature extraction model and extracts features from it to obtain first color face image features, which are also two-dimensional face image features. The terminal determines the second feature similarity between the three-dimensional face features and the standard three-dimensional face features of the target object, and the third feature similarity between the first color face image features and the standard color face image features of the target object. If both the second and third feature similarities are greater than or equal to the similarity threshold, the face recognition of the target object is deemed successful. If either the second or third feature similarity is less than the similarity threshold, the face recognition of the target object is deemed unsuccessful. The standard 3D face features and standard color face image features are both image features obtained from infrared and color face images acquired in a normal environment.

[0171] Alternatively, after the terminal determines the second feature similarity and the third feature similarity, the terminal performs a weighted fusion of the second feature similarity and the third feature similarity to obtain a third fused similarity. If the third fused similarity is greater than or equal to a similarity threshold, the face recognition of the target object is determined to have passed. If the third fused similarity is less than the similarity threshold, the face recognition of the target object is determined to have failed.

[0172] Example 4: The structured light system also includes a visible light camera. The terminal acquires a second infrared face image captured by the infrared camera and a first color face image captured by the visible light camera. The terminal determines the quality score of the first color face image. If the quality score of the first color face image is less than or equal to a second quality score threshold, the terminal performs image enhancement on the first color face image to obtain a second color face image. Based on the second infrared face image and the second color face image, the terminal performs face recognition on the target object.

[0173] The purpose of image enhancement on the first color face image is to improve its quality score, thereby increasing the success rate and accuracy of face recognition.

[0174] For example, the structured light system also includes a visible light camera. The terminal acquires a second infrared face image captured by the infrared camera and a first color face image captured by the visible light camera. The terminal uses an Image Quality Assessment (IQA) algorithm to determine the quality score of the first color face image. This image quality assessment algorithm includes subjective evaluation and objective evaluation. This embodiment of the specification does not limit the type of image quality assessment algorithm. If the quality score of the first color face image is less than or equal to a second quality score threshold, the terminal uses an image enhancement algorithm to enhance the first color face image to improve its quality, resulting in a second color face image. The image enhancement algorithm can be any type of algorithm, such as HDR (High-dynamic Range Rendering) or super-resolution algorithms. This embodiment of the specification does not limit this type. Based on the shape of the speckle in the second infrared face image, the terminal determines the three-dimensional face structure of the target object. This three-dimensional face structure includes the three-dimensional coordinates of multiple key points on the target object's face. The terminal inputs the 3D face structure into a 3D feature extraction model, which extracts features from the 3D face structure to obtain 3D face features. The terminal then inputs the second color face image into a 2D feature extraction model, which extracts features from the second color face image to obtain second color face image features, which are also 2D face image features. The terminal determines the second feature similarity between the 3D face features and the standard 3D face features of the target object, and the fourth feature similarity between the second color face image features and the standard color face image features of the target object. If both the second and fourth feature similarities are greater than or equal to a similarity threshold, the face recognition of the target object is deemed successful. If either the second or fourth feature similarity is less than the similarity threshold, the face recognition of the target object is deemed unsuccessful.

[0175] Alternatively, after the terminal determines the second feature similarity and the fourth feature similarity, the terminal performs a weighted fusion of the second feature similarity and the fourth feature similarity to obtain a fourth fused similarity. If the fourth fused similarity is greater than or equal to a similarity threshold, the face recognition of the target object is determined to have passed. If the fourth fused similarity is less than the similarity threshold, the face recognition of the target object is determined to have failed.

[0176] It should be noted that step 312 above is an example of face recognition performed by the terminal based on image information collected by the structured light system. In other possible implementations, after the structured light system collects image information, the image information can be sent to the server, and the server can perform face recognition based on the image. This specification does not limit this embodiment.

[0177] All the above-mentioned optional technical solutions can be combined in any way to form the optional embodiments of this specification, and will not be described in detail here.

[0178] The technical solution provided in the embodiments of this specification allows for the acquisition of a first infrared face image by an infrared camera within the structured light system when the infrared speckle projector in the structured light system is turned off. This first infrared face image is used to determine the environmental state of the terminal. After obtaining the environmental state, the infrared speckle projector is activated, and the operating parameters of the structured light system are adjusted based on the environmental state, enabling the structured light system to operate in a manner matching the environmental state. Based on the adjusted operating parameters, the structured light system performs face recognition on the target object, thereby improving the success rate and accuracy of face recognition under different environmental conditions.

[0179] Figure 5 This is a schematic diagram of a face recognition device provided in an embodiment of this specification, applied to a terminal. The terminal includes a structured light system, which includes an infrared speckle projector and an infrared camera. See [link to documentation]. Figure 5 The device includes: an image acquisition module 501, an environmental status determination module 502, a working parameter adjustment module 503, and a face recognition module 504.

[0180] The image acquisition module 501 is used to acquire a first infrared face image of the target object when the infrared speckle projector is turned off. The first infrared face image is captured by the infrared camera.

[0181] The environment status determination module 502 is used to determine the environment status of the terminal based on the first infrared face image.

[0182] The operating parameter adjustment module 503 is used to start the infrared speckle projector and adjust the operating parameters of the structured light system based on the environmental conditions.

[0183] The face recognition module 504 is used to perform face recognition on the target object based on the structured light system after the working parameters are adjusted.

[0184] In one possible implementation, the environment state determination module 502 is configured to perform any of the following:

[0185] If the brightness value of the first infrared face image is greater than or equal to the brightness threshold, the environmental state of the terminal is determined to be the first environmental state, which indicates that there is a light source in the environment where the terminal is located that meets the target lighting conditions.

[0186] If a speckle pattern is present in the first infrared face image, the environmental state of the terminal is determined to be the second environmental state, which indicates that there is interference from other infrared speckle projectors in the environment where the terminal is located.

[0187] In one possible implementation, the structured light system further includes a visible light camera, and the operating parameter adjustment module 503 is configured to perform at least one of the following when the environmental state of the terminal is the first environmental state:

[0188] Reduce the exposure time of the infrared camera.

[0189] Improve the signal-to-noise ratio of this infrared speckle projector.

[0190] Adjust the metering area of ​​the visible light camera to the face area of ​​the target object.

[0191] In one possible implementation, the structured light system further includes a visible light camera, and the operating parameter adjustment module 503 is configured to perform at least one of the following when the environmental state of the terminal is the second environmental state:

[0192] Adjust the operating time of the infrared speckle projector.

[0193] Adjust the operating frequency of the infrared speckle projector.

[0194] Improve the signal-to-noise ratio of this infrared speckle projector.

[0195] Reduce the exposure time of the infrared camera.

[0196] In one possible implementation, the device further includes a first adjustment module, configured to perform at least one of the following when the environmental state of the terminal is the first environmental state:

[0197] Increase the screen brightness of the terminal.

[0198] Increase the contrast of the current display interface on this terminal.

[0199] Increase the volume of this terminal.

[0200] In one possible implementation, the device further includes a second adjustment module, configured to perform at least one of the following when the environmental state of the terminal is the second environmental state:

[0201] A prompt message will be displayed.

[0202] Play a prompt message.

[0203] Both the text message and the voice prompt are used to indicate that the terminal's location needs to be changed.

[0204] In one possible implementation, the structured light system further includes a visible light camera. The face recognition module 504 is used to acquire a second infrared face image captured by the infrared camera and a first color face image captured by the visible light camera. Both the second infrared face image and the first color face image are acquired after the operating parameters of the structured light system have been adjusted. Based on the second infrared face image and the first color face image, face recognition is performed on the target object.

[0205] In one possible implementation, the face recognition module 504 is used to determine the quality score of the second infrared face image. If the quality score of the second infrared face image is less than or equal to a first quality score threshold, the infrared speckle projector is controlled to turn off. A third infrared face image is acquired by the infrared camera. Based on the third infrared face image and the first color face image, face recognition is performed on the target object.

[0206] In one possible implementation, the face recognition module 504 is further configured to reduce the weight of the second infrared face image in the face recognition process when the quality score of the second infrared face image is less than or equal to a first quality score threshold. This weight is positively correlated with the degree of influence on the face recognition result. Face recognition of the target object is then performed based on the second infrared face image and the first color face image.

[0207] In one possible implementation, the face recognition module 504 is further configured to, when the quality score of the second infrared face image is greater than the first quality score threshold, determine the three-dimensional face structure of the target object based on the second infrared face image. Feature extraction is then performed on the three-dimensional face structure and the first color face image to obtain the three-dimensional face features and two-dimensional face features of the target object. Face recognition of the target object is then performed based on the three-dimensional face features and the two-dimensional face features.

[0208] In one possible implementation, the face recognition module 504 is further configured to determine the quality score of the first color face image. If the quality score of the first color face image is less than or equal to a second quality score threshold, image enhancement is performed on the first color face image to obtain a second color face image. Based on the second infrared face image and the second color face image, face recognition is performed on the target object.

[0209] It should be noted that the face recognition device provided in the above embodiments is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the face recognition device and the face recognition method embodiments provided in the above embodiments belong to the same concept, and the specific implementation process can be found in the method embodiments, which will not be repeated here.

[0210] The technical solution provided in the embodiments of this specification allows for the acquisition of a first infrared face image by an infrared camera within the structured light system when the infrared speckle projector in the structured light system is turned off. This first infrared face image is used to determine the environmental state of the terminal. After obtaining the environmental state, the infrared speckle projector is activated, and the operating parameters of the structured light system are adjusted based on the environmental state, enabling the structured light system to operate in a manner matching the environmental state. Based on the adjusted operating parameters, the structured light system performs face recognition on the target object, thereby improving the success rate and accuracy of face recognition under different environmental conditions.

[0211] This specification provides a computer device for performing the above-described method. This computer device can be implemented as a terminal. The structure of the terminal is described below:

[0212] Figure 6 This is a schematic diagram of the structure of a terminal provided in an embodiment of this specification. The terminal 600 can be a smartphone, tablet computer, laptop computer, or desktop computer. The terminal 600 may also be referred to as user equipment, portable terminal, laptop terminal, desktop terminal, or other names.

[0213] Typically, terminal 600 includes one or more processors 601 and one or more memories 602.

[0214] Processor 601 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 601 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 601 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 601 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 601 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0215] The memory 602 may include one or more computer-readable storage media, which may be non-transitory. The memory 602 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 602 are used to store at least one computer program, which is executed by the processor 601 to implement the face recognition method provided in the method embodiments of this specification.

[0216] In some embodiments, the terminal 600 may optionally include a peripheral device interface 603 and at least one peripheral device. The processor 601, memory 602, and peripheral device interface 603 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 603 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 604, a display screen 605, a camera assembly 606, an audio circuit 607, and a power supply 608.

[0217] Peripheral interface 603 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 601 and memory 602. In some embodiments, processor 601, memory 602 and peripheral interface 603 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 601, memory 602 and peripheral interface 603 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0218] The radio frequency (RF) circuit 604 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 604 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 604 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 604 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc.

[0219] Display screen 605 is used to display a user interface (UI). This UI may include graphics, text, icons, video, and any combination thereof. When display screen 605 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 601 for processing. In this case, display screen 605 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard.

[0220] The camera assembly 606 is used to capture images or videos. Optionally, the camera assembly 606 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal.

[0221] The audio circuit 607 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals that are input to the processor 601 for processing, or input to the radio frequency circuit 604 to realize voice communication.

[0222] Power supply 608 is used to supply power to the various components in terminal 600. Power supply 608 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery.

[0223] In some embodiments, the terminal 600 further includes one or more sensors 609. The one or more sensors 609 include, but are not limited to, an accelerometer 610, a gyroscope 611, a pressure sensor 612, an optical sensor 613, and a proximity sensor 614.

[0224] Accelerometer 610 can detect the magnitude of acceleration on the three coordinate axes of a coordinate system established with terminal 600.

[0225] The gyroscope sensor 611 can detect the orientation and rotation angle of the terminal 600. The gyroscope sensor 611 can work in conjunction with the accelerometer sensor 610 to collect the user's 3D movements on the terminal 600.

[0226] The pressure sensor 612 can be installed on the side bezel of the terminal 600 and / or on the lower layer of the display screen 605. When the pressure sensor 612 is installed on the side bezel of the terminal 600, it can detect the user's grip signal on the terminal 600, and the processor 601 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 612. When the pressure sensor 612 is installed on the lower layer of the display screen 605, the processor 601 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 605.

[0227] An optical sensor 613 is used to collect ambient light intensity. In one embodiment, a processor 601 can control the display brightness of a display screen 605 based on the ambient light intensity collected by the optical sensor 613.

[0228] The proximity sensor 614 is used to detect the distance between the user and the front of the terminal 600.

[0229] Those skilled in the art will understand that Figure 6 The structure shown does not constitute a limitation on terminal 600, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0230] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including a computer program that can be executed by a processor to perform the face recognition method in the above embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0231] In an exemplary embodiment, a computer program product or computer program is also provided, which includes program code stored in a computer-readable storage medium. The processor of a computer device reads the program code from the computer-readable storage medium and executes the program code, causing the computer device to perform the above-described face recognition method.

[0232] In some embodiments, the computer program described in this specification may be deployed and executed on a single computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed across multiple locations and interconnected via a communication network. These multiple computer devices distributed across multiple locations and interconnected via a communication network may constitute a blockchain system.

[0233] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0234] The above are merely optional embodiments of this specification and are not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification shall be included within the scope of protection of this specification.

Claims

1. A face recognition method, executed by a terminal, the terminal comprising a structured light system, the structured light system comprising an infrared speckle projector and an infrared camera, the method comprising: With the infrared speckle projector off, a first infrared face image of the target object is acquired, the first infrared face image being captured by the infrared camera; Based on the first infrared face image, the environmental state of the terminal is determined; The infrared speckle projector is activated, and the operating parameters of the structured light system are adjusted based on the environmental conditions. The structured light system, after adjusting its operating parameters, performs facial recognition on the target object. The step of determining the environmental state of the terminal's environment based on the first infrared face image includes: If the brightness value of the first infrared face image is greater than or equal to the brightness threshold, the environmental state of the terminal is determined to be the first environmental state, which indicates that there is a light source in the environment where the terminal is located that meets the target lighting conditions. The structured light system also includes a visible light camera, and adjusting the operating parameters of the structured light system based on the environmental conditions includes: If the terminal is in the first environmental state, perform at least one of the following: Reduce the exposure time of the infrared camera; Improve the signal-to-noise ratio of the infrared speckle projector; The metering area of ​​the visible light camera is adjusted to the face area of ​​the target object.

2. The method according to claim 1, wherein determining the environmental state of the environment in which the terminal is located based on the first infrared face image further includes: If a speckle pattern is present in the first infrared face image, the environmental state of the terminal is determined to be a second environmental state, which indicates that there is interference from other infrared speckle projectors in the environment where the terminal is located.

3. The method according to claim 2, wherein the structured light system further includes a visible light camera, and adjusting the operating parameters of the structured light system based on the environmental conditions includes: If the terminal is in the second environmental state, perform at least one of the following: Adjust the operating time of the infrared speckle projector; Adjust the operating frequency of the infrared speckle projector; Improve the signal-to-noise ratio of the infrared speckle projector; Reduce the exposure time of the infrared camera.

4. The method according to claim 1, wherein when the environmental state of the terminal is the first environmental state, the method further comprises at least one of the following: Increase the screen brightness of the terminal; Increase the contrast of the current display interface of the terminal; Increase the volume of the terminal.

5. The method according to claim 2, wherein when the environmental state of the terminal is the second environmental state, the method further comprises at least one of the following: Display a prompt message; Play the prompt voice message; Both the prompt message and the prompt voice are used to prompt the user to change the location of the terminal.

6. The method according to claim 1, wherein the structured light system further comprises a visible light camera, and the face recognition of the target object based on the structured light system after adjusting the operating parameters comprises: The system acquires a second infrared face image captured by the infrared camera and a first color face image captured by the visible light camera. Both the second infrared face image and the first color face image are acquired after the operating parameters of the structured light system have been adjusted. Face recognition is performed on the target object based on the second infrared face image and the first color face image.

7. The method according to claim 6, wherein performing face recognition on the target object based on the second infrared face image and the first color face image comprises: Determine the quality score of the second infrared face image; If the quality score of the second infrared face image is less than or equal to the first quality score threshold, the infrared speckle projector is controlled to turn off. Acquire the third infrared face image captured by the infrared camera; Face recognition is performed on the target object based on the third infrared face image and the first color face image.

8. The method according to claim 7, wherein after determining the quality score of the second infrared face image, the method further comprises: If the quality score of the second infrared face image is less than or equal to the first quality score threshold, the weight of the second infrared face image in the face recognition process is reduced, and the weight is positively correlated with the degree of influence on the face recognition result. Face recognition is performed on the target object based on the second infrared face image and the first color face image.

9. The method according to claim 7, wherein after determining the quality score of the second infrared face image, the method further comprises: If the quality score of the second infrared face image is greater than the first quality score threshold, the three-dimensional face structure of the target object is determined based on the second infrared face image. Feature extraction is performed on the three-dimensional face structure and the first color face image respectively to obtain the three-dimensional face features and two-dimensional face features of the target object; Face recognition of the target object is performed based on the three-dimensional face features and the two-dimensional face features.

10. The method according to claim 6, wherein performing face recognition on the target object based on the second infrared face image and the first color face image comprises: Determine the quality score of the first color face image; If the quality score of the first color face image is less than or equal to the second quality score threshold, image enhancement is performed on the first color face image to obtain the second color face image. Face recognition is performed on the target object based on the second infrared face image and the second color face image.

11. A face recognition device applied to a terminal, the terminal including a structured light system, the structured light system including an infrared speckle projector and an infrared camera, the device comprising: The image acquisition module is used to acquire a first infrared face image of the target object when the infrared speckle projector is turned off, wherein the first infrared face image is captured by the infrared camera; An environment state determination module is used to determine the environmental state of the terminal based on the first infrared face image. The operating parameter adjustment module is used to start the infrared speckle projector and adjust the operating parameters of the structured light system based on the environmental conditions. A face recognition module is used to perform face recognition on the target object based on the structured light system after adjusting the working parameters; The environmental state determination module is further configured to determine the environmental state of the terminal as a first environmental state when the brightness value of the first infrared face image is greater than or equal to a brightness threshold. The first environmental state indicates that there is a light source in the environment where the terminal is located that meets the target lighting conditions. The structured light system also includes a visible light camera. The operating parameter adjustment module is further configured to perform at least one of the following when the environmental state of the terminal is the first environmental state: reduce the exposure time of the infrared camera; increase the signal-to-noise ratio of the infrared speckle projector; and adjust the metering area of ​​the visible light camera to the face area of ​​the target object.

12. A computer device comprising one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, the computer program being loaded and executed by the one or more processors to implement the face recognition method as claimed in any one of claims 1 to 10.

13. A computer-readable storage medium storing at least one computer program, the computer program being loaded and executed by a processor to implement the face recognition method as described in any one of claims 1 to 10.

14. A computer program product comprising a computer program that, when executed by a processor, implements the face recognition method according to any one of claims 1 to 10.

Images