Facial recognition methods, devices, equipment and storage media
By continuously acquiring image frame sequences and extracting facial time-series features for identity recognition in a desktop POS device, the problem of cumbersome operation in existing facial payment methods is solved, achieving seamless facial payment and efficient identity recognition.
Patent Information
- Application Number
- CN202110017428.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-01-07
- Publication Date
- 2026-11-17
- Estimated Expiration
- 2041-01-07
AI Technical Summary
Existing desktop POS systems require cashiers to manually select facial recognition payment methods, which is cumbersome and reduces the efficiency of order settlement.
By continuously acquiring image frame sequences, when a face appears in the image frame sequence, the time sequence features of the face that meet the face recognition conditions are extracted for identity recognition, eliminating the need for users to manually select the recognition method and improving the efficiency of face recognition.
It enables seamless facial recognition payment, improving the efficiency and accuracy of facial recognition. Users can complete identity verification and payment without manual operation, thus enhancing payment convenience.
Smart Images

Figure CN114742561B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of facial recognition, and in particular to a facial recognition method, apparatus, device, and storage medium. Background Technology
[0002] Facial recognition payment refers to a payment transaction method based on facial recognition technology. Users only need to face the camera of the cash register, and the cash register will match the facial information it recognizes with the user's pre-stored personal information. If the match is successful, the payment transaction is completed.
[0003] Taking a desktop POS system as an example, when a customer selects items and checks out, the desktop POS system generates a purchase order based on the selected items. The cashier then selects a payment method on the desktop POS system, such as facial recognition payment. The customer stands in front of the camera on the desktop POS system and pays by scanning their face.
[0004] In the above technical solutions, the desktop POS system requires cashiers to manually select the facial payment method before the camera can be activated to recognize the customer's face, which is cumbersome and reduces the efficiency of order settlement. Summary of the Invention
[0005] This application provides a method, apparatus, device, and storage medium for face recognition. By continuously acquiring image frame sequences, when a face appears in the image frame sequence, identity recognition is performed based on the extracted time-series features of the face that meet the face recognition conditions. This eliminates the need for the user to manually select a recognition method, thus improving face recognition efficiency. The technical solution includes the following: According to one aspect of this application, a face recognition method is provided, the method comprising the following steps: Acquire image frame sequences; In response to the appearance of a face in the image frame sequence, facial time-series features are extracted from the image frame sequence, wherein the facial time-series features are facial features contained in consecutive image frames; In response to the face time series features satisfying the face recognition conditions, the face features contained in the image frame sequence are used for identity recognition.
[0006] According to another aspect of this application, a face recognition device is provided, the device comprising the following parts: The acquisition module is used to acquire image frame sequences; An extraction module is used to extract face time-series features from the image frame sequence in response to the appearance of a face in the image frame sequence, wherein the face time-series features are face features contained in consecutive image frames; The processing module is used to perform identity recognition on the facial features contained in the image frame sequence in response to the facial time series features meeting the facial recognition conditions.
[0007] In an optional embodiment, the face time-series features satisfying the face recognition conditions include at least one of the following: The facial features in the facial time series features belong to the same face; The area of the face region in the face time series features continues to increase; The human eyes in the facial time-series features are in a state of continuous fixation.
[0008] In an optional embodiment, the acquisition module is used to acquire the first image frame in the image frame sequence in which a face appears, the first image frame including a first face feature; The processing module is configured to, in response to a second facial feature in n consecutive image frames following the first image frame in the image frame sequence matching the first facial feature, accumulate the number n of image frames whose facial features belong to the same face, where n is a positive integer; and in response to the number of image frames whose facial features belong to the same face reaching a first counting threshold, determine that the facial features in the facial time series features belong to the same face.
[0009] In an optional embodiment, the acquisition module is used to obtain the similarity between the first facial feature and the second facial feature; The processing module is used to determine that the second facial feature matches the first facial feature in response to the similarity reaching a similarity threshold.
[0010] In an optional embodiment, the acquisition module is used to acquire the j-th image frame and the (j+1)-th image frame in the image frame sequence in which a face appears. The j-th image frame contains a first face region, and the (j+1)-th image frame contains a second face region. The first face region corresponds to a first face region area, and the second face region corresponds to a second face region area, where j is a positive integer. The processing module is configured to, in response to the second face region area being greater than the first face region area, accumulate the number of image frames in which the face region area continuously increases; repeat the steps of acquiring the j-th image frame and the (j+1)-th image frame, and the step of accumulating the number of image frames in which the face region area continuously increases; and, in response to the number of image frames in which the face region area continuously increases reaching a second counting threshold, determine that the face region area in the face time series features is continuously increasing.
[0011] In an optional embodiment, the acquisition module is configured to acquire image frames in the image frame sequence in which a face appears, the image frames in which a face appears including the gaze direction of the human eye; the processing module is configured to, in response to the fact that the gaze direction included in n consecutive image frames in the image frame sequence belongs to the frontal gaze direction, accumulate the number n of image frames in which the human eye is in the continuous gaze state, where n is a positive integer; in response to the fact that the number of image frames in which the human eye is in the continuous state reaches a third counting threshold, determine that the human eye in the face time series features is in the continuous state.
[0012] In an optional embodiment, the device includes a reading module, a first transmitting module, and a first receiving module; The reading module is used to read the device identifier of the terminal; The first sending module is used to send an information acquisition request to the server, wherein the information acquisition request carries the device identifier; The first receiving module is used to receive configuration information sent by the server. The configuration information is obtained by the server based on the device identifier and the association relationship. The association relationship is used to characterize the correspondence between the device identifier and the configuration information. The configuration information includes the face recognition conditions.
[0013] According to another aspect of this application, a computer device is provided, the computer device comprising: a processor and a memory, the memory storing at least one instruction, at least one program, a code set or an instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the face recognition method as described above.
[0014] According to another aspect of this application, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the face recognition method as described above.
[0015] According to another aspect of this application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the face recognition method as described above.
[0016] The beneficial effects of the technical solutions provided in this application include at least the following: By continuously acquiring image frame sequences, when a face appears in the sequence, identity recognition is performed based on the extracted time-series features of the face that meet the facial recognition criteria. This eliminates the need for users to manually select a recognition method; as long as the user's face is in front of the camera, seamless identity recognition is achieved, improving efficiency and accuracy. For example, in unmanned convenience stores, where users carry many items (with their hands not free), the self-checkout equipment can use the facial recognition method provided in this application to identify the user and complete seamless payment for goods, eliminating the need for manual operation and increasing convenience. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a block diagram of a computer system provided in an exemplary embodiment of this application; Figure 2 This is a flowchart of a face recognition method provided in an exemplary embodiment of this application; Figure 3 This is a flowchart of a face recognition method provided in another exemplary embodiment of this application; Figure 4 This is a flowchart of a face recognition method provided in another exemplary embodiment of this application; Figure 5 This is a system framework diagram of face recognition provided in an exemplary embodiment of this application; Figure 6 This is a block diagram of a face recognition device provided in an exemplary embodiment of this application; Figure 7 This is a block diagram of a face recognition device provided in another exemplary embodiment of this application; Figure 8 This is a block diagram of a server provided in an exemplary embodiment of this application; Figure 9 This is a block diagram of a computer device provided in an exemplary embodiment of this application. Detailed Implementation
[0019] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.
[0020] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data used for analysis, data stored, data displayed, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices) involved in this application have all been fully authorized by the user or relevant parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.
[0021] First, the terms used in the embodiments of this application will be introduced: 3D cameras (3-Dimension cameras) detect distance information in the shooting space, typically by adding liveness detection software to achieve live face detection. They possess functions such as face recognition, gesture recognition, human skeleton recognition, 3D measurement, environmental perception, and 3D map reconstruction. They can be widely used in smart terminals, smart robots, drones, autonomous driving, virtual reality (VR), augmented reality (AR), smart homes, smart healthcare, and smart customer service, among other fields.
[0022] Serial Number (SN): A concept introduced to verify the legitimate identity of a product, used to uniquely identify a device. In this embodiment, the serial number uniquely identifies the terminal, enabling the server to determine the configuration information corresponding to the terminal, which is used to acquire image frame sequences.
[0023] SQLite database: refers to a lightweight database that adheres to ACID principles in relational data management systems. ACID is an abbreviation for the four fundamental requirements for the correct execution of database transactions: Atomicity, Consistency, Isolation, and Durability. In this embodiment, the terminal stores configuration information, including face recognition conditions, through an SQLite database.
[0024] Computer vision (CV) is a science that studies how to enable machines to "see." More specifically, it refers to machine vision, which uses cameras and computers to replace human eyes in recognizing, tracking, and measuring targets, and then performs image processing to create images more suitable for human observation or transmission to instruments. As a scientific discipline, computer vision studies related theories and technologies, attempting to build artificial intelligence systems capable of extracting information from images or multidimensional data. Computer vision technologies typically include image processing, image recognition, image semantic understanding, image retrieval, optical character recognition (OCR), video processing, video semantic understanding, video content / behavior recognition, 3D object reconstruction, 3D technology, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), and other technologies. The method provided in this application uses computer vision technology to identify individuals based on facial features.
[0025] The face recognition method provided in this application can be applied to the following scenarios: I. Facial recognition payment scenarios.
[0026] In this application scenario, the face recognition method provided in this embodiment can be applied to a POS device equipped with a camera. The POS device captures image frame sequences through the camera. When a face appears in the image frame sequence, face time-series features are extracted from the image frame sequence. If the face features in the face time-series features belong to the same face, or if the face region area in the face time-series features continuously increases, or if the eyes in the face time-series features are in a continuous gaze state, the face features contained in the image frame sequence are used for identity recognition. Payment is made to the purchase order based on the recognized payment account, realizing contactless face payment without requiring the user to manually select a payment method, thus improving the settlement efficiency of purchase orders.
[0027] II. Identity verification scenario.
[0028] In this application scenario, the face recognition method provided in this application embodiment can be applied to an identity verification device equipped with a camera. The identity verification device acquires image frame sequences through the camera. When a face appears in the image frame sequence, the device extracts the face time-series features from the image frame sequence. If the face features in the face time-series features belong to the same face, or if the face region in the face time-series features continuously increases in size, or if the eyes in the face time-series features are in a continuous gaze state, the device performs identity recognition on the face features contained in the image frame sequence. For example, in subway ticket gates, the ticket gates are equipped with cameras to verify the identity of the appearing faces, thereby determining the user's travel identity (e.g., determining the user's travel account by scanning the face); similarly, in train ticket gates, the ticket gates are equipped with cameras to verify the identity of the appearing faces, thereby determining the user's purchased ticket information.
[0029] The above examples illustrate only two application scenarios. The face recognition method provided in this application can also be applied to other face recognition scenarios (such as using face recognition to record employee attendance). This application does not limit the specific application scenarios.
[0030] The face recognition method provided in this application can be applied to computer devices with strong data processing capabilities. In one possible implementation, the face recognition method provided in this application can be applied to a personal computer, workstation, or server, that is, payment for purchase orders can be realized through a personal computer, workstation, or server. Illustratively, the face recognition method is applied to a server, which is the backend server of the application, enabling terminals with the application installed to perform face recognition on users with the help of the server.
[0031] Figure 1 A schematic diagram of a computer system provided in an exemplary embodiment of this application is shown. The computer system 100 includes a terminal 110 and a server 120, wherein the terminal 110 and the server 120 communicate with each other via a communication network. Optionally, the communication network can be a wired network or a wireless network, and the communication network can be at least one of a local area network, a metropolitan area network, and a wide area network.
[0032] Terminal 110 has an application that supports face recognition installed and running. When the application starts, it uses the terminal 110's camera to capture a sequence of face image frames and extracts facial time-series features from the sequence. These facial time-series features are the facial features contained in consecutive image frames. When the facial time-series features meet the face recognition criteria, they are sent to server 120 for identity verification.
[0033] Optionally, the terminal 110 can be a cash register device with a camera, a smartphone, a tablet computer, a laptop computer, a smart robot, or other mobile terminal, or a desktop computer, a projector computer, or other terminal. This application embodiment does not limit the type of terminal.
[0034] Server 120 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. In one possible implementation, server 120 is the backend server for the application in terminal 110.
[0035] like Figure 1 As shown, in this embodiment, terminal 110 is a terminal with a camera or a terminal connected to a camera. Terminal 110 runs an application that supports face recognition. When the application starts, it can call the camera component of terminal 110 to capture images. The image frame sequence captured by terminal 110 may or may not contain faces. Terminal 110 extracts face time series features from image frames containing faces.
[0036] When the facial time series features meet the facial recognition conditions, including at least one of the following conditions, the facial time series features are used for identity recognition.
[0037] Scenario 1: The camera captures the first image frame containing a face, which contains a first facial feature. The next n consecutive image frames (n is a positive integer) contain a second facial feature. When the first facial feature matches the second facial feature, it is determined that the first and second facial features belong to the same face. The number of image frames containing facial features belonging to the same face is accumulated. When the number of image frames containing facial features belonging to the same face reaches a first counting threshold, it is determined that the facial features in the facial time series characteristics belong to the same face.
[0038] Scenario 2: The camera captures two adjacent image frames containing a face, such as frame j and frame j+1, where j is a positive integer. Frame j contains a first face region with a corresponding area; frame j+1 contains a second face region with a corresponding area. When the area of the second face region is greater than the area of the first face region, it is determined that the face region area is continuously increasing. This process is repeated, accumulating the number of image frames with continuously increasing face region areas. When the number of image frames with continuously increasing face region areas reaches a second counting threshold, it is determined that the face region area in the face time series features is continuously increasing.
[0039] Case 3: The camera captures n consecutive image frames showing a human face. These n image frames contain the direction of the human eye's gaze (n is a positive integer). The number of image frames with the gaze direction being the frontal gaze (i.e., the image frames captured when the user is facing the camera) is accumulated. When the number of image frames with the gaze direction being the frontal gaze (the human eye is in a continuous gaze state) reaches the third counting threshold, it is determined that the human eye in the face time series features is in a continuous gaze state.
[0040] Schematic illustration: When terminal 110 is a POS device equipped with a camera, server 120 performs the following steps: Step 11, receiving facial time-series features; Step 12, performing identity verification on the facial time-series features; Step 13, processing payment for the purchase order based on the identity verification result. Server 120 queries a pre-stored payment account based on the facial time-series features, which corresponds to the facial time-series features, and processes the purchase order through this payment account. Server 120 sends payment result 14 to terminal 110, where a prompt message is displayed: Payment successful.
[0041] It is understood that the above embodiments only take the example of a terminal with an application that supports facial recognition installed. In actual use, the above facial recognition method can also be applied to mini-programs (programs that run dependent on a host program) or web pages. A mini-program is an application that can be used without downloading or installing. In order to provide users with more diversified business services, developers can develop corresponding mini-programs for terminal applications (such as instant messaging applications, shopping applications, email applications, etc.). The mini-program can be embedded as a sub-application into the terminal application. The terminal application is the host program of the mini-program. By running the sub-application (i.e., the corresponding mini-program) within the application, corresponding business services can be provided to users. This application embodiment does not limit this.
[0042] Figure 2 A flowchart illustrating an exemplary embodiment of this application provides a face recognition method. This embodiment applies the method to, for example... Figure 1 Taking terminal 110 in the computer system 100 shown as an example, the method includes the following steps: Step 201: Acquire image frame sequence.
[0043] An image frame sequence is captured by a camera connected to a terminal or part of a terminal, which may include at least one of a point-of-sale (POS) device, a smart camera (AI camera), a smartphone, or a platform computer. An image frame sequence refers to a sequence of multiple consecutive image frames captured by the camera; the image frame sequence may or may not contain a face.
[0044] In one example, the POS device is connected to a camera, and a POS-enabled application is installed on the device. This application can access the camera to capture images. When the application starts, the camera begins capturing images; when the application is closed or the POS device is turned off, the camera stops capturing images.
[0045] Step 202: In response to the appearance of a face in the image frame sequence, extract the face time series features from the image frame sequence. The face time series features are the face features contained in consecutive image frames.
[0046] When a face appears in an image frame sequence, the sequence contains temporal features of the face. That is, when a face appears in front of the camera while it is capturing images, the camera will capture multiple consecutive image frames containing the face, forming a sequence of face image frames. This sequence is a subset or partial sequence of image frames, and each frame contains a face. These faces appearing in each frame correspond to facial features. Since the face image frames are arranged according to their capture time, these facial features also exhibit temporal order. The faces contained in each frame may be the same or different, and the facial features extracted from each frame may be the same, similar, or different.
[0047] This illustration demonstrates how to extract facial time-series features using geometric methods. A human face consists of parts such as the eyes, nose, mouth, and chin. The geometric relationships between the shapes and structures of these parts can represent facial features. For example, a triangle formed by connecting the two eyes and the nose can represent a facial feature. This method extracts facial features from each frame of a face image sequence, forming a facial time-series feature.
[0048] This illustration demonstrates the extraction of facial time-series features using template matching. The terminal includes a pre-acquired facial feature database from a server, which contains facial feature templates. Facial features extracted from a sequence of facial image frames are matched against these templates. If a match is found, the facial feature is determined based on the template. This method extracts facial features from each frame of the facial image frame sequence, forming a facial time-series feature set.
[0049] The embodiments of this application do not limit the method of extracting facial features.
[0050] Step 203: In response to the face time series features meeting the face recognition conditions, perform identity recognition on the face features contained in the image frame sequence.
[0051] A face time-series feature that satisfies the face recognition condition includes at least one of the following: Facial features in a time-series facial feature belong to the same face; The area of the face region in the time series features of the face continues to increase; The human eyes in the face time series features are in a state of continuous fixation.
[0052] Facial features belonging to the same face means that facial features extracted from multiple consecutive image frames containing faces indicate that the faces belong to the same face.
[0053] The continuous increase in the face region area refers to the continuous increase in the face region area in multiple consecutive image frames containing faces. For example, the face region area in the (j+1)th image frame is greater than the face region area in the jth image frame, where j is a positive integer and the acquisition order of the (j+1)th image frame is after the jth image frame.
[0054] A sustained gaze state refers to a situation where the gaze direction (line of sight) of a person in multiple consecutive image frames containing a face points in the same direction. For example, if the person's eye in the j-th image frame is gazing straight ahead, and the person's eye in the (j+1)-th image frame is also gazing straight ahead, where j is a positive integer, then the gaze state of the person in the image frame is determined based on these two image frames. This application's embodiment uses the direction of the line of sight pointing towards the camera as an example, i.e., the direction the user faces the camera.
[0055] The terminal sends the facial time-series features to the server, which then determines the user's identity information based on these features.
[0056] In summary, the method provided in this embodiment continuously acquires image frame sequences. When a face appears in the image frame sequence, it performs identity recognition based on the extracted time-series features of the face that meet the face recognition conditions. This eliminates the need for the user to manually select an identity recognition method; as long as the user's face is in front of the camera, seamless identity recognition can be achieved, improving the efficiency and accuracy of face recognition and enabling the terminal to accurately identify the user's identity. For example, in unmanned convenience stores, when a user is carrying a large number of items (with both hands occupied), the self-checkout equipment can use the face recognition method provided in this embodiment to identify the user and complete seamless payment for the goods, eliminating the need for manual operation and improving the convenience of payment.
[0057] Figure 3 A flowchart of a face recognition method provided in another exemplary embodiment of this application is shown. This embodiment applies the method to, for example,... Figure 1 The method will be described using terminal 110 in the computer system 100 shown as an example. The method includes the following steps: Step 301: Read the device identifier of the terminal.
[0058] For illustrative purposes, a device identifier is represented by a string serial number (SN code). In some embodiments, the device identifier also includes a barcode or QR code corresponding to the SN code, which can be scanned to determine the device's SN code.
[0059] Step 302: Send an information retrieval request to the server. The information retrieval request carries the device identifier.
[0060] The terminal sends an information retrieval request carrying an SN code to the server. This information retrieval request is used to obtain the configuration information of each terminal. The configuration information refers to the parameter information that meets the face recognition conditions. For example, if the number of image frames whose facial features in the cumulative face time series features belong to the same face reaches 5 (configuration information), then the image frame containing that face time series feature is determined to meet the face recognition conditions.
[0061] Step 303: Receive configuration information sent by the server. The configuration information is obtained by the server based on the device identifier and the association relationship. The association relationship is used to characterize the correspondence between the device identifier and the configuration information. The configuration information includes face recognition conditions.
[0062] The server pre-stores configuration information and associations, and the configuration information corresponding to the terminal can be determined based on the device identifier and the association. These associations can be in the form of lookup tables or function relationships; however, this application embodiment does not limit the form of the association.
[0063] Table 1 illustrates the relationship between device identifiers and configuration information.
[0064] Table 1
[0065] SN1 indicates a terminal with SN code 1. This application does not limit the type of device identifier.
[0066] Indicatively, facial features can be used for identity recognition if at least one of the three conditions is met. Alternatively, weights can be set for each condition, and the sum of the weights for each condition can be calculated. If the sum of the weights is greater than the recognition condition threshold, the acquired image frame is considered to meet the facial recognition conditions, and facial features can be used for identity recognition.
[0067] The configuration information is identified by the device identifier, which allows users to perform facial recognition on different terminals.
[0068] Step 304: Acquire image frame sequence.
[0069] Taking a POS terminal with a camera as an example, an application is installed on the POS terminal. This application calls the camera to capture image frame sequences. To illustrate, when the application starts, it calls the camera to capture images; the captured images may or may not contain faces.
[0070] Step 305: In response to the appearance of a face in the image frame sequence, extract the face time series features from the image frame sequence. The face time series features are the face features contained in consecutive image frames.
[0071] When a human face appears in an image frame sequence, the acquired image frame sequence contains face time series features.
[0072] Case 1: The facial features in the face time series belong to the same face.
[0073] Steps 306a to 308a determine that the facial features in the facial time series features belong to the same face: Step 306a: Obtain the first image frame in the image frame sequence in which a face appears, and the first image frame includes the first face features.
[0074] When the camera captures the first image frame in which a face appears, the image frame is cached, and the image frame includes the first facial features.
[0075] Step 307a: In response to the matching of the second face feature with the first face feature in the n consecutive image frames following the first image frame in the image frame sequence, the number of image frames whose face features belong to the same face is accumulated n, where n is a positive integer.
[0076] Taking the capture of a user's face as an example, since the camera continuously captures a sequence of image frames, after the first image frame showing the face, there will be a second image frame, a third image frame, and an nth image frame (n is a positive integer).
[0077] Extract the second face feature from the second image frame containing the face, match the first face feature with the second face feature, obtain the similarity between the first face feature and the second face feature, and determine the match between the second face feature and the first face feature when the similarity reaches the similarity threshold.
[0078] To illustrate, similarity can be calculated using Euclidean distance. For example, if the Euclidean distance between the two eyes of the first face is 12 and the Euclidean distance between the two eyes of the second face is 13, then the first face and the second face are considered similar, and the features of the first face match the features of the second face.
[0079] To illustrate, similarity can also be calculated using cosine distance. The cosine of the angle between two vectors in vector space is used as a measure of the difference between two individuals. The closer the angle between two vectors is to 0, the closer the two vectors are, and the smaller the difference is, indicating that the two faces are more similar.
[0080] It should be noted that when a facial feature in one of n consecutive image frames does not match the first facial feature, the count for image frames containing the second facial feature is restarted. For example, if the second facial feature in the second image frame matches the first facial feature, the counter counts to 1. If the second facial feature in the third image frame does not match the first facial feature (e.g., the face in the first two image frames is user A, and the face in the third image frame is user B), the count for image frames with the same facial feature is restarted, and the counter is reset to zero. If the second facial feature in the fourth image frame matches the first facial feature, the counter counts to 1. If the second facial feature in the fifth image frame matches the first facial feature, the counter counts to 2, and so on. The cumulative number of image frames with the same facial feature does not include the first image frame.
[0081] Step 308a: In response to the number of image frames belonging to the same face reaching a first counting threshold, it is determined that the facial features in the facial time series features belong to the same face.
[0082] The first counting threshold is a threshold obtained from the configuration information. When the number of image frames belonging to the same face reaches this counting threshold, the terminal determines that the face features in the face time series features belong to the same face.
[0083] Scenario 2: The area of the face region in the face time series features continues to increase.
[0084] Steps 306b to 309b determine that the area of the face region in the face time series features continuously increases: Step 306b: Obtain the j-th image frame and the (j+1)-th image frame in the image frame sequence in which a face appears. The j-th image frame contains a first face region, and the (j+1)-th image frame contains a second face region. The first face region corresponds to the area of the first face region, and the area of the second face region corresponds to the area of the second face region. j is a positive integer.
[0085] To illustrate, each image frame contains a face region. In n consecutive image frames, each image frame contains a face region, and each face region has a corresponding face region area. The face region areas in any two adjacent image frames are compared. For example, the area of the first face region in the first image frame is 1, and the area of the second face region in the second image frame is 2.
[0086] Step 307b: In response to the second face region area being greater than the first face region area, accumulate the number of image frames in which the face region area continues to increase.
[0087] When the area of the second face region is greater than the area of the first face region, the counter counts to 1. If the area of the first face region in the first image frame is less than the area of the second face region in the second image frame, the counter counts to 1. If the area of the face region in the second image frame is greater than the area of the face region in the third image frame, the counting restarts for image frames where the face region area continues to increase, and the counter is reset to zero. If the area of the face region in the third image frame is less than the area of the face region in the fourth image frame, the counter counts to 1; if the area of the face region in the fourth image frame is less than the area of the face region in the fifth image frame, the counter counts to 2, and so on.
[0088] Step 308b: Repeat the steps of removing the j-th and (j+1)-th image frames, and the step of accumulating the number of image frames whose face area continues to increase.
[0089] Repeat steps 306b and 307b to calculate the number of image frames in which the face region area continuously increases. When the face region area in a certain image frame is no longer increasing, reset the counter to zero and start counting the number of image frames in which the face region area continuously increases again.
[0090] Step 309b: In response to the number of image frames whose face region area continues to increase reaching the second counting threshold, it is determined that the face region area in the face time series features continues to increase.
[0091] The second counting threshold is obtained from the configuration information. When the number of image frames showing a continuously increasing face area reaches this threshold, the terminal determines that the face area in the face time series features is continuously increasing. An increase in the face area indicates that the user is moving closer to the camera.
[0092] Scenario 3: The human eyes in the face time series features are in a continuous fixation state.
[0093] Steps 306c to 308c determine that the human eye in the face time series features is in a state of continuous enlargement: Step 306c: Obtain image frames in the image frame sequence that contain human faces, including the direction of the human eyes' gaze.
[0094] When the captured image frame sequence includes image frames containing a human face, the face in the image frame captured by the camera is facing the camera directly or from the side. When the face is facing the camera directly, the direction of gaze is the forward-looking direction. The image frames in the image frame sequence captured by the camera are then identified as having a forward-looking direction.
[0095] Step 307c: In response to the fact that the viewing direction of n consecutive image frames in the image frame sequence belongs to the frontal viewing direction, the number of image frames in which the human eye is in a continuous fixation state is accumulated, n, where n is a positive integer.
[0096] For illustration purposes, if the gaze direction in the first to fifth image frames is a frontal gaze direction, then the counter counts to 5.
[0097] For illustration purposes, in the first and second image frames, the gaze direction is in the frontal direction, so the counter counts to 2. In the third image frame, the gaze direction is in the sideal direction (e.g., the face is turned to the side and the gaze is directed to one side). The number of image frames in which the human eye is continuously focused is counted again, and the counter counts to zero. If the gaze direction in the fourth to sixth image frames is in the frontal direction, the counter counts to 3, and so on.
[0098] Step 308c: In response to the number of image frames in which the human eye is in a continuous gaze state reaching the third counting threshold, it is determined that the human eye in the face time series features is in a continuous gaze state.
[0099] The third counting threshold is a threshold obtained from the configuration information. When the number of image frames in which the human eye is in a continuous gaze state reaches the third counting threshold, the terminal determines that the human eye in the face time series features is in a continuous gaze state.
[0100] Step 310: In response to the face time series features satisfying the face recognition conditions, perform identity recognition on the face features contained in the image frame sequence.
[0101] When the facial time-series features meet the requirements, the terminal sends these features to the server, which then determines the user's identity based on them. Maintaining a continuous gaze is crucial to prevent misidentification of faces when other faces are also visible in the camera's view.
[0102] In summary, the method of this embodiment continuously acquires image frame sequences. When a face appears in the image frame sequence, it performs identity recognition based on the extracted time-series features of the face that meet the face recognition conditions. This eliminates the need for the user to manually select an identity recognition method; as long as the user's face is in front of the camera, seamless identity recognition can be achieved, improving the efficiency and accuracy of face recognition and enabling the terminal to accurately identify the user's identity. For example, in unmanned convenience stores, when a user is carrying a large number of items (with both hands occupied), the self-checkout equipment can use the face recognition method provided in this embodiment to identify the user and complete seamless payment for the goods, eliminating the need for manual operation and improving the convenience of payment.
[0103] In this embodiment, when the facial features in the facial time series features meet the facial recognition conditions, including at least one of the following: the facial features belong to the same face, the facial area is continuously increasing, and the eyes are in a continuous gaze state, the facial recognition process is initiated, enabling the terminal to achieve seamless recognition of identity information based on the above conditions.
[0104] The method in this embodiment matches the facial features in the first image frame where a face appears with the facial features in the next n consecutive image frames. It accumulates the number of image frames where the facial features belong to the same face. When the number of image frames where the facial features belong to the same face reaches a first counting threshold, it is determined that the facial features belong to the same face. This improves the accuracy of face recognition and avoids misidentification of faces.
[0105] The method in this embodiment also determines whether the first facial feature and the second facial feature match by judging whether the similarity between the first facial feature and the second facial feature reaches a similarity threshold, thereby improving the recognition accuracy of facial recognition and avoiding misidentification of faces.
[0106] The method in this embodiment further compares the area of the face region in any two consecutive image frames, accumulating the number of image frames with a continuously increasing face region area. When the number of image frames with a continuously increasing face region area reaches a second counting threshold, it is determined that the face region area is continuously increasing. This improves the accuracy of face recognition and avoids false recognition of faces.
[0107] In this embodiment, when the gaze direction in n consecutive image frames is the forward gaze direction, the number of image frames in which the human eye is in a continuous gaze state is accumulated. When the number of image frames in which the human eye is in a continuous gaze state reaches a third counting threshold, it is determined that the human eye is in a continuous gaze state. This improves the accuracy of face recognition and avoids misidentification of faces.
[0108] The method in this embodiment also obtains a device identifier from a server. The server then uses the device identifier to obtain the terminal's configuration information, enabling the terminal to perform seamless facial recognition based on the configuration information. Using a unique string serial number to identify the terminal as the device identifier distinguishes each terminal, allowing the server to configure corresponding information for each terminal.
[0109] It is understood that the above embodiments can be implemented individually or in any combination.
[0110] The face recognition method provided in the embodiments of this application will be described in conjunction with the server. Figure 4 A flowchart of a face recognition method provided in another exemplary embodiment of this application is shown. This embodiment applies the method to, for example... Figure 1 The method will be illustrated using server 120 in the computer system 100 shown as an example. The method includes the following steps: Step 401: Receive the face time series features sent by the receiving terminal. The face time series features are the face features contained in consecutive image frames.
[0111] The server receives the time-series facial features sent by the terminal.
[0112] Step 402: Perform identity recognition based on the face time-series features.
[0113] The server pre-stores user identity information and establishes a correspondence between user identity information and facial time-series features. The server retrieves user identity information based on the correspondence satisfied by the facial time-series features. This user identity information includes at least one of payment account, ID card number, and mobile phone number. In some embodiments, the user identity information may also be merchant membership information.
[0114] In summary, the method of this embodiment receives facial time-series features sent by the POS device through the server, determines the user's identity based on the facial time-series features, and enables the terminal to achieve seamless recognition of the user's identity information, thereby improving the efficiency of facial recognition.
[0115] based on Figure 4 In an alternative embodiment, the server also sends configuration information to the terminal.
[0116] Step 411: Receive the acquisition request sent by the terminal. The acquisition request carries the device identifier.
[0117] As an illustration, a terminal typically sends a request to the server upon initial use to obtain configuration information. When a user performs facial recognition using different terminals, the server distributes the configuration information to each terminal based on the device identifier.
[0118] Step 412: Determine the association relationship. The association relationship is used to characterize the correspondence between device identifier and configuration information. The configuration information includes face recognition conditions.
[0119] This relationship can be implemented with reference to the implementation method in Table 1, and will not be repeated here.
[0120] Step 413: Obtain configuration information based on device identifier and association.
[0121] Step 414: Send the configuration information to the terminal.
[0122] Steps 413 and 414 refer to the implementation method of step 303 in the above embodiments, and the source will not be repeated.
[0123] In summary, the method of this embodiment further obtains a device identifier from the server, and the server obtains the terminal's configuration information based on the device identifier, thereby enabling the terminal to achieve seamless facial recognition of the user based on the configuration information. Using a unique string serial number to identify the terminal as the device identifier allows each terminal to be distinguished, and the server can configure corresponding configuration information for each terminal.
[0124] based on Figure 4 In an optional embodiment, the server also makes payments for the purchase order based on the identification result obtained from the identity recognition.
[0125] Step 421: Obtain the payment account based on the identification result obtained from the identity verification.
[0126] In an illustrative manner, using a terminal as a payment device, when the payment device sends facial time-series features to a server, the server determines the payment account corresponding to that face based on the facial time-series features, such as determining the user's payment account in a payment application. In some embodiments, the identification result includes the merchant's membership information, which includes at least one of points, remaining balance, and coupons.
[0127] Step 422: Pay for the purchase order according to the payment account and get a payment success result.
[0128] The server transfers funds from the payment account to the merchant account via the payment account; in other embodiments, the server sends the payment account and purchase order to the payment server, the payment server pays for the purchase order, and returns the payment result to the server.
[0129] Step 423: Send the payment success result to the terminal.
[0130] As an illustration, after a successful payment, the server generates a payment success message (payment success result) and sends the payment success message to the terminal.
[0131] In summary, the method of this embodiment sends a payment success result to the terminal through the server, enabling the terminal to complete the settlement process of the purchase order and realize seamless payment for the user.
[0132] In some embodiments, if the terminal is a merchant's POS device, when the POS device first recognizes user A's face, after identifying user A's identity information, it checks whether user A has membership information at the merchant. If user A does not have membership information, with user A's permission, a frame is randomly selected from the captured image frame sequence containing user A's face as the photo for registering membership information, without needing to record user A's mobile phone number. When user A makes a purchase at the merchant the next time, user A's membership information can be determined through facial recognition.
[0133] In one example, taking a terminal with a camera as a POS device with payment functionality, the facial recognition method of the POS device is explained in conjunction with the user payment process.
[0134] Figure 5 A system framework diagram for face recognition provided in an exemplary embodiment of this application is shown. The system includes a server 120 and a cash register device 111.
[0135] The POS device 111 is a POS device containing a 3D camera 21, which is used to capture image frame sequences. An application is also installed in the POS device 111. This application is used for receiving payments. For example, if a customer presents a payment code, the application calls the 3D camera 21 to identify the payment account corresponding to the payment code, thereby collecting payment for the customer's purchase order. Alternatively, the application calls the 3D camera 21 to recognize the customer's face, identifying the customer's identity based on their facial features, and thus collecting payment for the purchase order.
[0136] When the application runs, it listens for the system startup broadcast of the POS device 111. After the system starts, the system calls the application, which then reads the serial number (SN code) of the POS device 111. This SN code uniquely identifies each POS device 111; that is, each POS device 111 has a different SN code. The POS device 111 sends the SN code to the server 120 and also requests facial features from the facial feature database from the server 120.
[0137] Server 120 pre-stores configuration information for each POS device 111, including face recognition conditions such as the number of sustained gazes, the number of sustained enlargements, and the number of sustained encounters with the same person. Server 120 queries the corresponding configuration information based on the received serial number (SN) and sends the configuration information corresponding to the SN to the POS device 111. Server 120 also sends facial features from its facial feature database to the POS device 111.
[0138] The POS device 111 caches the facial features and writes the configuration information into a local SQLite database (configuration database) 22. SQLite is a lightweight, ACID-compliant relational data management system used by the application for subsequent seamless facial recognition services. After the configuration information is written to the SQLite database, the application initiates the seamless facial recognition process.
[0139] The application calls the 3D camera 21 to capture image frame sequences, which may or may not contain faces. Illustratively, when a face is present in the image frame sequence, three image frame sequences are output: a color image frame sequence, an infrared image frame sequence, and a depth image frame sequence. These three image frame sequences are sent to the face acquisition module 23, which then sends them to the face selection module 24.
[0140] The face selection module 24 performs quality detection on each image frame in the acquired image frame sequence containing faces. The quality detection includes aspects such as the occlusion range of the face, the blur range, the illumination range, the pose angle range, and the face size. Illustratively, after detecting the quality of the image frames, the face selection module 24 outputs face image frame A, face image frame B, and face image frame C. The face selection module 24 then sends the output image frames to the face liveness detection module.
[0141] The face liveness detection module 25 is used to perform liveness detection on the selected face image frames. This embodiment uses depth images as an example for liveness detection. Liveness detection refers to a method for determining the true physiological characteristics of an object in some identity verification scenarios. In face recognition applications, liveness detection can verify whether the object is a real, living person by using combined actions such as blinking, opening the mouth, shaking the head, and nodding, and by employing technologies such as facial landmark localization and face tracking. It can effectively resist common attack methods such as photos, face swapping, facial occlusion, and screen re-photographing, thereby helping users identify fraudulent behavior and protecting users' personal information security. Liveness detection is a relatively mature technology in this field, and this application embodiment does not limit the method of liveness detection.
[0142] Extracting facial features from liveness-detected face image frames involves extracting temporal facial features from the image frame sequence. When the temporal facial features meet the face recognition criteria, the facial features in the image frame sequence are used for identity verification. Feature extraction methods include geometric feature extraction and template matching extraction, thereby transforming facial image information into a feature string that uniquely identifies a user. For example, facial feature data 'a' is obtained from image A.
[0143] After facial time-series feature extraction, facial feature data is cached in facial feature cache pool 26. Facial time-series features satisfying the facial recognition conditions include at least one of the following: 1. User Identity Verification (Continuous Same Person): This is illustrative. If there is no facial feature data 'a' for user A in the facial feature data, when facial feature data 'a' is extracted, it is stored in the facial feature database. When facial feature data 'b' is extracted from the same image frame sequence, facial feature data 'a' and facial feature data 'b' are compared. If the similarity between the two faces reaches a threshold, then the faces corresponding to facial feature data 'a' and facial feature data 'b' belong to the same face. A counter is incremented by 1 to accumulate the number of image frames belonging to the same face. When the number of image frames belonging to the same face reaches a first counting threshold, facial feature identification is performed. The first counting threshold is related to configuration information.
[0144] 2. Face Region Determination (Continuous Enlargement): Illustratively, a face feature database stores face feature data 'a', which corresponds to the area of a first face region. When subsequent image frames showing a face are captured, the area of a second face region in the image frame is compared with the area of the first face region. If the area of the second face region is larger than the area of the first face region, the face region is continuously enlarging, indicating that a user is approaching the POS device 111. The continuous enlargement of the face region is determined by comparing the face region areas in multiple consecutive image frames. The number of image frames with continuously enlarging face regions is accumulated. When the number of image frames with continuously enlarging face regions reaches a second counting threshold, facial recognition is performed. The second counting threshold is related to configuration information.
[0145] 3. Attention Determination (Continuous Gazing): Indicatively, the Openface facial recognition framework is used to detect eye gaze. The direction of the user's gaze is compared across multiple consecutive image frames to determine if the gaze is continuous. For example, if the gaze direction in the j-th and j+1-th image frames points in the same direction (in this embodiment, the gaze direction is towards the POS device, i.e., the image frame captured when the user is facing the POS device 111), it indicates that the user is gazing. When the gaze direction in multiple consecutive image frames all points in the same direction, it indicates that the user is in a continuous gaze state. The more consecutive image frames, the longer the gaze remains in a gaze state. The number of image frames with gaze directions pointing in the same direction is accumulated. When the number of image frames with gaze directions pointing in the same direction reaches a third counting threshold, facial recognition is performed. This third counting threshold is related to configuration information.
[0146] When the above facial recognition conditions are met, the POS device 111 sends the facial time-series features to the server 120. The server 120 performs identity recognition based on the received facial time-series features. The facial payment service pre-stores the correspondence between the user's corresponding payment account and the user's corresponding facial time-series features. Based on the correspondence and facial time-series features, the server determines the user's identity (payment account) and then makes payment for the purchase order through that payment account.
[0147] The following are device embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the device embodiments of this application, please refer to the method embodiments of this application.
[0148] Figure 6 A block diagram of a face recognition device provided in an exemplary embodiment of this application is shown. The device includes the following parts: Acquisition module 610 is used to acquire image frame sequences; The extraction module 620 is used to extract face time series features from the image frame sequence in response to the appearance of a face in the image frame sequence. The face time series features are the face features contained in consecutive image frames. The processing module 630 is used to perform identity recognition on the facial features contained in the image frame sequence in response to the facial time series features meeting the facial recognition conditions.
[0149] In an optional embodiment, the facial time-series features satisfying the facial recognition conditions include at least one of the following: Facial features in a time-series facial feature belong to the same face; The area of the face region in the time series features of the face continues to increase; The human eyes in the face time series features are in a state of continuous fixation.
[0150] In an optional embodiment, the acquisition module 610 is used to acquire the first image frame in the image frame sequence in which a face appears, and the first image frame includes a first face feature; The processing module 630 is configured to, in response to the matching of a second face feature with a first face feature in n consecutive image frames following the first image frame in the image frame sequence, accumulate the number n of image frames whose face features belong to the same face, where n is a positive integer; and in response to the number of image frames whose face features belong to the same face reaching a first counting threshold, determine that the face features in the face time series features belong to the same face.
[0151] In an optional embodiment, the acquisition module 610 is used to obtain the similarity between the first facial feature and the second facial feature; The processing module 630 is used to determine that the second facial feature matches the first facial feature in response to the similarity reaching a similarity threshold.
[0152] In an optional embodiment, the acquisition module 610 is used to acquire the j-th image frame and the (j+1)-th image frame in the image frame sequence in which a face appears. The j-th image frame contains a first face region, and the (j+1)-th image frame contains a second face region. The first face region corresponds to a first face region area, and the second face region corresponds to a second face region area. j is a positive integer. The processing module 630 is configured to, in response to the second face region area being greater than the first face region area, accumulate the number of image frames in which the face region area continuously increases; repeat the steps of acquiring the j-th image frame and the (j+1)-th image frame, and the step of accumulating the number of image frames in which the face region area continuously increases; and, in response to the number of image frames in which the face region area continuously increases reaching a second counting threshold, determine that the face region area in the face time series features is continuously increasing.
[0153] In an optional embodiment, the acquisition module 610 is used to acquire image frames in the image frame sequence in which a face appears, and the image frames in which a face appears include the direction of the gaze of the human eye; the processing module is used to, in response to the fact that the gaze direction included in n consecutive image frames in the image frame sequence belongs to the frontal gaze direction, accumulate the number n of image frames in which the human eye is in a continuous gaze state, where n is a positive integer; in response to the number of image frames in which the human eye is in a continuous state reaching a third counting threshold, determine that the human eye in the face time series features is in a continuous state.
[0154] In an optional embodiment, the device includes a reading module 640, a first transmitting module 650, and a first receiving module 660; The reading module 640 is used to read the device identifier of the terminal; The first sending module 650 is used to send an information acquisition request to the server, the information acquisition request carrying a device identifier; The first receiving module 660 is used to receive configuration information sent by the server. The configuration information is obtained by the server based on the device identifier and the association relationship. The association relationship is used to characterize the correspondence between the device identifier and the configuration information. The configuration information includes face recognition conditions.
[0155] In summary, the device in this embodiment continuously acquires image frame sequences. When a face appears in the image frame sequence, it performs identity recognition based on the extracted time-series features of the face that meet the face recognition conditions. This eliminates the need for the user to manually select an identity recognition method; as long as the user's face is in front of the camera, seamless identity recognition is achieved, improving the efficiency and accuracy of face recognition and enabling the terminal to accurately identify the user's identity. For example, in unmanned convenience stores, when a user is carrying a large number of items (with both hands occupied), the self-checkout equipment can use the face recognition method provided in this embodiment to identify the user and complete seamless payment for the goods, eliminating the need for manual operation and improving the convenience of payment.
[0156] In this embodiment, when the facial features in the facial time series features meet the facial recognition conditions, including at least one of the following: the facial features belong to the same face, the facial area is continuously increasing, and the eyes are in a continuous gaze state, the facial recognition process is initiated, enabling the terminal to achieve seamless recognition of identity information based on the above conditions.
[0157] The apparatus in this embodiment further matches the facial features in the first image frame where a face appears with the facial features in the next n consecutive image frames, accumulating the number of image frames where the facial features belong to the same face. When the number of image frames where the facial features belong to the same face reaches a first counting threshold, it is determined that the facial features belong to the same face. This improves the accuracy of face recognition and avoids misidentification of faces.
[0158] The device in this embodiment also determines whether the first facial feature and the second facial feature match by judging whether the similarity between the first facial feature and the second facial feature reaches a similarity threshold, thereby improving the recognition accuracy of facial recognition and avoiding misrecognition of faces.
[0159] The device in this embodiment further compares the area of the face region in any two consecutive image frames, accumulates image frames where the face region area continuously increases, and determines that the face region area is continuously increasing when the number of image frames with continuously increasing face region areas reaches a second counting threshold. This improves the accuracy of face recognition and avoids misidentification of faces.
[0160] In this embodiment, the device accumulates the number of image frames in which the human eye is in a continuous gaze state when the gaze direction in n consecutive image frames is in a forward gaze direction. When the number of image frames in which the human eye is in a continuous gaze state reaches a third counting threshold, it is determined that the human eye is in a continuous gaze state. This improves the accuracy of face recognition and avoids misidentification of faces.
[0161] The device in this embodiment also obtains a device identifier from a server. The server then obtains the terminal's configuration information based on the device identifier, enabling the terminal to perform seamless facial recognition based on the configuration information. Using a unique string serial number to identify the terminal as the device identifier distinguishes each terminal, allowing the server to set corresponding configuration information for each terminal.
[0162] Figure 7 A block diagram of a face recognition device provided in another exemplary embodiment of this application is shown. The device includes the following parts: The second receiving module 710 is used to receive face time-series features sent by the terminal. The face time-series features are face features contained in consecutive image frames. The recognition module 720 is used to identify individuals based on facial time-series features.
[0163] In an optional embodiment, the apparatus includes a second transmitting module 730 and an acquiring module 740; The second sending module 730 is used to receive an acquisition request sent by the terminal, the acquisition request carrying a device identifier; The acquisition module 740 is used to determine the association relationship, which represents the correspondence between the device identifier and the configuration information, and the configuration information includes face recognition conditions; and to acquire the configuration information based on the device identifier and the association relationship. The second sending module 730 is used to send configuration information to the terminal.
[0164] In an optional embodiment, the device includes a payment module 750; The identification module 720 is used to obtain the payment account based on the identification result obtained from the identity recognition. The payment module 750 is used to pay for the purchase order according to the payment account and obtain a payment success result; The second sending module 730 is used to send the payment success result to the terminal.
[0165] In summary, the device provided in this embodiment receives facial time-series features sent by the cash register device through a server, determines the user's identity based on the facial time-series features, enables the terminal to achieve seamless recognition of the user's identity information, and improves the efficiency of facial recognition.
[0166] The device provided in this embodiment also obtains a device identifier from a server. The server then uses the device identifier to obtain the terminal's configuration information, enabling the terminal to perform non-contact facial recognition based on the configuration information. Using a unique string serial number to identify the terminal as the device identifier ensures that each terminal is distinguishable, and the server can configure corresponding configuration information for each terminal.
[0167] The device provided in this embodiment also sends a payment success result to the terminal through the server, enabling the terminal to complete the settlement process of the purchase order and realize seamless payment for users.
[0168] Figure 8 This illustration shows a schematic diagram of a server provided in an exemplary embodiment of this application. The server may be as follows: Figure 1 The server 120 in the computer system 100 shown here specifically includes the following components.
[0169] Server 800 includes a Central Processing Unit (CPU) 801, a system memory 804 including Random Access Memory (RAM) 802 and Read Only Memory (ROM) 803, and a system bus 805 connecting the system memory 804 and the CPU 801. Server 800 also includes a basic input / output system (I / O system) 806 that facilitates the transfer of information between various devices within the computer, and a mass storage device 807 for storing the operating system 813, application programs 814, and other program modules 815.
[0170] The basic input / output system 806 includes a display 808 for displaying information and an input device 809 for user input, such as a mouse or keyboard. Both the display 808 and the input device 809 are connected to the central processing unit 801 via an input / output controller 810 connected to the system bus 805. The basic input / output system 806 may also include the input / output controller 810 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 810 also provides output to a display screen, printer, or other types of output devices.
[0171] Mass storage device 807 is connected to central processing unit 801 via a mass storage controller (not shown) connected to system bus 805. Mass storage device 807 and its associated computer-readable media provide non-volatile storage for server 800. That is, mass storage device 807 may include computer-readable media (not shown) such as hard disk or compact disc read-only memory (CD-ROM) drives.
[0172] Computer-readable media can include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, digital versatile optical disc (DVD), or solid-state drives (SSD), other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Random access memory can include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 804 and the mass storage device 807 mentioned above can be collectively referred to as memory.
[0173] According to various embodiments of this application, server 800 can also be connected to a remote computer on a network, such as the Internet. That is, server 800 can be connected to network 812 via network interface unit 811 connected to system bus 805, or it can also use network interface unit 811 to connect to other types of networks or remote computer systems (not shown).
[0174] The aforementioned memory also includes one or more programs, which are stored in the memory and configured to be executed by the CPU.
[0175] Figure 9 This illustration shows a structural block diagram of a computer device 900 provided in an exemplary embodiment of this application. The computer device 900 may be: a cash register device, an identity verification device, a smart camera, a smartphone, a tablet computer, a laptop computer, or a desktop computer. The computer device 900 may also be referred to as a user device, a portable computer device, a laptop computer device, a desktop computer device, or other names. For example, the computer device may be... Figure 1 Terminal 110 is shown.
[0176] Typically, computer device 900 includes a processor 901 and a memory 902.
[0177] Processor 901 may include one or more processing cores, such as a 9-core processor, an 8-core processor, etc. Processor 901 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 901 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 901 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content required to be displayed on the screen. In some embodiments, processor 901 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.
[0178] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 902 are used to store at least one instruction, which is executed by the processor 901 to implement the face recognition method provided in the method embodiments of this application.
[0179] In some embodiments, the computer device 900 may optionally include a peripheral device interface 903 and at least one peripheral device. The processor 901, memory 902, and peripheral device interface 903 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 903 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 904, a display screen 905, a camera assembly 906, an audio circuit 907, and a power supply 909.
[0180] Peripheral device interface 903 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 901 and memory 902. In some embodiments, processor 901, memory 902 and peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 901, memory 902 and peripheral device interface 903 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.
[0181] The radio frequency (RF) circuit 904 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 904 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 904 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 904 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 904 can communicate with other computer devices through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: metropolitan area networks (MANs), various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks (WLANs), and / or Wi-Fi (Wireless-Fidelity) networks. In some embodiments, the RF circuit 904 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.
[0182] Display screen 905 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 905 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 901 for processing. In this case, display screen 905 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 905, positioned as the front panel of the computer device 900; in other embodiments, there may be at least two display screens 905, respectively positioned on different surfaces of the computer device 900 or in a folded design; in still other embodiments, display screen 905 may be a flexible display screen, positioned on a curved or folded surface of the computer device 900. Furthermore, display screen 905 may be configured as a non-rectangular, irregular shape, i.e., a non-rectangular screen. Display screen 905 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).
[0183] The camera assembly 906 is used to acquire images or videos. Optionally, the camera assembly 906 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the computer device, and the rear-facing camera is located on the back of the computer device. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 906 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.
[0184] The audio circuit 907 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 901 for processing, or input to the radio frequency circuit 904 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located in a different part of the computer device 900. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 907 may also include a headphone jack.
[0185] Power supply 909 is used to supply power to the various components in computer device 900. Power supply 909 can be AC power, DC power, a disposable battery, or a rechargeable battery. When power supply 909 includes a rechargeable battery, the rechargeable battery can support wired or wireless charging. The rechargeable battery can also be used to support fast charging technology.
[0186] Those skilled in the art will understand that Figure 9 The structure shown does not constitute a limitation on the computer device 900, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.
[0187] Embodiments of this application also provide a computer device, which includes a processor and a memory. The memory stores at least one instruction, at least one program, code set, or instruction set. The at least one instruction, at least one program, code set, or instruction set is loaded and executed by the processor to implement the face recognition method described in the above embodiments.
[0188] Embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the face recognition method described above.
[0189] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the face recognition method as described in the above embodiments.
[0190] It should be understood that "multiple" as used in this article refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.
[0191] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.
[0192] The above description is merely an optional embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.
Claims
1. A face recognition method, characterized in that, The method includes: Read the device identifier of the terminal; send an information acquisition request to the server, the information acquisition request carrying the device identifier; receive configuration information sent by the server, the configuration information being obtained by the server based on the device identifier and the association relationship, the association relationship being used to characterize the correspondence between the device identifier and the configuration information, the configuration information including face recognition conditions; Acquire image frame sequences; In response to the appearance of a face in the image frame sequence, quality detection is performed on each image frame in the image frame sequence containing the face to obtain a preferred face image frame. The quality detection includes at least one of the following: the range of face occlusion, the range of blur, the range of illumination, the range of pose angle, and the face size. Liveness detection is performed on the preferred face image frame to obtain a detected face image frame. Face time-series features are extracted from the detected face image frame, where the face time-series features are face features contained in consecutive image frames. Obtain the first image frame from the detected face image frames, wherein the first image frame includes a first face feature; In response to a second face feature in n consecutive image frames following the first image frame after the detected face image frame matching the first face feature, the cumulative number n of image frames whose face features belong to the same face is calculated, where n is a positive integer; the cumulative number of image frames whose face features belong to the same face does not include the first image frame. When the facial features in any of the n consecutive image frames do not match the first facial features, the count of image frames containing the second facial features is restarted. In response to the number of image frames whose facial features belong to the same face reaching a first counting threshold, it is determined that the facial features in the facial time series features belong to the same face, and the facial time series features satisfy the facial recognition conditions; In response to the face time series features satisfying the face recognition conditions, the face features contained in the image frame sequence are used for identity recognition.
2. The method according to claim 1, characterized in that, The facial time-series features satisfying the facial recognition conditions also include at least one of the following: The area of the face region in the face time series features continues to increase; The human eyes in the facial time-series features are in a state of continuous fixation.
3. The method according to claim 1, characterized in that, The method further includes: Obtain the similarity between the first facial feature and the second facial feature; In response to the similarity reaching a similarity threshold, it is determined that the second facial feature matches the first facial feature.
4. The method according to claim 2, characterized in that, The method further includes: Obtain the j-th image frame and the (j+1)-th image frame in the image frame sequence in which a face appears. The j-th image frame contains a first face region, and the (j+1)-th image frame contains a second face region. The first face region corresponds to a first face region area, and the second face region corresponds to a second face region area. j is a positive integer. In response to the second face region area being larger than the first face region area, the number of image frames in which the face region area continuously increases is accumulated; Repeat the steps described above for obtaining the j-th image frame and the (j+1)-th image frame, as well as the step of accumulating the number of image frames in which the face region area continuously increases; When the number of image frames in which the face region area continues to increase reaches a second counting threshold, it is determined that the face region area in the face time series features continues to increase.
5. The method according to claim 2, characterized in that, The method further includes: Obtain image frames in the image frame sequence in which a human face appears, wherein the image frames in which a human face appears include the direction of the human eye's gaze; In response to the fact that the gaze direction of n consecutive image frames in the image frame sequence belongs to the frontal gaze direction, the number of image frames in which the human eye is in the continuous gaze state is accumulated, n, where n is a positive integer; In response to the number of image frames in which the human eye is in the sustained gaze state reaching a third counting threshold, it is determined that the human eye in the face time series features is in the sustained gaze state.
6. A face recognition device, characterized in that, The device includes: The reading module is used to read the device identifier of the terminal. The first sending module is used to send an information acquisition request to the server, wherein the information acquisition request carries the device identifier; The first receiving module is used to receive configuration information sent by the server. The configuration information is obtained by the server based on the device identifier and the association relationship. The association relationship is used to characterize the correspondence between the device identifier and the configuration information. The configuration information includes face recognition conditions. The acquisition module is used to acquire image frame sequences; An extraction module is configured to, in response to the appearance of a face in the image frame sequence, perform quality detection on each image frame in the image frame sequence containing a face to obtain a preferred face image frame. The quality detection includes at least one of the following: the range of face occlusion, the range of blur, the range of illumination, the range of pose angle, and the size of the face. Liveness detection is performed on the preferred face image frame to obtain a detected face image frame. Facial time-series features are extracted from the detected face image frame from the image frame sequence. The facial time-series features are facial features contained in consecutive image frames. The acquisition module is used to acquire the first image frame in the detected face image frame, and the first image frame includes a first face feature; The processing module is configured to respond to a second face feature in n consecutive image frames following the first image frame in the detected face image frame matching the first face feature, and to accumulate the number n of image frames whose face features belong to the same face, where n is a positive integer; the accumulated number of image frames whose face features belong to the same face does not include the first image frame; The module is used to perform the following steps: when the facial feature in any of the n consecutive image frames does not match the first facial feature, the count of image frames containing the second facial feature is restarted; The processing module is further configured to, in response to the number of image frames whose facial features belong to the same face reaching a first counting threshold, determine that the facial features in the facial time series features belong to the same face, and that the facial time series features satisfy the facial recognition conditions; The processing module is further configured to perform identity recognition on the facial features contained in the image frame sequence in response to the facial time series features satisfying the facial recognition conditions.
7. The apparatus according to claim 6, characterized in that, The facial time-series features satisfying the facial recognition conditions also include at least one of the following: The area of the face region in the face time series features continues to increase; The human eyes in the facial time-series features are in a state of continuous fixation.
8. The apparatus according to claim 6, characterized in that, The acquisition module is used to obtain the similarity between the first facial feature and the second facial feature; The processing module is used to determine that the second facial feature matches the first facial feature in response to the similarity reaching a similarity threshold.
9. The apparatus according to claim 7, characterized in that, The acquisition module is used to acquire the j-th image frame and the (j+1)-th image frame in the image frame sequence in which a face appears. The j-th image frame contains a first face region, and the (j+1)-th image frame contains a second face region. The first face region corresponds to a first face region area, and the second face region corresponds to a second face region area. j is a positive integer. The processing module is configured to, in response to the second face region area being greater than the first face region area, accumulate the number of image frames in which the face region area continuously increases; The processing module is used to repeat the steps of acquiring the j-th image frame and the (j+1)-th image frame, and to accumulate the number of image frames in which the face region area continuously increases. The processing module is configured to determine that the face region area in the face time series features is continuously increasing in response to the number of image frames in which the face region area is continuously increasing reaching a second counting threshold.
10. The apparatus according to claim 7, characterized in that, The acquisition module is used to acquire image frames in the image frame sequence in which a human face appears, and the image frames in which a human face appears include the direction of the human eye's gaze; The processing module is configured to, in response to the fact that the gaze direction of n consecutive image frames in the image frame sequence belongs to the frontal gaze direction, accumulate the number n of image frames in which the human eye is in the continuous gaze state, where n is a positive integer; The processing module is configured to determine that the human eye is in the continuous gaze state in the face time series features in response to the number of image frames in which the human eye is in the continuous gaze state reaching a third counting threshold.
11. A computer device, characterized in that, The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the face recognition method as described in any one of claims 1 to 5.
12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the face recognition method as described in any one of claims 1 to 5.
13. A computer program product comprising computer instructions stored in a computer-readable storage medium, wherein a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the face recognition method as described in any one of claims 1 to 5.
Citation Information
Patent Citations
Face image collection method and device and electronic equipment
CN111291737A
Payment method and device
US20200294060A1
Facial recognition method and device
WO2020038140A1
