Real-time face recognition terminal based on edge computing and fast matching method

Abstract

The application belongs to the technical field of face recognition, and discloses a real-time face recognition terminal based on edge computing and a fast matching method, which comprises a reconfigurable sensing front end, the output end of the reconfigurable sensing front end is connected with a reconfigurable computing array on a chip, the reconfigurable computing array on the chip is connected with a feature routing network on the chip; further comprising an environment perception unit, the output end of the environment perception unit is connected with a quality evaluation module, the output end of the quality evaluation module is connected with the feature routing network on the chip; further comprising a communication module, the communication module is connected with the reconfigurable computing array on the chip; further comprising a power management module. Through hardware dynamic on-demand reconfiguration and intelligent routing scheduling, combined with a two-level fast matching mechanism and a space-time context fingerprint, the application realizes the balance of high-precision recognition, fast response, low-power operation and anti-fake ability in a single system for the first time in a complex real scene.

Description

Technical Field

[0001] This invention relates to the field of facial recognition technology, and in particular to a real-time facial recognition terminal and a fast matching method based on edge computing. Background Technology

[0002] With the deep integration of IoT and AI technologies, edge computing has been widely adopted as a new computing paradigm. The core idea of ​​edge computing is to offload data processing and analysis tasks from the traditional cloud center to the network edge, performing computations closer to the data generation source (such as cameras, sensors, and other terminal devices). This significantly reduces network bandwidth requirements, decreases service response latency, and enhances the privacy and reliability of data processing. In the field of facial recognition, the introduction of edge computing has given rise to "edge facial recognition" technology. This technology directly performs key steps such as face detection and feature extraction on the acquisition terminal (e.g., a camera) or its nearby dedicated computing device, uploading only a small amount of feature data or recognition results to the cloud. This represents an architectural evolution from "cloud recognition" to "edge recognition," providing an effective technical path for scenarios requiring low latency and high concurrency response, such as access control, security monitoring, and smart retail.

[0003] However, existing edge computing-based facial recognition solutions still face several pressing technical challenges. At the terminal hardware level, most edge facial recognition terminals employ a "fixed sensor + general-purpose AI acceleration chip" architecture. Their computing resources are statically allocated, unable to dynamically adjust based on the specific quality of the facial image (such as partial occlusion, uneven lighting, or pose changes). This leads to either indiscriminate depth calculations across all image regions in complex real-world scenarios, resulting in resource waste and increased power consumption, or insufficient computing power sacrificing recognition accuracy. Secondly, at the recognition method level, existing solutions largely rely on single, static facial features for matching, lacking effective utilization of dynamic information such as individual micro-motion characteristics between consecutive frames. This makes recognition features easily forged (e.g., from high-definition photos or video playback) and exhibits poor robustness against brief occlusions. Summary of the Invention

[0004] The present invention aims to provide a real-time face recognition terminal and a fast matching method based on edge computing to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A real-time face recognition terminal based on edge computing includes a reconfigurable sensing front end, the output of which is connected to an on-chip reconfigurable computing array, and the on-chip reconfigurable computing array is connected to an on-chip feature routing network. An environmental sensing unit, the output of which is connected to a quality assessment module, and the output of which is connected to the on-chip feature routing network; A communication module is connected to the on-chip reconfigurable computing array; The power management module provides power to the reconfigurable sensing front end, the on-chip reconfigurable computing array, the on-chip feature routing network, the environmental perception unit, the quality assessment module, and the communication module.

[0006] An edge server; the edge server includes a Bloom filter and a feature database.

[0007] Preferably, the reconfigurable sensing front end includes a lens array and an image sensor, wherein the lens array is an electrically controlled liquid lens array.

[0008] Preferably, the electrically controlled liquid lens array consists of multiple liquid lens units, and the curvature of each liquid lens unit can be independently adjusted.

[0009] Preferably, the on-chip reconfigurable computing array includes a face detection network, a face alignment module, a local feature extraction unit, and a spatiotemporal feature fusion unit that can be dynamically reconfigured on hardware.

[0010] Preferably, the local feature extraction unit includes a left eye feature extractor, a right eye feature extractor, a nose feature extractor, and a mouth feature extractor arranged in parallel.

[0011] Preferably, it further includes an inertial measurement unit, the output of which is connected to the input of the on-chip reconfigurable computing array.

[0012] A fast face matching method based on edge computing is also provided, including the following steps: S1: The image sequence is acquired through the reconfigurable sensing front end of the terminal and processed in its on-chip reconfigurable computing array to generate the spatiotemporal context fingerprint of the target face and the corresponding binary hash code. S2: The binary hash code is sent to the edge server through the communication module of the terminal; S3: The edge server uses its Bloom filter to perform preliminary screening on the received binary hash codes to obtain a set of candidate identities; S4: The edge server extracts a subset of local features corresponding to the candidate identity from its feature database based on the quality mask uploaded by the terminal, and calculates the Hamming distance between the subset and the local features uploaded by the terminal. S5: Based on the similarity between the Hamming distance and the binary hash code, determine the final matching identity.

[0013] Preferably, the generation of the spatiotemporal context fingerprint in step S1 includes: extracting single-frame static face features; extracting micro-temporal dynamic behavior features; and fusing the static face features with the dynamic behavior features to generate the spatiotemporal context fingerprint.

[0014] Preferably, the step S4 of extracting the local feature subset corresponding to the candidate identity specifically involves: extracting only the feature blocks of the corresponding region as the local feature subset from the feature database based on the high-quality facial region identified by the quality mask.

[0015] Preferably, step S11 is included before step S1: environmental parameters are collected by the environmental sensing unit, and the quality assessment module generates a control signal based on the parameters to adjust the acquisition mode of the reconfigurable sensing front end.

[0016] The beneficial effects of this technical solution compared to existing technologies are as follows: (1) This solution achieves dynamic on-demand scheduling of hardware computing resources by introducing a reconfigurable sensing front-end, an on-chip reconfigurable computing array, and an on-chip feature routing network in a collaborative design. Specifically, the quality assessment module can generate a quality mask based on parameters such as illumination and distance collected by the environmental perception unit. The routing network dynamically activates or bypasses the corresponding local feature extractors in the on-chip computing array based on the mask and the key point coordinates generated after face alignment. This mechanism enables the system to accurately focus limited computing resources on high-quality, unoccluded areas of the face for deep feature extraction, while avoiding redundant computation on low-quality or invalid areas. Compared with existing edge terminals that use a fixed computing mode, this solution significantly improves the feature extraction accuracy and robustness in complex scenarios (such as side faces, partial occlusion, and uneven illumination) under the same computing power, while achieving a higher energy efficiency ratio.

[0017] (2) This scheme proposes a two-stage fast matching method that integrates static and dynamic features: "spatiotemporal context fingerprint" and "Bloom filter coarse screening plus local feature fine matching". The fingerprint generated by the terminal not only contains single-frame static features, but also integrates micro-temporal dynamic behavior features extracted based on continuous frames or inertial data, making the features more unique and difficult to be statically counterfeited. In the matching stage, the edge server first uses a Bloom filter to perform millisecond-level coarse screening on the binary hash code, quickly eliminating a large number of non-candidate identities; then, for only the few candidate identities obtained from the coarse screening, based on the quality mask uploaded by the terminal, it extracts the local feature subset of the corresponding high-quality region from the feature database for Hamming distance fine matching. Compared with the existing technology of directly uploading the complete feature vector and performing global comparison, this method greatly reduces the network transmission volume and server computing overhead, while ensuring extremely high recognition accuracy, achieving sub-second real-time matching response, and effectively resisting forgery attacks such as photos and screen re-photographing. Attached Figure Description

[0018] Figure 1 The system general diagram provided for this invention; Figure 2 Flowchart of the fast matching method provided by the present invention; Reference numerals: 100, Real-time face recognition terminal; 101, Reconfigurable sensing front end; 1011, Lens array; 1012, Image sensor; 102, On-chip reconfigurable computing array; 1021, Face detection network; 1022, Face alignment module; 1023, Local feature extraction unit; 1023a, Left eye feature extractor; 1023b, Right eye feature extractor; 1023c, Nose feature extractor; 1023d, Mouth feature extractor; 1024, Spatiotemporal feature fusion unit; 103, On-chip feature routing network; 104, Environmental perception unit; 105, Quality assessment module; 106, Communication module; 107, Power management module; 108, Inertial measurement unit; 200, Edge server; 201, Bloom filter; 202, Feature database; Detailed Implementation The present invention will now be described in further detail with reference to the accompanying drawings and embodiments: like Figure 1 The illustrated real-time face recognition terminal based on edge computing includes a real-time face recognition terminal 100 and an edge server 200. The terminal 100, acting as a data acquisition and front-end processing unit, establishes a network connection with the edge server 200 through its communication module 106. The edge server 200 has a built-in Bloom filter 201 and a feature database 202, responsible for efficient feature matching and identity determination.

[0019] The specific connection relationships of the various functional modules within terminal 100 are as follows: Figure 1 As shown, this constitutes a tightly coupled on-chip processing system: The output of the reconfigurable sensing front end 101 is electrically connected to the input of the on-chip reconfigurable computing array 102, through which raw image data is input.

[0020] The on-chip reconfigurable computing array 102 is interconnected with the on-chip feature routing network 103 to form a bidirectional control and data interaction path.

[0021] The output of the environmental sensing unit 104 is electrically connected to the input of the quality assessment module 105; the output of the quality assessment module 105 is electrically connected to the input of the on-chip feature routing network 103, and the control signal generated by it is fed back to the reconfigurable sensing front end 101.

[0022] The output of the inertial measurement unit 108 is electrically connected to the input of the on-chip reconfigurable computing array 102.

[0023] The communication module 106 is electrically connected to the on-chip reconfigurable computing array 102. The power management module 107 supplies power to all of the above modules.

[0024] After the system starts up, the environmental perception unit 104 continuously monitors parameters such as illumination and distance of the deployment scene. The quality assessment module 105 analyzes these parameters and generates control commands to dynamically adjust the reconfigurable sensing front end 101. For example, when the light is insufficient, the command can increase the gain of the image sensor 1012 and turn on the fill light; for multi-person wide-angle scenes, the command controls the electronically controlled liquid lens array 1011 to switch to wide-angle mode to ensure that the quality of the acquired basic image is optimal.

[0025] Once a face is captured, the on-chip reconfigurable computing array 102 is first configured as a face detection network 1021 to quickly locate all face regions in the image. Subsequently, the same hardware resource is dynamically reconfigured as a face alignment module 1022 to precisely locate key points such as the corners of the eyes and the tip of the nose for each face.

[0026] The matrix of sharpness and reliability of each region in the quality mask image generated in real time by the key point coordinate and quality assessment module 105 is sent to the on-chip feature routing network 103. Based on this, the on-chip feature routing network 103 makes real-time decisions and schedules the resources of the on-chip reconfigurable computing array 102. If the quality mask shows that the right cheek region of a face has low quality due to shadows, the routing network will instruct the computing array to activate only the dedicated extractors such as 1023a and 1023c in the local feature extraction unit 1023 corresponding to high-quality regions such as the left eye and nose in the following feature extraction cycle, and skip the calculation of the right cheek region, thereby achieving accurate and efficient use of computing resources.

[0027] Local features extracted from high-quality regions, along with micro-motion temporal data such as natural head tremors acquired synchronously from the inertial measurement unit 108, are input into a spatiotemporal feature fusion unit 1024 formed by dynamic reconstruction. The fusion unit encodes and fuses spatial static features and temporal dynamic features to generate a spatiotemporal context fingerprint with both recognizability and anti-spoofing capabilities, and further encodes it into a fixed-length binary hash code.

[0028] Terminal 100 uploads the generated binary hash code to edge server 200 via communication module 106. Edge server 200 uses its Bloom filter 201 to perform an instant query on the hash code. The Bloom filter can determine whether the hash code "definitely does not exist" or "may exist" in the registry with extremely high space efficiency and constant time complexity. This step can quickly filter out most irrelevant identities and output a very small set of candidate identities.

[0029] For each identity entering the candidate set, terminal 100 uploads its corresponding local feature vector and quality mask. Edge server 200, based on this quality mask, extracts a subset of features from its feature database 202, focusing on the same high-quality regions for each candidate identity. Subsequently, it calculates the Hamming distance between the local features uploaded by the terminal and the feature subset in the database. Finally, edge server 200 combines the coarse-screen confidence of the hash code with the precise measure of the Hamming distance, applies a preset threshold rule to determine the final matched identity, and returns the result to terminal 100 to execute the corresponding control action.

[0030] like Figure 2 The example shown is a fast face matching method based on edge computing, such as... Figure 1 As shown, it also includes an edge server 200, which includes a Bloom filter 201 and a feature database 202, and its steps are as follows: S11: Environmental parameters are collected by the environmental sensing unit 104, and the quality assessment module 105 generates a control signal based on the parameters to adjust the acquisition mode of the reconfigurable sensing front end 101.

[0031] S1: The image sequence is acquired by the reconfigurable sensing front end 101 of the terminal and processed in its on-chip reconfigurable computing array 102 to generate the spatiotemporal context fingerprint of the target face and the corresponding binary hash code. Generating a spatiotemporal context fingerprint includes: extracting static face features from a single frame; extracting micro-temporal dynamic behavior features; and fusing the static face features with the dynamic behavior features to generate the spatiotemporal context fingerprint.

[0032] S2: The binary hash code is sent to the edge server 200 through the communication module 106 of the terminal; S3: The edge server 200 uses its Bloom filter 201 to perform preliminary screening on the received binary hash codes to obtain a set of candidate identities; S4: The edge server 200 extracts a subset of local features corresponding to the candidate identity from its feature database 202 based on the quality mask uploaded by the terminal, and performs Hamming distance calculation with the local features uploaded by the terminal. Extracting the local feature subset corresponding to the candidate identity specifically involves extracting only the feature blocks of the corresponding region from the feature database 202 as the local feature subset based on the high-quality facial region identified by the quality mask.

[0033] The above descriptions are merely embodiments of the present invention, and common knowledge such as specific technical solutions and / or characteristics are not described in detail here. It should be noted that those skilled in the art can make various modifications and improvements without departing from the technical solutions of the present invention, and these should also be considered within the scope of protection of the present invention. These modifications and improvements will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. A real-time face recognition terminal based on edge computing, characterized in that: include: A reconfigurable sensing front end (101); the output of the reconfigurable sensing front end (101) is connected to an on-chip reconfigurable computing array (102), and the on-chip reconfigurable computing array (102) is connected to an on-chip feature routing network (103). An environmental sensing unit (104) is provided; the output of the environmental sensing unit (104) is connected to a quality assessment module (105), and the output of the quality assessment module (105) is connected to the on-chip feature routing network (103). Communication module (106); the communication module (106) is connected to the on-chip reconfigurable computing array (102); Power management module (107); the power management module (107) provides power to the reconfigurable sensing front end (101), the on-chip reconfigurable computing array (102), the on-chip feature routing network (103), the environmental perception unit (104), the quality assessment module (105) and the communication module (106); Edge server (200); the edge server (200) includes a Bloom filter (201) and a feature database (202).

2. The real-time face recognition terminal based on edge computing as described in claim 1, characterized in that: The reconfigurable sensing front end (101) includes a lens array (1011) and an image sensor (1012), wherein the lens array (1011) is an electrically controlled liquid lens array.

3. A real-time face recognition terminal based on edge computing as described in claim 2, characterized in that: The electrically controlled liquid lens array (1011) consists of multiple liquid lens units, and the curvature of each liquid lens unit can be independently adjusted.

4. A real-time face recognition terminal based on edge computing as described in claim 1, characterized in that: The on-chip reconfigurable computing array (102) includes a face detection network (1021) that can be dynamically reconfigured in hardware, a face alignment module (1022), a local feature extraction unit (1023), and a spatiotemporal feature fusion unit (1024).

5. A real-time face recognition terminal based on edge computing as described in claim 4, characterized in that: The local feature extraction unit (1023) includes a left eye feature extractor (1023a), a right eye feature extractor (1023b), a nose feature extractor (1023c), and a mouth feature extractor (1023d) arranged in parallel.

6. A real-time face recognition terminal based on edge computing as described in claim 1, characterized in that: It also includes an inertial measurement unit (108), the output of which is connected to the input of the on-chip reconfigurable computing array (102).

7. A fast face matching method based on edge computing, applied to a real-time face recognition terminal based on edge computing as described in any one of claims 1 to 6, characterized in that: Includes the following steps: S1: The image sequence is acquired by the reconfigurable sensing front end (101) of the terminal and processed in its on-chip reconfigurable computing array (102) to generate the spatiotemporal context fingerprint of the target face and the corresponding binary hash code. S2: The binary hash code is sent to the edge server (200) through the communication module (106) of the terminal. S3: The edge server (200) uses its Bloom filter (201) to perform preliminary screening on the received binary hash code to obtain a set of candidate identities; S4: The edge server (200) extracts a subset of local features corresponding to the candidate identity from its feature database (202) based on the quality mask uploaded by the terminal, and performs Hamming distance calculation with the local features uploaded by the terminal; S5: Based on the similarity between the Hamming distance and the binary hash code, determine the final matching identity.

8. The fast face matching method based on edge computing as described in claim 7, characterized in that: The generation of the spatiotemporal context fingerprint in step S1 includes: extracting single-frame static face features; extracting micro-temporal dynamic behavior features; and fusing the static face features with the dynamic behavior features to generate the spatiotemporal context fingerprint.

9. The fast face matching method based on edge computing as described in claim 7, characterized in that: The step S4 of extracting the local feature subset corresponding to the candidate identity specifically involves: extracting only the feature blocks of the corresponding region as the local feature subset from the feature database (202) based on the high-quality facial region identified by the quality mask.

10. The fast face matching method based on edge computing as described in claim 7, characterized in that: Before step S1, there is also step S11: environmental parameters are collected by the environmental sensing unit (104), and the quality assessment module (105) generates a control signal based on the parameters to adjust the acquisition mode of the reconfigurable sensing front end (101).

Images