A facial recognition authentication method based on the RK3588 platform

Abstract

This invention relates to a facial recognition authentication method based on the RK3588 platform, belonging to the field of biometric authentication. The method includes: algorithm model design and selection and optimization strategies, model optimization and performance verification, refined NPU resource configuration and parallel processing strategies, and system integration and application function implementation. This invention successfully implements a facial recognition authentication method based on the RK3588 platform. This method significantly improves the real-time processing capability of facial recognition authentication on edge devices while ensuring high recognition accuracy through optimized design of RetinaFace convolutional neural network models, optimized design of MobileFaceNet convolutional neural network models, effective model quantization and transformation, refined three-core NPU resource parallel scheduling strategies, and comprehensive system integration.

Description

Technical Field

[0001] This invention belongs to the field of biometric authentication, specifically relating to a facial recognition authentication method based on the RK3588 platform. Background Technology

[0002] Facial recognition technology, with its contactless, convenient, and user-friendly nature, has become one of the core technologies in the field of biometrics. It has been widely used in many scenarios such as public safety, financial payment, and access control systems, playing a vital role in improving social management efficiency and personal convenience.

[0003] Traditional facial recognition methods often struggle to meet the accuracy and robustness requirements of practical applications under complex conditions such as varying lighting, diverse poses, differences in facial expressions, and partial occlusion. In recent years, the rapid development of deep learning technology, especially the introduction and application of advanced models such as Convolutional Neural Networks (CNNs), has led to breakthroughs in facial feature extraction and comparison capabilities. Deep learning models can automatically learn and extract more discriminative high-dimensional facial features from large-scale data, thereby significantly improving the accuracy and robustness of facial feature authentication and effectively addressing the challenges posed by complex environments.

[0004] However, high-performance deep learning models typically involve significant computational complexity and a large number of parameters. Deploying them directly in the cloud can lead to issues such as network latency, data privacy and security concerns, and high concurrency processing pressure. Therefore, deploying deep learning models on edge devices for localized real-time processing at the source of data generation has become an important development trend. To achieve efficient deep learning inference on resource-constrained edge devices, dedicated hardware acceleration units, such as Neural Processing Units (NPUs), are used to optimize and accelerate the models, thereby improving inference speed and reducing power consumption. This meets the urgent real-time processing requirements of applications such as facial recognition authentication. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] The technical problem this invention aims to solve is how to provide a facial feature authentication method based on the RK3588 platform to address issues such as network latency, data privacy and security, and high concurrency processing pressure that may arise when high-performance deep learning models are directly deployed in the cloud for processing.

[0007] (II) Technical Solution

[0008] To address the aforementioned technical problems, this invention proposes a facial recognition authentication method based on the RK3588 platform, which includes the following steps:

[0009] S1. Algorithm Model Design, Selection, and Optimization Strategies

[0010] S1.1: The face detection module is followed by the face feature extraction module, and the functions of each module and the data flow are clearly defined;

[0011] S1.2: For the face detection module, the face detection model RetinaFace was optimized and designed;

[0012] S1.3: For the face feature extraction module, the face feature extraction model MobileFaceNet was optimized and designed;

[0013] S2, Model Optimization and Performance Verification

[0014] S2.1: Quantize the face detection model RetinaFace and the face feature extraction model MobileFaceNet respectively;

[0015] S2.2: Convert the two quantized models into a specific model format adapted to the hardware acceleration of the target RK3588 platform's Neural Processing Unit (NPU);

[0016] S2.3: Use a face dataset to evaluate and verify the performance metrics of the transformed face detection model and face feature extraction model to ensure that the performance meets the requirements;

[0017] S3 and NPU resource fine-grained configuration and parallel processing strategies

[0018] S3.1: For the RetinaFace face detection model, create an NPU runtime context and configure the context to bind and utilize the two NPU cores in the RK3588 platform for collaborative computing;

[0019] S3.2: For the face feature extraction model MobileFaceNet, create an independent NPU runtime context and configure this context to bind to and utilize one of the remaining NPU cores in the RK3588 platform for computation;

[0020] S3.3: Construct a multi-threaded parallel processing pipeline to enable the parallel execution of detection and feature extraction tasks on different NPU core groups;

[0021] S4. System Integration and Application Function Implementation

[0022] Construct a database table structure to store user facial feature vector data and corresponding identity information, complete the hardware driver debugging and system adaptation of image acquisition equipment, develop upper-layer application software, and realize system integration and operation.

[0023] (III) Beneficial Effects

[0024] This invention proposes a face feature authentication method based on the RK3588 platform. This invention successfully implements a face feature authentication method based on the RK3588 platform. This method significantly improves the real-time processing capability of face authentication on edge devices while ensuring high recognition accuracy by optimizing the design of the RetinaFace convolutional neural network model, optimizing the design of the MobileFaceNet convolutional neural network model, effectively quantizing and converting the model, implementing a refined three-core NPU resource parallel scheduling strategy, and perfecting system integration. Attached Figure Description

[0025] Figure 1 This is a flowchart of the face verification system of the present invention. Detailed Implementation

[0026] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0027] This invention provides a facial feature authentication method based on the RK3588 platform. The method includes the following steps:

[0028] S1. Algorithm Model Design, Selection, and Optimization Strategies

[0029] This step aims to build an efficient and lightweight facial recognition authentication algorithm architecture, specifically including:

[0030] S1.1: This method uses a face detection module followed by a face feature extraction module, clarifying the functions of each module and the data flow.

[0031] S1.2: For the face detection module, the RetinaFace convolutional neural network model for face detection is optimized. To improve real-time performance and reduce computational load, the optimization measures of this invention include: using the lightweight MobileNetV2 as the backbone network, which significantly reduces the number of parameters and computational complexity by utilizing depthwise separable convolutions; setting the input image resolution to 640x480 pixels to achieve a balance between detection performance and processing efficiency; and simplifying the Feature Pyramid Network (FPN) structure, specifically selecting only the feature maps of three levels, P3, P4, and P5, reducing the number of channels through 1x1 convolutions, and then performing element-wise addition and fusion via a top-down path, thereby efficiently constructing a multi-scale feature representation for detecting faces at different scales.

[0032] S1.3: For the face feature extraction module, the MobileFaceNet convolutional neural network model for face feature extraction is optimized. To achieve extremely low computational and storage overhead while ensuring high recognition accuracy, the optimization measures of this invention include: fully utilizing the core advantages of MobileFaceNet, using depthwise separable convolutions to construct the network body, and replacing the traditional fully connected layers with global depthwise convolutional layers at the end of the network to maintain the lightweight characteristics of the model. Simultaneously, a 128-dimensional low-dimensional face feature vector is chosen for output. This dimension ensures that the features have sufficiently high discriminative power while significantly reducing the computational load and database storage overhead for subsequent comparisons.

[0033] S2, Model Optimization and Performance Verification

[0034] This step specifically includes:

[0035] S2.1: The face detection model RetinaFace and the face feature extraction model MobileFaceNet selected in S1 are quantized to further reduce the computational load and parameter storage requirements during model operation.

[0036] S2.2: Convert the two quantized models into a specific model format adapted to the hardware acceleration of the target RK3588 platform Neural Processing Unit (NPU).

[0037] S2.3: The face detection model and face feature extraction model transformed in S2.2 are rigorously evaluated and verified on key performance indicators such as recognition accuracy and recall using the standard public face dataset (LFW dataset) to ensure that their performance meets the intended application requirements.

[0038] S3 and NPU resource fine-grained configuration and parallel processing strategies

[0039] To achieve efficient scheduling and full utilization of the three NPU core resources built into the RK3588 platform, this step specifically includes:

[0040] S3.1: For the computationally intensive RetinaFace face detection model, create an NPU runtime context and configure this context to bind and utilize the two NPU cores in the RK3588 platform for collaborative computing to accelerate the processing speed of the face detection module.

[0041] S3.2: For MobileFaceNet, a face feature extraction model with relatively low computational cost, an independent NPU runtime context is created, and this context is configured to bind to and utilize one of the remaining NPU cores in the RK3588 platform for computation, dedicated to face feature extraction.

[0042] S3.3: Construct a multi-threaded parallel processing pipeline to sequentially feed image data into the face detection module accelerated by a dual-core NPU and the face feature extraction module accelerated by a single-core NPU. Through mechanisms such as asynchronously calling the NPU inference interface, the detection task and the feature extraction task are executed in parallel on different NPU core groups, thereby maximizing the utilization of the platform's overall NPU computing power.

[0043] S4. System Integration and Application Function Implementation

[0044] Design and construct a database table structure for storing user facial feature vector data and corresponding identity information (such as name and ID number); complete the hardware driver debugging and system adaptation for image acquisition devices (such as cameras); develop upper-layer application software that integrates all core functional modules such as facial information input, facial information query, real-time facial detection and feature extraction based on the NPU parallel processing strategy described in S3, and subsequent facial feature comparison and identity verification, ultimately forming the operational foundation for a complete facial feature identity verification method and device. The system operation flowchart is as follows: Figure 1 As shown.

[0045] Example 1:

[0046] This invention provides a facial feature authentication method based on the RK3588 platform, and the specific implementation steps are as follows:

[0047] S1: Algorithm Model Design, Selection, and Optimization Strategies

[0048] This embodiment aims to build a facial recognition authentication system that can run efficiently on the RK3588 embedded platform.

[0049] S1.1: The overall algorithm architecture is designed as a two-stage cascaded structure, with the face detection module followed by the face feature extraction module. First, face detection is performed on the input image to obtain all face regions; then, the detected face regions are cropped, aligned, and normalized, and then fed into the face feature extraction network to obtain high-discrimination face feature vectors. Finally, identity verification is completed through feature comparison.

[0050] S1.2: For the face detection module, this invention selects an optimized RetinaFace convolutional neural network model. To meet the requirements of edge computing for real-time performance and computational efficiency, the specific optimization strategy is as follows:

[0051] (1) Backbone network and input resolution: The lightweight MobileNetV2 is used as its backbone network, which can significantly reduce the number of parameters and computation compared to complex backbones such as ResNet. At the same time, considering both detection accuracy and processing speed, the image resolution input to the RetinaFace model is set to 640x480 pixels to reduce the computational burden of subsequent convolutional layers.

[0052] (2) Simplification and Output Description of Feature Pyramid Network (FPN): The FPN structure in the model is simplified and adjusted. Specifically, this invention selects only the three feature layers of different scales, P3, P4, and P5, output by the backbone network. To reduce computational overhead, a 1x1 convolutional layer is first connected after the feature map of each layer, and the number of its output channels is uniformly adjusted to a small preset value (e.g., 256). Subsequently, the size of the high-level (P5) feature map is enlarged through an upsampling operation, and element-wise addition is performed with the low-level (P4) feature map after channel adjustment, and then element-wise addition is performed with the P3 feature map, realizing multi-scale feature fusion from top to bottom. These fused feature maps are finally fed into the classification and regression head of the model to predict the location information of faces.

[0053] In this invention, the RetinaFace model acts as a face detector, and its final output is the bounding box coordinates and facial landmark coordinates for each face in the image. This output is not directly used as input to the subsequent MobileFaceNet model, but rather as the basis for the next step of data processing: the system uses this coordinate information to crop precise face region image patches from the original image and performs affine transformations using the landmarks to align the faces. These cropped and aligned face image patches are then the final input to the next stage of the MobileFaceNet model for feature extraction.

[0054] S1.3: For the face feature extraction module, this invention selects an optimized MobileFaceNet convolutional neural network model. The specific optimization and implementation methods are as follows:

[0055] (1) Efficient Network Structure Utilization: To reduce computational complexity and the number of parameters, this invention fully utilizes the core advantages of the MobileFaceNet model. The main body of the network uses depthwise separable convolutions instead of standard convolutions, and global depthwise convolutional layers are used at the network ends instead of traditional fully connected layers. This series of designs greatly reduces the computational and storage requirements of the model, making it very suitable for deployment on resource-constrained edge devices.

[0056] (2) Input Processing and Feature Vector Generation: To balance recognition accuracy with the efficiency of subsequent feature storage and comparison, this invention chooses to generate a 128-dimensional facial feature vector to represent an individual's facial information. The specific generation process is as follows:

[0057] First, the face image blocks (which have been cropped and aligned) obtained in the previous step (S1.2) are uniformly scaled to 112x112 pixels and used as the standard input for the MobileFaceNet model.

[0058] Subsequently, the 112x112 pixel image is fed into the MobileFaceNet network for forward propagation computation. After processing through a series of convolutional layers and GDC layers within the network, a one-dimensional array containing 128 floating-point values ​​is finally generated at the network's output. This is the 128-dimensional feature vector representing the face. This vector has high discriminative power in the feature space and can effectively distinguish the identities of different individuals.

[0059] S2: Model Optimization and Performance Verification

[0060] S2.1: Model Quantization Processing: To achieve efficient inference on the RK3588 platform's NPU, this invention performs quantization processing on both the face detection model RetinaFace and the face feature extraction model MobileFaceNet selected in S1. Quantization aims to convert the original FP32 operations in the models into the more hardware-efficient INT8 operations.

[0061] This invention employs the INT8 symmetric quantization scheme. The core of this scheme is the linear mapping of a floating-point number range to an integer number range; the conversion relationship can be defined by the following formula:

[0062] real_value=scale*(quantized_value-zero_point)

[0063] in:

[0064] real_value is the original FP32 floating-point value.

[0065] quantized_value is the INT8 integer value obtained after quantization.

[0066] scale is a floating-point number that defines the step size or precision of quantization.

[0067] zero_point is an integer that ensures the original floating-point number 0 can be accurately mapped to a value in the quantized integer field.

[0068] (Scale and zero point) are not trainable parameters of the model, but rather coefficients used for data type conversion. Their specific values ​​are determined during a calibration step in the quantization process. In practice, this invention prepares a small, representative calibration dataset, inputs this data into the original FP32 model, and statistically analyzes the actual data distribution range (maximum and minimum values) of the network weights and activation values ​​for each layer. Based on these statistically derived ranges, the scale value that best preserves the original data information and minimizes quantization error is calculated.

[0069] This method maps model weights and activation values ​​from FP32 to INT8, which not only significantly reduces the model's storage requirements, but more importantly, enables the model to perform efficient integer operations on the NPU, thereby greatly improving inference speed.

[0070] S2.2: Model Format Conversion: The model is first converted to ONNX format, and then the RKNN-Toolkit2 toolkit provided by Rockchip is used to convert the ONNX (Open Neural Network Exchange) format model after INT8 quantization into the .rknn model format that can be directly loaded and executed by the RK3588 platform NPU. This conversion process includes model graph optimization, operator fusion, and instruction compilation tailored to the NPU hardware characteristics.

[0071] S2.3: Model Performance Validation: The face verification accuracy of the combined model (RetinaFace detection followed by MobileFaceNet feature extraction) after transformation in S2.2 was evaluated using the publicly available LFW (Labeled Faces in the Wild) face dataset. The evaluation metric was the verification accuracy (TAR@FAR = 0.001) under the condition of a false acceptance rate (FAR) of 0.1% (0.001). The validation results are shown in Table 1.

[0072] Table 1: Comparison of face verification accuracy before and after model quantization (LFW dataset, TAR@FAR = 0.001)

[0073]

[0074] As shown in Table 1, after INT8 quantization and .rknn format conversion, the validation accuracy of the model combination used in this invention on the LFW dataset decreased slightly, but the accuracy loss was controlled at 0.22%, which is still at a high level (99.30%), meeting the accuracy requirements of practical applications and laying the foundation for efficient NPU inference.

[0075] S3: Fine-grained NPU resource configuration and parallel processing strategies

[0076] To fully leverage the parallel processing capabilities of the RK3588 platform's built-in triple-core NPU (total computing power 6 TOPS), this embodiment features a refined configuration of NPU resources.

[0077] S3.1: Configuration of the RetinaFace NPU Face Detection Model: Given that the RetinaFace model requires significantly more computation than MobileFaceNet, it is deployed on two NPU cores of the RK3588 platform for collaborative operation. Specifically, an NPU context instance `detect_ctx` is created via the RKNPU runtime library API, and the `rk_npu_set_core_mask(detect_ctx, RK_NPU_CORE_0|RK_NPU_CORE_1)` interface function is called to bind this context to NPU core 0 and core 1. After loading the RetinaFace .rknn model into this context, when submitting an inference task, the runtime library automatically distributes the computational load across these two cores, achieving parallel acceleration.

[0078] S3.2: NPU Configuration for the MobileFaceNet Face Feature Extraction Model: For the relatively computationally intensive MobileFaceNet model, it is deployed on one of the remaining NPU cores (core 2) of the RK3588 platform. Another independent NPU context instance, `extract_ctx`, is created and bound to NPU core 2 by calling the `rk_npu_set_core_mask(extract_ctx, RK_NPU_CORE_2)` interface function. The MobileFaceNet .rknn model is loaded into this context and dedicated to handling face feature extraction tasks.

[0079] S3.3: Construction of Multi-threaded Parallel Processing Pipeline: Multi-threading technology is used to construct the processing pipeline to achieve parallelism between CPU tasks and NPU tasks, as well as between different NPU tasks.

[0080] (1) The main thread is responsible for video frame acquisition (e.g., acquiring from the camera via the V4L2 interface), task distribution, and result aggregation.

[0081] (2) Detection Thread: This thread retrieves preprocessed image frames from the main thread and submits them to `detect_ctx(RetinaFace)`, accelerated by the dual-core NPU (cores 0 and 1), for asynchronous inference. This asynchronous inference is implemented by calling the asynchronous execution interface provided by the NPU runtime library, using the `rk_npu_run_async` function provided by the Rockchip platform. This type of function submits the inference task to the NPU and returns immediately, without waiting for computation to complete. After inference is complete, the system notifies and returns the result through a preset callback function, in which the face detection bounding box can be obtained.

[0082] (3) Feature Extraction Thread: Receives the face bounding boxes output by the detection thread, performs preprocessing such as cropping, alignment, and normalization on each face bounding box, and then submits the processed face image blocks in batches or individually to extract_ctx (MobileFaceNet), which is accelerated by a single-core NPU (core 2), for asynchronous inference. After inference, a 128-dimensional face feature vector is obtained.

[0083] Data is transferred between threads through a thread-safe task queue. This design allows face detection and face feature extraction to be executed in near parallel on different NPU core groups, while the CPU can perform data preprocessing, post-processing, and thread scheduling tasks in parallel, significantly improving the overall processing throughput and real-time performance of the system.

[0084] To verify the effectiveness of the NPU resource configuration strategy of this invention, the system processing frame rate (Frames Per Second, FPS) under different NPU configurations was tested. The test input was a video stream with a resolution of 1920x1080, and each frame contained 1-3 faces. The results are shown in Table 2.

[0085] Table 2: Comparison of average system frame rate (FPS) under different NPU configurations

[0086]

[0087] As shown in Table 2, the fine-grained NPU resource configuration and parallel processing strategy (Scheme 3) adopted in this invention significantly improves the average processing frame rate to 31.2 FPS compared to single-NPU core serial processing (Scheme 1) and simple dual-NPU core division of labor (Scheme 2), meeting the requirements of real-time identity verification. Specifically, the face detection time is greatly reduced due to the collaborative computing of the dual-core NPU, and the feature extraction module running on a dedicated NPU core also ensures efficiency. The overall parallel pipeline effectively improves the system throughput.

[0088] S4: System Integration and Application Function Implementation

[0089] The final system integration and application functions achieved by this method are as follows:

[0090] (1) Database Design and Construction: A lightweight SQLite database is used to store user registration information. The database table structure mainly includes: User ID (UserID, INTEGER, primary key), User Name (UserName, TEXT), Registration Time (RegisterTime, DATETIME), and Face Feature Vector (FeatureVector, BLOB). Among them, the FeatureVector field is used to store 128-dimensional face feature data extracted by MobileFaceNet and quantized by INT8.

[0091] (2) Image Acquisition Equipment Adaptation: An 8-megapixel camera module with a MIPI interface is selected. Video stream acquisition is achieved through the LinuxV4L2 (Video4Linux2) framework. The original resolution can be configured to 1920x1080, with a frame rate of 30fps. The acquired YUV format video frames will undergo hardware-accelerated color space conversion (YUV to RGB) and image scaling (to the 640x480 required by RetinaFace) through the RGA (2D Raster GraphicAccelerationUnit) module built into the RK3588.

[0092] (3) Upper-layer application software development: Based on the C++ language and combined with the Qt framework, the graphical user interface (GUI) and backend service logic are developed. The core functional modules of the application software implementation include:

[0093] a) Face information entry: The user captures a face image through the camera. The system runs the process of S1-S3 to extract the face feature vector and stores it together with the user ID, name and other information into the database described in S4(1).

[0094] b) Face Information Query and Identity Verification: Real-time acquisition of camera video streams, performing S1-S3 process for face detection and feature extraction on each frame. The extracted real-time face features are then compared 1:N with registered face features in the database (typically using cosine similarity or Euclidean distance to calculate similarity, with a set threshold). When the similarity exceeds the preset threshold, a successful match is determined, the corresponding user information is displayed, and identity verification is completed.

[0095] c) System management: including auxiliary functions such as user management, log recording, and parameter configuration.

[0096] To further verify the performance of the method of this invention in practical application scenarios, a test environment simulating actual access control or identity verification scenarios was built. The database contained facial information of approximately 100 different individuals. The accuracy and response time of the system were comprehensively evaluated by simulating 1000 random identity verification requests (including attempts from registered and unregistered individuals). The test results are shown in Table 3.

[0097] Table 3: System Verification Performance Tests in Real-World Application Scenarios

[0098]

[0099] As shown in Table 3, the test results of this invention demonstrate that the method and apparatus described in this invention maintain a high verification accuracy rate (TAR reaches 99.25%, FAR is controlled within 0.08%) under simulated real-world application scenarios. Simultaneously, the average verification response time is 30.1 ms, and the longest response time is controlled within 36.5 ms. This indicates that the system can stably perform real-time face authentication processing at a rate close to or exceeding 30 FPS, fully meeting the real-time and accuracy requirements of most practical application scenarios.

[0100] This invention successfully implements a face feature authentication method based on the RK3588 platform. This method significantly improves the real-time processing capability of face authentication on edge devices while ensuring high recognition accuracy by optimizing the design of the RetinaFace convolutional neural network model, optimizing the design of the MobileFaceNet convolutional neural network model, effectively quantizing and converting the model, implementing a refined three-core NPU resource parallel scheduling strategy, and perfecting system integration.

[0101] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A facial recognition authentication method based on the RK3588 platform, characterized in that, The method includes the following steps: S1. Algorithm Model Design, Selection, and Optimization Strategies S1.1: The face detection module is followed by the face feature extraction module, and the functions of each module and the data flow are clearly defined; S1.2: For the face detection module, the face detection model RetinaFace was optimized and designed; S1.3: For the face feature extraction module, the face feature extraction model MobileFaceNet was optimized and designed; S2, Model Optimization and Performance Verification S2.1: Quantize the face detection model RetinaFace and the face feature extraction model MobileFaceNet respectively; S2.2: Convert the two quantized models into a specific model format adapted to the hardware acceleration of the target RK3588 platform's Neural Processing Unit (NPU); S2.3: Use a face dataset to evaluate and verify the performance metrics of the transformed face detection model and face feature extraction model to ensure that the performance meets the requirements; S3 and NPU resource fine-grained configuration and parallel processing strategies S3.1: For the RetinaFace face detection model, create an NPU runtime context and configure the context to bind and utilize the two NPU cores in the RK3588 platform for collaborative computing; S3.2: For the face feature extraction model MobileFaceNet, create an independent NPU runtime context and configure this context to bind to and utilize one of the remaining NPU cores in the RK3588 platform for computation; S3.3: Construct a multi-threaded parallel processing pipeline to enable the parallel execution of detection and feature extraction tasks on different NPU core groups; S4. System Integration and Application Function Implementation Construct a database table structure to store user facial feature vector data and corresponding identity information, complete the hardware driver debugging and system adaptation of image acquisition equipment, develop upper-layer application software, and realize system integration and operation; in, S1.2 includes: using a lightweight MobileNetV2 as the backbone network, setting the input image resolution to 640x480 pixels, simplifying the Feature Pyramid Network (FPN) structure, selecting only the feature maps of three levels, P3, P4, and P5, reducing the number of channels through 1x1 convolution, and then performing element-wise addition and fusion through a top-down path, thereby efficiently constructing a multi-scale feature representation for detecting faces of different scales; The simplification of the Feature Pyramid Network (FPN) structure specifically includes: First, a 1x1 convolutional layer is connected after the feature map at each level, and the number of output channels is uniformly adjusted to a small preset value. The feature map of the high-level P5 is enlarged by upsampling and then element-wise added to the lower-level P4 feature map after channel adjustment, and then element-wise added to the P3 feature map, realizing multi-scale feature fusion from top to bottom. These fused feature maps are finally fed into the classification and regression head of the model to predict the location information of the face. The RetinaFace face detection model outputs the bounding box coordinates and facial keypoint coordinates of each face in the image. Based on these coordinates, precise face region image patches are cropped from the original image, and affine transformations are performed using the keypoints to align the faces. These cropped and aligned face image patches are then fed into the MobileFaceNet face feature extraction model for feature extraction.

2. The facial recognition authentication method based on the RK3588 platform as described in claim 1, characterized in that, S1.3 includes: fully utilizing the core advantages of MobileFaceNet, using depthwise separable convolutions to construct the main body of the network, and using global depthwise convolutional layers at the end of the network to replace the traditional fully connected layers in order to maintain the lightweight characteristics of the model; at the same time, selecting to output a 128-dimensional low-dimensional face feature vector.

3. The facial recognition authentication method based on the RK3588 platform as described in claim 2, characterized in that, Choosing to output a 128-dimensional low-dimensional face feature vector includes: First, the face image blocks are uniformly scaled to 112x112 pixels and used as the standard input for the MobileFaceNet model; Subsequently, the 112x112 pixel image is fed into the MobileFaceNet network for forward propagation computation. After a series of convolutional layers and GDC layers within the network, a one-dimensional array containing 128 floating-point values ​​is finally generated at the network output, which is the 128-dimensional feature vector representing the face.

4. The facial recognition authentication method based on the RK3588 platform as described in claim 1, characterized in that, S2.1 includes: converting the original FP32 operation in the model into the more hardware-efficient INT8 operation; The INT8 symmetric quantization scheme linearly maps a range of floating-point numbers to a range of integers. The conversion relationship is defined by the following formula: real_value = scale (quantized_value - zero_point) in: real_value is the original FP32 floating-point value; quantized_value is the INT8 integer value obtained after quantization; scale is a floating-point number that defines the quantization step size or precision; zero_point is an integer that ensures that the original floating-point number 0 can be accurately mapped to a value in the quantized integer field; Prepare a small, representative calibration dataset, input this data into the original FP32 model, and statistically analyze the actual data distribution range of the network weights and activation values ​​for each layer in the model. Based on these statistically analyzed ranges, calculate the scale value that best preserves the original data information and minimizes the quantization error. This method maps model weights and activation values ​​from FP32 to INT8.

5. The facial recognition authentication method based on the RK3588 platform as described in claim 4, characterized in that, S2.2 includes: first converting the model to ONNX format, and then using the RKNN-Toolkit2 toolkit to convert the ONNX format model after INT8 quantization into the .rknn model format that can be directly loaded and executed by the RK3588 platform NPU; this conversion process includes model graph optimization, operator fusion, and instruction compilation for NPU hardware characteristics.

6. The facial recognition authentication method based on the RK3588 platform as described in claim 5, characterized in that, S3.1 includes: creating an NPU context instance detect_ctx through the RKNPU runtime library API, and calling the rk_npu_set_core_mask(detect_ctx, RK_NPU_CORE_0 | RK_NPU_CORE_1) interface function to bind the context to NPU core 0 and core 1; after loading the RetinaFace .rknn model into this context, when submitting an inference task, the runtime library will automatically distribute the computational load to these two cores to achieve parallel acceleration.

7. The facial recognition authentication method based on the RK3588 platform as described in claim 6, characterized in that, S3.2 includes: creating another independent NPU context instance extract_ctx and calling the rk_npu_set_core_mask(extract_ctx, RK_NPU_CORE_2) interface function to bind it to NPU core 2; loading the MobileFaceNet .rknn model into this context, dedicated to handling face feature extraction tasks.

8. The facial recognition authentication method based on the RK3588 platform as described in claim 7, characterized in that, S3.3 includes: The main thread is responsible for video frame acquisition, task distribution, and result aggregation; The detection thread retrieves preprocessed image frames from the main thread and submits them to the `detect_ctx` function, which is accelerated by a dual-core NPU, for asynchronous inference. This asynchronous inference is implemented by calling the asynchronous execution interface provided by the NPU runtime library. Using the `rk_npu_run_async` function provided by the Rockchip platform, the inference task is submitted to the NPU and returned immediately without waiting for the computation to complete. After the inference is completed, the result is notified and returned through a preset callback function, in which the face detection box can be obtained. Feature extraction thread: Receives face bounding boxes output by the detection thread, performs preprocessing such as cropping, alignment, and normalization on each face bounding box, and then submits the processed face image blocks in batches or individually to extract_ctx, which is accelerated by a single-core NPU, for asynchronous inference; after inference, it obtains a 128-dimensional face feature vector; Data can be transferred between threads using a thread-safe task queue.

Images