A high-concurrency real-time face detection method and system based on multi-GPU collaboration

Abstract

This invention relates to a high-concurrency real-time face detection method and system based on multi-GPU collaboration, belonging to the field of computer vision technology. It includes the following steps: concurrently pulling real-time video streams from multiple cameras via the RTSP protocol, and implementing heartbeat detection and automatic reconnection after disconnection using a state machine connection management mechanism; performing motion pre-filtering on decoded video frames, discarding frames with a difference ratio below a threshold; serializing qualified frames and their metadata and writing them into a persistent message queue supporting consumer groups. Based on the GPU memory capacity of the computing nodes, an appropriate number of inference worker processes are dynamically started. Each process preemptively reads video frames from the queue, performs face detection using a TensorRT-accelerated deep learning model, and then filters and classifies high-quality faces through multi-dimensional quality assessment. This invention can improve the efficiency and accuracy of face detection.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a high-concurrency real-time face detection method and system based on multi-GPU collaboration. Background Technology

[0002] Video stream-based intelligent analytics is widely used in security, attendance, smart parks, and traffic control. Face detection, as a core entry point technology for computer vision systems, directly determines the accuracy and response speed of subsequent face recognition and trajectory tracking. In large-scale scenarios such as large buildings, industrial parks, and transportation hubs, it is necessary to monitor and analyze dozens or even hundreds of 1080P or 4K high-definition cameras 24 / 7. However, existing video stream processing technologies and system architectures have many shortcomings in handling high concurrency and high real-time requirements, making them difficult to adapt to real-world application scenarios.

[0003] Existing technologies suffer from the following drawbacks: traditional monolithic and simple parallel architectures have concurrency bottlenecks; single-process architectures are limited by the GIL, while multi-process architectures face high context switching overhead, low IPC communication efficiency, and insufficient scalability; static resource allocation leads to wasted computing power and uneven load distribution; fixed binding strategies in multi-GPU scenarios cannot adapt to dynamic crowd flow and lack a mechanism for pre-filtering invalid images; video stream acquisition is vulnerable in weak network environments, RTSP transmission is easily affected by network jitter, and existing solutions lack a reliable automatic reconnection mechanism, easily leading to problems such as blocking and frame dropping; deep learning model inference efficiency is limited, native interface memory usage is high, hardware acceleration units are not fully utilized, and it is difficult to meet real-time requirements. In summary, there is an urgent need for a solution that decouples acquisition and computation, has strong network fault tolerance, can intelligently schedule multi-GPU resources, and integrates efficient pre-filtering and hardware acceleration technologies to overcome the current technological dilemmas. Summary of the Invention

[0004] To achieve the above objectives, the present invention employs the following technical solution:

[0005] This invention provides a high-concurrency real-time face detection method based on multi-GPU collaboration, comprising the following steps:

[0006] S1. Concurrently pull real-time video streams from multiple cameras via the RTSP protocol and maintain a state machine-based connection management mechanism to perform heartbeat detection and automatic reconnection for the video streams after disconnection.

[0007] S2. Perform motion pre-filtering on the decoded video frames, calculate the difference ratio between the current frame and the previous frame. If the difference ratio is lower than the preset threshold, discard the frame; otherwise, proceed to the next step.

[0008] S3. Serialize the filtered video frames and their metadata, and write them to a persistent message queue that supports consumer group mode.

[0009] S4. Based on the GPU memory capacity of the current computing node, dynamically calculate and start an appropriate number of inference worker processes. Each worker process acts as a consumer and preemptively reads video frames from the persistent message queue.

[0010] S5. Use a deep learning model with TensorRT acceleration engine to perform face detection on video frames and output face bounding boxes and key points.

[0011] S6. Perform multi-dimensional quality assessment on the detected face regions, and select high-quality face images based on the comprehensive score and classify and store them.

[0012] Furthermore, the connection management mechanism is as follows:

[0013] Establish a state machine with five states: disconnected, connected, connected, reconnecting, and error; send a heartbeat detection signal at 1-second intervals, and if the detection fails or the reading times out, the state machine automatically switches to the reconnection state; maintain a circular frame buffer with a duration of 5 seconds to ensure the continuity of video stream data during network jitter, and set the reconnection timeout to 3 seconds.

[0014] Furthermore, motion pre-filtering specifically includes:

[0015] Current frame And the previous frame Convert to grayscale and perform Gaussian blur denoising; calculate the absolute difference between the two frames, as shown in the formula below:

[0016] ,

[0017] in, Indicates the current time Video frames in coordinates The pixel grayscale value at that location; Represents the absolute difference between two frames; sets the pixel threshold. Generate a binary motion mask ,like ,but Otherwise, it is 0; calculate the binarized motion mask. The sum of non-zero pixels, and its proportion in the total number of pixels in the image. The formula is expressed as follows:

[0018] ,

[0019] in, Represents a binary motion mask The sum of non-zero pixels; Represents the total number of pixels in the image; if It is determined to be a static frame.

[0020] Furthermore, the persistent message queue adopts the Redis Stream architecture, specifically configured as follows:

[0021] Define a video frame data stream to store frames to be processed, and set a maximum length limit to prevent memory overflow; adopt a consumer group mode to allow multiple computing nodes to consume the same video stream in parallel without duplicate processing; if a consumer fails to process or times out without sending an acknowledgment, the message is automatically transferred to a dead-letter queue or preempted by other consumers.

[0022] Furthermore, dynamic scheduling strategies:

[0023] Real-time monitoring of the remaining video memory on each GPU; when the first type of model is selected and FP16 half-precision acceleration is enabled, due to the large context usage of the TensorRT deep learning inference framework, one inference process is allocated to each GPU; when the second type of model is selected, if the video memory of a single card exceeds a preset threshold, then... If one inference process is allocated, another process will be allocated; otherwise, one process will be allocated. Each inference process is a computational process running independently within the operating system. Each process independently loads a deep learning model instance and exclusively occupies a portion of the GPU memory context for executing parallel model inference tasks.

[0024] Furthermore, the multi-dimensional quality assessment algorithm is as follows:

[0025] Calculate the sharpness score The variance of the face region is calculated using the Laplacian operator to obtain the sharpness score. The face region specifically refers to a rectangular image block (Region of Interest) cropped from the original video frame based on the bounding box coordinates output by the face detection model.

[0026] Calculate the brightness score Calculate the average grayscale value of the face region. The brightness score is calculated using a fractional linear decay strategy. ;

[0027] Calculate the overall quality score ;in, Indicates the confidence level of the detection; This represents the posture angle score calculated based on the geometric coordinates of five key points (eyes, nose tip, and corners of mouth); This represents different empirical weighting coefficients preset based on the goal of maximizing the accuracy of face recognition.

[0028] Furthermore, brightness score The score is calculated using a fractional linear decay strategy: the optimal brightness range is defined as grayscale values ​​between [80, 180]. If the score falls within this range, full marks will be awarded. If it is below 80, then follow The ratio is attenuated; when If it is higher than 180, then follow the rules. The proportion of brightness reduction is adjusted; this ensures that the further the image brightness deviates from the optimal range, the lower its quality score, thus accurately reflecting the acquisition quality of the face image; the score is calculated using a piecewise function as follows:

[0029] like ,but, ;

[0030] like ,but, ;

[0031] like ,but, .

[0032] Furthermore, the attitude angle score is calculated based on the geometric coordinates of five key points. The calculation method is as follows: calculate the horizontal distance from the key point of the nose tip to the key point of the left eye, and the horizontal distance from the key point of the nose tip to the key point of the right eye; calculate the symmetry score, expressed by the following formula:

[0033] ,

[0034] in, This indicates the horizontal distance from the key point of the nose tip to the key point of the left eye. This indicates the horizontal distance from the key point of the nose tip to the key point of the right eye.

[0035] This invention also provides a high-concurrency real-time face detection system based on multi-GPU collaboration, which executes the aforementioned high-concurrency real-time face detection method based on multi-GPU collaboration, including:

[0036] Video acquisition module: concurrently pulls real-time video streams from multiple cameras via the RTSP protocol and maintains a state machine-based connection management mechanism to perform heartbeat detection and automatic reconnection after disconnection of the video stream; performs motion pre-filtering on the decoded video frames, calculates the difference ratio between the current frame and the previous frame, and if the difference ratio is lower than a preset threshold, the frame is discarded; otherwise, it proceeds to the next stage.

[0037] Message middleware module: Serializes filtered video frames and their metadata, and writes them to a persistent message queue that supports consumer group mode;

[0038] The detection and processing module dynamically calculates and starts an appropriate number of inference worker processes based on the GPU memory capacity of the current computing node. Each worker process acts as a consumer and preemptively reads video frames from the persistent message queue. It uses a deep learning model loaded with the TensorRT acceleration engine to perform face detection on the video frames and outputs face bounding boxes and key points.

[0039] Post-processing module: Performs multi-dimensional quality assessment on the detected face regions, selects high-quality face images based on the comprehensive score, and classifies and stores them.

[0040] The advantages of this invention are:

[0041] This invention abandons the traditional single-process serial processing architecture and introduces a Redis Stream producer-consumer decoupling model, supporting load balancing and dynamic horizontal scaling across multiple consumer groups. It enables concurrent processing of multiple high-definition video streams, resolving the blocking bottlenecks of traditional architectures. For complex network environments in industrial settings, a robust video acquisition mechanism based on a state machine, second-level heartbeat detection, and a circular frame buffer enable rapid recovery from stream interruptions and buffering of momentary jitter data, ensuring the continuity and integrity of video data, preventing keyframe loss, and improving the system's resilience and stability against network jitter. At the algorithm level, a "coarse-fine combined" two-layer filtering strategy and a memory-aware dynamic scheduling algorithm, combined with TensorRT acceleration technology, reduce... Ineffective computing power consumption and GPU memory usage are reduced, inference speed is improved, and hardware resource utilization is maximized. Through end-to-end performance optimization, FFmpeg hardware decoding, lightweight Base64 serialization, and the TensorRT inference engine are adopted to control the end-to-end latency of 1080P high-definition streaming to within 200ms, meeting the real-time requirements of scenarios such as security monitoring and facial recognition attendance. At the same time, a multi-dimensional facial quality assessment system is integrated to automatically filter low-quality facial data, saving disk storage space and providing high-quality base data for subsequent facial recognition, thereby improving the recognition accuracy of security systems from the source and forming a high-quality data closed loop and storage optimization. Attached Figure Description

[0042] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0043] Figure 1 This is a flowchart of the steps of the method of the present invention;

[0044] Figure 2 This is a diagram of the overall system architecture of the present invention;

[0045] Figure 3 This is a schematic diagram of the connection state machine transition for the video acquisition module;

[0046] Figure 4 A schematic diagram illustrating the working principle of multi-GPU resource allocation and consumer groups;

[0047] Figure 5 This is a complete business process diagram for face detection and processing. Detailed Implementation

[0048] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0049] Example 1

[0050] In this embodiment, as Figure 1 As shown, this invention provides a high-concurrency real-time face detection method based on multi-GPU collaboration, the specific steps of which include:

[0051] S1. Concurrently pull real-time video streams from multiple cameras via the RTSP protocol and maintain a state machine-based connection management mechanism to perform heartbeat detection and automatic reconnection for the video streams after disconnection.

[0052] Specifically, the connection management mechanism:

[0053] A state machine is established, encompassing five states: Disconnected, Connecting, Connected, Reconnecting, and Error. A heartbeat detection signal is sent at 1-second intervals; if the detection fails or a read times out, the state machine automatically switches to the reconnection state. A circular frame buffer with a duration of 5 seconds is maintained to ensure the continuity of video stream data during network jitter, with a reconnection timeout set to 3 seconds. This invention tightly couples the 5-second circular buffer with the 3-second reconnection mechanism, aiming to fill the "inference data gap" during network reconnection. This design ensures that downstream GPUs still have data to consume within the network recovery window, avoiding idle computing resources or system restarts.

[0054] S2. Perform motion pre-filtering on the decoded video frames, calculate the difference ratio between the current frame and the previous frame. If the difference ratio is lower than the preset threshold, discard the frame; otherwise, proceed to the next step.

[0055] Existing technologies typically feed all video frames directly into the GPU for inference or run optical flow tracking on the GPU, resulting in a large amount of invalid static images consuming the GPU's computing power. The innovation of this invention lies in designing a "CPU-GPU heterogeneous hierarchical filtering strategy," which utilizes the CPU to perform low-cost frame difference calculations as a pre-gating mechanism for high-cost GPU inference. The motion pre-filtering specifically involves:

[0056] Current frame And the previous frame Convert to grayscale and perform Gaussian blur denoising; calculate the absolute difference between the two frames, as shown in the formula below:

[0057] ,

[0058] in, Indicates the current time Video frames in coordinates The pixel grayscale value at that location; Represents the absolute difference between two frames; sets the pixel threshold. Generate a binary motion mask ,like ,but Otherwise, it is 0; calculate the binarized motion mask. The sum of non-zero pixels, and its proportion in the total number of pixels in the image. The formula is expressed as follows:

[0059] ,

[0060] in, Represents a binary motion mask The sum of non-zero pixels; Represents the total number of pixels in the image; if It is determined to be a static frame.

[0061] S3. Serialize the filtered video frames and their metadata, and write them to a persistent message queue that supports the ConsumerGroup mode.

[0062] The persistent message queue uses the Redis Stream architecture, specifically configured as follows:

[0063] This invention defines a video frame data stream (video:frames) to store frames to be processed, and sets a maximum length (maxlen) limit to prevent memory overflow. It employs a consumer group model, allowing multiple computing nodes to consume the same video stream in parallel without duplicate processing. If a consumer fails to process the video or times out without sending an acknowledgment (ACK), the message is automatically transferred to a dead letter queue or preempted by another consumer. This invention transforms Redis Stream into a "video frame persistent bus," leveraging its consumer group feature to achieve "heterogeneous consumption" (i.e., graphics cards with different performance levels (such as GPU0 and GPU1) can automatically preempt different numbers of frame tasks based on their processing capabilities). This architecture breaks through the performance bottleneck of traditional round-robin distribution and enables true dynamic load balancing.

[0064] S4. Based on the GPU memory capacity of the current computing node, dynamically calculate and start an appropriate number of inference worker processes. Each worker process acts as a consumer and preemptively reads video frames from the persistent message queue.

[0065] Specifically, the remaining VRAM of each GPU is monitored in real time. When the first type of model (high VRAM usage model, such as RetinaFace) is selected and FP16 half-precision acceleration is enabled, one inference process is allocated to each GPU due to the large context usage of the TensorRT deep learning inference framework. When the second type of model (low VRAM usage model, such as SCRFD) is selected, if the single-card VRAM is greater than a preset threshold (such as 10GB), then... 1 inference process (of which) ,and The allocation is determined based on the ratio of GPU memory capacity to model resource consumption; otherwise, one process is allocated. The inference process is an independently running computational process within the operating system. Each process independently loads a deep learning model instance and exclusively occupies a portion of the GPU memory context for executing parallel model inference tasks.

[0066] S5. Use a deep learning model with TensorRT acceleration engine to perform face detection on video frames and output face bounding boxes and key points.

[0067] S6. Perform multi-dimensional quality assessment on the detected face regions, and select high-quality face images based on the comprehensive score and classify and store them.

[0068] Specifically, the multi-dimensional quality assessment algorithm is as follows:

[0069] Calculate the sharpness score The variance of the face region is calculated using the Laplacian operator to obtain the sharpness score. The face region specifically refers to a rectangular image block (Region of Interest) cropped from the original video frame based on the bounding box coordinates output by the face detection model.

[0070] Calculate the brightness score Calculate the average grayscale value of the face region. The brightness score is calculated using a fractional linear decay strategy. ;

[0071] Specifically, brightness score The score is calculated using a fractional linear decay strategy: the optimal brightness range is defined as grayscale values ​​between [80, 180]. If the score falls within this range, full marks will be awarded. If it is below 80, then follow The ratio is attenuated; when If it is higher than 180, then follow the rules. The proportion of brightness reduction is adjusted; this ensures that the further the image brightness deviates from the optimal range, the lower its quality score, thus accurately reflecting the acquisition quality of the face image; the score is calculated using a piecewise function as follows:

[0072] like ,but, ;

[0073] like ,but, ;

[0074] like ,but, .

[0075] Calculate the overall quality score ;in, Indicates the confidence level of the detection; This represents the posture angle score calculated based on the geometric coordinates of five key points (eyes, nose tip, and corners of mouth); This represents the preset empirical weighting coefficients based on the goal of maximizing the accuracy of face recognition. Specifically, in a face recognition system, image sharpness has the greatest impact on the accuracy of feature extraction, and therefore is assigned the highest weight. =0.4); the detection confidence level directly determines whether the target is effective, and is assigned the second highest weight ( =0.3); while brightness and attitude angle can be compensated by the algorithm within a certain range, and their impact is relatively small, so they are given lower weights ( =0.2, =0.1). This parameter combination has been tested and can filter out high-quality images to the greatest extent.

[0076] Specifically, the attitude angle score is calculated based on the geometric coordinates of five key points. The calculation method is as follows: calculate the horizontal distance from the key point of the nose tip to the key point of the left eye, and the horizontal distance from the key point of the nose tip to the key point of the right eye; calculate the symmetry score, expressed by the following formula:

[0077] ,

[0078] in, This indicates the horizontal distance from the key point of the nose tip to the key point of the left eye. This indicates the horizontal distance from the key point of the nose tip to the key point of the right eye; if the nose tip is located at the exact center of both eyes (front view), then... The score is close to 100; if it is a side view, the difference between the two increases and the score decreases.

[0079] In one embodiment, such as Figure 2 As shown, the logical architecture is divided into four layers from top to bottom: video acquisition layer 100, message middleware layer 200, multi-GPU computing layer 300, and post-processing and storage layer 400. The specific data processing flow is as follows:

[0080] Video capture layer 100: Includes multiple surveillance cameras 101 (Camera 1 to Camera N) for generating RTSP real-time video streams. Video capture server 102 pulls these video streams, decodes them into BGR24 format frames via FFmpeg hardware decoding module 103, and transmits them to robust capture and preprocessing module 104. This module 104 performs motion pre-filtering and circular buffering logic to filter out valid frames.

[0081] Message middleware layer 200: Serving as the decoupling hub of the system, its core component is the Redis Stream message queue 201 (key: video:frames). Valid video frames from the upper layer are serialized and written to this queue for consumption by the lower layer.

[0082] Multi-GPU computing layer 300: This layer contains a GPU computing cluster 301, which consists of multiple GPU-based worker processes (Worker nodes 302). As shown in the diagram, Worker node A is bound to GPU 0 to run the SCRFD model, and Worker node B is bound to GPU 1 to run the RetinaFace model. Each Worker node 302 preemptively reads video frames from Redis Stream 201 in consumer group mode, performs face detection inference in parallel, and outputs the detection results.

[0083] Post-processing and storage layer 400: Receives the detection results and scores the clarity and brightness of the faces using the quality assessment and post-processing module 401. The finally selected high-quality face images are sent to the hierarchical storage module 402 and saved to the physical disk in a "year-month-day" hierarchical structure.

[0084] In one embodiment, the system hardware environment construction and dynamic resource initialization are as follows:

[0085] Hardware Deployment Architecture: This system is deployed on a high-performance computing server specifically designed for deep learning inference. The core processing unit includes a single AMD EPYC 7003 series processor (64 cores, 128 threads), equipped with 256GB of DDR4 ECC memory to support caching of large-scale image data. The compute acceleration layer uses four NVIDIA GeForce RTX 3090 graphics cards, each with 24GB of GDDR6X video memory and 10496 CUDA cores, communicating with the CPU via a PCIe 4.0 x16 bus. The storage layer uses 2TB NVMe SSDs, providing sequential write speeds exceeding 3500MB / s to meet the I / O requirements of high-concurrency image storage.

[0086] Software Dependencies and Middleware Configuration: The system runs on Ubuntu 20.04 LTS. The deep learning environment is built on CUDA 11.8 and TensorRT 8.5.3, utilizing cupy and torch to implement zero-copy operations in GPU memory. The message middleware uses Redis version 6.2, and the QueueConfigManager module is configured to perform the following initialization operations when the service starts:

[0087] Connection establishment: Establish a persistent connection pool with the local Redis service (port 6379), and set socket_timeout=5.0 to prevent connection blocking;

[0088] Stream structure initialization: Create the core video stream key-value video frame data stream video:frames. To prevent memory overflow, set the maximum stream length MAXLEN to 10000 and configure approximate=True to allow approximate pruning and reduce Redis memory fragmentation overhead;

[0089] Consumer group registration: Check if the consumer group `frame_processors` exists. If it does not exist, execute the `XGROUP CREATE video:frames frame_processors $ MKSTREAM` command, where the `$` symbol ensures that the consumer group only receives newly generated video frames from the time of its creation, ignoring historical backlog data.

[0090] When the memory-aware dynamic process scheduling master program starts, it first calls the pynvml library to traverse the GPU devices in the system. Based on the currently loaded model type (specified in the configuration file) and the real-time memory status of each GPU, it dynamically calculates and starts the optimal number of Worker processes:

[0091] Memory status scan: The system detected that GPUs 0-3 are online, and the free memory is greater than 22GB.

[0092] SCRFD Strategy (High Concurrency Mode): When the system is configured to use the SCRFD (InsightFace buffalo_l) model, since this model's single inference memory usage is approximately 3.5GB, the system determines that a single GPU's 24GB of memory is sufficient to support multi-path concurrency. The scheduling algorithm will allocate 2 inference processes (Workers) to each GPU. At this time, the system starts a total of 8 child processes through the multiprocessing module;

[0093] RetinaFace strategy (high-performance mode): When the system is configured to use the RetinaFace TensorRT (FP16) model, the scheduling algorithm forces each GPU to limit the concurrency to 1 inference process to avoid memory overflow (OOM) because the TensorRT engine needs to pre-allocate a large amount of context memory and workspace.

[0094] Process binding: When starting each child process, the program strictly binds the process to the specified GPU ID by setting the environment variable os.environ["CUDA_VISIBLE_DEVICES"], thereby achieving physical isolation of hardware resources and load balancing.

[0095] In one embodiment, such as Figure 3 The diagram shown illustrates the connection state machine flow of the video acquisition module of this invention. This state machine contains five core states, and the specific transition process is as follows:

[0096] S1 Disconnected State: This is the initial state of the system, where all network resources and memory buffers are in a released state.

[0097] S2 Connecting State: When the system calls the start interface, the state machine transitions from S1 to S2. In this state, the system attempts to establish an underlying RTSP socket connection. If the handshake is successful, it enters S3; if the connection fails, it jumps directly to the S5 error state.

[0098] S3 Connected State: This is the stable operating state of the system. In this state, the system performs three core tasks: continuously reading video frames, sending heartbeat detection signals at a frequency of 1 second, and filling the circular buffer for 5 seconds.

[0099] S4 Reconnecting State: When an abnormal trigger is detected in the S3 state (specifically, heartbeat loss or data read timeout exceeding 3 seconds), the state machine automatically switches to S4. In this state, the system attempts to restore the connection in non-blocking mode.

[0100] If the reconnection is successful (path 4), the status will immediately return to S3 and normal data collection will continue.

[0101] If the reconnection fails (path 5), the system will execute the backoff strategy and retry. As long as the number of retries (Count) is less than 3, it will remain in the reconnection attempt loop.

[0102] If the number of retries is exhausted (path 6), the failure is determined to be unrecoverable, and the status jumps to S5.

[0103] S5 Error / Stop State (ERROR): When this state is entered, the system records a detailed error log and stops the data collection task, waiting for manual intervention or a restart command.

[0104] In one embodiment, such as Figure 4 The diagram shown illustrates the working principle of multi-GPU resource allocation and consumer groups in this invention. This process demonstrates the complete path of data flow from the Redis server to the physical computing nodes:

[0105] Data source: The top-level Redis server maintains the core video frame data stream (Stream: video:frames). This stream stores video frame messages generated by the acquisition end in chronological order. Each message contains a unique message ID (e.g., 1001-0, 1002-0) and the corresponding camera identifier (e.g., Cam: 01, Cam: 02).

[0106] Logical Scheduling Layer: The middle layer is the logical scheduling layer, which manages and distributes messages uniformly through consumer groups (frame_processors). Each consumer group maintains a logical cursor and a pending list to ensure that the same message is not retrieved repeatedly by multiple consumers and to track messages that have been sent but not yet acknowledged.

[0107] GPU Physical Computation Layer: The underlying layer showcases a memory-aware differentiated resource allocation strategy.

[0108] GPU 0 (24GB VRAM): Configured to run a lightweight model (SCRFD). Due to the small single-process memory usage, the system launched two worker processes (Worker 0 and Worker 1) on this GPU to consume data in parallel to improve throughput;

[0109] GPU 1 (24GB VRAM): Configured to run high-performance models (RetinaFace TensorRT). Due to the large model context requirements, the system has allocated only one worker process (Worker 2) on this GPU to ensure inference stability.

[0110] Processing loop: Each worker process preempts tasks from the consumer group via the XREADGROUP command, completes face detection inference and business processing, and then sends an XACK confirmation signal to Redis. Upon receiving the confirmation, Redis removes the message from the Pending List, signifying that the message processing is complete.

[0111] In one embodiment, such as Figure 5 The diagram shown is a complete workflow diagram of the face detection and processing of this invention. The specific steps are as follows:

[0112] Acquiring video frames: The system first acquires the decoded raw video frames in BGR24 format.

[0113] Motion detection determination: The frame difference rate (Diff) is calculated using the frame difference method. If Diff > 0.01 (threshold), it is determined to be a moving frame and proceeds to S3; if it does not reach the threshold, it is determined to be a stationary frame and is discarded directly to save computing power.

[0114] Serialize and write: Base64 serialize the filtered motion frames and write them to the Redis message queue.

[0115] GPU model inference: Compute nodes retrieve data from the queue and perform inference using SCRFD or RetinaFace models.

[0116] Face detection decision: Determine whether the inference result contains a face. If no face is detected, execute the no-face policy (e.g., discard or only log); if a face is detected, proceed to S6.

[0117] Face quality scoring: This step involves multi-dimensional evaluation of detected face regions. It comprises three parallel subtasks: calculating sharpness using the Laplacian operator, calculating brightness from the mean grayscale value, and calculating pose / confidence. The scores from these three dimensions are then weighted to generate a comprehensive quality score Q.

[0118] Quality threshold determination: Determine whether the overall score Q is greater than a preset threshold. If it is lower than the threshold, it is considered a low-quality face and is discarded or archived only; if it is higher than the threshold, it is considered a high-quality face and proceeds to S8.

[0119] Image cropping and augmentation: Using the center of the face as a reference, the image is augmented and cropped at a ratio of 1.2 to preserve the background context.

[0120] Hierarchical storage: Finally, the cropped high-quality images are written to the storage system in a hierarchical structure of "year-month / day" (YYYY-MM / DD), and the process ends.

[0121] Example 2

[0122] This embodiment provides a high-concurrency real-time face detection system based on multi-GPU collaboration, which executes the high-concurrency real-time face detection method based on multi-GPU collaboration described in Embodiment 1, including:

[0123] Video acquisition module: It is used to concurrently pull real-time video streams from multiple cameras via the RTSP protocol and maintain a state machine-based connection management mechanism to perform heartbeat detection and automatic reconnection after disconnection of the video stream; it performs motion pre-filtering on the decoded video frames, calculates the difference ratio between the current frame and the previous frame, and if the difference ratio is lower than the preset threshold, the frame is discarded; otherwise, it proceeds to the next stage.

[0124] Message middleware module: Used to serialize filtered video frames and their metadata, and write them to a persistent message queue that supports consumer group mode;

[0125] The detection and processing module is used to dynamically calculate and start an appropriate number of inference worker processes based on the GPU memory capacity of the current computing node. Each worker process acts as a consumer and preemptively reads video frames from the persistent message queue. It uses a deep learning model loaded with the TensorRT acceleration engine to perform face detection on the video frames and outputs face bounding boxes and key points.

[0126] Post-processing module: Used to perform multi-dimensional quality assessment on the detected face regions, filter out high-quality face images based on the comprehensive score, and classify and store them.

[0127] Example 3

[0128] In this embodiment, the acquisition end uses FFmpeg hardware decoding, the transport layer uses lightweight Base64 serialization, and the inference layer uses the TensorRT engine. Actual testing shows that the system's end-to-end latency for processing 1080P high-definition streams is controlled within 200ms, as shown in Table 1, fully meeting the stringent real-time requirements of scenarios such as security monitoring and facial recognition attendance.

[0129] Table 1 Processing Schedule for Each Stage

[0130]

[0131] Example 4

[0132] This embodiment describes in detail how GPU computing nodes (Consumers) consume data, perform deep learning inference, and perform post-processing logic based on multidimensional quality scoring.

[0133] Preemptive consumption by consumer groups:

[0134] Each GPU worker process acts as an independent consumer, reading data from the Redis Stream in blocking mode (BLOCK 2000, i.e., a timeout of 2 seconds) using the XREADGROUP instruction;

[0135] The instruction parameter COUNT 1 ensures that each worker only preempts one frame of image at a time, and sends an XACK confirmation signal after processing. This mechanism naturally achieves load balancing: faster graphics cards (such as GPU 0) will automatically preempt more tasks without the need for a central node to allocate them.

[0136] TensorRT accelerates inference:

[0137] The Worker process incorporates an inference engine optimized for NVIDIA GPUs.

[0138] If the RetinaFace model is used, the system will load the pre-converted .engine file. The inference process runs entirely in FP16 (half-precision floating-point) mode, which reduces memory usage by approximately 50% compared to FP32 mode, and reduces inference latency from 45ms to approximately 12ms, as shown in Table 2.

[0139] Table 2 Comparison of Reasoning Time

[0140]

[0141] The inference output includes the face bounding box. The system first performs non-maximum suppression (NMS) with a threshold of 0.4 to remove overlapping detection boxes.

[0142] Multi-dimensional face quality assessment system:

[0143] The system performs a rigorous quality check on each detected region of interest (ROI) and calculates a comprehensive score. :

[0144] Clarity rating ( ): Grayscale conversion of the ROI region is performed using the Laplacian operator (kernel size). Edges are extracted, and image variance is calculated. A higher variance value indicates a sharper image. The system normalizes the variance values ​​to a range of 0-100.

[0145] Brightness rating ( ): Calculate the average gray value of the ROI. The ideal brightness range is set as follows: .like Within the range, you get 100 points; if you deviate from the range, the score decreases linearly.

[0146] Weighted average: Final score Calculate using the following formula:

[0147]

[0148] in To test the confidence level, This is a facial pose score calculated based on key points (high score for frontal view).

[0149] Image post-processing and hierarchical storage:

[0150] When the overall score When the preset threshold is reached, the system determines that the face has retention value:

[0151] Expand cropping: Expand the area 1.2 times (expand_ratio=1.2) around the face detection bounding box to retain more contextual features such as hairstyle and neck, which will facilitate subsequent manual review.

[0152] Hierarchical storage: To avoid file system performance degradation due to an excessive number of files in a single folder, the system generates hierarchical directories by date. For example, the generated path is records / faces / 2024-01 / 2024-01-04 / .

[0153] File naming: File names are in the format {camera_id}_{timestamp}_{face_idx}.jpg, ensuring globally unique file names. The final image is written to disk in JPEG format.

[0154] Ultimately, only face images with a comprehensive score higher than the threshold will be cropped (expanded by 1.2 times) and saved to the records / faces / YYYY-MM / DD / directory, thus achieving efficient use of storage space.

[0155] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A high-concurrency real-time face detection method based on multi-GPU cooperation, characterized in that, Includes the following steps: S1. Concurrently pull real-time video streams from multiple cameras via the RTSP protocol, and maintain a state machine-based connection management mechanism to perform heartbeat detection and automatic reconnection after disconnection of the video streams; the connection management mechanism is specifically as follows: Establish a state machine with five states: disconnected, connected, connected, reconnecting, and error; send a heartbeat detection signal at 1-second intervals, and if the detection fails or the reading times out, the state machine automatically switches to the reconnection state; maintain a circular frame buffer with a duration of 5 seconds to ensure the continuity of video stream data during network jitter, and set the reconnection timeout to 3 seconds; S2. Perform motion pre-filtering on the decoded video frames, calculate the difference ratio between the current frame and the previous frame. If the difference ratio is lower than the preset threshold, discard the frame; otherwise, proceed to the next step. S3. Serialize the filtered video frames and their metadata, and write them to a persistent message queue that supports consumer group mode; the persistent message queue adopts the Redis Stream architecture, and is specifically configured as follows: Define a video frame data stream to store frames to be processed, set a maximum length limit to prevent memory overflow; adopt a consumer group mode to allow multiple computing nodes to consume the same video stream in parallel without processing duplicates. If a consumer fails to process the message or fails to send an acknowledgment within a timeout period, the message is automatically transferred to the dead-letter queue or preempted by another consumer. S4. Based on the current GPU memory capacity of the computing node, dynamically calculate and start an appropriate number of inference worker processes. Each worker process acts as a consumer, preemptively reading video frames from the persistent message queue; dynamic scheduling strategy: Real-time monitoring of the remaining memory of each GPU; when the first type of model is selected and FP16 half-precision acceleration is enabled, allocate 1 inference process for each GPU; when the second type of model is selected, if the memory of a single card is greater than a preset threshold, allocate one inference process, otherwise allocate 1 process, . S5. Use a deep learning model with TensorRT acceleration engine to perform face detection on video frames and output face bounding boxes and key points. S6. Perform multi-dimensional quality assessment on the detected face regions, and select high-quality face images based on the comprehensive score and classify and store them. 2.The high-concurrency real-time face detection method based on multi-GPU cooperation according to claim 1, characterized in that, Motion pre-filtering specifically includes: Convert the current frame and the previous frame to grayscale and Gaussian blur denoising; calculate the absolute difference value map of the two frames, which is expressed as follows: ， in, Indicates the current time Video frames in coordinates The pixel grayscale value at that location; Represents the absolute difference between two frames; sets the pixel threshold. Generate a binary motion mask ,like ,but Otherwise, it is 0; calculate the binarized motion mask. The sum of non-zero pixels, and its proportion in the total number of pixels in the image. The formula is expressed as follows: ， in, Represents a binary motion mask The sum of non-zero pixels; Represents the total number of pixels in the image; if It is determined to be a static frame.

3. The high-concurrency real-time face detection method based on multi-GPU collaboration according to claim 2, characterized in that, The multi-dimensional quality assessment algorithm is as follows: Calculate the sharpness score The variance of the face region is calculated using the Laplacian operator to obtain the sharpness score. ; Calculate luminance score Calculate the average grayscale value of the face region. The luminance score is calculated using a fractional linear decay strategy. ; Calculate the overall quality score ;in, Indicates the confidence level of the detection; This represents the attitude angle score calculated based on the geometric coordinates of five key points; This represents different empirical weighting coefficients preset based on the goal of maximizing the accuracy of face recognition.

4. The high-concurrency real-time face detection method based on multi-GPU collaboration according to claim 3, characterized in that, Brightness score The score is calculated using a fractional linear decay strategy: the optimal brightness range is defined as grayscale values ​​between [80, 180]. If the score falls within this range, full marks will be awarded. If it is below 80, then follow The ratio is attenuated; when If it is higher than 180, then follow the rules. The proportion is attenuated; The piecewise function scores are calculated as follows: like ,but, ; like ,but, ; like ,but, .

5. The high-concurrency real-time face detection method based on multi-GPU collaboration according to claim 4, characterized in that, Attitude angle scoring based on geometric coordinates of five key points The calculation method is as follows: calculate the horizontal distance from the key point of the nose tip to the key point of the left eye, and the horizontal distance from the key point of the nose tip to the key point of the right eye; calculate the symmetry score, expressed by the following formula: ， in, This indicates the horizontal distance from the key point of the nose tip to the key point of the left eye. This indicates the horizontal distance from the key point of the nose tip to the key point of the right eye.

6. A high-concurrency real-time face detection system based on multi-GPU collaboration, executing the high-concurrency real-time face detection method based on multi-GPU collaboration as described in claim 1, characterized in that, include: Video acquisition module: concurrently pulls real-time video streams from multiple cameras via the RTSP protocol and maintains a state machine-based connection management mechanism to perform heartbeat detection and automatic reconnection after disconnection of the video stream; performs motion pre-filtering on the decoded video frames, calculates the difference ratio between the current frame and the previous frame, and if the difference ratio is lower than a preset threshold, the frame is discarded; otherwise, it proceeds to the next stage. Message middleware module: Serializes filtered video frames and their metadata, and writes them to a persistent message queue that supports consumer group mode; The detection and processing module dynamically calculates and starts an appropriate number of inference worker processes based on the GPU memory capacity of the current computing node. Each worker process acts as a consumer and preemptively reads video frames from the persistent message queue. It uses a deep learning model loaded with the TensorRT acceleration engine to perform face detection on the video frames and outputs face bounding boxes and key points. Post-processing module: Performs multi-dimensional quality assessment on the detected face regions, selects high-quality face images based on the comprehensive score, and classifies and stores them.

Images