Computer vision based monitoring image perception control method and system

Abstract

This invention discloses a computer vision-based method and system for sensing and controlling surveillance images. The method includes: performing face detection and upper body detection on video frame images respectively, outputting face bounding boxes and confidence scores, and upper body bounding boxes and confidence scores; associating face bounding boxes and upper body bounding boxes belonging to the same target in the same frame to form a joint detection vector; calculating a weighted inter-frame comprehensive association score based on the intersection-union ratio of upper body bounding boxes between adjacent frames, the product of face confidence scores, and the cosine similarity of face region features; and using a greedy matching algorithm to associate the same target between adjacent frames to construct a motion trajectory. Furthermore, the face images in the trajectory undergo multi-dimensional quality evaluation of brightness, dynamic range, sharpness, and pose, selecting high-quality face samples and performing feature fusion for identity recognition. This invention improves trajectory integrity and recognition accuracy.

Description

Technical Field

[0001] This invention belongs to the technical field of computer vision monitoring, specifically relating to a monitoring image perception and control method and system based on computer vision. Background Technology

[0002] With the rapid development of computer vision technology, real-time face recognition based on video streams has been widely used in fields such as security monitoring and smart cities. This technology typically first detects and locates facial targets in the monitored image, then constructs the target's motion trajectory through multi-frame tracking, and finally extracts features and compares identities from the facial images contained in the trajectory.

[0003] However, in real-world surveillance scenarios, the distance between the camera and the target person is often considerable, resulting in faces appearing at extremely small pixel scales in the video. Furthermore, the walking posture of the target person is random, with non-frontal facial features such as profile views, head tilts, or head lowering being very common. In addition, complex surveillance environments often involve pedestrians and objects occluding each other, making facial areas partially or completely invisible. These factors cause existing trajectory tracking algorithms based on single face detection to easily experience tracking drift or fragmentation when handling these situations, resulting in a large number of incomplete or erroneous facial trajectory fragments. This low-quality trajectory data directly impacts the subsequent identity recognition results.

[0004] Therefore, how to overcome the adverse factors such as small scale, varied postures and occlusion of human faces in monitoring scenarios, and achieve stable and continuous human target trajectory construction, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] A computer vision-based monitoring image perception and control method includes the following specific steps: The deep learning-based face detection model is applied to the input video frame images, and the face detection bounding box and face detection confidence score of each face target are output. The deep learning-based upper body detection model is applied to the same video frame image to output the upper body detection bounding box and upper body detection confidence score for each human target. The face detection bounding box and the upper body detection bounding box belonging to the same target in the same frame image are associated to form a joint detection vector; The inter-frame comprehensive association score is calculated by weighting the product of the intersection-union ratio of the upper body detection bounding box of the joint detection vector between adjacent frames, the face detection confidence, and the cosine similarity of the face region features extracted from the face detection bounding box. Based on the inter-frame comprehensive association score, a greedy matching algorithm is used to associate the same human target between adjacent frames, and to construct and maintain a motion trajectory containing information on the appearance of the human target in consecutive frames.

[0006] Furthermore, it also includes a step of performing a multi-dimensional facial quality assessment on the motion trajectory: Obtain each frame of face image contained in the motion trajectory; The luminance factor BR is calculated based on the grayscale histogram after converting the face image to grayscale. The formula for calculating the luminance factor BR is as follows:

[0007] in, The number of grayscale intervals. Let be the arithmetic mean of the grayscale values ​​of all pixels within the i-th interval. This represents the number of pixels whose grayscale values ​​fall within the i-th interval. Based on the grayscale histogram and the luminance factor BR, the dynamic range factor DR is calculated. The formula for calculating the dynamic range factor DR is as follows:

[0008] The Sobel operator is used to calculate the horizontal and vertical gradients of each pixel in a face image. Based on the gradient magnitude and a preset edge detection threshold T, the sharpness factor SH is calculated. The formula for calculating the sharpness factor SH is as follows:

[0009] in, For pixels The gradient magnitude; Five key feature points, including the corners of the eyes, the tip of the nose, and the corners of the mouth, are extracted using a facial feature point detection model. The extracted key feature points are then mapped to a homography matrix of the preset average facial coordinates by solving the homography matrix. Yaw and pitch components are extracted from the homography matrix as facial pose factors. The brightness factor BR, dynamic range factor DR, sharpness factor SH, yaw angle component, and pitch angle component are combined to form a feature vector, which is then input into a pre-trained second-order polynomial regression model to calculate the quality score of each frame of face image.

[0010] Furthermore, it also includes a trajectory-level high-quality face screening step: For each motion trajectory, obtain the face images of all frames contained in the motion trajectory and the quality score corresponding to each face image; Sort all the face images in the frames in descending order of their quality scores; A preset number of n face images with the highest quality scores in the sorting are selected as candidate high-quality face samples for the motion trajectory.

[0011] Furthermore, it also includes a step of feature fusion for candidate high-quality face samples with the same trajectory: Using a pre-trained face feature extraction network, 512-dimensional face feature vectors are extracted from N selected high-quality face sample images, where the i-th feature vector is denoted as:

[0012] The N 512-dimensional facial feature vectors are averaged dimension by dimension to generate a fused feature vector F. The formula for calculating the j-th dimension of the fused feature vector F is as follows: The fused feature vector F is output as the final identity recognition feature of the human target.

[0013] Furthermore, the calculation steps for the inter-frame composite correlation score are as follows: Let the joint detection vector of the i-th human target detected at time t be... The joint detection vector of the j-th human target detected at time t+1 is: ; Inter-frame composite correlation score The calculation formula is:

[0014] in, Let be the bounding box for detecting the upper part of the body of the i-th human target at time t. Let the bounding box of the upper part of the body of the j-th human target at time t+1 be the bounding box. Detect bounding box for the upper part of the body With the detection bounding box of the upper body The intersection and union ratio; Let be the confidence score of the face detection for the i-th human target at time t. Let be the confidence score of the face detection of the j-th human target at time t+1; Let cosine similarity be the facial region features of the i-th human target at time t and the facial region features of the j-th human target at time t+1. and These are the preset weighting coefficients.

[0015] Furthermore, the step of constructing and maintaining the motion trajectory using the greedy matching algorithm specifically includes: Calculate the inter-frame composite correlation score for all detection pairs between two adjacent frames, and sort all the inter-frame composite correlation scores from high to low; The detection pairs corresponding to each score are processed sequentially. If neither of the two human targets in the current detection pair is matched, the detection pair is accepted as a valid match, and the two human targets are associated with the same trajectory. After matching all detection pairs, the unmatched detection target in the next frame is taken as the starting point of the new trajectory, and the trajectory corresponding to the unmatched detection target in the current frame is marked as the end. Set the correlation score threshold When the highest correlation score calculated between a trajectory and the human target in the current frame and all human targets in the next frame is lower than the correlation score threshold... When the time comes, the trajectory is determined to terminate.

[0016] Furthermore, the steps for constructing the training dataset for the upper body detection model include: Obtain images containing human skeletal keypoint annotations from publicly available human keypoint datasets; Identify and acquire the sets of head key points, shoulder key points, and arm key points in the image; Traverse all keypoint coordinates in the head keypoint set, shoulder keypoint set, and arm keypoint set, find the minimum and maximum values ​​of the horizontal coordinate as the left and right boundaries of the upper body bounding box, find the minimum and maximum values ​​of the vertical coordinate as the upper and lower boundaries of the upper body bounding box, thereby generating the smallest bounding rectangle surrounding the head, neck, shoulders, and upper limbs as the label of the upper body true bounding box. The image and the corresponding ground truth bounding box label of the upper body are used to form a training sample pair to form a training dataset for upper body detection.

[0017] Furthermore, the training process of the face detection model uses the CIoU loss function as the bounding box regression loss function; The CIoU loss function The calculation formula is:

[0018] in, Indicates the predicted bounding box. Represents the true bounding box. It is the ratio of the intersection area to the union area of ​​the predicted bounding box and the true bounding box; This represents the function for calculating Euclidean distance. This represents the coordinates of the center point of the predicted bounding box. This represents the coordinates of the center point of the actual bounding box; The diagonal length of the smallest closed box that can simultaneously contain both the predicted bounding box and the true bounding box; These are the weighting coefficients. Used to measure the consistency of the aspect ratio between the predicted bounding box and the true bounding box.

[0019] Furthermore, a face alignment step is included before extracting facial features: The homogeneous coordinates of five key facial feature points in the face image to be processed are extracted using a facial feature point detection model. ; Solve for the homogeneous coordinates Homogeneous form mapped to preset average face coordinates affine transformation matrix The solution formula is:

[0020] Using the affine transformation matrix A geometric transformation is performed on the face image, and the midpoint of the line connecting the two corners of the eyes is used as the cropping center point to crop a standardized face image of 112 pixels × 112 pixels × 3 channels.

[0021] A computer vision-based surveillance image perception and control system includes: The face detection module is used to process the input video frame images using a deep learning-based face detection model, and outputs the face detection bounding box and face detection confidence score for each face target. The upper body detection module is used to perform deep learning-based upper body detection model processing on the same video frame image, and output the upper body detection bounding box and upper body detection confidence score for each human target; The detection fusion module is used to associate the face detection bounding box and the upper body detection bounding box belonging to the same target in the same frame image to form a joint detection vector; The trajectory tracking module is used to calculate a weighted inter-frame comprehensive association score based on the cross-union ratio of the body detection bounding box of the joint detection vector between adjacent frames, the product of the face detection confidence, and the cosine similarity of the face region features extracted from the face detection bounding box. Based on the inter-frame comprehensive association score, a greedy matching algorithm is used to associate the same human target between adjacent frames, and to construct and maintain a motion trajectory containing information on the appearance of the human target in consecutive frames.

[0022] In summary, this application includes at least one of the following beneficial technical effects: 1. This invention employs a joint detection strategy that combines face detection with upper body detection. By leveraging the spatial stability and geometric constraints of body parts, it effectively addresses tracking failures caused by factors such as small face size, side profile, and occlusion in surveillance scenarios, significantly improving the completeness and accuracy of trajectory construction. A multi-factor quality assessment mechanism is also included: through comprehensive scoring of multiple quality factors such as brightness, dynamic range, sharpness, and face pose, the highest quality face samples in the trajectory are selected, effectively filtering out low-quality inputs and providing a reliable data foundation for subsequent recognition.

[0023] 2. This invention uses trajectory-level face screening and quality filtering to send only high-quality faces into the feature extraction and comparison stages, reducing the number of invalid calculations and significantly improving the system's resource utilization efficiency; feature fusion improves recognition accuracy: by averaging the features of multiple high-quality faces of the same target, redundant information in the video sequence is fully utilized, effectively suppressing random errors in single-frame recognition, and improving the overall recognition accuracy.

[0024] 3. The logic between each processing step of this invention is clear and the interface is well-defined, which facilitates module replacement and algorithm upgrade in different application scenarios and has good scalability and maintainability; End-to-end processing flow: from the input of the original monitoring image to the final identity comparison output, a complete processing pipeline is formed, which can be directly applied to various monitoring and face recognition scenarios such as smart security, personnel deployment, and access control systems. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall technical solution for a computer vision-based surveillance image perception and control method. Figure 2 This is a schematic diagram illustrating the core principle of joint detection based on face detection and upper body detection; Figure 3 A schematic diagram illustrating the multi-level interaction relationships and data flow during the detection fusion and trajectory tracking stages; Figure 4 A flowchart illustrating the logic of multi-dimensional face quality assessment and feature extraction; Figure 5 This is a schematic diagram illustrating the core principles of feature fusion and face comparison. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the following description is provided in conjunction with the appendix. Figure 1 To be continued Figure 5 The present invention will be further described in detail below with reference to specific embodiments.

[0027] Example 1

[0028] Firstly, this invention discloses a computer vision-based surveillance image perception and control method. This method combines a joint detection strategy that integrates face detection with upper body detection, along with a multi-dimensional face quality assessment mechanism, to construct high-quality face trajectories and fuse features, effectively improving the accuracy of video face recognition and the system's processing efficiency. The method comprises eight core processing steps: face detection, upper body detection, detection fusion, trajectory tracking, high-quality face selection based on trajectory, face feature extraction, feature fusion, and face comparison.

[0029] The first step, S1, involves building and applying a face detection model to detect all facial targets from the input video frames and outputting their bounding box locations and confidence scores. This model is based on a deep learning-based object detection algorithm and involves three sub-stages: training data construction, model design and training, and inference application. These are explained below.

[0030] Step S101: Construct a face detection training dataset. First, obtain face images and their corresponding ground truth bounding box annotations from public datasets. Suitable public datasets include WIDER FACE, FDDB, or AFW, which cover face images under different scenes, poses, and lighting conditions. Each ground truth bounding box annotation is denoted as... .

[0031] in The x-coordinate of the top-left vertex of the ground truth bounding box in the image, in pixels; The vertical coordinate of the top-left vertex of the ground truth bounding box in the image, in pixels; Represents the width of the actual bounding box, in pixels; This represents the height of the actual bounding box, in pixels.

[0032] The acquired image is preprocessed sequentially as follows: the image size is normalized to a preset input size, such as 640×640 pixels; the pixel values ​​are linearly mapped from an integer range of 0 to 255 to a floating-point range of 0 to 1; and the color space format of the image is unified, for example, using the RGB three-channel format.

[0033] After preprocessing, the image is augmented to increase the diversity of training samples: the image is randomly flipped horizontally with a 50% probability; the image brightness is randomly adjusted within a coefficient range of -0.2 to 0.2; the image contrast and saturation are randomly adjusted within a preset coefficient range; the original image is cropped at a random ratio, and then the cropped result is scaled back to the preset input size.

[0034] The above process yields the final face training dataset. Each image in the dataset corresponds to several ground truth bounding box labels, which are used for subsequent supervised training.

[0035] Step S102: Design and train a face detection model using a convolutional neural network-based object detection algorithm. Applicable algorithms include the YOLO series, Faster R-CNN, or SSD. Taking the YOLO series as an example, the network structure consists of three parts: a backbone network, used to extract multi-scale deep feature maps from the input image; a neck network, used to fuse feature maps from different levels to enhance the model's ability to represent features of targets at different scales; and a detection head network, used to output the class probability, detection confidence, and bounding box regression parameters for each candidate target.

[0036] The model training employs a multi-task loss function, iteratively updating network weights on the dataset constructed in step S101 using the backpropagation algorithm and a stochastic gradient descent optimizer until the model converges. The optimizer chosen is either Adam or SGD.

[0037] Multi-task loss function Classification loss Confidence loss and bounding box regression loss The formula is obtained by weighted summation of the three parts:

[0038] in It is a preset balance coefficient used to adjust the weight of each part of the loss in the total loss.

[0039] Classification loss The cross-entropy loss function is used to measure the difference between the predicted class probability and the true class label. The calculation formula is as follows:

[0040] In the formula For sample index; For the first The true label of each sample is 1 when the sample contains a face, and 0 otherwise. Predict the probability that a sample belongs to the face category for the model.

[0041] Confidence loss A binary cross-entropy loss function is used to measure the difference between the predicted confidence score and the true confidence score. The true confidence score is defined as the intersection-union ratio (IUU) between the predicted and true bounding boxes.

[0042] Bounding box regression loss This is used to measure the positional difference between the predicted bounding box and the true bounding box. This embodiment uses the CIoU loss function, calculated as follows:

[0043] in: Indicates the prediction box; Represents a real bounding box; This is the ratio of the intersection area of ​​the predicted bounding box and the ground truth bounding box to the union area. This represents the function for calculating Euclidean distance; Indicates the coordinates of the center point of the prediction box; Represents the coordinates of the center point of the true bounding box; The diagonal length of the smallest closed box that can simultaneously contain both the predicted box and the ground truth box; These are the weighting coefficients; Used to measure the aspect ratio consistency between the predicted bounding box and the ground truth bounding box.

[0044] The goal of this loss function is to force the predicted bounding box to be close to the true bounding box in terms of spatial location, degree of overlap, and shape.

[0045] Step S103: Inference is performed using the trained face detection model. The video frame image to be detected is input into the trained face detection model. The model first extracts multi-scale feature maps of the input image through the backbone network, then performs feature pyramid fusion through the neck network, and finally outputs the detection results of all faces in the frame image through the detection head network.

[0046] Each detection result is represented by a five-dimensional vector. express: This represents the x-coordinate of the top-left corner vertex of the detected face bounding box in the input image coordinate system, in pixels. This represents the ordinate of the top-left corner vertex of the detected face bounding box in the input image coordinate system, in pixels. This represents the width of the detected face bounding box, in pixels. This represents the height of the detected face bounding box, in pixels. This represents the model's confidence score for the detection results, ranging from 0 to 1. A higher value indicates that the model has a higher confidence that the region contains a face.

[0047] When there are multiple faces in the input image, the model outputs multiple five-dimensional vectors, each vector corresponding to a detected face target.

[0048] Step S1 completes the face detection function: first, a training dataset with ground truth bounding box annotations is constructed; then, a deep learning detection model based on a multi-task loss function is designed and trained; finally, the model is used to infer the input image, outputting the position coordinates and confidence score of each face. The face detection results serve as the foundation for subsequent fusion with the upper body detection results and the construction of the target trajectory.

[0049] The next step, S2, involves constructing and applying a body upper region detection model to detect the upper regions of all visible human targets from the input video frames, and outputting the bounding box positions and confidence scores of these regions. The body upper region detection model is based on a deep learning-based object detection algorithm and involves three sub-stages: training data construction, model design and training, and inference application. These are explained below.

[0050] Step S201: Construct a training dataset for upper body detection. First, obtain images and corresponding skeletal keypoint annotations from publicly available human keypoint datasets. Suitable publicly available datasets include the COCO KeypointsDetection dataset, the MPII Human Pose dataset, or the LSP dataset. Each image in these datasets is annotated with the coordinates of human skeletal keypoints within the image, such as the top of the head, left shoulder, right elbow, and left wrist.

[0051] Using the acquired human body key point annotation information, bounding boxes for the upper body region are generated as annotations for subsequent model training. The specific generation process is as follows: based on the key point annotations, the set of head key points is identified, including the vertex of the head, the forehead point, and the back of the head point; the set of shoulder key points is identified, including the left shoulder point and the right shoulder point; and the set of arm key points is identified, including the left wrist point, the right wrist point, the left elbow point, and the right elbow point.

[0052] Iterate through the coordinates of all keypoints belonging to the upper body region, find the minimum and maximum values ​​of the x-coordinates, and use them as the left and right boundaries of the upper body bounding box, respectively; find the minimum and maximum values ​​of the y-coordinates, and use them as the upper and lower boundaries of the bounding box, respectively. This yields the smallest bounding rectangle enclosing the head, neck, shoulders, and upper limbs region, denoted as the training annotation box. .in This represents the x-coordinate of the top-left corner of the ground truth bounding box in the image, in pixels. This represents the vertical coordinate of the top-left corner of the ground truth bounding box in the image, in pixels. Represents the width of the actual bounding box, in pixels; This represents the height of the actual bounding box, in pixels.

[0053] The generated bounding boxes are paired with the corresponding input images to form training sample pairs, creating a training dataset for upper body detection. Each image in the dataset corresponds to several ground truth bounding box labels for the upper body, which are used for subsequent supervised training.

[0054] Step S202: Design and train the upper body detection model, using a target detection algorithm based on convolutional neural networks to construct the upper body detection model. Applicable algorithms include two-stage detection algorithms such as Faster R-CNN, or single-stage detection algorithms such as YOLO and SSD.

[0055] Taking the YOLO series of networks as an example, the backbone network can be the ResNet series, VGG series, or the lightweight MobileNet series, used to extract deep feature representations from the input image. To enhance the detection capability of targets on the upper part of the body at different scales, a feature pyramid network structure is connected after the backbone network to fuse feature maps from different levels.

[0056] The model training employs a multi-task loss function, iteratively updating network weights on the upper body detection training dataset constructed in step S201 using the backpropagation algorithm and stochastic gradient descent optimizer until the model converges. The optimizer chosen is either Adam or SGD.

[0057] Multi-task loss function Classification loss and bounding box regression loss The weighted sum of the two parts is obtained by the following formula:

[0058] in and It is a preset balancing coefficient used to adjust the weights of classification loss and regression loss in the total loss.

[0059] Classification loss A binary cross-entropy loss function is used to measure the prediction accuracy of whether each candidate region belongs to the upper body category. The calculation formula is as follows:

[0060] In the formula For the index of the candidate region; For the first The true label of each candidate region is 1 if the region contains the upper body, and 0 otherwise. The model predicts the probability that this region belongs to the upper body category.

[0061] Bounding box regression loss This is used to measure the positional difference between the predicted upper body bounding box and the ground truth bounding box. This embodiment uses the CIoU loss function, calculated as follows:

[0062] in: This represents the predicted upper bounding box of the body; Represents the actual upper bounding box of the body; This is the ratio of the intersection area to the union area of ​​the predicted upper body bounding box and the actual upper body bounding box. This represents the function for calculating Euclidean distance; This indicates the coordinates of the center point of the predicted upper bounding box of the body; Represents the coordinates of the center point of the actual upper bounding box of the body; The diagonal length of the smallest closed box that can simultaneously contain both the predicted upper body bounding box and the true upper body bounding box; These are the weighting coefficients; Used to measure the consistency of the aspect ratio between the predicted upper body bounding box and the true upper body bounding box.

[0063] The goal of this loss function is to force the predicted bounding box to be close to the true bounding box in terms of spatial location, degree of overlap, and shape.

[0064] Step S203: Inference is performed using the trained upper body detection model. The video frame image to be detected is input into the trained upper body detection model. The upper body detection model extracts features through a backbone network, and after feature pyramid fusion, the detection head network outputs the detection results of all visible human targets in the frame image.

[0065] Each detection result is represented by a five-dimensional vector. express: This represents the x-coordinate of the top-left corner vertex of the detected upper bounding box of the body in the input image coordinate system, in pixels; This represents the ordinate of the top-left corner vertex of the detected upper body bounding box in the input image coordinate system, in pixels. This represents the width of the detected upper bounding box of the body, in pixels. This indicates the height of the detected upper bounding box of the body, in pixels. This represents the confidence score of the upper body detection model for the detection results. The value ranges from 0 to 1. The higher the value, the higher the confidence of the upper body detection model in believing that the area contains the upper body.

[0066] When there are multiple visible human targets in the input image, the upper body detection model outputs multiple five-dimensional vectors as described above, with each vector corresponding to a detected upper body target.

[0067] Step S2 implements the upper body detection function: First, upper body bounding boxes are generated using the human keypoint dataset as annotations to construct a training dataset; then, a upper body detection model based on a multi-task loss function is designed and trained; finally, the trained upper body detection model is used to infer the input image, outputting the upper body bounding box position coordinates and confidence score for each human target. The upper body detection results will be fused with the face detection results from step S1 to provide spatial constraint information for subsequent trajectory tracking.

[0068] In step S3, the detected face results and upper body detection results in the same frame are associated to form a joint detection vector. Based on the intersection-union ratio of body detection boxes between adjacent frames, face detection confidence, and face feature similarity, an inter-frame association score is calculated to provide a matching basis for subsequent trajectory tracking. The specific implementation is as follows.

[0069] Step S301: Construct a joint detection vector by combining the face detection results and the upper body detection results into a joint detection vector. Define the joint detection vector. ,in For parameters measured in the upper body, These are the face detection parameters. The meanings of these parameters are consistent with the definitions in steps S2 and S1.

[0070] Assume at time... Detected in the input image Personal goals, Indicates time The detected first The joint detection vector corresponding to an individual human target, where Joint detection vector It contains two substructures: the first substructure stores the bounding box parameters detected on the upper part of the body. and confidence level The second substructure stores the bounding box parameters of the face detection. and confidence level This data structure links the body information of each detected target with facial information, facilitating subsequent trajectory correlation calculations.

[0071] Step S302: Calculate the crossover ratio (CROR) of the upper body detection boxes between adjacent frames, using the time intervals of two adjacent frames. and time Taking the test results as an example. Let time... The detected first The parameters of the upper body detection box of the individual target are: ,time The detected first The parameters of the upper body detection box of the individual target are: First, calculate the intersection-union ratio (CURRR) of the two upper bounding boxes of the body, as a measure of spatial correlation. The formula for calculating the CURRR is:

[0072] in This represents the area function of the bounding boxes; the numerator is the area of ​​the intersection region of the two bounding boxes; the denominator is the area of ​​the union region of the two bounding boxes. The intersection-union ratio (IUR) ranges from 0 to 1. A higher value indicates a greater degree of spatial overlap between the two bounding boxes, and a higher probability that they belong to the same target.

[0073] Step S303: Calculate the comprehensive metric of face detection confidence between adjacent frames, setting the time interval... No. The confidence level of face detection for individual body targets is ,time No. The confidence level of face detection for individual body targets is Calculate the combined result of the two confidence scores.

[0074] In one implementation, the product of the two confidence levels is directly calculated. A higher product indicates a higher confidence level for both face detections. In another implementation, the harmonic mean of the two confidence levels or other comprehensive indicators can also be calculated.

[0075] Step S304: Calculate the similarity of facial region features between adjacent frames, and set the time interval... No. Personal facial region images and time of the target No. Each human target's facial region image is cropped from the original image and fed into a facial feature extraction network to extract its respective facial feature vector. Then, the cosine similarity between the two feature vectors is calculated, denoted as . .

[0076] Cosine similarity is used to measure the directional consistency between two face images in the feature space. Its value ranges from -1 to 1. The higher the value, the more likely the two faces are to belong to the same person.

[0077] Step S305: Obtain the comprehensive correlation score by weighted combination.

[0078] The three metrics mentioned above are weighted and combined to obtain a comprehensive correlation score. Let... and Let be the adjustment coefficients for the intersection-union ratio (IU) and face similarity measures, respectively. Then, the formula for calculating the association score is:

[0079] in: Indicates time The Personal goals and time The The overall correlation score between individual goals; The preset weighting coefficient is used to adjust the proportion of the intersection-union ratio of the upper body detection box in the association score, and the value range is usually from 0 to 1; The intersection-union ratio of the upper body detection boxes calculated in step S302; and The confidence scores for face detection at two different time points are shown. These are preset weighting coefficients used to adjust the weight of facial feature similarity in the association score; The cosine similarity of the facial features calculated in step S304.

[0080] By adjusting the coefficients and The value of controls the weighting of different metrics in the correlation score calculation. When spatial consistency needs greater emphasis, increasing allows for better control. The value of ; when it is necessary to emphasize the similarity of facial features more, increase . The value of .

[0081] Step S3 associates the face detection results and the upper body detection results in the same frame into a joint detection vector, and calculates a comprehensive association score by combining three metrics: the intersection-union ratio of body detection boxes between adjacent frames, the face detection confidence product, and the cosine similarity of face features. This association score provides a quantitative matching basis for trajectory tracking in step S4, and can more robustly associate the detection results of the same target in different frames, thereby alleviating the tracking drift or breakage problems caused by small face scale, side profile, or occlusion.

[0082] In step S4 of the method in this embodiment, based on the association score calculated in step S3, a target correspondence is established between adjacent frames using a greedy matching algorithm, the detection results of the same human target in different frames are concatenated into a complete motion trajectory, and the necessary data structure is maintained for each trajectory. The following steps are followed.

[0083] Step S401: Calculate the association score of all detection pairs between adjacent frames for time... Each detected target, at time... Of all the detection results, the correlation score is calculated according to the formula given in step S305, and the calculation time is... No. Personal goals and time No. Comprehensive correlation score between individual goals .in , , and These represent the number of human targets detected at two different time points.

[0084] Step S402: A greedy matching strategy is used to select the optimal association pair. All calculated association scores are sorted in descending order. Starting from the highest score, the detection pairs corresponding to each score are processed sequentially. For the currently processed detection pair, if at time... The One goal and one moment The If none of the targets are matched by other detection pairs, then the detection pair is accepted as a valid match, and time is set to [time value]. The One goal and one moment The Each target is associated with the same trajectory. Then, the next highest-scoring detection pair is processed, and so on, until all detection pairs have been processed.

[0085] Step S403: Process unmatched detection targets. After the greedy matching in step S402 is completed, the remaining unmatched detection targets are processed as follows: For time... For unmatched detected targets, use them as the starting point for constructing a new trajectory. For time... For unmatched detected targets, their corresponding trajectories are marked as ending in the current frame and will not continue in subsequent frames.

[0086] Step S404: Determine the trajectory termination condition and set an association score threshold. The value typically ranges from 0.3 to 0.5. When a target on a trajectory cannot form a valid match with any target in the next frame, meaning all calculated association scores are below the threshold... If the trajectory ends or breaks, then it is determined that the trajectory has ended or broken. Specifically, for time... A certain target, if it is at time The highest correlation score If the target cannot be effectively associated with any target in subsequent frames, then the trajectory at time [time value missing] is considered to be [missing information]. termination.

[0087] During trajectory tracking, a data structure is maintained for each trajectory. This data structure contains the following fields: a unique trajectory identifier to distinguish different motion trajectories; a trajectory start frame number, recording the video frame number where the trajectory first appears; a trajectory end frame number, recording the video frame number where the trajectory last appears; index information of all detected targets included in the trajectory, used to backtrack the detection results of the upper body and face associated with each moment; accumulated confidence information of the trajectory, which can be obtained by summing or averaging the detection confidence of each frame; and a trajectory quality score, used for subsequent steps to select high-quality trajectories. Through the above greedy matching strategy, the detection results of the same human target in consecutive video frames are concatenated to form a complete target motion trajectory.

[0088] Step S4 utilizes a greedy matching algorithm to establish target matching relationships between adjacent frames based on the comprehensive association score calculated in step S3. It associates the detection results of the same human target into trajectories in chronological order, simultaneously processing the start of new trajectories and the end of old trajectories, and maintaining a trajectory data structure containing information such as identifiers, start and end frame numbers, detection indexes, accumulated confidence, and quality scores. The generated trajectories are then fed into step S5 for high-quality face selection, filtering out the most suitable face images for identity recognition from each trajectory.

[0089] In step S5, a multi-dimensional quality assessment is performed on all face images in each trajectory generated in step S4. The quality score is calculated from four aspects: brightness, dynamic range, sharpness, and face pose. Several face images with the highest scores in each trajectory are selected as candidate samples for subsequent feature extraction. The specific implementation is carried out according to the following steps.

[0090] Step S501: Calculate the brightness factor and convert the face image into a grayscale image. Assume the grayscale histogram is divided into... A grayscale range, The value range is from 16 to 256. For the... Let the intervals be denoted as . It is the arithmetic mean of the grayscale values ​​of all pixels within this interval. This represents the number of pixels whose grayscale values ​​fall within this range. (Brightness factor) The calculation formula is:

[0091] The numerator is the sum of the products of the average gray level of each gray level and the number of pixels, and the denominator is the total number of pixels in the image. It reflects the overall brightness and darkness of a facial image.

[0092] Step S502, calculate the dynamic range factor, dynamic range factor The calculation formula is:

[0093] The meanings of each symbol are the same as in step S501. It reflects the degree of dispersion of grayscale distribution in a face image; the larger the value, the richer the grayscale levels.

[0094] Step S503: Calculate the sharpness factor by using the Sobel operator to calculate the horizontal gradient of each pixel in the image. and vertical gradient The gradient magnitude is obtained as follows:

[0095] Set edge detection threshold The value ranges from 10 to 30. Clarity factor. The calculation formula is:

[0096] That is, for all gradient magnitudes greater than Sum the gradient magnitudes of the pixels. It reflects the sharpness of the edges of a human face image.

[0097] Step S504: Calculate facial pose factors and extract five key feature points from the facial image using a facial feature point detection model, including the corners of the eyes, the tip of the nose, and the left and right corners of the mouth, denoted as homogeneous coordinates. , The average face coordinates are set as the standard reference, denoted as . Solving the homography matrix Make:

[0098] in The 3×3 homography matrix is ​​obtained by solving the singular value decomposition. Extract the yaw angle component from it. and pitch components As a quantitative representation of facial pose, the smaller the value, the closer the pose is to a frontal view.

[0099] Step S505: Comprehensive Face Quality Score. Prepare a face dataset labeled with quality levels. Divide the images into 5 quality levels and assign quality scores of 1, 2, 3, 4, and 5, respectively, with higher scores indicating better quality. Extract feature vectors from each image. .

[0100] Train a second-order multinomial regression model as the quality scoring function. This model includes first-order and second-order terms for each feature, but excludes interaction terms between features. The model form is as follows:

[0101] in For feature vectors The One portion, For bias terms, and These are the regression coefficients for the linear and quadratic terms, totaling 11 parameters (1 + 5 + 5 = 11).

[0102] The mean squared error between predicted and manually labeled scores is used as the loss function, and the 11 parameters mentioned above are adjusted using a gradient descent optimization algorithm. After training, for any face image to be evaluated, its feature vector is extracted. You can get a quality score by inputting the model.

[0103] Step S506: Select high-quality faces from the trajectory. For each trajectory generated in step S4, acquire all face image frames contained in that trajectory. For each face image frame, calculate its quality score according to steps S501 to S505. Sort all face images in the trajectory from high to low quality score, and select the highest-scoring face image. Individual facial images are used as candidate samples for feature extraction. Parameters Configured according to the actual application scenario, for example This means selecting the three highest-quality faces from each trajectory for subsequent processing.

[0104] Step S5 evaluates the quality of face images from four dimensions: brightness, dynamic range, sharpness, and pose. It uses a second-order polynomial regression model to calculate the quality score of each face frame and selects the highest quality frames for each trajectory as input for subsequent feature extraction, thereby filtering out the negative impact of low-quality samples on recognition accuracy.

[0105] Next, regarding step S6 of the method in this embodiment, the high-quality face images selected in step S5 are geometrically aligned and their dimensions are standardized. Then, a fixed-dimensional feature vector is extracted using a deep neural network for subsequent identity recognition. This is specifically implemented according to the following steps.

[0106] Step S601: Extract key facial feature points and calculate the affine transformation matrix. For each high-quality face image to be processed, firstly, extract five key facial feature points using a facial feature point detection model. These key feature points include: the outer corner of the left eye, the inner corner of the left eye, the outer corner of the right eye, the inner corner of the right eye, the tip of the nose, the left corner of the mouth, and the right corner of the mouth. The feature point detection model can be a stacked hourglass network, an Hourglass network, or a cascaded regression network, trained on a dataset with labeled facial key points.

[0107] The homogeneous coordinates of the five detected key feature points are labeled as follows: ,in , and The first The horizontal and vertical coordinates of key points in the image are given, in pixels. The average face coordinates are used as a standard reference and denoted as [reference value]. , The average face coordinates are obtained by statistically analyzing the key point coordinates of a large number of frontal face images using a standard geometric template.

[0108] Solving the affine transformation matrix using two sets of coordinates This minimizes the sum of squared errors between the transformed keypoint coordinates and the average face coordinates. The solution formula is:

[0109] in This represents the homogeneous form of the average face coordinates. A 2×3 affine transformation matrix is ​​obtained through least squares estimation. This matrix contains geometric transformation parameters such as translation, rotation, and scaling.

[0110] Step S602: Perform affine transformation alignment and cropping on the face image, using the affine transformation matrix obtained in step S601. A geometric transformation is performed on the face image to align the input face to a standard frontal view. After alignment, the midpoint of the line connecting the outer corners of both eyes is used as the center point for image cropping, resulting in a standardized face image of a predetermined size.

[0111] The cropped size is 112 pixels × 112 pixels, and the image has 3 channels, i.e., RGB three channels. The standardized face image size is uniformly 112 × 112 × 3.

[0112] Step S603: Extract facial feature vectors and input the standardized facial image into a trained facial feature extraction neural network. This network can employ mature facial recognition architectures such as FaceNet, ArcFace, or CosFace. The network structure typically includes multiple convolutional layers, pooling layers, and fully connected layers: convolutional layers extract deep features from the image, pooling layers reduce feature dimensionality and enhance robustness, and the output of the final fully connected layer is the facial feature vector.

[0113] The training process of the face feature extraction network is as follows: On a large-scale face recognition dataset, such as MS-Celeb-1M, VGGFace2, or CASIA-WebFace, each image is labeled with a corresponding identity tag. During training, an identity classification loss function or a metric learning loss function, such as the ArcFace loss function, is used. The network parameters are optimized through backpropagation to enable the network to learn a face feature representation with good discriminative ability. After training, for an input face image, the network outputs a 512-dimensional feature vector, denoted as:

[0114] in For the eigenvector at the th The components of the dimension, The feature vector is the vector representation of a face image in a 512-dimensional Euclidean space. Feature vectors corresponding to different face images of the same identity are relatively close in the feature space, while feature vectors corresponding to face images of different identities are relatively far apart.

[0115] Step S6 uses a facial feature point detection model to extract 5 key points, calculates an affine transformation matrix to align the face to a standard viewpoint, and crops it into a standardized image of 112×112×3. Then, a pre-trained facial feature extraction network maps the standardized image into a 512-dimensional feature vector. This feature vector will serve as the basic input for feature fusion in step S7 and face comparison in step S8.

[0116] The next step, S7, involves averaging and fusing the feature vectors corresponding to high-quality faces from multiple frames within the same trajectory extracted in step S6 to generate a unified fused feature vector for subsequent identity comparison. The details are as follows.

[0117] Step S701: Obtain the feature vectors of candidate faces in the trajectory. Assume that step S506 selected faces from the same trajectory. One high-quality facial image, among which It is a positive integer. For each individual face image, extract its 512-dimensional facial feature vector according to step S603. Then... The eigenvectors are denoted as:

[0118] in: The index of the feature vector, with values ​​ranging from 1 to... ; Indicates the first A 512-dimensional feature vector corresponding to a high-quality face image; Indicates the first The eigenvector at the th eigenvector in the th th eigenvector The components of the dimension, ; superscript This represents the transpose of a vector, that is, converting a row vector into a column vector.

[0119] Step S702, calculate the average fusion of the feature vectors, for the above The eigenvectors are arithmetically averaged along each dimension to obtain the fused eigenvector. The calculation formula is: ,in: Indicates index From 1 to Perform a summation operation; The meaning is the same as step S701; The number of feature vectors participating in the fusion; This is the fused feature vector.

[0120] Expanding the above formula along its dimensions, the fusion result is at the [number]th dimension. The components of dimension are: ,in: Represents the fused feature vector In the Dimensional components; The meaning is the same as step S701; The meaning is the same as above.

[0121] Step S703: Output the fused feature vector. After the above calculations, the fused feature vector... It is a 512-dimensional floating-point vector. This vector integrates feature information from multiple high-quality face images of the same target along the same trajectory, which can suppress random errors and noise interference that may occur in single-frame recognition.

[0122] Step S7: Select from the same trajectory The feature vectors of high-quality faces are arithmetically averaged dimension by dimension to obtain a fused 512-dimensional feature vector. This fused vector will be used as the input for face comparison in step S8, and it has stronger robustness and stability compared to single-frame features.

[0123] Finally, in step S8, the feature vector obtained in step S7 is fused with the feature vector in the preset face database, and the cosine similarity is calculated. The identity is determined based on the similarity score. The steps are explained below.

[0124] Step S801: Obtain the fusion feature vector of the target to be identified, and obtain the fusion feature vector of the target to be identified from step S703. ,in For the fusion vector at the th The components of the dimension, superscript This indicates transpose.

[0125] Step S802: Obtain feature vectors from a preset face database. This database pre-stores facial feature vectors of individuals with known identities. Each stored record contains an identity identifier. and its corresponding 512-dimensional feature vector ,in , This represents the total number of people in the database. For the first The individual's feature vector in the th The components of the dimension. These feature vectors are obtained by sequentially performing face detection in step S1 and alignment and feature extraction in step S6 on face images of known identities.

[0126] Step S803: Calculate the cosine similarity one by one, and fuse the feature vectors of the target to be identified. With each pre-stored feature vector in the database Calculate the cosine similarity. The calculation formula is:

[0127] in: Indicates the dimension index Summing the numbers from 1 to 512; To fuse the feature vector at the th Dimensional components; For the first in the database The th pre-stored feature vector at the th th The components of the dimension; in the denominator To fuse the magnitude of the feature vectors, The magnitude of the pre-stored feature vector.

[0128] The range of values ​​for cosine similarity is: The higher the value, the more consistent the directions of the two feature vectors in the feature space, and the more likely the two faces belong to the same identity.

[0129] Step S804: Determine identity based on similarity scores, and process all calculated similarity scores. Sort in descending order of similarity score. The highest similarity score is considered the most significant. Exceeding the preset recognition threshold When the highest similarity score is below a certain threshold, the identity identifier corresponding to that score is output as the recognition result. When the identity is unknown, a message indicating "identity unknown" or "no match" is output. In one implementation, the top few with the highest similarity can also be output as a candidate identity list.

[0130] In summary, through step S8, the fused 512-dimensional feature vector is compared with the pre-stored face database one by one to calculate the cosine similarity, and the identity recognition result or unknown prompt is output according to the relationship between the highest score and the preset threshold, thus completing the final comparison process from surveillance image to identity confirmation.

[0131] For ease of understanding, the complete processing flow of the method of this invention in a specific application scenario is as follows: After the original monitoring video stream is input into the system, the video frames are first read and preprocessed to decompose the continuous video stream into independent image frame sequences.

[0132] Then, face detection processing and upper body detection processing are performed sequentially on each frame of the image to obtain the bounding boxes and confidence information of all face targets in the frame, as well as the upper body bounding boxes and confidence information of all visible human targets.

[0133] Next, based on the correspondence between the detection results of the face and the upper body, the detection results belonging to the same target are combined into a joint detection vector and stored.

[0134] Subsequently, based on the joint detection vectors between adjacent frames, the inter-frame correlation score is calculated, and the target motion trajectory is constructed using a greedy matching algorithm.

[0135] After the trajectory is formed, a multi-dimensional quality assessment is performed on all face frames in the trajectory, and the face frames with the highest quality scores are selected as a high-quality face candidate set.

[0136] Then, the high-quality face images are aligned, cropped to standard size, and fed into the face feature extraction network to extract 512-dimensional feature vectors.

[0137] Multiple high-quality facial feature vectors with the same trajectory are averaged and fused to obtain a unified feature representation.

[0138] Finally, the fused feature vector is compared with the known identity feature vectors in the preset face database using cosine similarity, and the identity recognition result is output based on the similarity score.

[0139] On the other hand, the computer vision-based monitoring image perception control system disclosed in this invention includes: The face detection module is used to process the input video frame images using a deep learning-based face detection model, and outputs the face detection bounding box and face detection confidence score for each face target. The upper body detection module is used to perform deep learning-based upper body detection model processing on the same video frame image, and output the upper body detection bounding box and upper body detection confidence score for each human target; The detection fusion module is used to associate the face detection bounding boxes and the upper body detection bounding boxes of the same target in the same frame image to form a joint detection vector; The trajectory tracking module is used to calculate the inter-frame comprehensive association score by weighting the cross-union ratio of the body detection bounding boxes of the joint detection vectors between adjacent frames, the product of the face detection confidence, and the cosine similarity of the face region features extracted from the face detection bounding boxes. Based on the inter-frame comprehensive association score, a greedy matching algorithm is used to associate the same human target between adjacent frames, and to construct and maintain the motion trajectory containing information on the appearance of the human target in consecutive frames.

[0140] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, the embodiments should be regarded as exemplary and non-limiting in all respects.

[0141] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A computer vision-based monitoring image perception control method, characterized by, Includes the following steps: The deep learning-based face detection model is applied to the input video frame images, and the face detection bounding box and face detection confidence score of each face target are output. The deep learning-based upper body detection model is applied to the same video frame image to output the upper body detection bounding box and upper body detection confidence score for each human target. The face detection bounding box and the upper body detection bounding box belonging to the same target in the same frame image are associated to form a joint detection vector; The inter-frame comprehensive association score is calculated by weighting the product of the intersection-union ratio of the upper body detection bounding box of the joint detection vector between adjacent frames, the face detection confidence, and the cosine similarity of the face region features extracted from the face detection bounding box. Based on the inter-frame comprehensive association score, a greedy matching algorithm is used to associate the same human target between adjacent frames, and to construct and maintain a motion trajectory containing information on the appearance of the human target in consecutive frames.

2. The computer vision based monitoring image perception control method according to claim 1, characterized in that, It also includes a step of performing a multi-dimensional facial quality assessment on the motion trajectory: Obtain each frame of face image contained in the motion trajectory; The luminance factor BR is calculated based on the grayscale histogram after converting the face image to grayscale. The formula for calculating the luminance factor BR is as follows: wherein, is the number of gray scale intervals, is the arithmetic mean of the gray scale values of all pixels within the i-th interval, is the number of pixels whose gray scale value falls within the i-th interval; Based on the grayscale histogram and the luminance factor BR, the dynamic range factor DR is calculated. The formula for calculating the dynamic range factor DR is as follows: The Sobel operator is used to calculate the horizontal and vertical gradients of each pixel in a face image. Based on the gradient magnitude and a preset edge detection threshold T, the sharpness factor SH is calculated. The formula for calculating the sharpness factor SH is as follows: in, For pixels The gradient magnitude; Five key feature points, including the corners of the eyes, the tip of the nose, and the corners of the mouth, are extracted using a facial feature point detection model. The extracted key feature points are then mapped to a homography matrix of the preset average facial coordinates by solving the homography matrix. Yaw and pitch components are extracted from the homography matrix as facial pose factors. The brightness factor BR, dynamic range factor DR, sharpness factor SH, yaw angle component, and pitch angle component are combined to form a feature vector, which is then input into a pre-trained second-order polynomial regression model to calculate the quality score of each frame of face image.

3. The computer vision-based monitoring image perception and control method according to claim 2, characterized in that, It also includes a high-quality face screening step at the trajectory level: For each motion trajectory, obtain the face images of all frames contained in the motion trajectory and the quality score corresponding to each face image; Sort all the face images in the frames in descending order of their quality scores; A preset number of n face images with the highest quality scores in the sorting are selected as candidate high-quality face samples for the motion trajectory.

4. The computer vision-based monitoring image perception and control method according to claim 3, characterized in that, It also includes a step of feature fusion on candidate high-quality face samples with the same trajectory: Using a pre-trained face feature extraction network, 512-dimensional face feature vectors are extracted from N selected high-quality face sample images, where the i-th feature vector is denoted as: The N 512-dimensional facial feature vectors are averaged dimension by dimension to generate a fused feature vector F. The formula for calculating the j-th dimension of the fused feature vector F is as follows: The fused feature vector F is output as the final identity recognition feature of the human target.

5. The computer vision-based monitoring image perception and control method according to claim 1, characterized in that, The specific steps for calculating the inter-frame composite correlation score are as follows: Let the joint detection vector of the i-th human target detected at time t be... The joint detection vector of the j-th human target detected at time t+1 is ; Inter-frame composite correlation score The calculation formula is: in, Let be the bounding box for detecting the upper part of the body of the i-th human target at time t. Let the bounding box of the upper part of the body of the j-th human target at time t+1 be the bounding box. Detect bounding box for the upper part of the body With the detection bounding box of the upper body The intersection and union ratio; Let be the confidence score of the face detection for the i-th human target at time t. Let be the confidence score of the face detection of the j-th human target at time t+1; Let cosine similarity be the facial region features of the i-th human target at time t and the facial region features of the j-th human target at time t+1. and These are the preset weighting coefficients.

6. The computer vision-based monitoring image perception and control method according to claim 5, characterized in that, The steps for constructing and maintaining the motion trajectory using the greedy matching algorithm specifically include: Calculate the inter-frame composite correlation score for all detection pairs between two adjacent frames, and sort all the inter-frame composite correlation scores from high to low; The detection pairs corresponding to each score are processed sequentially. If neither of the two human targets in the current detection pair is matched, the detection pair is accepted as a valid match, and the two human targets are associated with the same trajectory. After matching all detection pairs, the unmatched detection target in the next frame is taken as the starting point of the new trajectory, and the trajectory corresponding to the unmatched detection target in the current frame is marked as the end. Set the correlation score threshold When the highest correlation score calculated between a trajectory and the human target in the current frame and all human targets in the next frame is lower than the correlation score threshold... When the time comes, the trajectory is determined to terminate.

7. The computer vision-based monitoring image perception and control method according to claim 1, characterized in that, The steps for constructing the training dataset for the upper body detection model include: Obtain images containing human skeletal keypoint annotations from publicly available human keypoint datasets; Identify and acquire the sets of head key points, shoulder key points, and arm key points in the image; Traverse all keypoint coordinates in the head keypoint set, shoulder keypoint set, and arm keypoint set, find the minimum and maximum values ​​of the horizontal coordinate as the left and right boundaries of the upper body bounding box, find the minimum and maximum values ​​of the vertical coordinate as the upper and lower boundaries of the upper body bounding box, thereby generating the smallest bounding rectangle surrounding the head, neck, shoulders, and upper limbs as the label of the upper body true bounding box. The image and the corresponding ground truth bounding box label of the upper body are used to form a training sample pair to form a training dataset for upper body detection.

8. The computer vision-based monitoring image perception and control method according to claim 1, characterized in that, The training process of the face detection model uses the CIoU loss function as the bounding box regression loss function; The CIoU loss function The calculation formula is: in, Indicates the predicted bounding box. Represents the true bounding box. It is the ratio of the intersection area to the union area of ​​the predicted bounding box and the true bounding box; This represents the function for calculating Euclidean distance. This represents the coordinates of the center point of the predicted bounding box. This represents the coordinates of the center point of the actual bounding box; The diagonal length of the smallest closed box that can simultaneously contain both the predicted bounding box and the true bounding box; These are the weighting coefficients. Used to measure the consistency of the aspect ratio between the predicted bounding box and the true bounding box.

9. The computer vision-based monitoring image perception and control method according to claim 1, characterized in that, The process includes a face alignment step before extracting facial features. The homogeneous coordinates of five key facial feature points in the face image to be processed are extracted using a facial feature point detection model. ; Solve for the homogeneous coordinates Homogeneous form mapped to preset average face coordinates affine transformation matrix The solution formula is: Using the affine transformation matrix A geometric transformation is performed on the face image, and the midpoint of the line connecting the two corners of the eyes is used as the cropping center point to crop a standardized face image of 112 pixels × 112 pixels × 3 channels.

10. A computer vision-based monitoring image perception control system, applied to the method described in any one of claims 1-9, characterized in that, include: The face detection module is used to process the input video frame images using a deep learning-based face detection model, and outputs the face detection bounding box and face detection confidence score for each face target. The upper body detection module is used to perform deep learning-based upper body detection model processing on the same video frame image, and output the upper body detection bounding box and upper body detection confidence score for each human target; The detection fusion module is used to associate the face detection bounding box and the upper body detection bounding box belonging to the same target in the same frame image to form a joint detection vector; The trajectory tracking module is used to calculate a weighted inter-frame comprehensive association score based on the cross-union ratio of the body detection bounding box of the joint detection vector between adjacent frames, the product of the face detection confidence, and the cosine similarity of the face region features extracted from the face detection bounding box. Based on the inter-frame comprehensive association score, a greedy matching algorithm is used to associate the same human target between adjacent frames, and to construct and maintain a motion trajectory containing information on the appearance of the human target in consecutive frames.

Images