Face quality label generation method based on cooperation of multi-modal large model and computer vision algorithm

Abstract

The invention discloses a face quality label generation method based on cooperation of a multi-modal large model and a computer vision algorithm, and relates to the technical field of artificial intelligence and computer vision crossing, in particular to a face quality label generation method based on collaborative decision making of the multi-modal large model and the computer vision algorithm. The method is especially suitable for a high-security-level identity verification system, an automatic identification photo auditing platform, a large-scale face registration pre-screening process, a living body detection front level and other scenes, and is used for carrying out fine-grained, interpretable and high-robustness quality evaluation and defect positioning on an input face image. The core problems of weak semantic understanding ability, unstable discrimination precision, rigid fusion strategy, poor scene adaptability and the like in the prior art are solved by constructing a four-level technical architecture of dimension decoupling, double-channel parallelism, dynamic confidence fusion and scene adaptive evaluation.

Description

Technical Field

[0001] This invention relates to the interdisciplinary field of artificial intelligence and computer vision, specifically to a method for generating face quality labels based on collaborative decision-making between a multimodal large model and computer vision algorithms. It is particularly suitable for scenarios such as high-security identity verification systems, automated ID photo review platforms, large-scale face registration pre-screening processes, and pre-screening checkpoints for liveness detection, enabling fine-grained, interpretable, and robust quality assessment and defect localization of input face images. Background Technology

[0002] In modern facial recognition and authentication systems, the quality of the input image is a core prerequisite for determining the system's accuracy, security, and user experience. Low-quality images (such as those with defects like blurriness, occlusion, abnormal lighting, pose shifts, or screen re-captures) not only lead to feature extraction failures and decreased matching accuracy, but can also be exploited by malicious attackers to bypass liveness detection mechanisms, posing serious security risks.

[0003] Current mainstream face quality assessment technologies are mainly divided into two categories, but both have significant limitations: Methods based on traditional image processing and shallow machine learning rely on hand-designed features and rules, such as using Laplacian gradient variance to assess ambiguity, detecting re-images through DCT frequency domain energy entropy, and determining pose based on key point geometric relationships. Their advantages lie in high computational efficiency and strong interpretability. However, their discriminative ability is severely limited by the expressive power of hand-designed features, making it difficult to handle complex scenarios requiring high-level semantic understanding, such as "whether it's a screen re-image" or "whether there's strabismus," resulting in weak generalization ability and a high false positive rate in complex real-world scenarios.

[0004] End-to-end quality scoring models based on deep learning: These methods (such as using backbone networks like ResNet and EfficientNet to directly regress quality scores) are trained on large datasets and possess a certain degree of generalization ability. However, they suffer from a significant "black box" problem, lacking interpretability for specific quality dimensions and unable to provide detailed reasons for deductions. Furthermore, their performance is unstable in scenarios with small samples, long-tailed distributions, or adversarial examples, making it difficult to meet the stringent reliability requirements of industrial-grade systems.

[0005] In recent years, multimodal large models (such as Qwen-VL and LLaVA) have demonstrated powerful image-text joint understanding and instruction following capabilities, enabling them to directly respond to natural language queries such as "whether the person in the image has closed their eyes." However, directly using them for quality assessment faces three major bottlenecks: High computational cost: The model is large and the inference latency is high, making it difficult to meet the real-time requirements of high-concurrency online services; Insufficient detail perception: It is less sensitive to low-level image details (such as slight blurring or local occlusion) than dedicated CV algorithms, making it prone to missed detections; Unstructured output: The output results are probabilities or natural language, lacking structured and quantifiable quality dimension labels that can be directly connected to downstream business systems.

[0006] Furthermore, while existing technologies attempt to fuse the outputs of multiple models, they generally employ simple strategies such as static linear weighting (e.g., a fixed weight of 0.5). This fusion mechanism is rigid and cannot be dynamically adjusted based on image content, the confidence level of each channel's output, or specific business scenarios, resulting in suboptimal fusion effects and failing to fully realize the potential of multi-model collaboration.

[0007] Therefore, existing technologies suffer from core problems such as weak semantic understanding capabilities, unstable discrimination accuracy, rigid fusion strategies, and poor scene adaptability. There is an urgent need for a new face quality assessment architecture that can deeply integrate the advantages of multimodal large model semantic understanding with the high precision and low latency characteristics of traditional CV algorithms, and achieve dynamic and adaptive fusion mechanisms. Summary of the Invention

[0008] The purpose of this invention is to provide a method for generating face quality labels based on the collaboration of multimodal large models and computer vision algorithms, addressing the aforementioned problems.

[0009] The technical solution of the present invention is as follows: A method for generating face quality labels based on the collaboration of multimodal large models and computer vision algorithms includes the following steps: For each of the multiple predetermined quality dimensions of a face image, perform the following steps: Obtain the first probability output generated by the multimodal large model, which has been fine-tuned by instructions, based on the image and the corresponding dimension of the text prompt. ; Obtain the second probability output generated by a computer vision algorithm designed for this dimension based on low-level image feature analysis. ; The fusion score for this dimension is calculated using the dynamic confidence-aware fusion module. ,in and The fusion weights are dynamically generated based on the real-time confidence scores of the first and second probability outputs. Based on fusion score Generate quality labels for this dimension.

[0010] By introducing a dynamic confidence-aware fusion mechanism, the rigidity problem of traditional static weighted fusion strategies is fundamentally solved. This mechanism can intelligently adjust the level of trust in the large model and CV algorithm based on the specific performance of each image in each dimension, achieving intelligent decision-making with "high-confidence channels dominating and low-confidence channels assisting." This significantly improves the discrimination accuracy and system robustness in complex and edge cases (such as blurred boundaries and slight occlusion). Experiments show that the F1-score is improved by 12.3% in complex dimensions such as re-photograph detection and occlusion recognition.

[0011] Furthermore, the dynamically generated fusion weights and include: Calculate the semantic confidence index of the first channel ,in It is a binary cross-entropy function; Calculate the characteristic response intensity of the second channel ,in This represents the raw response value of the computer vision algorithm in this dimension. This represents the mean response of this dimension on the validation set. This is the scaling factor; Will and Input a lightweight meta-learner to generate fusion weights .

[0012] By quantifying the probability distribution entropy and feature response intensity—two key indicators—objective and reliable decision-making basis is provided for dynamic fusion. Combining uncertainty measurement with feature response intensity allows for a more refined capture of the performance stability of each model under different conditions, thus making the weight allocation more scientific and reasonable, avoiding the blindness of manually setting thresholds, and further improving the accuracy and adaptability of fusion decisions.

[0013] Furthermore, the lightweight meta-learner uses functions Generate weights, where For the Sigmoid function, , and Let be the learnable parameters obtained by training by minimizing the validation set loss, and .

[0014] This method achieves end-to-end adaptive weight allocation by automatically learning the optimal fusion strategy using a lightweight meta-learner. Through a data-driven approach, it learns the optimal fusion parameters offline, eliminating the need for complex calculations during online inference; weight generation is completed with only one forward propagation. While ensuring fusion performance, it introduces almost no additional computational overhead, with single-dimensional fusion taking less than 5ms, achieving a balance between accuracy and efficiency.

[0015] Furthermore, the multimodal large model employs dimension-level LoRA fine-tuning, independently training and saving low-rank adapter parameters for each quality dimension, and dynamically loading the corresponding dimension adapter on the shared backbone network during inference.

[0016] By employing a "dimension-level LoRA fine-tuning + shared backbone" architecture, the powerful semantic understanding capabilities of multimodal large models are preserved while significantly reducing computational overhead and storage costs. Compared to deploying independent large models for each dimension, this invention only requires storing a small number of adapter parameters and sharing the vast majority of model parameters, resulting in a 42% reduction in inference latency from 850ms to 210ms, meeting the stringent high-concurrency requirements of industrial applications.

[0017] Furthermore, in the dimension used to detect closed-eye states, specialized computer vision algorithms calculate the eye aspect ratio. Make a judgment, among which These are the vertical distances between the eyes. The horizontal distance to the eye.

[0018] Furthermore, in the dimension used to detect re-photographing, a dedicated computer vision algorithm calculates the energy entropy of high-frequency coefficients in the DCT frequency domain of the image. Make a judgment.

[0019] Using the methods described above, efficient and accurate dedicated CV algorithms are integrated to address specific challenges such as closed-eye detection and re-photographing. These algorithms are based on geometric features (such as EAR) or frequency domain features (such as DCT energy entropy), are computationally lightweight and highly interpretable, and effectively complement multimodal large models. When the semantic judgment of the model is ambiguous, these deterministic algorithms can provide strong underlying evidence support, effectively preventing missed detections and false positives, and improving the detection reliability of key dimensions.

[0020] Furthermore, it also includes: Summarize the quality labels from all dimensions to form a set. ; Loading dimension importance weight vector based on business scenario ,in ; Calculate the overall face quality score .

[0021] Furthermore, when the business scenario is liveness detection, the weight vector is configured as follows: ; When the business scenario is ID photo verification, the weight vector is configured as follows: .

[0022] By introducing a scenario-adaptive comprehensive evaluation model, flexible application of "one model for multiple scenarios" is achieved. By dynamically loading different dimension importance weight vectors, the same system can achieve the optimal evaluation strategy in different business scenarios such as liveness detection (focusing on re-photographing and occlusion interception) and ID photo verification (focusing on posture and absence of hats) without retraining, with a replacement rate of up to 97.2%, which greatly improves the system's practicality and deployability and reduces operation and maintenance costs.

[0023] This application also includes a face quality assessment device, comprising: One or more processors; The memory stores computer instructions that, when executed by one or more processors, cause the device to perform a face quality label generation method based on the collaboration of a multimodal large model and computer vision algorithms.

[0024] This application also includes a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements a method for generating face quality labels based on the collaboration of a multimodal large model and a computer vision algorithm.

[0025] Compared with existing technologies, the advantages of this invention are: 1. It pioneered the "dynamic confidence-aware fusion" mechanism: by quantifying the uncertainty of dual-channel output and introducing a lightweight meta-learner to dynamically generate fusion weights, it solves the rigidity problem of traditional static weighted fusion. It improves the F1-score by 12.3% in complex dimensions such as re-enactment and occlusion, and has significant non-obviousness. 2. Implement a “dimension-level LoRA fine-tuning + shared backbone” architecture: While maintaining the powerful semantic capabilities of Qwen2.5-VL, dimension-specific fine-tuning is achieved through low-rank adapters and shared visual encoders, reducing inference latency from 850ms to 210ms to meet industrial deployment requirements. 3. Construct a "scenario-adaptive evaluation" system: By dynamically loading dimension weight vectors, the system enables the switching of optimal strategies for the same model in different scenarios such as liveness detection and document verification, with a replacement rate of up to 97.2%, reducing the maintenance cost of multiple models; 4. Significantly improved overall performance: On a private dataset (10,000 labeled images), the method of this invention achieved an average accuracy of 96.4% and an AUC of 0.97, which is 9.7 percentage points higher than the pure large model solution and 13.2 percentage points higher than the traditional CV solution, verifying the effectiveness and advancement of the technical solution. Attached Figure Description

[0026] Figure 1 This is the overall flowchart of this application.

[0027] Figure 2 This is a schematic diagram of the dual-channel collaborative discrimination unit structure.

[0028] Figure 3 The flowchart for Qwen2.5-VL dimension-level LoRA fine-tuning training is shown.

[0029] Figure 4 Example of a visualization interface for outputting quality reports.

[0030] Figure 5 A bar chart comparing the experimental results.

[0031] Figure 6 A visualization of the experimental results. Detailed Implementation

[0032] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0033] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0034] Please see Figure 1-3 A method for generating face quality labels based on the collaboration of multimodal large models and computer vision algorithms, such as... Figure 1 The diagram illustrates the complete data flow and module interaction from image input to quality report output. It includes the following steps: Image preprocessing standardization accepts raw input face images (supported formats include: JPG, PNG, JPEG, BMP, HEIC, with a recommended resolution of no less than 640×480 pixels). Preprocessing includes face detection, key point localization and affine alignment, size normalization, and color space standardization to ensure the consistency and stability of subsequent dual-channel model input.

[0035] The preprocessed face image is input into the "quality dimension decomposition module" and distributed and decoupled in parallel according to nine preset independent and orthogonal quality dimensions to achieve fine-grained defect localization.

[0036] A dual-channel discrimination mechanism is used to evaluate each defined quality dimension.

[0037] The output results of the two channels in each dimension are input into the "Dynamic Confidence Perception Fusion Module". The fusion weights are dynamically generated by a lightweight meta-learner to replace static weighting, realize intelligent decision fusion, and output the final quality label of that dimension.

[0038] The binary labels of all nine dimensions are aggregated to form a structured quality label set. Based on the current business scenario, a preset dimension importance weight vector is loaded to calculate the comprehensive face quality score.

[0039] Output a structured JSON quality report, including an overall score, detailed tags for each dimension, and actionable retake suggestions, directly interfaced with downstream business systems. For example... Figure 4 As shown, this demonstrates the front-end presentation of the overall score, dimension tags, and retake suggestions.

[0040] More specifically, it includes the following steps: Face detection was performed using a lightweight RetinaFace-MobileNet 0.25 model, locating all face bounding boxes in the image and removing tiny face targets with a resolution lower than 30×30 pixels. Subsequently, a 106-point high-precision keypoint detection model was used to extract the coordinates of facial features, and an affine transformation matrix was calculated based on the eyes and the tip of the nose to rotate and scale the face region to a standard frontal pose. The cropped and aligned faces were then uniformly scaled to a resolution of 256×256 pixels. Finally, the image was converted from the BGR color space to RGB format, and pixel value normalization was performed. , where the mean vector Standard deviation vector It conforms to the ImageNet standard input specification, ensuring the stability of model inference.

[0041] The face detection model takes less than 30ms for single-image inference under an NVIDIA RTX 4090 GPU environment; the key point localization uses a 106-point model (such as PFLD or an improved version of MobileFaceNet), with an alignment accuracy error of less than 2 pixels; the normalization parameters are fixed using ImageNet statistics to ensure the consistency of the model input distribution and improve inference stability.

[0042] By constructing a dimensional routing mechanism, the same image is simultaneously distributed to nine independent processing channels. Each channel corresponds to a specific quality dimension: D1 - Number of Faces (whether it contains only a single valid face), D2 - Secondary Shooting (whether it is a screen or photo paper copy), D3 - Irregular Lighting (whether it is overexposed or underexposed), D4 - Image Blur (whether the facial area is out of focus or motion-blurred), D5 - Facial Occlusion / Incompleteness (whether facial features are obscured by more than 30%), D6 - Not Wearing a Hat or Headscarf (whether a hat or headscarf is worn), D7 - Pose Deviation (whether the head's Euler angle exceeds the limit), D8 - Closed Eyes (whether both eyes are closed), and D9 - Strabismus (whether the line of sight deviates from the camera axis by more than 15°). This decoupled design avoids interference between dimensions and supports independent optimization and evaluation.

[0043] By examining each quality dimension A dual-channel collaborative discrimination unit is constructed, utilizing a finely tuned multimodal large model for semantic-level understanding and a dedicated computer vision algorithm for deterministic detection, forming a complementary discrimination mechanism. For example... Figure 2 As shown, the parallel processing, confidence calculation, and dynamic fusion process of the VLM and CV channels are demonstrated in detail.

[0044] The first channel (semantic understanding channel): The preprocessed image is concatenated with dimension-specific text prompts and input into the Qwen2.5-VL model, which has been fine-tuned for dimension-level LoRA. The prompt format is: "Please determine whether this face image has a [dimension name] problem? Answer only 'yes' or 'no'." The model outputs a semantic summary of the existence of the dimension defect. Fine-tuning employs a low-rank adapter (LoRA). It freezes the backbone parameters, trains only the adapter layer, saves an independent adapter for each dimension, shares the visual encoder, dynamically loads during inference, reduces memory usage by 60%, and supports multi-task concurrency.

[0045] The Qwen2.5-VL model performs independent fine-tuning for each quality dimension during the training phase. The training dataset contains 100,000 labeled images, covering both real-world scenes and synthetic data. It employs LoRA (Low-Rank Adaptation) for efficient fine-tuning, reducing GPU memory usage by 60%. During inference, the fine-tuning parameters for all nine dimensions are loaded in batches, enabling multi-task discrimination within a shared backbone network.

[0046] The second channel (deterministic detection channel): invokes a dedicated computer vision algorithm module optimized for this dimension to output a deterministic score based on the low-level features of the image. The algorithm implementations for each dimension are as follows: Face count: YOLOv8s-face model detection, IoU threshold 0.65, clustering for deduplication; Secondary re-enhancing: Calculating the energy entropy of high-frequency coefficients in an 8×8 DCT block. ,like; It was determined to be a remake; Illumination anomalies: Calculate the mean value of the Y channel. ,when or It is judged to be either too dark or too exposed; Image blurring: Calculate the gradient variance of the Laplacian operator within the facial mask region. , The image is deemed blurry. Face occlusion: Combining the visibility score of 106 key points with Face-Parsing semantic segmentation, if the occlusion area is greater than... Disqualified; : Non-hat-free state: Train a hat detection classifier based on the SegFormer architecture and output probability. Judged as wearing a hat; Attitude offset: Euler angles are estimated based on 3DMM or Buffalo models, when , or The posture was deemed excessive. Eyes closed state: The EAR (Eye Aspect Ratio) algorithm is used to calculate... , It was determined to be closed eyes; Strabismus: Calculate the angle between the vectors of the eye center and the tip of the nose. ,like It was diagnosed as strabismus.

[0047] Dynamic fusion can be achieved through the following technical solutions: Confidence quantification: Large model confidence: ,in This is a binary cross-entropy function; the lower the value, the more uncertain the model. Confidence level of CV algorithm: ,in This is the original response value of CV. The mean response of this dimension on the validation set is normalized to [0,1] by Sigmoid.

[0048] Dynamic weight generation: Construct a lightweight meta-learner (single-layer MLP, input dimension 2, output dimension 1), parameters , Obtained through offline training: ; The training objective is to minimize the validation set weighted F1 loss. ,in This represents the dimensional importance coefficient.

[0049] Fusion score and label determination: , Set a specific threshold (This can be determined based on the validation set ROC curve), if If it is determined to be defective, a binary label is generated. ,otherwise ;like Figure 3 As shown, the process of data preparation, instruction construction, parameter freezing, and adapter training is illustrated.

[0050] The lightweight meta-learner is implemented in PyTorch, containing only one hidden layer (2→8→1), with ReLU activation function and Sigmoid output layer. The training data comes from 2,000 validation set images, and the parameters are fixed after 100 training rounds. There is no training overhead during the inference phase, and the single-dimensional fusion takes less than 5ms.

[0051] By using nine dimensions of labels Summarized into a set Then, the "Scene Adaptive Comprehensive Evaluation Model" is invoked, and the corresponding dimension weight vector is dynamically loaded based on the scene identifier (such as scene=live / id) passed in by the API. Calculate the overall score: ,satisfy ; Typical scenario weight configuration: Liveness detection scenarios: ; ID photo verification scenario: ; Weight configuration supports remote hot updates via the configuration center API without requiring a service restart.

[0052] The evaluation results are output using a standardized JSON Schema, with fields including: group_id (batch identifier), image_id (unique image ID), overall_score (overall quality score from 0 to 1), dimension_labels (a nine-dimensional binary label dictionary), and suggestion (natural language re-image suggestion, such as "Re-image suggestion: There are closed eyes and it is suspected to be a re-image"). The output results can be directly used in industrial applications such as automated image selection, user-end re-image prompts, facial recognition system access control, and quality statistical analysis dashboards, achieving an end-to-end closed loop.

[0053] The output results are pushed to downstream systems via RESTful API or Kafka message queue, and can be directly used in industrial-grade application scenarios such as automatic image filtering, user-end re-photo prompts, facial recognition system access control, and quality statistical analysis dashboards, to achieve end-to-end closed loop.

[0054] { "group_id": "ID_20251001_12345", "image_id": "img_001", "overall_score": 0.87, "dimension_labels": { "multiple_faces": 0, "replay": 1, "lighting": 0, "blur": 0, "occlusion": 0, "non_bareheaded": 0, "pose": 0, "closed_eye": 1, "gaze": 0 }, "suggestion": "Reshoot recommended: There are scenes with closed eyes and it appears to be a remake." } Experimental verification: To comprehensively evaluate the technical advancement and engineering applicability of the proposed method, a systematic comparative experiment was conducted on a private high-quality face quality assessment dataset (China Telecom Face-Quality Dataset, CTFQ-10K), comparing it with five representative mainstream methods across four dimensions: accuracy, robustness, efficiency, and scene adaptability. All experiments were performed in a unified hardware environment (NVIDIA RTX 4090, Intel Xeon Gold 6330, 64GB RAM) and software stack (Ubuntu 20.04, PyTorch 2.1, CUDA 12.1) to ensure fair and comparable results.

[0055] Experiment 1: Comparison of accuracy and robustness: For method selection, the traditional CV method uses FaceQnet v2, a widely deployed end-to-end quality regression model based on ResNet-50, which is one of the NIST FRVT's officially recommended solutions. The pure large-model method uses LLaVA-1.6-13B, one of the most powerful open-source image-text understanding models currently available, supporting instruction following, as the semantic discrimination baseline. The static fusion method uses MTCNN+CLIP Score Ensemble, statically weighting the MTCNN detection results with the CLIP image-text similarity (…). , (), representing the existing "simple fusion" scheme. The adaptive fusion method, using DynaFuse, a newly proposed multi-model dynamic fusion framework based on attention mechanism, is used to compare the superiority of the "meta-learner" scheme of this invention.

[0056] Dataset: CTFQ-10K, containing 10,000 real-world face images (from telecom liveness registration, ID photo uploads, and video call screenshots), annotated with nine dimensions (Kappa>0.87) by 5 professional annotators, divided into a training set of 7,000, a validation set of 2,000, and a test set of 1,000.

[0057] Evaluation metrics: Average accuracy, macro average F1 score, AUC, and F1 improvement in key dimensions (reproduction, occlusion, and squint). Figure 5 As shown, the method of this application is compared with traditional CV, pure LLM, and end-to-end models in terms of accuracy, F1 score, AUC, and latency.

[0058] Table 1. Experimental Comparison Results

[0059] Conclusion: This invention comprehensively outperforms the three core metrics of Accuracy, F1, and AUC, improving them by +5.1%, +4.5%, and +3.2% respectively compared to the second-best method, DynaFuse, thus verifying the effectiveness of the "dynamic confidence-aware fusion" mechanism. In the three semantically complex dimensions of re-photographing, occlusion, and squint, this invention improves the F1-score by +6.8%, +4.5%, and +5.7% respectively compared to DynaFuse, demonstrating the significant advantage of lightweight meta-learners in handling uncertain fusion. In terms of inference efficiency, this invention (210ms) far surpasses LLaVA (850ms) and DynaFuse (420ms), verifying the engineering superiority of the "dimensional LoRA + shared backbone" architecture.

[0060] Experiment 2: Verification of Scene Adaptability This application operates in two modes—Ours-Live (liveness detection weights) and Ours-ID (ID photo weights); the baseline for comparison is the fixed-weight model DynaFuse-Fixed and the scene-specific models FaceQnet-Live / FaceQnet-ID.

[0061] Evaluation metrics: attack interception rate (liveness) or human consistency (ID photo) on the corresponding scenario test set, false pass rate, and strategy switching time.

[0062] Liveness detection scenario (5,000 real registration requests): Table 2 Comparison Results of Liveness Detection Scenarios

[0063] ID photo verification scenario (3,000 manually annotated images): Table 3 Comparison Results of ID Photo Verification Scenarios

[0064] Strategy switching performance: Table 4: Performance Indicator of Strategy Switching

[0065] Figure 6 This is a visualization of the experimental results (Note: Due to issues such as facial privacy and portrait rights in the private dataset, AI-generated images were used, and the pixel count was reduced to simulate real images; the real images are of lower quality).

[0066] Conclusion: In the liveness detection scenario, the present invention achieves a re-photographing interception rate of up to 96.2% and a false pass rate of only 1.8%, which is significantly better than dedicated models and fixed weight models. In the ID photo scenario, the consistency with manual review reaches 96.5%, with a false judgment rate of 3.5%, which is close to the level of human review. The hot switching time of weight vector is less than 50ms, realizing true "one model for multiple scenarios", with a replacement rate of 97.2%, which greatly reduces the maintenance cost of multiple models.

[0067] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

Claims

1. A method for generating face quality labels based on the collaboration of multimodal large models and computer vision algorithms, characterized in that, Includes the following steps: For each of the multiple predetermined quality dimensions of a face image, perform the following steps: Obtain the first probability output generated by the multimodal large model, finely tuned by instructions, based on the image and corresponding dimension text prompts. ; Obtain the second probability output generated by a computer vision algorithm designed for this dimension based on low-level image feature analysis. ; The fusion score for this dimension is calculated using the dynamic confidence-aware fusion module. ,in and The fusion weights are dynamically generated based on the real-time confidence scores of the first and second probability outputs. Based on fusion score Generate quality labels for this dimension.

2. The face quality label generation method based on the collaboration of multimodal large model and computer vision algorithm according to claim 1, characterized in that, The dynamically generated fusion weights and include: Calculate the semantic confidence index of the first channel ,in It is a binary cross-entropy function; Calculate the characteristic response intensity of the second channel ,in This represents the raw response value of the computer vision algorithm in this dimension. This represents the mean response of this dimension on the validation set. This is the scaling factor; Will and Input a lightweight meta-learner to generate fusion weights .

3. The face quality label generation method based on the collaboration of multimodal large model and computer vision algorithm according to claim 2, characterized in that, The lightweight meta-learner uses functions Generate weights, where For the Sigmoid function, , and Let be the learnable parameters obtained by training by minimizing the validation set loss, and .

4. The face quality label generation method based on the collaboration of multimodal large model and computer vision algorithm according to claim 1, characterized in that, The multimodal large model employs dimension-level LoRA fine-tuning, independently training and saving low-rank adapter parameters for each quality dimension, and dynamically loading the corresponding dimension's adapter on the shared backbone network during inference.

5. The face quality label generation method based on the collaboration of multimodal large model and computer vision algorithm according to claim 1, characterized in that, In the dimension used to detect closed-eye states, specialized computer vision algorithms calculate the eye aspect ratio. Make a judgment, among which These are the vertical distances between the eyes. The horizontal distance to the eye.

6. The face quality label generation method based on the collaboration of multimodal large model and computer vision algorithm according to claim 1, characterized in that, In the dimension used to detect re-photographing, a dedicated computer vision algorithm calculates the energy entropy of high-frequency coefficients in the DCT frequency domain of the image. Make a judgment.

7. The face quality label generation method based on the collaboration of multimodal large model and computer vision algorithm according to claim 1, characterized in that, Also includes: Summarize all quality labels across all dimensions to form a set. ; Loading dimension importance weight vector based on business scenario ,in ; Calculate the overall face quality score .

8. The face quality label generation method based on the collaboration of multimodal large model and computer vision algorithm according to claim 7, characterized in that, When the business scenario is liveness detection, the weight vector is configured as follows: ; When the business scenario is ID photo verification, the weight vector is configured as follows: .

9. A face quality assessment device, characterized in that, include: One or more processors; The memory stores computer instructions that, when executed by one or more processors, cause the device to perform a face quality label generation method based on the collaboration of a multimodal large model and computer vision algorithms as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by the processor, the program implements a face quality label generation method based on the collaboration of a multimodal large model and computer vision algorithms as described in any one of claims 1-8.

Images