AI active learning-based label quality dynamic verification and authority control method

Abstract

The invention belongs to the technical field of AI training data annotation, and particularly relates to an AI active learning-based annotation quality dynamic verification and authority management and control method, which comprises the following steps of: initializing a core module, a rule base and an authority matrix; the AI active learning model samples and distributes samples based on data uncertainty and annotator ability matching; the dynamic verification module executes multi-dimensional verification and feeds back abnormal features; the authority management and control module dynamically adjusts the authority according to the verification result; and iteratively optimizing the model and the rule base based on feedback, and circulating until the annotated data meets a quality threshold value. According to the method, autonomous optimization of labeling quality, resource dynamic configuration and advanced avoidance of abnormal labeling are realized, the abnormal labeling recognition accuracy and the resource configuration efficiency are remarkably improved, the model iteration speed is increased, and excessive dependence on manual intervention is not needed.

Description

Technical Field

[0001] This invention belongs to the field of AI training data annotation technology, specifically a method for dynamic verification of annotation quality and access control based on AI active learning. Background Technology

[0002] With the deep application of artificial intelligence technology in fields such as image recognition, natural language processing, and voice interaction, high-quality labeled training data has become the core foundation for improving AI model performance. Its accuracy and consistency directly determine the model's training effect, and quality defects can easily lead to problems such as insufficient model generalization ability and decision bias. Therefore, quality control and access control of AI training data labeling have become key factors restricting the efficiency of AI technology implementation.

[0003] Current AI training data annotation technology mainly revolves around a linear process of sample annotation, quality checking, and model training, which has the following core shortcomings: 1. Existing verification methods mostly adopt a static rule-driven approach, with preset fixed thresholds and business rules, and checks are completed through manual sampling or simple algorithm comparison. Static rules cannot be updated with changes in data characteristics, making it difficult to identify new types of abnormal annotations; verification is disconnected from model training, and results are not fed back to the sampling stage, making it easy for high-risk samples to be repeatedly assigned to low-skilled annotators; cross-annotator consistency verification does not consider differences in annotation habits and abilities, resulting in low accuracy. 2. The existing access control system uses a fixed-level plus manual adjustment model, resulting in annotation staff levels and access ranges remaining unchanged for a long time. This leads to a disconnect between access and annotation quality, as access is not dynamically adjusted based on historical verification accuracy and the number of abnormal annotations, resulting in both resource waste and quality risks. Furthermore, access adjustment relies on manual evaluation, which is highly lagging and exacerbates the conflict between quality and efficiency. 3. The AI ​​active learning model only samples high-value samples based on data uncertainty, without combining the annotator's permission level and historical quality performance. Key samples are easily assigned to low-ability annotators, resulting in waste. The model iteration does not receive feedback on abnormal features from the verification process, making it impossible to optimize the sampling strategy. The permission control module does not participate in sample allocation, and the three work independently without forming a synergistic benefit. 4. The existing process exhibits a linear, unidirectional characteristic, with no reverse feedback mechanism at each stage. Abnormal data is only processed for rework and is not used to optimize the sampling strategy; permission adjustments do not guide sample allocation, and model iteration results do not update the verification rules. This model leads to annotation quality relying on manual intervention, a high rate of abnormal annotations, inefficient resource allocation, and the defects are further amplified as the annotation scale increases, making it difficult to meet the needs of large-scale, high-quality annotation data.

[0004] In summary, existing technologies suffer from core pain points such as rigid verification rules, inflexible permission allocation, lack of module collaboration, and insufficient closed-loop mechanisms, resulting in unstable annotation quality, inefficient resource allocation, and slow model iteration. There is an urgent need for a technical solution that enables deep collaboration between active learning, dynamic verification, and permission control to break through the existing bottlenecks. Summary of the Invention

[0005] To address the problems of rigid verification rules, inflexible permission allocation, lack of module collaboration, and insufficient closed-loop mechanisms in existing AI training data annotation technologies, this invention provides a dynamic verification and permission control method for annotation quality based on AI active learning. Through deep collaboration and closed-loop linkage of the AI ​​active learning model, dynamic verification module, and permission control module, it achieves autonomous optimization of annotation quality, dynamic allocation of annotation resources, and early avoidance of abnormal annotations, thus solving the problems of unstable annotation quality, inefficient resource allocation, and slow model iteration in existing technologies.

[0006] The technical solution adopted by this invention to solve its technical problem is: a method for dynamic verification of annotation quality and access control based on AI active learning, comprising the following steps: S1. Initialize the AI ​​active learning model, the annotation quality dynamic verification rule base, the annotator permission matrix, the dynamic verification module, and the permission control module. The permission matrix is ​​associated with the annotator's historical verification performance. S2. The AI ​​active learning model samples high-value samples to be labeled based on the feature uncertainty of the data to be labeled, and assigns them to junior annotators, intermediate annotators, senior annotators or reviewers according to the current permission matrix; S3. The dynamic verification module performs multi-dimensional verification on the annotation results returned by the annotator and generates a verification report including annotation accuracy, annotation consistency deviation, and logical compliance. S4. The access control module dynamically adjusts the annotation operator's access level, annotation data range, and operation permissions based on the verification report; S5. Input the verified labeled data into the AI ​​active learning model to complete the model iteration optimization, and update the feature extraction capability and sampling strategy; S6. Repeat steps S2 to S5 until the labeled data meets the preset quality threshold. The AI ​​active learning model, the dynamic verification module, and the access control module collaborate through targeted data interaction to form a closed-loop collaborative mechanism. The sampling results of the AI ​​active learning model are input into the dynamic verification module, and the verification report of the dynamic verification module is input into the access control module. The adjustment results of the access control module guide the sample allocation of the AI ​​active learning model in turn. Through the complete linkage of sampling, verification, access control adjustment, and model optimization, the annotation quality is dynamically improved.

[0007] Preferably, the sampling strategy of the AI ​​active learning model in step S2 is as follows: a multi-objective optimization function is constructed by combining the uncertainty of data features, the historical verification accuracy of the annotator and the permission level. The uncertainty of data features is determined by calculating the feature distribution entropy value of the data to be annotated, and data samples with high uncertainty and suitable for senior annotators are sampled first.

[0008] Preferably, the multi-dimensional verification of the dynamic verification module includes data internal feature consistency verification, cross-annotator same sample annotation result consistency verification, and annotation logic compliance verification with preset business rules. The annotation quality dynamic verification rule library is dynamically updated according to the iteration results of the AI ​​active learning model, and new verification rules corresponding to high-risk annotation features identified by the AI ​​active learning model are added.

[0009] Preferably, the dimensions of permission adjustment in step S4 include permission for annotation data complexity, permission for the upper limit of annotation quantity, permission for self-submission of annotation results, permission for annotation results to be reviewed, and permission for configuration of dynamic verification rule base for annotation quality. The adjustment is based on the accuracy trend of consecutive annotations, annotation consistency deviation, and number of abnormal annotations in the verification report.

[0010] Preferably, the iteration method of the AI ​​active learning model is semi-supervised learning, in which the labeled data that has passed the verification is used as labeled samples, and the unlabeled data and the abnormal data that has failed the verification are used as unlabeled samples. The model feature extractor is optimized by constructing positive and negative sample pairs for comparative learning, and the weight parameters of the data uncertainty assessment model are updated at the same time.

[0011] Preferably, the anomaly labeling and processing mechanism of the dynamic verification module is as follows: When the annotation result is detected to meet any of the following conditions, it is marked as an abnormal annotation: first, the internal feature consistency score is lower than a preset threshold; second, the cross-annotator consistency deviation is higher than a preset threshold; and third, it violates at least one core business rule. After an anomaly is marked, a review process by senior annotators or reviewers is triggered. At the same time, the anomaly marking features are fed back to the AI ​​active learning model. The AI ​​active learning model optimizes the next round of sampling strategy based on the anomaly marking features, reducing the allocation of similar high-risk samples to junior annotators.

[0012] Preferably, in the hierarchical system of the permission matrix, junior annotators can only annotate low-complexity data, are limited by the upper limit of the number of annotations, and the annotation results need to be reviewed; intermediate annotators can annotate low-complexity and medium-complexity data, and can independently submit the annotation results under high accuracy conditions; senior annotators can annotate full-complexity data, have partial review permissions, and have a higher upper limit of the number of annotations than junior annotators; reviewers only have review permissions and the permission to fine-tune the dynamic verification rule base of the annotation quality. The dynamic adjustment threshold of permissions is dynamically set according to the current performance of the AI ​​active learning model, and the current performance of the AI ​​active learning model includes feature extraction accuracy and sample classification accuracy.

[0013] Preferably, the sampling priority ranking rule of the AI ​​active learning model is as follows: the priority of high uncertainty data sample allocation is determined by the product of feature uncertainty score, historical verification accuracy of the annotator, and permission level coefficient, wherein the permission level coefficient is positively correlated with the current permission level of the annotator.

[0014] Preferably, the dynamic verification module employs a verification algorithm including a feature consistency verification algorithm based on an attention mechanism and a cross-annotator association verification algorithm based on a graph neural network. The feature consistency verification algorithm based on an attention mechanism focuses on the labeling matching degree of key data features, while the cross-annotator association verification algorithm based on a graph neural network models the association of labeling habits among annotators. The algorithm parameters are dynamically optimized according to the distribution characteristics of the labeled data after the permission adjustment.

[0015] Preferably, the sampling strategy of the AI ​​active learning model is directly related to the verification result of the dynamic verification module. The rule update of the dynamic verification module depends on the iterative output of the AI ​​active learning model, and the adjustment result of the permission control module guides the sample allocation of the AI ​​active learning model. The AI ​​active learning model optimizes its feature extraction capability through the abnormal feature feedback of the dynamic verification module. The dynamic verification module improves the verification rules through the iterative results of the AI ​​active learning model. The permission control module achieves precise permission adjustment through the verification report of the dynamic verification module. The directional interaction and bidirectional feedback between the above modules break through the technical bottlenecks of rigid verification rules, rigid permission allocation, and disconnect between model learning and quality control in traditional annotation quality control. This achieves the technical effects of autonomous optimization of annotation quality, dynamic allocation of annotation resources, and early avoidance of abnormal annotations. These technical effects cannot be achieved by running a single module independently or by simply stacking them together.

[0016] The beneficial effects of this invention are as follows: 1. The AI-based active learning-based dynamic annotation quality verification and access control method described in this invention optimizes the sampling strategy by iteratively linking the dynamic annotation quality verification rule base and the AI ​​active learning model with the back feedback of abnormal annotation features. This enables the verification rules to adapt to changes in data features and annotation scenario requirements in real time, significantly improving the accuracy of identifying novel abnormal annotations. 2. The AI-based active learning-based dynamic verification and access control method for annotation quality described in this invention achieves precise matching between permissions and annotation capabilities by dynamically adjusting permission levels based on the real-time performance of annotation personnel, avoiding resource waste and quality risks, and significantly improving the efficiency of annotation resource allocation. 3. The AI-based active learning-based dynamic verification and access control method for annotation quality described in this invention achieves a synergistic gain beyond the superposition of a single module by using the bidirectional feedback and linkage adaptation between the AI ​​active learning model dynamic verification module and the access control module to simultaneously improve annotation quality and efficiency. Attached Figure Description

[0017] The invention will now be further described with reference to the accompanying drawings.

[0018] Figure 1 This is a block diagram of the initialization phase of the present invention; Figure 2 This is a block diagram of the sample sampling and allocation stage of the present invention; Figure 3 This is a block diagram of the dynamic verification and anomaly handling stages of the present invention; Figure 4 This is a flowchart of the dynamic permission adjustment stage of the present invention; Figure 5 This is a flowchart of the model iteration and rule update stages of this invention; Figure 6 This is an overview block diagram of the closed-loop circulation of the present invention. Detailed Implementation

[0019] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figures 1-6 As shown, the core of the technical solution of this invention lies in constructing a collaborative mechanism for sampling, verification, permission adjustment, and model optimization. Functional linkage is achieved through targeted data interaction and two-way feedback. The specific working process and implementation details are as follows: The initialization phase is as follows: When the system starts, the following initialization operations are performed synchronously: AI active learning model initialization: A pre-trained feature extraction network is used as the basic architecture. The feature extraction network can be a convolutional neural network, a Transformer network, or a graph neural network, and the appropriate one is selected according to the data type to be labeled. The data uncertainty assessment model is initialized, and an entropy-based assessment framework is adopted. The feature extraction dimensions and uncertainty calculation baseline logic are preset. The entropy calculation is based on the full-dimensional feature distribution of the data. Initialization of the dynamic verification rule library for annotation quality: Three types of basic verification rules are preset, including internal data feature consistency verification rules, cross-annotator annotation consistency verification rules, and business rule compliance verification rules. Internal data feature consistency verification rules clarify the weight allocation principle for key features and auxiliary features, with key features having a higher weight than auxiliary features; cross-annotator annotation consistency verification rules define the judgment logic for the matching degree of annotation results, clarifying the core evaluation dimensions of the matching degree; business rule compliance verification rules input industry-wide core business logic and scenario-specific business constraints, forming a structured rule set. Annotator permission matrix initialization: This involves linking the annotators' historical verification performance data, including historical annotation accuracy, consistency deviation records, and abnormal annotation handling records, to clarify the initial permission scope for junior, intermediate, senior, and reviewer annotators. Initial permission allocation is based primarily on historical verification performance, combined with the complexity requirements of the annotation tasks, ensuring that initial permissions are appropriate for the annotators' capabilities. Dynamic verification module initialization: Load the feature consistency verification algorithm based on the attention mechanism and the cross-annotator association verification algorithm based on graph neural network, and preset the initial parameters of the algorithm, including the attention window size and the initial threshold of graph node connection. The initial parameters are preset according to the mainstream annotation data types and scenarios, and support subsequent adaptive adjustment; Access control module initialization: Configure the core dimensions and initial judgment logic for permission adjustment, and clarify the adjustment trigger condition framework for each level of permission. The framework includes the core indicator types and logical relationships that trigger the adjustment. Sample collection and allocation phase: Feature Extraction and Uncertainty Assessment: The AI ​​active learning model performs full-dimensional feature extraction on the labeled data. For text data, it focuses on semantic features, syntactic features, and entity features; for image data, it focuses on pixel features, texture features, and structural features; and for speech data, it focuses on acoustic features and prosodic features. The feature distribution entropy value is calculated through the data uncertainty assessment model. The higher the entropy value, the stronger the data uncertainty and the higher the sample value. The entropy value calculation process strictly follows the preset uncertainty calculation benchmark logic. Multi-objective optimization function construction: A multi-objective optimization function is constructed with the objectives of maximizing sample value, maximizing annotation accuracy, and rationalizing resource allocation. The function input variables include data feature uncertainty score, annotator's historical verification accuracy, and annotator's current permission level. The weights of each variable are dynamically allocated according to the priority of the annotation task. Quality-priority tasks receive a higher weight for accuracy, while efficiency-priority tasks receive a higher weight for resource allocation. The weight allocation ratio is preset through the task configuration interface. Sampling and Allocation Execution: Based on the output of the multi-objective optimization function, a greedy sampling algorithm is used to select high-value samples for annotation. The allocation priority of each sample is calculated through a sampling priority ranking rule, which is the product of feature uncertainty score, annotator's historical verification accuracy, and permission level coefficient. The permission level coefficient is positively correlated with the annotator's current permission level, and a corresponding correlation is preset according to the permission level system, with a larger coefficient for higher permission levels. High-uncertainty samples are preferentially allocated to senior annotators or reviewers, medium-uncertainty samples are allocated to intermediate annotators, and low-uncertainty samples are allocated to junior annotators, achieving precise matching between samples and annotators' capabilities. Dynamic verification phase: After receiving the annotation results returned by the annotator, the dynamic verification module performs multi-dimensional verification according to the following process: Internal feature consistency verification: The feature consistency verification algorithm based on the attention mechanism is called. The key features of the data are located through the self-attention mechanism. The matching degree between the key feature annotation results and the auxiliary feature annotation results is calculated. The matching degree is calculated using the cosine similarity algorithm to generate an internal feature consistency score. The higher the score, the better the consistency. Cross-annotator consistency verification of annotation results for the same sample: A cross-annotator association verification algorithm based on graph neural network is called to construct an association graph with annotators as graph nodes and annotation habit similarity as edge weights. Annotation habit similarity is calculated by comparing historical annotation results; the similarity of annotation results of the same batch of annotators for the same sample is calculated to generate a cross-annotator consistency score. Compliance verification of annotation logic and preset business rules: Traverse the business rules in the annotation results and annotation quality dynamic verification rule library, match them one by one to determine whether there is any violation, record the type and degree of violation, and classify the degree of violation according to the scope and severity of impact; Verification report generation and anomaly handling: Integrate the three types of verification results to generate a verification report that includes annotation accuracy, annotation consistency deviation, logical compliance, violation details, and abnormal annotation location; if the internal feature consistency score is lower than the preset threshold, the cross-annotator consistency score is lower than the preset threshold, or there is a violation of core business rules, it is marked as an abnormal annotation, triggering the review process. A senior annotator or reviewer will conduct a second annotation verification, and the second verification result will be used as the final annotation result to update the verification report. Dynamic permission adjustment phase: The adjustment is based on data collection and analysis: the access control module extracts the accuracy trend of consecutive annotations from the verification report, including three trend types: rising, stable, and falling, as well as changes in annotation consistency deviation, the number of abnormal annotations, and the distribution of abnormality types, to establish an annotation worker competency assessment model. The model uses the above indicators as input and outputs the annotation worker's current competency level assessment result. Dimensional execution of permission adjustment: Data complexity access permissions: Based on the capability assessment results, adjust the data complexity level that annotators can access. If the capability improves, access to higher complexity data will be expanded, and if the capability declines, access to low complexity data will be restricted. The complexity level classification is consistent with the preset complexity standard of the annotation task. Maximum Annotation Quantity: Set a reasonable maximum annotation quantity based on the annotator's current skill level to avoid quality degradation due to excessive quantity. The maximum value is set based on the optimal annotation efficiency of annotators with similar skills. Self-submission permission for annotation results: Annotators with high accuracy and low outlier annotation rate are granted self-submission permission and can directly enter the subsequent process without review; Annotation results require review authority: Annotators with low accuracy or high anomaly annotation rate will have their annotation results forcibly triggered to trigger the review process. Only after the review is passed can the subsequent process be carried out. Dynamic verification rule base configuration permissions for annotation quality: Only senior annotators and reviewers can obtain rule fine-tuning permissions to adapt to special annotation scenarios. Fine-tuning operations must be recorded for traceability. Dynamically adapting thresholds: The thresholds for adjusting permissions at each level are dynamically set based on the current performance of the AI ​​active learning model. The current performance of the model is evaluated by the accuracy of feature extraction and the accuracy of sample classification. If the accuracy of feature extraction of the model increases, the threshold for adjusting permissions will be increased appropriately. If the performance of the model decreases, the threshold will be decreased to strictly control permissions. Model iterative optimization phase: Semi-supervised learning iteration: Labeled data that passes the dynamic verification module is used as labeled samples, and unlabeled data and abnormal data that fails verification are used as unlabeled samples. A training dataset is constructed according to a preset ratio, which is set according to the scale of the labeling task and the complexity of the data. Comparative learning is carried out by constructing positive and negative sample pairs. Positive sample pairs are high-quality labeled samples of the same type of data, and negative sample pairs are heterogeneous data or abnormal labeled samples. The weight parameters of the model feature extractor and the data uncertainty assessment model are optimized using a contrastive loss function. The optimization process is executed according to a preset number of iterations and learning rate. Two-way feedback optimization: The dynamic verification module feeds back anomaly labeling features to the AI ​​active learning model, including anomaly feature type, distribution location, frequency of occurrence, and verification rule adaptability evaluation results. Based on this feedback, the AI ​​active learning model adjusts the feature extraction dimension and uncertainty evaluation weight to enhance the ability to identify high-risk anomaly features and optimize the next round of sampling strategy. The dynamic annotation quality verification rule base updates the verification rules based on the iteration results of the AI ​​active learning model, adds verification logic corresponding to high-risk annotation features identified by the model, deletes invalid and redundant rules, and adjusts the rule verification weights to ensure that the rule base adapts to changes in data features in real time. Algorithm parameter adaptation: The algorithm parameters of the dynamic verification module are adaptively adjusted according to the feature extraction results after the AI ​​active learning model iteration, including the weight allocation of the attention mechanism and the node connection threshold of the graph neural network, with the adjustment direction aimed at improving the verification accuracy and adaptability. Closed-loop cycle stage: Repeat the sample sampling and allocation, dynamic verification, dynamic permission adjustment, and model iterative optimization steps according to the following logic until the labeled data meets the preset quality threshold: Loop triggering mechanism: After each batch of annotations is completed, the next loop process is automatically started. The batch size is adaptively set according to the scale of the annotation task, data complexity and hardware processing capabilities. Dynamic adaptation and adjustment: During the loop, the sampling strategy of the AI ​​active learning model is dynamically optimized according to the results of permission adjustment and feedback of abnormal annotation, giving priority to sampling high-value feature samples that are not fully labeled; the algorithm parameters of the dynamic verification module are adaptively adjusted according to the distribution characteristics of the labeled data to improve the verification accuracy of complex features; the adjustment logic of the permission control module is updated with the model performance iteration to optimize the permission adjustment threshold and triggering conditions. Termination condition determination: The preset quality thresholds include three core indicators: annotation accuracy, annotation consistency deviation, and abnormal annotation rate. When all three indicators meet the preset requirements and remain stable for two consecutive batches, the loop terminates and the final annotation data is output.

[0021] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for dynamic verification of annotation quality and access control based on AI active learning, characterized in that, Includes the following steps: S1. Initialize the AI ​​active learning model, the annotation quality dynamic verification rule base, the annotator permission matrix, the dynamic verification module, and the permission control module. The permission matrix is ​​associated with the annotator's historical verification performance. S2. The AI ​​active learning model samples high-value samples to be labeled based on the feature uncertainty of the data to be labeled, and assigns them to junior annotators, intermediate annotators, senior annotators or reviewers according to the current permission matrix; S3. The dynamic verification module performs multi-dimensional verification on the annotation results returned by the annotator and generates a verification report including annotation accuracy, annotation consistency deviation, and logical compliance. S4. The access control module dynamically adjusts the annotation operator's access level, annotation data range, and operation permissions based on the verification report; S5. Input the verified labeled data into the AI ​​active learning model to complete the model iteration optimization, and update the feature extraction capability and sampling strategy; S6. Repeat steps S2 to S5 until the labeled data meets the preset quality threshold. The AI ​​active learning model, the dynamic verification module, and the access control module collaborate through targeted data interaction. The sampling results of the AI ​​active learning model are input to the dynamic verification module, the verification report of the dynamic verification module is input to the access control module, and the adjustment results of the access control module guide the sample allocation of the AI ​​active learning model.

2. The method for dynamic verification of annotation quality and access control based on AI active learning according to claim 1, characterized in that, The sampling strategy of the AI ​​active learning model in step S2 is as follows: a multi-objective optimization function is constructed by combining the uncertainty of data features, the historical verification accuracy of the annotator and the permission level. The uncertainty of data features is determined by calculating the feature distribution entropy value of the data to be annotated, and data samples with high uncertainty and suitable for senior annotators are sampled first.

3. The method for dynamic verification of annotation quality and access control based on AI active learning according to claim 1, characterized in that, The multi-dimensional verification of the dynamic verification module includes data internal feature consistency verification, cross-annotator same sample annotation result consistency verification, and annotation logic compliance verification with preset business rules. The annotation quality dynamic verification rule library is dynamically updated according to the iteration results of the AI ​​active learning model, and new verification rules corresponding to high-risk annotation features identified by the AI ​​active learning model are added.

4. The method for dynamic verification of annotation quality and access control based on AI active learning according to claim 1, characterized in that, The dimensions of permission adjustment in step S4 include access permissions for annotation data complexity, upper limit permissions for the number of annotations, permission for self-submission of annotation results, permission for annotation results to be reviewed, and permission for configuring the dynamic verification rule base for annotation quality. The adjustment is based on the accuracy trend of consecutive annotations, annotation consistency deviation, and number of abnormal annotations in the verification report.

5. The method for dynamic verification of annotation quality and access control based on AI active learning according to claim 2, characterized in that, The AI ​​active learning model iterates using semi-supervised learning, treating the verified labeled data as labeled samples and the unlabeled data and the abnormal data that failed verification as unlabeled samples. It optimizes the model's feature extractor by constructing positive and negative sample pairs for comparative learning, while simultaneously updating the weight parameters of the data uncertainty assessment model.

6. The method for dynamic verification of annotation quality and access control based on AI active learning according to claim 3, characterized in that, The anomaly annotation handling mechanism of the dynamic verification module is as follows: When the annotation result is detected to meet any of the following conditions, it is marked as an abnormal annotation: first, the internal feature consistency score is lower than a preset threshold; second, the cross-annotator consistency deviation is higher than a preset threshold; and third, it violates at least one core business rule. After an anomaly is marked, a review process by senior annotators or reviewers is triggered. At the same time, the anomaly marking features are fed back to the AI ​​active learning model. The AI ​​active learning model optimizes the next round of sampling strategy based on the anomaly marking features, reducing the allocation of similar high-risk samples to junior annotators.

7. The method for dynamic verification of annotation quality and access control based on AI active learning according to claim 4, characterized in that, In the hierarchical system of the permission matrix, junior annotators can only annotate low-complexity data, are limited by the maximum number of annotations, and the annotation results need to be reviewed. Intermediate annotators can annotate low-complexity and medium-complexity data, and can submit the annotation results independently when the accuracy is high. Senior annotators can annotate full-complexity data, have some review permissions, and have a higher maximum number of annotations than junior annotators. Reviewers only have review permissions and the permission to fine-tune the dynamic verification rule base of the annotation quality. The dynamic adjustment threshold of permissions is dynamically set according to the current performance of the AI ​​active learning model. The current performance of the AI ​​active learning model includes feature extraction accuracy and sample classification accuracy.

8. The method for dynamic verification of annotation quality and access control based on AI active learning according to claim 2, characterized in that, The sampling priority ranking rule of the AI ​​active learning model is as follows: the priority of high uncertainty data sample allocation is determined by the product of feature uncertainty score, historical verification accuracy of the annotator, and permission level coefficient. The permission level coefficient is positively correlated with the current permission level of the annotator.

9. The method for dynamic verification of annotation quality and access control based on AI active learning according to claim 3, characterized in that, The dynamic verification module employs two verification algorithms: a feature consistency verification algorithm based on an attention mechanism and a cross-annotator association verification algorithm based on a graph neural network. The feature consistency verification algorithm based on an attention mechanism focuses on the labeling matching degree of key data features, while the cross-annotator association verification algorithm based on a graph neural network models the association of labeling habits among annotators. The algorithm parameters are dynamically optimized according to the distribution characteristics of the labeled data after the permission adjustment.

10. The method for dynamic verification of annotation quality and access control based on AI active learning according to any one of claims 1 to 9, characterized in that, The sampling strategy of the AI ​​active learning model is directly related to the verification results of the dynamic verification module. The rule updates of the dynamic verification module depend on the iterative output of the AI ​​active learning model. The adjustment results of the permission control module guide the sample allocation of the AI ​​active learning model. The AI ​​active learning model optimizes its feature extraction capabilities through the abnormal feature feedback of the dynamic verification module. The dynamic verification module improves its verification rules through the iterative results of the AI ​​active learning model. The permission control module achieves precise permission adjustment through the verification report of the dynamic verification module.

Images