Sare sample marking system based on generative model and Bayesian calibration

Abstract

The invention discloses a scarce sample marking system based on a generative model and Bayesian calibration. A sample analysis module extracts multi-modal features and divides few-sample / long-tail classes; the joint generation module calls a diffusion or language model, converts the core feature constraint into a generation condition, and synchronously outputs a candidate label-evidence pair; the consistency measurement module calculates a consistency score from the three dimensions of feature matching degree, logic rationality and field conformity; the uncertainty calibration module constructs a Bayesian neural network, quantifies uncertainty values through Monte Carlo dropout sampling and divides risk levels; a decision verification module carries out secondary verification on medium-risk samples, carries out verification on high-risk samples by calling a confrontation model, and carries out integration to generate deviation data; and the iterative optimization module reversely transmits an optimization signal according to an error type, dynamically adjusts model parameters, and performs iterative optimization on a threshold value and a judgment standard. According to the method, a nonlinear iteration path is formed through bidirectional closed-loop interaction, and automatic labeling and continuous performance optimization without manual intervention are achieved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a rare sample labeling system based on generative models and Bayesian calibration. Background Technology

[0002] In the training of artificial intelligence models, high-quality labeled samples are a core prerequisite for ensuring model performance. However, in scenarios such as medical image diagnosis, industrial quality inspection, and text analysis in niche fields, there is a common problem of "scarce samples"—the few sample classes cannot learn complete category features due to the extremely small number of labeled samples, while the long-tail classes have biased learning during model training due to the extremely uneven distribution of samples, and the recognition accuracy of long-tail class samples is significantly lower than that of head classes.

[0003] Traditional sample labeling methods rely mainly on manual annotation, which has two major pain points for scarce samples: First, the cost is extremely high, as the annotation of scarce samples requires the participation of professionals in the field, and the annotation time is much longer than that of ordinary samples, resulting in high costs; second, the efficiency is extremely low, as the annotation cycle for long-tail samples in datasets such as medical images is very long, which is seriously lagging behind the need for rapid model iteration. Summary of the Invention

[0004] To address these issues, the present invention provides a rare sample labeling system based on a generative model and Bayesian calibration, thereby solving the aforementioned problems in the prior art.

[0005] To achieve the above objectives, this invention provides a scarce sample labeling system based on a generative model and Bayesian calibration, comprising:

[0006] The sample analysis module is used to perform multimodal feature extraction and class division on the input scarce sample set to obtain core feature constraints that include the few-sample class and the long-tail class;

[0007] The joint generation module is connected to the sample analysis module to call the corresponding model according to the modality type of the scarce sample set, and to transform the core feature constraints into generation conditions, and to synchronously output candidate labels and their bound supporting evidence to obtain label-evidence pairs.

[0008] The consistency measurement module, connected to the joint generation module, is used to calculate the multi-dimensional indicators of the label-evidence pair based on a preset real feature library, thereby obtaining a consistency score.

[0009] An uncertainty calibration module, connected to the consistency measurement module, is used to construct a calibration model by using the multi-dimensional indicators and the consistency score as input features, so as to quantify the uncertainty value and classify the risk level;

[0010] The decision verification module is connected to the uncertainty calibration module to implement an automatic differentiation strategy based on the risk level to generate deviation data.

[0011] The iterative optimization module, connected to the decision verification module, is used to transmit optimization signals in reverse according to the deviation data and preset error types, dynamically adjust the model parameters, consistency thresholds and risk level judgment criteria of each module, and trigger the next round of iteration.

[0012] Furthermore, the joint generation module includes:

[0013] The model selection unit is used to select and call the diffusion generation model or the language generation model according to the modality type, and to convert the core feature constraints into the generation condition input of the corresponding model;

[0014] A diffusion generation unit, connected to the model selection unit, is used to generate candidate samples through noise addition and denoising iteratively and automatically extract feature matching regions to construct supporting evidence;

[0015] A language generation unit, connected to the diffusion generation unit, is used to generate candidate labels based on the generation conditions and simultaneously output semantic logic chains as supporting evidence.

[0016] The pairing filtering unit, which is connected to the diffusion generation unit and the language generation unit respectively, binds the candidate tags with supporting evidence and assigns a unique identifier, and outputs the tag-evidence pair after removing logical contradictions.

[0017] Furthermore, after receiving the generation conditions, the diffusion generation unit embeds the core feature constraints into the condition input of the diffusion model, generates 5-10 candidate samples through no less than 100 steps of noise addition and denoising iteration, automatically extracts the regional features that match the core feature constraints for each candidate sample, and constructs supporting evidence containing pixel coordinates, morphological parameters and grayscale range. After binding the labels of the candidate samples with the supporting evidence, the results are output to the pairing filtering unit.

[0018] Furthermore, the consistency measurement module includes:

[0019] The feature library construction unit is used to integrate machine-extracted features and human-made gold standard features to construct the real feature library;

[0020] The indicator calculation unit is connected to the feature library construction unit and is used to calculate the multi-dimensional indicator based on the label-evidence pair from three dimensions: feature matching degree, logical rationality and domain conformity.

[0021] The score determination unit is connected to the index calculation unit and is used to obtain the consistency score through weighted calculation.

[0022] Furthermore, the indicator calculation unit includes:

[0023] The feature matching calculation subunit is used to calculate the degree of overlap between the feature description of the label-evidence pair and the features in the real feature library to obtain the feature matching degree index.

[0024] The logical rationality calculation subunit is used to determine the logical consistency between the label and the evidence in the label-evidence pair according to the preset scenario rules to obtain the logical rationality index;

[0025] The domain compliance calculation subunit is used to determine the domain professionalism of the label-evidence pair based on preset domain knowledge to obtain a domain compliance index.

[0026] Furthermore, the uncertainty calibration module includes:

[0027] A Bayesian modeling unit is used to construct a Bayesian neural network as the calibration model by taking the multi-dimensional indicators and the consistency score as input features.

[0028] An uncertainty calculation unit, connected to the Bayesian modeling unit, is used to calculate the uncertainty value based on the Bayesian neural network through Monte Carlo dropout sampling;

[0029] A risk classification unit, connected to the uncertainty calculation unit, is used to determine the risk level based on the uncertainty value and a preset risk threshold.

[0030] Furthermore, the risk allocation unit includes:

[0031] The threshold dynamic adjustment subunit is used to calculate the actual verification error rate of samples of each risk level in the historical iteration. When the error rate of low-risk samples exceeds 5%, the upper limit of the risk threshold is lowered, and when the error rate of medium-risk samples exceeds 20%, the lower limit of the risk threshold is raised.

[0032] The risk fusion subunit is used to fuse the uncertainty value with the feature matching degree deviation value and the logical rationality deviation value in the multi-dimensional indicators to generate a comprehensive risk index.

[0033] The risk level determination subunit is connected to the risk fusion subunit and classifies the risk level according to the comprehensive risk index.

[0034] Furthermore, the decision verification module includes:

[0035] The risk grading unit classifies the risk level into medium-risk samples and high-risk samples;

[0036] A medium-risk verification unit, connected to the risk classification unit, triggers a secondary generation verification for the medium-risk sample to generate a first verification conclusion.

[0037] A high-risk verification unit, connected to the risk classification unit, automatically verifies the high-risk sample by calling the adversarial verification model to generate a second verification conclusion.

[0038] The deviation integration unit is connected to the medium-risk verification unit and the high-risk verification unit respectively, and is used to integrate the first verification conclusion and the second verification conclusion to generate the deviation data.

[0039] Furthermore, the iterative optimization module includes:

[0040] The deviation analysis unit is used to classify the deviation data into generation deviation, consistency deviation and calibration deviation according to a preset error type;

[0041] An optimized signal generation unit is connected to the deviation analysis unit, and generates corresponding parameter adjustment signals, threshold adjustment signals, and level adjustment signals according to the generation deviation, consistency deviation, and calibration deviation, respectively.

[0042] The parameter dynamic adjustment unit is connected to the optimization signal generation unit. Based on the parameter adjustment signal, threshold adjustment signal, and level adjustment signal, it adjusts the model weight of the joint generation module, the multi-dimensional index weight and consistency threshold of the consistency measurement module, and the risk level judgment standard of the uncertainty calibration module.

[0043] An iterative gating unit, connected to the parameter dynamic adjustment unit, is used to evaluate the current system performance after the adjustment is completed. If the performance improvement reaches a preset threshold, the next iteration is triggered; otherwise, the existing parameters are maintained and new deviation data is accumulated.

[0044] Furthermore, the iterative gating unit includes:

[0045] The performance evaluation subunit is used to calculate the change in the low consistency return rate of the consistency measurement module and the change in the proportion of low-risk samples of the uncertainty calibration module before and after the current iteration, and to generate performance improvement indicators.

[0046] A threshold adjustment subunit is connected to the performance evaluation subunit and adaptively adjusts the preset threshold value based on historical data of the performance improvement index.

[0047] The cooling period control subunit is connected to the threshold adjustment subunit. When the number of consecutive trigger iterations reaches the preset cooling threshold, the next round of iteration is forcibly paused and the data accumulation stage is entered. The cooling is lifted only after the amount of deviation data generated by the verification decision module reaches the preset sample size.

[0048] Compared with existing technologies, the advantages of this invention are as follows: the core feature constraints extracted by the sample analysis module directly determine the generation direction and evidence quality of the joint generation module. The more accurate the features, the higher the matching degree between the generated samples and the real distribution. The consistency measurement module quantifies the three dimensions of matching degree, logicality, and domain relevance into a consistency score. After this score is input into the uncertainty calibration module, low consistency will inevitably lead to high uncertainty values, thereby triggering the decision verification module to implement more stringent differential verification. The risk level is positively correlated with the verification intensity. The deviation data generated by the verification is fed back to the iterative optimization module according to three types of errors: generation, consistency, and calibration. The module dynamically adjusts the model parameters and judgment thresholds, making the next round of generation more in line with real features, the consistency assessment more accurate, and the uncertainty quantification more reliable, forming a self-reinforcing positive cycle. The system can automatically balance the generation diversity and accuracy, the assessment rigor and flexibility, and the risk sensitivity and stability in the iteration without manual intervention, significantly reducing the annotation cost and improving the annotation quality of scarce samples. Attached Figure Description

[0049] Figure 1 A schematic diagram of the rare sample labeling system based on generative model and Bayesian calibration provided by the present invention;

[0050] Figure 2 A schematic diagram of the joint generation module in the scarce sample labeling system based on generative model and Bayesian calibration provided by the present invention;

[0051] Figure 3 This is a schematic diagram of the consistency measurement module in the scarce sample labeling system based on generative model and Bayesian calibration provided by the present invention. Detailed Implementation

[0052] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0053] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0054] It should be noted that in the description of this invention, the terms "upper", "lower", "left", "right", "inner", "outer", etc., which indicate directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. This is only for the convenience of description and is not intended to indicate or imply that the device or element must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this invention.

[0055] Furthermore, it should be noted that, in the description of this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0056] Please see Figure 1 As shown, this invention provides a scarce sample labeling system based on a generative model and Bayesian calibration, comprising:

[0057] The sample analysis module 10 is used to perform multimodal feature extraction and class division on the input scarce sample set to obtain core feature constraints that include the few-sample class and the long-tail class.

[0058] Specifically, it receives multimodal scarce sample sets, supports automatic parsing of image formats (JPG / PNG), text formats (TXT / JSON), and standard audio formats, assigns a unique ID to each sample, and establishes an index containing storage path, data type, and original annotation information. Simultaneously, it verifies data integrity and records missing or corrupted samples. For image samples, it uses a pre-trained ResNet-50 model to extract 2048-dimensional pixel-level feature vectors (covering edges, texture, and color histograms) and high-level semantic feature vectors (target contours, region features). For text samples, it uses a BERT model to extract 768-dimensional word embedding features and sentence-level semantic feature vectors. For cross-modal samples, it simultaneously extracts and associates multimodal features. It counts the number of labeled samples in each class, automatically classifying categories with ≤10 samples as minority classes; calculates the proportion of each category in the total samples, classifying categories with a proportion ≤5% and a difference of ≥10 times from the largest category as long-tail classes; calculates the cosine similarity of features among samples of the same category, and removes samples with a similarity <0.5 and marks them as noise samples. The classification results and feature data are integrated to generate a "Scarce Sample Analysis Report". The report clearly lists the category names, sample quantity and proportion of the scarce sample class and the long-tail class, the core feature description of each class (such as the typical defect morphological parameters of long-tail class images and the high-frequency keyword set of scarce sample class text), the list of noisy sample IDs and the reasons for removal, and outputs the report and feature vectors to the joint generation module.

[0059] The joint generation module 20 is connected to the sample analysis module 10. It is used to call the corresponding model according to the modality type of the scarce sample set, and convert the core feature constraints into generation conditions. It synchronously outputs candidate labels and their bound supporting evidence to obtain label-evidence pairs.

[0060] Specifically, such as Figure 2 As shown, the joint generation module 20 includes:

[0061] The model selection unit 21 is used to select and call the diffusion generation model or the language generation model according to the modality type, and to convert the core feature constraints into the generation condition input of the corresponding model;

[0062] Specifically, the system receives core feature constraints and modality type identifiers. If the modality is image, it selects a diffusion generation model (such as Stable Diffusion); if the modality is text, it selects a language generation model (such as GPT-4); if the modality is cross-modality, it calls both models in parallel. The core feature constraints are transformed into structured generation conditions—for image modalities, parameters such as lesion size, location, and grayscale range are encoded into conditional embedding vectors; for text modalities, high-frequency keywords, sentence structure, and semantic features are constructed into Prompt templates. The generation conditions are input to the corresponding generation unit, and the number of generation parameters are passed (5-10 for images, 3-5 for text).

[0063] The diffusion generation unit 22 is connected to the model selection unit 21 and is used to generate candidate samples through noise addition and denoising iteration and automatically extract feature matching regions to construct supporting evidence.

[0064] Specifically, after receiving the generation conditions, the diffusion generation unit embeds the core feature constraints into the condition input of the diffusion model, generates 5-10 candidate samples through no less than 100 steps of noise addition and denoising iteration, automatically extracts the regional features that match the core feature constraints for each candidate sample, and constructs supporting evidence containing pixel coordinates, morphological parameters and grayscale range. After binding the labels of the candidate samples with the supporting evidence, the results are output to the pairing filtering unit.

[0065] In this embodiment, the conditional embedding vector is injected into the cross-attention layer of the Stable Diffusion UNet network. The number of iterations for adding and removing noise is set to 100-200, the guiding weight is 7.5, and the random seed is fixed to ensure reproducibility. After the iteration is completed, candidate image samples are generated, and ResNet-50 is called to extract feature maps. Gradient-weighted class activation mapping (Grad-CAM) is used to automatically locate image regions with a matching degree > 0.8 with the core feature constraints. The pixel coordinate range, morphological parameters (major axis, minor axis, area) and gray-level statistics (mean, variance) of the region are recorded to construct structured supporting evidence. The candidate label (such as "early lung cancer lesion") is bound to the supporting evidence as an initial pair.

[0066] The language generation unit 23 is connected to the diffusion generation unit 21 and is used to generate candidate labels based on the generation conditions and simultaneously output semantic logic chains as supporting evidence.

[0067] Specifically, the Prompt template is input into the GPT-4 model, and the temperature coefficient is set to 0.7, Top-p to 0.9, and the maximum number of tokens to 500 to balance diversity and accuracy. While the model generates candidate labels, it extracts key token sequences in the generation process through an attention mechanism to construct semantic logic chain evidence. For example, when generating the "Category A" label, it simultaneously outputs the inference path of "first feature corresponds to first attribute → second feature matches second attribute → excludes interference from third attribute". The candidate labels and semantic logic chains are used as the initial output pairs.

[0068] The pairing filtering unit 24 is connected to the diffusion generation unit 22 and the language generation unit 23 respectively. It binds the candidate tags with supporting evidence and assigns unique identifiers. After eliminating logical contradictions, it outputs the tag-evidence pairs.

[0069] Specifically, a unique identifier is assigned to each initial pair (format: modality-category-timestamp-serial number, such as IMG-LUNG-20241223-001); logical contradictions are detected through a rule engine—if the grayscale range described in the evidence does not match the pathological features corresponding to the label, or the reasoning rules in the semantic logic chain conflict with the domain knowledge base, the pair is determined to be invalid and removed; the validity of valid pairs is verified (the evidence must contain no less than 3 feature dimensions), and finally the label-evidence pair is output to the consistency measurement module.

[0070] The consistency measurement module 30 is connected to the joint generation module 20 and is used to calculate the multi-dimensional indicators of the label-evidence pair based on a preset real feature library, thereby obtaining a consistency score.

[0071] Specifically, such as Figure 3 As shown, the consistency measurement module 30 includes:

[0072] Feature library construction unit 31 is used to integrate machine-extracted features and human gold standard features to construct the real feature library;

[0073] Specifically, the system receives machine-extracted features (image ResNet-50 feature vectors, text BERT feature vectors) and manually annotated gold-standard features (feature descriptions formed by independent annotation and cross-validation by more than 3 domain experts) from the sample analysis module to construct a structured real feature library. Each record in the library contains a feature type field (pixel-level / semantic-level), a feature description field (such as "crack length 2-5mm"), and a feature weight field (dynamically allocated according to the importance of the category determination, with core features having a weight of 0.7-0.8 and secondary features having a weight of 0.3-0.4). The feature library is iteratively updated through version number management.

[0074] The indicator calculation unit 32 is connected to the feature library construction unit 31 and is used to calculate the multi-dimensional indicator based on the label-evidence pair from three dimensions: feature matching degree, logical rationality and domain conformity.

[0075] Specifically, the indicator calculation unit includes:

[0076] The feature matching calculation subunit is used to calculate the degree of overlap between the feature description of the label-evidence pair and the features in the real feature library to obtain the feature matching degree index.

[0077] Specifically, the supporting evidence portion of the tag-evidence pair is received, and its described feature set (such as "lesion diameter 3mm" in the evidence) is extracted and compared with the corresponding category features in the real feature library one by one; the feature vector matching degree is calculated using cosine similarity, and a similarity ≥ 0.8 is judged as a match. The feature matching degree index is calculated according to the formula "(number of matching features × sum of weights) / (total number of features × sum of weights)", with an output range of 0-1.

[0078] The logical rationality calculation subunit is used to determine the logical consistency between the label and the evidence in the label-evidence pair according to the preset scenario rules to obtain the logical rationality index;

[0079] Specifically, a preset scenario rule library (including the distribution rules of industrial defect locations, the evolution rules of medical lesion morphology, text semantic logic rules, etc.) is loaded, and the tags in the tag-evidence pair are matched with the evidence according to the rules. If the evidence description conforms to the real scenario rules (such as the bearing crack only appears in the inner or outer ring, and the evidence description location is consistent with it), it is assigned a value of 1. If there is a contradiction (such as the evidence description that the crack is located in the ball), it is assigned a value of 0, and the logical rationality index is output.

[0080] The domain compliance calculation subunit is used to determine the domain professionalism of the label-evidence pair based on preset domain knowledge to obtain a domain compliance index.

[0081] Specifically, a pre-defined domain knowledge base (such as medical clinical guidelines, industrial testing standards, and classical Chinese grammar norms) is loaded, and parameters in the evidence (such as a patient's heart rate of 50 beats / min) are compared with the knowledge base standard (heart rate in acute myocardial infarction should be >60 beats / min). A perfect match is assigned 1 point, a partial match is assigned 0.2-0.8 points, and a non-match is assigned 0 points, and the domain compliance index is output.

[0082] The score determination unit 33 is connected to the index calculation unit 32 and is used to obtain the consistency score through weighted calculation.

[0083] Specifically, the consistency score is calculated using the weighted formula "feature matching degree × 0.6 + logical rationality × 0.2 + domain compliance × 0.2". A static threshold is set (≥0.8 for high consistency, 0.5-0.8 for medium consistency, and <0.5 for low consistency), and historical audit error rate data is retained through a closed loop. If the error rate of medium consistency samples is >20%, the lower limit of the medium consistency threshold is automatically increased from 0.5 to 0.6. If the error rate of low-risk samples is >5%, the upper limit of the risk threshold is decreased, thus achieving dynamic adaptation of the threshold.

[0084] An uncertainty calibration module, connected to the consistency measurement module, is used to construct a calibration model by using the multi-dimensional indicators and the consistency score as input features, so as to quantify the uncertainty value and classify the risk level;

[0085] Specifically, the uncertainty calibration module includes:

[0086] A Bayesian modeling unit is used to construct a Bayesian neural network as the calibration model by taking the multi-dimensional indicators and the consistency score as input features.

[0087] Specifically, the system receives multi-dimensional metrics (feature matching degree, logical rationality, and domain conformity) and consistency scores output by the consistency measurement module, and constructs a four-layer Bayesian neural network as a calibration model. The input layer contains 4 neurons (corresponding to 4 input features), the two hidden layers each contain 32 neurons and use the ReLU activation function, and the output layer has 2 neurons (outputting the mean and variance of a Gaussian distribution, respectively). The network weights adopt a normal prior distribution (mean 0, variance 1), and the bias adopts a uniform prior distribution. The training dataset consists of historical validation samples accumulated by the iterative optimization module (no less than 1000 samples). The loss function is negative log-likelihood loss, the optimizer is Adam (learning rate 0.001), and the training epochs are 100. Early stopping detection is performed in each epoch (the validation loss stops if it does not decrease for 10 consecutive epochs).

[0088] An uncertainty calculation unit, connected to the Bayesian modeling unit, is used to calculate the uncertainty value based on the Bayesian neural network through Monte Carlo dropout sampling;

[0089] Specifically, Monte Carlo dropout sampling is performed on each input label-evidence pair; the dropout rate is set to 0.2, the number of samplings is 50, and each sampling is forward propagated to obtain one prediction variance value; the 50 sampling results are input into the variance calculation subunit of the uncertainty calculation unit, and after removing outlier sampling values ​​that deviate from the mean by more than 3 times the standard deviation, the arithmetic mean of the remaining variance values ​​is taken to obtain the final uncertainty value (range 0-1), which is then output to the risk classification unit.

[0090] A risk classification unit, connected to the uncertainty calculation unit, is used to determine the risk level based on the uncertainty value and a preset risk threshold.

[0091] Specifically, the risk allocation unit includes:

[0092] The threshold dynamic adjustment subunit is used to calculate the actual verification error rate of samples of each risk level in the historical iteration. When the error rate of low-risk samples exceeds 5%, the upper limit of the risk threshold is lowered, and when the error rate of medium-risk samples exceeds 20%, the lower limit of the risk threshold is raised.

[0093] Specifically, after verifying every 100 label-evidence pairs, historical data is retrieved from the verification decision module, and the actual verification error rate of low-risk samples (number of incorrect samples / total number of low-risk samples) is calculated. If the error rate is greater than 5%, the upper limit of the low-risk threshold is lowered from 0.1 to 0.08 (step size 0.01). The actual verification error rate of medium-risk samples is calculated, and if the error rate is greater than 20%, the lower limit of the medium-risk threshold is raised from 0.1 to 0.15 (step size 0.01). The adjusted thresholds are synchronized in real time to the risk fusion subunit and the level determination subunit.

[0094] The risk fusion subunit is used to fuse the uncertainty value with the feature matching degree deviation value and the logical rationality deviation value in the multi-dimensional indicators to generate a comprehensive risk index.

[0095] Specifically, the uncertainty value output by the uncertainty calculation unit is received, and the feature matching deviation value (the absolute difference between the preset feature value and the evidence feature value) and the logical rationality deviation value (1 - logical rationality index) are extracted from the multi-dimensional indicators. The weighted fusion formula is used: Comprehensive risk index = uncertainty value × 0.6 + feature matching deviation value × 0.3 + logical rationality deviation value × 0.1. The comprehensive risk index normalized to the [0,1] interval is output to the level determination subunit.

[0096] The risk level determination subunit is connected to the risk fusion subunit and classifies the risk level according to the comprehensive risk index.

[0097] Specifically, the comprehensive risk index is received, and a three-level classification is performed using dynamically adjusted thresholds: a comprehensive risk index ≤ 0.08 indicates low risk, 0.08 < index ≤ 0.4 indicates medium risk, and an index > 0.4 indicates high risk. The judgment result is output to the verification decision module, and low-risk pairs are directly marked as "valid labeled samples" and stored in the sample library, while medium- and high-risk pairs are output with an accompanying risk index.

[0098] The decision verification module is connected to the uncertainty calibration module to implement an automatic differentiation strategy based on the risk level to generate deviation data.

[0099] Specifically, the decision verification module includes:

[0100] The risk grading unit classifies the risk level into medium-risk samples and high-risk samples;

[0101] Specifically, the system receives the risk level labels (low risk / medium risk / high risk) and corresponding comprehensive risk indices output by the uncertainty calibration module; samples with a comprehensive risk index ∈ (0.08, 0.4] are classified as medium-risk samples, and samples with an index > 0.4 are classified as high-risk samples; a graded timestamp and a risk source label are added to each sample (marking the dominant risk factor as uncertainty bias, feature matching bias, or logical rationality bias), and the results are output to the corresponding verification unit; the distribution ratio of the two types of risk samples in the current batch is statistically analyzed in real time, and if the proportion of high-risk samples exceeds 30% of the total sample size or the proportion of medium-risk samples exceeds 50%, an early warning signal is triggered to the iterative optimization module, indicating that there is a systematic bias in the generated model.

[0102] A medium-risk verification unit, connected to the risk classification unit, triggers a secondary generation verification for the medium-risk sample to generate a first verification conclusion.

[0103] Specifically, the process involves receiving the medium-risk samples, extracting their original core feature constraints, and applying structured perturbations—in the image modality, applying a ±10% random fluctuation to the constraint parameters of lesion size, location coordinates, and grayscale range; in the text modality, randomly replacing 20% ​​of synonyms with high-frequency keywords and adding or deleting conjunctions in sentence structures; inputting the perturbated constraints into the joint generation module for secondary generation to obtain a second set of label-evidence pairs; calculating the cosine similarity of the feature matching degree between the original pair and the new pair; if the similarity is ≥0.85 and the labels are consistent, the sample is determined to be stable and "First verification conclusion: Pass" is generated; if the similarity is <0.85 or the labels conflict, the sample is determined to be unstable and "First verification conclusion: Deviation" is generated; and recording the feature difference (corresponding Euclidean distance of feature vectors) and logical difference (Jaccard similarity of evidence text) generated twice as the first deviation data.

[0104] A high-risk verification unit, connected to the risk classification unit, automatically verifies the high-risk sample by calling the adversarial verification model to generate a second verification conclusion.

[0105] Specifically, the high-risk samples are received, and an adversarial verification model is invoked (a dual-tower adversarial structure consisting of a discriminator and a generator; the discriminator uses an Inception-v3 or BERT pre-trained model with a risk-aware bypass added; the generator structure is consistent with the diffusion / language generation unit). The label-evidence pair is input into the discriminator. If the discriminator outputs a confidence score ≥ 0.9 and classifies it as a "real sample", a "second verification conclusion: credible" is generated. If the confidence score < 0.9 or it is classified as a "synthetic sample", a "second verification conclusion: high-risk bias" is generated. At the same time, the Euclidean distance between the feature vector of the second-to-last layer of the discriminator and the feature vector of the evidence is extracted. Combined with the discriminator's attention weight distribution and the matching deviation value of the key region of the evidence, the second bias data is constructed.

[0106] The deviation integration unit is connected to the medium-risk verification unit and the high-risk verification unit respectively, and is used to integrate the first verification conclusion and the second verification conclusion to generate the deviation data.

[0107] Specifically, the system receives the first and second verification conclusions and encodes them uniformly into structured deviation records (fields include: sample ID, original risk level, verification conclusion type, deviation quantification value, deviation source module marker, and verification timestamp). For samples marked as "deviation" and "high-risk deviation," the system extracts the feature difference degree and logical difference degree for medium-risk samples, and the feature Euclidean distance and attention matching deviation for high-risk samples, integrating them into a 4-dimensional deviation data matrix. The system automatically classifies the deviations according to preset error type mapping rules—if the deviation mainly stems from differences in generated results, it is marked as "generational deviation"; if it stems from conflicts between evidence and the real feature library, it is marked as "consistency deviation"; if it stems from an underestimation of uncertainty, it is marked as "calibration deviation." The classified deviation data is output to the iterative optimization module and written into the verification log library to support full-process traceability and auditing.

[0108] The iterative optimization module, connected to the decision verification module, is used to transmit optimization signals in reverse according to the deviation data and preset error types, dynamically adjust the model parameters, consistency thresholds and risk level judgment criteria of each module, and trigger the next round of iteration.

[0109] Specifically, the iterative optimization module includes:

[0110] The deviation analysis unit is used to classify the deviation data into generation deviation, consistency deviation and calibration deviation according to a preset error type;

[0111] Specifically, deviations are automatically classified into three categories according to preset rules: if the deviation data is marked as "generational deviation" or the verification conclusion is "deviation" and the difference between the two generated results is >0.5, it is classified as generational deviation; if the deviation data is marked as "consistency deviation" or the feature Euclidean distance is >0.8, it is classified as consistency deviation; if the deviation data is marked as "calibration deviation" or the high-risk sample verification error rate is >20%, it is classified as calibration deviation. Each type of deviation is stored independently and its frequency is counted. When a certain type of deviation accumulates to more than 15 in 100 samples, the module is marked as a "key optimization target".

[0112] An optimized signal generation unit is connected to the deviation analysis unit, and generates corresponding parameter adjustment signals, threshold adjustment signals, and level adjustment signals according to the generation deviation, consistency deviation, and calibration deviation, respectively.

[0113] Specifically, three adjustment signals are generated: for generation bias, the mean of feature difference and the mean of logistic difference are extracted to generate a parameter adjustment signal (diffusion model learning rate adjustment factor = 1 - feature difference × 0.5, language model temperature coefficient adjustment magnitude = logistic difference × 0.1); for consistency bias, the mean of feature matching bias is extracted to generate a threshold adjustment signal (consistency threshold lower limit increase magnitude = bias mean × 0.2); for calibration bias, the proportion of samples with underestimated uncertainty is extracted to generate a level adjustment signal (risk threshold upper limit decrease magnitude = underestimated proportion × 0.05). All adjustment signals are encoded in JSON format, including adjustment module identifier, adjustment parameter name, adjustment direction (increase / decrease), and adjustment step size.

[0114] The parameter dynamic adjustment unit is connected to the optimization signal generation unit. Based on the parameter adjustment signal, threshold adjustment signal, and level adjustment signal, it adjusts the model weight of the joint generation module, the multi-dimensional index weight and consistency threshold of the consistency measurement module, and the risk level judgment standard of the uncertainty calibration module.

[0115] Specifically, the parameter update interface of the corresponding module is called through the API interface. For the joint generation module, the parameter adjustment signal is read. If the signal is a learning rate adjustment for the diffusion model, the original learning rate (1e-5) is multiplied by the adjustment factor and the fine-tuning process is restarted (50 iterations). If the signal is a temperature coefficient adjustment for the language model, the generation configuration is directly updated and written to the model metadata. For the consistency measurement module, the threshold adjustment signal is read, the feature matching weight, logical rationality weight, or consistency threshold is updated (adjustment step size is 0.01), and the weight configuration table in the feature library construction unit is updated synchronously. For the uncertainty calibration module, the level adjustment signal is read, the upper limit of the low-risk threshold or the lower limit of the medium-risk threshold in the risk classification unit is modified (adjustment step size is 0.01), and the retraining of the Bayesian neural network is triggered (30 iterations). All adjustment operations are recorded in the adjustment log library, including the parameter values ​​before / after adjustment, the deviation sample ID on which the adjustment is based, and the adjustment timestamp.

[0116] An iterative gating unit, connected to the parameter dynamic adjustment unit, is used to evaluate the current system performance after the adjustment is completed. If the performance improvement reaches a preset threshold, the next iteration is triggered; otherwise, the existing parameters are maintained and new deviation data is accumulated.

[0117] Specifically, the iterative gating unit includes:

[0118] The performance evaluation subunit is used to calculate the change in the low consistency return rate of the consistency measurement module and the change in the proportion of low-risk samples of the uncertainty calibration module before and after the current iteration, and to generate performance improvement indicators.

[0119] Specifically, after the parameters are adjusted, the low consistency return rate (number of low consistency samples / total number of samples) of 100 samples before and after this iteration is extracted from the consistency measurement module, and the change Δ1 = before - after is calculated; the change in the proportion of low-risk samples Δ2 = after - before is extracted from the uncertainty calibration module; and the performance improvement index = Δ1 + Δ2 is generated. If the index > 0, it is determined to be a performance improvement; if the index ≤ 0, it is determined to be no performance improvement.

[0120] A threshold adjustment subunit is connected to the performance evaluation subunit and adaptively adjusts the preset threshold value based on historical data of the performance improvement index.

[0121] Specifically, a sliding window of length 10 is maintained to record historical performance improvement metrics. If the average of the last 5 metrics is less than 0.02, the preset threshold value is lowered from the default 0.05 to 0.03 to reduce the difficulty of triggering iterations. If the average of the last 3 metrics is greater than 0.1, the threshold value is raised to 0.08 to avoid excessive iterations. The adjusted threshold value is synchronized to the cooling-off period control subunit.

[0122] The cooling period control subunit is connected to the threshold adjustment subunit. When the number of consecutive trigger iterations reaches the preset cooling threshold, the next round of iteration is forcibly paused and the data accumulation stage is entered. The cooling is lifted only after the amount of deviation data generated by the verification decision module reaches the preset sample size.

[0123] Specifically, the system monitors the number of consecutive iterations. When the number reaches a preset cooling threshold (5 times by default), the next iteration is forcibly paused and the system enters the data accumulation phase. During the pause, the system continues to receive and cache the deviation data generated by the decision verification module. When the amount of cached data reaches a preset sample size (50 by default), the cooling is automatically lifted and a new iteration is allowed. During the cooling period, the performance evaluation subunit continues to calculate the performance improvement index. If the index is negative for 3 consecutive times, an alarm signal is sent to the system monitoring interface.

[0124] Specifically, this invention uses core feature constraints extracted by the sample analysis module to directly determine the generation direction and evidence quality of the joint generation module. The more precise the features, the higher the matching degree between the generated samples and the real distribution. The consistency measurement module quantifies the three dimensions of matching degree, logicality, and domain relevance into a consistency score. After this score is input into the uncertainty calibration module, low consistency inevitably leads to high uncertainty values, thereby triggering the decision verification module to implement more stringent differential verification. The risk level is positively correlated with the verification intensity. The deviation data generated by the verification is fed back to the iterative optimization module according to three types of errors: generation, consistency, and calibration. This dynamically adjusts the model parameters and judgment thresholds, making the next round of generation more closely match the real features, the consistency assessment more accurate, and the uncertainty quantification more reliable, forming a self-reinforcing positive cycle. The system can automatically balance generation diversity and accuracy, assessment rigor and flexibility, and risk sensitivity and stability during iteration without manual intervention, significantly reducing annotation costs and improving the annotation quality of scarce samples.

[0125] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

[0126] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A scarce sample labeling system based on generative models and Bayesian calibration, characterized in that, include: The sample analysis module is used to perform multimodal feature extraction and class division on the input scarce sample set to obtain core feature constraints that include the few-sample class and the long-tail class; The joint generation module is connected to the sample analysis module to call the corresponding model according to the modality type of the scarce sample set, and to transform the core feature constraints into generation conditions, and to synchronously output candidate labels and their bound supporting evidence to obtain label-evidence pairs. The consistency measurement module, connected to the joint generation module, is used to calculate the multi-dimensional indicators of the label-evidence pair based on a preset real feature library, thereby obtaining a consistency score. An uncertainty calibration module, connected to the consistency measurement module, is used to construct a calibration model by using the multi-dimensional indicators and the consistency score as input features, so as to quantify the uncertainty value and classify the risk level; The decision verification module is connected to the uncertainty calibration module to implement an automatic differentiation strategy based on the risk level to generate deviation data. The iterative optimization module, connected to the decision verification module, is used to transmit optimization signals in reverse according to the deviation data and the preset error type, dynamically adjust the model parameters, consistency thresholds and risk level judgment criteria of each module, and trigger the next round of iteration.

2. The scarce sample labeling system based on generative model and Bayesian calibration according to claim 1, characterized in that, The joint generation module includes: The model selection unit is used to select and call the diffusion generation model or the language generation model according to the modality type, and to convert the core feature constraints into the generation condition input of the corresponding model; A diffusion generation unit, connected to the model selection unit, is used to generate candidate samples through noise addition and denoising iteratively and automatically extract feature matching regions to construct supporting evidence; A language generation unit, connected to the diffusion generation unit, is used to generate candidate labels based on the generation conditions and simultaneously output semantic logic chains as supporting evidence. The pairing filtering unit, which is connected to the diffusion generation unit and the language generation unit respectively, binds the candidate tags with supporting evidence and assigns a unique identifier, and outputs the tag-evidence pair after removing logical contradictions.

3. The scarce sample labeling system based on generative model and Bayesian calibration according to claim 2, characterized in that, After receiving the generation conditions, the diffusion generation unit embeds the core feature constraints into the condition input of the diffusion model, generates 5-10 candidate samples through at least 100 steps of noise addition and denoising iteration, automatically extracts the regional features that match the core feature constraints for each candidate sample, and constructs supporting evidence containing pixel coordinates, morphological parameters and grayscale range. After binding the labels of the candidate samples with the supporting evidence, the samples are output to the pairing filtering unit.

4. The scarce sample labeling system based on generative model and Bayesian calibration according to claim 3, characterized in that, The consistency measurement module includes: The feature library construction unit is used to integrate machine-extracted features and human-made gold standard features to construct the real feature library; The indicator calculation unit is connected to the feature library construction unit and is used to calculate the multi-dimensional indicator based on the label-evidence pair from three dimensions: feature matching degree, logical rationality and domain conformity. The score determination unit is connected to the index calculation unit and is used to obtain the consistency score through weighted calculation.

5. The scarce sample labeling system based on generative model and Bayesian calibration according to claim 4, characterized in that, The indicator calculation unit includes: The feature matching calculation subunit is used to calculate the degree of overlap between the feature description of the label-evidence pair and the features in the real feature library to obtain the feature matching degree index. The logical rationality calculation subunit is used to determine the logical consistency between the label and the evidence in the label-evidence pair according to the preset scenario rules to obtain the logical rationality index; The domain compliance calculation subunit is used to determine the domain professionalism of the label-evidence pair based on preset domain knowledge to obtain a domain compliance index.

6. The scarce sample labeling system based on generative model and Bayesian calibration according to claim 5, characterized in that, The uncertainty calibration module includes: A Bayesian modeling unit is used to construct a Bayesian neural network as the calibration model by taking the multi-dimensional indicators and the consistency score as input features. An uncertainty calculation unit, connected to the Bayesian modeling unit, is used to calculate the uncertainty value based on the Bayesian neural network through Monte Carlo dropout sampling; A risk classification unit, connected to the uncertainty calculation unit, is used to determine the risk level based on the uncertainty value and a preset risk threshold.

7. The scarce sample labeling system based on generative model and Bayesian calibration according to claim 6, characterized in that, The risk allocation unit includes: The threshold dynamic adjustment subunit is used to calculate the actual verification error rate of samples of each risk level in the historical iteration. When the error rate of low-risk samples exceeds 5%, the upper limit of the risk threshold is lowered, and when the error rate of medium-risk samples exceeds 20%, the lower limit of the risk threshold is raised. The risk fusion subunit is used to fuse the uncertainty value with the feature matching degree deviation value and the logical rationality deviation value in the multi-dimensional indicators to generate a comprehensive risk index. The risk level determination subunit is connected to the risk fusion subunit and classifies the risk level according to the comprehensive risk index.

8. The scarce sample labeling system based on generative model and Bayesian calibration according to claim 7, characterized in that, The decision verification module includes: The risk grading unit classifies the risk level into medium-risk samples and high-risk samples; A medium-risk verification unit, connected to the risk classification unit, triggers a secondary generation verification for the medium-risk sample to generate a first verification conclusion. A high-risk verification unit, connected to the risk classification unit, automatically verifies the high-risk sample by calling the adversarial verification model to generate a second verification conclusion. The deviation integration unit is connected to the medium-risk verification unit and the high-risk verification unit respectively, and is used to integrate the first verification conclusion and the second verification conclusion to generate the deviation data.

9. The scarce sample labeling system based on generative model and Bayesian calibration according to claim 8, characterized in that, The iterative optimization module includes: The deviation analysis unit is used to classify the deviation data into generation deviation, consistency deviation and calibration deviation according to a preset error type; An optimized signal generation unit is connected to the deviation analysis unit, and generates corresponding parameter adjustment signals, threshold adjustment signals, and level adjustment signals according to the generation deviation, consistency deviation, and calibration deviation, respectively. The parameter dynamic adjustment unit is connected to the optimization signal generation unit. Based on the parameter adjustment signal, threshold adjustment signal, and level adjustment signal, it adjusts the model weight of the joint generation module, the multi-dimensional index weight and consistency threshold of the consistency measurement module, and the risk level judgment standard of the uncertainty calibration module. An iterative gating unit, connected to the parameter dynamic adjustment unit, is used to evaluate the current system performance after the adjustment is completed. If the performance improvement reaches a preset threshold, the next iteration is triggered; otherwise, the existing parameters are maintained and new deviation data is accumulated.

10. The scarce sample labeling system based on generative model and Bayesian calibration according to claim 9, characterized in that, The iterative gating unit includes: The performance evaluation subunit is used to calculate the change in the low consistency return rate of the consistency measurement module and the change in the proportion of low-risk samples of the uncertainty calibration module before and after the current iteration, and to generate performance improvement indicators. A threshold adjustment subunit is connected to the performance evaluation subunit and adaptively adjusts the preset threshold value based on historical data of the performance improvement index. The cooling period control subunit is connected to the threshold adjustment subunit. When the number of consecutive trigger iterations reaches the preset cooling threshold, the next round of iteration is forcibly paused and the data accumulation stage is entered. The cooling is lifted only after the amount of deviation data generated by the verification decision module reaches the preset sample size.

Images