Data set automatic annotation construction method and system

Abstract

The invention discloses a data set automatic annotation construction method and system, and mainly relates to the technical field of artificial intelligence and computer vision. Comprising the following steps: carrying out preprocessing and feature extraction on labeled source domain data and unlabeled target domain data; calculating a mean value prototype based on the marked category of the source domain and fusing neighborhood information to generate a priori enhanced prototype; performing balanced clustering initialization on the target domain data; fusing the sample features, the neighbor features and the prototype through a hybrid offset refocusing operator, and generating an enhanced embedding vector; performing cross-domain redistribution based on the attention weight of the source domain, and correcting feature distribution of the target domain; performing comparative learning by using enhanced embedding, and generating a pseudo tag in combination with a multi-view fusion strategy; and finally, outputting an automatic labeling result with confidence. The method has the beneficial effects that the accuracy and the robustness of automatic labeling can be improved and the manual labeling cost can be reduced while the pseudo label noise is reduced, the feature drift is inhibited and the embedding discrimination and the clustering stability are improved.

Description

Technical Field

[0001] This invention relates to the field of technology, specifically to a method and system for automatically labeling and constructing datasets based on a cross-domain attention redistribution mechanism. Background Technology

[0002] In the fields of artificial intelligence and computer vision, the success of deep learning technology in tasks such as image classification and object detection heavily relies on large-scale, high-quality labeled data. However, data labeling is costly, and unknown categories are constantly emerging in the real open world. This necessitates that models possess the ability to learn from labeled data and automatically discover new categories in unlabeled data. To address this, researchers have proposed novel category discovery and generalized category discovery tasks. The latter requires models to simultaneously identify known categories and cluster unknown categories, serving as a crucial bridge connecting supervised and unsupervised learning.

[0003] Currently, GCD methods primarily rely on powerful pre-trained visual models to extract features, combined with contrastive learning and clustering algorithms to achieve category classification. Despite some progress, significant drawbacks still exist, limiting their performance and practicality. First, these methods overly depend on the feature extraction capabilities of pre-trained models; when using advanced encoders like DINOv2, performance improvements plateau, limiting generalization. Second, the widely adopted instance-level contrastive learning fails to fully utilize the structural relationships between categories, resulting in insufficient discriminative power in the learned feature representations and blurred inter-class boundaries. Furthermore, commonly used semi-supervised clustering algorithms are sensitive to initial centers, easily leading to inter-cluster overlap and unstable clustering results. In addition, many methods still assume the number of unknown categories is known in practical applications, limiting their applicability in real-world open environments. Some improvements attempt to introduce parameterized classifiers, iteratively optimize cluster centers, or utilize pseudo-labels to enhance supervision, but these introduce new problems such as high computational complexity, unstable convergence, and pseudo-label error propagation.

[0004] The root cause of these problems lies in the fact that existing methods fail to fully and effectively integrate task-related prior knowledge with the local distribution characteristics of the data itself. On the one hand, the models lack a mechanism to reliably transfer structured semantic information from labeled data to unlabeled data, resulting in insufficient utilization of known category knowledge. On the other hand, they fail to explicitly enhance intra-class cohesion and inter-class segregation during the feature learning stage, while also lacking the ability to dynamically correct for feature distribution shifts and cluster imbalances. These shortcomings collectively lead to bottlenecks in the existing GCD methods in terms of feature separability, cluster robustness, and overall performance.

[0005] Therefore, there is an urgent need for a method and system for automatically labeling and constructing datasets based on cross-domain attention redistribution mechanisms to solve the above problems. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for automatic dataset annotation construction based on a cross-domain attention redistribution mechanism. This method can improve the accuracy and robustness of automatic annotation while reducing pseudo-label noise, suppressing feature drift, improving embedding discriminativeness and clustering stability, and reducing the cost of manual annotation.

[0007] To achieve the above objectives, the present invention employs the following technical solution: On the one hand, the present invention provides a method for automatically labeling and constructing datasets, including the following steps: Step S1: Preprocess the labeled source domain data and the unlabeled target domain data, and extract normalized feature vectors through a pre-trained encoder; Step S2: Calculate the class mean prototype based on the labeled categories of the source domain data, and combine it with nearest neighbor information to generate a priori enhanced prototype through neighborhood smoothing; Step S3: Use the prior-enhanced prototype to perform balanced initialization of the target domain data and determine the initial cluster centers of unknown categories; Step S4: Mix the original features, nearest neighbor features, and corresponding class prototypes of the feature vector of each sample proportionally using the hybrid offset refocusing operator to generate an enhanced embedding vector; Step S5: Based on the category-level attention weights of the source domain data, the attention weights are redistributed to the enhanced embedding vectors of the target domain samples through a cross-domain attention redistribution mechanism to perform semantic alignment and distribution offset correction. Step S6: Perform contrastive learning using enhanced embedding vectors, combine supervised and unsupervised contrastive loss, periodically generate or update pseudo-labels, and adopt a multi-view fusion strategy to improve the robustness of pseudo-labels; Step S7: Output the automatic annotation results with confidence levels.

[0008] Preferably, in step S2, generating the prior-enhanced prototype includes: For the Class, its mean prototype The calculation formula is: ; in, Indicates the first The feature vector of each sample Indicates the corresponding label, Indicates belonging to a category The sample set, The number of elements in the set; Neighborhood smoothing representation of samples The calculation formula is: ; in, Indicated by Centered A nearest neighbor set The size of the neighborhood; This is the enhanced representation obtained by weighting the neighborhood mean with the original vector; By combining the mean prototype and the neighborhood smoothing representation, a prior-enhanced prototype is generated for subsequent clustering initialization.

[0009] Preferably, in step S4, the use of the hybrid offset refocusing operator includes: Through space transformation kernel function Mix the sample itself, its neighborhood, and the corresponding prototype proportionally: ; in, Indicates that it is extracted by the encoder and then... The normalized input feature vector, This represents the vector nearest neighbor set This represents the class prototype vector generated by the prior-enhanced prototype. Used to control the mixing ratio of neighborhood / prototype and original features. Used to balance prototype weights; right conduct Normalization enhances embedding : ; in, The Mix-Shift augmented embedding vectors are used for subsequent comparative learning.

[0010] Preferably, in step S5, the cross-domain attention redistribution mechanism includes: Based on source domain data, a category-level attention template is constructed. By calculating the similarity between the target domain sample and the source domain prototype, the attention weights are redistributed to the target domain sample according to a configurable strategy. The enhanced embedding vectors of the target domain samples are reweighted to correct inter-domain distribution shifts and suppress pseudo-label noise; Attention weight redistribution is achieved through a learnable attention function, and the parameters of attention weight redistribution are dynamically adjusted according to the feature distribution of the source and target domains.

[0011] Preferably, in step S6, the comparative learning includes: Unsupervised contrastive loss constructed using enhanced embedding : ; in, Indicates the current training batch. and For an enhanced embedding of a pair of positive samples, Temperature coefficient; Unsupervised comparison loss Losses compared with supervision Weighted fusion to form the final optimization target : ; in, To control the hyperparameters of the two types of loss weights, adjustments are made based on the labeled proportion of the dataset and the noise level.

[0012] Preferably, in step S6, the multi-view fusion strategy includes at least one of the following methods: Pre-fusion: fusing feature representations from different enhanced views before generating pseudo-labels; Post-fusion: Fusing the pseudo-label results from multiple views after the pseudo-labels are generated; Dual fusion: Simultaneous feature-level and label-level fusion.

[0013] On the other hand, the present invention also provides an automatic dataset annotation and construction system for implementing the above-described automatic dataset annotation and construction method, comprising: The data preprocessing module is used to preprocess labeled source domain data and unlabeled target domain data; The feature extraction module is used to extract normalized feature vectors from preprocessed data through a pre-trained encoder. The prior and prototype generation module is used to calculate the category mean prototype based on the labeled categories of the source domain data, and perform neighborhood smoothing by combining nearest neighbor information to generate a prior-enhanced prototype. The hybrid offset refocusing module is used to apply the hybrid offset refocusing operator to the feature vector of each sample, mixing its original features, nearest neighbor features and corresponding class prototypes in a proportional manner to generate an enhanced embedding vector. The cross-domain attention redistribution module is used to redistribute attention weights to the enhanced embedding vectors of the target domain samples based on the category-level attention weights of the source domain data through a cross-domain attention redistribution mechanism, thereby performing semantic alignment and distribution offset correction. The pseudo-label generation and fusion module is used to perform contrastive learning using the enhanced embedding vectors, combine supervised and unsupervised contrastive loss, periodically generate or update pseudo-labels, and adopt a multi-view fusion strategy to improve the robustness of pseudo-labels. The annotation output module is used to output automatic annotation results with confidence levels.

[0014] Preferably, the prior and prototype generation module is specifically used for: Calculate the prototype mean of known categories in the source domain ; Combination Nearest neighbor information is used to smooth the sample features in the neighborhood and generate an enhanced representation. ; Unknown class centers are initialized in a balanced manner by excluding nearest neighbor samples to avoid clustering bias.

[0015] Preferably, the hybrid offset refocusing module is specifically used for: By configuring parameters and Adjust the mixing ratio to generate diverse enhanced embeddings; Efficient use of approximate indexes Nearest neighbor retrieval ensures scalability under large-scale data; The embedded vector input contrastive learning module will be enhanced to expand the number of effective contrastive sample pairs.

[0016] Preferably, the system also includes an engineering optimization module for: Parallel computation is used to accelerate cluster initialization and pseudo-label update; Provides confidence thresholds to filter low-quality labeled samples and generates labels to be reviewed; Integrated visual quality inspection tools, including t-SNE distribution maps and prototype sample displays, are used for manual review and correction.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. By integrating cross-domain attention redistribution and prior augmentation prototype learning, this invention breaks away from the simple dependence on pre-trained model features and achieves in-depth mining and effective transfer of structured knowledge of known categories. 2. By adopting a priori-enhanced prototype contrastive learning and a hybrid offset refocusing mechanism, clearer and more stable cluster boundaries are constructed in the feature space; 3. By using balanced clustering initialization and dynamic cross-domain attention correction, noise accumulation and error propagation in the pseudo-label generation process are significantly reduced; 4. Modular design and efficient algorithm implementation ensure annotation accuracy while meeting the computational efficiency requirements of large-scale dataset construction. Attached Figure Description

[0018] Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the LCMB and feature enhancement process according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the system structure according to an embodiment of the present invention; Figure 4This is a block diagram of the automated annotation construction system module and a schematic diagram of runtime data flow according to an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the automatic labeling and attention visualization results for object categories in industry and nature according to an embodiment of the present invention. Detailed Implementation

[0019] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined in this application.

[0020] In this invention, terms such as "upper," "lower," "left," "right," "front," "back," "vertical," "horizontal," "side," and "bottom" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used only to facilitate the description of the structural relationships of the various components or elements of this invention and do not specifically refer to any component or element in this invention. They should not be construed as limiting the invention.

[0021] Example: like Figure 1 As shown, this embodiment provides a method for automatically labeling and constructing a dataset, including the following steps: Step S1: Preprocess the labeled source domain data and the unlabeled target domain data, and extract normalized feature vectors through a pre-trained encoder; Step S2: Calculate the class mean prototype based on the labeled categories of the source domain data, and combine it with nearest neighbor information to generate a priori enhanced prototype through neighborhood smoothing; Step S3: Use the prior-enhanced prototype to perform balanced initialization of the target domain data and determine the initial cluster centers of unknown categories; Step S4: Mix the original features, nearest neighbor features, and corresponding class prototypes of the feature vector of each sample proportionally using the hybrid offset refocusing operator to generate an enhanced embedding vector; Step S5: Based on the category-level attention weights of the source domain data, the attention weights are redistributed to the enhanced embedding vectors of the target domain samples through a cross-domain attention redistribution mechanism to perform semantic alignment and distribution offset correction. Step S6: Perform contrastive learning using enhanced embedding vectors, combine supervised and unsupervised contrastive loss, periodically generate or update pseudo-labels, and adopt a multi-view fusion strategy to improve the robustness of pseudo-labels; Step S7: Output the automatic annotation results with confidence levels.

[0022] This embodiment proposes an automated dataset annotation construction method based on prior enhanced prototype contrastive learning, Mix-Shift refocusing, and cross-domain attention redistribution. This method can be used for automated annotation of cross-domain image / visual datasets, open-world category discovery, and semi-supervised training scenarios. First, the source domain (annotated) and target domain (unannotated) samples are uniformly preprocessed, and a pre-trained encoder extracts normalized feature vectors. Then, robust prototypes are constructed based on the labeled class mean and sample k-neighbor information, and used to evenly initialize unknown class centers (LCMB) to reduce center initialization bias and generate reliable prototypes. Next, the Mix-Shift operator is applied to each sample, proportionally mixing its original features, k-nearest neighbor features, and the corresponding class prototype to obtain an enhanced embedding, thereby expanding the number of effective contrast pairs. Based on this, cross-domain attention redistribution is introduced, assigning source domain category-level attention templates to the target domain embedding according to similarity to correct distribution shifts and suppress pseudo-label noise. During training, augmented embeddings were used as a comparison. Pseudo-labels were periodically generated / updated using LCMB, and a multi-view fusion strategy was employed to improve the robustness of the pseudo-labels. Finally, automatically labeled results with confidence scores were exported for downstream use and manual review. Details of Mix-Shift and attention weighting are as follows... Figure 2 As shown.

[0023] like Figure 3 As shown, this embodiment also provides an automatic dataset annotation and construction system, including: The system comprises a data preprocessing module, a feature extraction module, a prior and prototype generation module, a Mix-Shift enhancement module, a cross-domain attention redistribution module, a pseudo-label generation and fusion module, and a labeled output module. The system is implemented using a modular software / hardware collaborative approach: feature extraction can be deployed on GPU servers or edge computing nodes; k-NN retrieval uses approximate indexes (such as FAISS) to ensure scalability; the candidate / exclusion and clustering steps of LCMB are parallelized to reduce latency; Mix-Shift and attention redistribution are configurable operators, allowing adjustment of the mixing weights and attention templates according to dataset features. For engineering stability, the system provides confidence thresholds, low-confidence sample "pending review" marking, and visual quality checks (t-SNE, prototype examples) subsystems. Figure 4 The system module block diagram and runtime data flow illustration are shown. The system supports end-to-end automated operation, as well as manual interruption and correction processes, to balance efficiency and annotation quality.

[0024] This preferred embodiment is a university-industry collaboration project of an industrial manufacturing enterprise, applied in the industrial manufacturing field. It aims to build a high-quality labeled dataset for large-scale visual inspection systems (such as PCB board defect detection, surface scratch detection, and product quality inspection). In this scenario, the labeled source domain data (such as laboratory environment or early production line data) and the target domain data (such as real-time data collected from a new production line or new mold) exhibit differences in distribution regarding lighting, background, camera angle, and product style. The target domain data also contains a large number of unknown defect categories that the model has never encountered before. The automatic labeling system is deployed as follows: 1. Prior Enhancement Prototype Generation and Initialization (LCMB): The system's prior and prototype generation module calculates class centers after feature extraction, and then the system uses source domain (labeled) samples. Calculate the mean prototype of known defect categories Simultaneously, K-nearest neighbor information is used to smooth the sample features within their neighborhoods, enhancing local consistency and intra-class compactness. Then, for samples containing both known and unknown defect categories... The data is initialized for clustering by excluding nearest neighbor samples to select initial centers. This avoids cluster imbalance and incorrect clustering caused by random initialization or center drift, providing a reliable starting point for subsequent discovery of unknown defects. 2. Mix-Shift hybrid refocusing and sample enhancement will feature: Neighborhood and prototype information are mixed proportionally to increase the number of effective contrastive sample pairs and improve representational power. For each sample feature... The module uses a spatial transformation kernel function to mix the features with the k nearest neighbors and the corresponding class prototypes to generate a Mix-Shift enhanced embedding. 3. Cross-domain attention reallocation and generation label correction: Utilize the labeled source domain A category-level attention weight is constructed, and this weight is then redistributed based on the similarity between the sample and the source domain prototype, and applied to the reweighting and semantic alignment of the target domain (to be labeled) samples. In this way, the semantic prior of known defects such as "typical scratches" or "typical blemishes" is reliably transferred to unlabeled samples, thereby improving the consistency and accuracy of generated labels. Figure 5 As shown, this system can effectively discover and label categories on objects in both industrial and natural environments, and automatically allocate attention to complete the feature discovery of specific categories. 4. Joint optimization and automated annotation export: The system's pseudo-label generation and fusion module incorporates unsupervised contrastive loss. Compared with supervision, the loss This achieves end-to-end optimization. During training, the system periodically generates or updates labels for unlabeled samples and adjusts these labels using a multi-view fusion strategy to further improve the stability and coverage of the annotations. Finally, the system exports automatic annotations and corresponding confidence scores for manual review, ensuring a balance between annotation quality and efficiency in industrial applications.

[0025] This embodiment, based on an automated annotation system with cross-domain attention redistribution, achieves optimal accuracy in both the identification and clustering of known and unknown defect categories. For industrial categories, it achieves over 80% accuracy without human intervention, over 95% accuracy with 1% human involvement, and over 99% accuracy with only 1.5% human involvement. It maintains stable clustering even with inter-domain distribution differences introduced by new production lines, enabling fully automated discovery and annotation of over 20 defect categories. This significantly reduces reliance on and cost of manual annotation, and improves the efficiency of large-scale dataset construction.

[0026] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of this application.

Claims

1. A method for automatically labeling and constructing a dataset, characterized in that, Includes the following steps: Step S1: Preprocess the labeled source domain data and the unlabeled target domain data, and extract normalized feature vectors through a pre-trained encoder; Step S2: Calculate the class mean prototype based on the labeled categories of the source domain data, and combine it with nearest neighbor information to generate a priori enhanced prototype through neighborhood smoothing; Step S3: Use the prior-enhanced prototype to perform balanced initialization of the target domain data and determine the initial cluster centers of unknown categories; Step S4: Mix the original features, nearest neighbor features, and corresponding class prototypes of the feature vector of each sample proportionally using the hybrid offset refocusing operator to generate an enhanced embedding vector; Step S5: Based on the category-level attention weights of the source domain data, the attention weights are redistributed to the enhanced embedding vectors of the target domain samples through a cross-domain attention redistribution mechanism to perform semantic alignment and distribution offset correction. Step S6: Perform contrastive learning using enhanced embedding vectors, combine supervised and unsupervised contrastive loss, periodically generate or update pseudo-labels, and adopt a multi-view fusion strategy to improve the robustness of pseudo-labels; Step S7: Output the automatic annotation results with confidence levels.

2. The automatic dataset annotation and construction method according to claim 1, characterized in that, In step S2, generating the prior-enhanced prototype includes: For the Class, its mean prototype The calculation formula is: ； in, Indicates the first The feature vector of each sample Indicates the corresponding label, Indicates belonging to a category The sample set, The number of elements in the set; Neighborhood smoothing representation of samples The calculation formula is: ； in, Indicates Centered A nearest neighbor set The size of the neighborhood; This is the enhanced representation obtained by weighting the neighborhood mean with the original vector; By combining the mean prototype and the neighborhood smoothing representation, a prior-enhanced prototype is generated for subsequent clustering initialization.

3. The automatic dataset annotation and construction method according to claim 1, characterized in that, In step S4, the use of the hybrid offset refocusing operator includes: Through space transformation kernel function Mix the sample itself, its neighborhood, and the corresponding prototype proportionally: ； in, Indicates that it is extracted by the encoder and then... The normalized input feature vector, This represents the vector Nearest neighbor set This represents the class prototype vector generated by the prior-enhanced prototype. Used to control the mixing ratio of neighborhood / prototype and original features. Used to balance prototype weights; right conduct Normalization enhances embedding : ； in, The Mix-Shift augmented embedding vectors are used for subsequent comparative learning.

4. The automatic dataset annotation and construction method according to claim 1, characterized in that, In step S5, the cross-domain attention reallocation mechanism includes: Based on source domain data, a category-level attention template is constructed. By calculating the similarity between the target domain sample and the source domain prototype, the attention weights are redistributed to the target domain sample according to a configurable strategy. The enhanced embedding vectors of the target domain samples are reweighted to correct inter-domain distribution shifts and suppress pseudo-label noise; Attention weight redistribution is achieved through a learnable attention function, and the parameters of attention weight redistribution are dynamically adjusted according to the feature distribution of the source and target domains.

5. The automatic dataset annotation and construction method according to claim 1, characterized in that, In step S6, the contrastive learning includes: Unsupervised contrastive loss constructed using enhanced embedding : ； in, Indicates the current training batch. and For an enhanced embedding of a pair of positive samples, Temperature coefficient; Unsupervised comparison loss Losses compared with supervision Weighted fusion to form the final optimization target : ； in, To control the hyperparameters of the two types of loss weights, adjustments are made based on the labeled proportion of the dataset and the noise level.

6. The automatic dataset annotation and construction method according to claim 1, characterized in that, In step S6, the multi-view fusion strategy includes at least one of the following methods: Pre-fusion: fusing feature representations from different enhanced views before generating pseudo-labels; Post-fusion: Fusing the pseudo-label results from multiple views after the pseudo-labels are generated; Dual fusion: Simultaneous feature-level and label-level fusion.

7. A dataset automatic annotation and construction system, used to implement the dataset automatic annotation and construction method as described in any one of claims 1-6, characterized in that, include: The data preprocessing module is used to preprocess labeled source domain data and unlabeled target domain data; The feature extraction module is used to extract normalized feature vectors from preprocessed data through a pre-trained encoder. The prior and prototype generation module is used to calculate the category mean prototype based on the labeled categories of the source domain data, and perform neighborhood smoothing by combining nearest neighbor information to generate a prior-enhanced prototype. The hybrid offset refocusing module is used to apply the hybrid offset refocusing operator to the feature vector of each sample, mixing its original features, nearest neighbor features and corresponding class prototypes in a proportional manner to generate an enhanced embedding vector. The cross-domain attention redistribution module is used to redistribute attention weights to the enhanced embedding vectors of the target domain samples based on the category-level attention weights of the source domain data through a cross-domain attention redistribution mechanism, thereby performing semantic alignment and distribution offset correction. The pseudo-label generation and fusion module is used to perform contrastive learning using the enhanced embedding vectors, combine supervised and unsupervised contrastive loss, periodically generate or update pseudo-labels, and adopt a multi-view fusion strategy to improve the robustness of pseudo-labels. The annotation output module is used to output automatic annotation results with confidence levels.

8. The automatic dataset annotation and construction system according to claim 7, characterized in that, The prior and prototype generation module is specifically used for: Calculate the prototype mean of known categories in the source domain ; Combination Nearest neighbor information is used to smooth the sample features in the neighborhood and generate an enhanced representation. ; Unknown class centers are initialized in a balanced manner by excluding nearest neighbor samples to avoid clustering bias.

9. The automatic dataset annotation and construction system according to claim 7, characterized in that, The hybrid offset refocusing module is specifically used for: By configuring parameters and Adjust the mixing ratio to generate diverse enhanced embeddings; Efficient use of approximate indexes Nearest neighbor retrieval ensures scalability under large-scale data; The embedded vector input contrastive learning module will be enhanced to expand the number of effective contrastive sample pairs.

10. The automatic dataset annotation and construction system according to claim 7, characterized in that, The system also includes an engineering optimization module for: Parallel computation is used to accelerate cluster initialization and pseudo-label update; Provides confidence thresholds to filter low-quality labeled samples and generates labels to be reviewed; Integrated visual quality inspection tools, including t-SNE distribution maps and prototype sample displays, are used for manual review and correction.

Images