An image data labeling method

Abstract

The application belongs to the technical field of image processing, and specifically discloses an image data labeling method, which comprises the following steps: constructing an image pyramid structure, performing feature extraction on multiple resolution image levels respectively to obtain multi-scale feature maps; performing self-adaptive fusion processing on the multi-scale feature maps by using an attention mechanism to obtain a fusion feature representation; determining position information and edge contour information of a labeling target based on the fusion feature representation to generate a labeling result. The application can effectively improve small target labeling precision and complex edge contour fitting degree, enhance video cross-frame labeling consistency, significantly improve labeling efficiency, and reduce redundant labeling cost.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to an image data annotation method. Background Technology

[0002] Image data annotation is a crucial preliminary step in computer vision and deep learning model training. The quality of annotation directly impacts the performance of models for tasks such as object detection, image segmentation, and pose estimation.

[0003] In existing technologies, image annotation methods typically employ a single-scale image processing strategy. Specifically, the image to be annotated is directly input into the annotation tool, and the annotator manually selects the target region or draws its outline. This approach has the following technical drawbacks: When images contain targets of different scales, small targets have limited feature information, resulting in lower annotation accuracy. Annotated bounding boxes often deviate from the actual target boundaries, leading to significant noise in the training samples. For complex scenes such as blurred regions and adhered targets, traditional methods cannot accurately fit the true contours of the targets, easily resulting in missed targets or selection errors. Furthermore, traditional manual annotation lacks intelligent assistance, resulting in low annotation efficiency and a decline in annotation consistency due to fatigue. Summary of the Invention

[0004] The purpose of this invention is to provide an image data annotation method to solve the technical problems of poor scale adaptability, rough edge contour processing, and low annotation efficiency in the prior art.

[0005] This invention provides an image data annotation method, comprising the following steps: acquiring an image to be annotated; performing downsampling processing on the image to be annotated to obtain multiple image layers of different resolutions to form an image pyramid structure; extracting features from the multiple image layers of different resolutions to obtain multi-scale feature maps corresponding to each layer; performing adaptive fusion processing on the multi-scale feature maps using an attention mechanism to obtain fused feature representations; and determining the location information and edge contour information of the annotation target based on the fused feature representations to generate annotation results.

[0006] Preferably, constructing the image pyramid structure includes: performing downsampling processing on the image to be labeled at least three times to obtain at least four image levels with different resolutions.

[0007] Preferably, when the image to be labeled is a video frame sequence, the method further includes the following steps: based on the labeling result of the current frame, using an optical flow prediction model to predict the initial position of the label box of the adjacent frame; calculating the coordinate difference value between the labeling results of the current frame and the adjacent frame; when the coordinate difference value exceeds a preset first threshold, triggering a manual review mechanism; when the coordinate difference value does not exceed the first threshold, automatically using the predicted initial position of the label box as the labeling result of the adjacent frame.

[0008] Preferably, the method further includes the following steps: obtaining the category prediction probability distribution of adjacent frames; calculating the difference between the category prediction probability distribution of the current frame and adjacent frames; and triggering a manual review mechanism when the difference exceeds a preset second threshold.

[0009] Preferably, the method further includes the following steps: constructing an active learning sample selection module; performing hard example mining processing on unlabeled images to obtain hard example samples; calculating the comprehensive score of the hard example samples; and sorting the samples in descending order of the comprehensive score and pushing them to the labeling queue in order of score from high to low.

[0010] Preferably, the difficult example mining process includes: calculating the prediction uncertainty index of the sample.

[0011] Preferably, the calculation of the prediction uncertainty index of the sample includes: training an initial classification model using labeled images, performing multiple forward propagations on unlabeled images, calculating the entropy value of the prediction probability distribution based on the prediction results of the multiple forward propagations, and using the entropy value as the prediction uncertainty index.

[0012] Preferably, the difficult example mining process further includes: calculating the feature diversity index of the sample.

[0013] Preferably, the calculation of the feature diversity index of the sample includes: extracting the depth feature vector of the unlabeled image, performing cluster analysis on the depth feature vector to obtain multiple cluster centers, calculating the distance value between the current sample and each cluster center, and using the minimum distance value as the feature diversity index.

[0014] It should be noted that a sample refers to a single image instance in unlabeled image data; in the context of active learning and hard example mining, each image to be labeled is referred to as a sample. The current sample refers to the unlabeled image for which the feature diversity index is currently being calculated.

[0015] Preferably, the comprehensive score is calculated by weighting the prediction uncertainty index and the feature diversity index; wherein the weight coefficient of the prediction uncertainty index is greater than the weight coefficient of the feature diversity index.

[0016] This invention constructs an image pyramid to obtain multi-scale feature maps and uses an attention mechanism for adaptive fusion, which can highlight important features, suppress redundant information, and improve the accuracy of target location and edge contour annotation. For video frame sequences, the initial positions of adjacent frame annotation boxes are predicted based on an optical flow prediction model. The system automatically determines whether to trigger manual review based on coordinate differences, reducing the workload of frame-by-frame annotation while ensuring quality. Simultaneously, a review mechanism is triggered by calculating the difference in inter-frame category probability distributions, further improving the continuity and reliability of video annotation.

[0017] This invention constructs an active learning sample selection module, which obtains prediction uncertainty indicators, such as the entropy value of multiple forward propagations, and feature diversity indicators, such as the minimum distance between deep features and cluster centers, through hard example mining. These are weighted and calculated into a comprehensive score, then pushed to the labeling queue in descending order. The uncertainty indicator has a higher weight than the diversity indicator, allowing for priority screening of samples that the model struggles to determine, while also considering the representativeness of the sample distribution. This reduces redundant labeling, lowers labeling costs, accelerates model performance iteration, and improves generalization ability. Attached Figure Description

[0018] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn to scale.

[0019] Figure 1 Here is a flowchart of the adaptive image annotation method based on multi-scale feature fusion in Example 1; Figure 2 This is a flowchart of the video cross-frame annotation method with temporal consistency constraints in Example 2; Figure 3 This is a flowchart of the active learning annotation method based on difficult example mining in Example 3. Detailed Implementation

[0020] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0021] It should be noted that, unless otherwise stated, the technical or scientific terms used in this application should have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0022] Example 1: An Adaptive Image Annotation Method Based on Multi-Scale Feature Fusion Reference Figure 1 The specific steps in this embodiment are as follows: Step S101: Obtain the image to be labeled.

[0023] The images to be labeled can be acquired in various ways, including but not limited to: reading image files from local storage devices, downloading image data from a web server, and acquiring images in real time via a camera. The images to be labeled can be common still images in JPEG, PNG, or BMP formats, or image data in any other format. The acquired images are first normalized to a preset standard resolution for subsequent processing.

[0024] Step S102: Construct the image pyramid structure.

[0025] The acquired images to be labeled are downsampled to construct an image pyramid structure. Specifically, a combination of Gaussian blur and downsampling is used to scale the image in multiple levels. The downsampling process for each layer includes: first, applying Gaussian filtering to the current layer image to eliminate jagged edges that may occur during downsampling; then, halving the resolution according to a preset downsampling factor.

[0026] In this embodiment, the image to be labeled undergoes at least three downsampling processes to obtain at least four image layers with different resolutions. For example, the first layer is the original resolution image, the second layer is the original resolution image, and the third layer is the original resolution image. Figure 2 The first layer is a 1 / 3 resolution image, the second layer is a 1 / 4 resolution image, and the third layer is a 1 / 8 resolution image. Ideally, the downsampling factor between adjacent layers is set to 2, meaning that the resolution of each layer is half that of the layer above it.

[0027] Analysis and Judgment Steps: The number of downsampling layers needs to be determined based on the distribution of target scales in the image. If the image contains a large number of small targets, the number of image layers should be increased to ensure sufficient feature information at high-resolution layers. If the image mainly contains large targets, the number of layers can be appropriately reduced to save computational resources. Furthermore, the parameters used in the downsampling process also need to be adjusted according to the actual situation. For example, the size of the Gaussian kernel and the downsampling interpolation algorithm will all affect the final annotation effect.

[0028] Step S103: Multi-scale feature extraction.

[0029] Feature extraction is performed on the constructed image layers of different resolutions to obtain multi-scale feature maps corresponding to each layer. Specifically, a convolutional neural network is used as the feature extractor to perform convolution operations on each image layer to extract the depth features of that layer.

[0030] Feature extraction networks typically consist of multiple convolutional layers, pooling layers, and activation layers. Shallower convolutional layers extract features with higher resolution, making them suitable for detecting small targets; deeper convolutional layers extract features with richer semantic information, making them suitable for detecting large targets. By extracting features from images at different levels separately, multi-scale feature maps containing information at different scales can be obtained.

[0031] Analysis and Judgment Steps: The choice of feature extraction network needs to be determined based on the actual application scenario and annotation accuracy requirements. For scenarios with high accuracy requirements, networks with deeper layers and more parameters, such as ResNet-50 and ResNet-101, can be selected; for scenarios with high real-time requirements, lightweight networks, such as MobileNetV3 and ShuffleNet, can be selected. The feature dimensions extracted by the network also need to be adapted according to the needs of subsequent processing.

[0032] Step S104: Attention fusion processing.

[0033] An attention mechanism is used to adaptively fuse the multi-scale feature maps obtained in step S103 to obtain fused feature representations.

[0034] The specific process of attention fusion processing includes: First, spatial attention weights are calculated for each feature map. Specifically, global average pooling and global max pooling are applied to each feature map to compress it into two one-dimensional vectors. These two vectors are then mapped to spatial attention weights through a shared fully connected layer, normalized using the sigmoid function, and finally multiplied element-wise with the original feature map to achieve spatial recalibration of the features.

[0035] Secondly, channel attention weights are calculated for each feature map. Specifically, global average pooling is performed on each feature map to obtain two one-dimensional vectors. These vectors are then mapped to channel attention weights through a fully connected layer. After normalization using the Sigmoid function, the weights are multiplied element-wise with the original feature map to achieve channel recalibration of the features.

[0036] Then, the feature maps at each level, after spatial and channel attention processing, are upsampled to be uniform to the original image resolution. Upsampling can be achieved using methods such as bilinear interpolation, deconvolution, or nearest neighbor interpolation.

[0037] Finally, the upsampled feature maps at each level are added or concatenated element by element to obtain the fused feature representation.

[0038] Analysis and Judgment Steps: Parameters in the attention mechanism, such as the number of neurons in the fully connected layer and the type of activation function, will affect the fusion effect. If the fusion effect is not ideal, consider increasing the number of layers in the attention mechanism or using a more complex attention structure, such as non-local attention or Transformer attention. Furthermore, the feature fusion method also needs to be selected based on the actual situation. Addition operations are more suitable for situations where the feature maps have the same dimension, while concatenation operations can retain more feature information.

[0039] Step S105: Generate annotation results.

[0040] Based on the fusion feature representation obtained in step S104, the location information and edge contour information of the labeled target are determined, and the labeling result is generated.

[0041] Specifically, a fully convolutional network is used to process the fused feature representation, outputting two types of information: The first type of information is the category heatmap. A category heatmap is a two-dimensional array, where each value represents the probability that a given location belongs to a particular category. By performing maximum suppression on the heatmap, the center point location and category information of the target can be obtained.

[0042] The second type of information is the edge regression map. Edge regression maps are used to accurately regress the bounding box coordinates or segmentation contour point coordinates of the target. For bounding box annotation tasks, the edge regression map outputs four coordinate values, representing the x and y coordinates of the top-left and bottom-right corners of the bounding box. For polygon annotation tasks, the edge regression map outputs a sequence of coordinates for multiple contour points, used to accurately depict the irregular edges of the target.

[0043] Analysis and Judgment Steps: The accuracy of edge regression maps is affected by several factors. First, the quality of the training data directly determines the accuracy of edge regression. If the edge annotations in the training data are not accurate enough, the edge information learned by the model will also be biased. Second, the choice of loss function is also important. Different loss functions can be used to balance localization accuracy and bounding box quality, such as L1 loss, L2 loss, or IoU loss, or a combination of loss functions can be used. Finally, the non-maximum suppression threshold setting in the post-processing step also needs to be adjusted according to the actual situation. Setting the threshold too high may lead to the omission of overlapping targets, while setting the threshold too low may produce duplicate detection results.

[0044] The technical advantages of this embodiment are as follows: By utilizing the image pyramid to preserve high-resolution hierarchical detail information, the features of small targets are significantly enhanced after attention fusion, thus greatly improving the annotation accuracy of small targets and effectively solving the problem of low annotation accuracy of small targets in traditional methods. Simultaneously, the edge regression map is specifically trained for contour refinement, accurately identifying the true boundaries of blurred regions and adhered targets, and significantly reducing the fitting error of complex edge contours.

[0045] Furthermore, since multi-scale feature fusion can adaptively select and combine feature information at different levels, it can achieve the best annotation effect for targets of different scales, and the annotation results have a wider range of applicability and better consistency.

[0046] Example 2: A Video Cross-Frame Annotation Method with Temporal Consistency Constraints like Figure 2 As shown, based on Embodiment 1, when the image to be labeled is a video frame sequence, this embodiment adds the following technical means: Step S201: Initialization of optical flow prediction module and propagation of annotation boxes.

[0047] Optical flow field calculation is performed using a pre-trained optical flow prediction network model. Given the annotation result B_t of the t-th frame in the video sequence, the annotation result includes the target's location information (coordinates of the top left and bottom right corners of the bounding box), size information, and category information.

[0048] The optical flow prediction network first receives two frames, frame t and frame (t+1), as input. It then extracts features from these two frames using a convolutional neural network and calculates the optical flow field from frame t to frame (t+1) using an optical flow decoder. The optical flow field is a two-dimensional vector field, where each location has two components representing the displacement of that point in the x and y directions, respectively.

[0049] After obtaining the optical flow field, it is applied to the annotation result B_t of frame t. Specifically, the coordinates of the four corner points of the annotation box are added to the corresponding optical flow vector offsets to obtain four new corner point coordinates. These four new coordinates constitute the predicted annotation box position of frame t+1.

[0050] Analysis and Judgment Steps: The accuracy of optical flow prediction is affected by video quality. When there are complex situations such as motion blur, occlusion, or rapid movement in the video, the accuracy of optical flow prediction may decrease. In this case, a more robust optical flow prediction model can be considered, or a comprehensive prediction can be made by combining other information (such as the optical flow fields of multiple frames before and after). In addition, when there are multiple targets in the video, optical flow propagation processing needs to be performed on each target separately, and possible target occlusion and disappearance situations need to be handled.

[0051] Step S202: Calculate coordinate differences.

[0052] Calculate the coordinate difference between the actual annotation result of frame t+1 and the predicted annotation box position obtained in step S201.

[0053] The coordinate difference value can be calculated in several ways: the first method is to calculate the Euclidean distance between the center points of the two bounding boxes; the second method is to calculate the Manhattan distance between the top-left and bottom-right corners of the two bounding boxes; the third method is to calculate the intersection-union ratio (IU / U) of the two bounding boxes and then subtract that value from 1 to obtain the difference. Preferably, the first or third method is used to calculate the coordinate difference value.

[0054] Analysis and Judgment Steps: The calculation method for coordinate difference values ​​needs to be selected based on the actual application scenario. If the overall positional offset of the target is of greater concern, the center point distance is more appropriate; if the fit of the bounding box is of greater concern, the intersection-union ratio (IUGR) is more accurate. Furthermore, the threshold setting for coordinate difference values ​​also needs to be adjusted according to the actual situation. If the target in the video moves at a high speed, the threshold can be increased appropriately; if high annotation accuracy is required, the threshold should be decreased.

[0055] Step S203: First threshold judgment and manual review trigger.

[0056] A first threshold is set to determine whether manual intervention is needed. The coordinate difference value calculated in step S202 is compared with the first threshold.

[0057] When the coordinate difference exceeds the first threshold, it indicates a significant deviation between the predicted and actual results, triggering a manual review mechanism. The system automatically marks the current frame as an abnormal frame and pushes this frame and related frames to the manual annotation queue, where annotators re-annotate the (t+1)th frame.

[0058] When the coordinate difference value does not exceed the first threshold, the predicted bounding box position is automatically used as the annotation result of the t+1 frame, and the process directly enters the next frame without manual intervention.

[0059] Analysis and Judgment Steps: The setting of the first threshold needs to strike a balance between annotation efficiency and annotation quality. If the threshold is set too low, a large number of normal frames will be misclassified as abnormal frames, increasing the workload of manual review and reducing annotation efficiency. If the threshold is set too high, problematic annotation results may be missed, affecting annotation quality. Ideally, the first threshold can be adaptively adjusted based on the average motion speed of the target in the video, or the optimal threshold can be determined through validation set experiments.

[0060] Step S204: Obtain the category prediction probability distribution.

[0061] While steps S201 to S203 are being executed, the category prediction probability distributions for frame t and frame t+1 are obtained. The category prediction probability distribution reflects the model's confidence in the target category and is a probability vector of length equal to the number of categories, with the sum of all elements being 1.

[0062] The class prediction probability distribution can be obtained by adding a classification head to the annotation network. The classification head typically consists of a global pooling layer and a fully connected layer. The output of the fully connected layer is processed by the Softmax function to obtain the class prediction probability distribution.

[0063] Analysis and Judgment Steps: The accuracy of class prediction depends on the design of the classifier head and the quality of the training data. If the classifier head is poorly designed, or if there is class imbalance in the training data, the probability distribution of the predicted classes may not be accurate enough. In this case, consider using a class-weighted loss function for training, or using label smoothing techniques to improve the prediction performance.

[0064] Step S205: Calculate the difference.

[0065] Calculate the dissimilarity between the class prediction probability distributions of frame t and frame t+1. The dissimilarity can be calculated in several ways: the first is to calculate the Euclidean distance between the two probability vectors; the second is to calculate the cosine similarity between the two probability vectors; and, preferably, the third is to calculate the KL divergence between the two probability vectors.

[0066] KL divergence is a measure of the difference between two probability distributions. Its calculation involves considering the logarithm of a probability value in one distribution to the logarithm of the corresponding probability value in another distribution. When the probability of a certain category is zero, the probability distribution needs to be smoothed to avoid infinity.

[0067] Analysis and Judgment Steps: KL divergence is a good measure of the difference between two probability distributions. However, it's important to note that when some probabilities in one of the distributions are zero, the KL divergence may become infinite. In this case, the probability distributions need to be smoothed, for example, by adding a small constant to all probability values ​​to avoid zero probabilities.

[0068] Step S206: Second threshold judgment and manual review trigger.

[0069] When the difference calculated in step S205 exceeds the preset second threshold, it indicates that there may be a category jump between adjacent frames. At this time, the manual review mechanism is triggered, and the annotator confirms whether the target category is correct.

[0070] Analysis and Judgment Steps: The setting of the second threshold needs to be adjusted based on the number and characteristics of the categories. If there are many categories, the differences between different categories may be large, in which case the second threshold can be appropriately increased. If some categories are easily confused, the second threshold should be appropriately decreased to improve the annotation quality. In addition, when a new target appears or a target disappears in the video, the category prediction probability distribution may change drastically. In this case, even if the difference does not exceed the second threshold, manual review should still be performed.

[0071] The technical advantages of this embodiment are as follows: By utilizing optical flow prediction to guide the propagation of bounding boxes between frames, the annotation results of adjacent frames exhibit strong continuity, significantly improving the consistency of cross-frame annotation in the video. The annotation results are more stable, effectively avoiding the box jitter problem caused by independent frame-by-frame annotation in traditional methods.

[0072] By setting a dual-threshold judgment mechanism, images with abnormal annotation results can be automatically identified and manual review can be triggered, significantly reducing the workload of subsequent corrections. At the same time, since the annotation results of most frames are automatically inherited by the algorithm, annotation efficiency is significantly improved, and the workload of manual annotation is greatly reduced.

[0073] Example 3: Active Learning Annotation Method Based on Hard Example Mining like Figure 3 As shown, based on Embodiment 1 or Embodiment 2, this embodiment adds an active learning sample selection module to optimize the configuration of annotation resources: Step S301: Initial model training and update.

[0074] An initial classification or detection model is trained using a subset of labeled samples. This initial model is then used to predict unlabeled images and assess the difficulty level of the samples.

[0075] The initial model was trained using a standard deep learning training process. First, the labeled image data was divided into a training set and a validation set. Then, the model was trained iteratively multiple times using the training set data, while the model's performance was monitored using the validation set data. Training was stopped when the validation set accuracy stabilized, resulting in the initial model.

[0076] In the subsequent annotation process, when a new batch of labeled samples is added, the new samples need to be added to the training set, the model needs to be incrementally trained or retrained, the initial model needs to be updated, and the next round of sample selection needs to be carried out.

[0077] Analysis and Judgment Steps: The quality of the initial model directly affects the accuracy of subsequent sample selection. If the initial model is not sufficiently trained, its prediction results may be unreliable, and the accuracy of hard example mining will also be affected. Therefore, during initial training, it is essential to ensure that the model reaches a certain accuracy level on the validation set. Furthermore, the hyperparameter settings during training, such as learning rate, batch size, and regularization strength, need to be adjusted according to the actual situation to obtain the optimal initial model.

[0078] Step S302: Uncertainty assessment.

[0079] For each image in the unlabeled image set, perform multiple forward propagations using the initial model trained in step S301 to calculate the prediction uncertainty index of the sample.

[0080] There are two ways to implement multiple forward propagation: The first is to enable the dropout mechanism in the model and perform multiple forward propagations on the same image. During each forward propagation, the neurons in the network are randomly deactivated, thus producing different prediction results. The second is to perform multiple data augmentations on the same image (such as random cropping, random flipping, color jittering, etc.) and then input the augmented images into the model for forward propagation.

[0081] Each forward propagation yields a predicted probability distribution, reflecting the model's confidence in the image category. The mean of these multiple predicted probability distributions is taken, and the entropy of this mean probability distribution is calculated. Entropy is a concept in information theory used to measure the uncertainty of a random variable. A higher entropy value indicates greater uncertainty in the model's prediction of the image, making the image more likely to be a hard example. The calculated entropy value is used as an indicator of the image's predicted uncertainty.

[0082] Analysis and Judgment Steps: The accuracy of the predicted uncertainty index is affected by the number of forward propagations. If the number of propagations is too small, the statistical mean and entropy values ​​may not be stable enough and are easily affected by randomness; if the number of propagations is too large, it will increase the computational cost and reduce the efficiency of sample selection. Ideally, the number of forward propagations should be set between 10 and 20. In addition, the dropout probability setting also needs to be reasonable. If the probability is too high, the difference between each prediction will be too large, resulting in an artificially high entropy value; if the probability is too low, the difference between multiple predictions will be too small, making it difficult to distinguish between easy and difficult samples.

[0083] Step S303: Characteristic diversity assessment.

[0084] The depth feature vector is extracted from each unlabeled image. This depth feature vector is the output of the penultimate layer of the initial model. The depth feature vector contains high-level semantic information about the image and can reflect the essential features of the image.

[0085] Clustering algorithms are used to perform cluster analysis on the deep feature vectors of labeled samples to obtain several cluster centers. The clustering algorithm can be K-Means, hierarchical clustering, or DBSCAN, among others. Preferably, K-Means is used, with the number of clusters set to the number of categories in the labeled samples. The resulting cluster centers represent the feature distribution regions of different categories.

[0086] For the currently unlabeled image, calculate the distance between its depth feature vector and each cluster center. Distance calculation can use Euclidean distance, Manhattan distance, or cosine similarity, etc. The minimum distance value is taken as the feature diversity index for that image. A larger distance value indicates that the image is more sparse in the feature space, meaning it differs more from labeled samples and has higher diversity.

[0087] Analysis and Judgment Steps: The accuracy of the feature diversity index is affected by the number of clusters. If the number of clusters is set too large, the representativeness of each cluster center deteriorates, and the calculated distance values ​​may not be accurate enough; if the number of clusters is set too small, it may not fully reflect the distribution characteristics of the data. In addition, when the number of labeled samples is small, the stability of the clustering results may be affected. In this case, hierarchical clustering or density-based methods can be considered to obtain more stable clustering results.

[0088] Step S304: Calculate the overall score.

[0089] The prediction uncertainty index obtained in step S302 and the feature diversity index obtained in step S303 are weighted and fused to calculate the comprehensive score.

[0090] Specifically, the overall score equals the prediction uncertainty index multiplied by its corresponding weighting coefficient, plus the feature diversity index multiplied by its corresponding weighting coefficient.

[0091] In some implementations, the weighting coefficient for the prediction uncertainty index is set to be greater than the weighting coefficient for the feature diversity index. Alternatively, the weighting coefficient for the prediction uncertainty index may be set to 0.6, and the weighting coefficient for the feature diversity index may be set to 0.4.

[0092] Analysis and Judgment Steps: The weight coefficients need to be adjusted according to the actual application scenario. If the model's current prediction uncertainty is high, the weight of the prediction uncertainty index can be appropriately increased to prioritize the labeling of the most uncertain samples. If the model's uncertainty is already low, but its generalization ability is insufficient, the weight of the feature diversity index can be appropriately increased to select more diverse samples. In addition, the weight coefficients can also be optimized through validation set experiments to select the weight combination that maximizes the improvement of model performance.

[0093] Step S305: Batch sample selection and labeling queue push.

[0094] Based on the comprehensive score calculated in step S304, all unlabeled images are sorted in descending order, and a number of samples at the top of the list are selected to form a labeling batch. The size of the labeling batch is usually set to 32 or 64, and the specific value can be adjusted according to the labeling resources and time requirements.

[0095] The selected batch of labeled samples is pushed to the labeling queue, awaiting labeling by the labelers. After labeling is completed, the labeling results are added to the labeled sample set, and the process returns to step S301 to update the model before continuing with the next round of sample selection.

[0096] Analysis and Judgment Steps: The batch size setting needs to balance annotation efficiency and model improvement. If the batch size is too large, the cost of each annotation is high, and it may include some low-value samples, resulting in wasted resources. If the batch size is too small, the model update frequency decreases, and the iteration convergence speed slows down. Additionally, when selecting samples, a lower score threshold can be set to skip samples with excessively low scores, thus avoiding the selection of very low-value samples.

[0097] The technical advantages of this embodiment are: by prioritizing the annotation of difficult examples that contribute the most to model improvement, and avoiding wasting annotation resources on a large number of redundant examples, the amount of labeled data required to achieve the same model accuracy is significantly reduced. Annotation costs are significantly reduced.

[0098] Meanwhile, because the samples used in each training iteration are those with the highest information content, the accuracy of the model's validation set increases faster, and the iteration convergence cycle is significantly shortened. Model development efficiency is significantly improved.

[0099] Furthermore, by simultaneously considering both prediction uncertainty and feature diversity, we can ensure sample diversity while selecting high-value samples, avoiding model bias caused by samples being concentrated in a certain area. The selected labeled samples are more representative, and the model has stronger generalization ability.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions described in the foregoing embodiments; and such modifications or substitutions should all be covered within the scope of the claims and specification of the present invention.

Claims

1. An image data annotation method, characterized in that, Includes the following steps: Obtain the image to be labeled; The image to be labeled is downsampled to obtain multiple image layers of different resolutions to form an image pyramid structure; Feature extraction is performed on the multiple image layers with different resolutions to obtain multi-scale feature maps corresponding to each layer; An attention mechanism is used to adaptively fuse the multi-scale feature maps to obtain a fused feature representation. Based on the fused feature representation, the location information and edge contour information of the labeled target are determined, and the labeling result is generated.

2. The image data annotation method as described in claim 1, characterized in that, The construction of the image pyramid structure includes: performing downsampling processing on the image to be labeled at least three times to obtain at least four image levels with different resolutions.

3. The image data annotation method as described in claim 1, characterized in that, When the image to be labeled is a video frame sequence, the following steps are also included: Based on the annotation results of the current frame, the initial position of the annotation box in the adjacent frame is predicted using the optical flow prediction model; Calculate the coordinate difference between the current frame and the annotation results of adjacent frames; When the coordinate difference value exceeds a preset first threshold, a manual review mechanism is triggered; When the coordinate difference value does not exceed the first threshold, the predicted initial position of the bounding box is automatically used as the annotation result of the adjacent frame.

4. The image data annotation method as described in claim 3, characterized in that, It also includes the following steps: Obtain the category prediction probability distribution of adjacent frames; Calculate the difference between the class prediction probability distributions of the current frame and those of neighboring frames; When the difference exceeds a preset second threshold, a manual review mechanism is triggered.

5. The image data annotation method according to any one of claims 1 to 3, characterized in that, It also includes the following steps: Construct an active learning sample selection module; Hard example mining is performed on unlabeled images to obtain hard example samples; Calculate the overall score of the difficult sample; The samples are sorted in descending order of their comprehensive scores and then pushed to the labeling queue in descending order of their scores.

6. The image data annotation method as described in claim 5, characterized in that, The difficult case mining process includes: calculating the prediction uncertainty index of the sample.

7. The image data annotation method as described in claim 6, characterized in that, The prediction uncertainty index of the calculated sample includes: training an initial classification model using labeled images, performing multiple forward propagations on unlabeled images, calculating the entropy value of the prediction probability distribution based on the prediction results of the multiple forward propagations, and using the entropy value as the prediction uncertainty index.

8. The image data annotation method as described in claim 7, characterized in that, The difficult case mining process also includes: calculating the feature diversity index of the sample.

9. The image data annotation method as described in claim 8, characterized in that, The calculation of the feature diversity index of the sample includes: extracting the depth feature vector of the unlabeled image, performing cluster analysis on the depth feature vector to obtain multiple cluster centers, calculating the distance between the current sample and each cluster center, and using the minimum distance value as the feature diversity index.

10. The image data annotation method as described in claim 9, characterized in that: The overall score is calculated by weighting the prediction uncertainty index and the feature diversity index; wherein the weight coefficient of the prediction uncertainty index is greater than the weight coefficient of the feature diversity index.

Images