Image annotation boundary automatic correction method and system based on semi-supervised learning

Abstract

This invention relates to the field of image recognition technology, and more particularly to a method and system for automatic correction of image annotation boundaries based on semi-supervised learning. The system includes: an image processing module, comprising an acquisition unit for acquiring the image to be corrected and the features of the labeled boundary; a model training module, comprising a model training unit for training an initial model based on the labeled boundary features; a boundary correction module for detecting and automatically correcting the boundary features to be corrected according to a semi-supervised learning model; a matching degree adjustment module for determining the matching degree of the temporal features of the image to be corrected based on the acquisition clock offset of the image to be corrected; a pseudo-label adjustment module for determining the iterative evolution rate of pseudo-labels based on the rate of increase of the test set boundary error in the prediction model; and a learning rate adjustment module for determining a noise robustness learning rate scaling factor based on the noise proportion in the pseudo-labels. This invention improves the stability of image annotation boundary correction.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a method and system for automatic correction of image annotation boundaries based on semi-supervised learning. Background Technology

[0002] In the field of modern computer vision, image annotation boundary automatic correction technology based on semi-supervised learning has become a key technical support for ensuring the quality of annotation data due to its core advantages of low dependence on annotation data, high correction efficiency, and strong bias adaptability. Its correction accuracy directly determines the reliability of the annotation data and the training effect of models such as object detection and image segmentation. As semi-supervised learning-based image annotation boundary automatic correction technology develops towards scene generalization and accuracy adaptation, the data and scene features involved in the annotation correction process exhibit characteristics of multi-source, high-frequency, and high heterogeneity: different types of data have significantly different requirements for "real-time correction response, bias recognition accuracy, and cross-scene compatibility." Against this backdrop, traditional semi-supervised learning-based image annotation boundary automatic correction methods have gradually exposed many technical bottlenecks, making it difficult to meet the needs of accurate annotation in complex scenes.

[0003] Chinese Patent Publication No. CN118608554A discloses a semi-supervised learning-based ultrasound medical image segmentation method, comprising the following steps: training an image restoration model based on a first image dataset and a degraded image dataset obtained by downsampling the first image dataset; initializing a first pre-trained model and a second pre-trained model based on the image restoration model; training the first pre-trained model based on a second image dataset carrying true segmentation labels to obtain a first segmentation model; inputting a third image dataset into the first segmentation model to obtain pseudo-segmentation labels for the third image dataset output by the first segmentation model; and training the second pre-trained model based on the third image dataset carrying pseudo-segmentation labels and a fourth image dataset carrying true segmentation labels to obtain an image segmentation model. It is evident that this semi-supervised learning-based ultrasound medical image segmentation method suffers from a problem: due to the lack of control over the confidence threshold when selecting pseudo-segmentation labels, an excessive number of noisy pseudo-labels leads to incorrect model learning and insufficient stability in correcting image annotation boundaries. Summary of the Invention

[0004] To address this issue, the present invention provides an automatic image annotation boundary correction method and system based on semi-supervised learning, which overcomes the problem in the prior art where the lack of confidence threshold control during pseudo-segmentation label selection leads to an excessive number of noisy pseudo-labels, causing the model to learn incorrect annotation information and resulting in insufficient stability of image annotation boundary correction.

[0005] To achieve the above objectives, this invention provides a method and system for automatic correction of image annotation boundaries based on semi-supervised learning, comprising:

[0006] The image processing module includes an acquisition unit for acquiring the image to be corrected, labeled boundary features, and unlabeled boundary features, respectively, and a preprocessing unit connected to the acquisition unit for preprocessing the image to be corrected to output the boundary features to be corrected.

[0007] The model training module, which is connected to the image processing module, includes a model training unit for training an initial model based on the labeled boundary features to output a prediction model, a pseudo-label generation unit connected to the model training unit for predicting the unlabeled boundary features based on the prediction model to generate pseudo-labels, and a model update unit connected to the pseudo-label generation unit for updating the prediction model based on the pseudo-labels and the labeled boundary features to obtain a semi-supervised learning model.

[0008] A boundary correction module, which is connected to the model training module, is used to detect and automatically correct the boundary features to be corrected based on the semi-supervised learning model.

[0009] A matching degree adjustment module, which is connected to the image processing module, is used to determine the matching degree of the time features of the image to be corrected based on the acquisition clock offset of the image to be corrected.

[0010] A pseudo-label adjustment module, which is connected to the model training module and the matching degree adjustment module respectively, is used to determine the iterative evolution rate of pseudo-labels based on the growth rate of the boundary error of the test set in the prediction model.

[0011] The learning rate adjustment module is connected to both the model training module and the pseudo-label adjustment module, and is used to determine the noise robustness learning rate scaling factor based on the noise ratio in the pseudo-labels.

[0012] Furthermore, the matching degree adjustment module responds to the acquisition clock offset of the image to be corrected being less than or equal to a preset first offset, and determines that the correction stability of the image annotation boundary meets the requirements.

[0013] The matching degree adjustment module determines that the stability of the image annotation boundary correction does not meet the requirements when the acquisition clock offset of the image to be corrected is greater than the preset first offset.

[0014] Furthermore, in response to the acquisition clock offset of the image to be corrected being greater than the preset first offset and less than or equal to the preset second offset, the matching degree adjustment module initially determines that the generalization of the prediction model does not meet the requirements.

[0015] Furthermore, the matching degree adjustment module increases the matching degree of the time features of the image to be corrected in response to the acquisition clock offset of the image to be corrected being greater than the preset second offset.

[0016] The increase in the matching degree of the temporal features of the image to be corrected is determined by the difference between the acquisition clock offset of the image to be corrected and a preset second offset.

[0017] Furthermore, the pseudo-label adjustment module determines that the generalization of the prediction model meets the requirements when the growth rate of the test set boundary error in the prediction model is less than or equal to a preset first growth rate.

[0018] The pseudo-label adjustment module determines that the generalization of the prediction model does not meet the requirements when the growth rate of the test set boundary error in the prediction model is greater than the preset first growth rate.

[0019] Furthermore, the pseudo-label adjustment module reduces the iterative evolution rate of the pseudo-label in response to the fact that the growth rate of the test set boundary error in the prediction model is greater than the preset first growth rate and less than the preset second growth rate.

[0020] The pseudo-label adjustment module responds to the fact that the growth rate of the test set boundary error in the prediction model is greater than the preset second growth rate, and preliminarily determines that the boundary noise immunity of the prediction model does not meet the requirements.

[0021] Furthermore, the magnitude of the reduction in the iterative evolution rate of the pseudo-label is determined by the difference between the test set boundary error growth rate and the preset first growth rate in the prediction model.

[0022] Furthermore, the learning rate adjustment module responds to the fact that the noise ratio in the pseudo-label is less than or equal to the preset noise ratio, thus determining that the boundary noise immunity of the prediction model meets the requirements.

[0023] The learning rate adjustment module responds to the fact that the noise ratio in the pseudo-label is greater than the preset noise ratio, determines that the boundary noise immunity of the prediction model does not meet the requirements, and increases the noise robustness learning rate scaling factor.

[0024] Furthermore, the increase in the noise robustness learning rate scaling factor is determined by the difference between the noise proportion in the pseudo-label and the preset noise proportion.

[0025] This invention also provides a method for automatic correction of image annotation boundaries based on semi-supervised learning, comprising:

[0026] The image to be corrected, labeled boundary features, and unlabeled boundary features are collected separately. The image to be corrected is then cleaned, denoised, transformed, and feature extracted sequentially to output the boundary features to be corrected.

[0027] The initial model is trained using the labeled boundary features to obtain a prediction model. The unlabeled boundary features are then predicted based on the prediction model to generate pseudo-labels. The prediction model is updated based on the pseudo-labels and the labeled boundary features to obtain a semi-supervised learning model. The boundary features to be corrected are then detected and automatically corrected based on the semi-supervised learning model.

[0028] Obtain the acquisition clock offset of the image to be corrected, and determine whether the correction stability of the image annotation boundary meets the requirements based on the acquisition clock offset of the image to be corrected.

[0029] If the stability of the image annotation boundary correction does not meet the requirements, then determine whether it is necessary to increase the matching degree of the temporal features of the image to be corrected;

[0030] If it is not necessary to increase the matching degree of the temporal features of the image to be corrected, then obtain the growth rate of the boundary error of the test set in the prediction model to determine whether the generalization of the prediction model meets the requirements.

[0031] If the generalization of the prediction model does not meet the requirements, then determine whether it is necessary to reduce the iterative evolution rate of the pseudo-labels;

[0032] If it is not necessary to reduce the iterative evolution rate of pseudo-labels, then the noise robustness learning rate scaling factor is determined based on the noise proportion in the pseudo-labels.

[0033] Compared with existing technologies, the beneficial effects of this invention are as follows: The system of this invention, by setting up an image processing module, a model training module, a boundary correction module, a matching degree adjustment module, a pseudo-label adjustment module, and a learning rate adjustment module, adjusts the matching degree of the temporal features of the image to be corrected according to the acquisition clock offset of the image to be corrected. Since the image acquisition device may experience a clock offset, causing the image's own temporal metadata to mismatch with the actual scene time, subsequent annotation directly uses the incorrect temporal metadata, leading to time annotation errors. By increasing the matching degree of the temporal features of the image to be corrected, the incorrect metadata time labels can be brought closer to the actual time features implicit in the image content. By adjusting the metadata time through the content temporal sequence features, the correct temporal sequence is restored. The iterative evolution rate of the pseudo-labels is adjusted according to the boundary error growth rate of the test set in the prediction model. Since the pseudo-labels are the core of semi-supervised learning utilizing unlabeled data, the initial pseudo-labels are generated by the model's prediction of the unlabeled data. If the model's initial judgment of the boundary position is biased, subsequent iterations will use these incorrect pseudo-labels as supervision information. The forced learning of incorrect boundary features by pseudo-labels can lead to the model remembering local patterns of erroneous pseudo-labels, resulting in poor generalization. By reducing the iteration rate of pseudo-labels, the influence of erroneous pseudo-labels on the model can be reduced, creating a window for error correction. This allows the correct supervision signals from labeled data to more effectively guide the model and offset the negative impact of erroneous pseudo-labels. The noise robustness learning rate scaling factor can be adjusted by controlling the proportion of noise in the pseudo-labels. Since the model's ability to judge boundaries is weak in the early stages, if there is noise in the input image, the model may mistake this noise for real boundary features and generate pseudo-labels containing noise. In subsequent iterations, these noisy pseudo-labels will serve as supervision signals, forcing the model to learn the incorrect association between noise and boundaries, causing the noise to be continuously amplified. Increasing the noise robustness learning rate scaling factor can provide protection during the stage when the model's judgment is weakest. Even if the quality of the input image pseudo-labels is poor, the model will not learn incorrectly quickly due to the high learning rate, but will explore real boundary features at a more gradual pace, improving the stability of image label boundary correction.

[0034] Furthermore, the system of the present invention adjusts the matching degree of the time features of the image to be corrected by setting a preset first offset and a preset second offset. Since the image acquisition device may have a device clock offset, the time metadata of the image itself may not match the actual scene time. In subsequent annotation, the incorrect time metadata is directly used, resulting in time annotation errors. By increasing the matching degree of the time features of the image to be corrected, the incorrect metadata time label can be brought closer to the actual time features implied in the image content. By adjusting the metadata time through the content time sequence features, the correct time sequence is restored, which further improves the stability of the correction of the image annotation boundary.

[0035] Furthermore, the system of the present invention adjusts the iterative evolution rate of pseudo-labels by setting a preset first growth rate and a preset second growth rate. Since pseudo-labels are the core of semi-supervised learning using unlabeled data, the initial pseudo-labels are generated by the model's prediction of the unlabeled data. If the model's initial judgment of the boundary position is biased, subsequent iterations will use these erroneous pseudo-labels as supervision signals, forcing the model to learn incorrect boundary features. Ultimately, this causes the model to remember the local patterns of erroneous pseudo-labels, resulting in the inability to generalize. By reducing the iterative evolution rate of pseudo-labels, the influence of erroneous pseudo-labels on the model can be reduced, creating a window period for the system to correct errors. This allows the correct supervision signals of labeled data to more effectively guide the model, offsetting the negative impact of erroneous pseudo-labels and further improving the stability of image annotation boundary correction.

[0036] Furthermore, the system described in this invention adjusts the noise robustness learning rate scaling factor by setting a preset noise ratio. Since the model's ability to judge boundaries is weak in the early stages, if there is noise in the input image, the model may mistakenly identify this noise as real boundary features and generate pseudo-labels containing noise. In subsequent iterations, these noise pseudo-labels will serve as supervision signals, forcing the model to learn the incorrect association between noise and boundaries, resulting in the noise being continuously reinforced. By increasing the noise robustness learning rate scaling factor, protection can be provided in the stage where the model's judgment is weakest. Even if the quality of the input image pseudo-labels is poor, the model will not learn incorrectly quickly due to the high learning rate, but will explore real boundary features at a more gradual pace, thereby improving the stability of the correction of image annotation boundaries and further improving the stability of the correction of image annotation boundaries. Attached Figure Description

[0037] Figure 1 This is a block diagram of the overall structure of the image annotation boundary automatic correction system based on semi-supervised learning according to an embodiment of the present invention;

[0038] Figure 2 This is an overall flowchart of the image annotation boundary automatic correction method based on semi-supervised learning according to an embodiment of the present invention;

[0039] Figure 3 This is a flowchart illustrating the process of determining the matching degree of temporal features of an image to be corrected in an image annotation boundary automatic correction system based on semi-supervised learning, according to an embodiment of the present invention.

[0040] Figure 4 This is a flowchart illustrating the iterative evolution rate of the image annotation boundary automatic correction system based on semi-supervised learning in an embodiment of the present invention. Detailed Implementation

[0041] To make the objectives and advantages of the present invention clearer, the present invention will be further described below with reference to embodiments; it should be understood that the specific embodiments described herein are merely for explaining the present invention and are not intended to limit the present invention.

[0042] Preferred embodiments of the present invention will now be described with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are merely illustrative of the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0043] Please see Figure 1 , Figure 2 , Figure 3 as well as Figure 4 The diagrams shown are, respectively, the overall structural block diagram, the overall flowchart, the logical flowchart for determining the matching degree of the temporal features of the image to be corrected, and the logical flowchart for determining the iterative evolution rate of the pseudo-labels, all based on the semi-supervised learning-based automatic image annotation boundary correction method and system of this invention. This invention provides an automatic image annotation boundary correction system based on semi-supervised learning, comprising:

[0044] The image processing module includes an acquisition unit for acquiring the image to be corrected, labeled boundary features, and unlabeled boundary features, respectively, and a preprocessing unit connected to the acquisition unit for preprocessing the image to be corrected to output the boundary features to be corrected.

[0045] The model training module, which is connected to the image processing module, includes a model training unit for training an initial model based on the labeled boundary features to output a prediction model, a pseudo-label generation unit connected to the model training unit for predicting the unlabeled boundary features based on the prediction model to generate pseudo-labels, and a model update unit connected to the pseudo-label generation unit for updating the prediction model based on the pseudo-labels and the labeled boundary features to obtain a semi-supervised learning model.

[0046] A boundary correction module, which is connected to the model training module, is used to detect and automatically correct the boundary features to be corrected based on the semi-supervised learning model.

[0047] A matching degree adjustment module, which is connected to the image processing module, is used to determine the matching degree of the time features of the image to be corrected based on the acquisition clock offset of the image to be corrected.

[0048] A pseudo-label adjustment module, which is connected to the model training module and the matching degree adjustment module respectively, is used to determine the iterative evolution rate of pseudo-labels based on the growth rate of the boundary error of the test set in the prediction model.

[0049] The learning rate adjustment module is connected to both the model training module and the pseudo-label adjustment module, and is used to determine the noise robustness learning rate scaling factor based on the noise ratio in the pseudo-labels.

[0050] Specifically, preprocessing includes image cleaning, denoising, and conversion.

[0051] Specifically, the prediction model can be Faster R-CNN, U-Net, or HRNet.

[0052] Specifically, the images to be corrected include laptop images with background images, tablet images at different resolutions, and mobile phone images taken from different angles.

[0053] Specifically, the labeled boundary features are the image boundary features that have been accurately labeled.

[0054] Specifically, unlabeled boundary features are image boundary features that have not yet been accurately labeled.

[0055] Specifically, the boundary features to be corrected include the core outline of the tablet computer image, the boundary offset of the laptop image, and the matching degree between the texture of the mobile phone frame and the marked boundary.

[0056] Specifically, the process of training the initial model based on the marked boundary features to output a prediction model involves dividing the marked boundary features into a training set, a test set, and a validation set. The input layer of the initial model receives the preprocessed image, and the output layer corresponds to the boundary prediction result. Using the marked boundary features as the standard answer, a loss function (such as Dice loss or IoU loss) is used to calculate the deviation between the model's predicted boundary and the marked boundary. The model parameters are optimized through backpropagation, and the training is iteratively continued until the boundary prediction accuracy on the validation set reaches the target. After training, the model with stable parameters that can accurately learn the marked boundary rules is saved, which is the prediction model that can be used for subsequent boundary correction.

[0057] Specifically, the process of generating pseudo-labels by predicting unlabeled boundary features using a prediction model involves inputting the unlabeled boundary features into the prediction model. Based on the boundary rules of electronic products learned from labeled data (such as component outlines and size ratios), the prediction model predicts the boundary positions and shapes in the unlabeled features. Results with high confidence in the model prediction (such as the predicted circular boundary of a mobile phone camera having a high degree of matching with the prior structure) are selected and labeled as pseudo-labels with reference value for use as supplementary data for subsequent semi-supervised training.

[0058] Specifically, the process of updating the prediction model to obtain a semi-supervised learning model based on pseudo-labels and labeled boundary features involves merging the selected high-confidence pseudo-labels (such as the accurate electronic product component boundaries predicted by the model) with the labeled boundary features to form a hybrid dataset for semi-supervised training. The supervised loss (such as IoU loss) is calculated using the labeled boundary features as the true labels to constrain the model's accuracy. The consistency loss (such as the deviation between the prediction results and the pseudo-labels) is calculated using the pseudo-labels as a reference to allow the model to learn the patterns of unlabeled data. The two losses are simultaneously optimized through backpropagation, and the model parameters are iteratively adjusted until the model can stably learn the electronic product boundary features on both labeled and pseudo-labeled data, ultimately resulting in a semi-supervised learning model.

[0059] Specifically, semi-supervised learning models can be semi-supervised U-Net, semi-supervised Mask R-CNN, and self-trained prediction models.

[0060] Specifically, the matching degree of temporal features of the image to be corrected measures the consistency of boundary annotation features (such as component outlines, positions, and shapes) of the same (or similar) images to be corrected collected at different times.

[0061] Specifically, the iterative evolution rate of pseudo-labels is the rate at which the quality of pseudo-labels changes during the semi-supervised training iteration process as the prediction model iterates.

[0062] Specifically, the noise robustness learning rate scaling factor is a coefficient that dynamically adjusts the learning rate weights based on the noise level of the data (such as pseudo-labels and annotation boundaries).

[0063] In implementation, the system of this invention adjusts the matching degree of the temporal features of the image to be corrected based on the clock offset of the image acquisition. Since the image acquisition device may experience clock offset, the image's temporal metadata may not match the actual scene time. Subsequent annotations directly use the incorrect temporal metadata, leading to time labeling errors. By increasing the matching degree of the temporal features of the image to be corrected, the incorrect metadata time labels can be brought closer to the actual time features implicit in the image content. Adjusting the metadata time using the content's temporal sequence features restores the correct temporal sequence. The iterative evolution rate of the pseudo-labels is adjusted based on the rate of increase of the test set boundary error in the prediction model. Since pseudo-labels are the core of semi-supervised learning using unlabeled data, the initial pseudo-labels are generated by the model's prediction of the unlabeled data. If the model's initial judgment of the boundary position is biased, subsequent iterations will use these incorrect pseudo-labels as supervision signals to force the model to learn. Learning incorrect boundary features ultimately leads to the model remembering local patterns of erroneous pseudo-labels, resulting in poor generalization. By reducing the iteration rate of pseudo-labels, the influence of erroneous pseudo-labels on the model can be reduced, creating a window for the system to correct errors. This allows the correct supervision signals from labeled data to more effectively guide the model, offsetting the negative impact of erroneous pseudo-labels. The noise robustness learning rate scaling factor can be adjusted by controlling the proportion of noise in the pseudo-labels. Since the model's ability to judge boundaries is weak in the early stages, if there is noise in the input image, the model may mistake this noise for real boundary features and generate pseudo-labels containing noise. In subsequent iterations, these noisy pseudo-labels will serve as supervision signals, forcing the model to learn the incorrect association between noise and boundaries, causing the noise to be continuously reinforced. By increasing the noise robustness learning rate scaling factor, protection can be provided in the stage where the model's judgment is weakest. Even if the quality of the input image pseudo-labels is poor, the model will not learn incorrectly quickly due to the high learning rate, but will explore real boundary features at a more gradual pace, improving the stability of image annotation boundary correction.

[0064] Specifically, the matching degree adjustment module responds to the acquisition clock offset of the image to be corrected being less than or equal to a preset first offset, and determines that the correction stability of the image annotation boundary meets the requirements.

[0065] The matching degree adjustment module determines that the stability of the image annotation boundary correction does not meet the requirements when the acquisition clock offset of the image to be corrected is greater than the preset first offset.

[0066] Specifically, the matching degree adjustment module responds to the acquisition clock offset of the image to be corrected being greater than the preset first offset and less than or equal to the preset second offset, initially determining that the generalization of the prediction model does not meet the requirements, and then determines whether the generalization of the prediction model meets the requirements based on the growth rate of the boundary error of the test set in the prediction model.

[0067] It is understandable that the preset first offset is less than the preset second offset, and the three intervals divided by the preset first offset and the preset second offset correspond to three different cases:

[0068] The first interval is when the acquisition clock offset of the image to be corrected is less than or equal to the preset first offset. The corresponding situation is that the correction stability of the image annotation boundary meets the requirements.

[0069] The second interval is when the acquisition clock offset of the image to be corrected is greater than the preset first offset and less than or equal to the preset second offset. The corresponding situation is as follows: Since pseudo-labels are the core of semi-supervised learning to utilize unlabeled data, the initial pseudo-labels are generated by the model's prediction of the unlabeled data. If the model's initial judgment of the boundary position is biased, subsequent iterations will use these erroneous pseudo-labels as supervision signals to force the model to learn the incorrect boundary features, which will eventually cause the model to remember the local patterns of the erroneous pseudo-labels and fail to generalize.

[0070] The third interval is when the acquisition clock offset of the image to be corrected is greater than the preset second offset. The corresponding situation is: due to the possibility of a clock offset in the image acquisition device, the time metadata of the image itself does not match the actual scene time. Subsequent annotation directly uses the incorrect time metadata, resulting in time annotation errors.

[0071] Understandably, in a semi-supervised learning-based automatic image annotation boundary correction system, using the first and second offsets to characterize the correction stability of image annotation boundaries is essentially a design logic based on pseudo-label iteration bias gradient adaptation in semi-supervised learning scenarios. This avoids the limitations of a single bias standard. By using a hierarchical bias standard, the intuitive and quantitative indicator of annotation boundary offset is closely linked to different states of correction stability, subsequent boundary correction, and pseudo-label iteration rate adjustment. This not only meets the rigorous requirements of semi-supervised learning for annotation correction accuracy but also satisfies the practical needs of boundary hierarchical adjustment and model generalization capability protection. The core function of the first offset is to quickly filter stable annotation states that do not require adjustment; essentially, it is the lowest acceptable standard based on the semi-supervised learning sample distribution and the initial prediction characteristics of pseudo-labels. The core function of the second offset is to serve as a graded line for reasons why correction stability does not meet requirements. The preset first and second offsets can be set according to actual working conditions. The setting of the preset first and second offsets aims to ensure the correction stability and practicality of image annotation boundaries. Optionally, the preset first offset and preset second offset are determined through a limited number of experiments by evaluating the correction effect of different clock offsets on the image annotation boundaries. The determined preset first offset and preset second offset should satisfy the condition that they are neither too small nor cause excessive interference to the correction process of the image annotation boundaries. For example, the preset first offset is generally selected in the range of [14μs, 16μs], and the preset second offset is generally selected in the range of [29μs, 31μs].

[0072] Preferably, the first preset offset is 15 μs, and the second preset offset is 30 μs.

[0073] Specifically, the acquisition clock offset of the image to be corrected is the difference between the timestamp recorded in the metadata of the image file and the corresponding real-world time.

[0074] Specifically, the matching degree adjustment module increases the matching degree of the time features of the image to be corrected in response to the acquisition clock offset of the image to be corrected being greater than the preset second offset.

[0075] The increase in the matching degree of the temporal features of the image to be corrected is determined by the difference between the acquisition clock offset of the image to be corrected and a preset second offset.

[0076] Specifically, when the difference between the acquisition clock offset of the image to be corrected and the preset second offset is within 4μs, the matching degree of the time feature of the image to be corrected increases to 1.1 times the original value. When the difference between the acquisition clock offset of the image to be corrected and the preset second offset exceeds 4μs, on the basis of increasing to 1.1 times the original value, the matching degree of the time feature of the image to be corrected increases by 1% for every 1μs exceeding the original value. For example, if the difference between the acquisition clock offset of the image to be corrected and the preset second offset is 4μs, and the current matching degree of the time feature of the image to be corrected is 80%, the matching degree of the time feature of the image to be corrected after the increase is 80×1.1+1×2=90%.

[0077] In practice, the system of the present invention adjusts the matching degree of the time features of the image to be corrected by setting a preset first offset and a preset second offset. Since the image acquisition device may have a device clock offset, the time metadata of the image itself may not match the actual scene time. In subsequent annotation, the incorrect time metadata is directly used, resulting in time annotation errors. By increasing the matching degree of the time features of the image to be corrected, the incorrect metadata time label can be brought closer to the actual time features implied in the image content. By adjusting the metadata time through the content time sequence features, the correct time sequence is restored, further improving the correction stability of the image annotation boundary.

[0078] Specifically, the pseudo-label adjustment module determines that the generalization of the prediction model meets the requirements when the growth rate of the test set boundary error in the prediction model is less than or equal to a preset first growth rate.

[0079] The pseudo-label adjustment module determines that the generalization of the prediction model does not meet the requirements when the growth rate of the test set boundary error in the prediction model is greater than the preset first growth rate.

[0080] Specifically, the pseudo-label adjustment module reduces the iterative evolution rate of pseudo-labels in response to the fact that the growth rate of the test set boundary error in the prediction model is greater than the preset first growth rate and less than the preset second growth rate.

[0081] The pseudo-label adjustment module responds to the fact that the growth rate of the test set boundary error in the prediction model is greater than the preset second growth rate, initially determines that the boundary noise immunity of the prediction model does not meet the requirements, and determines whether the boundary noise immunity of the prediction model meets the requirements based on the noise ratio in the pseudo-label.

[0082] It is understandable that the preset first growth rate is less than the preset second growth rate, and the three intervals divided by the preset first growth rate and the preset second growth rate correspond to three different scenarios:

[0083] The first interval is when the growth rate of the test set boundary error in the prediction model is less than or equal to the preset first growth rate, which corresponds to the generalization of the prediction model meeting the requirements.

[0084] The second interval is when the growth rate of the test set boundary error in the prediction model is greater than the preset first growth rate and less than or equal to the preset second growth rate. The corresponding situation is as follows: Since pseudo-labels are the core of semi-supervised learning to utilize unlabeled data, the initial pseudo-labels are generated by the model's prediction of unlabeled data. If the model's initial judgment of the boundary position is biased, subsequent iterations will use these erroneous pseudo-labels as supervision signals to force the model to learn the erroneous boundary features, which will eventually cause the model to remember the local patterns of erroneous pseudo-labels and thus fail to generalize.

[0085] The third interval is when the growth rate of the test set boundary error in the prediction model is greater than the preset second growth rate. The corresponding situation is that the model's ability to judge the boundary is weak in the early stage. If there is noise in the input image, the model may mistakenly identify this noise as the real boundary feature and generate pseudo-labels containing noise. In subsequent iterations, these noise pseudo-labels will be used as supervision signals, forcing the model to learn the wrong association between noise and boundary, resulting in the noise being continuously reinforced.

[0086] Understandably, in a semi-supervised learning-based automatic image annotation boundary correction system, using the first and second growth rates to characterize the generalization ability of the prediction model is essentially to align with the semi-supervised learning approach, which aims to both adapt to the rate gradient of pseudo-label iteration and avoid generalization imbalance caused by training the model with erroneous pseudo-labels. The first growth rate essentially serves as the basic acceptable threshold for the prediction model's generalization ability, defining the upper limit of acceptable error for pseudo-label boundary annotation. The second growth rate distinguishes whether the prediction model has lost generalization ability, differentiating between high and low risk of generalization failure and providing clear triggering conditions for generalization protection actions. The preset first and second growth rates can be set according to actual working conditions. The setting of the preset first and second growth rates aims to ensure the stability and practicality of the image annotation boundary correction. Optionally, the preset first and second growth rates are determined through a limited number of experiments by evaluating the effect of different error growth rates on the image annotation boundary correction. The determined preset first and second growth rates should be neither too small nor excessively interfere with the image annotation boundary correction process. For example, the preset first growth rate is generally selected in the range of [1.5%, 2.5%], and the preset second growth rate is generally selected in the range of [3.5%, 4.5%].

[0087] Preferably, the preset first growth rate is 2% in a preferred embodiment, and the preset second growth rate is 4% in a preferred embodiment.

[0088] Specifically, the test set boundary error growth rate in the prediction model is the ratio of the new error of the pseudo-label boundary to the pseudo-label boundary labeling error of the previous round.

[0089] Specifically, the reduction in the iterative evolution rate of the pseudo-label is determined by the difference between the test set boundary error and the preset first growth rate in the prediction model.

[0090] Specifically, when the difference between the growth rate of the test set boundary error in the prediction model and the preset first growth rate is within 1%, the iterative evolution rate of the pseudo-label is reduced to 0.9 times its original value. When the difference exceeds 1%, in addition to reducing it to 0.9 times its original value, for every 0.2% increase, the iterative evolution rate of the pseudo-label decreases by 0.005. For example, if the difference between the growth rate of the test set boundary error in the prediction model and the preset first growth rate is 1.2%, and the current iterative evolution rate of the pseudo-label is 0.03, the reduced iterative evolution rate of the pseudo-label is 0.03 × 0.9 - 0.005 × 1 = 0.022.

[0091] In implementation, the system of this invention adjusts the iterative evolution rate of pseudo-labels by setting a preset first growth rate and a preset second growth rate. Since pseudo-labels are the core of semi-supervised learning using unlabeled data, the initial pseudo-labels are generated by the model's prediction of the unlabeled data. If the model's initial judgment of the boundary position is biased, subsequent iterations will use these erroneous pseudo-labels as supervision signals, forcing the model to learn incorrect boundary features. Ultimately, this causes the model to remember the local patterns of erroneous pseudo-labels, resulting in the inability to generalize. By reducing the iterative evolution rate of pseudo-labels, the influence of erroneous pseudo-labels on the model can be reduced, creating a window period for the system to correct errors. This allows the correct supervision signals from labeled data to more effectively guide the model, offsetting the negative impact of erroneous pseudo-labels and further improving the stability of image annotation boundary correction.

[0092] Specifically, the learning rate adjustment module responds to the fact that the noise ratio in the pseudo-label is less than or equal to the preset noise ratio, thus determining that the boundary noise immunity of the prediction model meets the requirements.

[0093] The learning rate adjustment module responds to the fact that the noise ratio in the pseudo-label is greater than the preset noise ratio, determines that the boundary noise immunity of the prediction model does not meet the requirements, and increases the noise robustness learning rate scaling factor.

[0094] It is understandable that the two intervals defined by the preset noise percentage correspond to two different scenarios:

[0095] The first interval is when the noise proportion in the pseudo-label is less than or equal to the preset noise proportion, which corresponds to the situation where the boundary noise immunity of the prediction model meets the requirements.

[0096] The second interval is when the noise ratio in the pseudo-label is greater than the preset noise ratio. The corresponding situation is as follows: Since the model's ability to judge the boundary is weak in the early stage, if there is noise in the input image, the model may mistakenly regard this noise as the real boundary feature and generate pseudo-labels containing noise. In subsequent iterations, these noise pseudo-labels will be used as supervision signals to force the model to learn the wrong association between noise and boundary, resulting in the noise being continuously amplified.

[0097] Understandably, in a semi-supervised learning-based automatic image annotation boundary correction system, using a preset noise percentage to characterize the boundary noise robustness of the prediction model is based on the core logic of relating the noise percentage in pseudo-labels to the model's anti-interference capability. It transforms the abstract boundary noise robustness into a quantifiable noise percentage threshold judgment, essentially using the model's tolerance to pseudo-label noise to reverse-verify the effectiveness of the noise robustness adjustment mechanism. The preset noise percentage serves as the basic acceptable threshold for boundary noise robustness, defining the maximum acceptable noise percentage upper limit in pseudo-labels. The preset noise percentage can be set according to actual working conditions. The setting of the preset noise percentage aims to ensure the stability and practicality of the image annotation boundary correction. Optionally, the preset noise percentage is determined through a limited number of experiments by evaluating the correction effect of different noise percentages on the image annotation boundaries. The determined preset noise percentage should be neither too small nor cause excessive interference to the image annotation boundary correction process. For example, the preset noise percentage is generally selected within the range of [3%, 7%].

[0098] Preferably, the preset noise percentage is 5% in the preferred embodiment.

[0099] Specifically, the noise percentage in pseudo-labels is the ratio of the number of noisy pseudo-labels to the total number.

[0100] Specifically, the increase in the noise robustness learning rate scaling factor is determined by the difference between the noise proportion in the pseudo-label and the preset noise proportion.

[0101] Specifically, when the difference between the noise percentage in the pseudo-label and the preset noise percentage is within 1%, the noise robustness learning rate scaling factor is increased to 1.1 times the original value. When the difference between the noise percentage in the pseudo-label and the preset noise percentage exceeds 1%, the noise robustness learning rate scaling factor is increased by 0.05 for every 0.5% increase beyond the original value, in addition to the original 1.1 times. For example, if the difference between the noise percentage in the pseudo-label and the preset noise percentage is 2%, the current noise robustness learning rate scaling factor is 0.3, and the increased noise robustness learning rate scaling factor is 0.3×1.1+0.05×2=0.43.

[0102] In implementation, the system of this invention adjusts the noise robustness learning rate scaling factor by setting a preset noise ratio. Since the model's ability to judge boundaries is weak in the early stages, if there is noise in the input image, the model may mistakenly identify this noise as real boundary features and generate pseudo-labels containing noise. In subsequent iterations, these noise pseudo-labels will serve as supervision signals, forcing the model to learn the incorrect association between noise and boundaries, resulting in the noise being continuously reinforced. By increasing the noise robustness learning rate scaling factor, protection can be provided in the stage where the model's judgment is weakest. Even if the quality of the input image pseudo-labels is poor, the model will not learn incorrectly quickly due to the high learning rate, but will explore real boundary features at a more gradual pace, thereby improving the stability of the correction of image annotation boundaries and further improving the stability of the correction of image annotation boundaries.

[0103] An automatic image annotation boundary correction method based on semi-supervised learning includes:

[0104] Step S1: Collect the image to be corrected, labeled boundary features, and unlabeled boundary features respectively, and sequentially clean, denoise, transform, and extract features from the image to be corrected to output the boundary features to be corrected;

[0105] Step S2: Train the initial model using the labeled boundary features to obtain a prediction model; predict the unlabeled boundary features based on the prediction model to generate pseudo-labels; update the prediction model based on the pseudo-labels and the labeled boundary features to obtain a semi-supervised learning model; and detect and automatically correct the boundary features to be corrected based on the semi-supervised learning model.

[0106] Step S3: Obtain the acquisition clock offset of the image to be corrected, and determine whether the correction stability of the image annotation boundary meets the requirements based on the acquisition clock offset of the image to be corrected.

[0107] Step S4: If the stability of the correction of the image annotation boundary does not meet the requirements, determine whether it is necessary to increase the matching degree of the temporal features of the image to be corrected.

[0108] Step S5: If it is not necessary to increase the matching degree of the temporal features of the image to be corrected, then obtain the growth rate of the boundary error of the test set in the prediction model to determine whether the generalization of the prediction model meets the requirements.

[0109] Step S6: If the generalization of the prediction model does not meet the requirements, determine whether it is necessary to reduce the iterative evolution rate of the pseudo-label.

[0110] Step S7: If it is not necessary to reduce the iterative evolution rate of pseudo-labels, then determine the noise robustness learning rate scaling factor based on the noise ratio in the pseudo-labels.

[0111] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.

Claims

1. An image annotation boundary automatic correction system based on semi-supervised learning, characterized in that, include: The image processing module includes an acquisition unit for acquiring the image to be corrected, labeled boundary features, and unlabeled boundary features, respectively, and a preprocessing unit connected to the acquisition unit for preprocessing the image to be corrected to output the boundary features to be corrected. The model training module, which is connected to the image processing module, includes a model training unit for training an initial model based on the labeled boundary features to output a prediction model, a pseudo-label generation unit connected to the model training unit for predicting the unlabeled boundary features based on the prediction model to generate pseudo-labels, and a model update unit connected to the pseudo-label generation unit for updating the prediction model based on the pseudo-labels and the labeled boundary features to obtain a semi-supervised learning model. A boundary correction module, which is connected to the model training module, is used to detect and automatically correct the boundary features to be corrected based on the semi-supervised learning model. A matching degree adjustment module, which is connected to the image processing module, is used to determine the matching degree of the time features of the image to be corrected based on the acquisition clock offset of the image to be corrected. A pseudo-label adjustment module, which is connected to the model training module and the matching degree adjustment module respectively, is used to determine the iterative evolution rate of pseudo-labels based on the growth rate of the boundary error of the test set in the prediction model. A learning rate adjustment module, which is connected to the model training module and the pseudo-label adjustment module respectively, is used to determine the noise robustness learning rate scaling factor based on the noise ratio in the pseudo-label. The matching degree of the temporal features of the image to be corrected measures the consistency of the boundary annotation features of the same image to be corrected acquired at different times. The iterative evolution rate of the pseudo-label is the rate at which the pseudo-label changes in quality as the prediction model iterates during the semi-supervised training iteration process. The noise robustness learning rate scaling factor is a coefficient that dynamically adjusts the learning rate weights based on the noise level of the data. The matching degree adjustment module responds to the acquisition clock offset of the image to be corrected being less than or equal to a preset first offset, and determines that the correction stability of the image annotation boundary meets the requirements. The matching degree adjustment module determines that the stability of the image annotation boundary correction does not meet the requirements when the acquisition clock offset of the image to be corrected is greater than the preset first offset.

2. The image annotation boundary automatic correction system based on semi-supervised learning according to claim 1, characterized in that, The matching degree adjustment module responds to the fact that the acquisition clock offset of the image to be corrected is greater than the preset first offset and less than or equal to the preset second offset, and initially determines that the generalization of the prediction model does not meet the requirements.

3. The image annotation boundary automatic correction system based on semi-supervised learning according to claim 2, characterized in that, The matching degree adjustment module increases the matching degree of the time features of the image to be corrected in response to the acquisition clock offset of the image to be corrected being greater than the preset second offset. The increase in the matching degree of the temporal features of the image to be corrected is determined by the difference between the acquisition clock offset of the image to be corrected and a preset second offset.

4. The image annotation boundary automatic correction system based on semi-supervised learning according to claim 3, characterized in that, The pseudo-label adjustment module determines that the generalization of the prediction model meets the requirements when the growth rate of the test set boundary error in the prediction model is less than or equal to a preset first growth rate. The pseudo-label adjustment module determines that the generalization of the prediction model does not meet the requirements when the growth rate of the test set boundary error in the prediction model is greater than the preset first growth rate.

5. The image annotation boundary automatic correction system based on semi-supervised learning according to claim 4, characterized in that, The pseudo-label adjustment module reduces the iterative evolution rate of the pseudo-label in response to the fact that the growth rate of the test set boundary error in the prediction model is greater than the preset first growth rate and less than the preset second growth rate. The pseudo-label adjustment module responds to the fact that the growth rate of the test set boundary error in the prediction model is greater than the preset second growth rate, and preliminarily determines that the boundary noise immunity of the prediction model does not meet the requirements.

6. The image annotation boundary automatic correction system based on semi-supervised learning according to claim 5, characterized in that, The reduction in the iterative evolution rate of the pseudo-label is determined by the difference between the test set boundary error growth rate and the preset first growth rate in the prediction model.

7. The image annotation boundary automatic correction system based on semi-supervised learning according to claim 6, characterized in that, The learning rate adjustment module responds to the fact that the noise ratio in the pseudo-label is less than or equal to the preset noise ratio, thus determining that the boundary noise immunity of the prediction model meets the requirements. The learning rate adjustment module responds to the fact that the noise ratio in the pseudo-label is greater than the preset noise ratio, determines that the boundary noise immunity of the prediction model does not meet the requirements, and increases the noise robustness learning rate scaling factor.

8. The image annotation boundary automatic correction system based on semi-supervised learning according to claim 7, characterized in that, The increase in the noise robustness learning rate scaling factor is determined by the difference between the noise proportion in the pseudo-label and the preset noise proportion.

9. A correction method applied to the image annotation boundary automatic correction system based on semi-supervised learning as described in any one of claims 1-8, characterized in that, include: The image to be corrected, labeled boundary features, and unlabeled boundary features are collected separately. The image to be corrected is then cleaned, denoised, transformed, and feature extracted sequentially to output the boundary features to be corrected. The initial model is trained using the labeled boundary features to obtain a prediction model. The unlabeled boundary features are then predicted based on the prediction model to generate pseudo-labels. The prediction model is updated based on the pseudo-labels and the labeled boundary features to obtain a semi-supervised learning model. The boundary features to be corrected are then detected and automatically corrected based on the semi-supervised learning model. Obtain the acquisition clock offset of the image to be corrected, and determine whether the correction stability of the image annotation boundary meets the requirements based on the acquisition clock offset of the image to be corrected. If the stability of the image annotation boundary correction does not meet the requirements, then determine whether it is necessary to increase the matching degree of the temporal features of the image to be corrected; If it is not necessary to increase the matching degree of the temporal features of the image to be corrected, then obtain the growth rate of the boundary error of the test set in the prediction model to determine whether the generalization of the prediction model meets the requirements. If the generalization of the prediction model does not meet the requirements, then determine whether it is necessary to reduce the iterative evolution rate of the pseudo-labels; If it is not necessary to reduce the iterative evolution rate of pseudo-labels, then the noise robustness learning rate scaling factor is determined based on the noise proportion in the pseudo-labels.

Images