SAM2 segmentation-based YOLO automatic labeling tool and method
By using a YOLO automatic annotation tool based on SAM2 segmentation, the problems of low annotation efficiency and unstable quality in existing technologies are solved, and efficient and accurate target detection datasets are generated, which are applicable to fields such as autonomous driving, intelligent monitoring, medical image analysis and industrial inspection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- FUDAN UNIVERSITY
- Filing Date
- 2026-01-29
- Publication Date
- 2026-05-22
AI Technical Summary
Existing object detection algorithms suffer from low annotation efficiency, high cost, and unstable quality in fields such as autonomous driving, intelligent monitoring, medical image analysis, and industrial inspection. They are particularly difficult to accurately separate objects in dense or occluded scenarios, and lack efficient temporal consistency and a unified data processing flow.
The YOLO automatic annotation tool based on SAM2 segmentation is adopted. The SAM2 automatic segmentation module generates binary mask images, which are combined with morphological erosion processing and target separation modules, connected component analysis and bounding box generation modules to provide interactive annotation correction. Finally, the data is converted into YOLO format dataset, forming a closed-loop process to improve annotation efficiency and consistency.
It significantly reduces manual workload, improves annotation quality and consistency, generates high-quality datasets suitable for YOLO training, reduces missing and incorrect annotations, reduces repetitive operations, improves the stability and usability of the dataset, and adapts to image data of different sizes and devices.
Smart Images
Figure CN122072967A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of computer vision and artificial intelligence, specifically a YOLO automatic annotation tool and method based on SAM2 segmentation. Background Technology
[0002] With the rapid development of deep learning technology, object detection algorithms have been widely used in fields such as autonomous driving, intelligent monitoring, medical image analysis, and industrial inspection. The YOLO series of algorithms, as one of the most advanced object detection frameworks currently available, requires a large amount of high-quality labeled data for model training. Traditional manual annotation methods suffer from low efficiency, high cost, and unstable quality, severely hindering the development and application of object detection technology.
[0003] In existing technologies, automated annotation tools mainly employ the following methods:
[0004] The first type is based on traditional image processing algorithms, such as background subtraction, inter-frame differencing, and optical flow. While these methods are computationally simple, their accuracy in target recognition in complex scenes is limited, especially in cases of target occlusion, lighting variations, and complex backgrounds, making it difficult to generate accurate labeled data.
[0005] The second approach is based on deep learning, such as using instance segmentation models like Faster R-CNN and Mask R-CNN for automatic annotation. While these methods improve annotation accuracy, they require a large amount of pre-trained data and computational resources, and typically can only process single images, lacking effective temporal consistency handling capabilities for video sequences.
[0006] The third method is the semi-automatic annotation method provided by YOLO, based on YOLO and the original SAM. This type of method still relies heavily on manual annotation in the early stages of video sequence annotation.
[0007] In addition, the existing technology also has the following technical defects:
[0008] In scenarios where targets are densely packed or occluded, traditional methods struggle to accurately separate adjacent targets, leading to multiple targets being incorrectly labeled as a single entity.
[0009] Existing automatic annotation tools often generate bounding boxes that deviate significantly from the actual boundaries of the target, requiring extensive manual adjustments to meet training requirements.
[0010] For video data, existing tools typically require frame-by-frame processing and lack an effective mechanism for maintaining temporal consistency, which leads to a significant increase in the workload of annotation.
[0011] Existing solutions often consist of multiple independent tools, lacking a unified data format and workflow, requiring users to manually perform complex tool switching and data conversion.
[0012] Therefore, there is an urgent need for a semi-automated annotation toolset that can achieve high accuracy, high efficiency, and high quality, effectively solving the problems of annotation accuracy, efficiency, and integration in existing technologies, and providing high-quality training datasets for object detection algorithms such as YOLO. Summary of the Invention
[0013] The purpose of this invention is to provide a YOLO automatic annotation tool and method based on SAM2 segmentation to solve the technical problems mentioned in the background.
[0014] Based on the above ideas, the present invention provides the following technical solution:
[0015] A YOLO automatic annotation tool based on SAM2 segmentation includes:
[0016] The SAM2 automatic segmentation module receives video files input by the user, automatically segments the video files using the SAM2 deep learning segmentation model, generates binary mask images for identifying pixel-level boundaries of targets, and supports video target tracking.
[0017] The mask image processing module is used to perform format conversion and preprocessing on the binary mask image so that the binary mask image is suitable for subsequent connected component analysis. The binary mask image includes multi-target instance segmentation results, and each instance corresponds to an independent binary mask region.
[0018] The morphological erosion processing and target separation module is used to perform morphological erosion processing on the binary mask image before connected component analysis, and to separate the adhered targets by means of configurable erosion kernel size and iteration number parameters;
[0019] The connected component analysis and bounding box generation module is used to perform connected component analysis on the binary mask image after preprocessing and morphological erosion. It uses the 8-connectivity algorithm to identify independent connected regions and calculates the minimum bounding rectangle coordinates for each connected region to generate initial bounding box data.
[0020] The interactive annotation correction module provides a graphical user interface to support users in visually inspecting and manually correcting the initial bounding box data, and enables interactive operations such as selecting, moving, scaling, and deleting bounding boxes; it also provides size filtering and batch dilation / erosion adjustment functions.
[0021] The YOLO format conversion and dataset generation module is used to convert the corrected bounding box coordinate data into the YOLO standard format, perform coordinate normalization and class encoding processing, and automatically divide the training set and validation set according to a preset ratio to generate a standard dataset for YOLO training.
[0022] By integrating the SAM2 automatic segmentation module, mask image processing module, morphological erosion processing and target separation module, connected component analysis and bounding box generation module, interactive annotation correction module, and YOLO format conversion and dataset generation module, a closed-loop process is formed from video input, target segmentation, separation of connected targets, bounding box generation, manual correction to YOLO format dataset output. This significantly reduces the workload of manual frame-by-frame annotation, improves annotation efficiency, and enhances the consistency and usability of output annotations. At the same time, the automatic generation and interactive correction methods balance automation and accuracy, making the generated dataset more suitable for YOLO training requirements.
[0023] Preferably, the SAM2 automatic segmentation module is configured to support automatic segmentation mode and video target tracking mode, and output binary mask data to characterize the pixel-level boundaries of the targets; each target corresponds to an independent mask region.
[0024] By configuring the SAM2 automatic segmentation module to support both automatic segmentation mode and video target tracking mode, and outputting independent binary mask regions for each target, it is easier to maintain consistent segmentation results for the same target across multiple frames. This reduces omissions, mislabeling, or boundary drift caused by inter-frame changes, thereby improving the stability and consistency of subsequent connected component analysis, bounding box generation, and dataset output.
[0025] Preferably, the size of the corrosion kernel and the number of iterations in the morphological corrosion processing and target separation module are configurable parameters, and a preview display of the corrosion effect is provided to support users in adjusting the configurable parameters.
[0026] By setting the morphological erosion processing and target separation module's erosion kernel size and iteration count as configurable parameters and providing a preview of the erosion effect, users can quickly adjust the separation intensity for targets with different densities, scales, or degrees of adhesion. This reduces erroneous merging of adhesion targets while preserving the main body shape of the target as much as possible, improving the accuracy of bounding box generation and reducing subsequent manual correction costs.
[0027] Preferably, the user interface of the interactive annotation correction module is developed based on the Python Tkinter framework and adopts an event-driven architecture; the bounding box operations include creation, selection, movement, scaling, and deletion; it supports drawing and adjusting bounding boxes with the left mouse button and deselecting with the right mouse button; the size filtering function filters bounding boxes based on normalized width, height, and area thresholds; the batch dilation / erosion adjustment function supports previewing the effect and batch application to multiple frames of data;
[0028] The tool includes two separate software tools:
[0029] The first tool is an inference tool developed based on SAM2, configured to perform deep learning image segmentation and video object tracking to generate mask image files;
[0030] The second tool is a YOLO annotation tool, configured to process masked images, generate bounding boxes, provide an interactive correction interface, and output YOLO format datasets; the first and second tools exchange data through a standard file format, and users use them in sequence to complete the annotation process.
[0031] By implementing an event-driven interactive annotation correction interface based on Python Tkinter, and providing operations such as creation, selection, movement, scaling, and deletion, as well as size filtering and batch dilation / erosion adjustment functions, users can efficiently review and batch correct automatically generated annotations, reducing repetitive operations and improving overall annotation consistency. At the same time, the dual-tool separation architecture, which uses a first tool to generate mask image files and a second tool to generate and correct bounding boxes and output YOLO datasets, is conducive to module decoupling and process clarification, facilitating sequential use and maintenance upgrades, and reducing coupling between tools through file exchange, thereby improving deployment and expansion flexibility.
[0032] A YOLO automatic annotation method based on SAM2 segmentation, based on the aforementioned YOLO automatic annotation tool based on SAM2 segmentation, includes:
[0033] S1. Use the SAM2 automatic segmentation module to automatically segment the user-input video file, generate a binary mask image, and perform video target tracking;
[0034] S2. The binary mask image is format-converted and preprocessed using the mask image processing module so that the binary mask image is suitable for subsequent connected component analysis;
[0035] S3. Morphological erosion processing and target separation module is used to perform morphological erosion processing on the binary mask image before connected component analysis, and the adhered targets are separated by configurable erosion kernel size and iteration number parameters.
[0036] S4. The connected component analysis and bounding box generation module is used to perform connected component analysis on the binary mask image after preprocessing and morphological erosion. The 8-connectivity algorithm is used to identify independent connected regions, and the coordinates of the minimum bounding rectangle are calculated for each connected region to generate initial bounding box data.
[0037] S5. The initial bounding box data is displayed in the graphical user interface using the interactive annotation correction module, and user interaction is received to correct the bounding box by selecting, moving, scaling and deleting, while size filtering and batch dilation / erosion adjustment are performed.
[0038] S6. Using the YOLO format conversion and dataset generation module, the corrected bounding box coordinate data is converted into the YOLO standard format, and coordinate normalization and class encoding are performed. At the same time, the training set and validation set are automatically divided according to the preset ratio to generate a standard dataset for YOLO training.
[0039] By establishing a clear step chain for automatic segmentation, mask preprocessing, erosion separation, connected component analysis, bounding box generation, interactive correction, and YOLO format output and dataset partitioning, the automatic annotation process becomes reproducible, implementable, and applicable to batch video data. It can improve the degree of automation while ensuring controllable annotation quality. At the same time, human intervention is limited to the display and correction stages, which facilitates rapid correction. This achieves a better balance between efficiency and accuracy, and ultimately produces datasets that can be directly used for YOLO training more efficiently.
[0040] Preferably, the coordinate normalization in the YOLO format conversion and dataset generation module converts the pixel coordinates of the bounding box into relative coordinates according to the YOLO annotation specification. The relative coordinates include the relative x-coordinate, relative y-coordinate, relative width, and relative height of the bounding box center point. The relative width is the ratio of the bounding box width to the image width, the relative height is the ratio of the bounding box height to the image height, and the relative x-coordinate and relative y-coordinate of the center point are the ratios of the bounding box center point coordinates to the image width and image height, respectively.
[0041] By converting the pixel coordinates of the bounding box into relative coordinates such as the center point, relative width, and relative height according to the YOLO annotation specification, the output annotation is decoupled from the image resolution. This facilitates maintaining the consistency of annotation expression under different image input conditions and reduces training bias caused by resolution differences. At the same time, the standardized relative coordinate output is conducive to direct interface with the YOLO training framework, reducing data preprocessing costs and format conversion errors.
[0042] Preferably, the method includes the following operation steps:
[0043] The user runs the first tool and inputs a video file to generate a mask image file; the user runs the second tool and loads the mask image file and the corresponding original image, sets the erosion parameters and performs connected component analysis to generate bounding boxes, corrects the bounding boxes through the interactive interface, and finally outputs a YOLO format labeled dataset.
[0044] By clearly defining the operational flow of the first tool generating the mask image file, the second tool loading the mask image file and the original image and generating bounding boxes, interactive correction, and outputting the YOLO format dataset, the barrier to entry can be lowered, the number of missed steps or misoperations caused by process confusion can be reduced, and the source and processing path of the labeled data can be clearly traced. At the same time, the sequential use of tools is conducive to forming a fixed work process in batch tasks, improving overall production efficiency and delivery stability.
[0045] The technical solution of the present invention may include the following beneficial effects:
[0046] The SAM2 automatic segmentation module, mask image processing module, morphological erosion processing and target separation module, connected component analysis and bounding box generation module, interactive annotation correction module, and YOLO format conversion and dataset generation module are linked together into a complete closed loop, covering the entire process from video file input to standard YOLO training dataset output. This avoids the multiple rounds and repeated imports and exports required in traditional workflows, such as segmentation, manual screening, bounding box drawing, format modification, and dataset splitting. Since automatic segmentation and target tracking directly output binary mask images, initial bounding boxes are automatically generated through connected components and minimum bounding rectangles, and necessary manual corrections are then performed through the interface. This transforms manual work from drawing from scratch to reviewing results and making minor corrections, significantly reducing the annotation cycle, increasing the throughput of batch video data processing, and standardizing and reproducible the annotation process, facilitating team collaboration and large-scale production.
[0047] Controllable automation improves annotation quality: On one hand, the morphological erosion and target separation module performs erosion preprocessing on the mask before connected component analysis, allowing configurable and previewable erosion kernel size and iteration count. This enables users to quickly adjust the separation intensity for different degrees of adhesion, target densities, and morphological differences, reducing misclassification of multiple targets as the same connected component and improving the reliability of bounding box generation. On the other hand, the interactive annotation correction module provides editing capabilities such as creating, selecting, moving, scaling, and deleting bounding boxes. Combined with size filtering and batch dilation / erosion adjustments, it allows users to quickly locate abnormal boxes (too large, too small, offset, missing separation, etc.) and perform batch-level corrections, reducing repetitive frame-by-frame and box-by-box operations. These mechanisms combine the efficiency advantages of automatic generation with the quality assurance of manual control, significantly reducing the impact of missing, mislabeled, off-target, and inconsistent boxes on training data quality, thus improving dataset usability and stability from the source.
[0048] The module offers direct benefits in data delivery and training adaptation: The YOLO format conversion and dataset generation module converts bounding box pixel coordinates into relative coordinates such as the center point's horizontal and vertical coordinates, relative width, and relative height, according to the YOLO annotation specifications. This decouples annotation from image resolution, ensuring consistent data representation across images of different sizes, cropping strategies, or acquisition devices, thereby reducing training bias and additional preprocessing costs caused by resolution differences. Simultaneously, the module automatically performs category encoding and divides the training and validation sets according to a preset ratio, reducing the risk of data leakage, distribution imbalance, or misplacement caused by manual splitting. This allows the generated dataset to be directly used for YOLO training and iterative validation, shortening the closed-loop cycle of annotation, training, evaluation, and re-annotation. Furthermore, the dual-tool separation architecture—where the first tool generates mask image files and the second tool generates, corrects, and outputs the YOLO format dataset—facilitates process clarity and module decoupling. Standard file exchange reduces system coupling and deployment complexity, facilitating the reuse, upgrading, or replacement of individual tool modules in different environments, thus improving the overall maintainability and scalability of the project. Attached Figure Description
[0049] Figure 1 This is a toolset architecture diagram of the YOLO automatic annotation tool and method based on SAM2 segmentation of this invention.
[0050] Figure 2 This is a flowchart illustrating the workflow of the SAM2 automatic segmentation module in the YOLO automatic annotation tool and method based on SAM2 segmentation of this invention.
[0051] Figure 3This is a schematic diagram illustrating the connected component analysis and bounding box generation principle of the YOLO automatic annotation tool and method based on SAM2 segmentation of this invention.
[0052] Figure 4 This is a comparison of the morphological erosion processing effects of the YOLO automatic annotation tool and method based on SAM2 segmentation of this invention.
[0053] Figure 5 This is a comparison of the morphological erosion processing effects of the YOLO automatic annotation tool and method based on SAM2 segmentation of this invention.
[0054] Figure 6 This is a flowchart illustrating the YOLO format conversion and dataset generation process of the YOLO automatic annotation tool and method based on SAM2 segmentation of this invention.
[0055] Figure 7 This is a complete annotation workflow diagram of the YOLO automatic annotation tool and method based on SAM2 segmentation of this invention.
[0056] Figure 8 This is a mask image illustrating an example of the YOLO automatic annotation tool and method based on SAM2 segmentation according to the present invention.
[0057] Figure 9 This is a schematic diagram illustrating the YOLO automatic annotation tool and method based on SAM2 segmentation of the present invention when S4 corrosion separation is turned off.
[0058] Figure 10 for Figure 9 This is a schematic diagram illustrating the S4 erosion separation process when the YOLO automatic annotation tool and method based on SAM2 segmentation of this invention is enabled. Detailed Implementation
[0059] Example 1
[0060] This implementation includes the following technical solutions:
[0061] like Figure 1 and 7 The tools comprise two independent software tools, forming a set of technical tools:
[0062] The first tool is an inference tool developed based on SAM2, configured to perform deep learning image segmentation and video object tracking, and generate binary mask data;
[0063] Based on the SAM2 model developed by Meta, this tool supports automatic segmentation and target tracking modes. It can handle various input formats, including video files (MP4, AVI, MOV, etc.). The output is a binary mask image, with each target corresponding to a separate mask file.
[0064] The second tool is the YOLO annotation tool, configured to process masked images, generate bounding boxes, provide an interactive correction interface, and output YOLO format datasets; the two tools exchange data via standard file formats, and users complete the annotation process by using them sequentially.
[0065] It includes functions such as mask loading, connected component analysis, bounding box generation, interactive correction, and format conversion. The tool is developed in Python, uses OpenCV for image processing, and Tkinter to build the user interface.
[0066] It also includes the following steps:
[0067] S1. Construct the SAM2 automatic segmentation module: Configured to receive video files input by the user, automatically segment the input data using the SAM2 deep learning segmentation model, and generate a binary mask image. The mask image identifies the pixel-level boundaries of the target in the image; supports video target tracking function and can maintain the consistency of the target between consecutive frames.
[0068] like Figure 2 The SAM2 automatic segmentation module consists of four main steps: model loading, data preprocessing, segmentation inference, and result output.
[0069] The model loading step supports loading SAM2 pre-trained models from the HuggingFace model library or local configuration files. Model loading employs a delayed loading strategy, loading the model into memory only when needed, optimizing resource usage.
[0070] The data preprocessing step standardizes the input image, including resizing and pixel value normalization. For video input, it also includes frame extraction and index management functions.
[0071] The segmentation inference step provides an automatic target segmentation and tracking mode: the user specifies the target in the starting frame or any frame, and the model tracks the target in subsequent video frames.
[0072] The output step saves the segmentation results as binary mask images in PNG format, with each target corresponding to a separate mask file. Simultaneously, a metadata JSON file is generated, recording segmentation parameters and result statistics.
[0073] S2. Develop a mask image processing module: Configure it to perform format conversion and preprocessing on the mask image generated by SAM2 to ensure that the mask image is suitable for subsequent connected component analysis; the mask image includes multi-target instance segmentation results, with each instance corresponding to an independent binary mask region;
[0074] S3. Connectivity analysis and bounding box generation module: configured to perform connectivity analysis on the preprocessed mask image, use the 8-connectivity algorithm to identify independent connected regions in the image; calculate the coordinates of the minimum bounding rectangle for each connected region, and generate initial bounding box data, which includes geometric information such as the pixel coordinates, width, and height of the target;
[0075] like Figure 3 Connectivity analysis is implemented using OpenCV's `connectedComponentsWithStats` function. This function uses a Two-Pass algorithm: 1. First scan: Assign a temporary label to each pixel. 2. Second scan: Merge equivalent labels to generate the final connected component labels.
[0076] Connectivity analysis algorithms have the following characteristics: - Time complexity: O(n), where n is the number of image pixels - Supports 8-connectivity and 4-connectivity - Output includes region labels, area, bounding box, centroid, and other information.
[0077] The bounding box calculation uses the minimum bounding rectangle algorithm, ensuring that the rectangle completely contains all pixels of the connected region. The calculation formula is:
[0078]
[0079] in The minimum coordinates of the connected region. The maximum coordinates.
[0080] The parameters for connected component analysis are configured as follows:
[0081] Parameter name default value Parameter Description Connection method 8-connected Connectivity is determined using eight adjacent pixels. Minimum area 10 pixels Filter connected regions smaller than this area Minimum width 3 pixels Filter areas with a width smaller than this value Minimum height 3 pixels Filter areas with a height less than this value Background Tags 0 Label values of background pixels
[0082] S4. Morphological Erosion Processing and Target Separation Module: Configured to preprocess the mask image before connected component analysis using morphological erosion operations. Through configurable erosion kernel size and iteration number parameters, it effectively separates mutually adhered target objects. The erosion operation is based on mathematical morphology principles and uses structuring elements to process the mask image pixel by pixel, removing boundary pixels and retaining the core region, thereby achieving the target separation effect.
[0083] This embodiment tested 377 frames of data and found that enabling S4 erosion separation (5×5 erosion kernels, 3 iterations) can effectively reduce the number of erroneous mergings.
[0084] S4 corrosion separation shut off Enable S4 corrosion separation Number of incorrectly merged boxes 247 0 Number of manual interventions 247 0
[0085] like Figure 8 This is a mask image;
[0086] like Figure 9This is a schematic diagram showing the S4 corrosion separation process when it is turned off; the incorrectly merged boxes are visible.
[0087] like Figure 10 This is a schematic diagram showing the S4 corrosion separation process when enabled; there are no incorrectly merged boxes.
[0088] like Figure 4 Morphological etching is used to separate mutually adhered targets, improving the accuracy of connected domain analysis.
[0089] The mathematical expression for the corrosion operation is:
[0090]
[0091] in, Indicates size is The corrosion operation, Represents a structural element. This represents the input binary image.
[0092] Key parameters for the erosion process include: - Structural element size: 1×1 to 9×9 pixels, typically an odd number - Number of iterations: 0 to 5, controlling the erosion intensity - Structural element shape: square (current implementation)
[0093] The evaluation criteria for corrosion effect are as follows:
[0094] 1. Separation degree: The effectiveness of separating adhered targets.
[0095] 2. Retention: The degree to which the target dimensions are retained.
[0096] 3. Completeness: The integrity of the target structure
[0097] The applicable scenarios for corrosion parameters are shown in the table below:
[0098] nuclear size Number of iterations Separation effect Applicable Scenarios 1×1 0 No separation Non-adhesive target 3×3 1 slight separation slight adhesion 5×5 2-3 intermediate separation moderate adhesion 7×7 3-4 Strong separation Severe adhesion 9×9 4-5 Strong separation Extreme adhesion
[0099] S5. Construct an interactive annotation correction module: Configured to provide a graphical user interface, supporting users to visually inspect and manually correct automatically generated bounding boxes; realize interactive operations such as selecting, moving, scaling, and deleting bounding boxes; provide optimization functions such as size filtering and batch dilation / erosion adjustment to ensure the accuracy and consistency of annotation data;
[0100] like Figure 5 The interactive annotation correction module provides an intuitive user interface and supports manual adjustment of automatically generated bounding boxes.
[0101] The user interface was developed using the Tkinter framework and includes the following main components:
[0102] Image display area: Displays the original image overlaid with an automatically generated bounding box;
[0103] Tools panel: Provides editing tools such as selection, drawing, and deletion;
[0104] Parameter settings area: Configure corrosion parameters and screening conditions;
[0105] Status bar: Displays current frame information and operation prompts;
[0106] Interactive operations include:
[0107] Left mouse button: Draw a new bounding box;
[0108] Left-click and drag: Move the bounding box;
[0109] Right-click: Delete bounding box;
[0110] Keyboard shortcuts: Switch frames, save annotations;
[0111] The size filtering function is based on normalized parameters and supports the following settings:
[0112] Width range: 0.0-1.0 (relative to image width);
[0113] Height range: 0.0-1.0 (relative to image height);
[0114] Area range: 0.0-1.0 (relative to image area);
[0115] The batch dilation / erosion adjustment function allows users to uniformly adjust the selected bounding boxes:
[0116] Positive value: Dilation operation, expands the bounding box;
[0117] Negative values: Erosion operation, shrinks the bounding box;
[0118] Supports preview mode and batch application.
[0119] S6. Develop a YOLO format conversion and dataset generation module: Configure it to convert the corrected bounding box coordinate data into the YOLO standard format, including coordinate normalization, class encoding and other processing; automatically divide the training set and validation set according to a preset ratio to generate a standard dataset that meets the YOLO training requirements, including image files, annotation files and dataset partitioning files.
[0120] Specifically, the SAM2 model is a Meta open-source model that supports automatic segmentation mode and video target tracking mode, and can maintain the consistency of target IDs between consecutive video frames. The model is based on the Transformer architecture, uses an attention mechanism to achieve global feature modeling, and supports GPU-accelerated inference. The segmentation results generate pixel-level precision binary mask data, with each target corresponding to an independent mask region.
[0121] Specifically, the morphological corrosion processing module in S4 adopts the following technical solution:
[0122] The erosion operation uses square structuring elements with an odd number of pixels in size; the erosion kernel size is dynamically adjusted according to the target size and separation requirements, ranging from 1×1 to 9×9 pixels; the number of erosion iterations controls the erosion intensity, ranging from 0 to 5 times; the erosion effect is displayed in real time through the preview function, and users can adjust the parameters according to the actual situation to obtain the best target separation effect.
[0123] Specifically, the interactive annotation correction module in S5 has the following technical characteristics:
[0124] The user interface is developed based on the Python Tkinter framework and adopts an event-driven architecture. Bounding box operations include basic functions such as creation, selection, movement, scaling, and deletion. It supports drawing and adjusting bounding boxes with the left mouse button and deselecting with the right mouse button. It provides a size filtering function, filtering bounding boxes based on normalized width, height, and area ranges. It implements batch dilation / erosion adjustment functions, supports previewing effects, and batch application to all frames.
[0125] Specifically, such as Figure 6 The YOLO format conversion module in S6 implements the following technical process:
[0126] The coordinate transformation uses a normalization algorithm to convert pixel coordinates into relative coordinates; the coordinate transformation formula is:
[0127]
[0128]
[0129]
[0130]
[0131] in These are the pixel coordinates of the bounding box. The image size is specified; the dataset is partitioned using a random sampling algorithm to ensure a balanced distribution of data between the training and validation sets.
[0132] YOLO annotation file format: class_id x_center y_center width height
[0133] Each line represents the annotation information for one target. Explanation of each field:
[0134] class_id: Category ID (integer, starting from 0);
[0135] x_center: x-coordinate of the bounding box center (normalized value, 0-1);
[0136] y_center: The y-coordinate of the bounding box center (normalized value, 0-1);
[0137] width: Bounding box width (normalized value, 0-1);
[0138] height: Bounding box height (normalized value, 0-1);
[0139] The dataset is split using a random sampling algorithm: 1. Collect all labeled image files. 2. Randomly shuffle the file list. 3. Divide the validation set according to a preset ratio (default 0.2). 4. Generate train.txt and valid.txt index files.
Claims
1. A YOLO automatic annotation tool based on SAM2 segmentation, characterized in that, include: The SAM2 automatic segmentation module receives video files input by the user, automatically segments the video files using the SAM2 deep learning segmentation model, generates binary mask images for identifying pixel-level boundaries of targets, and supports video target tracking. The mask image processing module is used to perform format conversion and preprocessing on the binary mask image so that the binary mask image is suitable for subsequent connected component analysis. The binary mask image includes multi-target instance segmentation results, and each instance corresponds to an independent binary mask region. The morphological erosion processing and target separation module is used to perform morphological erosion processing on the binary mask image before connected component analysis, and to separate the adhered targets by means of configurable erosion kernel size and iteration number parameters; The connected component analysis and bounding box generation module is used to perform connected component analysis on the binary mask image after preprocessing and morphological erosion. It uses the 8-connectivity algorithm to identify independent connected regions and calculates the minimum bounding rectangle coordinates for each connected region to generate initial bounding box data. An interactive annotation correction module is used to provide a graphical user interface to support users in visually inspecting and manually correcting the initial bounding box data, and to realize interactive operations such as selecting, moving, scaling and deleting bounding boxes. It also provides size filtering and batch expansion / erosion adjustment functions; The YOLO format conversion and dataset generation module is used to convert the corrected bounding box coordinate data into the YOLO standard format, perform coordinate normalization and class encoding, and automatically divide the training set and validation set according to a preset ratio to generate a standard dataset for YOLO training.
2. The YOLO automatic annotation tool based on SAM2 segmentation according to claim 1, characterized in that: The SAM2 automatic segmentation module is configured to support automatic segmentation mode and video target tracking mode, and output binary mask data to characterize the pixel-level boundaries of the targets; each target corresponds to an independent mask region.
3. The YOLO automatic annotation tool and method based on SAM2 segmentation according to claim 2, characterized in that, The morphological etching process and target separation module has configurable parameters for the etching kernel size and the number of iterations, and provides a preview of the etching effect to support user adjustment of the configurable parameters.
4. A YOLO automatic annotation tool based on SAM2 segmentation according to claim 3, characterized in that, The user interface of the interactive annotation correction module is developed based on the Python Tkinter framework and adopts an event-driven architecture; bounding box operations include creation, selection, movement, scaling, and deletion; It supports drawing and adjusting bounding boxes with the left mouse button and deselecting with the right mouse button; the size filtering function filters bounding boxes based on normalized width, height, and area thresholds; the batch dilation / erosion adjustment function supports previewing the effect and batch application to multiple frames of data. The tool includes two separate software tools: The first tool is an inference tool developed based on SAM2, configured to perform deep learning image segmentation and video object tracking to generate mask image files; The second tool is a YOLO annotation tool, configured to process masked images, generate bounding boxes, provide an interactive correction interface, and output YOLO format datasets; the first and second tools exchange data through a standard file format, and users use them in sequence to complete the annotation process.
5. A YOLO automatic annotation method based on SAM2 segmentation, characterized in that, The YOLO automatic annotation tool based on SAM2 segmentation as described in any one of claims 1-4 includes: S1. Use the SAM2 automatic segmentation module to automatically segment the user-input video file, generate a binary mask image, and perform video target tracking; S2. The binary mask image is format-converted and preprocessed using the mask image processing module so that the binary mask image is suitable for subsequent connected component analysis; S3. Morphological erosion processing and target separation module is used to perform morphological erosion processing on the binary mask image before connected component analysis, and the adhered targets are separated by configurable erosion kernel size and iteration number parameters. S4. The connected component analysis and bounding box generation module is used to perform connected component analysis on the binary mask image after preprocessing and morphological erosion. The 8-connectivity algorithm is used to identify independent connected regions, and the coordinates of the minimum bounding rectangle are calculated for each connected region to generate initial bounding box data. S5. The initial bounding box data is displayed in the graphical user interface using the interactive annotation correction module, and user interaction is received to correct the bounding box by selecting, moving, scaling and deleting, while size filtering and batch dilation / erosion adjustment are performed. S6. Using the YOLO format conversion and dataset generation module, the corrected bounding box coordinate data is converted into the YOLO standard format, and coordinate normalization and class encoding are performed. At the same time, the training set and validation set are automatically divided according to the preset ratio to generate a standard dataset for YOLO training.
6. The YOLO automatic annotation method based on SAM2 segmentation according to claim 5, characterized in that: The coordinate normalization in the YOLO format conversion and dataset generation module converts the pixel coordinates of the bounding box into relative coordinates according to the YOLO annotation specification. The relative coordinates include the relative x-coordinate, relative y-coordinate, relative width, and relative height of the bounding box center point. The relative width is the ratio of the bounding box width to the image width, and the relative height is the ratio of the bounding box height to the image height. The relative x-coordinate and relative y-coordinate of the center point are the ratios of the bounding box center point coordinates to the image width and image height, respectively.
7. A YOLO automatic annotation method based on SAM2 segmentation according to claim 6, characterized in that, The method includes the following operational steps: The user runs the first tool and inputs a video file to generate a mask image file; the user runs the second tool and loads the mask image file and the corresponding original image, sets the erosion parameters and performs connected component analysis to generate bounding boxes, corrects the bounding boxes through the interactive interface, and finally outputs a YOLO format labeled dataset.