Water conservancy element semi-automatic labeling method based on MAFs and SAM large model

Abstract

The invention discloses a water conservancy element semi-automatic labeling method based on MAFs and SAM large models, and relates to the technical field of water conservancy element semi-automatic labeling methods. The method comprises the steps of firstly performing operations such as overturning, mirroring, cutting and rotating on an unmanned aerial vehicle water conservancy element image to expand a sample data volume, then inputting water conservancy element prompt point information (pixel coordinates) by utilizing interface interaction operation, inputting an SAM Marge segmentation engine, generating an initial segmentation result, and in the face of changes of complex water conservancy elements, weeds, colors and the like, generating an initial segmentation result; the SAM initial segmentation result cannot completely distinguish the water conservancy elements from the non-water conservancy elements, in view of the result, each connected component of a binary image is calculated by using morphology, noise points of the non-water conservancy elements are filtered according to attribute information such as the water conservancy element area and the diagonal length, and finally a high-precision and high-accuracy water conservancy element sample image is obtained. The method is used for subsequent water conservancy factor semantic segmentation training.

Description

Technical Field

[0001] This invention relates to the technical field of semi-automatic AI annotation methods for water conservancy elements, and particularly to a semi-automatic annotation method for water conservancy elements based on morphological attribute filtering (MAFs) and SAM large model. Background Technology

[0002] Semantic segmentation is a computer vision task that aims to classify each pixel in an image, thereby assigning a category label to each pixel. Unlike image classification and object detection, semantic segmentation not only focuses on the types of objects present in an image but also aims to accurately identify the contours and regions of objects. This makes semantic segmentation an important component of image segmentation, forming a complete framework for image understanding along with instance segmentation and panoptic segmentation.

[0003] Traditional methods for identifying water conservancy elements typically rely on manual annotation and experience-based judgment, which are inefficient and easily affected by human factors. With the development of technologies such as artificial intelligence, the Internet of Things, and cloud computing, AI image recognition technology is gradually being widely applied. In the fields of artificial intelligence and machine learning, image annotation is a crucial task. Whether performing object detection, image segmentation, or image classification, a large amount of labeled data is needed to train the model. An efficient image annotation tool is essential for shortening data preparation time and improving the quality and efficiency of working samples.

[0004] Currently, mainstream open-source annotation tools include LabelMe and LabelImg. LabelImg is an open-source image annotation tool primarily used for object recognition and detection. It is written in Python and has a graphical interface based on QT. It supports rectangular bounding box annotation and can directly generate TXT and JSON file formats that YOLO can read, often used for object detection bounding box labeling. LabelMe, developed by MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL), is an image annotation tool with richer and more flexible features compared to LabelImg. It supports annotation in various forms, including polygons, rectangles, circles, polylines, line segments, and points. It can be used not only for object detection and image segmentation tasks but also for flag-based annotation for image classification and cleanup tasks. However, when facing segmentation tasks, LabelMe primarily uses manual outlining and drawing along the object's contour with the mouse for labeling (e.g., ...). Figure 1 (As shown). Training an AI semantic recognition model with broad general capabilities often requires a lot of manpower to draw each point one by one, which is time-consuming.

[0005] Currently, open-source AI semantic annotation software or tools rely on manually tracing and outlining each object's edge point by point to generate the mask file needed for training samples. Figure 1 As the number of local samples continues to increase, issues arise such as low manual efficiency and continuous manual output leading to a mismatch between the edge points of the sample delineation and the actual edge points of the water conservancy elements, resulting in low sample quality and impacting the quality and efficiency of subsequent AI semantic training of the images.

[0006] To address the aforementioned issues, this invention proposes a semi-automatic annotation method for water conservancy elements based on morphological attribute filtering (MAFs) and the SAM large model. Summary of the Invention

[0007] The purpose of this invention is to propose a semi-automatic annotation method for hydraulic elements based on morphological attribute filtering (MAFs) and the SAM large model. By combining the powerful object segmentation capabilities of the SAM general segmentation large model with the powerful manipulation capabilities of morphology on mask files, this method aims to solve the following problems:

[0008] (1) Using the SAM Large general segmentation model, the initial seed points are used as prompt points to input the prompt encoder to obtain the initial segmentation results. The relatively simple hydraulic elements can obtain high-quality segmentation mask files (mask images).

[0009] (2) For complex hydraulic elements and their underlying surface conditions, the initial SAM segmentation results often have unsatisfactory problems such as "islands", "burrs" and "voids". In this case, a post-processing method needs to be designed to optimize the segmentation results.

[0010] To achieve the above objectives, the present invention adopts the following technical solution:

[0011] A semi-automatic annotation method for hydraulic elements based on morphological attribute filtering (MAFs) and the SAM large model includes the following steps:

[0012] S1. Sample Data Augmentation: Perform geometric transformations on the original UAV water conservancy element images, including one or more of flipping, mirroring, cropping, and rotating, to increase the amount of sample data;

[0013] S2. Interactive input prompts: Select water conservancy element targets through manual interaction in the image interface, generate sparse point prompts and record the coordinates of other pixels;

[0014] S3, SAM Large Model Initial Segmentation: Input the sparse point hints generated in S2 into the SAM Large segmentation engine to generate the initial segmentation binary mask map of the hydraulic elements.

[0015] S4. Morphological Attribute Calculation and Analysis: Connected component labeling is performed on the initial segmentation binary mask obtained in S3, and the morphological attributes of each connected component are extracted; the morphological attributes include one or more of the following: area, diagonal length, moment of inertia, and second moment.

[0016] S5. Morphological Attribute Filtering (MAFs): Based on the set characteristics of water conservancy elements, attribute threshold rules are set to filter out connected components that do not conform to the rules, and optimized water conservancy element annotation results are generated.

[0017] S6. Output labeled samples: Use the optimized labeled results from S5 as training samples for training the semantic segmentation model of water conservancy elements.

[0018] Preferably, the cue information of the SAM large model includes sparse cue and dense cue, and the sparse point cue information in S2 is generated in the following way:

[0019] The foreground point of the target area of ​​the water conservancy element can be selected by clicking on the image interface through manual interaction;

[0020] The prompt information is embedded in the SAM prompt encoder in the form of location encoding.

[0021] Preferably, the SAM Large segmentation engine in S3 includes an image encoder, a cue word encoder, and a mask encoder;

[0022] The image encoder is used to encode images. A visual self-attention model (ViT) pre-trained with a masked autoencoder (MAE) is used as the image encoder to adapt to high-resolution input.

[0023] The cue word encoder is used to encode cue information, where sparse cues are represented by points and boxes through positional encoding, plus embeddings for each cue type and free-form text from the CLIP off-the-shelf text encoder; dense cues use convolutional embeddings and are element-wise summed with image embeddings.

[0024] The mask decoder block updates all embeddings using cue-based self-attention and cross-attention; after two decoders, the image embeddings are upsampled, the MLP maps the output to a linear classifier, and then the dynamic linear classifier computes the mask foreground probability for each image location.

[0025] Preferably, the connected component marking in S4 is implemented using a connection component marking algorithm, specifically including the following:

[0026] Scan each pixel in the binary image, group pixels with the same value that are connected to each other into the same group, and obtain all connected components of the pixels in the image;

[0027] A morphological analysis is performed on the area, diagonal length, moment of inertia, and second moment of the connected component to comprehensively describe its characteristics and identify attribute combinations that meet specific objectives of hydraulic elements for differentiation.

[0028] Preferably, S5 specifically includes the following:

[0029] Based on morphological filtering algorithms, the fixed structuring elements of attribute morphology are transformed into flexible attribute structuring elements that can be expressed by equations and numbers, so as to make up for the shortcomings of fixed-size structuring elements and the insufficient adaptability of adaptive threshold segmentation methods to blurry image filtering; based on two processes of connected component calculation and attribute logic judgment, attribute filtering is defined by predefined attributes of the target itself, and filtering rules are formulated according to the characteristics of the target.

[0030] Preferably, the attribute threshold rule in S5 specifically includes the following:

[0031] The area attribute is calculated using pixel recursion and boundary chain code method;

[0032] The diagonal length attribute is calculated using the diagonal length of the bounding rectangle of the connected components;

[0033] The moment of inertia property is calculated using the invariance of the second moment of the connected components.

[0034] The present invention further protects a computer device, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the instruction, program, code set or instruction set being loaded and executed by the processor to implement the above-mentioned semi-automatic annotation method for water conservancy elements based on morphological attribute filtering (MAFs) and SAM large model.

[0035] The present invention further protects a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the instruction, program, code set, or instruction set is loaded and executed by a processor to implement the above-mentioned semi-automatic annotation method for water conservancy elements based on morphological attribute filtering (MAFs) and SAM large model.

[0036] Compared with the prior art, the present invention has the following beneficial effects:

[0037] (1) In view of the shortcomings of existing open-source annotation software, which requires complex manual operations such as outlining and drawing in the process of semantic recognition sample annotation, resulting in low efficiency and time and effort, this invention proposes to use the SAM general segmentation large model engine to perform batch initial segmentation of water conservancy element targets with interactive prompt information.

[0038] (2) In view of the problem that the initial segmentation results of SAM in the face of complex water conservancy elements are low quality due to "spots" and "islands" noise, this invention proposes to use the morphological attribute filtering (MAFs) method to operate on the initial segmentation results of SAM to obtain high-quality and high-precision samples.

[0039] (3) The experiments conducted in the embodiments of the present invention show that, compared with the currently open-source semantic sample annotation tools (such as labelme), the present invention introduces the SAM general segmentation large model for the annotation process, which greatly improves the annotation efficiency. The overall test level is about 2-3s / image recognition efficiency under the configuration of the test computer (CPU / THR / MEM:24C / 32T / 192G, GPU:GeForce4080-16G, CUDA:12.4). In addition, the morphological attribute filtering (MAFs) method proposed in the present invention can better balance the preservation of semantic sample details and the "denoising" task, thereby improving the overall sample quality and annotation accuracy. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings involved in the embodiments are now briefly described. Obviously, the drawings in the following description are merely illustrative of some embodiments of the present invention. For those skilled in the art, other forms of drawings can be constructed based on these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram of the labelme segmentation sample annotation process mentioned in the background art of this invention;

[0042] Figure 2 This is a technical roadmap for the semi-automatic annotation method for water conservancy elements based on morphological attribute filtering (MAFs) and the SAM large model mentioned in Embodiment 1 of the present invention;

[0043] Figure 3 This is a schematic diagram of the interactive prompt sparse point input mentioned in Embodiment 1 of the present invention;

[0044] Figure 4 This is a schematic diagram of the SAM segmentation process mentioned in Embodiment 1 of the present invention;

[0045] Figure 5 This is the initial SAM segmentation result diagram mentioned in Embodiment 2 of the present invention;

[0046] Figure 6 This is a comparison diagram of the morphological attribute filtering (MAFs) results and the initial SAM segmentation results mentioned in Embodiment 2 of the present invention;

[0047] Figure 7This is a diagram illustrating the combined SAM+MAFs results mentioned in Embodiment 2 of the present invention;

[0048] Figure 8 This is a schematic diagram of the morphological attribute filtering (MAFs) process mentioned in Embodiment 2 of the present invention. Detailed Implementation

[0049] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0050] This invention proposes a semi-automatic annotation method for water conservancy elements based on morphological attribute filtering (MAFs) and SAM large model. The method uses morphological methods to perform morphological attribute filtering (MAFs) on the mask file to filter out noise problems such as "islands" and "burrs" in the obtained initial segmentation mask, resulting in a cleaner and more accurate mask file (target mask image).

[0051] The development of AI sample semi-automatic annotation software based on the SAM general segmentation model mainly serves the following purposes:

[0052] First, it significantly increases the time and manpower costs of labeling training data for self-developed AI products, while greatly shortening the time for preparing data samples.

[0053] Secondly, it allows R&D personnel to focus more on optimizing and transferring the network structure of algorithm models.

[0054] Third, we will gradually iterate to form a huge target dataset within the industry, providing data support for further training of large industry models.

[0055] The semi-automatic annotation method for water conservancy elements based on morphological attribute filtering (MAFs) and SAM large model proposed in this invention will be described below with reference to relevant figures and specific examples.

[0056] Example 1:

[0057] This invention proposes a semi-automatic annotation method for hydraulic elements based on morphological attribute filtering (MAFs) and a large SAM model. The technical approach is as follows: Figure 2As shown, the main components include data augmentation, interactive input prompts, SAM segmentation, calculation of connected components in the water conservancy element segmentation results, and morphological attribute rule filtering. To increase the richness of sample types, the UAV water conservancy element images are first expanded by flipping, mirroring, cropping, and rotating. Then, the water conservancy element prompt point information (pixel coordinates) is input into the SAM Large segmentation engine through interactive operation to generate initial segmentation results. When faced with complex water conservancy elements, weeds, color, and other variations, the initial SAM segmentation results cannot completely distinguish between water conservancy elements and non-water conservancy elements. Therefore, it is proposed to use morphological calculation of each connected component of the binary image and filter out "noise points" of non-water conservancy elements based on attributes such as the area and diagonal length of water conservancy elements. Finally, high-precision and high-accuracy water conservancy element sample images are obtained for subsequent water conservancy element semantic segmentation training. The specific steps include:

[0058] (1) Data Augmentation

[0059] Commonly used geometric transformation methods include: flipping, rotating, cropping, scaling, translating, and dithering; their main functions are as follows:

[0060] 1) Avoid overfitting. When the dataset has certain obvious features, such as the images in the dataset being taken in the same scene, methods such as Cutout and style transfer can be used to prevent the model from learning information that is irrelevant to the target.

[0061] 2) Improve model robustness and reduce model sensitivity to images. When the training data is in a relatively ideal state, it is easy to make mistakes when encountering some special situations, such as occlusion, brightness, blur, etc. Adding noise or masking to the training data can improve the robustness of the model.

[0062] 3) Increase training data to improve the model's generalization ability.

[0063] 4) Avoid sample imbalance. In cases of complex ground features, extreme imbalance between positive and negative samples can easily occur. By applying data augmentation techniques to the smaller sample size, the proportion of sample imbalance can be reduced.

[0064] From the perspective of sample richness, the purpose of this invention to carry out sample augmentation work on water conservancy targets in UAV images is mainly to test the mean generalization ability of the SAM large model in water conservancy element target segmentation.

[0065] (2) Interactive input prompts

[0066] SAM provides two sets of cue information: sparse (points, boxes, text) and dense (masks). Sparse cues represent points and boxes using positional encoding, plus embeddings for each cue type and free-form text from an off-the-shelf text encoder from CLIP. Dense cues (i.e., masks) use convolutional embeddings and are element-wise summed with the image embeddings. This invention primarily uses manual, rapid comparison with hydraulic targets to select sparse points as SAM cue input; please refer to [link to relevant documentation]. Figure 3 , Figure 3 The process of manually selecting points of interest using the software system integrated in this invention is demonstrated.

[0067] (3) Initial division of water conservancy elements

[0068] In April 2023, Meta released a new image segmentation model, SAM (Segment Anything Model), which can generate high-quality object masks from input cues (such as points or bounding boxes) and can be used to generate masks for all objects in an image. It has been trained on a dataset containing 11 million images and 11 billion masks, and can perform zero-shot transfer to new image distributions and tasks. Its segmentation results are impressive, making it the current state-of-the-art (SOTA) algorithm for segmentation. See also... Figure 4 SAM borrows the Prompt approach from NLP tasks, providing a prompt to the image segmentation task to achieve fast segmentation of any target.

[0069] The cues can be foreground / background point sets, rough bounding boxes or masks, text of any form, or any information indicating that the image needs to be segmented. The input to this task is the original image and some cues; the output is a mask of the target in the image. Its network structure mainly consists of three parts: an image encoder, a cue word encoder, and a mask decoder, specifically including:

[0070] 1) Image encoder

[0071] The image encoder in SAM uses a Vision Transformer (ViT) pre-trained with MAE (Visual Self-Attention) to adapt to high-resolution input. Specifically, the image encoder supports vit-h, vit-l, and vit-b. The vit encoding is pre-trained using MAE, a type of self-supervised learning in image processing.

[0072] 2) Prompt encoder

[0073] Sparse cues (dots, boxes) are embedded using positional encoding and then added to the learned embedding of the corresponding cue type. Text cues are embedded using CLIP's text encoder. Dense cues (masks) are embedded using convolutional embedding and then added to the image embedding.

[0074] 3) Mask Decoder

[0075] A modified version of the Transformer decoder is used. The decoder block updates all embeddings in two directions (cue-to-image embedding, image-to-cue embedding) using cue self-attention and cross-attention. After two decoder blocks, the image embeddings are upsampled, and the MLP maps the output to a dynamic linear classifier, which then computes the masked foreground probability for each image location.

[0076] For a given output, if the hint is ambiguous, the model will average multiple valid masks. To address the ambiguity of the segmentation results, multiple output masks are predicted for a single hint. Specifically, three output masks (overall, partial, and sub-part) are set for each hint, and the model simultaneously predicts the confidence score (estimated IoU) for each mask. During training, only the minimum loss of the predicted masks is calculated, using a linear combination of Focal loss and Dice loss as the loss function.

[0077] (4) Morphological attribute calculation and analysis

[0078] Morphological attribute calculation is based on the extraction of connected components and the calculation of morphological attributes in morphological binary images. Its core is the connected component labeling algorithm. This algorithm essentially scans each pixel in the binary image, grouping pixels with the same value and those that are interconnected into the same group, ultimately obtaining all connected components in the image. Then, through morphological analysis of the connected components, such as area, diagonal length, moment of inertia, and second moment, the characteristics of the connected component are comprehensively described, and attribute combinations that meet specific objectives of hydraulic elements are identified for differentiation.

[0079] (5) Morphological Attribute Filters (MAFs)

[0080] Morphological attribute filters (MAFs) are built upon classic morphological filtering algorithms (Alternating Sequential Filters, ASFs). They transform the fixed structural elements of attribute morphology into flexible attribute structural elements that can be expressed by equations and numbers, thus overcoming the shortcomings of fixed-size structural elements and adaptive thresholding methods in filtering blurred images. Similar to morphological reconstruction of connected components, MAFs do not distort the shape, structure, or edge details of the image target. Instead, they utilize predefined attributes inherent in the target, primarily involving two processes: connected component calculation and attribute logic judgment, to formulate filtering rules based on the target's characteristics.

[0081] Example 2:

[0082] Based on Example 1 but with a difference, the semi-automatic annotation method for water conservancy elements based on morphological attribute filtering (MAFs) and SAM large model proposed in this invention is divided into four parts, namely (1) sample enhancement; (2) initial segmentation of water conservancy elements; (3) morphological attribute calculation and analysis; (4) general segmentation (SAM) + morphological attribute filtering (MAFs), specifically including the following contents.

[0083] Step 1: Sample Enhancement

[0084] The data augmentation operations in this invention mainly utilize torch.Transforms, and include the following categories:

[0085] Flip: This function allows you to flip images horizontally and vertically. Some frames do not provide a vertical flip function. However, a vertically flipped image is equivalent to rotating the image 180 degrees before performing a horizontal flip. Below is an example of image flipping based on this invention.

[0086] Rotation: In order to ensure that the aspect ratio of the image remains unchanged, this invention mainly performs 180-degree rotation operation on UAV water images.

[0087] Center cropping: This invention performs center cropping on UAV water images, with a cropping range of (width:2000, height:2000) and a cropping ratio of 0.5.

[0088] Step 2: Initial Segmentation of Water Conservancy Elements

[0089] To verify the high efficiency and accuracy of the proposed method in water feature sample annotation, this invention patent used PyCharm 2024 as the experimental development platform and conducted experiments using over 12,000 UAV river patrol images. The UAV river patrol images primarily consisted of high-resolution images of 3000*4000px size with a bit depth of 24bit; additionally, images of complex water features such as different angles, lighting conditions, resolutions, colors, and water surface reflections were added as mixed samples to test the homogeneous generalization ability of the SAM (Search Engine Analysis). The initial SAM segmentation mask results and the mask-overlay results are shown below. Figure 5 As shown.

[0090] Figure 5 The image consists of four sets of images (a)-(d). Each set, from top to bottom, comprises three rows representing the original UAV image, the initial SAM segmentation mask image, and the overlay image of the mask and the original image, respectively. Figure 5 It can be seen that the SAM large model can show good homogeneous segmentation ability for various complex hydraulic factors, but it may exhibit "island" noise problems for some complex underlying surfaces, such as... Figure 5 (c) and Figure 5 The elliptical box in (d) is shown.

[0091] Step 3: Morphological attribute calculation and analysis;

[0092] (1) Calculation of morphological attributes

[0093] Since morphological attribute calculation is performed on the connected components of urban river network water features, accurately and completely extracting these connected components from the initial SAM segmentation results is a crucial step in subsequent attribute calculation and filtering. Morphological connected component extraction algorithms, based on image set theory, utilize a combination of morphological dilation and foreground image intersection for labeling. They are widely used in image segmentation and object detection due to their simplicity and fast connected component search. This paper utilizes morphological connected component extraction algorithms for connected component extraction. Attribute calculation includes connected component opening and closing transformations and attribute transformations.

[0094] Suppose we have an initially segmented binary image I, and a connected component c on I, and a connected set X = {c | c ∈ X}. Then the open transformation of the connected component c is defined as:

[0095]

[0096] By adding attribute constraints T(X) to the open transformation of the connected components, the attribute transformation of the connected components is obtained:

[0097]

[0098] In the formula, λ is the set attribute threshold, which is determined based on the minimum water conservancy element attribute value of the UAV image. The minimum water conservancy element values ​​involved in this invention are obtained through repeated calculations and statistics on different attribute dimensions. T(X) is the attribute definition of the connected component. Combining equations (1) and (2), the binary image attribute transformation expression is obtained:

[0099]

[0100] (2) Attribute criteria judgment

[0101] Considering that water conservancy elements possess certain attribute characteristics in high-resolution UAV imagery, such as area (A), diagonal length of connected components (D), and moment of inertia (W), a multi-dimensional water conservancy element attribute feature set is constructed based on the initial segmentation of water conservancy elements using the SAM model, incorporating the aforementioned three attribute characteristics. The specific attribute filtering criteria are as follows:

[0102] a. Area attribute criterion: The area attribute of connected components reflects the overall land area characteristics of water conservancy elements. The pixel area A(c) of each connected component is obtained through pixel recursion and boundary chain code. Area filtering criterion Ψ A Then it is defined as:

[0103] When the area attribute value A(c(i)) of connected component i is greater than the attribute threshold value A λ When the condition is met, the connected component will be preserved; otherwise, it will be filtered. This can be described as:

[0104]

[0105] b. Diagonal length attribute criterion: The diagonal length D of a connected component reflects the length-width structure and overall extensibility of the river channel. The larger the D value, the larger its length and width, and the better its overall extensibility. In this case, the probability that the connected component belongs to the characteristics of hydraulic elements is greater. Conversely, the smaller the D value, the greater the probability.

[0106] c. Rotational inertia property criterion: The rotational inertia W of a connected component reflects the invariance of its second moment. A larger W value indicates that the centroid of the connected component is farther from its two endpoints, suggesting better overall extensibility. The latter two property filtering criteria are the same as Ψ. A Consistent with the above attribute conditions, connected components that meet the above attribute conditions are retained as foreground, while connected components that do not meet the attribute criteria are masked as background.

[0107] Step 4: General Segmentation (SAM) Large Model + Morphological Attribute Filtering (MAFs)

[0108] (1) Computer threshold calibration of morphological attributes

[0109] according to Figure 5Compared to reality, the initial SAM segmentation results for water conservancy elements exhibit "spot" and "island" noise in complex situations with unclear segmentation, affecting the accuracy of subsequent network training. Morphological Attribute Filtering (MAFs) is used for analysis. First, attributes are calculated for all connected components of the entire labeled image, including area, diagonal length, and moment of inertia. Multi-attribute feature values ​​of the connected components are obtained. The attribute threshold λ is obtained through semi-empirical statistics, specifically derived from the statistical analysis of all water conservancy element samples in the experimental area. The area, diagonal, and moment of inertia attributes of the UAV river network involved in this invention are [2000, 200, 0.46]. To avoid directional selection differences, an 8-neighborhood search rule is used to track the structure of the connected components of the water conservancy elements. Then, these multi-attribute feature sets are logically judged according to filtering criteria. Connected components that do not meet the water conservancy element feature attributes are masked by surrounding background pixels, while those that do are retained, thus completing the attribute filtering of the initial SAM segmentation results. Figure 6 The results of attribute filtering and the initial SAM segmentation are presented.

[0110] Figure 6 In the image matrix, the first row is the original image, the second row is the initial SAM segmentation result, the third row is a magnified image of the local noise in the initial SAM segmentation result, the fourth row is a labeled image of the extracted connected components in terms of area, diagonal length, and moment of inertia, the fifth row is the result image after morphological attribute filtering (MAFs), and the sixth row is the result image after attribute filtering. Figure 6 As can be seen, the mask after morphological attribute filtering (MAFs) significantly reduces the influence of "spots" and "islands" noise, effectively optimizing the initial SAM segmentation results and improving the accuracy and quality of the mask as a subsequent semantic recognition sample.

[0111] (2) Accuracy Evaluation

[0112] In order to accurately evaluate the impact of the above three annotation methods on the annotation accuracy of water conservancy element samples, this invention adopts a wide range of evaluation indicators.

[0113] Table 1. Accuracy Evaluation of Recognition Results

[0114] Note: The bolded data in the table represents the optimal value for that item.

[0115] As can be seen from Table 1, although the manual outlining annotation method has an advantage in recall, it is inferior in accuracy and false alarm rate, and the annotation efficiency is low. In addition, manual annotation cannot be carried out continuously and there is the problem of fatigue error. Secondly, using only the initial segmentation results of SAM as the annotation sample results will result in a loss of accuracy and an increase in false alarm rate due to the interference of "noise" from complex hydraulic elements. However, the efficiency is significantly improved compared to manual annotation.

[0116] Further reference Figures 7-8 , Figure 7 The output image is a combination of SAM and MAFs. The user can interactively input the pixel where the target is located and automatically submit the input of the SAM segmentation engine and the morphological attribute filtering (MAFs) engine. Figure 8 This is a schematic diagram of the Morphological Attribute Filtering (MAFs) process. Experiments conducted on over 12,000 UAV river and lake images demonstrate that, compared to currently available open-source semantic sample annotation tools (such as Labelme), the introduction of the SAM general segmentation model in this invention significantly improves annotation efficiency. The overall performance, under local CPU / THR / MEM: 24C / 32T / 192G and GPU_0: GeForce4080-16G configuration, is approximately 2-3 seconds per image. Furthermore, the proposed Morphological Attribute Filtering (MAFs) method effectively balances semantic sample detail preservation with denoising, improving overall sample quality and annotation accuracy.

[0117] Compared to the previous two methods, this invention, based on the initial SAM segmentation, utilizes the connectivity component attribute features of water conservancy elements for further screening. By using attribute features, it further eliminates the influence of noise such as "spots" and "islands" of non-water conservancy elements. While ensuring annotation efficiency, it further improves the accuracy of semi-automatic annotation of samples and reduces the false alarm rate, resulting in higher quality and more accurate annotated samples.

[0118] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A semi-automatic annotation method for hydraulic elements based on MAFs and SAM large models, characterized in that, Includes the following steps: S1. Sample Data Augmentation: Perform geometric transformations on the original UAV water conservancy element images, including one or more of flipping, mirroring, cropping, and rotating, to increase the amount of sample data; S2. Interactive Input Prompts: Select water conservancy element targets through manual interaction in the image interface, generate sparse point prompts, and record the coordinates of other pixels; when facing complex underlying surfaces, only the coordinates of positive and negative prompt points need to be entered; S3, SAM Large Model Initial Segmentation: Input the sparse point hints generated in S2 into the SAM Large segmentation engine to generate the initial segmentation binary mask map of the hydraulic elements. S4. Morphological Attribute Calculation and Analysis: Connected component labeling is performed on the initial segmentation binary mask obtained in S3, and the morphological attributes of each connected component are extracted; the morphological attributes include one or more of the following: area, diagonal length, moment of inertia, and second moment. S5. Morphological attribute filtering: Based on the set characteristics of water conservancy elements, attribute threshold rules are set to filter out connected components that do not conform to the rules and generate optimized water conservancy element annotation results. S6. Output labeled samples: Use the optimized labeled results from S5 as training samples for training the semantic segmentation model of water conservancy elements.

2. The semi-automatic annotation method for hydraulic elements based on MAFs and SAM large models according to claim 1, characterized in that, The SAM large model's cue information includes sparse cue and dense cue. The sparse point cue information mentioned in S2 is generated as follows: The foreground point of the target area of ​​the water conservancy element can be selected by clicking on the image interface through manual interaction; The prompt information is embedded in the SAM prompt encoder in the form of location encoding.

3. The semi-automatic annotation method for hydraulic elements based on MAFs and SAM large models according to claim 2, characterized in that, The SAM Large segmentation engine described in S3 includes an image encoder, a cue word encoder, and a mask encoder; The image encoder is used to encode the image. The ViT pre-trained by MAE is used as the image encoder to adapt to high-resolution input. The cue word encoder is used to encode cue information, where sparse cues are represented by points and boxes through positional encoding, plus embeddings for each cue type and free-form text from the CLIP off-the-shelf text encoder; dense cues use convolutional embeddings and are element-wise summed with image embeddings. The mask decoder block updates all embeddings using cue-based self-attention and cross-attention; after two decoders, the image embeddings are upsampled, the MLP maps the output to a linear classifier, and then the dynamic linear classifier computes the mask foreground probability for each image location.

4. The semi-automatic annotation method for hydraulic elements based on MAFs and SAM large models according to claim 3, characterized in that, The connected component labeling described in S4 is implemented using a connection component labeling algorithm, specifically including the following: Scan each pixel in the binary image, group pixels with the same value that are connected to each other into the same group, and obtain all connected components of the pixels in the image; A morphological analysis is performed on the area, diagonal length, moment of inertia, and second moment of the connected component to comprehensively describe its characteristics and identify attribute combinations that meet specific objectives of hydraulic elements for differentiation.

5. The semi-automatic annotation method for hydraulic elements based on MAFs and SAM large models according to claim 4, characterized in that, S5 specifically includes the following: Based on morphological filtering algorithms, the fixed structuring elements of attribute morphology are transformed into flexible attribute structuring elements that can be expressed by equations and numbers, so as to make up for the shortcomings of fixed-size structuring elements and the insufficient adaptability of adaptive threshold segmentation methods to blurry image filtering; based on two processes of connected component calculation and attribute logic judgment, attribute filtering is defined by predefined attributes of the target itself, and filtering rules are formulated according to the characteristics of the target.

6. The semi-automatic annotation method for hydraulic elements based on MAFs and SAM large models according to claim 5, characterized in that, The attribute threshold rules described in S5 specifically include the following: The area attribute is calculated using pixel recursion and boundary chain code method; The diagonal length attribute is calculated using the diagonal length of the bounding rectangle of the connected components; The moment of inertia property is calculated using the invariance of the second moment of the connected components.

7. A computer device, characterized in that, The computer device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set, or instruction set, and the instruction, program, code set, or instruction set is loaded and executed by the processor to implement the semi-automatic annotation method for hydraulic elements based on MAFs and SAM large models as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, at least one program, code set, or instruction set, which is loaded and executed by a processor to implement the semi-automatic annotation method for hydraulic elements based on MAFs and SAM large models as described in any one of claims 1-6.

Images