A flotation foam image segmentation method based on SAM prediction mask concatenation post-processing and morphological gradient watershed synergy.
By combining SAM prediction mask concatenation post-processing with morphological gradient watershed, the problems of nested redundancy, adhesion splitting, and inaccurate boundaries in flotation bubble image segmentation are solved, achieving high-precision and robust bubble segmentation results.
Patent Information
- Application Number
- CN202611483477.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-22
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies for flotation foam image segmentation suffer from problems such as unstable label extraction, difficulty in splitting adhered bubbles, redundant nested SAM prediction masks, and inaccurate boundaries. They cannot simultaneously solve the problems of nested redundancy, adhesion and splitting, preservation of small bubbles, and rounded boundaries.
A method based on SAM prediction mask concatenated post-processing and morphological gradient watershed is adopted. By combining filtering to enhance image contrast, a gradient topographic map is constructed. SAM prediction mask is used and concatenated post-processing is performed, including dynamic topology anti-nesting and adaptive elliptic erosion. The immersion watershed algorithm is then used for segmentation.
It effectively eliminates nested redundant masks, accurately splits adhering bubbles, retains small bubbles, and achieves smooth and continuous bubble boundaries and high-precision segmentation, thereby improving the robustness and accuracy of segmentation.
Smart Images

Figure CN123118673A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of mineral processing technology, specifically to a flotation foam image segmentation method based on SAM prediction mask concatenation post-processing and morphological gradient watershed collaboration. Background Technology
[0002] Flotation is one of the most important separation methods in mineral processing, and its performance directly determines the recovery rate of the target mineral and the grade of the concentrate. During flotation, the state of the froth layer (including froth size, color, quantity, formation rate, and disappearance rate) is closely related to flotation parameters. Traditionally, these parameters relied on years of experience from on-site technicians for judgment. With the advancement of intelligent mineral processing strategies and the continuous reduction in the number of on-site technicians, the automatic analysis of flotation froth images using machine vision technology to achieve intelligent monitoring and parameter optimization of the flotation process has become an inevitable trend in the industry.
[0003] Fine segmentation of flotation foam images is a prerequisite for subsequent foam feature extraction (such as size distribution, morphological parameters, and color features) and flotation state identification. However, flotation foam images have unique visual characteristics: a large number of bubbles, a wide range of particle size distribution (from tens of pixels to tens of thousands of pixels), dense adhesion between bubbles, varying shades of bubble wall shadows, and high-brightness reflections on the bubble surface. These characteristics place extremely high technical demands on image segmentation algorithms.
[0004] In the field of flotation froth image segmentation, the adaptive marker watershed algorithm is the most widely used traditional method. Mehrshad et al. disclosed an adaptive marker watershed transformation method, the technical solution of which is: to extract watershed markers from flotation froth images using an adaptive threshold method, to smooth small intensity changes in flat areas using an adaptive median filter, to generate markers using morphological operations, and finally to achieve segmentation through the watershed algorithm ([2] Mehrshad N, Massinaei M. New image-processing algorithm for measurement of bubble size distribution from flotation froth images [J]. Minerals&Metallurgical Processing, 2011, 28(3): 146-150.). Jahedsaravani et al. made improvements on this basis. Their technical solution is as follows: Based on the adaptive thresholding method, two sets of labeled images with soft threshold and hard threshold are extracted by morphological operation. After fusing multiple sets of labels, the watershed algorithm is applied to deal with the scene of mixed large and small bubbles ([3] Jahedsaravani A, Marhaban MH, Massinaei M, et al. Development of a new algorithm for segmentation offlotation froth images [J]. Minerals and Metallurgical Processing, 2014, 31(1): 66-72.). Zhang et al. proposed a watershed segmentation algorithm based on an optimal marker. The technical solution is as follows: the sub-image category is determined by the sub-image classification model (SVM), the three sets of marker regions, sub-image category information and bubble light reflection features are fused to generate the optimal marker, and then the marker-based watershed segmentation is used ([4] Zhang H, Tang Z, Xie Y, et al. A watershed segmentation algorithm based on an optimal marker for bubblesize measurement [J]. Measurement, 2019, 138: 182-193.).Yang et al. proposed an improved watershed algorithm based on adaptive label extraction. The technical solution is as follows: after Gaussian filtering the flotation froth image, the label is adaptively extracted using the morphology-based extended maximum technique. The label is used to modify the gradient image, and then the watershed algorithm is used to segment the modified gradient image ([5] Yang JM, Yang D D. An flotation frothimage segmentation method utilizing improved watershed algorithm [J]. CoalPreparation Technology, 2012, 40(5): 82-86.). Liang et al. proposed a foam image segmentation method based on highlight correction and parameter adaptation. The technical solution is as follows: homomorphic filtering is used to improve the uneven brightness and shadow of the image, fuzzy C-means clustering is used to classify similar highlights, after Otsu threshold segmentation, parameter adaptive morphological operation is used to extract the marker points and edge bands, and the highlight markers are used as the local minimum points of the watershed for segmentation ([6] Liang XM, TianT, Liu WT, et al. Flotation froth image segmentation based on highlightcorrection and parameter adaptation [J]. Mining, Metallurgy&Exploration,2020, 37(2): 467-474.). In addition, Central South University disclosed a key feature extraction method for flotation foam image analysis. The technical solution is as follows: construct a flotation foam image acquisition platform, combine morphological operation and watershed method to segment the foam image and extract size features ([7] Central South University. Key feature extraction method for flotation foam image analysis [P]. CN101334844A. 2008-12-31.). CITIC Heavy Industries Machinery Co., Ltd. disclosed a processing method for foam adhesion image region segmentation. The technical solution is as follows: extract the feature information such as bright spots and foam edges of the foam image, and use them as the target starting point and boundary restriction point for image segmentation, respectively. The watershed algorithm is used for segmentation ([8] CITIC Heavy Industries Machinery Co., Ltd. A processing method for foam adhesion image region segmentation [P]. CN111524143A.2020-08-11.).
[0005] The aforementioned existing technologies based on adaptive labeling watersheds suffer from the following common drawbacks. First, label extraction is highly dependent on the image's inherent grayscale features (such as thresholds, morphological maxima, and specular highlights). In complex industrial environments, factors like varying illumination, surface reflections on foam, and uneven shadows lead to unstable grayscale features, resulting in poor reliability of label extraction and a high likelihood of missing or redundant labels, ultimately causing under-segmentation or over-segmentation in the watershed algorithm. Second, none of the above methods utilize the predictive capabilities of deep learning models. The label extraction process is essentially based on manually designed image features and manually adjusted threshold parameters, exhibiting poor adaptability and insufficient robustness to different types of ores and flotation conditions. Third, for the problem of splitting large-area adhered bubbles, the above methods lack an effective adhesion-removal mechanism. When multiple bubbles are tightly adhered, the grayscale-feature-based labeling fails to generate a sufficient number of independent labels within the adhered area, leading to misclassification of adhered bubbles as single bubbles and a significant decrease in segmentation accuracy.
[0006] Wu et al. proposed a watershed color image segmentation algorithm based on morphological gradient. The technical solution is as follows: by obtaining the component gradient of the color image in the new color space, the gradient image is reconstructed by opening and closing, the threshold is automatically obtained by the maximum inter-class variance algorithm, the original gradient image is forcibly calibrated by the obtained binary labeled image, and then the watershed algorithm is used to segment the corrected gradient image ([9] Wu YY, Li Q. The algorithm of watershed colorimage segmentation based on morphological gradient [J]. Sensors, 2022, 22(21): 8202.). Although this technical solution introduces the idea of constructing gradient topographic map by morphological gradient, its label source is still the binary labeled image based on threshold segmentation, and the label extraction method has not deviated from the traditional image processing framework. In flotation foam images, the grayscale changes are drastic in the deepest shadows of the bubble walls, while the grayscale changes are weak in the flat areas inside the bubbles. Although morphological gradient operations can delineate the bubble edges, if the marker positions are inaccurate (e.g., the markers are off-center from the bubble or the number of markers is insufficient), watershed evolution will still establish watershed lines at incorrect locations, causing the segmentation boundary to deviate from the actual bubble walls. Furthermore, this technical solution is designed for general color images and does not consider the special scenarios of densely packed and widely spaced bubbles in flotation foam images. Directly transferring this solution to flotation foam segmentation makes it difficult to guarantee the segmentation effect.
[0007] In recent years, visual basic models represented by SAM (Segment Anything Model) have provided a new technical path for image segmentation. Wang et al. applied SAM to the field of flotation foam image segmentation. Their technical solution is as follows: using the zero-shot generalization ability of SAM, no additional training or fine-tuning is required for flotation foam images, a fully automatic point prompting segmentation scheme is developed, and a mask filtering module and a compensation segmentation module are introduced to reduce missegmentation ([1] Wang Jinglong, Jia Runda, Zheng Jun, et al. Segment anything model-based method for precise froth size determination in flotation process [J]. Chemical Engineering Science, 2025,301: 121504.). This technical solution makes full use of the zero-shot prediction ability of SAM, and to a certain extent overcomes the defect of traditional methods that rely on manual adjustment of thresholds.
[0008] However, Wang et al.'s technical solution has the following deep-seated flaws. First, their approach is end-to-end SAM segmentation, meaning it relies entirely on the prediction output of the SAM model as the final segmentation result, without deeply coupling the SAM prediction results with traditional image processing methods. In densely clustered flotation foam scenarios, although SAM can predict the entire size of the foam region, the blurred boundaries and smooth grayscale transitions between foams cause the SAM prediction mask to produce inaccurate boundaries in the clustered areas, leading to deviations in the boundary positioning of clustered bubbles. Second, SAM inherently exhibits nested redundancy when generating multi-scale masks (i.e., a large mask contains multiple smaller masks). Although Wang et al. introduced a mask filtering module, the specific implementation mechanism of this module was not disclosed, and judging from its implementation effect, the suppression of nested redundant masks is not thorough, and the problem of the same bubble being repeatedly segmented still exists. Third, SAM end-to-end segmentation lacks an explicit constraint mechanism on physical boundaries. The boundary shape of the predicted mask is determined by the internal decision of the model. It is difficult to ensure a smooth break at the deepest part of the bubble wall shadow (i.e. the real physical boundary position), resulting in insufficient continuity and smoothness of the segmentation boundary.
[0009] In the field of medical image segmentation, there have been attempts to combine SAM with traditional image processing. Jiangxi Yizhichu Medical Pathology Diagnosis Management Co., Ltd. disclosed a method for segmenting cervical liquid-based thin-layer cell images based on the SAM segmentation model. The technical solution is as follows: a brightness histogram is constructed by sampling to obtain cell nucleus seed points. The seed points and surrounding auxiliary points are used as prompt points to input the SAM model for mask segmentation. The maximum contour is obtained to remove holes and noise and optimize the mask area (
[10] Jiangxi Yizhichu Medical Pathology Diagnosis Management Co., Ltd. Method for segmenting cervical liquid-based thin-layer cell images based on the SAM segmentation model [P]. CN117876402A. 2024-04-12.). Although this technical solution combines SAM with traditional seed point extraction, its application field is cervical cell images, which are fundamentally different from floating foam images in terms of visual features (clear boundary between cell nucleus and cytoplasm vs. blurred bubble wall shadow), segmentation targets (isolated cells vs. densely adhered bubbles), and technical requirements (cell classification vs. foam size statistics). More importantly, the input of SAM in this technical solution relies on manually designed seed point prompts, and the acquisition of seed points still requires the use of traditional image processing methods (brightness histogram analysis). In the scenario of densely adhered and blurred boundaries of flotation foam, the reliable extraction of seed points is itself a problem. Therefore, this technical solution cannot be directly transferred to the field of flotation foam segmentation.
[0010] Lu Caiwu et al. proposed a flotation foam image segmentation method based on the SoftEdge soft edge detection model and the improved watershed. The technical solution is as follows: the SoftEdge soft edge detection model is used to extract the edge features of the flotation foam image, and the improved watershed algorithm is used for segmentation (
[13] Lu Caiwu, Cao Yue, Liu Di, et al. Research on flotation foam image segmentation method based on SoftEdge soft edge detection model and improved watershed [J]. Metal Mine, 2025(8): 158-164.). This technical solution combines deep learning with the traditional watershed, which improves the accuracy of edge detection to a certain extent. However, this technical solution has the following defects: First, SoftEdge is an edge detection model rather than a segmentation model. Its output is an edge probability map rather than a region mask. The edge probability map needs to be further converted into watershed labels to drive the watershed evolution. Inappropriate threshold selection during the conversion process can easily lead to label breakage or label redundancy. Second, the edge detection model has a weak response to the edges of extremely small bubbles. The edge features of small bubbles are easily submerged by noise in the edge probability map, leading to missed detection of small bubbles. Third, this technical solution does not involve an explicit splitting mechanism for adhered bubbles. When multiple bubbles are tightly adhered, the edge features of the adhered area are blurred, making it difficult for the edge detection model to generate effective splitting boundaries within the adhered area, resulting in poor splitting performance for large-area adhered bubbles.
[0011] Sadr-Kazemi et al. studied the watershed method based on morphology, using the maximum region of the image as the seed point of the algorithm, and preprocessed the flotation foam image using morphological reconstruction technology (
[11] Sadr-Kazemi N, Cilliers J J. An image processing algorithm for measurement of flotation froth bubble size and shape distributions [J]. Minerals Engineering, 1997, 10(10): 1075-1083.). Lin et al. applied the erosion and dilation algorithm in mathematical morphology to the flotation foam image of coal slime, which solved the oversegmentation and undersegmentation problems of the classic watershed algorithm, and made the adhering coal slime bubbles effectively segmented (
[12] Lin XZ, Gu YY, Zhao G Q. Feature extraction based on imagesegmentation of coal flotation froth [J]. Journal of China Coal Society, 2007, 32(3): 304-308.). While the aforementioned technical solution combines morphological corrosion-expansion with watershed techniques, its corrosion-expansion operation uses fixed-scale structural elements and does not adaptively adjust according to the bubble area. In flotation froth images, the bubble size distribution spans a wide range, and the fixed-scale corrosion operation presents a dilemma: if the corrosion nucleus is too large, tiny bubbles are over-corroded and disappear, leading to missed detection of small bubbles; if the corrosion nucleus is too small, the adhered areas of large-area bubbles cannot be effectively separated, resulting in under-segmentation of adhered bubbles. Therefore, the fixed-scale corrosion-expansion strategy cannot simultaneously achieve strong fragmentation of large-area bubbles and the backup preservation of tiny-area bubbles.
[0012] In summary, existing technologies in the field of flotation foam image segmentation have the following unresolved technical problems. First, traditional adaptive labeling watershed methods rely on image grayscale features and manually adjusted thresholds for label extraction, resulting in poor robustness in complex industrial environments and an inability to effectively handle the splitting of large-area adhered bubbles. Second, while the SAM end-to-end segmentation method possesses zero-shot prediction capabilities, it is prone to nested redundant masks and inaccurate boundaries in densely adhered foam scenarios, and lacks explicit constraints on physical boundaries. Third, existing deep learning and traditional watershed combination schemes (such as SoftEdge+watershed) use edge detection models rather than segmentation models, making the edge-to-label conversion process unreliable and lacking an explicit splitting mechanism for adhered bubbles. Fourth, existing morphological erosion-dilation-based segmentation schemes employ fixed-scale structuring elements, failing to balance the strong splitting of large-area bubbles with the backup survival of extremely small-area bubbles. Therefore, there is an urgent need in this field for a flotation foam image segmentation method that can fully utilize the zero-sample prediction capability of SAM, while simultaneously optimizing the SAM prediction results through traditional image processing methods, and applying an adaptive corrosion strategy to bubbles of different areas, in order to solve the comprehensive technical problems of nested redundancy, adhesion and splitting, small bubble preservation and smooth boundary breakage that cannot be overcome simultaneously by the above-mentioned existing technologies. Summary of the Invention
[0013] The purpose of this invention is to provide a flotation foam image segmentation method based on SAM prediction mask concatenation post-processing and morphological gradient watershed collaboration, in order to solve the following technical problems existing in the prior art: the problem of SAM generating nested redundant masks in densely adhered foam scenes; the problem of traditional watershed algorithm label extraction relying on manual threshold adjustment, over-segmentation or under-segmentation; the problem that fixed-scale erosion kernel cannot simultaneously take into account the splitting of large-area bubbles and the preservation of extremely small-area bubbles; and the problem of non-rounded and inaccurate segmentation boundaries.
[0014] To achieve the above objectives, the present invention adopts the following technical solution.
[0015] A method for segmenting flotation foam images based on SAM prediction mask concatenation post-processing includes the following steps.
[0016] S1. Perform combined filtering and contrast enhancement preprocessing on the original foam image, and construct a gradient topographic map based on morphological gradient operations.
[0017] Let the input original bubble image be I. a First, it is converted into a single-channel grayscale image I, where (x, y) are the pixel coordinates currently being processed. The grayscale conversion process is represented as follows: I(x,y) = RGB2Gray(I a (x,y))(1) (1) Dual filtering and histogram equalization (CLAHE) To remove surface reflections from bubbles while preserving edges, the grayscale image is subjected to dual filtering processes: median filtering and bilateral filtering.
[0018] Median filtering: Taking a neighborhood window W(x, y) centered at pixel (x, y) and measuring (2k+1)×(2k+1), the median of the gray values of all pixels within the window is used as the filtered gray value of the center pixel. I med (x,y)=median (i,j)∈W(x,y) [I a (i,j)] In the formula, I a (i, j) represents the grayscale value of the original bubble image at pixel (i, j), I med (x, y) represents the gray value of the image at pixel (x, y) after median filtering.
[0019] Bilateral filtering: The image after median filtering is further subjected to bilateral filtering, which simultaneously weights spatial proximity and gray-level similarity. The output is:
[0020] Wherein, the normalized weight W(x, y) is:
[0021] In the formula, k is the window radius of the bilateral filter, and σ s Let σ be the standard deviation of the spatial domain. r I represents the standard deviation of the range. med (i, j) represents the gray value of the image after median filtering at the neighboring pixel (i, j), where (i, j) is the coordinate of the neighboring pixel within the current window. The first exponential term is the spatial proximity weight, and the second exponential term is the gray-level similarity weight. The product of the spatial proximity weight and the gray-level similarity weight constitutes the combined weight of the bilateral filter, thereby smoothing the reflective noise on the bubble surface while preserving the bubble edge.
[0022] Adaptive Histogram Equalization (CLAHE): The bilaterally filtered image is divided into several sub-blocks. Each sub-block's histogram is contrast-limited and cropped before equalization. The enhanced grayscale value of pixels (x, y) within sub-block (k, l) is:
[0023] In the formula, L is the number of gray levels, h k,l (m) represents the number of pixels corresponding to gray level m in the histogram of the sub-block (k, l) after contrast-limited cropping, N TLet (k, l) ∈ W(x, y) be the total number of pixels in the sub-block, and (k, l) ∈ W(x, y) be the sub-block to which pixel (x, y) belongs.
[0024] The enhanced image I obtained after the above dual filtering and histogram equalization processing enh The foam has clear edges and enhanced grayscale contrast, serving as the input image for subsequent gradient topographic mapping. Here, L is 256, and N... T W represents the total number of pixels within a single sub-block. k,l (x, y) represents the bilinear interpolation weights of the pixel relative to the (k, l)th network center. The histogram of the sub-blocks after being clipped by a clipping threshold β and evenly distributed is shown in the following equation:
[0025] h k,l (m) indicates the gray level of the grid. m The original histogram frequency at T clip The cropping threshold is defined by Δ, which represents the uniform distribution of excess pixels across all gray levels, and the cropping threshold is T. clip The following calculations were performed:
[0026] CLAHE divides the image into several sub-blocks, applies a clipping threshold to the histogram of each sub-block, redistributes the blocks, and then fuses the results of each sub-block using bilinear interpolation to enhance local contrast and suppress noise amplification.
[0027] Where L = 256, N T W represents the total number of pixels within a single sub-block. k,l (x, y) represents the bilinear interpolation weights of the pixel relative to the center of the (k, l)th grid. This is the histogram of the sub-blocks after being clipped by a cutting threshold β and evenly distributed.
[0028] (2) Morphological gradient topographic map.
[0029] The enhanced image is dilated and eroded, and the difference between the two is 0 in flat areas, while the grayscale changes drastically at the bubble boundaries. After subtraction, steep ridges are formed, which accurately delineate the bubble edges.
[0030] G(x,y) = (I enh ⊕K3)(x,y) - (I enh K3)(x,y) Where ⊕ represents the expansion operation. This represents the erosion operation, and K3 is a 3×3 structuring element.
[0031] Finally, the processed image is restored to a three-channel image I. F .
[0032] S2. Input the preprocessed image into the SAM model to obtain the original bubble mask set.
[0033] The enhanced three-channel image I F Input the SAM model to obtain the original bubble mask set.
[0034] M = {M1, M2, ..., M n} The SAM model, leveraging its zero-shot generalization capability, can predict full-size foam regions without requiring additional training or fine-tuning on flotation foam images.
[0035] S3. Perform concatenated post-processing on the original bubble mask set, including sorting by area from largest to smallest, calculating the coverage area ratio based on the occupied mask matrix to dynamically filter out nested masks, and then applying an adaptive elliptical erosion kernel to the remaining masks by area segment to generate marker seeds.
[0036] (1) Dynamic topology anti-nesting filtering.
[0037] Sort all masks in descending order of area, and define the occupied mask matrix O0(x,y)=0. For the k-th mask M in the sequence... k Calculate the percentage of its area that is covered by the already occupied area.
[0038]
[0039] Where S O M k Let S be an indicator function. O (M k If M > 0.5, then the mask is determined to be a nested mask and is discarded; otherwise, M is retained. k And update the occupancy map.
[0040] O k (x,y)=O k-1 (x,y)∪M k (x,y)·255 This dynamic topology anti-nesting mechanism ensures that large masks are reserved first by sorting by area, accurately determines the nesting relationship by using a coverage area ratio threshold of 0.5, and achieves dynamic space occupancy tracking by accumulating and updating the occupied mask matrix. It effectively solves the nested redundant mask problem generated by SAM in dense foam scenes.
[0041] (2) Adaptive piecewise elliptic erosion.
[0042] Piecewise adaptive elliptic erosion based on area gradient is performed on the remaining individual masks to eliminate adhesion between foams. An elliptic structuring element B is defined. d The diameter d(A) k Values are taken in segments based on area.
[0043]
[0044] The structuring element is defined as an elliptical mask.
[0045]
[0046] Where u is the number of pixels horizontally from the center, v is the number of pixels vertically from the center, and d is the diameter of the structure kernel.
[0047] Morphological erosion is performed on each mask.
[0048] E k =M k B d The adaptive segmented elliptical erosion core applies erosion at different scales according to the area. Large-area adhered bubbles are strongly peeled off into independent cores, while extremely small-area bubbles are kept alive, achieving a balance between splitting and keeping them alive.
[0049] (3) Construction of the tag matrix.
[0050] Assign an incrementally unique label to the k-th bubble that passes the screening, and construct a label matrix.
[0051]
[0052] Where i increments from 1, 2, ..., and unmarked background pixels are set to 0.
[0053] S4. Input the labeled seed and the gradient topographic map into the immersion watershed algorithm to achieve accurate segmentation of the adhering foam.
[0054] The constructed label matrix D(x,y) and gradient topographic map G(x,y) are input into the immersion watershed algorithm.
[0055] (1) Treat the modulus of the three-channel gradient map as the terrain height.
[0056]
[0057] The set of connected water bodies is defined as the water depth at a depth of h, starting from the bottom minimum point (marked by seed D(x,y) = i) where the water begins to flood.
[0058] C h(i) = {(x,y) H(x,y) ≤ h and (x,y) belongs to the connected region spreading from the seed D(x,y) = i} The total water area for all seeds at height h is [value missing].
[0059]
[0060] (2) The physical boundary line is broken.
[0061] A watershed line is established when waters from different labels i and j (i ≠ j) collide at the gradient ridgeline (where the bubble wall shadow is deepest).
[0062]
[0063] Record the boundaries in the marking matrix.
[0064]
[0065] Final segmentation result.
[0066]
[0067] By combining gradient topographic maps, SAM prediction, adaptive seeds, and watersheds, the marker seeds spread and grow along the edge gradient topographic maps under the driving force of adaptive generation. When they grow to the physical boundary, they are smoothly broken off by the watershed algorithm, which effectively improves the accuracy and robustness of flotation foam image segmentation. Attached Figure Description
[0068] Figure 1 Image of flotation foam; Figure 2 This is for using the method of the present invention to Figure 1 The image shown is the result of segmenting the flotation foam image. Detailed Implementation
[0069] The present invention will be further described below with reference to embodiments, but this is not intended to limit the present invention in any way. Any modifications or substitutions made based on the teachings of the present invention shall fall within the protection scope of the present invention.
[0070] Example
[0071] Select an image of flotation foam taken on-site, such as... Figure 1 As shown in the image, this picture contains a large number of densely packed bubbles of varying sizes. Some bubble surfaces exhibit high-reflectivity, and the bubble walls show varying shades of shadow, representing a typical complex flotation foam scene.
[0072] The flotation foam image segmentation method proposed in this invention, based on SAM prediction mask concatenation post-processing and morphological gradient watershed synergy, is used to segment the image. Figure 1 The image shown is segmented, and the specific steps are as follows.
[0073] Step 1: Image preprocessing and gradient topographic map construction.
[0074] The original bubble image was converted from RGB to grayscale to obtain a single-channel grayscale image I. Median filtering and bilateral filtering were then applied to the grayscale image. A 3×3 window was used for median filtering to remove impulse noise generated by specular reflection from the bubble surface. A 5-radius k-radius was used for bilateral filtering, with a spatial domain standard deviation σs of 15 and a value domain standard deviation σr of 25, to smooth regional noise while preserving bubble edge information.
[0075] The bilaterally filtered image was enhanced using CLAHE with an 8×8 sub-block size and a clipping threshold β of 4.0. The contrast amplification was limited to prevent excessive noise amplification. The enhanced image showed a significant improvement in local contrast, with more dramatic grayscale changes in the deepest shadows of the bubble walls.
[0076] Morphological gradient operations were performed on the CLAHE-enhanced image. A 3×3 square structuring element K3 was used for dilation and erosion, and the difference between the two was subtracted to obtain the gradient topographic map G(x,y). The difference in flat areas approached zero, while the grayscale changes were drastic at the bubble boundaries, resulting in steep ridges that precisely delineated the bubble edges. Finally, the processed image was restored to a three-channel IF image to preserve multi-channel gradient information.
[0077] Step 2: SAM mask generation and dynamic topology anti-nesting filtering.
[0078] Input the three-channel image IF into the SAM model (ViT-H version) to obtain the original bubble mask set M={M1,M2,...,M... n The SAM model does not require training or fine-tuning on flotation foam images; it directly utilizes its zero-shot generalization ability to predict foam regions across the entire particle size. In this embodiment, SAM generates approximately 200 initial masks, including a large number of nested redundant masks.
[0079] Sort all masks in descending order of area, and define the occupied mask matrix O0(x,y)=0. Start traversing from the mask with the largest area, and for the k-th mask M... k Calculate the percentage of its area covered by the already occupied area, S. O (M k If S O (M k If M > 0.5, then the mask is determined to be a nested mask and is discarded; otherwise, M is retained. k And update the occupied graph O k (x,y)=O k-1 (x,y)∪(M k(x,y)·255).
[0080] After dynamic topology anti-nesting filtering, nested redundant masks are effectively eliminated, and the number of independent masks retained is about 60% of the initial number, which significantly reduces the computational burden of subsequent processing and the risk of over-segmentation.
[0081] Step 3: Adaptive piecewise elliptic erosion and marker matrix construction.
[0082] The filtered individual masks are subjected to adaptive elliptic erosion based on area segmentation. This is based on area A. k Piecewise mapping of erosion core diameter d(A) k ): when A k When the pixel count is >10000, d=9 (extra-large bubble, strong peeling); when the pixel count is 3000... k When A is ≤10000 pixels, d=5 (medium bubble); k When the pixel count is ≤3000, d=3 (small bubble, bottom-line protection).
[0083] Elliptical structural element B d Defined as Where u is the number of pixels horizontally from the center, and v is the number of pixels vertically from the center. Morphological erosion E is performed on each mask. k =M k B d Large-area adhered bubbles are forcefully peeled off to separate independent cores, while extremely small-area bubbles are kept alive due to the small diameter of the corrosion core.
[0084] Assign an incrementing unique label i (starting from 1, 2, ...) to the eroded mask to construct a label matrix. The background pixels are set to 0. Each non-zero pixel in the marker matrix is the marker seed for the watershed algorithm.
[0085] Step 4: Evolution of the watershed in the gradient topographic map.
[0086] The label matrix D(x,y) and the gradient terrain map G(x,y) are input into the immersion watershed algorithm. Terrain height is defined. The flooding begins from the seed location D(x, y)=i.
[0087] A watershed line is established when water bodies from different labels i and j (i≠j) collide at the gradient ridgeline (where the bubble wall shadow is deepest). And record the boundary D(x,y)=-1 in the marking matrix.
[0088] Final segmentation result M final In (x, y), the pixel belonging to the i-th seed is labeled as i, and the watershed boundary pixel is labeled as -1. The segmentation result is as follows: Figure 2 As shown.
[0089] from Figure 2 It can be seen that the method of the present invention can effectively segment... Figure 1 In the densely clustered flotation foam, large-area clustered bubbles are precisely split into independent bubbles, while extremely small-area bubbles are completely preserved. The bubble boundaries are rounded and continuous, without obvious over-segmentation or under-segmentation.
[0090] Test case To verify the technical effectiveness of the method of the present invention, the method of the present invention was compared with the following three comparative methods on the same flotation foam image (e.g., Figure 1 A segmentation experiment was conducted on the sample (as shown), and the segmentation results were qualitatively compared and analyzed.
[0091] Comparison Method 1: Standalone SAM Segmentation Method. The technical solution of this comparison method is as follows: The flotation bubble image to be segmented is directly input into the SAM model (using the same ViT-H version as this invention), utilizing SAM's zero-shot generalization capability to generate an initial bubble mask set; the generated mask set undergoes simple overlap deduplication processing, i.e., when the intersection-union ratio (IUU) of two masks is greater than a preset threshold, the mask with the larger area is retained, and the mask with the smaller area is discarded; the deduplicated mask set is directly output as the final segmentation result, without concatenated post-processing or watershed evolution. This comparison method employs end-to-end SAM segmentation, and its segmentation result relies entirely on the prediction output of the SAM model.
[0092] Comparison Method 2: Traditional Adaptive Labeling Watershed Method. The technical solution of this comparison method is as follows: Gaussian filtering is applied to the flotation bubble image to be segmented to smooth image noise; an adaptive thresholding method is used to extract watershed labels from the filtered image, i.e., the threshold is automatically determined based on the local gray-level statistical characteristics of the image, and pixels with gray-level values below the threshold are used as candidate labels; morphological filtering is applied to the candidate labels to remove isolated noise labels and retain connected label regions; the retained labels are used to correct the gradient map of the image, forcing the gradient value corresponding to the label position to a local minimum; the watershed algorithm is used to segment the corrected gradient map, and the watershed line is established at the local maximum of the gradient. This comparison method does not involve the SAM model; the source of the labels depends entirely on the gray-level features and morphological operations of the image itself.
[0093] Comparison Method 3: SoftEdge Edge Detection vs. Improved Watershed Method. The technical solution of this comparison method is as follows: The image of the float foam to be segmented is input into the SoftEdge soft edge detection model. This model is based on an encoder-decoder structure and extracts multi-scale edge features through a deep convolutional network, outputting an edge probability map of the same size as the input image. The value of each pixel in the map represents the probability that the pixel belongs to the bubble edge. Thresholding is performed on the edge probability map, with pixels having a probability value greater than a preset threshold as edge points and pixels having a probability value less than a preset threshold as non-edge points. Morphological thinning is performed on the binary edge map to obtain a single-pixel-width edge skeleton. The center point of the closed region enclosed by the edge skeleton is used as the watershed marker, and the edge skeleton serves as the constraint boundary for watershed evolution. An improved watershed algorithm is used for segmentation, and water inundation is performed along the edge constraints under marker-driven conditions, finally establishing a watershed line at the edge location. This comparison method uses an edge detection model instead of a segmentation model, and the edge probability map needs to undergo thresholding and morphological processing before it can be converted into a watershed marker.
[0094] The comparison and analysis of the segmentation effects of the above three comparison methods and the method of the present invention on the same flotation foam image are as follows.
[0095] (1) Nested redundancy mask suppression effect.
[0096] Comparison Method 1 (Single SAM) generates a large number of nested redundant masks in dense bubble scenes. When generating multi-scale masks, SAM generates both a large mask covering the entire bubble and a small mask covering a local area of the bubble for large bubble regions. Comparison Method 1, using only a simple intersection-union (IUU) deduplication strategy, cannot effectively identify and eliminate this nesting relationship (the IUU of the large mask and the small mask may be low, but the small mask is completely inside the large mask), resulting in the same bubble being repeatedly segmented. The method of this invention uses a dynamic topology anti-nesting mechanism, with area sorting as a pre-constraint and the coverage area percentage S... O With a threshold of >0.5 and a state propagation mechanism based on the cumulative update of the occupied mask matrix, nested redundant masks are effectively eliminated, and each bubble retains only one independent mask.
[0097] (2) The effect of splitting the adhering bubbles.
[0098] In contrast to Method 2 (traditional adaptive marker watershed), when processing large-area adhered bubbles, the marker extraction relies on image grayscale features. Since the grayscale changes in the adhered region are gradual, the adaptive thresholding method struggles to generate a sufficient number of independent markers within the adhered region, leading to large-area adhered bubbles being misclassified as single bubbles. Method 3 (SoftEdge + watershed), while capable of detecting the edges of the adhered region, exhibits a weak response in the edge probability map within the adhered region. After threshold segmentation, edge points break, making it difficult to form closed regions and generate effective split markers. The method of this invention obtains the initial mask for the full-size foam region through SAM prediction, and then uses adaptive piecewise elliptical erosion (d=9 / 5 / 3) to forcefully peel away the adhered bubbles, effectively splitting large-area adhered regions into multiple independent bubbles.
[0099] (3) Retention effect of small bubbles.
[0100] Compared to Method 1 (SAM alone), which struggles with extremely small bubbles due to the limited resolution of tiny regions, some small bubbles have overly small predicted mask areas and are misclassified as noise during the overlap deduplication process. In contrast to Method 3 (SoftEdge + Watershed), the SoftEdge edge detection model has a weak edge response to extremely small bubbles. The edge features of these small bubbles are easily submerged by background noise in the edge probability map, and after thresholding, the edges of the small bubbles break, failing to form closed regions and resulting in missed detections. This invention addresses this issue by employing a fallback keep-alive mechanism in adaptive piecewise elliptic erosion (A...). k When ≤3000, d=3), even after slight corrosion, the extremely small bubbles still retain independent cores and are completely preserved in the watershed evolution.
[0101] (4) Boundary roundness.
[0102] In contrast, Method 2 (traditional watershed) tends to produce jagged segmentation boundaries at the bubble boundary. Because the markers in traditional watersheds are derived from grayscale thresholding, their positions are randomly offset by noise, causing the watershed line to deviate from the actual bubble wall, resulting in a jagged boundary. In contrast, Method 1 (Single SAM) determines the boundary shape through internal model decisions, lacking explicit constraints at the deepest point of the bubble wall shadow (i.e., the true physical boundary). The predicted mask boundary may penetrate the bubble interior or extend into the background region, resulting in insufficient continuity and smoothness. The method of this invention constructs steep ridges through morphological gradient topography (dilatational erosion), providing precise edge constraints for watershed evolution. Simultaneously, the SAM predicted mask, after erosion, serves as a marker seed, positioned close to the bubble center. The watershed line smoothly breaks at the gradient ridge line, resulting in continuous and smooth segmentation boundaries.
[0103] (5) Zero-sample adaptability.
[0104] In contrast, method 3 (SoftEdge + Watershed) requires training or fine-tuning the SoftEdge model to adapt to flotation foam images. The SoftEdge model is pre-trained on a general edge detection dataset, and its edge detection strategy is designed for natural images. It has poor adaptability to special visual features in flotation foam images, such as bubble wall shadows and high-brightness reflections. Fine-tuning the model requires collecting flotation foam edge annotation data, which is costly and difficult to cover all working conditions. The method of this invention utilizes the zero-shot generalization capability of SAM, eliminating the need for annotation data in the flotation foam domain, significantly reducing application costs, and demonstrating stronger adaptability to new scenarios.
[0105] In summary, the method of this invention, through the synergistic combination of SAM prediction mask concatenation post-processing and morphological gradient watershed, outperforms the three comparative methods mentioned above in terms of nested redundancy suppression, bubble splitting, small bubble preservation, boundary roundness, and zero-sample adaptability, and can effectively improve the accuracy and robustness of flotation foam image segmentation.
Claims
1. A flotation foam image segmentation method based on SAM prediction mask concatenation post-processing and morphological gradient watershed synergy, characterized in that, Includes the following steps: S1. Perform combined filtering and contrast enhancement preprocessing on the original foam image, and construct a gradient topographic map based on morphological gradient operation; S2. Input the preprocessed image into the SAM model to obtain the original bubble mask set; S3. Perform concatenated post-processing on the original bubble mask set, including sorting by area from largest to smallest, calculating the coverage area ratio based on the occupied mask matrix to dynamically filter out nested masks, and then applying an adaptive elliptical erosion kernel to the remaining masks by area segment to generate marker seeds. S4. Input the labeled seed and the gradient topographic map into the immersion watershed algorithm to achieve accurate segmentation of the adhering foam.
2. The method according to claim 1, characterized in that, The combined filtering and contrast enhancement preprocessing described in step S1 includes: The original bubble image was sequentially converted from RGB to grayscale, then subjected to median filtering, bilateral filtering, and CLAHE enhancement. Bilateral filtering achieved noise reduction and edge preservation through joint weighting of spatial Gaussian kernel and range Gaussian kernel. CLAHE divided the image into several sub-blocks, and after the histogram of each sub-block was subject to a clipping threshold limit, the blocks were redistributed. Finally, the results of each sub-block were fused through bilinear interpolation.
3. The method according to claim 1 or 2, characterized in that, The formula for constructing a gradient topographic map using morphological gradient operations in step S1 is as follows: , Where ⊕ represents the expansion operation. This represents the erosion operation, where K3 is a 3×3 structuring element; and the processed image is restored to a three-channel image I. F .
4. The method according to claim 1, characterized in that, The nested mask dynamic filtering in step S3 includes: removing the original bubble mask set M = {M1, M2, ..., M} n Sort the masks by area from largest to smallest, define the occupied mask matrix O0(x,y) = 0, and for the k-th mask M... k Calculate the coverage area percentage If S O (M k If the value is greater than 0.5, it is considered a nested mask and is removed; otherwise, it is retained and O is updated. k (x,y)=O k-1 (x,y)∪M k (x,y)·255.
5. The method according to claim 1 or 4, characterized in that, The diameter d(A) of the adaptive elliptical erosion core described in step S3 k According to area A k Segmented mapping is 。 6. The method according to claim 5, characterized in that, The elliptical structural element B d Defined as , Where u is the number of pixels from the center in the horizontal direction, v is the number of pixels from the center in the vertical direction, and d is the diameter of the structuring element; the erosion operation is E k =M k B d .
7. The method according to claim 1 or 6, characterized in that, The labeling seed is represented by the labeling matrix D(x,y). , Where E k Let i be the k-th eroded mask, i be the unique label that increments, and the background pixels be set to 0.
8. The method according to claim 1, characterized in that, The immersion watershed algorithm described in step S4 includes: inputting the label matrix D(x,y) and the gradient topographic map G(x,y) into the watershed algorithm, and defining the topographic height. The flooding begins at the seed D(x,y) = i, and a watershed line is established when waters from different labels i and j (i ≠ j) collide. And record the boundary D(x,y) = -1 in the label matrix.
9. The method according to claim 8, characterized in that, The terrain height H(x, y) is expressed using the Euclidean modulus of a three-channel gradient map, and the gradient terrain map is composed of a three-channel image I. F Obtained through morphological gradient calculation.
10. The method according to claim 1 or 8, characterized in that, Final segmentation result M final (x,y) is , Where C H(x,y) (i) represents the connected water area belonging to the i-th seed, and W is the watershed boundary.
Citation Information
Patent Citations
Critical characteristic extraction method for flotation foam image analysis
CN101334844A
