An image object segmentation method based on position-bound information guidance

Abstract

The application discloses a kind of based on location-bound information guidance image object segmentation method belongs to computer vision field.The present application obtains pollen segmentation image by combining the information of two modules: target positioning module obtains positioning map by the classification network trained by image level label, provides accurate location information;Boundary guidance module utilizes pollen contour prior knowledge to match target pollen to obtain saliency map, obtains fine-grained boundary information.This method can make full use of the complementary relationship between the two, obtain accurate target pollen boundary, and discard non-target object (such as impurity) pixel, significantly improve pollen image segmentation quality.

Description

Technical Field

[0001] This invention relates to an image object segmentation method guided by position-boundary information, which belongs to the field of computer vision. Background Technology

[0002] This method segments target objects in microscopic pollen images, counts pollen grains in the images, and predicts their concentration, which helps pollen allergy sufferers prevent allergies. In real-world scenarios, pollen microscopic images contain many impurities, air bubbles, and other interfering substances. Semantic segmentation can effectively utilize the unique contour features of pollen to distinguish interfering substances from pollen, completing the acquisition of pollen region and category information, which is helpful for subsequent pollen counting.

[0003] Existing research on semantic segmentation of pollen images has two main directions: traditional predefined feature methods and deep learning methods. Regarding traditional predefined feature methods, Landsmeer et al. [1] A method for segmenting pollen images based on color and shape information was proposed. This method integrates pollen detection through Hough transform, color thresholding, and other methods, and uses a clustering algorithm to determine the affiliation of pollen in different planes. Sebastiano et al. [2] This study integrates multiple methods for pollen segmentation. First, mean-transfer filtering and Otsu's method are used for coarse segmentation of the foreground and background. Then, flooding, area detection, and morphological operations are performed on the coarsely segmented image to further remove impurities and bubbles, resulting in accurate segmentation. Xiong et al. [3] This study focuses on the color features of pollen, processing pollen images using a mean-transfer filtering algorithm. The pollen images are then binarized in HSV space using these color features, and finally, morphological operations and contour extraction are performed to obtain the segmentation results. Regarding deep learning methods, Jahanshahi... [4] Applying the Unet network to pollen image segmentation, he used a public dataset containing 11 pollen types, manually annotating the pollen pixels with pixel-level contour labels to obtain image masks and train the Unet network. (James A. Grant-Jacob et al.) [5] A two-step neural network segmentation model is proposed to segment pollen grains using pollen scattering images. First, an image generation network generates images of pollen grains from pollen scattering patterns. Then, the images are used to identify pollen grains in the image by a semantic segmentation network (DeepLabV3).

[0004] The above methods can segment pollen regions to a certain extent, but they have the following limitations: (1) Traditional predefined feature methods can obtain fine-grained boundary information of target pollen, but cannot eliminate interference from non-target objects (such as impurities); (2) Pixel-level labeling of deep learning methods is time-consuming and laborious; (3) Weak supervision methods using weak supervision labels are rarely used in pollen segmentation. They can obtain accurate target pollen location information, but lack boundary information constraints.

[0005] To overcome these challenges, this invention proposes a pollen image segmentation method guided by location-boundary information. This method obtains segmented pollen images by combining information from two modules: a target localization module obtains a localization map through a classification network trained on image-level labels, providing accurate location information; and a boundary guidance module uses prior knowledge of pollen contours to match target pollen to obtain a saliency map, yielding fine-grained boundary information. This method fully utilizes the complementary relationship between the two modules to obtain accurate target pollen boundaries and discards pixels of non-target objects (such as impurities), significantly improving the quality of pollen image segmentation. The results are shown in the appendix. Figure 1 As shown. Summary of the Invention

[0006] Existing weakly supervised image segmentation utilizes image-level labels to complete the segmentation task. Class activation mapping (CAM) based on classification networks generates target interest images, which can obtain accurate location information of the target object. However, in pollen image data, pollen grains are small and their proportion to the whole image differs significantly. Images generated by CAM suffer from inaccurate segmentation region coverage and incomplete target object boundaries, which seriously affects the segmentation quality.

[0007] This invention employs the following technical solution: Based on the CAM image from the classification network, pollen boundary information is specifically introduced to constrain the boundaries of the segmented image. We integrate the original classification network CAM into a target localization module to generate a localization map and extract the target pollen location information. On this basis, a boundary guidance module is added to obtain a saliency map, containing fine-grained pollen boundary information. To fully utilize the location information of the localization map and the boundary information of the saliency map, we fuse the two modules. The target localization module uses an image multi-classification network to generate a pollen localization map, supplementing the pollen location information missing in the boundary guidance module. The boundary guidance module introduces the idea of ​​template matching, using prior knowledge of pollen contours to match the target region boundary, determining the target pollen boundary and category, obtaining a pollen saliency map, and guiding the optimization of boundary information in the target localization module. Finally, our model uses an image fusion algorithm to complement the advantages of the localization map and the saliency map, obtaining an accurate pollen segmentation image.

[0008] The specific solution of the present invention is attached. Figure 2 As shown.

[0009] Step 1: Image cropping

[0010] Image cropping refers to cropping an oversized image obtained from a digital slice scanner into an optimal pollen image size that is easy for the program to process while minimizing the loss of pollen information.

[0011] Step 2: The target localization module obtains the localization map.

[0012] The purpose of this step is to train a classification network using image-level labels and obtain a localization map through class activation mapping. It consists of three main stages.

[0013] Step 2.1: Construct the dataset: Based on the image content, the cropped images are divided into three categories: pollen, impurities, and others. Images containing clear pollen are classified as pollen, images containing obvious impurities but not pollen are classified as impurities, and the remaining images are classified as others. Then, rotation and multi-scale transformations are used to balance the number of images in each category, facilitating the training of the classification network.

[0014] Step 2.2 Constructing the Classification Network: This invention constructs a multi-classification network for pollen images based on a CNN network, and trains the network using image-level label supervision. In this stage, the network adopts the classic DenseNet network, introducing a dense connection method. This method allows each layer of the network to receive the results of all preceding layers as additional input, achieving feature reuse and improving network efficiency.

[0015] Step 2.3 Obtaining the localization map using class activation mapping: This invention utilizes class activation mapping to obtain the pollen class activation map. For deep classification networks such as DenseNet that contain global average pooling layers, an activation map containing class information can be obtained by using the weighted combination of each channel of the highest layer feature map. Then, the pollen class is activated according to a set threshold. Figure 2 Value-based methods yield the location map.

[0016] Step 3: Boundary guidance module obtains saliency map

[0017] The purpose of the boundary guidance module is to filter contour regions in an image using the prominent features of pollen outlines and generate a saliency map. First, the target contour is extracted from the original image, and the contour features are represented by two feature descriptors (shape context features and shape metric features). Then, based on the feature descriptors, the target contour is matched with pollen templates in our constructed template library, establishing a correspondence between the target contour and the template. Target contour regions with strong correspondences are selected as the salient regions for this module, generating a saliency map and simultaneously obtaining boundary extent and region category information.

[0018] Step 3.1 Contour Extraction: First, the image is processed by migration mean filtering and Gaussian filtering to weaken the influence of internal texture and smooth the image edges; after converting the image to grayscale, the Canny operator is used to extract all edge lines in the image; finally, the outermost closed lines are retained, and the area and color of the closed region are checked. Lines that meet the pollen characteristics requirements are taken as the contour extraction results.

[0019] Step 3.2 Shape Context Feature Extraction: The context information of each point in the contour point set is statistically analyzed using the shape context feature descriptor. The shape context features of the target contour are obtained through contour point sampling, shape context feature descriptor extraction and other operations.

[0020] Step 3.3 Shape Measurement Feature Extraction: In order to measure the overall shape of the contour, this invention proposes shape measurement features, which include four feature dimensions: roundness ratio, roundness variance, rectangularity and concavity. We extract the features of the above four dimensions of the target contour to obtain its shape measurement features.

[0021] Step 3.4 Template matching: Calculate two feature descriptors for the template contour and the target contour respectively. Match the extracted target contour with the template image based on the feature distance. Select the contour region with high matching degree. After contour filtering and K-NN classification, obtain the saliency map.

[0022] Step 4: Generate pollen segmentation image

[0023] This invention fuses the pollen localization map generated by the target localization module and the pollen saliency map generated by the boundary guidance module to obtain the final pollen segmentation image. To obtain an accurate pollen segmentation image, we use the activation region of the saliency map (referred to as A) S Using ) as a baseline, the active region of the localization map (referred to as A) L As a condition, choose A. L A with intersection S This serves as the final pollen segmentation result.

[0024] Compared with existing technologies, the advantages of this invention are as follows:

[0025] I. The image object segmentation method guided by position-boundary information adopted in this invention utilizes the idea of ​​template matching to fully consider the prior features of pollen contours to screen the contour regions of the image, while obtaining the boundary range information and region category information of the target contour.

[0026] Second, the image object segmentation method guided by position-boundary information adopted in this invention makes full use of the complementary relationship between the saliency map boundary information and the position map position information, so that the activation regions of the two types of images complement each other, thereby improving the accuracy of segmented image region coverage and the integrity of the boundaries.

[0027] References:

[0028] [1] Landsmeer S H, Hendriks E A, Weger L, et al. Detection of pollengrains in multifocal optical microscopy images of air samples[J]. MicroscopyResearch and Technique, 2010, 72(6): 424 - 430.

[0029] [2] Sebastiano Battiato, et al. POLLEN13K: A Large Scale MicroscopePollen Grain Image Dataset. 2020 IEEE International Conference on ImageProcessing(ICIP), 2020, pp. 2456–2460.

[0030] [3] Xiong, C., Li, J., et al. An Automatic Pollen Grain Detector Using DeepLearning. Frontier Computing, 2021, pp: 34–44.

[0031] [4] Jahanshahi, Maxime. Segmentation and classification of pollen grainswith CNN's. Ecole polytechnique de Louvain, Universitécatholique de Louvain, 2020. Prom.: De Vleeschouwer, Christophe.

[0032] [5] Grant - Jacob J A, Praeger M, Eason R W, et al. Semantic segmentation ofpollen grain images generated from scattering patterns via deep learning[J].Journal ofPhysics Communications, 2021, 5(5): 055017(11pp). Attached Figure Description

[0033] Figure 1 This is a diagram illustrating the effect of the method proposed in this invention.

[0034] Figure 2 This is an overall model diagram of the method proposed in this invention. Detailed Implementation

[0035] The following detailed description of embodiments of the present invention is provided in conjunction with the accompanying drawings:

[0036] Figure 1 This is a diagram illustrating the effect of the method proposed in this invention. The black areas in the saliency map, localization map, and result image represent the identified background areas. The saliency map obtained from the input image contains detailed pollen boundary information but contains non-target interference (such as the white dashed box in the saliency map); the localization map contains accurate pollen location information but lacks boundary information constraints; the result image of this invention fully integrates the two parts of information to obtain an accurate pollen target segmentation image.

[0037] This invention is an image object segmentation method guided by location-boundary information. It is a weakly supervised pollen image segmentation method that integrates the location information of the pollen localization map and the boundary information of the saliency map. The specific process of pollen image segmentation is as follows: the original classification network CAM is integrated into a target localization module to generate a localization map and extract the target pollen location information; on this basis, a boundary guidance module is added to obtain a saliency map, which contains fine-grained pollen boundary information. In order to make full use of the location information of the localization map and the boundary information of the saliency map, we fuse the two modules. The target localization module uses an image multi-classification network to generate a pollen localization map, supplementing the pollen location information missing in the boundary guidance module; the boundary guidance module introduces the idea of ​​template matching, using prior knowledge of pollen contours to match with the target region boundary, determining the target pollen boundary and category, obtaining the pollen saliency map, and guiding the optimization of the boundary information of the target localization module. Finally, our model uses an image fusion algorithm to complement the advantages of the localization map and the saliency map, resulting in an accurate pollen segmentation image.

[0038] Specifically, the method includes the following steps:

[0039] Step 1: Image cropping

[0040] We cropped and filtered the large pollen images obtained from digital slicing scanners. We cropped the entire pollen image to obtain the optimal pollen image size for easy processing by the program. Optical microscope scans typically produce extremely large images (e.g., images larger than 100,000 × 50,000 pixels obtained using a digital slicing scanner), which is very detrimental to model training. Furthermore, the pollen in these images is very sparse, and the pollen grains are generally very small (in this experiment, we found pollen grains to be only 50 to 120 pixels in diameter), significantly disproportionate to the overall image size, making automatic processing of such data very difficult. Under the guidance of the Beijing Meteorological Bureau, we cropped the entire pollen image to obtain the optimal pollen image size (512 × 512 pixels in this experiment). This size preserves as much pollen as possible, minimizes image information loss, and facilitates program processing.

[0041] Step 2: The target localization module obtains the localization map.

[0042] The target localization module aims to train a classification network using image-level labels and obtain pollen localization maps through class activation mapping. This module consists of three main stages.

[0043] Step 2.1 Dataset Construction and Image Preprocessing: Based on image content, this invention categorizes cropped images into three classes: pollen, impurities, and others. Images containing clear pollen are classified as pollen, images containing obvious impurities but not pollen are classified as impurities, and the remaining images are classified as others. Rotation and multi-scale transformations are then used to balance the number of images in each category, facilitating the training of the classification network.

[0044] Step 2.2 Constructing the Classification Network: This invention constructs a multi-classification network for pollen images based on a CNN network, and trains the network using image-level label supervision. In this stage, the network adopts the classic DenseNet network, introducing a dense connection method. This method allows each layer of the network to receive the results of all preceding layers as additional input, achieving feature reuse and improving network efficiency.

[0045] Step 2.3 Obtaining the localization map using class activation mapping: This invention utilizes class activation mapping to obtain the pollen class activation map. For deep classification networks such as DenseNet that contain global average pooling layers, the activation map containing class information can be obtained by weighted combination of each channel of the highest layer feature map, denoted as M. CAM As shown in formula (1).

[0046]

[0047] In the formula, C refers to the number of feature maps in the last layer of the network, and Z... cRefers to the c-th feature map, β c This refers to the weight of the c-th feature map.

[0048] Then, pollen will be activated according to the set threshold. Figure 2 Value-based mapping yields the location map, denoted as M. L As shown in formula (2).

[0049]

[0050] In the formula, ε is a pre-set threshold, which is set to 0.65 in this invention. M L (a, b) represents the pixel value in row a, column b of the location map, M CAM (a,b) represents the pixel value in row a and column b of the pollen activation map.

[0051] Step 3: Boundary guidance module obtains saliency map

[0052] The purpose of the boundary guidance module is to filter contour regions in an image using the prominent features of pollen outlines and generate a saliency map. First, the target contour is extracted from the original image, and the contour features are represented by two feature descriptors (shape context features and shape metric features). Then, based on the feature descriptors, the target contour is matched with pollen templates in our constructed template library, establishing a correspondence between the contour and the template. Contour regions with strong correspondences are selected as the salient regions for this module, generating a saliency map and simultaneously obtaining boundary extent and region category information.

[0053] Step 3.1 Contour Extraction: First, the image is processed by migration mean filtering and Gaussian filtering to weaken the influence of internal texture and smooth the image edges; after converting the image to grayscale, the Canny operator is used to extract all edge lines in the image; finally, the outermost closed lines are retained, and the area and color of the closed region are checked. Lines that meet the pollen characteristics requirements are taken as the contour extraction results.

[0054] Step 3.2 Shape Context Feature Extraction: This invention utilizes shape context feature descriptors to statistically analyze the context information of each point in the contour point set.

[0055] Step 3.2.1 Contour Point Sampling: For n contour points of a single contour, this invention calculates the distance between each point and the remaining n-1 points, resulting in an n*n relative distance matrix. Each time, any point in the shortest distance point pair is removed, and its distance relationship with other points is terminated. This process is repeated until only m contour points remain in the point pair (m<=n, this invention sets m to 100).

[0056] Step 3.2.2 Calculate the shape context feature descriptor: Among m contour sampling points, for one point p iWe construct a polar logarithmic coordinate system centered at p. To ensure the descriptor has rotation invariance, we define p... i The direction of the line connecting the centroid of the contour is the positive direction of the X-axis of the coordinate system. The centroid g is calculated as shown in formula (3):

[0057]

[0058] Where m represents the total number of contour points, x i ,y i Let x and y represent the x and y coordinates of the i-th point, respectively.

[0059] The coordinate system is divided into 12 angular regions and 5 distance regions (bins), resulting in a total of 60 polar coordinate regions (denoted as K). The number of sample points falling into different regions is counted, and finally, normalization is performed to obtain the shape context descriptor F. SC As shown in formula (4):

[0060]

[0061] The expression #{U:V} represents the number of V under condition U. p′ represents the remaining boundary sample points, and bin(k) represents p. i The k-th polar coordinate region of the point. Each contour sampling point p i Each will generate a shape context histogram. This represents the value corresponding to the kth polar coordinate region in the histogram. The m histograms obtained from m sampling points in the image are defined as the top and bottom features of the image's shape.

[0062] Step 3.3 Shape Measurement Feature Extraction: In order to measure the overall shape of the contour, this invention proposes shape measurement features, which include four feature dimensions: roundness ratio, roundness variance, rectangularity, and concavity. We extract the features of the above four dimensions of the target contour to calculate its shape measurement features.

[0063] Step 3.3.1 Calculate the roundness ratio: The roundness ratio measures the degree to which the profile is round, and is defined as the ratio between the area enclosed by the profile and the area of ​​a circle with the same circumference, denoted as F. cr As shown in formula (5).

[0064]

[0065] Where A represents the area, P represents the perimeter, and A boundary A circle P boundary These are the area enclosed by the target outline, the area of ​​the circle, and the perimeter of the target outline, respectively.

[0066] Step 3.3.2 Calculate the circularity variance: Circularity variance measures the degree of circularity of the profile. It is defined as the ratio of the standard deviation to the mean of the set of distances between the profile points and the centroid, denoted as F. cv As shown in formulas (6) and (7).

[0067] ρ i =|p i -g|| (6)

[0068]

[0069] Where |||| represents the radial distance, g represents the centroid (calculated using formula (3)), and p i ρ represents the i-th contour point. i σ represents the radial distance between the i-th point and the centroid. ρ μ represents the standard deviation of radial distance. ρ This represents the average radial distance.

[0070] Step 3.3.3 Calculate Rectangularity: Rectangularity measures the density of shapes within a bounding rectangle, i.e., the similarity between the outline and the rectangle. It is defined as the ratio between the area enclosed by the outline and the area of ​​its smallest bounding rectangle, denoted as F. re As shown in formula (8).

[0071]

[0072] Where A box This represents the area of ​​the smallest bounding rectangle.

[0073] Step 3.3.4 Calculate concavity: Concavity measures the degree of indentation of a contour and is defined as the ratio between the perimeter of the convex hull and the perimeter of the contour, denoted as F. co As shown in formula (9).

[0074]

[0075] Where P hull This represents the perimeter of the convex hull.

[0076] Step 3.3.5 Calculate the shape metric feature descriptor: Finally, this invention calculates the above four feature values ​​for the sample contour points and integrates them to obtain the shape metric feature descriptor F. SM , as in formula (10).

[0077] F SM =[F cr ,F cv ,F re ,F co (10)

[0078] Step 3.4 Template matching: Calculate two feature descriptors for the template contour and the target contour respectively. Match the extracted target contour with the template image based on the feature distance. Select the contour region with high matching degree. After contour filtering and K-NN classification, obtain the saliency map.

[0079] Step 3.4.1 Calculate the distance of the shape context features: The shape context feature descriptor can be used to calculate the distance for each point p on the first contour. i Find the "best" matching point q on the second contour. j The feature distance between all matching points is used as a feature distance metric between contours.

[0080] For a point p on the target contour i and a point q on the template outline j Using chi-square statistics C(p) i ,q j The feature distance between two points is represented by F. The feature distances between every two points form an m*m feature distance matrix C, as shown in formula (11). We use F to represent the feature of the target contour and T to represent the feature of the template contour.

[0081]

[0082] and Let K represent the feature values ​​of the k-th polar coordinate regions of the i-th target contour point and the j-th template contour point, respectively, where K represents the total number of polar coordinate regions.

[0083] Shape context feature distance metric D SC The sum of the feature distances between the best matching points on the target contour and the template contour is obtained by symmetric summation of the feature distance matrices, as shown in formula (12).

[0084]

[0085] Let p represent the minimum value of V under condition U, m represent the number of sample points, and p represent the minimum value of V under condition U. i and q j These represent the i-th target contour sample point and the j-th template contour sample point, respectively.

[0086] Step 3.4.2 Calculate the distance of the shape metric feature: For the shape metric feature of the contour, this invention uses the Euclidean distance between the target contour and the template contour feature to represent the distance D of the shape metric feature. SM As shown in formula (13).

[0087]

[0088] Where Ftype and T type These represent the type features of the target contour and the template contour, respectively. The type can be ce, cv, re, or co.

[0089] Step 3.4.3 Calculate the joint distance representation: We define the shape distance D between the target contour and the template contour as the shape context distance D. SC Distance D of shape metric features SM The weighted sum is shown in formula (14):

[0090] D=ω×D SC +(1-ω)×D SM (14)

[0091] In the formula, ω is the weighting parameter corresponding to the two distances, which is defined as 0.8 in this invention.

[0092] Step 3.4.4 K-NN Classification: For each target contour extracted from the image, 400 distance features can be generated with 400 templates. We use the idea of ​​K-NN classification to count the proportion of each category among the 100 templates that are closest to the sample contour (i.e., the smallest distance feature), and take the category with the largest proportion as the category of this contour.

[0093] Finally, a pollen saliency map M is generated based on the contours obtained from the above steps. S .

[0094] Step 4: Generate pollen segmentation image

[0095] This invention integrates the pollen localization map M generated by the target localization module. L Pollen saliency map M generated by the boundary guidance module S The final pollen segmentation image M is obtained. To obtain an accurate pollen segmentation image, we use the activation regions of the saliency map (called A) S Using ) as a baseline, the active region of the localization map (referred to as A) L As a condition, choose A. L A with intersection S The final result of pollen segmentation is shown in formulas (15) and (16).

[0096]

[0097] M = {A h} (16)

[0098] Where {} represents a set, A h These represent the h-th activation region on the saliency map, localization map, and final pollen segmentation image, respectively.

Claims

1. An image object segmentation method guided by position-boundary information, characterized in that, Includes the following steps: Step 1: Image cropping The large image of pollen obtained by digital slice scanner is cropped and filtered; The entire pollen image was cropped to obtain the optimal size. The size used was [size missing]. ; Step 2: The target localization module obtains the localization map. The purpose of the target localization module is to train a classification network using image-level labels and obtain pollen localization maps through class activation mapping. This module consists of three main stages; Step 2.1 Dataset Construction and Image Preprocessing: Based on the image content, the cropped images are divided into three categories: pollen, impurities, and others. Images containing clear pollen are classified as pollen, images containing obvious impurities but not pollen are classified as impurities, and the remaining images are classified as others. Then, the number of images in each category is balanced to facilitate the training of the classification network. Step 2.2 Constructing the classification network: Based on the CNN network, a multi-classification network for pollen images is constructed and trained using image-level label supervision; the classic DenseNet network is used in this stage. Step 2.3 Obtaining the localization map using class activation mapping: For deep classification networks like DenseNet with global average pooling layers, a pollen class activation map is obtained using weighted combinations of channels from the highest-level feature maps. This activation map, denoted as [missing information], is used to obtain the pollen class activation map. As shown in formula (1); (1) In the formula, C indicates that the last layer of the network has C feature maps. Refers to the c-th feature map. The weight assigned to the c-th feature map; Then, the pollen activation map is binarized according to the set threshold to obtain the localization map, denoted as . As shown in formula (2); (2) In the formula The pre-defined threshold is set to 0.65; This represents the pixel value in row a and column b of the location map. This represents the pixel value in row a and column b of the pollen activation map; Step 3: Boundary guidance module obtains saliency map The purpose of the boundary guidance module is to filter the contour regions of the image using the obvious features of the pollen contour and generate a saliency map. First, the target contour is extracted from the original image. The contour features are represented by two feature descriptors: shape context features and shape metric features. Then, the target contour is matched with the pollen templates in the constructed template library according to the feature descriptors to form a correspondence between the contour and the template. The contour regions with strong correspondence are selected as the saliency regions of this module to generate a saliency map, and the boundary range and region category information are obtained at the same time. Step 3.1 Contour Extraction: First, the image is processed by migration mean filtering and Gaussian filtering to weaken the influence of internal texture and smooth the image edges; after converting the image to grayscale, the Canny operator is used to extract all edge lines in the image; finally, the outermost closed lines are retained, and the area and color of the closed region are checked. Lines that meet the pollen characteristics requirements are taken as the contour extraction results. Step 3.2 Shape Context Feature Extraction: The context information of each point in the contour point set was statistically analyzed using shape context feature descriptors; Step 3.2.1 Contour point sampling: For n contour points of a single contour, the distance between each point and the other n-1 points is counted to obtain an n*n relative distance matrix; each time, any point in the shortest distance point pair is removed and its distance relationship with other points is cancelled, and this process is repeated until only m contour points remain in the point pair; m<=n, and m is set to 100; Step 3.2.2 Calculate the shape context feature descriptor: Among m contour sampling points, for one of the points... Construct a polar logarithmic coordinate system centered at this point. To ensure the descriptor has rotation invariance, define... The line connecting the centroid of the contour is in the positive direction of the X-axis of the coordinate system; find the centroid. As shown in formula (3): (3) Where m represents the total number of contour points. , These represent the x and y coordinates of the i-th point, respectively. The coordinate system is divided into 12 angular regions and 5 distance regions (bins), resulting in a total of 60 polar coordinate regions, denoted as K. The number of sample points falling into different regions is counted, and finally, normalization is performed to obtain the shape context descriptor. As shown in formula (4): (4) The expression #{U:V} represents the number of V under condition U; Represents the remaining boundary sample points. express The k-th polar coordinate region of the point; each contour sampling point Each will generate a shape context histogram. , This represents the value corresponding to the kth polar coordinate region in the histogram. The m histograms obtained from m sampling points in the image are defined as the top and bottom features of the image's shape. Step 3.3 Shape Measurement Feature Extraction: In order to measure the overall shape of the contour, shape measurement features are proposed, which include four feature dimensions: roundness ratio, roundness variance, rectangularity and concavity. The shape measurement features are calculated by extracting the features of the above four dimensions of the target contour. Step 3.3.1 Calculate the roundness ratio: The roundness ratio measures the degree of roundness of the profile, defined as the ratio between the area enclosed by the profile and the area of ​​a circle with the same circumference, denoted as . As shown in formula (5); (5) Where A represents the area and P represents the perimeter. , , These are the area enclosed by the target outline, the area of ​​the circle, and the perimeter of the target outline, respectively. Step 3.3.2 Calculate the circularity variance: Circularity variance measures the degree of circularity of the profile. It is defined as the ratio between the standard deviation and the mean of the set of distances between the profile points and the centroid, denoted as . As shown in formulas (6) and (7); (6) (7) Wherein the formula Indicates radial distance. The center of gravity is indicated by formula (3); This indicates that the i-th point is used to construct the contour. This represents the radial distance between the i-th point and the centroid. The standard deviation of radial distance, This represents the average radial distance. Step 3.3.3 Calculate Rectangularity: Rectangularity measures the density of shapes within a bounding rectangle, i.e., the similarity between the outline and the rectangle. It is defined as the ratio between the area enclosed by the outline and the area of ​​its smallest bounding rectangle, denoted as . As shown in formula (8); (8) in This represents the area of ​​the smallest bounding rectangle; Step 3.3.4 Calculate concavity: Concavity measures the degree of indentation of a contour and is defined as the ratio between the perimeter of the convex hull and the perimeter of the contour, denoted as . As shown in formula (9); (9) in Indicates the perimeter of the convex hull; Step 3.3.5 Calculate the shape metric feature descriptor: Finally, calculate the four feature values ​​above for the sample contour points and integrate them to obtain the shape metric feature descriptor. , as in formula (10); (10) Step 3.4 Template matching: Calculate two feature descriptors for the template contour and the target contour respectively. Match the extracted target contour with the template image based on the feature distance. Select the contour region with high matching degree. After contour filtering and K-NN classification, obtain the saliency map. Step 3.4.1 Calculate the distance of the shape context features: The shape context feature descriptor can be used to calculate the distance for each point on the first contour. Find the "best" matching point on the second contour. The feature distance between all matching points is used as a measure of the feature distance between contours. For a point on the target contour and a point on the template outline Using chi-square statistics , to represent the feature distance between the two points; the feature distance between each pair of points forms an m*m feature distance matrix C, as shown in formula (11); F represents the feature of the target contour, and T represents the feature of the template contour; (11) and Let K represent the feature values ​​of the k-th polar coordinate regions of the i-th target contour point and the j-th template contour point, respectively, where K represents the total number of polar coordinate regions. Shape context feature distance metric The sum of the feature distances between the best matching points on the target contour and the template contour is obtained by symmetric summation of the feature distance matrix, as shown in formula (12). (12) This represents the minimum value of V under condition U, where m represents the number of sample points. and These represent the i-th target contour sample point and the j-th template contour sample point, respectively. Step 3.4.2 Calculate the distance of shape metric features: For the shape metric features of a contour, the distance of the shape metric features is represented by the Euclidean distance between the target contour and the template contour features. As shown in formula (13); (13) in and These represent the type features of the target contour and the template contour, respectively. The type can take various values. ; Step 3.4.3 Calculate the joint distance representation: Define the shape distance D between the target contour and the template contour as the shape context distance. Distance to shape metric features The weighted sum is shown in formula (14): (14) In the formula, It is the weighting parameter corresponding to the two distances, which is 0.8; Step 3.4.4 K-NN Classification: For each target contour extracted from the image, 400 distance features can be generated with 400 templates. Using the idea of ​​K-NN classification, the proportion of each category in the 100 templates that are closest to the sample contour (i.e., the smallest distance feature) is counted, and the category with the largest proportion is taken as the category of this contour. Finally, a pollen saliency map is generated based on the contours obtained from the above steps. ; Step 4: Generate pollen segmentation image Pollen localization map generated by the fusion target localization module Pollen saliency map generated by the boundary guidance module The final pollen segmentation image is obtained. To obtain an accurate pollen segmentation image, the activation region of the saliency map is called... As a benchmark, the active region of the localization map is called... As a condition, choose and There is an intersection The final result of pollen segmentation is shown in formulas (15) and (16); (15) (16) Where {} represents a set, , , These represent the h-th activation region on the saliency map, localization map, and final pollen segmentation image, respectively.

Images