A pollen image feature extraction and recognition method based on deep learning

Abstract

The application discloses a pollen image feature extraction and recognition method based on deep learning, relates to the technical field of computer vision and image recognition, and comprises the following steps: an image preprocessing step: obtaining a target pollen image and converting the target pollen image into a single-channel grayscale data matrix; a manifold field reconstruction step: inputting the matrix into a surface reconstruction encoder to generate a pseudo-three-dimensional geometric feature map representing surface space structure; a topological feature extraction step: using a geodesic line convolution unit to perform non-Euclidean space sampling and extracting surface geometric curvature features with rotation invariance; and a Riemann classification recognition step: inputting the features into a classifier based on Riemann metric, calculating geodesic line distance in a manifold space, and determining a category label; the application solves the problem of topological structure information loss caused by rotation of a two-dimensional image by constructing a pseudo-three-dimensional geometric feature map and applying geodesic line convolution, and high-precision recognition of close species is realized.

Description

Technical Field

[0001] This invention relates to the field of computer vision and image recognition technology, specifically to a method for pollen image feature extraction and recognition based on deep learning. Background Technology

[0002] Computer vision technology is increasingly being applied in the field of biological microscopy, and automated pollen image identification has become an important auxiliary tool in palynological research. Currently, pollen species identification mainly relies on two-dimensional digital images acquired by optical microscopes. Existing identification techniques typically analyze pixel grayscale or texture features in Euclidean space, treating three-dimensional pollen grains as planar images and determining the pollen category by comparing image features.

[0003] However, the microscopic imaging process is limited by the depth of field of the optical system and the random placement angle of the pollen samples during preparation: First, the limited depth of field can cause local out-of-focus or blurry images, making it difficult for traditional algorithms to capture key texture details; second, when three-dimensional pollen grains are projected into two-dimensional images, spatial topological information is inevitably lost, and rotational changes in the shooting angle can cause the same pollen to present drastically different visual forms. Furthermore, subtle morphological differences between closely related species are often compressed or distorted in a flat Euclidean space, resulting in poor robustness of feature extraction and making it difficult to meet the requirements of high-precision classification.

[0004] Therefore, how to effectively recover and extract rotation-invariant topological features from a single two-dimensional microscopic image to improve recognition accuracy under complex imaging conditions has become an urgent problem to be solved in this field. Summary of the Invention

[0005] The purpose of this invention is to provide a pollen image feature extraction and recognition method based on deep learning, and to solve the following technical problems: Existing pollen identification technologies are mainly based on the analysis of two-dimensional image features in Euclidean space. However, they have significant shortcomings in dealing with problems such as the loss of spatial topological information caused by the projection of three-dimensional pollen grains, visual morphological differences caused by random rotation of the shooting angle, blurring caused by the depth of field limitation of microscopic imaging, and the compression and distortion of the fine morphological differences of closely related species in flat space. There is an urgent need to propose an intelligent image processing method that can effectively recover pseudo-three-dimensional manifolds from a single two-dimensional image, extract surface geometric curvature features with rotation invariance, and accurately identify closely related species using Riemannian metric.

[0006] The objective of this invention can be achieved through the following technical solutions: S1. Obtain the target pollen image to be identified and convert the target pollen image into a single-channel grayscale data matrix; S2. Input the single-channel grayscale data matrix into the surface reconstruction encoder in the surface geometric feature reconstruction network to generate a pseudo-three-dimensional geometric feature map characterizing the spatial structure of the target pollen surface; S3. Based on the pseudo-3D geometric feature map, geodesic convolution units are used to perform feature sampling in non-Euclidean space to extract surface geometric curvature features with rotation invariance. S4. Input the surface geometric curvature features into a Riemannian metric-based classifier, and use the classifier to calculate the geodesic distance of the feature vector corresponding to the surface geometric curvature features in the non-Euclidean space to determine the species label of the target pollen image.

[0007] As a further aspect of the present invention, S2 includes the following sub-steps: S21. Perform convolution operation on the single-channel grayscale data matrix using the photometric constraint layer in the surface reconstruction encoder; S22. Based on the preset Lambertian illumination reflection model constraints, the normal vector field distribution corresponding to the pixel is decoupled from the result of the convolution operation; S23. Calculate the depth value of each pixel based on the normal vector field distribution; S24. The normal vector field distribution and depth value are concatenated to generate a pseudo-3D geometric feature map.

[0008] As a further aspect of the present invention, S3 includes the following sub-steps: S31. Traverse each feature point in the pseudo-3D geometric feature map and obtain the local normal vector at the current feature point; S32. Determine the surface path along the object's surface based on the direction of the local normal vector; S33. Map the sampling points of the convolution kernel from a rectangular grid distribution to a curved path to generate an adaptive convolution sampling point set; S34. Use the adaptive convolution sampling point set to perform weighted summation on the pseudo-3D geometric feature map and output the surface geometric curvature features.

[0009] As a further aspect of the present invention, S4 includes the following sub-steps: S41. Extract two-dimensional texture features from the target pollen image; S42. Fuse the two-dimensional texture features with the surface geometric curvature features to generate a multimodal feature vector; S43. In the Riemannian manifold space, calculate the Riemannian metric distance between the multimodal feature vector and the preset pollen class center points; S44. Select the category corresponding to the category center point with the smallest Riemann metric distance as the category label.

[0010] As a further aspect of the present invention, the decoupling of the normal vector field distribution corresponding to the pixel in S22 specifically includes: A local neighborhood window is set with the pixel as the center, and the pixel gradient variance within the local neighborhood window is calculated. The pixel gradient variance is compared with a preset texture threshold. If the pixel gradient variance is less than the texture threshold, it is determined that there is high-frequency texture loss in the local neighborhood window. The pixel with the largest gradient variance is searched within the set neighborhood of the local neighborhood window as a reference pixel. Based on the Jacobian matrix of the normal vector field of the reference pixel, the normal vector direction of the local neighborhood window is completed by interpolation through differential operation. If the pixel gradient variance is greater than or equal to the texture threshold, the normal vector field distribution of the local neighborhood window remains unchanged.

[0011] As a further aspect of the present invention, a model training step is included before S1: S01. Construct an initial model that includes a network for reconstructing surface geometric features; S02. Supervised training of the initial model is performed using sample images and their corresponding pollen species labels; S03. During the training process, a surface consistency loss function is introduced to constrain the normal vector field output by the surface reconstruction encoder to satisfy the integrability condition.

[0012] As a further aspect of the present invention, in S33, an adaptive convolution sampling point set is generated, specifically by performing the following operations: Calculate the projection relationship between the local normal vector and the tangent plane; Based on the projection relationship, the coordinate offset of the sampling points of the convolution kernel is dynamically adjusted so that the sampling points are distributed on the contour trajectory centered on the current feature point.

[0013] As a further aspect of the present invention, the surface geometric curvature features include at least one of the following: The average curvature characteristics of the surface protrusions, the Gaussian curvature characteristics of the surface pores and depressions, and the principal curvature direction field characteristics of the surface texture direction are used to characterize the surface protrusions.

[0014] As a further aspect of the present invention, the target pollen image in S1 is a two-dimensional projection image acquired by a microscopic imaging device at a single focal length, and the two-dimensional projection image includes a defocused area caused by depth-of-field limitations.

[0015] As a further aspect of the present invention, the Riemannian metric distance in S43 is calculated using a logarithmic Euclidean metric algorithm to quantify the manifold differences between different topologies.

[0016] The present invention has the following advantages: 1. This invention decouples the normal vector field and depth value from the single-channel grayscale data matrix through the photometric constraint layer and Lambertian illumination reflection model in the surface reconstruction encoder. Addressing the defocusing problem in microscopic imaging, it calculates the pixel gradient variance and compares it with a texture threshold. For high-frequency texture loss areas, it uses the neighborhood light intensity gradient trend for differential interpolation to complete the data, generating a pseudo-3D geometric feature map. This method effectively utilizes changes in light and shadow to restore the spatial structure of the pollen surface, solving the problem of texture detail loss caused by depth-of-field limitations.

[0017] 2. This invention applies geodesic convolution units based on pseudo-3D geometric feature maps. It determines the curved path along the object's surface based on the direction of the local normal vector, and maps the sampling points of the convolution kernel from a rectangular grid to the curved path and contour trajectory to perform non-Euclidean spatial feature sampling. This sampling mechanism extracts surface geometric curvature features with rotation invariance, ensuring that feature information characterizing the geometric properties can still be accurately extracted when the two-dimensional pollen image undergoes rotational changes.

[0018] 3. This invention extracts two-dimensional texture features from target pollen images and fuses them with surface geometric curvature features to generate multimodal feature vectors. Using a Riemannian metric-based classifier, the geodesic distance between the feature vector and the class center point in the manifold space is calculated using a log-Euclidean metric algorithm. This method combines the advantages of two-dimensional texture and three-dimensional topology, and determines the species label by quantifying manifold differences, thereby improving the accuracy of pollen species identification. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 This is a flowchart illustrating a pollen image feature extraction and recognition method based on deep learning, provided in an embodiment of the present invention. Detailed Implementation

[0020] The following description provides numerous specific details to offer a deeper understanding of this application; however, it will be apparent to those skilled in the art that embodiments of this application can be implemented without one or more of these details. In other instances, certain technical features well-known in the art have not been described to avoid confusion with embodiments of this application.

[0021] Example 1: Please see Figure 1A pollen image feature extraction and recognition method based on deep learning includes the following steps: S1. Obtain the target pollen image to be identified and convert the target pollen image into a single-channel grayscale data matrix; S2. Input the single-channel grayscale data matrix into the surface reconstruction encoder in the surface geometric feature reconstruction network to generate a pseudo-three-dimensional geometric feature map characterizing the spatial structure of the target pollen surface; S3. Based on the pseudo-3D geometric feature map, geodesic convolution units are used to perform feature sampling in non-Euclidean space to extract surface geometric curvature features with rotation invariance. S4. Input the surface geometric curvature features into a Riemannian metric-based classifier, and use the classifier to calculate the geodesic distance of the feature vector corresponding to the surface geometric curvature features in the non-Euclidean space to determine the species label of the target pollen image.

[0022] This embodiment discloses a pollen image feature extraction and recognition method based on deep learning. The method aims to solve the technical problem in the prior art where the loss of topological structure information in two-dimensional microscopic pollen images due to shooting angle rotation and depth of field limitation leads to a decrease in recognition accuracy. The core logic of this embodiment is to map the target pollen image to be identified from the traditional pixel grayscale space to a pseudo-three-dimensional geometric feature map, and use the geodesic concept in differential geometry to replace Euclidean distance for feature extraction.

[0023] S1. Perform the data acquisition step by acquiring digital images containing pollen grains using an optical microscope as target pollen images to be identified; Since the color of pollen is greatly affected by dyes and light sources, and its surface texture, such as germination pores and spiky protrusions, is mainly reflected in changes in light and shadow, the target pollen image is converted into a single-channel grayscale data matrix. This matrix preserves the light intensity gradient information of the image and eliminates redundant interference in the color channels.

[0024] S2. Input the single-channel grayscale data matrix into the surface reconstruction encoder in the surface geometric feature reconstruction network to generate a pseudo-3D geometric feature map; the encoder is a deep neural network model constructed in this invention, and the specific architecture adopts a U-shaped network variant with residual connections, consisting of 4-level downsampling coding blocks and corresponding decoding blocks.

[0025] To ensure that those skilled in the art can accurately construct and reproduce this network, this embodiment specifies the following network structure parameters: Input single-channel grayscale data matrix Standardized adjustment Size; number of output feature channels for each coding block in the network. The values ​​were configured as 64, 128, 256, and 512 respectively; except for the photometric constraint layer which was initialized using a specific physical model, all other convolutional layers used the Ho's initialization method to optimize gradient propagation in deep networks.

[0026] To clarify the data flow and connection method, the downsampling operation in the downsampling coding block specifically uses a step size of 2. Convolutional layers are used to replace traditional max pooling layers, thereby reducing the loss of spatial information. Residual connections are implemented by adding elements one by one to connect the input and output of the coding block to alleviate the gradient vanishing problem. The corresponding decoding block is upsampled using bilinear interpolation and skips connections are established with the feature maps of the same level coding block through channel concatenation, thereby fusing shallow texture details with deep semantic features. Each coded block contains two Convolutional layers and rectified linear unit activation layers are used to extract lighting features at different scales and map them to a high-dimensional space, thereby generating pseudo-3D geometric feature maps. ; The pseudo-3D geometric feature map here is not a real 3D point cloud, but a high-dimensional feature tensor. Its channels encode the normal vector direction and relative depth potential of the pixel. Through this step, the network recovers the wrinkles and protrusions of the pollen surface from the 2D light and shadow.

[0027] S3. Based on the pseudo-3D geometric feature map, geodesic convolution units are used to perform feature sampling in non-Euclidean space; the geodesic convolution unit simulates the path extending along the geodesic trajectory of the object surface to extract surface geometric curvature features; these features refer to feature vectors that describe the geometric properties of pollen surfaces, such as Gaussian curvature and mean curvature.

[0028] Regardless of how pollen is rotated under a microscope, its surface topology, such as the number of pores and the density of spines, remains unchanged, thus this feature has inherent rotation invariance. It should be noted that the surface geometric curvature features output in this step are constructed as a feature map format with spatial dimensions to ensure that they correspond to the spatial position of the original image.

[0029] S4. Input the surface geometric curvature features into a Riemannian metric-based classifier. Traditional Euclidean distance cannot accurately measure the difference between two distributions in manifold space. Therefore, this step uses a Riemannian metric-based classifier to calculate the shortest path, i.e., the geodesic distance, between the sample and the class center in the curved feature space, thereby outputting the final class label. This embodiment successfully decouples topological features that are insensitive to rotation and viewpoint changes from a single two-dimensional microscopic image by constructing a pseudo-three-dimensional geometric feature map and applying geodesic convolution. In the microscopic imaging scenario, this method actually endows two-dimensional images with depth perception capabilities, so that the recognition algorithm no longer depends on the specific placement posture of pollen. Even with a limited sample size and without large-scale rotational data augmentation, this scheme can still achieve high-precision differentiation of closely related species by capturing the essential geometric invariants of the pollen surface, significantly reducing the requirements for the imaging specifications of micromanipulators.

[0030] Example 2: S2 includes the following sub-steps: S21. Using the photometric constraint layer in the surface reconstruction encoder, perform convolution operation on the single-channel grayscale data matrix; S22. Based on the preset Lambertian illumination reflection model constraints, the normal vector field distribution corresponding to the pixel is decoupled from the result of the convolution operation; S23. Calculate the depth value of each pixel based on the normal vector field distribution; S24. Channel-stitch the normal vector field distribution and depth values ​​to generate a pseudo-3D geometric feature map; S22 decouples the normal vector field distribution corresponding to the pixel, specifically including: setting a local neighborhood window centered on the pixel, calculating the pixel gradient variance within the local neighborhood window; comparing the pixel gradient variance with a preset texture threshold; if the pixel gradient variance is less than the texture threshold, it is determined that there is high-frequency texture loss in the local neighborhood window, and the pixel with the largest gradient variance is searched within the set neighborhood of the local neighborhood window as a reference pixel. Based on the Jacobian matrix of the normal vector field of the reference pixel, the normal vector direction of the local neighborhood window is completed by interpolation through differential operation; if the pixel gradient variance is greater than or equal to the texture threshold, the normal vector field distribution of the local neighborhood window remains unchanged.

[0031] This embodiment further specifies step S2, focusing on how to inversely reconstruct the surface normal vector based on single-view photometric constraints and handle the defocus problem; using the photometric constraint layer in the surface reconstruction encoder to perform convolution operation on the single-channel grayscale data matrix; in order to satisfy the interpretability of the physical model, the photometric constraint layer is not a randomly initialized ordinary convolution layer, but is initialized with a convolution kernel containing two specific weights, which are used to extract the light intensity gradients in the horizontal and vertical directions respectively; Specifically, horizontal gradient convolution kernel Initialize to the standard Sobel operator: Vertical gradient convolution kernel Initialize to It is worth noting that in the model training phase of step S03, these convolutional kernels are set as learnable tensor parameters, which allows the network to fine-tune the weights of the gradient operator through backpropagation to adapt to the diffraction characteristics of a specific microscope lens, thereby obtaining the optimal photometric gradient response while minimizing the loss of surface consistency. Its output is configured as a dual-channel feature map, representing the pixel points on the image plane. shaft and Physical intensity gradient components along the axial direction and , i.e., surface depth coordinates The partial derivatives; where, Represents the discrete row and column indices of the image matrix, with values ​​ranging from 1 to 2. The specific calculation formula for the above gradient components is as follows: in, This represents a two-dimensional convolution operation; although the convolution kernel is initialized to extract the light intensity gradient by the Sobel operator, after end-to-end photometric constraint training, this layer has learned and mapped the changes in light and shadow into geometric slope information that characterizes the three-dimensional shape of the surface. Based on the pre-defined Lambertian illumination reflection model constraints, the normal vector field distribution corresponding to each pixel is decoupled from the convolution operation result. To ensure the stability of the numerical calculation and clarify the source of the interpolation, this step adopts a two-stage calculation strategy of global coarse estimation and local fine refinement. The first stage, namely global coarse estimation: The system directly applies the Lambert model vector normalization formula as follows: in, Representing coordinates The rough normal vector at that location; Representing coordinates along Negative gradient component in the axial direction; : Represents coordinates along Negative gradient component in the axial direction; As a normalization factor, ensure that the generated normal vectors are unit vectors; generate an initial coarse normal vector field based on the gradient features of the entire image. At this point, we do not consider the effect of defocus, and assume that all pixels satisfy the ideal diffuse reflection condition. The second stage, local refinement: In order to deal with the common defocusing and blurring problem in microscopic images, the system traverses the coarse normal vector field. For each local window Size set to Pixel, calculate its pixel gradient variance The specific formula for calculating this variance is: in, This represents the total number of pixels within the window. For pixels The gradient magnitude at point is defined as follows: ; This is the average gradient magnitude of all pixels within the window.

[0032] In this preferred embodiment, for pollen samples with rich surface texture features, the variance is compared with a set texture threshold. Comparison, among which The threshold is determined by the 30th percentile value of the gradient variance distribution of the training set images. The basis for selecting the 30th percentile as the threshold is that, after statistical analysis of a large number of microscopic slide samples, about 30% of the area belongs to the background noise or smooth medium area without texture. The gradient variance of this area is mainly composed of sensor thermal noise. Setting the threshold at this point can effectively distinguish the high-frequency texture loss area from the original smooth background area. like The system determines that the region is a defocused area with lost high-frequency texture and performs differential interpolation: the system centers on the window containing the currently defocused pixel, and... Search within the neighborhood The largest pixel is used as the reference point and extract reference points In the rough normal vector field The values ​​in the table are used as a benchmark; To clarify the specific meaning of the light intensity gradient trend, in this embodiment, the trend is based on the light intensity gradient components. and The rate of change of the constructed normal vector field is used to characterize it; specifically, to avoid directly relying on the entire graph... Calculating partial derivatives can introduce gradient estimation biases that introduce erroneous information in the defocus region. This system employs a confidence region derivative migration strategy: i.e., a reference point... The selection of the value must satisfy the condition that the gradient variance is greater than a specific high signal-to-noise ratio threshold; this means that the calculation of the Jacobian matrix is ​​strictly limited to the reference point. Performed within a local neighborhood; Meanwhile, in order to suppress the interference of random noise in the original gradient field on the second-order differential calculation, the system pre-processes... Centered The local normal vector field is Gaussian smoothed, where .

[0033] Based on this smoothed high-confidence local field, using The Sobel operator performs convolution calculations, extracts the partial derivatives of each component of the normal vector along the image coordinate axes, and constructs the value at the reference point. Jacobian matrix That is, it includes There are 6 partial derivative terms; the formula is: Among them, the function This represents normalizing a vector to a unit vector; calculating the current defocus pixel. The corrected normal vector, this for The image coordinate column vector, and Multiplying the Jacobian matrices yields The normal correction quantity; in response to Then keep it directly. The values ​​in the data are used; after traversal correction, the final refined normal vector field distribution is output. ; Based on the normal vector field distribution, a three-layer cascaded transposed convolutional layer is used to fit the integration process, reconstruct the relative height, and obtain the depth value. The normal vector field distribution and depth values ​​are concatenated via channels to generate a pseudo-3D geometric feature map. The formula is as follows: in, This indicates a concatenation operation performed along the feature channel dimension; This represents the refined normal vector field distribution after decoupling and correction in step S22; This represents the depth value of each pixel obtained through integration and reconstruction in step S23.

[0034] This embodiment introduces a global coarse estimation-local fine refinement strategy, clarifying that the Jacobian matrix required for differential interpolation is calculated based on a smoothed, high-confidence local field, thereby solving the circular dependency problem in the variable calculation process and effectively combating the unavoidable depth-of-field blurring problem in optical microscopy.

[0035] Example 3: S3 includes the following sub-steps: S31. Traverse each feature point in the pseudo-3D geometric feature map and obtain the local normal vector at the current feature point; S32. Determine the surface path along the object's surface based on the direction of the local normal vector; S33. Map the sampling points of the convolution kernel from the rectangular grid distribution to the surface path to generate an adaptive convolution sampling point set; S34. Use the adaptive convolution sampling point set to perform weighted summation on the pseudo-3D geometric feature map and output the surface geometric curvature features; In S33, an adaptive convolution sampling point set is generated by performing the following operations: calculating the projection relationship between the local normal vector and the tangent plane; and dynamically adjusting the coordinate offset of the convolution kernel sampling points according to the projection relationship, so that the sampling points are distributed on the contour trajectory centered on the current feature point.

[0036] This embodiment describes in detail the specific implementation mechanism of geodesic convolution, that is, how to perform convolution sampling on a pseudo-surface; Traversing any feature point in the pseudo-3D geometric feature map Obtain its local normal vector And based on the direction of the local normal vector, determine the surface path along the object's surface; The sampling points of the convolution kernel are mapped from a rectangular grid distribution onto a curved path to generate an adaptive convolution sampling point set. Specifically, the coordinate offset of the convolution kernel sampling points is dynamically adjusted based on the projection relationship. This ensures that the sampling points are distributed along the contour line trajectory centered on the current feature point; prior to this, the feature point is defined. tangent plane at the location To satisfy the equation The set of vectors; the calculation formula is as follows: in, This is the local normal vector of the current feature point; The local mean curvature is calculated from the divergence of the normal vector field; These are the network training parameters; parameters Defined as a dimensionless scaling factor for the projection of the tangent plane, this embodiment sets... Used to maintain Defined as pixel grid coordinates with units of The dimensions of the length remain unchanged after projection; parameters As a weight for the curvature coupling term, it is initialized to And participate in backpropagation optimization along with the network; For the first Each sampling point is relative to the center point The coordinate offset; Original regular grid direction vector The three-dimensional upscaling form is specifically defined as: in, For standard The discrete sampling coordinate offset of the convolution kernel on the two-dimensional image plane, whose set of values ​​is defined as: This coordinate system has the current calculated pixel as its origin. ; To feature points The projection operator of the tangent plane is calculated using the following formula: Meanwhile, this embodiment implicitly defines a 1:1 mapping ratio between image pixel grids and three-dimensional spatial coordinates, meaning one pixel step corresponds to one unit length in three-dimensional space; regarding the physical rationality of dimensional matching in the formula, the parameters... Defined as the curvature space projection coupling coefficient, its physical dimension is set as area, and in digital image processing it is specified as pixel square. This coefficient aims to address the scale incompatibility issue between 3D curvature and 2D pixel mesh: given the local mean curvature. Having inverse length dimension By using a quantity with area as its unit of Multiplying them together yields a quantity with the dimension of length. displacement correction term This allows the term to be physically consistent with the tangent plane projection vector of the first term, thus correctly mapping the curvature properties of the three-dimensional surface to the coordinate offset of the image plane sampling points in mathematical logic. The second term in the formula Geometrically, this constitutes a second-order Taylor expansion approximation of the geodesic, that is, using the normal curvature vector to perform bending correction on the first-order linear prediction of the tangent space, so that the sampling points are closer to the real curved manifold, rather than just staying on the tangent plane. The geometric and physical significance of this formula lies in: the first term Responsible for aligning the planar mesh to the local tangent plane and handling viewpoint differences; the second item This introduces curvature-related normal compensation, which simulates pushing the sampling points along the normal direction to a geodesic level equipotential to the center point; The pseudo-3D geometric feature map is weighted and summed using an adaptive convolutional sampling point set to output the surface geometric curvature features. It should be noted that the above formula calculates... A vector in the tangent space of a three-dimensional manifold When performing feature map sampling, the system performs a projection mapping operation: only the planar components of the vector are taken. Superimposed on the current feature point Grid coordinates The actual sampling coordinates are obtained from the above, and the calculation formula is as follows: Regarding the specific implementation logic and rationality of the exponential mapping, this embodiment provides further mathematical derivation and disclosure: the above operation is essentially a specific implementation of the contraction mapping on the Riemannian manifold onto a single-valued height field surface; since the pollen surface is parameterized as: in, This represents the parameterized coordinate vector of the three-dimensional manifold on the pollen surface. Represents coordinates in the image plane The depth value at that location, i.e., the depth value generated in step S24 above. A point on any manifold uniquely corresponds to a coordinate in the image plane. The three-dimensional spatial displacement vector of the sampling point relative to the center point Represented as: It is a three-dimensional spatial displacement vector determined by the tangent plane and curvature correction; this vector is superimposed on... and discard The operation is not a simple dimensional truncation, but rather an inverse projection from three-dimensional Euclidean space to the two-dimensional parametric domain: in, : Represents the inverse projection mapping operator, used to define the functional relationship that projects a vector in three-dimensional space back to a two-dimensional plane; : Represents three-dimensional real Euclidean space, here referring to a space containing depth information. The pseudo-3D manifold feature space; : Represents two-dimensional real Euclidean space, here referring to the two-dimensional pixel coordinate parameter domain of the image. .

[0037] The mathematical proof regarding the technical effect of automatic shortening of planar displacement in steep slope areas is as follows: Let the length of the local tangent vector in three-dimensional space be... , representing the geodesic distance step size, whose projection length on the image plane is . Based on solid geometry, the calculation formula is as follows: in, For surface normal vector The angle with the optical axis; This represents cosine trigonometric function operations; in areas with steep slopes, the surface normal vector deviates from the Z-axis. Enlargement, leading to The sampling step size of the projected plane is reduced. It must be less than This geometric property ensures that although the sampling points become denser on the image plane, the geodesic distance between the sampling points remains constant on the curved three-dimensional pollen surface, i.e., equal to... This is precisely the fundamental reason why geodesic convolution can resist perspective distortion and achieve rotation invariance. The system employs a bilinear interpolation algorithm based on sampling points. The feature value at the non-integer coordinate is calculated from the four nearest integer grid pixel values, and then multiplied and added with the convolution kernel weights to obtain the output feature. This embodiment achieves omnidirectional isotropy of feature extraction by forcibly mapping convolutional sampling points onto contour lines based on surface normal vectors, thus eliminating the dependence of traditional convolutional neural networks on data augmentation.

[0038] Example 4: S4 includes the following sub-steps: S41. Extract the two-dimensional texture features of the target pollen image; S42. Fuse the two-dimensional texture features with the surface geometric curvature features to generate a multimodal feature vector; S43. In the Riemannian manifold space, calculate the Riemannian metric distance between the multimodal feature vector and the preset pollen class center points; S44. Select the category corresponding to the category center point with the smallest Riemann metric distance as the category label; The Riemann metric distance in S43 is calculated using a logarithmic Euclidean metric algorithm to quantify the manifold differences between different topologies.

[0039] This embodiment illustrates the specific construction of the classifier, particularly the metric computation in the Riemannian manifold space; and the extraction of two-dimensional texture features from the target pollen image. This step specifically uses a multi-scale Gabor filter bank to convolve and extract the original grayscale image. The filter bank design includes 4 directions and 3 frequency scales to generate a 12-channel feature response map as a two-dimensional texture feature to preserve high-frequency texture details in the image. The two-dimensional texture feature and the aforementioned surface geometric curvature feature Feature vector fusion is performed to generate multimodal feature vectors. ; Integrate the splicing operation based on the specific execution channel dimension: Let the number of texture feature channels be... The number of topological feature channels is This includes one channel of average curvature, one channel of Gaussian curvature, and three channels of principal direction field vectors. The fused... for dimensional vector; To prevent the differences in the numerical ranges of different modal features from affecting the subsequent calculation of the covariance matrix, the system performs separate checks on each modality before concatenation. and Execution layer normalization operation; To adapt to Riemannian metrics, the features in vector form need to be converted into points on the manifold, i.e., covariance descriptors in symmetric positive definite matrix form; specifically, the extracted texture features and surface geometric curvature features are feature maps that preserve spatial dimensions, and their size is denoted as . Here, and Defined as the spatial resolution of the original input image after its height and width have been downsampled by the network. This represents the total number of feature channels. To ensure that the covariance matrix has full rank and is statistically significant, the network structure is configured to ensure the total number of spatial pixels in the feature map, calculated as follows: It should be at least three times the feature dimension, i.e. To meet the degree of freedom requirements for covariance estimation; the system will assign each spatial location On A dimensional vector as an independent local descriptor Thus constructing a system containing A set of eigenvectors From multimodal feature vectors To multimodal feature matrix The conversion process is as follows: in, The transpose operator represents a matrix or vector, used to convert a column vector into a row vector for outer product operations; Represents the feature vector set The mean vector is calculated using the following formula: It was set to 1×10⁻⁵. This is the identity matrix; the regularization term not only ensures the positive definiteness of the matrix, but also prevents numerical overflow or computational instability caused by excessively small eigenvalues ​​in subsequent logarithmic operations, thus obtaining a robust symmetric positive definite matrix form of the multimodal eigenma matrix. ; In the Riemannian manifold space, the Riemannian metric distance between the multimodal eigenvectors and the predefined pollen class center points is calculated. A log-Euclidean metric algorithm is used here to quantify the manifold differences between different topologies. The specific calculation formula is as follows: in, The multimodal feature matrix of the image to be identified is obtained by fusing feature vectors and calculating covariance in step S42, and the coordinates of the current sample in the manifold space are also given. For the first The cluster center matrix of pollen-like organisms was calculated using the Riemann mean algorithm during the training phase. The Frobenius norm of a matrix is ​​used to calculate the square root of the sum of the squares of the matrix elements. This represents the matrix logarithm operation, specifically based on the natural logarithm base. After performing eigenvalue decomposition on a symmetric positive definite matrix, the natural logarithm of the eigenvalues ​​is calculated.

[0040] To ensure geometrical consistency with the log-Euclidean metric used in the classification phase, this embodiment abandons the traditional computationally complex iterative mean algorithm for affine invariant metrics. Instead, it employs a closed-form solution to calculate the log-Euclidean mean as follows: in, For sample index, For the first The total number of training samples of each class For the first in this category The feature matrix of each sample; this formula ensures the center point. It can minimize the sum of squared log-Euclidean distances to all samples in that class; Specifically, this is achieved through eigenvalue decomposition, that is, by decomposing a symmetric positive definite matrix. ,in, A diagonal matrix is ​​an orthogonal matrix composed of eigenvectors. Taking the natural logarithm / exponent of the diagonal elements yields Reconstruction ; The category corresponding to the category centroid with the smallest Riemannian distance is selected as the category label; that is, comparison. With all The distance, take the minimum value corresponding to For the recognition results; This embodiment employs a logarithmic Euclidean metric algorithm, which shifts the classification decision boundary from a flat Euclidean space to a curved Riemannian manifold. In the context of biological taxonomy, pollen from different subspecies often differs only in subtle topological structures, and these differences are often compressed or distorted in Euclidean space. Logarithmic Euclidean metric can accurately amplify the key geometric differences that determine species classification by mapping Riemannian manifold homeomorphisms to Euclidean tangent space and restoring the true geodesic distance between two points on the manifold surface, thus significantly improving the system's ability to identify closely related species.

[0041] Example 5: Before S1, there is also a model training step: S01. Construct an initial model containing a surface geometry feature reconstruction network; S02. Supervised training of the initial model is performed using sample images and their corresponding pollen species labels; S03. During the training process, a surface consistency loss function is introduced to constrain the normal vector field output by the surface reconstruction encoder to satisfy the integrability condition.

[0042] This embodiment describes the training process of the model, especially the importance of introducing the surface consistency loss function and the overall optimization objective; Construct an initial model that includes a surface geometry feature reconstruction network, and prepare a training dataset containing sample images and their corresponding pollen species labels; The initial model is trained under supervision using sample images and their corresponding pollen species labels; in this step, the system constructs the total loss function. The formula for calculating the parameters used for backpropagation update is as follows: in, To balance the hyperparameters of the classification task and geometric constraints, this embodiment sets it to . This value is an empirical value determined based on grid search experiments on the validation set, and the search range is set to... Step size is Experiments show that when At that time, poor surface reconstruction quality leads to high curvature feature noise; when At times, the network focuses excessively on geometric constraints and neglects classification discriminativeness; this setting can balance the classification loss. loss of surface consistency The magnitude difference prevents gradient dominance or training non-convergence due to an excessively large loss term. To address the issue of dimensionality compatibility between the Riemann metric distance and the cross-entropy loss function, this embodiment employs a distance-based negative exponential probability mapping mechanism; the multimodal feature matrix is ​​then calculated. With each category center Riemann distance: in, Let represent the logarithmic Euclidean metric function; the distance is converted into a predicted probability using the following formula: in, This represents the index of the specific pollen category currently undergoing probability prediction, with a value range of [value missing]. to ;symbol Used as a conditional probability separator; This indicates that, given a multimodal feature matrix Under the given conditions, the predicted category label of the target pollen image Belongs to the The posterior probability of pollen-like substances; This represents the index of the summation variable that iterates through all possible pollen types in the summation operation of the denominator, with values ​​ranging from... Traversal to ; This represents the total number of pollen types. Indicates the predicted category label; and These represent the multimodal feature matrices respectively. With the Class and No. Riemannian distance between pollen category centers; The bandwidth parameter for the radial basis functions is fixed at 5.0 in this embodiment to accommodate the large range of geodesic distances between symmetric positive definite matrices on a Riemannian manifold. Classification loss function The calculation formula is: in, Represents the true category label corresponding to the current input image; It should be noted that, in order to eliminate the interdependence of parameter updates between feature distribution and cluster centers, this embodiment focuses on the category centers. A dynamic update strategy using Riemann che space moving average was adopted; Specifically, at the initial moment All Initialize to identity matrix ; in the training In this iteration, updates are not performed directly via gradient descent. Instead, it calculates the number of items belonging to the current batch. The logarithmic Euclidean mean of the feature matrices of all samples of a class is denoted as . The global center is updated according to the following formula, which is calculated as follows: in, The momentum coefficient is set to 0.1; this strategy utilizes the linear properties of the logarithmic field to achieve smooth tracking of cluster centers in the manifold space, ensuring... Convergence; During training, the loss function is defined as follows: in, This is the surface consistency loss function, used to quantify the degree to which the normal vector field violates the surface gradient integrability constraint; For discrete pixel coordinate indices in the image plane; and The surfaces are respectively and The gradient component of the direction, which is related to the unit normal vector. The relationship is and This formula is based on Schwarz's theorem, which forces the surface gradient field to satisfy... This ensures that the normal vector field generated by the network can be integrated to recover a physically closed continuous surface.

[0043] Example 6: Surface geometric curvature features include at least one of the following: average curvature features characterizing surface protrusions, Gaussian curvature features characterizing surface pores and depressions, and principal curvature direction field features characterizing surface texture orientation.

[0044] This embodiment lists specific types of surface geometric curvature features, which are calculated based on the aforementioned pseudo-3D geometric feature map; specifically, surface geometric curvature features include: average curvature features. It characterizes the degree of local protrusion on the surface, such as the sharpness of hay fever acne; its calculation formula is as follows: in, It is the mean curvature and has the inverse length dimension. It is used to describe the degree of local protrusion or depression on the surface at the feature point; It is a divergence operator that characterizes the degree of divergence of a vector field; unit normal vector On the image plane coordinate axes and Component of direction; For partial differential operators along the horizontal and vertical directions of the image, in discrete image data they are typically implemented using the Sobel operator or difference convolution kernel; this feature is constructed at output as Single-channel feature map; Gaussian curvature characteristics Characterizing the intrinsic geometric properties of a surface, such as the depth of pores and depressions; its calculation formula is as follows: in, The projection components of the normal vector onto the tangent plane of The Jacobian matrix, in physical terms, is the area scaling factor of the normal vector mapping; This represents determinant operations, and this feature is also constructed in the output. Single-channel feature map; The principal curvature direction field feature characterizes the orientation of surface textures, such as brain-like grooves; this feature is obtained by applying the shape operator matrix. The shape operator is defined as follows: (Eigenvalue decomposition is performed.) Calculate matrix The two eigenvectors, namely the principal curvature directions, correspond to the directions of maximum and minimum surface curvature. To enable this orientation field feature to be effectively processed by subsequent neural network layers, this embodiment performs a vector lifting operation from the two-dimensional parameter space to the three-dimensional manifold space: calculate matrix The unit eigenvector corresponding to the maximum principal curvature This vector only represents the texture gradient direction on the image plane; using the tangent space basis vectors to... Mapped to principal curvature direction vectors in three-dimensional space ; Specifically, given the surface slope It is not directly included in the final manifold field; it is obtained here through inverse calculation: although the original These are intermediate operation variables, but the input data for this step—the pseudo-3D geometric feature map—is... The normal vector field is fully preserved. This provides a complete data foundation for reverse derivation, ensuring that the calculation can be performed; the specific calculation formula is as follows: in, Is the unit normal vector in The axial component, whose value represents the degree of tilt of the current pixel surface relative to the imaging plane, The closer to 1, the flatter the surface; the closer to 0, the steeper the surface. The numerical stability constant is set to . To prevent overflow during division by zero.

[0045] Constructing the 3D tangent vector: in, and Let A be the basis vector of the tangent space constructed from the surface slope. and Two-dimensional principal curvature direction vector In the image coordinate system shaft and The projection component weights along the axis are then normalized to obtain: in, The Euclidean norm of a vector or Norm operations are used to calculate vectors. The system will determine the magnitude of the three-dimensional vector. The three components Each feature channel is mapped to an independent feature channel, thereby generating The three-channel directional field feature map; therefore, combining the above three curvature features, the final output surface geometric curvature feature is... The total number of channels is .

[0046] This embodiment explicitly extracts these curvature features with clear geometric meaning, enabling the network to understand the microscopic topological geometric semantics of the pollen surface. In botanical identification scenarios, the spines, pores, and grooves on the pollen surface are key distinguishing features for differentiating species. Compared to traditional implicit feature extraction, this feature descriptor based on differential geometry has stronger interpretability and can keenly capture those minute morphological differences that are extremely difficult for human experts to discern, thus demonstrating superior performance in fine-grained classification tasks.

[0047] This application has been described through the above embodiments; however, it should be understood that the above embodiments are for illustrative purposes only and are not intended to limit this application to the described embodiments. Those skilled in the art will understand that many more variations and modifications can be made based on the teachings of this application, and all such variations and modifications fall within the scope of protection claimed in this application.

Claims

1. A method for pollen image feature extraction and recognition based on deep learning, characterized in that, Includes the following steps: S1. Obtain the target pollen image to be identified, and convert the target pollen image into a single-channel grayscale data matrix; S2. Input the single-channel grayscale data matrix into the surface reconstruction encoder in the surface geometric feature reconstruction network to generate a pseudo-three-dimensional geometric feature map characterizing the spatial structure of the target pollen surface; S3. Based on the pseudo-three-dimensional geometric feature map, feature sampling of non-Euclidean space is performed using geodesic convolution units to extract surface geometric curvature features with rotation invariance; S4. Input the surface geometric curvature features into a Riemannian metric-based classifier, use the classifier to calculate the geodesic distance of the surface geometric curvature feature vector in the non-Euclidean space, and determine the species label of the target pollen image; S2 includes the following sub-steps: S21. Using the photometric constraint layer in the surface reconstruction encoder, perform convolution operation on the single-channel grayscale data matrix; S22. Based on the preset Lambertian illumination reflection model constraints, the normal vector field distribution corresponding to the pixel is decoupled from the result of the convolution operation; S23. Calculate the depth value of each pixel based on the normal vector field distribution; S24. The normal vector field distribution and the depth value are concatenated by channels to generate the pseudo-3D geometric feature map.

2. The pollen image feature extraction and recognition method based on deep learning according to claim 1, characterized in that, S3 includes the following sub-steps: S31. Traverse each feature point in the pseudo-3D geometric feature map and obtain the local normal vector at the current feature point; S32. Determine the surface path along the object surface based on the direction of the local normal vector; S33. Map the sampling points of the convolution kernel of the geodesic convolution unit from the rectangular grid distribution to the surface path to generate an adaptive convolution sampling point set; S34. The pseudo-3D geometric feature map is weighted and summed using the adaptive convolution sampling point set to output the surface geometric curvature feature.

3. The pollen image feature extraction and recognition method based on deep learning according to claim 1, characterized in that, S4 includes the following sub-steps: S41. Extract the two-dimensional texture features of the target pollen image; S42. The two-dimensional texture features and the surface geometric curvature features are fused to generate a multimodal feature vector; S43. In the Riemannian manifold space, calculate the Riemannian metric distance between the multimodal feature vector and the preset center points of each pollen category; S44. Select the category corresponding to the category center point with the smallest Riemann metric distance as the category label.

4. The pollen image feature extraction and recognition method based on deep learning according to claim 1, characterized in that, The decoupling of the normal vector field distribution corresponding to the pixel in S22 specifically includes: A local neighborhood window is set with the pixel as the center, and the pixel gradient variance within the local neighborhood window is calculated. The pixel gradient variance is compared with a preset texture threshold. If the pixel gradient variance is less than the texture threshold, it is determined that there is high-frequency texture loss in the local neighborhood window. The pixel with the largest gradient variance is searched within the set neighborhood of the local neighborhood window as a reference pixel. Based on the Jacobian matrix of the normal vector field of the reference pixel, the normal vector direction of the local neighborhood window is completed by interpolation through differential operation. If the pixel gradient variance is greater than or equal to the texture threshold, the normal vector field distribution of the local neighborhood window is kept unchanged.

5. The pollen image feature extraction and recognition method based on deep learning according to claim 1, characterized in that, Prior to S1, a model training step is also included: S01. Construct an initial model that includes a network for reconstructing surface geometric features; S02. Supervised training of the initial model is performed using sample images and their corresponding pollen species labels; S03. During the training process, a surface consistency loss function is introduced to constrain the normal vector field output by the surface reconstruction encoder to satisfy the integrability condition.

6. The pollen image feature extraction and recognition method based on deep learning according to claim 2, characterized in that, In step S33, an adaptive convolution sampling point set is generated, specifically by performing the following operations: Calculate the projection relationship between the local normal vector and the tangent plane; Based on the projection relationship, the coordinate offset of the convolution kernel sampling points is dynamically adjusted so that the sampling points are distributed on the contour trajectory centered on the current feature point.

7. The pollen image feature extraction and recognition method based on deep learning according to claim 1, characterized in that, The surface geometric curvature features include at least one of the following: The average curvature characteristics of the surface protrusions, the Gaussian curvature characteristics of the surface pores and depressions, and the principal curvature direction field characteristics of the surface texture direction are used to characterize the surface protrusions.

8. The pollen image feature extraction and recognition method based on deep learning according to claim 1, characterized in that, The target pollen image in S1 is a two-dimensional projection image acquired by a microscopic imaging device at a single focal length, and the two-dimensional projection image includes a defocused area caused by depth of field limitations.

9. The pollen image feature extraction and recognition method based on deep learning according to claim 3, characterized in that, The Riemann metric distance in S43 is calculated using a logarithmic Euclidean metric algorithm to quantify the manifold differences between different topologies.

Images