A diatom fossil image segmentation extraction method based on radial profile pre-training ViT

Abstract

The application discloses a diatom fossil image segmentation extraction method based on a radial contour pre-training ViT, which comprises the following steps: generating a radial contour geometric data set with a spatial organization rule, pre-training a ViT backbone network to obtain geometric prior features; loading the pre-training weight to a target detection framework, reconstructing a multi-scale feature pyramid through a feature adaptation module; fine-tuning the network by using a real diatom data set with polygon annotation and a multi-task joint loss function; inputting a to-be-processed microscopic image into a model to infer a binary mask, and performing pixel-by-pixel spatial filtering to extract a fossil monomer image. The application effectively eliminates complex microscopic background interference, and realizes high-precision segmentation and zero-background automatic extraction of diatom fossils.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of artificial intelligence and paleontology, and in particular to a method for segmenting and extracting diatom fossil images based on radial contour pre-trained ViT. Background Technology

[0002] Diatoms, as widely distributed microscopic single-celled algae, have their siliceous shells preserved in strata as fossils for extended periods, holding significant scientific value in paleontology and sedimentary geology research. By analyzing the morphological and assemblage characteristics of diatom fossils in microscopic images, the ecological evolutionary process over geological history can be accurately reconstructed. With the rapid development of deep learning technology, computer vision has gradually been introduced into the analysis of microscopic paleontological images. Traditional object detection and segmentation methods are mostly based on convolutional neural network (CNN) architectures, often employing single-stage detection algorithms to locate small diatoms. In recent years, the Vision Transformer (ViT), with its superior long-range dependency and global feature modeling capabilities, has demonstrated great potential in general image recognition and segmentation tasks, providing a new technological direction for the efficient and automated processing of microscopic images.

[0003] However, existing microscopic image segmentation techniques still have significant shortcomings when applied to diatom fossils. First, microscopic images often contain interference such as mineral debris and phytoliths in the background. Existing general-purpose pre-trained models (such as those trained on ImageNet or COCO datasets) lack the ability to represent the low-level features of the specific regular geometric contours and internal ordered patterns of diatoms, making it easy to misclassify impurities with locally similar textures as targets, thus hindering the accurate extraction of morphological features. Second, conventional segmentation schemes struggle to completely remove complex background interference at the pixel level, resulting in low signal-to-noise ratios in the output samples and severely impacting the purity of the feature space in subsequent classification tasks. Furthermore, existing high-precision segmentation models often involve significant computational redundancy, making it difficult to balance inference efficiency with segmentation accuracy. Moreover, the process of obtaining standardized fossil samples from raw microscopic images still heavily relies on tedious manual image cutout, lacking an efficient end-to-end automated workflow. Summary of the Invention

[0004] The purpose of this section is to outline some aspects of embodiments of the present invention and to briefly describe some preferred embodiments. Simplifications or omissions may be made in this section, as well as in the abstract and title of this application, to avoid obscuring the purpose of these documents; however, such simplifications or omissions should not be construed as limiting the scope of the invention.

[0005] In view of the aforementioned existing problems, this invention is proposed. Therefore, this invention provides a diatom fossil image segmentation and extraction method based on radial contour pre-trained ViT to solve the problems mentioned in the background art.

[0006] To address the aforementioned technical problems, this invention provides the following technical solution: a method for segmenting and extracting diatom fossil images based on radial contour pre-trained ViT, comprising: Generate a radial contour geometric image dataset containing spatially organized regularity characteristics, and use the radial contour geometric image dataset as input to pre-train the Vision Transformer (ViT) backbone network to obtain pre-trained ViT weights with geometric prior features. The pre-trained ViT weights are loaded into the target detection framework, and the single-scale feature map output by the ViT backbone network is reconstructed through the feature adaptation module to build a multi-scale feature pyramid. The dataset of real diatom fossil microscopic images with polygon annotations is input into the target detection framework. The network weights are minimized and optimized using a multi-task joint loss function to complete the domain transfer fine-tuning from the general geometric space to the diatom fossil recognition task space, thus obtaining the trained diatom fossil segmentation model. The microscopic image to be processed is input into the diatom fossil segmentation model for inference, generating a binarized instance segmentation mask for the target region. The binarized instance segmentation mask is then mapped back to the original microscopic image coordinate system to perform pixel-by-pixel spatial filtering, extracting irregularly shaped diatom fossil individual images.

[0007] Compared with existing technologies, the beneficial effects of the invention are: 1. Since diatom fossils generally possess regular geometric contours and orderly internal decorative structures, this invention introduces a ViT model pre-trained using the Radial Contour Geometric Image Dataset (RCDB), enabling the backbone network to possess the geometric prior ability to recognize regular geometric features and spatial organization patterns. Compared to general pre-trained models, this approach can effectively distinguish diatom fossils from impurities with locally similar textures (such as mineral debris, phytoliths, etc.), improving the accuracy of target localization and boundary segmentation.

[0008] 2. In the inference stage, this invention generates a binary instance segmentation mask that closely fits the complex edges of diatoms, and uses it as a spatial filter to perform pixel-by-pixel dot product filtering with the original image. This completely eliminates various background interferences in the microscopic environment, achieves zero background in the output sample, and provides a pure feature space for subsequent fossil classification tasks.

[0009] 3. This invention seamlessly integrates the non-hierarchical ViT model, which possesses powerful global feature modeling capabilities, into mature object detection frameworks (such as Mask R-CNN) through the Feature Adaptation Module (ViTDet). While fully leveraging the long-range dependency advantage of Transformer, a multi-scale feature pyramid is constructed, reducing computational redundancy in the inference process and achieving a good balance between recognition speed and segmentation accuracy. Attached Figure Description

[0010] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein: Figure 1 This is a flowchart illustrating the overall process of a diatom fossil image segmentation and extraction method based on radial contour pre-trained ViT, as described in one embodiment of the present invention. Detailed Implementation

[0011] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0012] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0013] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0014] This invention is described in detail with reference to the schematic diagrams. When detailing the embodiments of this invention, for ease of explanation, the cross-sectional views illustrating the device structure may be partially enlarged, not adhering to the usual scale. Furthermore, the schematic diagrams are merely examples and should not be construed as limiting the scope of protection of this invention. In actual fabrication, the three-dimensional spatial dimensions of length, width, and depth should be included.

[0015] Furthermore, in the description of this invention, it should be noted that the terms "upper," "lower," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used solely for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. In addition, the terms "first," "second," or "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0016] Unless otherwise explicitly specified and limited, the terms "installation," "connection," and "joining" in this invention should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; similarly, they can refer to mechanical connections, electrical connections, or direct connections, or indirect connections through an intermediate medium, or internal connections between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances. Example 1

[0017] Reference Figure 1 This is the first embodiment of the present invention, which provides a method for segmenting and extracting diatom fossil images based on radial contour pre-trained ViT. The method employs a three-layer architecture combined with a fine-tuning engine, sequentially passing through a feature pre-learning layer, an object detection adaptation layer, and a precise segmentation and extraction layer from bottom to top. It also incorporates a model transfer fine-tuning engine to drive feature transfer from a general geometric space to a specific biological task space. Specifically, it includes the following steps: S1, Feature Pre-learning Layer: Generates a Radial Contour Geometric Image Dataset (RCDB) containing spatially organized geometric features, and uses the RCDB as input to pre-train the Vision Transformer (ViT) backbone network to obtain pre-trained ViT weights with geometric prior features.

[0018] Furthermore, based on the radial contour and its vertex evolution algorithm, a synthetic image set is generated through parameterized control, as follows: First, define the radial profile. Depend on The union of ... in, Indicates the first There are 3 polygons, each polygon is composed of 12 polygons. Composed of strips.

[0019] Secondly, by adjusting the positive The size and position of the vertices of the polygon generate the basic outline: in, This indicates the radial radius of the polygon. and These represent the x and y coordinates of the center point of the initial regular polygonal profile, respectively. They are used to define the center position of the radial profile base regular n-gon and are the reference parameters for generating the basic profile of the diatom shell. Represented as the first The coordinates of the vertex in the first iteration.

[0020] Finally, to simulate the different geometric patterns of diatom shells and the nonlinear edge deviations formed during natural growth or deposition, the vertices were modified by incorporating a random noise sequence. Expand outwards: in, Indicates the first The vertex at the th The coordinates at the next iteration The step length, This is the noise gain. and These are the noise gain coefficients along the x-axis and y-axis, respectively.

[0021] It should be noted that inputting the images from the aforementioned radial contour geometric image dataset into the ViT backbone network for pre-training, since the dataset has a regular geometric structure, can effectively enable the model to grasp the spatial distribution relationship and structural organization features of regular geometric patterns in the images, thereby enhancing the model's ability to perceive and extract features from the regular arrangement, regular contours, and internal repeating structural regions of diatom fossils.

[0022] S2, Object Detection Adaptation Layer: The pre-trained ViT weights are loaded into the object detection framework, and the single-scale feature map output by the ViT backbone network is reconstructed through the feature adaptation module (ViTDet adaptation module) to build a multi-scale feature pyramid (FPN).

[0023] It should be noted that traditional object detection frameworks (such as Mask R-CNN in Detectron2) rely on a hierarchical backbone network, while ViT adopts a non-hierarchical architecture with a global fixed resolution. Therefore, a feature adaptation mechanism is required for conversion.

[0024] Furthermore, from the single-scale feature map of ViT Construct a multi-scale feature pyramid The mathematical expression for the feature adaptation process is: in, Presentation layer normalization processing operation. This represents the convolution operation. This represents a transformation operation applied to different scale levels, and the rules for this transformation operation are as follows: Obtaining first-scale features It employs two deconvolution operations with a stride of 2 (transposed convolution).

[0025] Obtaining second-scale features : Use a deconvolution operation with a stride of 2.

[0026] Obtaining third-scale features : Use direct mapping operation (no scaling).

[0027] Obtaining fourth-scale features Use a max pooling operation with a step size of 2.

[0028] It should be noted that, through the above-mentioned scale adaptation, ViT is losslessly integrated into the Detectron2 framework to meet the multi-task requirements of subsequent target localization, bounding box regression, and instance segmentation.

[0029] S3, Model Transfer Fine-tuning Engine: Input the dataset of real diatom fossil microscopic images with polygon annotations into the object detection framework, and use the multi-task joint loss function to minimize and optimize the network weights, thereby completing the domain transfer fine-tuning from the general geometric space to the diatom fossil recognition task space.

[0030] It should be noted that after weight loading and architecture adaptation are completed, a polygonal supervised learning operator and a multi-task loss constraint are introduced using a fine-tuning engine to complete the domain transfer from the pre-learning layer to the recognition layer. By optimizing various parameters of the model, the consistency between the segmentation mask and the real physical edge is ensured, while improving the target recognition accuracy. In the scheme of this invention, polygonal contours are selected to accurately annotate the contours of diatoms, and training is performed on a real diatom dataset with polygonal annotations. This helps the model learn the geometric morphological features of diatom targets, enhancing its understanding and segmentation ability of target boundaries.

[0031] Specifically, on the diatom fossil dataset, a polygon-supervised learning operator is introduced to accurately label the edges of diatoms using polygonal contours. During the fine-tuning phase, the model simultaneously learns localization and segmentation, and its multi-task joint loss function... for: Furthermore, the calculation process for each part of the above formula is as follows: (1) Target discrimination loss : Used to constrain the model's ability to distinguish whether candidate regions belong to diatom fossil targets or background, employing a binary cross-entropy loss function: in, This represents the number of samples used in training. Indicates the first The true label of each sample This indicates the probability that the model predicts the sample to be a diatom fossil target.

[0032] (2) Bounding box regression loss : Optimize the positioning of the bounding rectangle of the polygon using smooth L1 loss: in, This represents the regression parameters of the u-th target bounding rectangle predicted by the model. This represents the regression parameters corresponding to the actual bounding boxes; and These represent the x and y coordinates of the center point of the circumscribed rectangle, respectively. and These represent the width and height of the circumscribed rectangle, respectively.

[0033] (3) Mask segmentation loss Calculate the binary cross-entropy loss for each pixel: in, pixels within the mask The true label, For the predicted probability, it is represented as the confidence score of the model predicting that the pixel belongs to the diatom fossil target. This is expressed as the side length (pixel dimension) of the output feature map.

[0034] S4. Precise Segmentation and Extraction Layer: The microscopic image to be processed is input into the trained diatom fossil segmentation model for inference, generating a binary instance segmentation mask for the target region, and performing pixel-by-pixel spatial filtering to extract the pure diatom fossil individual images with irregular shapes.

[0035] Specifically, during the inference phase, the model outputs a value of size [missing information]. floating-point matrix This represents the probability that each pixel belongs to a diatom fossil. It is activated using the Sigmoid function and a threshold is set. (Usually preset to 0.5) Generate binary mask B: in, It is the Sigmoid activation function, which ultimately activates by... Edge extraction yields the polygon boundary. It represents the binary mask value corresponding to the pixel with coordinates (i, j) in the original image coordinate system, and the value can only be 1 or 0; when the value is 1, it means that the pixel belongs to the diatom fossil target area, and when the value is 0, it means that the pixel belongs to the background area. It is the core spatial filtering parameter for achieving pixel-level accurate extraction of diatom fossils.

[0036] Furthermore, during the cropping stage, the inferred binary mask B is mapped back to the original high-resolution image coordinate system. Let the pixel matrix of the original microscopic image be... A binary mask B (with pixel values ​​of 1 for diatom fossil regions and a preset fixed value of 0 for background pixels) is used as a spatial filter and spatially multiplied pixel-by-pixel (Hadamard product) with the original image. The resulting purified diatom fossil image... The calculation formula is: in, This represents a pixel-by-pixel multiplication operation.

[0037] It should be noted that through this operation, only the pixels within the polygon outline retain their original color and texture information in the output image matrix, while all background pixels outside the outline (such as mineral debris, phytoliths, plant debris, etc.) are set to 0 (pure black) or transparent via the alpha channel.

[0038] It should be noted that this invention completely eliminates the interference of impurities in the microscopic environment, achieving a high signal-to-noise ratio and zero background in the output image, and providing an end-to-end automated solution for constructing a high-quality, standardized diatom fossil morphology database.

[0039] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of this application can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0040] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0041] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0042] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0043] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0044] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for diatom fossil image segmentation and extraction based on radial contour pre-trained ViT, characterized in that, include: Generate a radial contour geometric image dataset containing spatially organized regularity characteristics, and use the radial contour geometric image dataset as input to pre-train the Vision Transformer (ViT) backbone network to obtain pre-trained ViT weights with geometric prior features. The pre-trained ViT weights are loaded into the target detection framework, and the single-scale feature map output by the ViT backbone network is reconstructed through the feature adaptation module to build a multi-scale feature pyramid. The dataset of real diatom fossil microscopic images with polygon annotations is input into the target detection framework. The network weights are minimized and optimized using a multi-task joint loss function to complete the domain transfer fine-tuning from the general geometric space to the diatom fossil recognition task space, thus obtaining the trained diatom fossil segmentation model. The microscopic image to be processed is input into the diatom fossil segmentation model for inference, generating a binarized instance segmentation mask for the target region. The binarized instance segmentation mask is then mapped back to the original microscopic image coordinate system to perform pixel-by-pixel spatial filtering, extracting irregularly shaped diatom fossil individual images.

2. The diatom fossil image segmentation and extraction method based on radial contour pre-trained ViT as described in claim 1, characterized in that, Generating the radial contour geometry image dataset includes: Define a radial profile consisting of the union of multiple polygons, where each polygon consists of multiple edges; The basic outline is generated by adjusting the vertex size and position parameters of a regular polygon; A random noise sequence with a preset step length and noise gain parameter is iteratively added to the vertex position of the basic contour to expand the vertex outward and generate a synthetic image with nonlinear deviation of the edge and internal texture features, which is constructed as the radial contour geometric image dataset.

3. The diatom fossil image segmentation extraction method based on the radial profile pre-trained ViT according to claim 1, wherein, The object detection framework is a Mask R-CNN architecture based on the Detectron2 framework, which uses the pre-trained ViT weights as the backbone network of the Mask R-CNN architecture to replace the hierarchical convolutional neural network (CNN).

4. The diatom fossil image segmentation extraction method based on a radial profile pre-trained ViT according to claim 1, wherein, The multi-scale feature pyramid is constructed using a feature adaptation module, including: Four different levels of scaling operations were performed on the global fixed-resolution single-scale feature map output by the ViT backbone network to obtain four different resolution multi-scale features: The first-scale features are obtained by using two deconvolution operations with preset strides; The second-scale features are obtained by using a deconvolution operation with a preset stride. Third-scale features are obtained through direct mapping; Fourth-scale features are obtained using a max pooling operation with a preset step size.

5. The diatom fossil image segmentation and extraction method based on radial contour pre-trained ViT as described in claim 1, characterized in that, The multi-task joint loss function includes target discrimination loss, bounding box regression loss, and mask segmentation loss. The fine-tuning is achieved by simultaneously calculating the sum of the target discrimination loss, bounding box regression loss, and mask segmentation loss to update the various parameters in the model used for target localization and instance segmentation.

6. The diatom fossil image segmentation extraction method based on a radial profile pre-trained ViT according to claim 1, wherein, The process of constructing the dataset of real diatom fossil microscopic images with polygon annotations includes: Collect microscopic images of diatom fossils in real-world environments; Polygonal outlines are used to fit and annotate the complex physical edges of diatom fossils in microscopic images, generating polygonal labels to construct a polygonal annotation dataset for supervised learning.

7. The diatom fossil image segmentation extraction method based on a radial profile pre-trained ViT according to claim 5, wherein, The calculation process of the multi-task joint loss function includes: For target discrimination loss, a binary cross-entropy loss is calculated based on the probability that the model predicts the sample as a diatom fossil target and the true label. For bounding box regression loss, the smoothed L1 loss is calculated based on the center point coordinates, width, and height parameters of the target bounding box predicted by the model and the corresponding parameters of the actual labeled box. For mask segmentation loss, binary cross-entropy loss is calculated pixel-by-pixel based on the predicted probability and the true label of each pixel within the mask.

8. The diatom fossil image segmentation extraction method based on a radial profile pre-trained ViT according to claim 1, wherein, The generated binary instance segmentation mask for the target region includes: The diatom fossil segmentation model outputs a floating-point matrix representing the probability that each pixel belongs to a diatom fossil. The floating-point matrix is ​​non-linearly mapped using the Sigmoid activation function; The mapped floating-point matrix is ​​binarized according to a preset probability threshold to generate the binarized instance segmentation mask, and the edges of the binarized instance segmentation mask are extracted to obtain the polygon boundary.

9. The diatom fossil image segmentation extraction method based on a radial profile pre-trained ViT according to claim 1 or 8, wherein, Perform pixel-by-pixel spatial filtering operations, including: The binarized instance segmentation mask is used as a spatial filter, wherein the pixel state within the diatom fossil target area is defined as the preserved state, and the background pixel state outside the target area is defined as the zeroed state. Perform a pixel-by-pixel multiplication operation between the matrix of the spatial filter and the pixel matrix of the microscopic image to be processed.

10. The diatom fossil image segmentation extraction method based on a radial profile pre-trained ViT according to claim 9, wherein, Images of irregularly shaped diatom fossil cells were extracted, including: After the pixel-by-pixel multiplication operation, the original color and texture information of the pixels located within the polygon boundary contour are preserved. Set the values ​​of all background pixels located outside the polygon boundary outline to pure black, or set them to transparent by adding an alpha channel, to output a single image of diatom fossils without background features.

Images