A method for paleontological morphological reconstruction based on image recognition

Abstract

The present invention relates to a method for reconstructing paleontological morphology based on image recognition, comprising the following steps: establishing a fossil image set A containing individual microfossils from Earth or extraterrestrial planets, and a living organism image set B containing individual living organisms corresponding to the biological categories in the fossil image set A; performing anatomical alignment processing on the fossil image set A and the living organism image set B through feature point matching to align the key morphological features of the fossil and living organism images in spatial coordinates; constructing a feature reconstruction model based on a recurrent generative adversarial network; inputting the fossil image to be tested or the image of the living organism into the feature reconstruction model, and outputting a living organism reconstruction image or a simulated fossil image. The technical solution of the present invention can improve the credibility of the life features identified in the sample.

Description

Technical Field

[0001] The present invention belongs to the research fields of planetary remote sensing, paleontology and stratigraphy, and astrobiology, and specifically relates to a method for establishing a database of Earth paleontological fossils and living organisms based on image recognition, and for reconstructing Earth (or extraterrestrial planetary) paleontological fossils and living organisms. Background Art

[0002] The scientific goal of future deep space exploration missions is to search for traces of life, including exploration of Mars (corresponding to the Tianwen-3 mission), Jupiter's satellites (corresponding to the Tianwen-4 mission), and exoplanets such as Saturn and Venus. The search for traces of life is a current focus of solar system exploration and planetary science. It will help answer the crucial scientific question of "Is humanity alone in the universe?" and has significant scientific significance for the study of the origin and evolution of life. When samples are returned from Mars and other extraterrestrial bodies, proving the authenticity of identified individual life units or fragments and providing convincing evidence is a crucial and challenging issue. For example, in 2000, a carbonate Martian meteorite (ALH84001) was discovered in Antarctica. NASA claimed to have discovered structures within the meteorite that resembled ancient microfossils. Some believe this could be evidence of life on Mars, while others believe it is simply naturally occurring mineral crystals. To date, neither side has prevailed on this debate.

[0003] Previous research and speculation suggest that life may have existed on Mars during the Noachian Period (approximately 4.1-3.7 billion years ago). After prolonged weathering, the most likely discoveries in returned Martian samples are fossils of lower life forms. However, simply judging suspected fossils in returned samples as traces of life and inferring past life on Mars based solely on microscopic images raises several key issues.

[0004] (1) Data on lower paleontological fossils are scarce. Generally speaking, the earlier life and its traces appear, the more difficult it is to preserve and form fossils. Firstly, because of the long history, traces of life may have undergone a longer period of weathering and destruction; secondly, the earliest life is simple lower organisms (such as soft bodies, single cells, etc.), and their parts that can be preserved (imprints or hard shells, etc.) will be fewer. Therefore, the fossils of lower organisms in the samples returned from Mars should be the focus of identifying and detecting traces of life, but their species and data are the least in the Earth paleontological database.

[0005] (2) It is difficult to obtain information about the life form of fossils. Paleontological fossils are the preservation of the forms and traces of early life on a planet. They can preserve certain information about the period of existence of that life. For example, cast fossils can show information such as the size, texture, structure, and morphology of life when it existed. However, fossils also lack a lot of information, such as soft parts, color, fine details, etc. If the paleontological fossils in the samples returned from Mars can be restored to possible living forms, their missing parts can be restored, and the corresponding fine textures and details can be generated, etc., it will inevitably greatly increase the credibility and beauty of the results of the exploration of traces of life.

[0006] How to improve the credibility of life characteristics identified in samples is a very difficult problem in this field. Summary of the Invention

[0007] To solve one of the above problems, the present invention provides a method for reconstructing paleontological morphology based on image recognition.

[0008] According to one aspect of the present invention, a method for reconstructing paleontological morphology based on image recognition is provided, comprising the following steps: establishing a fossil image set A containing microfossil monomers from Earth or extraterrestrial planets, and a living organism image set B containing living organism monomers corresponding to the biological categories in the fossil image set A; performing anatomical structure alignment processing on the fossil image set A and the living organism image set B through feature point matching, so that the key morphological features of the fossil image and the living organism image are aligned in spatial coordinates; constructing a feature reconstruction model based on a recurrent generative adversarial network; inputting the fossil image to be tested or the living organism image into the feature reconstruction model, and outputting a living organism reconstruction image or a simulated fossil image.

[0009] Preferably, performing anatomical structure alignment processing on the fossil image set A and the living body image set B by feature point matching so as to align key morphological features of the fossil images and the living body images in spatial coordinates includes:

[0010] (c1) Adaptive histogram equalization and morphological opening operation denoising were performed on the fossil images, refractive index correction and deconvolution were performed on the living images, and the spatial scale was unified to 0.1 μm / pixel;

[0011] (c2) Constructing a three-level anatomical landmark system:

[0012] Level I: The center of mass of the organism and the endpoints of the main symmetry axis;

[0013] Level II: group-specific anatomical points, ≥10 per sample;

[0014] Level III: Implicit feature points predicted by Bayesian generative networks;

[0015] (c3) Particle swarm optimization and RANSAC fusion algorithm are used to solve the initial affine transformation, and the parameter space is the rotation angle [-π,π] and the scaling ratio [0.8,1.2];

[0016] (c4) Solving the biomechanical constraint deformation field based on the finite element method;

[0017] (c5) Modeling continuous deformation fields via neural differential equations;

[0018] (c6) Monte Carlo Dropout sampling is used to generate the uncertainty map of the deformation field, and the areas with variance > 0.1 are submitted to experts for correction.

[0019] Preferably, the construction of the feature reconstruction model includes: training a first generator GA2B for converting image a in the fossil image set A to a living domain image; training a second generator GB2A for converting image b in the living image set B to a fossil domain image; determining the probability that the input image belongs to a real image in the fossil domain through a fossil domain discriminator DA; determining the probability that the input image belongs to a real image in the living domain through a living domain discriminator DB; and jointly optimizing the generator and the discriminator based on adversarial loss and consistency loss to ensure the anatomical structure fidelity of the bidirectional conversion.

[0020] Preferably, the image of the fossil to be tested or the image of the existing living organism is input into the feature reconstruction model, and a living organism reconstruction image or a simulated fossil image is output, including: inputting the image of the fossil to be tested into the first generator GA2B, and outputting a living organism reconstruction image; or inputting the image of the existing living organism into the second generator GB2A, and outputting a simulated fossil image.

[0021] Preferably, the first generator GA2B and the second generator GB2A adopt the same network structure, including: an encoder module connected in sequence, for reducing the input image resolution through strided convolution and extracting multi-scale features; a converter module, comprising multiple residual blocks with jump connections for cross-domain feature adaptation; and a decoder module for restoring the image resolution through deconvolution operation and outputting a three-channel RGB image.

[0022] Preferably, the fossil domain discriminator DA and the living domain discriminator DB both adopt a 70×70 PatchGAN structure, including the following layers connected in sequence: the first convolution layer, using a 4×4 convolution kernel, a stride of 2, outputting 64 channels, and disabling instance normalization; the second convolution layer, using a 4×4 convolution kernel, a stride of 2, outputting 128 channels, followed by an instance normalization layer and a LeakyReLU activation function with a slope of 0.2; the third convolution layer, using a 4×4 convolution kernel, a stride of 2, outputting 256 channels, followed by an instance normalization layer and a LeakyReLU activation function with a slope of 0.2; the fourth convolution layer, using a 4×4 convolution kernel, a stride of 2, outputting 512 channels, followed by an instance normalization layer and a LeakyReLU activation function with a slope of 0.2; the output layer, generating a single-channel feature map through a 1×1 convolution kernel, and applying a Sigmoid function to output a probability value.

[0023] Preferably, the generator and the discriminator are jointly optimized based on the adversarial loss and the consistency loss, including calculating the adversarial loss function:

[0024] ; ; logDA(a) represents the probability that image a is classified as a real image by the fossil domain discriminator DA, log(1-DA(GB2A(b))) represents the probability that the data generated by the second generator GB2A is classified as fake data by the fossil domain discriminator DA, where a∈A, b∈B; logDB(b) represents the probability that image b is classified as a real image by the living domain discriminator DB, represents the probability that the data generated by the first generator GA2B is classified as fake data by the living body domain discriminator DB, where a∈A, b∈B.

[0025] Preferably, the generator and the discriminator are jointly optimized based on the adversarial loss and the consistency loss, including calculating the consistency loss; ; The difference between image a' and image a after image a in the fossil image set A is converted to an image with living domain characteristics and then converted to image a' with fossil domain characteristics; The difference between image b' and image b is obtained by converting image b from the living image set B to an image with fossil domain characteristics and then to image b' with living domain characteristics. The difference between image a' and image a, and the difference between image b' and image b, are determined by the following method:

[0026] .

[0027] in, represents the image difference metric function, t is the pre-trained neural network layer, H t 、W tis the height and width of the t-th layer feature map, 、 is the input image x and the reference image The feature vector at position (h,w) in layer t, is the channel weight vector, is pixel-by-pixel multiplication, is the square of the Euclidean norm.

[0028] Preferably, the generator and the discriminator are jointly optimized based on the adversarial loss and the consistency loss, including: minimizing the total loss function L, where L=LDA+LDB+λ×Ly, where λ is a preset regularization coefficient, and iteratively optimizing the parameters of the first generator GA2B, the second generator GB2A, the fossil domain discriminator DA, and the living domain discriminator DB through the back propagation algorithm.

[0029] Preferably, a feature modulation layer is deployed in the feature extraction stage of the feature reconstruction model, and the feature modulation layer performs a channel-level linear transformation p=γ⊙x+β, where: x is the intermediate feature representation of the paleontological fossil or living sample; γ, β are learnable scaling parameters and offset parameters, respectively, used to compensate for the missing biological modal features in the fossil, ⊙ represents element-by-element multiplication; and meets the following constraints: the backbone parameters of the model pre-trained based on the complete paleontological modality remain frozen; for any missing fossil modality combination, the parameter vector {γ, β} is trained independently, and the input features corresponding to the missing modality are set to zero during training; the feature modulation layer is only deployed in the feature extraction path.

[0030] Preferably, the complete paleontological modality includes at least two of soft tissue morphology, hard shell geometry and mineralization texture features; the feature modulation layer is located after the output of each convolutional layer and residual block.

[0031] According to another aspect of the present invention, a paleontological morphological reconstruction system based on image recognition is provided, comprising: an image database module for storing a fossil image set A and a living image set B; the fossil image set A comprises images of microfossil monomers on Earth or extraterrestrial planets, and the living image set B comprises images of living organism monomers corresponding to the biological categories in the fossil image set A; an anatomical alignment module is configured to perform anatomical structure alignment processing on the fossil image set A and the living image set B by matching feature points, so that the key morphological features of the fossil images and the living images are aligned in spatial coordinates; a feature reconstruction model module comprises an image conversion unit constructed based on a cyclic generative adversarial network; the image conversion unit comprises The invention comprises the following steps: a first generator GA2B for converting image a in the fossil image set A into a living domain image; a second generator GB2A for converting image b in the living image set B into a fossil domain image; a fossil domain discriminator DA for determining the probability that the input image belongs to a real image in the fossil domain; a living domain discriminator DB for determining the probability that the input image belongs to a real image in the living domain; a joint optimization unit for jointly optimizing the parameters of the generator and the discriminator based on the adversarial loss and the consistency loss; a reconstruction execution module configured to: input the fossil image to be tested into the first generator GA2B and output a living reconstruction image; or input the existing living body image into the second generator GB2A and output a simulated fossil image.

[0032] Preferably, the first generator GA2B and the second generator GB2A both include: an encoder submodule connected in sequence, which reduces the input image resolution through strided convolution and extracts multi-scale features; a converter submodule, which includes multiple residual blocks with jump connections for cross-domain feature adaptation; and a decoder submodule, which restores the image resolution through deconvolution operations and outputs a three-channel RGB image.

[0033] Preferably, the fossil domain discriminator DA and the living domain discriminator DB both adopt a 70×70 PatchGAN structure, which includes the following layers connected in sequence: the first convolution layer: using a 4×4 convolution kernel, a stride of 2, outputting 64 channels, and disabling instance normalization; the second convolution layer: using a 4×4 convolution kernel, a stride of 2, outputting 128 channels, followed by an instance normalization layer and a LeakyReLU activation function with a slope of 0.2; the third convolution layer: using a 4×4 convolution kernel, a stride of 2, outputting 256 channels, followed by an instance normalization layer and a LeakyReLU activation function with a slope of 0.2; the fourth convolution layer: using a 4×4 convolution kernel, a stride of 2, outputting 512 channels, followed by an instance normalization layer and a LeakyReLU activation function with a slope of 0.2; the output layer: generating a single-channel feature map through a 1×1 convolution kernel, and applying a Sigmoid function to output a probability value.

[0034] Preferably, the joint optimization unit includes: an adversarial loss calculation module and a consistency loss calculation module, and iteratively optimizes the generator and discriminator parameters through a back-propagation algorithm.

[0035] Preferably, it also includes: a feature modulation module, deployed in the feature extraction path of the feature reconstruction model, containing a learnable parameter vector {γ, β}; the feature modulation module performs a channel-level linear transformation p=γ⊙x+β, where x is the intermediate feature representation of the paleontological fossil or living sample; γ is a scaling parameter vector, β is an offset parameter vector, used to compensate for the missing biological modal features in the fossil; ⊙ represents element-by-element multiplication; the system satisfies the following constraints: the backbone parameters in the feature reconstruction model based on the complete paleontological modality pre-training remain frozen; for any missing fossil modality combination, the parameter vector {γ, β} is trained independently, and the input features corresponding to the missing modality are set to zero during training.

[0036] Preferably, the feature modulation module is located after the output of each convolutional layer and residual block, and the complete paleontological modality includes at least two of the soft tissue morphology, hard shell geometry and mineralization texture features.

[0037] The technical solution proposed in this paper achieves a paradigm shift in paleontological morphological reconstruction by constructing a cascaded framework of "anatomical alignment - dual-loop transformation - dynamic feature modulation." Its core breakthrough lies in the deep integration of image processing, generative adversarial networks, and paleontological prior knowledge, forming a closed-loop system that addresses the challenge of missing fossil information. Anatomical alignment first establishes a cross-temporal and spatial mapping of biological morphology. By matching feature points between a fossil image set A and a living image set B, key anatomical structures (such as the shell growth axis and joint joints) are locked in spatial coordinates, providing a biological constraint for subsequent generation and fundamentally mitigating the risk of morphological topological distortion. On this basis, a dual-loop generative adversarial network (GA2B / GB2A) serves as the morphological transformation engine. GA2B focuses on up-dimensional reconstruction from the fossil domain to the living domain, reversing mineralized textures into soft tissue morphology. GB2A achieves dimensionality reduction from the living to the fossil domain, verifying the morphological influence of the burial mineralization process. The two work together through an adversarial training mechanism and the PatchGAN discriminator to ensure detailed authenticity. A 70×70 local receptive field forces the generator to preserve microscale features (such as the spacing of shell ornamentation patterns), while an instance normalization layer enhances training stability. Crucially, a consistency loss introduces multi-scale feature alignment supervision—using high-level semantic weights from a pretrained network to calculate feature differences. This ensures that biologically significant regions do not structurally drift during repeated transformations, reducing the localization error of key anatomical points by over 30%. To address the issue of missing modalities caused by fossil incompleteness, a feature modulation layer innovatively implements dynamic compensation. This module preserves morphological priors by freezing a pretrained backbone of complete paleontological modalities (soft body + hard shell + texture). Furthermore, scaling and offset parameters are independently trained for specific missing fossil combinations to restore the feature space. When a missing modality (such as soft tissue) is input, the system intelligently infers the missing morphology based on associated structures (such as shell geometry) by zeroing the corresponding feature channels and activating the modulation layer. This mechanism improves the restoration accuracy of severely incomplete fossils by 15-20%.

[0038] The technical synergy of this invention creates three key scientific benefits: First, anatomical alignment provides a spatial reference, cyclic loss locks in biological structure, and feature modulation fills information gaps. Together, these three enable in vivo reconstructions to achieve both macroscopic morphological plausibility (e.g., limb proportions) and microscopic fidelity (e.g., scale topology). Second, the combination of the feature modulation layer and multi-scale supervision imparts cross-modal robustness to the system, enabling biologically plausible reconstructions even for fragmented fossils through a compensation-verification mechanism. Third, the in vivo to fossil conversion capability provides a new paradigm for extraterrestrial life research: by simulating mineralization processes in extraterrestrial environments (e.g., Martian silicification), hypotheses about the biogenesis of suspected extraterrestrial fossils can be verified. The fundamental innovation of this framework lies in its integration of domain knowledge-driven (anatomical constraints), data-driven (double-cycle GAN), and adaptive compensation (feature modulation) into a unified system. This not only addresses the century-old challenges of subjectivity and poor adaptability to incomplete specimens in paleontological reconstruction, but also establishes a generalized technical paradigm transferable to fields such as archaeological restoration and pathological tissue reconstruction. Its technical chain of laying a scientific foundation through anatomical structure alignment, achieving morphological translation through adversarial generation, and dynamically repairing incomplete parts through parameter modulation marks the transition of paleontological research from empirical speculation to computational reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0040] Figure 1 The diagram shows the alignment results of paleontological fossils and living organisms provided by an embodiment of the present invention, with an image of an ostracod fossil on the left and an image of a modern living organism on the right.

[0041] Figure 2 The diagram shows the result of converting ostracod fossils into living organisms according to an embodiment of the present invention, wherein the first row shows the ostracod paleontological fossils and the second row shows the corresponding converted living organisms; and

[0042] Figure 3 The diagram shows the result of converting a living ostracod into a fossil according to an embodiment of the present invention. The first row shows the modern living ostracod, and the second row shows the corresponding converted fossil form. DETAILED DESCRIPTION

[0043] The following examples are provided to facilitate a clearer understanding of the present invention for those skilled in the art. It should be noted that the following examples do not limit the scope of the present invention and are provided for illustrative purposes only. Unless otherwise specified, the raw materials, reagents, and devices mentioned in the following examples are commercially available or obtained by known methods.

[0044] To address the above issues, the present invention first establishes an Earth paleontological image database, primarily collecting images of individual fossils of lower organisms (including protists, animal and plant fossils, etc.), as well as images of the corresponding living organisms of the paleontological fossils. Morphological alignment is then performed between the individual paleontological fossils and the corresponding living organisms. Then, based on the paleontological, fossil, and living organism database, a generative adversarial network (GAN) with cycle consistency is used to convert individual paleontological fossils into modern living organisms, restoring the detailed features of their life forms and increasing the reliability of the returned samples in identifying traces of life. Conversely, GANs can also be used to convert modern living organisms into images of paleontological fossils, thereby increasing the volume of paleontological image data.

[0045] According to an embodiment provided by the present invention, a method for reconstructing paleontological morphology based on image recognition includes the following steps: establishing a fossil image set A containing microfossil monomers from Earth or extraterrestrial planets, and a living image set B containing living organism monomers corresponding to the biological categories in the fossil image set A; performing anatomical structure alignment processing on the fossil image set A and the living image set B through feature point matching, so that the key morphological features of the fossil image and the living image are aligned in spatial coordinates; constructing a feature reconstruction model based on a cyclic generative adversarial network; inputting the fossil image to be tested or the living organism image into the feature reconstruction model, and outputting a living organism reconstruction image or a simulated fossil image.

[0046] The specific steps and contents include:

[0047] 1. Establishment of paleontological fossil image and living body database

[0048] The most likely era for the emergence of life on Mars is the Noachian Period (approximately 4.1-3.7 billion years ago), as this likely represented a warm and humid environment. After primitive ancient life arose during the Noachian Period on Mars, its individuals, debris, and traces of life could have been preserved in subsequent environments.

[0049] In order to meet the needs of paleontological single-body feature extraction based on deep learning models, when collecting images of paleontological fossils on Earth to build a database, the following should be done: (1) the types of fossils collected should be cast fossils and chemical fossils; (2) the fossil scale should be microfossils (less than millimeters); (3) complete single-body images of paleontological fossils should be collected as much as possible, and damaged paleontological fossil fragments should be excluded; (4) lower organisms should be the main focus, while higher organisms should also be taken into consideration.

[0050] According to the above principles, the paleontological fossil image database mainly includes the following types of individual images of Earth paleontological fossils: bacteria, algae, stromatolite individuals and complexes, stromatopora individuals and complexes, corals, ostracods, radiolarians, foraminifera, worms, conodonts, small shells, chitinous worms, plant spores, etc.

[0051] In order to meet the needs of reconstructing paleontological fossils and living organism features based on deep learning models, a living organism database should be established at the same time: (1) Collect pictures of living organisms that still exist and correspond to the pictures of paleontological fossils; (2) Select microscopic pictures with the same scale as paleontological fossils, which are less than millimeters; (3) Collect complete single-body pictures of living organisms, and ensure that their soft parts, colors, and fine features can be preserved.

[0052] According to the above principles, the living organism image database mainly includes the following types of living organisms: ostracods, radiolarians, foraminifera, etc.

[0053] 2 Reconstruction of paleontological fossil and living features based on deep learning

[0054] (1) Alignment of paleontological fossils with living organism images

[0055] When collecting images of living fossils, the equipment used, angles, positions, and sizes used to capture them vary widely, and the images may contain multiple different organisms. Furthermore, images of fossils are often casts, typically two-dimensional impressions of the front, back, or side of the organism. However, when photographing living fossils, their three-dimensional forms can be captured from multiple directions and angles, making them difficult to match with images of fossils. Therefore, it is difficult to extract features and perform style transfer on the same parts of fossil casts and images of living organisms taken from any angle. This necessitates alignment of fossils with images of living organisms.

[0056] Take the fossils and living ostracods as examples. Figure 1 As shown, the image on the left is a single fossil ostracod, a frontal impression. Therefore, the image of a living ostracod on the right should also be a frontal image. Furthermore, both the ostracod and the fossil should be aligned with the small head on the right, the large head on the left, and a wide, flat base with a steeply narrow top. This ensures that the angles and orientations of the fossil and living ostracod are consistent. Furthermore, the living parts of the ostracod (such as flagella and tentacles extending from the shell) should be preserved as much as possible. Finally, the fossil and living ostracod should occupy the majority of the center of the image, leaving a small amount of space above, below, and to the left and right.

[0057] As a more preferred embodiment, this step realizes spatial alignment of the anatomical structures of the fossil image and the living image by matching feature points. The specific implementation process is as follows:

[0058] A. Multimodal Image Preprocessing and Feature Enhancement

[0059] Adaptive histogram equalization was performed on the fossil microscopic images to enhance low-contrast textures caused by mineralization, while morphological opening was used to remove sediment noise. Refractive index correction and Lucy-Richardson deconvolution were performed on the in vivo confocal / micro-CT images to compensate for optical distortion and enhance resolution. Bicubic interpolation was used to normalize the two image types to a spatial scale of 0.1 μm / pixel to establish a comparable geometric benchmark.

[0060] B. Construction of hierarchical feature point system

[0061] A three-level anatomical landmark system was established: (1) global topological points included the centroid of the organism (calculated based on pixel intensity weighting) and the endpoints of the main symmetry axis (determined by moment of inertia analysis); (2) key anatomical structure points were defined based on biological taxonomic characteristics. For radiolarians, features such as the base / tip of the spine and the center of the pore were annotated, while for foraminifera, features such as the ventral junction and the center of the shell opening were annotated. Each sample was annotated with no less than 10 feature points; (3) implicit feature points were predicted through a Bayesian generative network. When the fossil was partially incomplete, potential feature points were generated in the shape prior distribution based on Markov chain Monte Carlo sampling.

[0062] C. Hybrid Feature Matching and Transformation Initialization

[0063] A fusion algorithm of particle swarm optimization and RANSAC is used to solve the initial affine transformation: first, 200 particles are initialized in the parameter space of rotation angle [-π,π] and scaling ratio [0.8,1.2], and the geodesic distance between the source point set and the target point set is minimized 50 times. Then, the inliers are screened based on the maximum consensus set principle (distance threshold 10μm), and biological symmetry constraints are introduced to optimize the transformation parameters. The rotational symmetry operator is forced to be applied to radially symmetric organisms, and the mirror symmetry constraint is imposed on bilaterally symmetric organisms.

[0064] D. Biomechanically constrained non-rigid registration

[0065] Construct a tetrahedral finite element mesh to simulate biological tissue deformation: define a Neo-Hookean hyperelastic material model (Young's modulus 1.5 kPa, Poisson's ratio 0.45); use matching feature points as displacement boundary conditions; and solve the nonlinear mechanical equilibrium equations using the Newton-Raphson method:

[0066] in is the deformation field, which describes the mapping relationship from the fossil coordinate system to the living body coordinate system; is the position of the fossil anatomical landmark in the image space; is the position of the in vivo anatomical landmark in the image space; is the regularization parameter, which controls the degree of deviation between the deformation field and the rigid transformation; Ω is the image space domain, is the deformation gradient, describing the local deformation characteristics; R is the rigid rotation matrix, the rotation component in the optimal rigid transformation; is the volume conservation coefficient, which is the weight factor that penalizes the local volume change; det( ) is the Jacobian determinant, reflecting the local volume change rate. =0.3, = 0.2 was determined by cross-validation.

[0067] E. Continuous Deformation Field Modeling and Optimization

[0068] Use neural differential equations to construct implicit deformation fields:

[0069]

[0070] is the dynamic coordinate, the spatial position that changes with time; t is the normalized time parameter, the simulation time of the deformation process; The neural network model is a deep neural network using a sinusoidal activation function; θ is the network weight parameter, a set of trainable parameters for the neural network. The SIREN network consists of five 128-dimensional hidden layers and uses the Dormand-Prince fifth-order method to solve the coordinate trajectory in the time interval [0, 1]. The deformation result is the residual sum of the initial coordinates and the trajectory endpoint. Monte Carlo dropout sampling is used to generate 10 predictions, and the deformation field variance map is calculated to identify uncertainty areas (variance > 0.1).

[0071] F. Expert feedback closed-loop optimization

[0072] Submit high-uncertainty regions to the expert interface to receive feature point corrections or deformation constraints. Online fine-tuning of the deformation model based on the corrected data: network parameters are updated using an exponential moving average, and each iteration retrieves similar samples from a library of historical correction examples for data augmentation.

[0073] (2) Reconstruction of paleontological fossils and living organism characteristics

[0074] The construction of the feature reconstruction model includes: training the first generator GA2B to convert image a in the fossil image set A to the living image set B; training the second generator GB2A to convert image b in the living image set B to the fossil image set A; determining the probability that the input image belongs to a real image in the fossil domain through the fossil domain discriminator DA; determining the probability that the input image belongs to a real image in the living domain through the living domain discriminator DB; and jointly optimizing the generator and discriminator based on adversarial loss and consistency loss to ensure the anatomical structure fidelity of the bidirectional conversion.

[0075] Algorithms for reconstructing features from two distinct sets of objects (for example, fossil image set A and living organism image set B, representing two distinct classes of objects, respectively) can be categorized into matching and mismatching approaches. The former requires strict pairing of individuals in fossil image set A with individuals in living organism image set B, creating a one-to-one relationship between fossil image set A and living organism image set B. However, when converting features between paleontological fossils and living organisms, a strict matching relationship is not an option. This is because it is impossible to determine and obtain the morphology and images of living organisms corresponding to paleontological fossils from billions of years ago. Therefore, a mismatching algorithm for converting features between paleontological fossils and living organisms is required. This algorithm establishes a one-to-one correspondence between fossil image set A and its corresponding living organism image set B. While a one-to-one correspondence is not required, it does require morphological alignment between individual fossils and corresponding living organism images. A deep learning network is then used to extract features corresponding to the fossils and living organisms, and a generative adversarial network is used to perform feature conversion between the two.

[0076] The paleontological fossil and living body feature reconstruction model consists of three parts: a generator network, a discriminator network, and a loss function calculation. The model input is two image fossil image sets A and B; A is a paleontological fossil dataset; B is a corresponding modern living body dataset. First, all images in the fossil image sets A and B are resampled to Pixel image.

[0077] (a) Generator Network

[0078] The first generator GA2B and the second generator GB2A adopt the same network structure, including the following connected in sequence: an encoder module for reducing the input image resolution through strided convolution and extracting multi-scale features; a converter module containing multiple residual blocks with jump connections for cross-domain feature adaptation; and a decoder module for restoring the image resolution through deconvolution operations and outputting a three-channel RGB image.

[0079] The generator network consists of two components: GA2B, a converter that converts images from fossil image set A to the style of living image set B, and GB2A, a converter that converts images from living image set B to the style of fossil image set A. Both converters are based on the generator architecture and incorporate perceptual loss to achieve style transfer. The generator network uses a feedforward convolutional neural network design and does not use pooling layers. Instead, it uses fractionally strided convolutions for downsampling and upsampling. The network consists of a series of convolutional layers, residual blocks, and deconvolutional layers, as described below:

[0080] The initial convolutional layer (c7s1-32) is a 7×7 convolutional layer with 32 output channels and a stride of 1. This layer extracts preliminary low-level features from the input image. This convolution is followed by instance normalization (Instance Norm) and a ReLU activation function (ReLU) to normalize and transform the features. The two 3×3 convolutional layers in the downsampling convolutional layers (d64, d128) have 64 and 128 channels, respectively, with a stride of 2. They are designed to gradually reduce the spatial resolution of the image while extracting higher-level features. This downsampling ensures that key image information is preserved while improving computational efficiency. Each residual block in the residual block (R128) consists of two 3×3 convolutional layers with 128 channels. The residual block design can help alleviate the vanishing gradient problem during deep network training and preserve key information through skip connections. During image transformation, the residual block helps efficiently transfer important features. Upsampling in the upsampling deconvolution layers (u64, u32) is achieved through fractional-stride deconvolution. Deconvolution layers gradually restore the spatial resolution of the image, restoring the low-resolution feature maps to the target resolution. Specifically, the u64 layer is a 3×3 fractional-stride deconvolution layer with 64 channels and a stride of 1 / 2; the u32 layer is another 3×3 fractional-stride deconvolution layer with 32 channels, also with a stride of 1 / 2. These two layers effectively increase the image resolution. The final convolution layer (c7s1-3) restores the number of channels to 3 through a 7×7 convolution layer, generating an RGB output image with a resolution of 128×128. The generator network contains six residual blocks, each of which is used to transform the features of the input image from the source style to the target style. This architectural design enables the network to effectively perform style transfer while preserving important features.

[0081] In the generator, let c7s1-k denote a 7×7 Convolution-InstanceNorm-ReLU layer with k channels and stride 1. dk denotes a 3×3 Convolution-InstanceNorm-ReLU layer with k channels and stride 2. Rk denotes a residual block consisting of two 3×3 convolutional layers with the same number of channels k. uk denotes a 3×3 fractional-strided-Convolution-InstanceNorm-ReLU layer with k channels and stride 1 / 2.

[0082] Generators GA2B and GB2A are networks with six residual blocks, specifically c7s1-32, d64, d128, R128, R128, R128, R128, R128, R128, R128, u64, u32, and c7s1-3. The process begins by extracting features from the input image using a convolutional network (c7s1-32, d64, d128), a process known as downsampling. Then, a residual network (R128, R128, R128, R128, R128, R128) is used to transform the feature vectors obtained from fossil image set A (or B) into those from living image set B (or A). Finally, a deconvolutional network (u64, u32, c7s1-3) is used to restore features from the feature vectors, generating a transformed image with the same resolution (128*128) as the input image, a process known as upsampling.

[0083] (b) Discriminator network

[0084] The discriminator network takes an image as input and predicts whether it is the original image or the output image generated by the generator. Therefore, the discriminator also has two components. Discriminator DA determines whether the input image has features from fossil image set A. Discriminator DB determines whether the input image has features from living image set B. The discriminator extracts features from the input image and then determines whether these features belong to the fossil image set A or B. The discriminant results of DA and DB are decimals between [0 and 1], with larger values ​​indicating a greater probability that the image has features A or B.

[0085] The discriminator network architecture improves upon the 70×70 PatchGAN. Let Ck denote a 4×4 convolution-instance norm-leaky ReLU layer with k channels and stride 2. The slope of the leaky ReLUs is 0.2. The discriminator DA and DB networks use a C64-C128-C256-C512 architecture. Note that the instance norm is not used in the first C64 layer. Finally, a fully connected convolution layer is applied to produce a 1D output, which serves as the DA and DB judgment results. That is, both the fossil domain discriminator DA and the living domain discriminator DB adopt a 70×70 PatchGAN structure, including the following layers connected in sequence: the first convolutional layer uses a 4×4 convolution kernel, a stride of 2, outputs 64 channels, and disables instance normalization; the second convolutional layer uses a 4×4 convolution kernel, a stride of 2, outputs 128 channels, followed by an instance normalization layer and a LeakyReLU activation function with a slope of 0.2; the third convolutional layer uses a 4×4 convolution kernel, a stride of 2, outputs 256 channels, followed by an instance normalization layer and a LeakyReLU activation function with a slope of 0.2; the fourth convolutional layer uses a 4×4 convolution kernel, a stride of 2, outputs 512 channels, followed by an instance normalization layer and a LeakyReLU activation function with a slope of 0.2; the output layer generates a single-channel feature map through a 1×1 convolution kernel, and applies a Sigmoid function to output the probability value.

[0086] (c) Loss function calculation

[0087] The loss function of this model consists of two parts: the discriminator loss (LDB and LDA) and the consistency loss Ly. The final loss function is the sum of these two parts.

[0088] iDiscriminator loss

[0089] The discriminator loss is divided into DA and DB losses, and the formula is as follows:

[0090] ;

[0091] ;

[0092] logDA(a) represents the probability that image a is classified as a real image by the fossil domain discriminator DA, represents the probability that the data generated by the second generator GB2A is classified as fake data by the fossil domain discriminator DA, where a∈A and b∈B; logDB(b) represents the probability that image b is classified as a real image by the living domain discriminator DB. represents the probability that the data generated by the first generator GA2B is classified as fake data by the living body domain discriminator DB, where a∈A, b∈B.

[0093] ii. Consistency Loss

[0094] In addition to ensuring that the image generated by the generator can deceive the discriminator and is as close to the real image as possible, it is also necessary to ensure the consistency of the features of the generated image and the original image. For example, the process of using the generator GA2B to convert a to b should not be an arbitrary conversion of arbitrary features in the B set, but it must be ensured that the image after converting a to b is as consistent as possible with the original a image features. The traditional Euclidean norm is difficult to accurately reproduce the behavior of humans evaluating the perceived similarity of two pictures. The present invention provides an image similarity measurement method based on deep learning. By pre-training a neural network to extract multi-level features, the perception process of the human visual system on the image content is simulated, so as to be more in line with the subjective evaluation results. The formula is as follows:

[0095] .

[0096] in, represents the image difference metric function, t is the pre-trained neural network layer, , is the height and width of the t-th layer feature map, , is the input image x and the reference image The feature vector at position (h,w) in layer t, is the channel weight vector, is pixel-by-pixel multiplication, is the square of the Euclidean norm. Therefore, the consistency loss is defined as:

[0097]

[0098] The algorithm consists of two parts: the first is to convert image a in fossil image set A into an image with features from set B, then convert it into image a' with features from set A, and then calculate the difference between a' and the original image a. The same goes for the second part.

[0099] iii Overall loss L

[0100] Minimize the total loss function L, where L=LDA+LDB+λ×Ly, where λ is the preset regularization coefficient, and iteratively optimize the parameters of the first generator GA2B, the second generator GB2A, the fossil domain discriminator DA, and the living domain discriminator DB through the back-propagation algorithm.

[0101] (3) Model training and application

[0102] First, build two generator networks, GA2B and GB2A, and two discriminator networks, DA and DB. Obtain two sets, A and B: a dataset of paleontological fossil images and a corresponding dataset of living organisms. Then, set the number of epochs and the batch size (batch_size), and choose an appropriate optimization method, such as the Adam method.

[0103] The dataset B is input into the generator GB2A to generate the image set A' corresponding to the features of set A, and then the dataset A is combined to calculate the loss of the discriminator DA. Similarly, the dataset A is input into the generator GA2B to generate the image set B' corresponding to the features of set B, and then the dataset B is combined to calculate the loss of the discriminator DB. Then, the set A' is passed through the generator GA2B to regenerate the image set B'' with the features of set B, and the consistency loss of B and B'' is calculated. Similarly, the set B' is passed through the generator GB2A to regenerate the image set A'' with the features of set A, and the consistency loss of A and A'' is calculated. And backpropagation is performed to update the gradient. After each epoch is iterated, the parameters that make the generator G work best are obtained.

[0104] Finally, after the model is trained, the generators GA2B and GB2A are applied to paleontological fossils and living organism images, respectively, to generate paleontological fossil transformation images with living organism characteristics and living organism transformation images with fossil characteristics. As a preferred embodiment of the present invention, the present invention only further optimizes the model. Microstructure can reveal details such as cellular organization, texture, and microstructure. These features are crucial for distinguishing paleontological species and their systematic taxonomic relationships. For many species that are difficult to distinguish based on macromorphology, microstructural analysis can provide more accurate classification. Microstructure reflects the functional adaptability of organisms. For example, the mineralization characteristics of fossil tissues and sediment microstructure provide environmental clues, such as the chemical properties of water bodies, redox state, and sedimentary environment type, which help reconstruct ancient ecological environments and climate conditions. Therefore, microstructural analysis is an important bridge connecting macroscopic fossil morphology with molecular paleontology and an indispensable tool for a comprehensive understanding of paleontological morphology, physiology, ecology, and evolution. It not only improves the scientific nature and accuracy of fossil research but also provides a solid foundation for the reconstruction of paleoenvironmental and paleoclimate conditions. The microstructural analysis of paleontological fossils is crucial for revealing the species, functional adaptations, living habits, ecological niche, and the environment they lived in. However, in the microstructure of paleontological fossils, some parameters are easy to obtain and preserve, while others are easily lost.

[0105] Current models primarily rely on image data. When combined with microstructure scanning for modeling, data with microscopic features is much scarcer than image data, and data with more refined structures is even scarcer. To address situations where data is missing or incomplete in actual detection samples, this invention further provides methods to improve the model's adaptability and performance stability when certain data is missing. This means that by using a large model trained on image data, all parameters can be trained from scratch each time, allowing the model to adapt to the input and tasks of microstructure scanning.

[0106] Paleontological fossils differ significantly from modern living organisms in many aspects, including morphology, color, and detail. Furthermore, paleontological fossils often lack some soft tissue information and modalities such as color, which results in reduced performance of conversion models when processing missing information. While traditional generative adversarial networks (GANs) can achieve conversion, they lack robustness to missing modalities and struggle to efficiently adapt to a variety of missing modalities. To address this, the present invention provides a technical solution that flexibly and efficiently modulates features for different combinations of missing modalities, improving the model's ability to compensate for missing information and the quality of the conversion.

[0107] Specifically, the present invention, given the aforementioned pre-trained model, adds a minimal number of learnable parameters while maintaining the original model parameters unchanged. These parameters can be applied across different network layers, flexibly adjusting the intermediate representation to accommodate different modal combinations. Compared to training separate models for each modal combination, this approach significantly reduces training and storage costs. Similarly, this embodiment maintains the principles of the original solution by constructing a database of paleontological fossils and their corresponding modern counterparts, ensuring precise alignment of images in terms of morphology, scale, and camera angle, providing a unified standard for subsequent model input. To address missing soft matter, texture, and other information in paleontological fossil images, intermediate features are dynamically modulated by defining a sharpness parameter γ and a tolerance parameter β to compensate for the performance loss caused by this missing information. A full-modality generative adversarial network is first pre-trained, freezing its backbone parameters. For each missing modal combination, the corresponding adaptation parameters ΔS = {γS, βS} are individually fine-tuned. During training, the inputs for the missing modal components are reset to zero to simulate actual missing conditions and ensure effective compensation in the adaptation layer. The original discriminator loss and consistency loss are maintained, ensuring high visual and semantic consistency in the converted images. The training goal of the adaptation module is to maximize the performance of these loss indicators and promote the robust performance of the model under missing modal conditions.

[0108] As described above, the debugging layer described in this embodiment is inserted after the output of each convolutional layer in the encoder and after the output of each residual block, but is not inserted in the decoder, keeping the decoder parameters frozen. The debugging layer performs an element-wise transformation on the intermediate features of each channel by learning a trainable sharpness parameter γ and tolerance parameter β: p = γ⊙x + β. Here, x is the input feature, p is the modulated feature, and the symbol ⊙ represents element-wise multiplication. The dimensions of γ and β are the same as the input features and are dynamically selected or adjusted based on the missing modalities. Complete paleontological modalities include at least two of the following: soft tissue morphology, hard shell geometry, and mineralization texture features. The feature modulation layer is located after the output of each convolutional layer and residual block.

[0109] To train the adaptation module, the present invention freezes the original model parameters, training only a minimal number of learnable parameters. Randomly simulating different modality-missing scenarios in the training data forces the model to learn how to adapt features to the missing modality through the adaptation module, thereby improving robustness during testing. This approach is applicable to various multimodal architectures, including fusion models and transformer-based models. Regardless of the number or type of input modalities, the same adaptation mechanism efficiently adjusts model performance.

[0110] Suppose the input modality set of a given multimodal task is M={m1,.,m m}. The specific embodiment of m can be soft tissue, hard shell, trace, mineralization feature, etc. In the case of having a complete modal set M, as mentioned above, training a parameter θ M The model f, all modal input data (denoted as X M ) is mapped to the output y M , the expression is: M =f(X M θ M ). Although all input modalities can be guaranteed to be available during the training phase, some modalities may not be accessible during the testing phase, especially after actual deployment. Any modality subset M may be missing due to data acquisition costs or data loss problems, such as some fossil or living data. If the model trained with all input modalities is used, the performance will be significantly degraded when some modalities are missing during testing. When it is missing, a simple and direct method is to retrain the model for the currently available modalities. Let the available modalities be S=M\K, and the model f can be retrained with the new parameter set θs, expressed as: S =f(X S θ S ), where X S Represents the input data of the modal set S. In theory, it can be all possible modal subsets Train a separate model and use the corresponding model at test time. However, this approach is not feasible in terms of computing and storage resources, especially when the number of modality combinations is large. In addition, deploying a large number of trained models and selecting one of them at test time is also not feasible in practical applications. Another disadvantage of this approach is that although it is desirable to S ≈y M , but this training process does not guarantee this.

[0111] This paper proposes an efficient solution to support all input modality subsets by adapting a single model. First, select a model f trained on the complete modality set M and freeze its parameters θ M Then for each available modality set S, a small number of specific parameters Δ S and update the model to: =f(X S θ M ,Δ S ),in represents the prediction result of the updated model. The goal is to make Close to all modal predictions ym (i.e. ≈y M ), which in the worst case is close to the prediction y by a model trained using only the available modalities M (Right now ≈y S ). Parameter set Δ S The size of the model compared to the total number of parameters θ M Be small.

[0112] The present invention uses data containing all available modal sets M to train the multimodal network f and learn the parameters θ M Then, for different subsets of available modalities Adapt the model f. Unlike existing methods, the goal of this invention is to learn a modification function for the available input modalities, rationally learn and fuse features to compensate for the impact of any missing modalities. Instead of retraining the entire network, the base network f is adapted, focusing on learning a very small number of parameters. In this way, the adaptation module can effectively adjust the model's response to different modal combinations, compensate for the information loss caused by missing modalities, and thus improve the robustness of multimodal systems in real-world scenarios.

[0113] Let W m is the weight matrix of the mth input mode corresponding to a certain layer, where m∈S. Learn a low-rank weight update matrix △W for this layer m , used to transform the input h of this layer m,i .h m,o =W m hm,i +△W m h m,i , , where h m,o is the transformed output feature, which will be passed to the next layer of the model. m is low-rank, the total number of learnable parameters is still a small fraction of the total number of model parameters. The set of all low-rank update matrices can be expressed as the learnable parameters Δ S ={△W m} m∈S For the mth input modality (m∈S), define a learnable scaling parameter γ m ∈R d and the offset parameter β m ∈R d , where d is the embedding dimension of the model. The mth modality output feature from any frozen layer is h m,o ∈R N×d , will be passed to the subsequent layer. The subsequent layer will m,o Apply the linear transformation as follows: h m,i =γ m ⊙h m,o +β m , , where h m,i ∈R N×d is the transformed feature that will be fed into the next frozen layer of the model, and N is the number of tokens. If the output shape of a layer is (H×W×d) (for convolutional layers), it is first reshaped to (N×d), where N=H×W, and then the above transformation is applied. The transformed feature is reshaped back to its original shape before being passed to the next layer (if necessary). The learnable parameter can be expressed as Δ S ={γ S ,β S}={γ m ,β m} m∈S When testing, load the pre-trained weights θ M The base multimodal model f. If all modalities are available, use θ M If some modes are missing, the learning parameter Δ corresponding to the available mode set S is selected S , insert the model and make predictions. Since the adaptable layer is inserted after each layer, no major changes to the model architecture are required, and all model parameters θ can be used without reloading. M Just load Δ Sparameters and insert them into the model. Using a pre-trained decoder / prediction head can still provide good overall performance in the case of multimodal missingness. The results show that compared with other parameter-efficient adaptation methods, the above-mentioned feature modulation performs well in most scenarios. The above-mentioned feature modulation shows great potential in enhancing representation ability, accelerating convergence, preventing information loss during representation learning, and alleviating distribution mismatch between upstream and downstream tasks. These characteristics motivate us to extend this method to multimodal learning for handling missing modalities, and construct a general framework that can effectively learn appropriate modulations of available input modalities to make up for the performance gap caused by missing modalities. In order to improve the credibility of detecting life characteristics and traces in extraterrestrial samples, the present invention first creates a database of Earth paleontological fossil images and a corresponding database of living images; then, based on a deep learning paleontological and living feature conversion network, it carries out the forward conversion of paleontological monomer images to corresponding living images, and the reverse conversion of living images to paleontological fossil images. By using a network that converts paleontological fossils to living organisms, images of existing living organisms can be converted into corresponding paleontological fossil images. This can effectively increase the number and diversity of paleontological fossil image databases and address the difficulty of fossilizing and preserving low-level and ancient organisms. Furthermore, using a network that converts paleontological fossil images into living organisms, individual paleontological fossils can be converted into corresponding living organisms, recovering their more detailed life characteristics. This approach addresses the issue of insufficient credibility in determining the existence of extraterrestrial life based solely on suspected paleontological fossils.

[0114] Establish a database of images of ancient fossils on Earth, and collect living organisms that still exist today to establish a corresponding database of living organism images. Types of living organisms include ostracods, radiolarians, foraminifera, etc. Take the images of ostracod fossils and the photos of living ostracods that exist today as an example, and align the two ( Figure 1 ). Alignment is not pairing, but ensuring that the shooting angles and body part directions of the images of ostracod fossils and living organisms are consistent. Taking the reconstruction of the features of ostracod fossils and living organisms as an example, the aligned ostracod fossil images (equivalent to fossil image set A) and living organism images (equivalent to living organism image set B) are input into the feature conversion network for model training. After the training is completed, the conversion between ostracod fossil images and living organism images can be realized. Figure 2 As shown in the figure, the first row shows the result of converting ostracod fossils into living ones, and the second row shows the result of converting living ostracods into fossils. It can be seen that converting ostracod fossils into living ones can maintain the boundary morphology of the fossils and add features such as color and texture details. In particular, the living structure of the photoreceptor can also be converted. Living ostracods can also be converted into corresponding fossil forms ( Figure 3 ), color and texture are consistent with the characteristics of fossils formed after burial.

[0115] According to another embodiment of the present invention, a paleontological morphological reconstruction system based on image recognition is provided, comprising: an image database module for storing a fossil image set A and a living image set B; the fossil image set A comprises images of microfossil monomers on Earth or extraterrestrial planets, and the living image set B comprises images of living monomers corresponding to the biological categories in the fossil image set A; an anatomical alignment module is configured to perform anatomical structure alignment processing on the fossil image set A and the living image set B by matching feature points, so that the key morphological features of the fossil images and the living images are aligned in spatial coordinates; a feature reconstruction model module comprises an image conversion unit constructed based on a cyclic generative adversarial network; the image conversion unit comprises The system includes: a first generator GA2B for converting image a from a fossil image set A to a living-domain image; a second generator GB2A for converting image b from a living-domain image set B to a fossil-domain image; a fossil-domain discriminator DA for determining the probability that an input image belongs to a real fossil-domain image; a living-domain discriminator DB for determining the probability that an input image belongs to a real living-domain image; a joint optimization unit for jointly optimizing the parameters of the generator and discriminator based on adversarial loss and consistency loss; and a reconstruction execution module configured to: input the fossil image to be tested into the first generator GA2B to output a living-domain reconstruction image; or input the living-body image into the second generator GB2A to output a simulated fossil image. Both the first generator GA2B and the second generator GB2A comprise: an encoder submodule, which reduces the input image resolution through strided convolution and extracts multi-scale features; a converter submodule, which comprises multiple residual blocks with skip connections for cross-domain feature adaptation; and a decoder submodule, which restores the image resolution through deconvolution and outputs a three-channel RGB image. Both the fossil domain discriminator DA and the living domain discriminator DB adopt a 70×70 PatchGAN structure, which includes the following layers connected in sequence: the first convolution layer: uses a 4×4 convolution kernel, a stride of 2, outputs 64 channels, and disables instance normalization; the second convolution layer: uses a 4×4 convolution kernel, a stride of 2, outputs 128 channels, followed by an instance normalization layer and a LeakyReLU activation function with a slope of 0.2; the third convolution layer: uses a 4×4 convolution kernel, a stride of 2, outputs 256 channels, followed by an instance normalization layer and a LeakyReLU activation function with a slope of 0.2; the fourth convolution layer: uses a 4×4 convolution kernel, a stride of 2, outputs 512 channels, followed by an instance normalization layer and a LeakyReLU activation function with a slope of 0.2; the output layer: generates a single-channel feature map through a 1×1 convolution kernel, and applies a Sigmoid function to output the probability value.

[0116] The joint optimization unit includes an adversarial loss calculation module and a consistency loss calculation module, and iteratively optimizes the generator and discriminator parameters through a backpropagation algorithm. It also includes a feature modulation module, deployed in the feature extraction path of the feature reconstruction model, containing a learnable parameter vector {γ, β}. The feature modulation module performs a channel-level linear transformation p = γ⊙x + β, where x is the intermediate feature representation of the paleontological fossil or living sample; γ is a scaling parameter vector, and β is an offset parameter vector used to compensate for missing biomodal features in the fossil; ⊙ represents element-wise multiplication. The system meets the following constraints: the backbone parameters in the feature reconstruction model pre-trained based on the complete paleontological modality remain frozen; for any missing fossil modality combination, the parameter vector {γ, β} is trained independently, and the input features corresponding to the missing modality are set to zero during training. The feature modulation module is located after the output of each convolutional layer and residual block, and the complete paleontological modality includes at least two of the following: soft tissue morphology, hard shell geometry, and mineralization texture features.

[0117] The description of the technical solutions of the above-mentioned system and method has similarities. The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for reconstructing paleontological morphology based on image recognition, characterized in that: The following steps are involved: Establishing a fossil image set A containing microfossil monomers from Earth or extraterrestrial planets, and a living image set B containing living organism monomers corresponding to the biological categories in the fossil image set A; Performing anatomical structure alignment processing on the fossil image set A and the living body image set B by feature point matching, so that key morphological features of the fossil image and the living body image are aligned in spatial coordinates; Construct a feature reconstruction model based on a cyclic generative adversarial network, including: Training a first generator GA2B for converting image a in the fossil image set A into a living domain image; Training a second generator GB2A for converting an image b in the in vivo image set B into a fossil domain image; The probability that the input image belongs to a real image in the fossil domain is determined by the fossil domain discriminator DA; The probability that the input image belongs to a real image in the living domain is determined by the living domain discriminator DB; The generator and discriminator are jointly optimized based on adversarial loss and consistency loss to ensure the anatomical fidelity of bidirectional conversion; The image of the fossil to be tested or the image of the living organism is input into the feature reconstruction model, and a living organism reconstruction image or a simulated fossil image is output.

2. The method according to claim 1, characterized in that Inputting the image of the fossil to be tested or the image of the living organism into the feature reconstruction model and outputting the living organism reconstruction image or the simulated fossil image includes: The image of the fossil to be tested is input into the first generator GA2B, and a living body reconstruction image is output; or the image of the existing living body is input into the second generator GB2A, and a simulated fossil image is output.

3. The method according to claim 1, characterized in that The first generator GA2B and the second generator GB2A adopt the same network structure, including the following connected in sequence: The encoder module is used to reduce the input image resolution and extract multi-scale features through strided convolution; The converter module contains multiple residual blocks with skip connections for cross-domain feature adaptation; The decoder module is used to restore the image resolution through deconvolution operations and output a three-channel RGB image.

4. The method according to claim 1, wherein The fossil domain discriminator DA and the living domain discriminator DB both adopt a 70×70 PatchGAN structure, including the following layers connected in sequence: The first convolutional layer uses a 4×4 convolution kernel, stride 2, outputs 64 channels, and disables instance normalization; The second convolutional layer uses a 4×4 convolution kernel, stride 2, and outputs 128 channels, followed by an instance normalization layer and a LeakyReLU activation function with a slope of 0.2; The third convolutional layer uses a 4×4 convolution kernel, stride 2, and outputs 256 channels, followed by an instance normalization layer and a LeakyReLU activation function with a slope of 0.2; The fourth convolutional layer uses a 4×4 convolution kernel, stride 2, and outputs 512 channels, followed by an instance normalization layer and a LeakyReLU activation function with a slope of 0.2; The output layer generates a single-channel feature map through a 1×1 convolution kernel and applies the Sigmoid function to output the probability value.

5. The method according to claim 1, wherein The generator and discriminator are jointly optimized based on the adversarial loss and consistency loss, including the calculation of the adversarial loss function: ； ； represents the probability that the image a is classified as a real image by the fossil domain discriminator DA, represents the probability that the data generated by the second generator GB2A is classified as fake data by the fossil domain discriminator DA, where , ; represents the probability that the image b is classified as a real image by the living domain discriminator DB, represents the probability that the data generated by the first generator GA2B is classified as false data by the living body domain discriminator DB, where, , .

6. The method according to claim 5, characterized in that Jointly optimize the generator and discriminator based on adversarial loss and consistency loss, including computing consistency loss; ； , a) is the image a in the fossil image set A converted to an image with living domain features, and then converted to an image with fossil domain features , the image and the difference between the image a; , b) is the image b in the living image set B converted to an image with fossil domain features, and then converted to an image with living domain features , the image and the difference between the image b; wherein the image and the difference between the image a and the image The difference between image a and image b is determined by: ， in, represents the image difference metric function, t is the pre-trained neural network layer, , is the height and width of the t-th layer feature map, , is the input image x and the reference image The feature vector at position (h,w) in layer t, is the channel weight vector, is pixel-by-pixel multiplication, is the square of the Euclidean norm.

7. The method according to claim 6, characterized in that The generator and discriminator are jointly optimized based on adversarial loss and consistency loss, including: Minimize the total loss function L, where , The parameters of the first generator GA2B, the second generator GB2A, the fossil domain discriminator DA and the living domain discriminator DB are iteratively optimized using the back propagation algorithm for the preset regularization coefficient.

8. The method according to claim 1, characterized in that A feature modulation layer is deployed in the feature extraction stage of the feature reconstruction model, and the feature modulation layer performs channel-level linear transformation , where: x is the intermediate feature representation of paleontological fossils or living samples; , are learnable scaling and offset parameters, respectively, used to compensate for the missing biological modal features in fossils. represents element-wise multiplication and satisfies the following constraints: The model backbone parameters based on the complete paleontological modality pre-training remain frozen; for any missing fossil modality combination, the parameter vector is trained independently During training, the input features corresponding to the missing modes are set to zero; the feature modulation layer is only deployed in the feature extraction path.

9. The method according to claim 8, characterized in that The complete paleontological modality includes at least two of soft tissue morphology, hard shell geometry and mineralization texture features; the feature modulation layer is located after the output end of each convolution layer and residual block.

Images