Corneal image postmortem time prediction method based on multi-modal transformer-diffusion model

Abstract

The present application provides a corneal image death time prediction method based on a multi-modal Transformer-diffusion model, belonging to the field of forensic science technology, which first acquires corneal image data and environmental parameter data; the corneal image data is preprocessed; the diffusion model is used for image quality enhancement, effectively processing the noise problem in low-quality images; the enhanced corneal image is feature-extracted by VisionTransformer, capturing complex nonlinear relationships; the corneal image features and environmental parameter data are multi-modal fused, considering various influencing factors; finally, the fusion features are used to generate a death time prediction result, and the model is compressed and deployed on a lightweight device to realize on-site application; the method realizes high-accuracy prediction within 0-72 hours, especially having obvious advantages in long-time prediction of more than 24 hours.

Description

Technical Field

[0001] This invention relates to the field of forensic science and technology, specifically to a method for predicting the time of death from corneal images based on a multimodal Transformer-diffusion model, and particularly to using deep learning technology to analyze corneal images to achieve high-precision prediction of the time of death. Background Technology

[0002] In forensic practice, accurately determining the time of death is crucial for criminal investigations. Traditional methods for determining the time of death mainly include post-mortem thermometry, gastric contents analysis, and ocular chemistry. Post-mortem thermometry is significantly affected by ambient temperature, gastric contents analysis is greatly affected by individual differences, and ocular chemistry can damage the cornea and is complex to perform. These methods generally suffer from low accuracy, sensitivity to environmental factors, or the need for complex procedures.

[0003] With the development of computer vision and artificial intelligence technologies, image analysis-based methods for estimating time of death have gradually emerged. The cornea, as a transparent tissue exposed to the outside world, undergoes significant clouding changes over time after death. This change exhibits certain regularities, providing a possibility for predicting time of death through image analysis. However, existing methods for predicting time of death based on corneal images mainly rely on simple image processing techniques and traditional machine learning algorithms, making it difficult to capture complex nonlinear relationships and limiting their ability to process low-quality images. Especially under conditions of high image noise and uneven illumination, the prediction accuracy decreases significantly.

[0004] Furthermore, existing technologies also have shortcomings in data acquisition and processing. The lack of standardized, high-quality corneal image databases, especially long-term data exceeding 24 hours under diverse environmental conditions, limits the effective training of deep learning models. At the same time, existing methods often focus only on the corneal image itself, ignoring the influence of environmental parameters (such as temperature and humidity) on corneal changes, resulting in limited accuracy and reliability of prediction results. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a corneal image time-of-death prediction method based on a multimodal Transformer-diffusion model, which can overcome the limitations of existing methods in handling complex nonlinear relationships and low-quality images, and improve the accuracy and reliability of time-of-death prediction.

[0006] This invention proposes a method for predicting corneal image time of death based on a multimodal Transformer-diffusion model, comprising:

[0007] Acquire corneal image data and environmental parameter data;

[0008] The corneal image data is preprocessed to obtain a preprocessed corneal image;

[0009] Based on the preprocessed corneal image, image quality enhancement is performed using a diffusion model to obtain an enhanced corneal image.

[0010] The enhanced corneal image was subjected to feature extraction using VisionTransformer to obtain corneal image features;

[0011] The corneal image features are fused with the environmental parameter data in a multimodal manner to obtain fused features;

[0012] Based on the fusion features, a time of death prediction result is generated.

[0013] Preferably, the acquisition of corneal image data and environmental parameter data includes:

[0014] Acquire corneal images and record environmental parameters such as temperature and humidity at the time of image capture;

[0015] The image quality score is obtained by scoring the sharpness, illumination uniformity, corneal integrity, focus accuracy, shooting angle, and environmental interference of the corneal image.

[0016] Based on the image quality score, qualified corneal image data are selected.

[0017] Preferably, the method for scoring the sharpness, illumination uniformity, corneal integrity, focus accuracy, shooting angle, and environmental interference of the corneal image to obtain an image quality score is as follows:

[0018] Each dimension was scored on a scale of 1 to 5.

[0019] An image is considered good when the total score is 25 or higher.

[0020] Images with a total score between 20 and 24 are deemed to require review.

[0021] An image is considered unqualified if its total score is less than 20 points or its score in any dimension is less than or equal to 2 points.

[0022] Preferably, the preprocessing of the corneal image data to obtain the preprocessed corneal image includes:

[0023] The corneal image is segmented into corneal regions;

[0024] Adjust the segmented corneal region images to a uniform size;

[0025] Perform pixel value normalization;

[0026] Image enhancement is performed, including adaptive contrast adjustment, illumination equalization, and edge enhancement.

[0027] Preferably, the image quality enhancement using a diffusion model includes:

[0028] The forward noise addition process is designed to gradually add noise to the original corneal image;

[0029] The diffusion model is trained to learn the reverse denoising process from a noisy image to the original image;

[0030] A trained diffusion model is used to denoise and repair low-quality corneal images to obtain enhanced corneal images.

[0031] Generate diverse corneal image samples to expand the training dataset.

[0032] Preferably, the feature extraction of the enhanced corneal image using VisionTransformer includes:

[0033] The enhanced corneal image is divided into several image blocks;

[0034] Linear projection is performed on the image patch, and positional encoding information is added to obtain the image patch embedding representation;

[0035] The image patch embedding representation is input into the Transformer encoder;

[0036] By employing a multi-head self-attention mechanism and a feedforward neural network, global relationships between image patches are captured to obtain corneal image features.

[0037] Preferably, the step of multimodal fusion of the corneal image features and the environmental parameter data to obtain fused features includes:

[0038] The environmental parameter data is converted into a parameter embedding vector;

[0039] Design a feature fusion module that combines corneal image features and parameter embedding vectors;

[0040] By dynamically adjusting the weights of different modal features using an attention mechanism, fused features are obtained.

[0041] Preferably, generating the time of death prediction result based on the fusion features includes:

[0042] Design a classification head module to map the fused features to a probability distribution of the death time interval;

[0043] One-hot encoding is used to represent the time intervals of death, with 12 intervals corresponding to different time periods of death;

[0044] Calculate the confidence score of the prediction results;

[0045] Low-confidence predictions are marked, indicating that manual review may be required.

[0046] As a preferred option, it also includes:

[0047] Joint optimization of the Transformer model and the diffusion model improves the overall system performance.

[0048] Design a feedback mechanism so that the prediction error of the Transformer guides the improvement of the diffusion model;

[0049] A serial processing flow is implemented, with the diffusion model serving as the preprocessing engine and the Transformer model serving as the main prediction engine.

[0050] Preferably, the method also includes the step of deploying it onto a lightweight device, including:

[0051] Compression of Transformer and diffusion models includes knowledge distillation, parameter quantization, and model pruning;

[0052] Convert the compressed model into a format suitable for edge devices;

[0053] A complete system for data acquisition, processing, and result display is implemented on an integrated hardware platform;

[0054] The integrated hardware platform includes a high-resolution camera module, a lightweight computing unit, a multi-parameter sensor, a display screen, and an integrated battery power system.

[0055] The beneficial effects of this invention include:

[0056] 1. By establishing a multimodal fusion framework that combines corneal image data and environmental parameter data, we can comprehensively capture various factors that affect corneal changes and improve the accuracy and robustness of the prediction model.

[0057] 2. A diffusion model is used for image quality enhancement, which effectively addresses noise issues in low-quality corneal images, improves image clarity, and provides a better data foundation for subsequent analysis.

[0058] 3. Leveraging the powerful feature extraction capabilities of VisionTransformer, complex patterns and nonlinear relationships in corneal images are captured, enabling more accurate prediction of time of death.

[0059] 4. Establish a complete workflow for shooting, importing and analyzing data, and presenting the time of death, simplifying operational steps and improving the efficiency of forensic work.

[0060] 5. Through lightweight model technology, deployment on portable devices can be achieved to meet the needs of on-site forensic work. Attached Figure Description

[0061] Figure 1 This is an overall flowchart of the corneal image time-of-death prediction method based on the multimodal Transformer-diffusion model of the present invention.

[0062] Figure 2 This is a schematic diagram of the corneal image quality assessment system of the present invention.

[0063] Figure 3 This is a schematic diagram of the image quality enhancement process of the diffusion model of the present invention.

[0064] Figure 4 This is a schematic diagram of the VisionTransformer structure of the present invention.

[0065] Figure 5 This is a schematic diagram of the structure of the multimodal feature fusion module of the present invention. Detailed Implementation

[0066] Please refer to Figure 1 - Figure 5 The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. Those skilled in the art should understand that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention.

[0067] Reference Figure 1 This invention provides a method for predicting corneal image death time based on a multimodal Transformer-diffusion model, which mainly includes the following steps:

[0068] First, corneal image data and environmental parameter data are acquired. In practical applications, a high-resolution camera (such as the Canon EOS M650) equipped with a polarizer can be used to capture corneal images, while simultaneously recording parameters such as temperature and humidity of the shooting environment. Preferably, the ambient temperature is controlled within the range of 18°C ​​to 26°C, because corneal changes are more regular within this temperature range, which helps to improve prediction accuracy.

[0069] Next, the acquired corneal image data is preprocessed to obtain a preprocessed corneal image. The preprocessing steps include corneal region segmentation, image size adjustment, pixel value normalization, and preliminary image enhancement. In one embodiment of the invention, the image can be uniformly adjusted to a size of 224×224 pixels, and pixel value normalization can be performed to unify the pixel value range to between 0 and 1. This helps improve the stability and accuracy of subsequent processing.

[0070] Then, based on the preprocessed corneal image, image quality enhancement is performed using a diffusion model to obtain the enhanced corneal image. The diffusion model effectively removes noise and improves image quality through forward noise addition and backward noise reduction processes. Furthermore, the diffusion model can generate diverse training samples, enhancing the model's generalization ability and robustness.

[0071] Next, VisionTransformer is used to extract features from the enhanced corneal image to obtain corneal image features. VisionTransformer first segments the image into multiple image patches, and then uses a multi-head self-attention mechanism to capture the global relationship between the image patches, thereby extracting the deep features of the corneal image.

[0072] Subsequently, corneal image features and environmental parameter data are fused in a multimodal manner to obtain fused features. Multimodal fusion dynamically adjusts the weights of different modal features through an attention mechanism to achieve effective information integration.

[0073] Finally, based on the fused features, the predicted time of death is generated. The predicted results can be represented in one-hot encoding form to indicate different time intervals of death. At the same time, the prediction confidence is calculated, and low-confidence predictions are marked to indicate that manual review may be required.

[0074] In this invention, the quality of the corneal image has a significant impact on the prediction results. Therefore, this invention establishes a rigorous image quality evaluation system.

[0075] Reference Figure 2 This invention scores the acquired corneal images in multiple dimensions, including six dimensions: image clarity, illumination uniformity, corneal integrity, focus accuracy, shooting angle, and environmental interference. Each dimension is scored from 1 to 5 points.

[0076] Regarding image sharpness, a score of 5 indicates clear texture and visible corneal fine structures, 3 indicates slight blurring but still clear edges, and 1 indicates severe blurring where corneal texture cannot be discerned. Preferably, the image sharpness score is not lower than 3 points, which ensures that corneal texture information in the image can be effectively extracted.

[0077] Regarding illumination uniformity, a score of 5 indicates no glare and uniform illumination, 3 indicates localized glare but covering less than 20% of the cornea, and 1 indicates strong glare or shadows covering more than 50% of the corneal area. Preferably, the illumination uniformity score is not lower than 3, which helps reduce interference caused by uneven illumination.

[0078] For corneal integrity, a score of 5 indicates complete and undamaged cornea, a score of 3 indicates minor damage but less than 10% of the area, and a score of 1 indicates corneal tear or loss exceeding 30% of the area. Preferably, the corneal integrity score is not lower than 4 points, which ensures the integrity of corneal morphological information.

[0079] Regarding focus accuracy, 5 points indicates perfect focus across the entire cornea, 3 points indicates partial focus, and 1 point indicates complete defocus causing blurred iris texture. Preferably, the focus accuracy score is no less than 4 points, which ensures overall image sharpness.

[0080] Regarding the shooting angle, 5 points indicates that the image is perpendicular to the center of the cornea with an error of less than 5°, 3 points indicates a tilt of 10-20°, and 1 point indicates a tilt of more than 30°. Preferably, the shooting angle score is not lower than 4 points, which reduces the distortion caused by changes in viewing angle.

[0081] Regarding environmental interference, 5 points indicates no contamination, 3 points indicates slight contamination that can be corrected by software, and 1 point indicates obvious contaminants such as bloodstains or dust. Preferably, the environmental interference score is not lower than 4 points, which reduces the impact of external factors on image quality.

[0082] Based on the scoring of the above six dimensions, this invention uses the following criteria for image selection:

[0083] When the total score is greater than or equal to 25 points, it is judged as a good image and directly enters the analysis process;

[0084] When the total score is between 20 and 24 points, the image is deemed to require review, and a third party decides whether to use it.

[0085] An image is deemed unqualified and removed if the total score is less than 20 points or the score in any dimension is less than or equal to 2 points.

[0086] Through this rigorous image quality assessment system, the present invention ensures that the corneal images entering the analysis process have sufficiently high quality, providing a good data foundation for subsequent processing.

[0087] After acquiring qualified corneal images, a series of preprocessing operations are required to prepare for subsequent analysis. The corneal image preprocessing of this invention mainly includes four steps:

[0088] First, corneal region segmentation is performed on the corneal image. Since the original image may contain non-corneal areas such as eyelids and eyelashes, the corneal region needs to be extracted using image segmentation techniques. In one embodiment of the invention, a U-Net-based semantic segmentation method can be used to identify and extract the corneal region. Preferably, the segmentation accuracy (IoU, Intersection over Union) is not less than 0.9, which ensures accurate extraction of the corneal region.

[0089] Secondly, the segmented corneal region images are adjusted to a uniform size. Considering the input requirements of the subsequent VisionTransformer model, this invention uniformly adjusts the corneal images to a size of 224×224 pixels. During the adjustment process, a bicubic interpolation algorithm is used to preserve image detail information.

[0090] Next, pixel value normalization is performed. Normalizing the pixel values ​​of the RGB three channels to the range of 0-1 helps improve the stability of model training. The normalization formula is:

[0091] ,

[0092] in These are the normalized pixel values. This is the minimum value of the channel (usually 0). This is the maximum value for the channel (usually 255).

[0093] Finally, image enhancement processing is performed. This invention employs multiple image enhancement techniques, including adaptive contrast adjustment, illumination equalization, and edge enhancement. Adaptive contrast adjustment uses the CLAHE (Contrast Limited Adaptive Histogram Equalization) algorithm, dividing the image into several sub-regions (e.g., an 8×8 grid), performing histogram equalization on each sub-region, and then performing bilinear interpolation, which effectively improves the local contrast of the image. Illumination equalization removes the effects of uneven illumination through Gaussian filtering and subtraction operations. Edge enhancement uses a nonlinear edge enhancement filter to strengthen corneal edges and texture information.

[0094] Through the above preprocessing steps, the present invention effectively improves the quality and standardization of corneal images, laying the foundation for subsequent deep learning analysis.

[0095] Preprocessed corneal images may still have some degree of noise and quality issues, especially images acquired under non-ideal shooting conditions. Therefore, this invention introduces a diffusion model for further image quality enhancement.

[0096] The diffusion model is a generative model that generates high-quality images by progressively adding noise and learning a denoising process. The diffusion model of this invention comprises two key parts: a forward noise addition process and a backward denoising process.

[0097] The forward noise addition process is designed as follows:

[0098] From the original corneal image Begin, through Gaussian noise is gradually added step by step to obtain a series of noisy images. The noise addition at each step can be represented as:

[0099] ,

[0100] in: For the noisy image at step t, For the noisy image at step t-1, This is the noise scheduling parameter (usually set as a decreasing sequence, ranging from 0.999 to 0.9). It is standard Gaussian noise, conforming to a normal distribution with a mean of 0 and a variance of 1. , The identity matrix is ​​used. Preferably, the noise step number T is set to 1000, which provides a sufficiently smooth noise addition process.

[0101] The reverse denoising process is the process of learning to recover the original image from a noisy image. This invention trains a neural network. The noise added at each step is predicted, and the original image is then reconstructed through reverse reasoning. The training objective is to minimize the following loss function:

[0102] ,

[0103] in: For loss function, For neural network parameters, The actual noise added. For network prediction noise, For time steps, Indicates the expected value. This represents the square of the Euclidean norm.

[0104] After training, the reverse denoising process can be represented as:

[0105] ,

[0106] in: To recover a lower noise image, For cumulative noise parameters, Represents all values ​​from i=1 to t. The product of Add small noise to each step (usually set to) ), Standard Gaussian noise is used to increase sampling diversity.

[0107] The diffusion model of this invention is not only used for image quality enhancement, but also for generating diverse training samples to expand the training dataset. In particular, for time periods with scarce data (such as corneal images older than 24 hours), more samples are synthesized through conditional generation techniques, improving the model's prediction accuracy in these time periods.

[0108] In practical applications, the diffusion model outperforms traditional methods in denoising performance. For example, for corneal images with 80% random noise, the diffusion model of this invention can achieve a PSNR (Peak Signal-to-Noise Ratio) of 31.2 dB, which is significantly higher than traditional denoising methods such as APGL (26.63 dB), TNNR (27.47 dB), and DNN (28.79 dB).

[0109] The corneal image enhanced by the diffusion model is then fed into VisionTransformer (ViT) for feature extraction. ViT is a deep learning model that applies the Transformer architecture to computer vision tasks, effectively capturing global dependencies in images.

[0110] The feature extraction process of the VisionTransformer in this invention is as follows:

[0111] First, the enhanced corneal image (224×224 pixels) is divided into multiple 16×16 pixel patches, resulting in 196 patches. Each patch is transformed into an embedding vector with a dimension of 768 through linear projection. The patch embedding representation can be expressed as:

[0112] ,

[0113] in: This is the initial embedding representation (dimension 197×768). For special classification labels (learnable parameter vectors with 768 dimensions). This is the first image patch (dimension 256, i.e., 16×16 pixels). The embedding matrix (256×768, which maps 256-dimensional image patches to a 768-dimensional feature space). The location is encoded (a learnable parameter matrix with dimensions 197×768, used to preserve spatial location information), and the semicolon ";" indicates a vector concatenation operation.

[0114] Then, the image patch embeddings are input into the Transformer encoder. The Transformer encoder consists of multiple layers, each containing a multi-head attention mechanism and a feed-forward network. The multi-head attention mechanism allows the model to simultaneously focus on information from different regions of the image, capturing the complex relationships between image patches. The self-attention calculation formula is:

[0115] ,

[0116] in: For query matrix (Query, dimension n×) (where n is the sequence length) The key matrix (Key, dimension n×) ), Value matrix (Value, dimension n×d_v) For the dimension of the key, Let K be the transpose of K, and softmax be the softmax activation function, which transforms the input into a probability distribution. This is a scaling factor to prevent the softmax gradient from vanishing due to excessively large dot product values.

[0117] In multi-head attention, the attention mechanism is run in parallel h times (h=8 in this invention), and then the results are concatenated:

[0118] ,

[0119] Where: head Attention (Dimension is d_modelxd_k) (Dimension is d_modelxd_k) (Dimensions are d_model x d_v) and (The dimension is hd_v x d_model) is the learnable parameter matrix. Concat represents the concatenation operation, which concatenates the outputs of h heads together. d_model is the model dimension (768 in this invention).

[0120] In the feedforward neural network part, a two-layer fully connected network is used for nonlinear transformation:

[0121] ,

[0122] Where: GELU is the activation function of the Gaussian error linear unit. , , , These are learnable parameters. Preferably, the hidden layer dimension of the feedforward network is set to 3072, which provides sufficient model capacity.

[0123] After processing by an L-layer Transformer encoder (L=12 in this invention), the deep feature representation of the corneal image is finally obtained. Specifically, classification labels... The corresponding output is used as global image features for subsequent time-of-death prediction.

[0124] The VisionTransformer of this invention has the following advantages compared to traditional convolutional neural networks (CNNs):

[0125] 1. It can capture long-range dependencies in corneal images and identify subtle patterns of corneal changes over time;

[0126] 2. Attention mechanisms provide interpretability for model decisions, which helps in understanding the image regions that the model focuses on;

[0127] 3. It has stronger robustness to image distortion and positional changes, and can adapt to corneal images under different shooting conditions.

[0128] Changes in the cornea are influenced by a variety of factors, and relying solely on image features may not be able to fully capture the relationship between time of death and corneal condition. Therefore, this invention introduces multimodal fusion technology to fuse corneal image features with environmental parameter data (such as temperature and humidity) to improve the accuracy and robustness of predictions.

[0129] The multimodal feature fusion process is as follows:

[0130] First, the environmental parameter data is converted into parameter embedding vectors. Assuming the environmental parameters contain m dimensions (e.g., temperature, humidity), each parameter is first normalized, and then transformed into an embedding vector of dimension d through a linear layer.

[0131] ,

[0132] in: The parameter embedding vector (with dimension d, d=768 in this invention). This is the normalized environment parameter vector (with dimension m). This is the weight matrix (dimension m×d). It is the bias vector (dimension d).

[0133] Then, a feature fusion module is designed to combine corneal image features and parameter embedding vectors. This invention employs an attention-based fusion method to dynamically adjust the weights of features from different modalities. The fusion process can be represented as:

[0134] ,

[0135] in: For fusion features (dimension d). Image features (dimension d) extracted for Vision Transformer. Embed the parameter vector (with dimension d). and For attention weights, satisfying .

[0136] Attention weights are calculated using a learnable attention network:

[0137] ,

[0138] in: This is the weight matrix (2×2d). It is a bias vector (with a dimension of 2). This means concatenating image features and parametric features into a vector (2d in dimension). The softmax function ensures that the sum of the two weights in the output is 1.

[0139] In one embodiment of the invention, a cross-modal attention mechanism is further introduced, allowing for mutual influence between features of different modalities. The attention of image features to parametric features and the attention of parametric features to image features are calculated separately and then fused, which enhances the model's ability to capture complex relationships between modalities.

[0140] A key advantage of multimodal fusion is its enhanced adaptability to diverse environmental conditions. For instance, corneal opacity typically accelerates at higher temperatures (above 26°C) and slows down at lower temperatures (below 18°C). By fusing environmental parameter information, the model can automatically adjust its interpretation of image features, thereby improving prediction accuracy.

[0141] Based on the fused features, this invention designs a classification head module to generate the predicted time of death. The classification head module maps the fused features to the probability distribution of the time of death intervals, and uses one-hot encoding to represent different time of death intervals.

[0142] Specifically, the classification head module contains a multilayer perceptron (MLP) that maps the fused features $f_{fused}$ to k output units (k=12 in this invention, corresponding to 12 death time intervals):

[0143] ,

[0144] in: This represents the probability distribution of the predicted time intervals of death (with dimension k). This is the first layer weight matrix (dimension d×d_h, where d_h is the hidden layer dimension, set to 1024). This is the first layer bias vector (dimension d_h). This is the second-layer weight matrix (dimension d_h×k). The second layer bias vector (with dimension k) is GELU, which is the Gaussian error linear unit activation function, and the softmax function converts the output into a probability distribution.

[0145] The 12 time intervals for death are divided as follows: 0-6 hours, 6-12 hours, 12-18 hours, 18-24 hours, 24-30 hours, 30-36 hours, 36-42 hours, 42-48 hours, 48-54 hours, 54-60 hours, 60-66 hours, and 66-72 hours. This interval division maintains sufficient accuracy while taking into account the needs of actual forensic work.

[0146] For the confidence assessment of the prediction results, this invention calculates the maximum value of the prediction probability distribution as the confidence score:

[0147] ,

[0148] in: The confidence score is... For the predicted probability distribution, The function returns the maximum value in the vector.

[0149] When the confidence score is below a threshold (e.g., 0.75), the system marks the prediction as low confidence, indicating that manual review may be necessary. This mechanism increases the reliability of the prediction, especially when dealing with boundary cases or atypical corneal images.

[0150] During model training, the cross-entropy loss function is used:

[0151] ,

[0152] in: This is the loss value. The real label (one-hot encoded, dimension k) Let be the predicted probability for the i-th time interval. It is the natural logarithm function. This represents the summation over k time intervals.

[0153] To address the issue of imbalanced sample sizes across different time intervals, this invention introduces a weighted cross-entropy loss, assigning higher weights to time intervals with fewer samples:

[0154] ,

[0155] in: The weighted loss value. Let be the weight of the i-th time interval, inversely proportional to the number of samples in that interval, calculated using the following formula: , The total number of samples, Let be the number of samples in the i-th interval.

[0156] Experimental results demonstrate that the time-of-death prediction method of this invention achieved a 99% accuracy rate in a 72-hour canine experiment, significantly outperforming traditional methods. In particular, this method excels in long-term predictions exceeding 24 hours, which is of significant value in expanding the applicability of time-of-death determination in forensic medicine.

[0157] To further improve system performance, this invention performs joint optimization of the Transformer model and the diffusion model, and designs a collaborative working mechanism.

[0158] The joint optimization process employs a two-stage training strategy: first, the Transformer model and the diffusion model are pre-trained separately, and then end-to-end joint fine-tuning is performed. The loss function in the joint fine-tuning stage consists of two parts:

[0159] ,

[0160] in: For joint losses, Classification loss (used for time of death prediction) For diffusion model loss (used for image enhancement). and These are the weighting coefficients (set to 1.0 and 0.5 in this invention). This joint optimization method enables the two models to coordinate with each other and jointly improve the overall performance.

[0161] In practical applications, the Transformer and diffusion models employ a sequential processing flow: the diffusion model acts as the preprocessing engine, responsible for image quality enhancement; the Transformer model serves as the main prediction engine, responsible for feature extraction and temporal prediction. This sequential architecture simplifies data flow and reduces system complexity.

[0162] Furthermore, this invention incorporates a feedback mechanism that allows the Transformer's prediction errors to guide improvements in the diffusion model. Specifically, when the Transformer's prediction accuracy for a certain type of corneal image is low, the system automatically labels these images and increases the processing weights of the diffusion model for these images in subsequent training, thereby improving the quality enhancement effect of the relevant images.

[0163] Experimental results show that this collaborative optimization strategy can significantly improve system performance. Compared with using the Transformer or diffusion model alone, the jointly optimized system performs more stably in low-quality image processing and boundary case prediction, with an average accuracy improvement of 3.5%.

[0164] To meet the needs of on-site forensic work, this invention deploys the method onto lightweight equipment, enabling portable operation.

[0165] Model compression is a key step in lightweight deployment. This invention employs techniques such as knowledge distillation, parameter quantization, and model pruning to compress Transformer and diffusion models.

[0166] In the knowledge distillation process, a large teacher model is first trained, and then the output of the teacher model is used to guide the training of a small student model. The distillation loss function is:

[0167] ,

[0168] in For distillation loss, Cross-entropy loss (the loss between the student model output and the true label), The KL divergence loss is used to calculate the difference between the student model output and the teacher model output. For real labels, For the output of the student model, For the output of the teacher model, The balance coefficient (set to 0.5 in this invention) is used to measure the difference between two probability distributions.

[0169] Parameter quantization converts 32-bit floating-point parameters into 8-bit integer representations, significantly reducing model size and computational complexity. Pruning techniques remove network connections that contribute little to prediction, further alleviating the model's burden. Through these techniques, this invention reduces the model size by more than 80% while maintaining at least 97% of the original accuracy.

[0170] Preferably, the lightweight model is converted to ONNX format and further optimized using TensorRT to fully utilize the computing power of edge devices. This reduces inference time from several seconds to less than one second, meeting real-time processing requirements.

[0171] The integrated hardware platform of this invention includes a high-resolution camera module (Canon EOS M650), a lightweight computing unit (NVIDIA Jetson TX2), a multi-parameter sensor, a 7-inch touchscreen display, and an integrated battery power system. The entire system weighs less than 1.5 kg and has a battery life of over 8 hours, making it suitable for on-site forensic work environments.

[0172] In terms of software, this invention features an intuitive user interface that simplifies the operation process. Users only need to capture corneal images, and the system automatically performs preprocessing, quality enhancement, and time-of-death prediction, generating a detailed report. The report includes the predicted time-of-death interval, confidence score, and key corneal feature markers, making it easy for forensic personnel to understand and use the prediction results.

[0173] This invention provides a method for predicting time of death from corneal images based on a multimodal Transformer-diffusion model. By combining advanced deep learning techniques and multimodal data fusion strategies, it effectively improves the accuracy and reliability of time of death prediction. Furthermore, through model lightweighting and hardware integration, it enables deployment on portable devices, meeting the needs of on-site forensic work. This method performs excellently within the 0-72 hour range, exhibiting significant advantages in long-term predictions exceeding 24 hours, providing a new solution for technological advancement and practical application in the field of forensic science.

[0174] The embodiments described above are merely illustrative of specific implementations of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for predicting postmortem interval from corneal images based on a multi-modal Transformer-diffusion model, characterized in that, include: Acquire corneal image data and environmental parameter data; The corneal image data is preprocessed to obtain a preprocessed corneal image; The preprocessing includes: segmenting the corneal region of the corneal image using a U-Net-based semantic segmentation method, with a segmentation accuracy IoU of not less than 0.9; adjusting the segmented corneal region image to a size of 224×224 pixels; normalizing pixel values; and performing image enhancement, including adaptive contrast adjustment, illumination equalization, and edge enhancement. Based on the preprocessed corneal image, image quality enhancement is performed using a diffusion model to obtain an enhanced corneal image. A forward noise addition process is designed to gradually add noise to the original corneal image. The diffusion model is trained to learn the reverse denoising process from the noisy image to the original image. The trained diffusion model is then used to denoise and repair low-quality corneal images to obtain an enhanced corneal image. Diverse corneal image samples are generated to expand the training dataset. The reverse denoising process of the diffusion model is implemented using the following formula: ， wherein: is the restored lower noise image, is the noise image of the first step, is the noise schedule parameter, denotes the product of all from , , is the neural network predicted noise, is the neural network parameter, is the small noise added at each step, is a standard Gaussian noise, following a normal distribution with mean 0 and variance 1 , is the identity matrix; Feature extraction was performed on the enhanced corneal image using Vision Transformer; the enhanced corneal image was then segmented into multiple 16×16 pixel image blocks, resulting in 196 image blocks; linear projection was performed on the image blocks, and positional encoding information was added to obtain the image block embedding representation. : ， in: For special classification markers, For the first Image blocks, For embedding matrix, The image patch is embedded and input into a Transformer encoder. Through a multi-head self-attention mechanism and a feedforward neural network, the global relationship between the image patches is captured to obtain corneal image features. The corneal image features and environmental parameter data are fused in a multimodal manner to obtain fused features; the environmental parameter data is converted into parameter embedding vectors; a feature fusion module is designed to combine corneal image features and parameter embedding vectors; the weights of different modal features are dynamically adjusted using an attention mechanism to obtain fused features; multimodal fusion is achieved through the following formula: ， in: As a feature of fusion, Image features extracted for Vision Transformer Embed the vector for the parameters. and For attention weights, satisfying ; Attention weights are calculated using the following formula: ， in: This is the weight matrix. For bias vectors, This means concatenating image features and parametric features into a single vector. The softmax function ensures that the sum of the two weights in the output is 1. Furthermore, a cross-modal attention mechanism is introduced, where the attention of image features to parametric features and the attention of parametric features to image features are calculated separately, and then fused together. Based on the fused features, a time-of-death prediction result is generated, including: designing a classification head module to map the fused features to a probability distribution of time-of-death intervals; using one-hot encoding to represent the time-of-death intervals, with 12 intervals corresponding to different time periods of death; calculating the confidence score of the prediction result; and marking low-confidence predictions to indicate the need for manual review. The classification head module is implemented using the following formula: ， in: This represents the probability distribution of the predicted time intervals of death. This is the first layer weight matrix. This is the first layer bias vector. This is the weight matrix for the second layer. The second layer bias vector is GELU, which is the Gaussian error linear unit activation function, and the softmax function converts the output into a probability distribution. The model training uses a weighted cross-entropy loss function, assigning higher weights to time intervals with fewer samples: ， in: The weighted loss value. For the first Weights for each time interval For real labels, For the first The predicted probability for each time interval. The number of time intervals; the weight is inversely proportional to the number of samples in that interval, and the calculation formula is: , The total number of samples, For the first The number of samples in each interval; The Transformer model and the diffusion model are jointly optimized using a two-stage training strategy: first, the Transformer model and the diffusion model are pre-trained separately, and then end-to-end joint fine-tuning is performed. A feedback mechanism is designed so that the prediction errors of the Transformer guide the improvement of the diffusion model. A serial processing flow is implemented, with the diffusion model as the pre-processing engine and the Transformer model as the main prediction engine. The loss function for the joint optimization is: ， in: For joint losses, For classifying losses, For the diffusion model loss, and These are the weighting coefficients.

2. The method according to claim 1, characterized in that, The acquisition of corneal image data and environmental parameter data includes: Acquire corneal images and record environmental parameters such as temperature and humidity at the time of image capture; The image quality score is obtained by scoring the sharpness, illumination uniformity, corneal integrity, focus accuracy, shooting angle, and environmental interference of the corneal image. Based on the image quality score, qualified corneal image data are selected.

3. The method according to claim 2, characterized in that, The method for scoring the corneal image based on its sharpness, illumination uniformity, corneal integrity, focus accuracy, shooting angle, and environmental interference to obtain an image quality score is as follows: Each dimension was scored on a scale of 1 to 5. An image is considered good when the total score is 25 or higher. Images with a total score between 20 and 24 are deemed to require review. An image is considered unqualified if its total score is less than 20 points or its score in any dimension is less than or equal to 2 points.

4. The method according to claim 1, characterized in that, It also includes the step of deploying the method to a lightweight device, including: Compression of Transformer and diffusion models includes knowledge distillation, parameter quantization, and model pruning; Convert the compressed model into a format suitable for edge devices; A complete system for data acquisition, processing, and result display is implemented on an integrated hardware platform; The integrated hardware platform includes a high-resolution camera module, a lightweight computing unit, a multi-parameter sensor, a display screen, and an integrated battery power system.

Images