Forensic medicine intelligent diagnosis method and system based on multi-modal image and deep learning

Abstract

The application discloses a kind of forensic science intelligent diagnosis method and system based on multi-modal image and deep learning, it is related to forensic science diagnostic technique field, disclosed forensic science intelligent diagnosis method and system based on multi-modal image and deep learning, by automatic acquisition and processing multi-modal image data, characteristic fusion is carried out in combination with environmental parameter, diagnosis analysis and visual report generation, solve the problem of manual interpretation in prior art, low efficiency, strong subjectivity, can improve the degree of automation of forensic science diagnosis, objectivity and accuracy, reduce artificial dependence, realize efficient, interpretable diagnosis report generation.

Description

Technical Field

[0001] This application relates to the field of forensic diagnostic technology, and in particular to forensic intelligent diagnostic methods and systems based on multimodal imaging and deep learning. Background Technology

[0002] With the rapid development of medical imaging technology, virtual autopsy, as a non-invasive post-mortem examination method, has shown great potential in the field of forensic medicine. High-precision three-dimensional image data obtained through technologies such as computed tomography (CT) or magnetic resonance imaging (MRI) provides new digital evidence for key forensic issues such as cause of death deduction and time of death determination. However, current analysis of virtual autopsy images mainly relies on manual interpretation and experience-based judgment by forensic experts. This is not only inefficient and time-consuming, but also prone to subjective influences, resulting in poor standardization and repeatability. Furthermore, the complexity of forensic diagnosis lies not only in identifying anatomical abnormalities but also in the need for multi-dimensional correlation and comprehensive analysis of imaging features with external environmental parameters (such as temperature, humidity, and scene conditions) to form more accurate comprehensive conclusions. Existing single image analysis algorithms or simple multimodal fusion models struggle to effectively integrate temporal-spatial contextual information between high-dimensional image slices and lack mechanisms for refined feature fusion of specific anatomical regions and dynamic coupling with the external environment. Specifically, traditional methods often neglect the spatial order and contextual dependencies between slices when processing multi-slice image data, resulting in incomplete three-dimensional feature representation. In anatomical region analysis, they fail to perform adaptive feature weighting and fusion based on the characteristics of different organ regions, affecting the accuracy of cause-of-death classification. Furthermore, the lack of an interactive analysis mechanism between environmental parameters and image features makes it difficult to adaptively adjust the time of death estimation in conjunction with dynamic factors at the scene. Thus, these shortcomings restrict the objectivity, accuracy, and interpretability of forensic diagnosis.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a forensic intelligent diagnostic method and system based on multimodal imaging and deep learning, which aims to improve the automation, objectivity and accuracy of forensic diagnosis.

[0005] To achieve the above objectives, this application proposes a forensic intelligent diagnostic method based on multimodal imaging and deep learning, the method comprising:

[0006] Acquire forensic virtual autopsy image datasets and environmental parameter datasets;

[0007] The forensic virtual autopsy image dataset is preprocessed to obtain a standardized image dataset;

[0008] The standardized image dataset is subjected to multi-slice feature extraction processing to obtain a set of multi-slice feature vectors;

[0009] The multi-slice feature vector set is subjected to temporal-spatial fusion processing to obtain a three-dimensional context feature vector;

[0010] The three-dimensional context feature vector is subjected to anatomical region feature fusion processing to obtain a multi-region fused feature vector;

[0011] Forensic diagnostic analysis is performed based on the multi-region fusion feature vector and the environmental parameter dataset to obtain a forensic intelligent diagnostic result set; including performing cause-of-death classification processing on the multi-region fusion feature vector to obtain cause-of-death classification results, and performing time-of-death inference processing based on the multi-region fusion feature vector and the environmental parameter dataset to obtain time-of-death inference results;

[0012] The forensic intelligent diagnostic result set, the standardized image dataset, and the environmental parameter dataset are processed to generate a visualization report, resulting in an interpretable forensic diagnostic report.

[0013] In one embodiment, the step of performing multi-slice feature extraction processing on the standardized image dataset to obtain a set of multi-slice feature vectors includes:

[0014] The standardized image dataset is subjected to key organ region extraction processing to obtain a key organ region dataset;

[0015] The key organ region dataset is subjected to key slice selection processing to obtain a key slice dataset.

[0016] Feature extraction processing is performed on the key slice dataset to obtain the multi-slice feature vector set.

[0017] In one embodiment, the step of performing key slice selection processing on the key organ region dataset to obtain a key slice dataset includes:

[0018] The key organ region dataset is subjected to three-dimensional spatial analysis to obtain a three-dimensional distribution dataset of organs.

[0019] The organ three-dimensional distribution dataset is sampled at equal intervals to obtain an equally spaced slice dataset.

[0020] The equally spaced slice dataset is subjected to feature importance analysis to obtain a slice importance dataset;

[0021] Based on the slice importance dataset, key slices are selected from the equally spaced slice dataset to obtain the key slice dataset.

[0022] In one embodiment, the step of performing temporal-spatial fusion processing on the multi-slice feature vector set to obtain a three-dimensional context feature vector includes:

[0023] The multi-slice feature vector set is spatially ordered to obtain a sequential feature sequence;

[0024] Perform inter-slice correlation analysis on the sequential feature sequence to obtain an inter-slice contextual relationship dataset;

[0025] The fusion weights of each slice are calculated based on the inter-slice context relationship dataset to obtain the slice weight dataset;

[0026] The multi-slice feature vector set is weighted and fused based on the slice weight dataset to obtain the three-dimensional context feature vector.

[0027] In one embodiment, the step of performing anatomical region feature fusion processing on the three-dimensional context feature vector to obtain a multi-region fused feature vector includes:

[0028] The standardized image dataset is subjected to anatomical region segmentation processing to obtain multiple anatomical region image datasets;

[0029] Feature extraction and temporal-spatial fusion processing were performed on the image datasets of each anatomical region to obtain the feature vectors of each anatomical region.

[0030] The feature vectors of each anatomical region are dynamically weighted and fused to obtain the multi-region fused feature vector.

[0031] In one embodiment, the step of performing cause-of-death classification processing on the multi-region fused feature vector to obtain the cause-of-death classification result includes:

[0032] The multi-region fusion feature vectors are subjected to feature combination analysis to obtain a candidate feature combination dataset;

[0033] The candidate feature combination dataset is matched with the preset cause of death feature template to obtain the feature matching degree dataset.

[0034] Based on the feature matching degree dataset, the corresponding cause of death feature patterns are identified to obtain the cause of death feature pattern dataset.

[0035] The cause of death category is determined based on the aforementioned cause of death feature pattern dataset, and the cause of death classification result is obtained.

[0036] In one embodiment, the step of performing death time inference processing based on the multi-region fused feature vector and the environmental parameter dataset to obtain the death time inference result includes:

[0037] The environmental parameter dataset is subjected to feature encoding processing to obtain an environmental feature vector;

[0038] The environmental feature vector and the multi-region fusion feature vector are subjected to feature interaction processing to obtain the environmental modulation vector.

[0039] Based on the environmental modulation vector, the multi-region fusion feature vector is adaptively adjusted to obtain the environmental adaptive feature vector.

[0040] The time of death is inferred by performing time regression analysis on the environmental adaptive feature vector.

[0041] In one embodiment, the step of performing feature interaction processing on the environmental feature vector and the multi-region fusion feature vector to obtain the environmental modulation vector includes:

[0042] Use the environmental feature vector as the query vector and the multi-region fusion feature vector as the key vector;

[0043] Calculate the matching degree between the query vector and the key value vector to obtain the environment-image matching degree dataset;

[0044] Modulation coefficients are generated based on the environment-image matching degree dataset to obtain the environment modulation vector.

[0045] In one embodiment, the step of performing visualization report generation processing on the forensic intelligent diagnostic result set, the standardized image dataset, and the environmental parameter dataset to obtain an interpretable forensic diagnostic report includes:

[0046] The decision basis analysis and processing of the forensic intelligent diagnosis result set are performed to obtain the model decision basis dataset;

[0047] The model decision-making dataset and the standardized image dataset are subjected to heatmap mapping to obtain an attention heatmap dataset.

[0048] The attention heatmap dataset is subjected to 3D reconstruction processing to obtain a 3D visualization heatmap dataset;

[0049] The forensic intelligent diagnostic result set, the three-dimensional visualization heat map dataset, and the environmental parameter dataset are integrated and processed to obtain the interpretable forensic diagnostic report.

[0050] Furthermore, to achieve the above objectives, this application also proposes a forensic intelligent diagnostic system based on multimodal imaging and deep learning. The system includes: a memory, a processor, and a forensic intelligent diagnostic program based on multimodal imaging and deep learning stored in the memory and executable on the processor. The forensic intelligent diagnostic program based on multimodal imaging and deep learning is configured to implement the steps of the forensic intelligent diagnostic method based on multimodal imaging and deep learning.

[0051] The forensic intelligent diagnostic method and system proposed in this application, based on multimodal imaging and deep learning, automatically acquires and processes multimodal image data, combines environmental parameters for feature fusion, diagnostic analysis, and visualization report generation. This solves the problems of reliance on manual interpretation, low efficiency, and strong subjectivity in existing technologies, and can improve the automation, objectivity, and accuracy of forensic diagnosis, reduce reliance on manual intervention, and achieve efficient and interpretable diagnostic report generation. Attached Figure Description

[0052] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0053] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 This is a flowchart illustrating an embodiment of the forensic intelligent diagnostic method based on multimodal imaging and deep learning provided in this application.

[0055] Figure 2 This is a schematic diagram of a structure provided for an embodiment of the forensic intelligent diagnostic system based on multimodal imaging and deep learning according to this application.

[0056] Explanation of icon numbers:

[0057] 10. Memory; 20. Processor.

[0058] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0059] The technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of this application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0060] It should be understood that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0061] In existing technologies, forensic virtual autopsy image analysis mainly relies on manual interpretation and experience-based judgment by forensic experts. This is not only inefficient and time-consuming, but also makes diagnostic results susceptible to subjective factors, resulting in poor standardization and repeatability. Furthermore, existing single image analysis algorithms or simple multimodal fusion models struggle to effectively integrate temporal-spatial contextual information between high-dimensional image slices, and lack mechanisms for refined feature fusion of specific anatomical regions and dynamic coupling with the external environment. Consequently, it is difficult to form accurate and comprehensive forensic diagnostic conclusions.

[0062] Based on this, embodiments of this application provide a forensic intelligent diagnostic method based on multimodal imaging and deep learning, referring to... Figure 1 The forensic intelligent diagnostic method based on multimodal imaging and deep learning includes steps S100 to S700, wherein:

[0063] Step S100: Obtain the forensic virtual autopsy image dataset and environmental parameter dataset;

[0064] Step S200: Perform image preprocessing on the forensic virtual autopsy image dataset to obtain a standardized image dataset;

[0065] Step S300: Perform multi-slice feature extraction processing on the standardized image dataset to obtain a set of multi-slice feature vectors;

[0066] Step S400: Perform temporal-spatial fusion processing on the multi-slice feature vector set to obtain a three-dimensional context feature vector;

[0067] Step S500: Perform anatomical region feature fusion processing on the three-dimensional context feature vector to obtain a multi-region fused feature vector;

[0068] Step S600: Perform forensic diagnostic analysis based on the multi-region fusion feature vector and the environmental parameter dataset to obtain a forensic intelligent diagnostic result set; including performing cause-of-death classification processing on the multi-region fusion feature vector to obtain cause-of-death classification results, and performing time-of-death inference processing based on the multi-region fusion feature vector and the environmental parameter dataset to obtain time-of-death inference results.

[0069] Step S700: Perform visualization report generation processing on the forensic intelligent diagnostic result set, the standardized image dataset, and the environmental parameter dataset to obtain an interpretable forensic diagnostic report.

[0070] In this embodiment, the forensic virtual autopsy image dataset refers to a collection of three-dimensional images of the deceased human body acquired through medical imaging technologies such as computed tomography (CT) and magnetic resonance imaging (MRI) for forensic analysis. This dataset typically contains multiple slice images, reflecting the morphology, structure, and pathological changes of internal organs and tissues. The environmental parameter dataset refers to a collection of external environmental information related to the forensic case, such as temperature, humidity, lighting conditions at the crime scene, the posture of the body at the time of discovery, clothing, and other physical or chemical parameters that may affect the process of changes in the body. These parameters are helpful for estimating the time of death and analyzing the cause of death. The standardized image dataset refers to the forensic virtual autopsy image dataset after a series of image preprocessing operations (such as denoising, correction, registration, and normalization). Standardization eliminates noise and artifacts in the original image data, unifies the image size, resolution, and grayscale range, and provides consistent and high-quality input for subsequent feature extraction and analysis. The multi-slice feature vector set refers to a set of numerical vectors extracted from multiple slices in the standardized image dataset that characterize the image content and pathological features. Each vector represents an abstract feature representation of a slice, used to capture local and regional image information.

[0071] In this embodiment, the three-dimensional context feature vector refers to a feature vector obtained by temporal-spatial fusion processing of a multi-slice feature vector set, which comprehensively reflects the overall structure and interrelationship of image data in three-dimensional space. This vector captures the spatial dependencies between and within different slices, providing more comprehensive three-dimensional information. The multi-region fusion feature vector refers to a comprehensive feature vector obtained by anatomical region feature fusion processing of the three-dimensional context feature vector, which integrates the features of different anatomical regions. This vector considers the different importance of different organ or tissue regions in forensic diagnosis and weights or combines their features to form a more discriminative representation. The forensic intelligent diagnostic result set refers to the final diagnostic conclusion set obtained after forensic diagnostic analysis based on the multi-region fusion feature vector and environmental parameter dataset. This result set typically includes the cause of death classification result and the time of death estimation result. The interpretable forensic diagnostic report refers to a report generated by visually integrating the forensic intelligent diagnostic result set, standardized image dataset, and environmental parameter dataset. This report not only provides diagnostic conclusions but also demonstrates the basis of model decision-making through visualization, enhancing the transparency and credibility of the diagnostic process and facilitating review and understanding by forensic experts.

[0072] In this embodiment, the forensic intelligent diagnostic method based on multimodal imaging and deep learning first acquires a forensic virtual autopsy image dataset and an environmental parameter dataset. The forensic virtual autopsy image dataset can be obtained in various ways, such as by directly importing existing CT or MRI scan data from a hospital or forensic institution's image database, or by generating it through a new CT or MRI scan of the cadaver. The environmental parameter dataset can be manually recorded and input into the system by on-site investigators, or automatically collected by sensors deployed at the scene, such as recording information like temperature, humidity, and light intensity at the crime scene.

[0073] Subsequently, in this embodiment, the acquired forensic virtual autopsy image dataset is preprocessed to obtain a standardized image dataset. The purpose of image preprocessing is to eliminate noise and artifacts in the original images and to standardize the image format and size. Specifically, traditional image processing algorithms such as median filtering and Gaussian filtering can be used for noise reduction; image contrast can be enhanced through methods such as grayscale stretching and histogram equalization; and the images can be scaled and their resolution adjusted to meet the input requirements of subsequent deep learning models. For example, all images can be uniformly adjusted to 256x256 pixels and grayscale value normalization can be performed.

[0074] Furthermore, in this embodiment, multi-slice feature extraction processing is performed on the standardized image dataset to obtain a multi-slice feature vector set. This processing aims to extract discriminative features from the image data. One implementation is to independently apply a pre-trained convolutional neural network (CNN) model, such as VGGNet or ResNet, to each slice image in the standardized image dataset, converting each slice image into a fixed-dimensional feature vector. Thus, the feature vectors of all slice images collectively constitute the multi-slice feature vector set.

[0075] Building upon this, this embodiment performs temporal-spatial fusion processing on the multi-slice feature vector set to obtain a 3D context feature vector. This fusion process aims to integrate information from different slices and capture the 3D structural features of the image data. One implementation approach is to perform simple concatenation or average pooling operations on the multi-slice feature vector set according to their relative positions in 3D space. For example, all slice feature vectors can be directly concatenated into a longer vector, or the average value of all slice feature vectors can be calculated as the fused 3D context feature vector.

[0076] Next, in this embodiment, the anatomical region feature fusion processing is performed on the three-dimensional context feature vector to obtain a multi-region fused feature vector. This processing aims to highlight the role of different anatomical regions in diagnosis. One implementation is to predefine multiple anatomical regions of interest, then extract feature subsets related to these regions from the three-dimensional context feature vector, and perform simple summation or concatenation operations on these subsets to form a multi-region fused feature vector.

[0077] Furthermore, in this embodiment, forensic diagnostic analysis is performed based on multi-region fused feature vectors and environmental parameter datasets to obtain a forensic intelligent diagnostic result set. This analysis includes cause-of-death classification and time-of-death inference. For cause-of-death classification, the multi-region fused feature vectors can be directly input into a multi-classifier, such as a Support Vector Machine (SVM) or a Multilayer Perceptron (MLP), to predict possible cause-of-death categories. For time-of-death inference, the multi-region fused feature vectors can be simply concatenated with features from the environmental parameter dataset and then input into a regression model, such as a linear regression model or a random forest regression model, to predict the time of death.

[0078] Finally, in this embodiment, the forensic intelligent diagnostic result set, standardized image dataset, and environmental parameter dataset are processed to generate a visual report, resulting in an interpretable forensic diagnostic report. This processing aims to provide intuitive diagnostic evidence and results. One implementation method is to present the diagnostic results directly in text form and simply display the slice images from the standardized image dataset. Simultaneously, the environmental parameter dataset can be attached to the report in the form of a table or list for forensic experts to refer to.

[0079] In this embodiment, multimodal images and environmental parameters are automatically acquired, and multi-level feature extraction and fusion are performed on the image data, including multi-slice feature extraction, temporal-spatial fusion, and anatomical region feature fusion, effectively integrating the contextual information between high-dimensional image slices. Simultaneously, this method jointly analyzes image features and environmental parameters, achieving intelligent classification of causes of death and estimation of time of death. Therefore, it overcomes the problems of low efficiency, strong subjectivity, and insufficient standardization in traditional forensic diagnosis, significantly improving the accuracy, objectivity, and interpretability of forensic diagnosis.

[0080] In one feasible implementation, the step of performing multi-slice feature extraction processing on the standardized image dataset to obtain a multi-slice feature vector set includes: performing key organ region extraction processing on the standardized image dataset to obtain a key organ region dataset; performing key slice selection processing on the key organ region dataset to obtain a key slice dataset; and performing feature extraction processing on the key slice dataset to obtain the multi-slice feature vector set.

[0081] In this embodiment, key organ region extraction is performed on the standardized image dataset to identify and separate specific organs or anatomical regions closely related to forensic diagnosis from the complete image data. This process can be achieved through various image segmentation techniques. For example, deep learning-based semantic segmentation models (such as U-Net and MaskR-CNN) can be used. These models, trained on a large amount of labeled data, can automatically and accurately delineate the boundaries of key organs such as the heart, lungs, liver, and brain. Alternatively, traditional image processing methods, such as thresholding, region growing, and active contour models, can be used in conjunction with anatomical prior knowledge or medical image registration techniques to separate key organ regions from the standardized image dataset, forming a key organ region dataset. This step effectively focuses the analysis scope and eliminates interference from background noise and irrelevant regions.

[0082] In this embodiment, key organ region datasets undergo key slice selection processing. The aim is to further filter out the most representative and diagnostically valuable image slices from the extracted key organ regions. For example, the importance of a slice can be assessed based on statistical characteristics of its content (such as pixel intensity variance, texture complexity, and information entropy), selecting the slice with the highest information content. Alternatively, machine learning models can be used to classify or regress slices to predict their contribution to the diagnostic results, thus selecting slices with high contribution. Furthermore, expert experience or pre-defined anatomical landmarks can be incorporated, for example, selecting slices containing specific lesions or important anatomical structures. This processing significantly reduces the amount of data requiring further analysis, improving the efficiency and focus of subsequent processing.

[0083] In this embodiment, feature extraction is performed on the key slice dataset to convert selected key slices into numerical feature vectors that can be processed by deep learning models. This process typically utilizes deep convolutional neural networks (CNNs) as feature extractors; for example, pre-trained models such as ResNet, VGG, and Inception can be used, and fine-tuned according to the characteristics of forensic images. These CNN models can automatically learn and extract high-level semantic features from images, such as texture, shape, edges, and lesion patterns. After feature extraction, each key slice generates a high-dimensional feature vector. These feature vectors together form a multi-slice feature vector set, providing rich and refined input for subsequent diagnostic analysis.

[0084] In this embodiment, the above-described technical solution first extracts key organ regions from the standardized image dataset, focusing the analysis on anatomical structures closely related to forensic diagnosis and effectively eliminating background noise and interference from irrelevant areas. Based on this, the dataset further selects key slices, ensuring that only the slices with the most information and diagnostic value are processed subsequently, significantly reducing data redundancy and improving the efficiency and specificity of feature extraction. Finally, feature extraction is performed on these selected key slice datasets, resulting in a more refined and discriminative set of multi-slice feature vectors. This provides high-quality input for subsequent temporal-spatial fusion, anatomical region feature fusion, and the final forensic diagnostic analysis, thereby improving the accuracy and reliability of the overall diagnostic method.

[0085] In one feasible implementation, the step of performing key slice selection processing on the key organ region dataset to obtain a key slice dataset includes: performing three-dimensional spatial analysis processing on the key organ region dataset to obtain an organ three-dimensional distribution dataset; performing equal-interval sampling processing on the organ three-dimensional distribution dataset to obtain an equal-interval slice dataset; performing feature importance analysis processing on the equal-interval slice dataset to obtain a slice importance dataset; and selecting key slices from the equal-interval slice dataset based on the slice importance dataset to obtain the key slice dataset.

[0086] In this embodiment, the key organ region dataset undergoes three-dimensional spatial analysis to gain a deeper understanding of the morphology, structure, and lesion distribution of key organs in three-dimensional space. Through three-dimensional spatial analysis, information such as organ volume, surface area, shape features, internal density distribution, lesion location, and size can be obtained. Specifically, various three-dimensional image processing techniques can be employed. For example, morphological operations such as connected component analysis, skeleton extraction, and distance transformation can be performed on the key organ region dataset to quantify the organ's geometric characteristics; or voxel-level analysis can be performed to statistically analyze the voxel distribution of different densities or grayscale values ​​within the organ to identify abnormal regions; algorithms such as Marching Cubes can be used to reconstruct the organ's three-dimensional surface model and then analyze its surface features; image segmentation algorithms can also be combined to automatically identify and quantify lesions within the organ and record their three-dimensional coordinates and dimensions. This step ensures that the analysis results accurately reflect the overall structure and local details of the organ, providing comprehensive spatial information for subsequent slice selection.

[0087] In this embodiment, the organ's three-dimensional distribution dataset is sampled at equal intervals. The purpose is to generate a series of slices at uniform intervals within the organ's three-dimensional space, ensuring coverage of all parts of the organ and providing a basic, unbiased slice set for subsequent feature importance analysis. Specifically, equal-interval slices can be taken along the organ's principal axis (e.g., the Z-axis, which is typically perpendicular to the scanning plane in CT / MRI images), with a fixed slice interval set according to the organ's size and the required sampling density. In addition to axial sampling, equal-interval sampling in the coronal and sagittal planes can also be considered to obtain a more comprehensive view. The setting of the sampling interval should balance information completeness and data volume, ensuring that no key information is missed while avoiding the generation of too many redundant slices.

[0088] In this embodiment, feature importance analysis is performed on the equidistant slice dataset to evaluate the amount of information contained in each slice and its contribution to diagnosis, thereby identifying the most representative and diagnostically valuable slices. Implementation methods may include: calculating the image information entropy of each slice, with higher entropy values ​​generally indicating greater information content; analyzing gray-level gradient changes within the slice, with high-gradient regions typically corresponding to important structures such as tissue boundaries and lesion edges; extracting texture features of the slices and evaluating importance through texture complexity and uniqueness; or combining lesion information identified in the organ's three-dimensional distribution dataset to calculate the coverage of lesions or the size of the lesion region contained in each slice, with slices containing more lesion information generally considered more important. The selection of importance assessment indicators should be closely related to the forensic diagnostic objectives, ensuring that key pathological features related to cause of death and time of death estimation are highlighted.

[0089] In this embodiment, key slices are selected from the equally spaced slice dataset based on the slice importance dataset to form the key slice dataset. This is the final screening step, selecting the most representative and diagnostically valuable slices from a large number of equally spaced slices based on the previously calculated slice importance, forming a concise and efficient set of key slices. Selection strategies may include: setting an importance threshold, where all slices with importance scores higher than the threshold are selected as key slices; or sorting slices by importance score from high to low and selecting a predetermined number of slices as key slices; or performing cluster analysis on the slice importance dataset and selecting representative slices near each cluster center. In addition to considering importance scores, the distribution of slices in three-dimensional space can also be considered during selection to ensure that the selected key slices uniformly cover important areas of the organ, avoiding excessive concentration of key slices in a small area.

[0090] In this embodiment, the aforementioned technical solution enables three-dimensional spatial analysis of the key organ region dataset, providing a comprehensive understanding of the organ's overall morphology, structure, and internal lesion distribution, thus offering a solid spatial basis for subsequent slice selection. Based on this, equidistant sampling ensures unbiased, uniformly distributed slice samples from all dimensions and depths of the organ, avoiding the omission of crucial information due to subjective selection or insufficient sampling. Subsequently, feature importance analysis of these equidistant slices quantifies the potential contribution of each slice to forensic diagnosis, accurately identifying slices containing rich pathological information and playing a decisive role in cause-of-death classification and time-of-death estimation. Finally, selecting key slices based on the slice importance dataset significantly reduces the amount of data to be processed, improves the efficiency of subsequent feature extraction and deep learning model training and inference, and ensures that the selected slices represent the organ's pathological characteristics to the greatest extent possible, thereby enhancing the accuracy and reliability of intelligent forensic diagnosis.

[0091] In one feasible implementation, the step of performing temporal-spatial fusion processing on the multi-slice feature vector set to obtain a three-dimensional context feature vector includes: spatially arranging the multi-slice feature vector set to obtain a sequential feature sequence; performing inter-slice correlation analysis on the sequential feature sequence to obtain an inter-slice context relationship dataset; calculating the fusion weight of each slice based on the inter-slice context relationship dataset to obtain a slice weight dataset; and performing weighted fusion processing on the multi-slice feature vector set based on the slice weight dataset to obtain the three-dimensional context feature vector.

[0092] In this embodiment, the feature vector set of multiple slices is spatially ordered, aiming to logically sort the extracted feature vectors according to the relative positions of the original image slices in three-dimensional space. For example, the slices can be sorted in ascending or descending order based on their Z-axis coordinates or scan sequence numbers in the virtual anatomical image, thereby ensuring that the feature sequence accurately reflects the layering relationship of the anatomical structure. In this way, a sequential feature sequence can be obtained, in which the position of each feature vector corresponds to the position of its corresponding image slice in three-dimensional space.

[0093] In this embodiment, inter-slice correlation analysis is performed on the sequential feature sequence to quantify the degree of interdependence between different slices in terms of content or features. This can be achieved by calculating the similarity between feature vectors of adjacent slices or slices within a certain distance (e.g., cosine similarity, the reciprocal of Euclidean distance). Alternatively, a deep learning-based attention mechanism (such as self-attention) can be used to allow the model to learn and output the degree of attention each slice pays to other slices, thereby forming an inter-slice contextual relationship dataset that reflects the complex spatial relationships between slices.

[0094] In this embodiment, the fusion weights of each slice are calculated based on the inter-slice contextual relationship dataset, aiming to assign a numerical value representing the importance or contribution of each slice. For example, the weights can be determined based on the correlation strength between a slice and other key slices, with slices with stronger correlations potentially being assigned higher weights. Alternatively, a small neural network module can be used, taking the inter-slice contextual relationship dataset as input, to learn and predict the optimal contribution ratio of each slice in the final 3D representation, thereby obtaining a slice weight dataset. These weights reflect the roles and influences that different slices should play in constructing the overall 3D context.

[0095] In this embodiment, a weighted fusion process is performed on the feature vector set of multiple slices based on the slice weight dataset. This step generates a single 3D context feature vector by multiplying each slice feature vector by its corresponding fusion weight and then summing or averaging all the weighted feature vectors. This weighted fusion method ensures that, when integrating information, slices deemed more important or more context-dependent have a greater impact on the final 3D representation, thereby more accurately capturing the comprehensive information of the entire 3D anatomical structure.

[0096] In this embodiment, the above-described technical solution first arranges the feature vector sets of multiple slices in spatial order, ensuring the preservation of the spatial continuity of the three-dimensional anatomical structure. Subsequently, through inter-slice correlation analysis, the intrinsic connections and contextual information between different slices can be deeply explored, avoiding the isolation of local information caused by treating each slice as an independent entity. Based on this, fusion weights are calculated according to the contextual relationships between slices, allowing for dynamic adjustment during feature fusion based on the importance and relevance of each slice in the overall three-dimensional structure. This effectively avoids information redundancy or dilution of key information that may result from simple averaging or splicing. Finally, through weighted fusion processing, the generated three-dimensional contextual feature vector can more comprehensively and accurately represent the three-dimensional information of the forensic virtual autopsy image, providing a more discriminative feature representation for subsequent cause-of-death classification and time-of-death estimation, significantly improving the accuracy and reliability of forensic intelligent diagnosis.

[0097] In one feasible implementation, the step of performing anatomical region feature fusion processing on the three-dimensional context feature vector to obtain a multi-region fused feature vector includes: performing anatomical region segmentation processing on the standardized image dataset to obtain multiple anatomical region image datasets; performing feature extraction and temporal-spatial fusion processing on each anatomical region image dataset to obtain each anatomical region feature vector; and performing dynamic weighted fusion processing on each anatomical region feature vector to obtain the multi-region fused feature vector.

[0098] In this embodiment, anatomical region segmentation is performed on the standardized image dataset to divide the complete standardized image dataset into sub-regions with specific anatomical significance. This helps to decompose complex overall images into more easily analyzed and understood local units, thereby enabling differentiated processing based on the characteristics of different regions. During implementation, various image segmentation techniques can be employed. For example, deep learning-based semantic segmentation models (such as U-Net, Mask R-CNN, etc.) can be pre-trained on a large amount of labeled forensic image data to automatically identify and segment key organ or tissue regions such as the heart, lungs, liver, kidneys, and brain. Alternatively, traditional image processing methods such as thresholding, region growing, and level set methods can be combined, or predefined anatomical templates can be used for registration and segmentation. The result of the segmentation process is multiple independent image datasets, each corresponding to a specific anatomical region.

[0099] In this embodiment, feature extraction and temporal-spatial fusion processing are performed separately for each anatomical region image dataset. This aims to perform specialized feature extraction for each independent anatomical region image dataset and fuse it while considering its internal temporal and spatial information. This ensures that the unique information of each anatomical region is fully captured, and its internal structure and variation patterns are effectively encoded, avoiding potential feature confusion in the overall image. For feature extraction, convolutional neural networks (CNNs) can be used as feature extractors, such as pre-trained models like ResNet, VGG, and Inception, or 3D CNNs specifically designed for medical imaging. These networks can learn high-level visual features such as texture, shape, and density from anatomical region images. Regarding temporal-spatial fusion processing, considering that forensic virtual anatomical images are typically multi-slice or three-dimensional data, each anatomical region image dataset may contain multiple slices. Temporal-spatial fusion aims to integrate information between these slices, which can be achieved in the following ways: directly processing 3D image data of the anatomical region using 3D convolutional layers to naturally capture spatial and slice-related contextual information (which can be viewed as the "temporal" dimension); or inputting the slice feature sequence of each anatomical region into a recurrent neural network (RNN) or Transformer model to learn the dependencies and contextual information between slices, and then aggregating the output (e.g., average pooling, max pooling, or attention mechanisms) to obtain the fused feature vector of the region; alternatively, different weights can be assigned to different slices or different spatial locations within each anatomical region to highlight important information. Ultimately, each anatomical region will generate a vector representing its core features.

[0100] In this embodiment, the feature vectors of each anatomical region are dynamically weighted and fused. This aims to integrate feature vectors obtained from different anatomical regions to form a comprehensive, multi-region fused feature vector representing the overall pathological state. The key here is "dynamic weighting," which allows the system to adaptively adjust the importance of different anatomical region features based on the current diagnostic task or image content, thereby more accurately reflecting pathological changes. During implementation, an attention network can be introduced. This network receives all anatomical region feature vectors as input and outputs a set of weights, each corresponding to an anatomical region. These weights are dynamically generated, reflecting the relative importance of each region under the current diagnostic task. For example, when diagnosing heart disease, the heart region will have a higher weight; when diagnosing lung disease, the lung region will have a higher weight. Alternatively, a gating unit can be designed to control the inflow and fusion degree of feature information from different regions, or a small multilayer perceptron (MLP) network can be used to learn how to combine these region feature vectors, where the weights of the MLP can be considered implicitly dynamically weighted. During training, through the backpropagation algorithm, the model can learn how to assign appropriate weights to each anatomical region based on the input image and diagnostic target to maximize diagnostic accuracy. Finally, by multiplying and summing the feature vectors of each anatomical region with their corresponding dynamic weights (or performing other forms of aggregation), a comprehensive multi-region fusion feature vector is obtained.

[0101] In this embodiment, the standardized image dataset is first finely segmented into anatomical regions using the aforementioned technical solution. This allows the system to independently analyze the characteristics of different organ or tissue regions, effectively avoiding mutual interference between features from different regions. Subsequently, feature extraction and temporal-spatial fusion processing are performed on each anatomical region's image dataset, ensuring that the internal structure and variation patterns of each anatomical region are fully captured, forming representative regional feature vectors. Based on this, by dynamically weighting and fusing the feature vectors of each anatomical region, the system can adaptively adjust the importance of different anatomical regions according to the diagnostic task or image content, thereby highlighting information from key pathological regions and suppressing interference from secondary regions. This regional, dynamically weighted fusion strategy enables the final multi-region fused feature vector to more comprehensively and accurately reflect the pathological information required for forensic diagnosis, significantly improving the accuracy and reliability of intelligent forensic diagnosis.

[0102] In one feasible implementation, the step of performing cause-of-death classification processing on the multi-region fused feature vector to obtain the cause-of-death classification result includes: performing feature combination analysis processing on the multi-region fused feature vector to obtain a candidate feature combination dataset; performing matching calculation processing on the candidate feature combination dataset and a preset cause-of-death feature template to obtain a feature matching degree dataset; identifying the corresponding cause-of-death feature pattern based on the feature matching degree dataset to obtain a cause-of-death feature pattern dataset; and performing cause-of-death category judgment processing based on the cause-of-death feature pattern dataset to obtain the cause-of-death classification result.

[0103] In this embodiment, feature combination analysis is performed on the multi-region fused feature vector to obtain a candidate feature combination dataset. The aim is to mine representative feature combinations relevant to cause-of-death determination from the multi-region fused feature vector. The multi-region fused feature vector may contain a large amount of information, but not all information is equally important for identifying a specific cause of death. Feature combination analysis can identify subsets of features that co-occur, are correlated, or have strong indicative power for the cause of death. This can be achieved through various machine learning or deep learning techniques. For example, attention-based methods can be used to allow the model to automatically learn which feature combinations contribute most to cause-of-death classification; or feature selection algorithms, such as recursive feature elimination or Lasso regression, can be used to select the optimal feature subset; or cluster analysis or association rule mining can be used to discover potential combination patterns between features. Finally, these identified feature sets with potential diagnostic significance are organized into a candidate feature combination dataset.

[0104] In this embodiment, the candidate feature combination dataset is matched with a preset cause-of-death feature template to obtain a feature matching degree dataset. The purpose of this step is to compare the candidate feature combinations extracted from the image data with known feature templates representing specific causes of death, quantifying their similarity or correlation. The preset cause-of-death feature templates are constructed based on a large number of historical forensic cases, expert knowledge, or medical literature, and contain combination patterns of typical imaging features, pathological features, or biological indicators of various known causes of death. Matching calculations can employ various similarity measurement methods, such as cosine similarity, Euclidean distance, Pearson correlation coefficient, etc., or learn the matching relationship between feature combinations and templates through deep learning models (such as Siamese Network or metric learning). The results of the matching calculations form a feature matching degree dataset, which reflects the degree of fit between each candidate feature combination and each cause-of-death feature template.

[0105] In this embodiment, the corresponding cause-of-death feature patterns are identified based on the feature matching degree dataset, resulting in a cause-of-death feature pattern dataset. After obtaining the feature matching degree dataset, this step aims to identify the cause-of-death feature patterns that best match the current case. This typically involves analyzing and filtering the matching degree dataset. For example, a matching degree threshold can be set, and cause-of-death feature templates with matching degrees higher than the threshold can be selected; or a sorting mechanism can be used to select several cause-of-death feature templates with the highest matching degrees as candidate patterns. Alternatively, an expert system or rule engine can be combined to comprehensively judge and identify the most likely cause-of-death feature patterns based on matching degrees, the weight of feature combinations, and other contextual information. The identified cause-of-death feature patterns are organized into a cause-of-death feature pattern dataset, which represents the most likely pathophysiological manifestations or injury types corresponding to the current case.

[0106] In this embodiment, the cause of death category is determined based on the aforementioned cause-of-death feature pattern dataset to obtain the cause-of-death classification result. The final cause-of-death category is determined based on the identified cause-of-death feature pattern dataset. This step maps the cause-of-death feature patterns to specific cause-of-death classification labels. For example, if the identified feature patterns are "extensive intracranial hemorrhage" and "brainstem compression," the classification might be "death due to traumatic brain injury." The determination process can utilize multi-classifiers, such as support vector machines, random forests, and neural networks, taking the cause-of-death feature patterns as input and outputting the final cause-of-death category. Alternatively, a decision tree or rule base can be constructed to directly provide a cause-of-death judgment based on different combinations of feature patterns. The final output cause-of-death classification result is a clear cause-of-death category for this forensic case.

[0107] In this embodiment, through the above technical solution, when classifying causes of death using multi-region fused feature vectors, this application no longer simply performs end-to-end classification. Instead, it first performs feature combination analysis on the fused feature vectors, thereby extracting more diagnostically significant and interrelated feature subsets from complex image features, effectively reducing the dimensionality and noise of the feature space. Subsequently, by matching these candidate feature combinations with preset cause-of-death feature templates, the similarity between the current case and known cause-of-death patterns is quantified, making the cause-of-death judgment process more objective and quantitative. Furthermore, specific cause-of-death feature patterns are identified based on the matching degree. This not only improves the accuracy of cause-of-death classification but also provides clear pathological evidence for the final cause-of-death category judgment, enhancing the interpretability of the diagnostic results. This step-by-step, pattern-matching classification strategy effectively solves the problems of insufficient accuracy and poor interpretability that may exist in direct classification. It is particularly suitable for the complex and ever-changing cause-of-death judgment scenarios in forensic medicine, ensuring the scientificity and reliability of the diagnostic results.

[0108] In one feasible implementation, the step of performing death time inference processing based on the multi-region fusion feature vector and the environmental parameter dataset to obtain the death time inference result includes: performing feature encoding processing on the environmental parameter dataset to obtain an environmental feature vector; performing feature interaction processing on the environmental feature vector and the multi-region fusion feature vector to obtain an environmental modulation vector; performing adaptive adjustment processing on the multi-region fusion feature vector based on the environmental modulation vector to obtain an environmental adaptive feature vector; and performing time regression analysis processing on the environmental adaptive feature vector to obtain the death time inference result.

[0109] In this embodiment, the environmental parameter dataset typically includes various external environmental factors such as temperature, humidity, light intensity, wind speed, and water immersion. These factors significantly influence the decomposition and rate of change of the corpse. Feature encoding of the environmental parameter dataset aims to transform these raw, heterogeneous environmental parameters into unified, numerical environmental feature vectors for processing by deep learning models. For example, continuous parameters (such as temperature and humidity) can be normalized or standardized; discrete parameters (such as clothing type and exposure environment type) can be represented using one-hot encoding or embedding layers. Furthermore, time series analysis (such as calculating averages and rates of change) can be used to extract more representative environmental features.

[0110] In this embodiment, the environmental feature vector and the multi-region fused feature vector are subjected to feature interaction processing to obtain an environmental modulation vector. The purpose of feature interaction processing is to deeply explore the intrinsic relationship between the environmental feature vector and the multi-region fused feature vector, generating an environmental modulation vector that reflects their synergistic effect. This processing goes beyond simple feature splicing and aims to capture how environmental factors affect the expression and changes of anatomical features within the cadaver. In addition to the method of matching the environmental feature vector as a query vector and the multi-region fused feature vector as a key vector to generate modulation coefficients, other interaction methods can be used. For example, a multilayer perceptron (MLP) or attention mechanism layer can be used to learn the nonlinear mapping relationship between environmental features and image features, or a gating mechanism (such as multiplication gating) can be used to control the degree of influence of environmental information on image features, thereby generating a more informative environmental modulation vector.

[0111] In this embodiment, the multi-region fusion feature vector is adaptively adjusted based on the environmental modulation vector to obtain an environmentally adaptive feature vector. This adaptive adjustment process uses the environmental modulation vector to dynamically correct or enhance the original multi-region fusion feature vector, thus yielding the environmentally adaptive feature vector. This process enables the model to intelligently adjust the weights and interpretations of cadaver image features according to specific environmental conditions when inferring the time of death. For example, this can be achieved by performing element-wise multiplication or addition between the environmental modulation vector and the multi-region fusion feature vector, or by using a small neural network with the environmental modulation vector as input to generate adjustment parameters, thereby transforming the multi-region fusion feature vector. This adjustment ensures higher accuracy and robustness in interpreting image features under different environments.

[0112] In this embodiment, time regression analysis is performed on the environmentally adaptive feature vector to obtain the time of death estimation result. Time regression analysis maps the environmentally adaptively adjusted feature vector to a specific time of death estimation result. This process is typically implemented using a regression model that learns the complex nonlinear relationship between the feature vector and the time of death. Specifically, a deep regression neural network (such as a multilayer perceptron) can be used, with its output layer designed to predict a continuous numerical value (i.e., the time of death). Alternatively, machine learning models such as Support Vector Regression (SVR), Gaussian Process Regression, or ensemble learning methods (such as Random Forest Regression or Gradient Boosting Regression) can be used to establish a mapping relationship between features and the time of death through training, thereby outputting an accurate time of death estimation result.

[0113] In this embodiment, the environmental parameter dataset is feature-encoded using the aforementioned technical solution, transforming heterogeneous environmental information into a unified environmental feature vector, laying the foundation for subsequent deep interaction. Furthermore, the environmental feature vector and the multi-region fusion feature vector undergo feature interaction processing to generate an environmental modulation vector. This step delves deeper into the intrinsic correlation between environmental factors and cadaver image features, rather than simply superimposing them. Based on this environmental modulation vector, the multi-region fusion feature vector is adaptively adjusted, enabling the interpretation of image features to dynamically adapt to different environmental conditions, thereby generating a more environmentally sensitive adaptive environmental feature vector. Finally, time regression analysis of this adaptive environmental feature vector significantly improves the accuracy and robustness of time-of-death estimation, especially under complex and variable environmental conditions. This effectively overcomes the inference bias caused by insufficient consideration of environmental factors or simplistic processing methods in traditional methods, resulting in more refined and reliable forensic diagnostic results.

[0114] In one feasible implementation, the step of performing feature interaction processing on the environmental feature vector and the multi-region fusion feature vector to obtain an environmental modulation vector includes: using the environmental feature vector as a query vector and the multi-region fusion feature vector as a key vector; calculating the matching degree between the query vector and the key vector to obtain an environment-image matching degree dataset; and generating modulation coefficients based on the environment-image matching degree dataset to obtain the environmental modulation vector.

[0115] In this embodiment, in the step of using the environmental feature vector as the query vector and the multi-region fusion feature vector as the key-value vector, the environmental feature vector is designated as the query vector, and its role is to actively "ask" or "explore" the correlation between it and the multi-region fusion feature vector. The multi-region fusion feature vector is designated as the key-value vector, which contains comprehensive feature information representing different anatomical regions extracted from forensic virtual autopsy images, and serves as the object of the query. This role allocation draws on the concept of attention mechanisms, aiming to evaluate the correlation strength between the query vector and each part of the key-value vector, thereby laying the foundation for subsequent feature interactions. For example, the query vector can be encoded environmental information, such as temperature, humidity, and discovery time, while the key-value vector can be a vector containing the morphological, density, and pathological changes of various anatomical regions of the corpse, such as the brain, heart, and liver.

[0116] In this embodiment, the matching degree between the query vector and the key-value vector is calculated to obtain an environment-image matching degree dataset. The matching degree reflects the correlation or similarity between environmental features and image features. This matching degree can be calculated in various ways; for example, a dot product can be used to measure the similarity between two vectors, or cosine similarity can be used to assess their directional proximity. Alternatively, a small neural network layer can be used to learn and output a matching score. The calculation results form the environment-image matching degree dataset, where each value represents the matching strength between a specific environmental feature and a specific image feature. For example, if the environmental feature vector represents a low-temperature environment, and the image feature vector contains features of cadaver changes caused by low temperature, the matching degree will be high.

[0117] In this embodiment, modulation coefficients are generated based on the environment-image matching degree dataset to obtain an environment modulation vector. After obtaining the environment-image matching degree dataset, this dataset is used to generate modulation coefficients. The modulation coefficients are weights or factors dynamically generated according to the matching degree, used to adjust or "modulate" the multi-region fusion feature vector. Methods for generating modulation coefficients may include normalizing the matching degree dataset, for example, by using a Softmax function to make its sum equal to 1, thereby obtaining a set of weights; or by using a gating mechanism to determine the degree of information flow based on the matching degree. These generated modulation coefficients are then applied to the multi-region fusion feature vector, for example, by element-wise multiplication or weighted summation, to obtain the environment modulation vector. The environment modulation vector is a feature representation "tuned" by environmental information, which more accurately reflects the contribution of image features to the time of death inference under specific environmental conditions.

[0118] In this embodiment, the above-described technical solution uses the environmental feature vector as the query vector and the multi-region fusion feature vector as the key vector, calculating the degree of matching between the two. This allows for explicit modeling of the interaction between environmental factors and cadaver image features. This modulation coefficient generation mechanism based on matching degree enables environmental information to adaptively and finely weight and adjust the multi-region fusion feature vector, thereby generating an environmental modulation vector with greater environmental context awareness. Compared to simple feature combinations, this solution can more accurately capture the impact of the environment on changes in the cadaver, resulting in more accurate and reliable results for subsequent death time inference processing based on the environmental modulation vector, significantly improving the accuracy and reliability of forensic intelligent diagnosis.

[0119] In one feasible implementation, the steps of generating an interpretable forensic diagnostic report by visualizing the forensic intelligent diagnostic result set, the standardized image dataset, and the environmental parameter dataset include: performing decision basis analysis on the forensic intelligent diagnostic result set to obtain a model decision basis dataset; performing heatmap mapping on the model decision basis dataset and the standardized image dataset to obtain an attention heatmap dataset; performing three-dimensional reconstruction on the attention heatmap dataset to obtain a three-dimensional visual heatmap dataset; and performing report integration processing on the forensic intelligent diagnostic result set, the three-dimensional visual heatmap dataset, and the environmental parameter dataset to obtain the interpretable forensic diagnostic report.

[0120] In this embodiment, the decision basis analysis of the forensic intelligent diagnostic result set is performed to obtain the model decision basis dataset. This processing aims to reveal the key features or data points relied upon by the internal decision-making mechanism of the intelligent diagnostic model when arriving at a specific diagnostic result. Specifically, various interpretable artificial intelligence (XAI) techniques can be employed, such as gradient-based saliency mapping methods (e.g., Grad-CAM), locally interpretable models (e.g., LIME), or game theory methods (e.g., SHAP), to quantify which regions in the standardized image dataset or which features in the multi-region fused feature vector contribute most to the final cause-of-death classification result or time-of-death inference result. Through this analysis, a model decision basis dataset reflecting the model's decision weights can be generated, thus providing a foundation for subsequent visualization.

[0121] In this embodiment, after obtaining the model decision-making basis dataset, this application further performs heatmap mapping processing on it and the standardized image dataset to obtain an attention heatmap dataset. The purpose of this step is to visualize the abstract model decision-making basis onto the original image data, enabling forensic experts to intuitively see the areas "focused" by the model. Specifically, the weights or importance scores in the model decision-making basis dataset can be superimposed on each slice of the corresponding standardized image dataset in the form of color coding. For example, high-weight areas can be represented by warm colors (such as red and yellow), while low-weight areas are represented by cool colors (such as blue and green), thereby forming a series of two-dimensional attention heatmaps. These heatmaps can clearly indicate the pathological areas or anatomical structures in the images that have a decisive influence on the diagnostic results.

[0122] In this embodiment, to provide a more comprehensive spatial understanding, the attention heatmap dataset undergoes 3D reconstruction processing to obtain a 3D visualization heatmap dataset. Since forensic virtual autopsy images are inherently 3D, elevating the 2D attention heatmap to 3D space better aligns with the cognitive habits of forensic experts regarding anatomical structures. This processing utilizes slice information from the attention heatmap dataset, employing techniques such as volume rendering or surface reconstruction to fuse a series of 2D heatmaps into a continuous 3D visualization model. For example, isosurface extraction algorithms (such as Marching Cubes) or direct volume rendering techniques can be used to interpolate and visualize the attention weights on different slices in 3D space, thereby generating a 3D visualization heatmap that can be viewed from any angle, clearly demonstrating the model's focus of attention within the entire 3D anatomical structure.

[0123] In this embodiment, the application integrates the forensic intelligent diagnostic result set, the 3D visualization heatmap dataset, and the environmental parameter dataset into a report to obtain an interpretable forensic diagnostic report. This step aims to gather all key information into a structured, easy-to-understand document. The report not only includes the final results of the intelligent diagnosis (such as the cause-of-death classification results and the estimated time of death results), but also visually demonstrates the imagery basis for the model's judgments through 3D visualization heatmaps, and provides comprehensive contextual information by combining relevant environmental parameter datasets. The report can be in various formats, such as a PDF document or an interactive digital report, which may include text descriptions, 2D slice images, 3D interactive models, and data tables, ensuring that forensic experts can fully and deeply understand every step and decision-making logic of the intelligent diagnosis.

[0124] In this embodiment, the above-described technical solution effectively addresses the lack of transparency and interpretability in intelligent diagnostic results. By conducting in-depth analysis of the model's decision-making basis and mapping it onto a standardized image dataset in the form of an intuitive attention heatmap, followed by further 3D reconstruction, forensic experts can clearly see the key areas the model focuses on within the 3D anatomical structure. This visualization not only enhances the reliability and credibility of the diagnostic results but also provides experts with a basis for verifying the model's judgments. Ultimately, integrating the diagnostic results, the 3D visualization heatmap, and the environmental parameter dataset into an interpretable forensic diagnostic report enables forensic experts to comprehensively and systematically understand the entire process and decision-making logic of intelligent diagnosis, thereby greatly promoting the practical application and dissemination of intelligent diagnostic technology in the field of forensic medicine and improving diagnostic efficiency and accuracy.

[0125] In the embodiments of this application, the forensic intelligent diagnosis method based on multimodal imaging and deep learning automatically acquires and processes multimodal image data, combines environmental parameters for feature fusion, diagnostic analysis, and visualization report generation, solving the problems of reliance on manual interpretation, low efficiency, and strong subjectivity in the prior art. It can improve the automation, objectivity, and accuracy of forensic diagnosis, reduce reliance on manual intervention, and achieve efficient and interpretable diagnostic report generation.

[0126] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the forensic intelligent diagnostic method based on multimodal imaging and deep learning. Any simple modifications based on this technical concept are within the scope of protection of this application.

[0127] This application also provides a forensic intelligent diagnostic system based on multimodal imaging and deep learning, referenced in [reference]. Figure 2The system includes: a memory 10, a processor 20, and a forensic intelligent diagnostic program based on multimodal imaging and deep learning stored in the memory 10 and executable on the processor 20. The forensic intelligent diagnostic program based on multimodal imaging and deep learning is configured to implement the steps of the forensic intelligent diagnostic method based on multimodal imaging and deep learning.

[0128] The forensic intelligent diagnostic system based on multimodal imaging and deep learning provided in this application, employing the forensic intelligent diagnostic method based on multimodal imaging and deep learning in the above embodiments, can improve the automation, objectivity, and accuracy of forensic diagnosis. Compared with the prior art, the beneficial effects of the forensic intelligent diagnostic system based on multimodal imaging and deep learning provided in this application are the same as those of the forensic intelligent diagnostic method based on multimodal imaging and deep learning provided in the above embodiments, and other technical features of the forensic intelligent diagnostic system based on multimodal imaging and deep learning are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0129] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0130] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. All equivalent structural transformations made under the technical concept of this application using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included within the scope of patent protection of this application.

Claims

1. A forensic intelligent diagnostic method based on multimodal imaging and deep learning, characterized in that, The method includes: Acquire forensic virtual autopsy image datasets and environmental parameter datasets; The forensic virtual autopsy image dataset is preprocessed to obtain a standardized image dataset; The standardized image dataset is subjected to multi-slice feature extraction processing to obtain a set of multi-slice feature vectors; The multi-slice feature vector set is subjected to temporal-spatial fusion processing to obtain a three-dimensional context feature vector; The three-dimensional context feature vector is subjected to anatomical region feature fusion processing to obtain a multi-region fused feature vector; Forensic diagnostic analysis is performed based on the multi-region fusion feature vector and the environmental parameter dataset to obtain a forensic intelligent diagnostic result set; including performing cause-of-death classification processing on the multi-region fusion feature vector to obtain cause-of-death classification results, and performing time-of-death inference processing based on the multi-region fusion feature vector and the environmental parameter dataset to obtain time-of-death inference results; The forensic intelligent diagnostic result set, the standardized image dataset, and the environmental parameter dataset are processed to generate a visualization report, resulting in an interpretable forensic diagnostic report. The steps for performing temporal-spatial fusion processing on the multi-slice feature vector set to obtain a three-dimensional context feature vector include: The multi-slice feature vector set is spatially ordered to obtain a sequential feature sequence; Perform inter-slice correlation analysis on the sequential feature sequence to obtain an inter-slice contextual relationship dataset; The fusion weights of each slice are calculated based on the inter-slice context relationship dataset to obtain the slice weight dataset; The multi-slice feature vector set is weighted and fused based on the slice weight dataset to obtain the three-dimensional context feature vector. The steps for performing anatomical region feature fusion processing on the three-dimensional context feature vector to obtain a multi-region fused feature vector include: The standardized image dataset is subjected to anatomical region segmentation processing to obtain multiple anatomical region image datasets; Feature extraction and temporal-spatial fusion processing were performed on the image datasets of each anatomical region to obtain the feature vectors of each anatomical region. The feature vectors of each anatomical region are dynamically weighted and fused to obtain the multi-region fused feature vector. The steps for performing death time inference processing based on the multi-region fused feature vector and the environmental parameter dataset to obtain the death time inference result include: The environmental parameter dataset is subjected to feature encoding processing to obtain an environmental feature vector; The environmental feature vector and the multi-region fusion feature vector are subjected to feature interaction processing to obtain the environmental modulation vector. Based on the environmental modulation vector, the multi-region fusion feature vector is adaptively adjusted to obtain the environmental adaptive feature vector. The time of death is inferred by performing time regression analysis on the environmental adaptive feature vector.

2. The forensic intelligent diagnostic method based on multimodal imaging and deep learning as described in claim 1, characterized in that, The steps for performing multi-slice feature extraction on the standardized image dataset to obtain a set of multi-slice feature vectors include: The standardized image dataset is subjected to key organ region extraction processing to obtain a key organ region dataset; The key organ region dataset is subjected to key slice selection processing to obtain a key slice dataset. Feature extraction processing is performed on the key slice dataset to obtain the multi-slice feature vector set.

3. The forensic intelligent diagnostic method based on multimodal imaging and deep learning as described in claim 2, characterized in that, The steps for performing key slice selection processing on the key organ region dataset to obtain the key slice dataset include: The key organ region dataset is subjected to three-dimensional spatial analysis to obtain a three-dimensional distribution dataset of organs. The organ three-dimensional distribution dataset is sampled at equal intervals to obtain an equally spaced slice dataset. The equally spaced slice dataset is subjected to feature importance analysis to obtain a slice importance dataset; Based on the slice importance dataset, key slices are selected from the equally spaced slice dataset to obtain the key slice dataset.

4. The forensic intelligent diagnostic method based on multimodal imaging and deep learning as described in claim 1, characterized in that, The steps for performing cause-of-death classification processing on the multi-region fused feature vector to obtain the cause-of-death classification result include: The multi-region fusion feature vectors are subjected to feature combination analysis to obtain a candidate feature combination dataset; The candidate feature combination dataset is matched with the preset cause of death feature template to obtain the feature matching degree dataset. Based on the feature matching degree dataset, the corresponding cause of death feature patterns are identified to obtain the cause of death feature pattern dataset. The cause of death category is determined based on the aforementioned cause of death feature pattern dataset, and the cause of death classification result is obtained.

5. The forensic intelligent diagnostic method based on multimodal imaging and deep learning as described in claim 1, characterized in that, The step of performing feature interaction processing on the environmental feature vector and the multi-region fused feature vector to obtain the environmental modulation vector includes: Use the environmental feature vector as the query vector and the multi-region fusion feature vector as the key vector; Calculate the matching degree between the query vector and the key value vector to obtain the environment-image matching degree dataset; Modulation coefficients are generated based on the environment-image matching degree dataset to obtain the environment modulation vector.

6. The forensic intelligent diagnostic method based on multimodal imaging and deep learning as described in claim 1, characterized in that, The steps for generating an interpretable forensic diagnostic report by visualizing the forensic intelligent diagnostic result set, the standardized image dataset, and the environmental parameter dataset include: The decision basis analysis and processing of the forensic intelligent diagnosis result set are performed to obtain the model decision basis dataset; The model decision-making dataset and the standardized image dataset are subjected to heatmap mapping to obtain an attention heatmap dataset. The attention heatmap dataset is subjected to 3D reconstruction processing to obtain a 3D visualization heatmap dataset; The forensic intelligent diagnostic result set, the three-dimensional visualization heat map dataset, and the environmental parameter dataset are integrated and processed to obtain the interpretable forensic diagnostic report.

7. A forensic intelligent diagnostic system based on multimodal imaging and deep learning, characterized in that, The system includes: a memory, a processor, and a forensic intelligent diagnostic program based on multimodal imaging and deep learning stored in the memory and executable on the processor, wherein the forensic intelligent diagnostic program based on multimodal imaging and deep learning is configured to implement the steps of the forensic intelligent diagnostic method based on multimodal imaging and deep learning as described in any one of claims 1 to 6.

Images