A deep learning forensic CT image assisted cause of death inference system and method

Abstract

This invention relates to the fields of medical image processing, artificial intelligence, and forensic identification technology. It discloses a deep learning-based forensic CT image-assisted cause-of-death deduction system and method, addressing the knowledge barriers and talent bottlenecks in CT image interpretation during forensic identification. The system includes a data acquisition module, an image preprocessing module, a multimodal feature extraction and fusion module, and an auxiliary inference module. The method acquires PMCT images and case information, preprocesses the images, extracts and fuses multimodal features, uses a pre-trained classification model to calculate the probability of the cause of death category, and outputs an auxiliary report. This invention integrates forensic knowledge with deep learning technology to achieve automated and standardized interpretation of CT images. By combining multimodal data from images and case information for comprehensive analysis, the output results are interpretable, providing forensic personnel with an objective and quantitative auxiliary judgment tool, significantly improving the efficiency, scientific rigor, and objectivity of forensic identification.

Description

Technical Field

[0001] This invention belongs to the fields of medical image processing, artificial intelligence and forensic identification technology, and specifically relates to a deep learning forensic CT image-assisted cause of death prediction system and method. Background Technology

[0002] In modern forensic identification, computed tomography (CT) scanning technology is widely used in autopsy examinations due to its advantages such as being non-invasive, rapid, and providing permanent data, making it an important technical tool in forensic identification. However, the application of forensic CT technology currently faces significant knowledge barriers and a talent bottleneck, specifically manifested in the following ways: Mismatch of professional experience: Forensic practitioners often lack systematic clinical radiology reading experience, making it difficult to accurately identify key pathological or injury features from complex CT images; while clinical radiologists, although proficient in image interpretation, lack forensic pathology knowledge and are unable to accurately establish forensic correlations between injuries, diseases, and causes of death.

[0003] High communication costs: Cross-disciplinary collaboration between forensic and radiologists suffers from information discrepancies, resulting in low communication efficiency and image interpretation results that easily deviate from the actual needs of forensic identification.

[0004] Highly subjective and lacking standardization: Traditional manual image interpretation relies heavily on personal experience, resulting in highly subjective results. At the same time, it is difficult to standardize and quantify the analysis and description of minute features such as minor fractures, air embolism, and micro-bleeds.

[0005] Inefficient: Manual image review and feature annotation are time-consuming and cannot meet the timeliness requirements of forensic identification.

[0006] Therefore, there is an urgent need for a forensic CT image analysis system that integrates forensic medical expertise with artificial intelligence technology to achieve automated and standardized interpretation of CT images, overcome talent bottlenecks, provide forensic personnel with objective and quantitative tools to infer the cause of death, and improve the efficiency and scientific rigor of forensic identification. Summary of the Invention

[0007] The purpose of this invention is to provide a deep learning-based forensic CT image-assisted cause-of-death estimation system and method.

[0008] To achieve the above objectives, the present invention provides the following technical solution: An Automatic Recognition and Auxiliary Inference System for Forensic CT Images The system includes a data acquisition module, an image preprocessing module, a multimodal feature extraction and fusion module, and an auxiliary inference module, which are sequentially connected in communication. It also includes a result output module that is connected in communication with the auxiliary inference module. 1. Data Acquisition Module This is used to acquire post-mortem computed tomography (PMCT) image data, case text data, and basic individual information of the target subject. The PMCT image data is in standard DICOM format and is the raw data from the forensic CT scan; the case text data includes either structured or unstructured text describing the location of the body discovery, preliminary on-site investigation, and descriptions of injuries; and the basic individual information includes fundamental data such as the deceased's age and gender, providing supplementary reference for determining the cause of death.

[0009] 2. Image preprocessing module PMCT image data undergoes enhancement and standardization processing. Core operations include window width and level adjustment, bone artifact removal, voxel normalization, and multi-scale contrast enhancement. The display effects of bone windows, lung windows, and soft tissue windows are optimized for forensic focus areas such as bones, soft tissues, and hollow organs. This removes bone artifacts from soft tissue observation, unifies voxel data scales, and enhances the contrast of minute lesion features to ensure the accuracy of subsequent feature extraction.

[0010] 3. Multimodal feature extraction and fusion module This module is the core module of the system, realizing the extraction and deep fusion of image visual features and text semantic features: Image feature extraction: Using the self-developed 3D ResFormer model, targeting the three-dimensional characteristics of forensic CT images, CNN is used in parallel to extract local texture and edge features, and Transformer is used to model global context features, finally outputting a 1024-dimensional visual feature vector. Text feature extraction: The BioMed-Forensic BERT model is adopted, which is pre-trained on PubMed biomedical corpus and fine-tuned with forensic identification reports. It can accurately identify forensic entities, extract text semantic features and map them into a 1024-dimensional semantic feature vector. Feature fusion: A two-stage cross-attention fusion algorithm is adopted. First, text-guided image enhancement focuses image features on areas related to the case. Then, bidirectional feature interaction between image and text and between text and image is achieved. Finally, a comprehensive feature vector is generated through multi-scale fusion methods such as splicing, element-level maximum value, and element-level average value to fully capture the deep semantic relationships between modalities.

[0011] 4. Auxiliary Inference Module The comprehensive feature vector is input into either a pre-trained classification model, MLP or XGBoost, to output the probability distribution of the cause of death category. At the same time, a heat map of key areas is generated using Grad-CAM technology, highlighting the core basis for the model's judgment on the original CT images. The built-in confidence assessment unit evaluates the credibility of the prediction results. Finally, the probability distribution and heat map results are combined to automatically generate a text-based auxiliary analysis report, clarifying the suspected cause of death, key positive findings, and imaging descriptions.

[0012] 5. Result Output Module The system enables visualization and document export of results, including: visualization of the probability ranking of cause of death categories, interactive viewing of CT images with overlaid heatmaps, and export of auxiliary analysis reports in either Word or PDF format, facilitating review by forensic personnel and inclusion in identification reports.

[0013] A method for forensic CT imaging-assisted cause of death estimation Based on the above system, the present invention also provides a method for forensic CT image-assisted cause of death estimation, comprising the following steps: S1, Data Acquisition The data acquisition module imports PMCT image data in DICOM format, along with case text data and basic individual information such as the deceased's age and gender, to complete the initial data acquisition.

[0014] S2, Image Preprocessing The image preprocessing module sequentially performs window width and level adjustment, bone artifact removal, voxel normalization, and multi-scale contrast enhancement on PMCT images to generate enhanced three-dimensional CT image data that highlights forensic identification features.

[0015] S3, Multimodal Feature Extraction The enhanced 3D CT images were encoded using the 3D ResFormer model to automatically identify forensic imaging features such as fractures, bleeding, and air embolism, and to extract 1024-dimensional visual feature vectors. The BioMed-Forensic BERT model was used to segment the case text, identify forensic entities, extract semantic features, and map them into a 1024-dimensional semantic feature vector. Age is numerically normalized and gender is categorically embedded, integrating basic individual information into the feature vector.

[0016] S4, Multimodal Feature Fusion Feature fusion using the DSCMAF algorithm: Phase 1: Text-guided image enhancement, using text features as queries to enhance the feature response of areas in the image that are relevant to the case. The second stage: collaborative cross-attention to achieve bidirectional feature interaction between image and text, and between text and image, allowing image features to absorb text semantics and text features to absorb image visual information; Multi-scale fusion: The features after bidirectional interaction are concatenated, and the element-level maximum and element-level average values ​​are fused. Then, they are compressed into a comprehensive feature vector suitable for classification through a fully connected layer.

[0017] S5, Auxiliary Cause of Death Deduction The comprehensive feature vectors are input into the pre-trained MLP / XGBoost classification model, which outputs the probability distribution of various causes of death. The Grad-CAM technology generates a heat map of key areas on the original CT images. Combining the probability distribution and heat map results, a text-based auxiliary analysis report containing suspected causes of death, key positive findings, and imaging descriptions is automatically generated.

[0018] S6. Output Results The results output module visualizes the ranking of causes of death probabilities and overlays CT images with heatmaps, supports interactive viewing by forensic personnel, and exports auxiliary analysis reports in Word / PDF format.

[0019] 7. Model Iterative Optimization As new forensic identification cases accumulate, the image data, case information, and final identification conclusions of the new cases are added to the training dataset. The classification model is incrementally trained regularly to continuously optimize the model's prediction accuracy and form a virtuous cycle of model iteration.

[0020] Classification model training methods The core classification model in the system is trained through the following steps: 1. Constructing a training dataset: Collect forensic CT image data from clinical medical examination cases and historical closed cases. Senior forensic and radiology experts jointly annotate key injury areas in the images and associate them with the corresponding final cause of death identification conclusions to form a gold standard for annotation.

[0021] 2. Data Augmentation: Randomly rotate, translate, scale, and flip the CT images in the training dataset to expand the data volume and effectively prevent model overfitting.

[0022] 3. Pre-training and fine-tuning: First, the 3DResFormer backbone network and BioMed-Forensic BERT language model are pre-trained using large public medical image datasets such as NIH ChestX-ray to enable them to have general medical feature extraction capabilities; then, the entire model is fine-tuned end-to-end using a forensic-specific dataset to optimize the model parameters and make it accurately suited for forensic cause-of-death inference tasks.

[0023] 4. Model Validation: The trained model is tested using an independent validation set to evaluate core metrics such as accuracy, recall, and AUC. Once the metrics meet the preset thresholds for forensic industry applications, the model training is completed, and a classification model that can be put into practical application is obtained.

[0024] The beneficial effects of this invention are as follows: This invention's system integrates forensic medicine and radiology expertise, enabling ordinary forensic personnel to obtain expert-level CT image analysis support without relying on external radiologists, thus resolving the mismatch between the professional experience of forensic and radiologists. Analysis results based on historical big data eliminate the subjectivity of personal experience, achieving standardized and quantitative descriptions of forensic image features and improving the objectivity of identification results.

[0025] Breaking through the limitations of single-image analysis, this system achieves multimodal deep fusion of CT images with case text and individual information. By capturing semantic relationships between modalities through bidirectional cross-attention, the resulting cause-of-death deductions better align with the actual needs of forensic practice. It not only outputs the probability distribution of the cause of death but also highlights key judgment areas using heatmaps, generating supplementary reports with detailed imaging descriptions. This makes the AI's judgment basis transparent, not replacing forensic decision-making but providing accurate references for forensic review.

[0026] The system supports incremental training with new case data. As forensic cases accumulate, the model's prediction accuracy can be continuously optimized, resulting in a virtuous cycle of technological and practical application. The system enhances the features of minor injuries of interest to forensic scientists, supports the DICOM standard image format, and auxiliary reports can be directly exported to commonly used document formats, seamlessly integrating with the daily forensic identification workflow. Detailed Implementation

[0027] The specific embodiments of the present invention will be described in further detail below with reference to the examples. These examples are for illustrative purposes only and are not intended to limit the scope of the invention.

[0028] Unless otherwise specified, the technical solutions described in this invention are all conventional solutions in the field; unless otherwise specified, the reagents or materials described are all from commercial sources.

[0029] The actual application process of the system This embodiment uses a forensic examination of a case involving an accidental death at a construction site as an example to illustrate the actual application process of this system: 1. Data Collection: The forensic doctor imported the deceased's PMCT image DICOM file and entered the case text: Male, 45 years old, found at the construction site, with external injuries to the head and crush marks on the neck, as well as personal information: 45 years old, male; 2. Image preprocessing: The system automatically adjusts the window width and level of CT images, removes bone artifacts, normalizes voxels, and enhances multi-scale contrast to highlight the characteristics of minor intracranial and cervical injuries. 3. Multimodal feature extraction: The 3D ResFormer model extracts image features to identify the fracture line of the 3rd-4th cervical vertebrae, intracranial subdural hematoma, and soft tissue contusion of the neck; the BioMed-Forensic BERT model extracts text features to identify key information such as head trauma, construction site, and male age of 45. 4. Multimodal feature fusion: The DSCMAF algorithm uses head trauma and neck compression in the text as a guide to enhance the feature response of the skull, brain tissue and cervical spine regions in the image, and then realizes bidirectional feature interaction between the image and the text to generate a comprehensive feature vector. 5. Assisted in determining the cause of death: The classification model outputs the probability distribution of the cause of death: mechanical injury: 85%, asphyxiation: 10%, sudden death: 5%. Using Grad-CAM technology, key areas of cervical spine fracture and intracranial hematoma are highlighted on CT images. An auxiliary report is automatically generated. Based on the CT image features and case information, mechanical injury is highly suspected. The key findings are fracture of the 3rd-4th cervical vertebrae, intracranial subdural hematoma, and soft tissue contusion of the neck. It is recommended to focus on examining the correlation between neck and head injuries and the cause of death. 6. Results Output: Forensic experts can view the CT images with overlaid heat maps in the system, review key injury areas, export auxiliary reports in PDF format, and incorporate them into the final forensic identification report.

[0030] Example 2: Model Training and Validation This example illustrates the training and validation process of the system classification model: 1. Dataset Construction: Collect 10,000 historical forensic case records and 5,000 clinical medical imaging cases. Five senior forensic experts and three radiology specialists (associate chief physicians or above) jointly annotated the images. The annotations included more than 20 forensic imaging signs such as fractures, bleeding, and air embolism, and associated them with eight causes of death, including mechanical injury, asphyxiation, poisoning, and sudden death, forming the gold standard dataset for annotation. 2. Data Augmentation: Randomly rotate, translate, scale, and flip the CT images in the dataset to expand the dataset to 50,000 cases; 3. Pre-training and fine-tuning: First, 3D ResFormer and BioMed-Forensic BERT were pre-trained using the NIH ChestX-ray14 dataset, and then end-to-end fine-tuning was performed using a forensic dataset. The training batch size was 32, the learning rate was 1e-4, and the number of training epochs was 50. 4. Model Validation: The model was tested using a 2000-case independent validation set. The results showed that the model achieved an accuracy of 92.5% and a recall of 90.3% for identifying major causes of death, with an AUC of 0.94, meeting the practical application needs of the forensic industry.

[0031] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made under the design concept of the present invention should be included within the scope of protection of the present invention. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered to be within the scope of this specification.

[0032] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention. Therefore, the protection scope of this invention patent should be determined by the appended claims.

Claims

1. A deep learning forensic CT image-assisted cause-of-death inference system, characterized by, include: The data acquisition module is used to acquire cadaver tomographic image data of the target object, as well as related case text data and individual basic information; the cadaver tomographic image data is in DICOM format, the case text data includes the discovery location and preliminary investigation information, and the individual basic information includes age and gender; The image preprocessing module is communicatively connected to the data acquisition module and is used to enhance and standardize the cadaver tomographic image data to highlight forensic identification features. The processing operations include at least window width and window level adjustment, bone artifact removal, voxel normalization, and multi-scale contrast enhancement, which are optimized for bone window, lung window, and soft tissue window respectively, to highlight the injury characteristics of bones, soft tissues, and hollow organs. The multimodal feature extraction and fusion module is communicatively connected to the image preprocessing module. It is used to extract visual features of the preprocessed image based on a deep learning network, extract semantic features of the case text data through a domain-adapted language model, and deeply fuse the visual features and semantic features to generate a comprehensive feature vector. The auxiliary inference module is communicatively connected to the multimodal feature extraction and fusion module. It is used to input the comprehensive feature vector into the pre-trained classification model and output auxiliary judgment information about the cause of death category. The auxiliary judgment information includes the probability distribution of the cause of death category, the visual annotation of key image evidence, and the text-based auxiliary analysis report.

2. The system according to claim 1, characterized in that, The deep learning network in the multimodal feature extraction and fusion module is the 3DResFormer model with a 3D hybrid CNN-Transformer architecture, which is used to automatically identify one or more forensic imaging features such as fracture, hemorrhage, air embolism, pneumothorax, fat embolism, and subdural hematoma from 3D CT data; the domain-adapted language model is the BioMed-Forensic BERT model, which is pre-trained on biomedical corpus and fine-tuned with forensic identification reports, and can identify forensic entities and extract high-dimensional semantic features.

3. The system according to claim 1, characterized in that, The multimodal feature extraction and fusion module uses a two-stage cross-attention fusion algorithm for feature fusion. First, the image enhancement stage guided by text focuses the image features on areas related to the case. Then, the collaborative cross-attention stage realizes bidirectional feature interaction between image and text, and between text and image. Finally, a multi-scale fusion method using splicing, element-level maximum value, and element-level average value is used to generate a comprehensive feature vector.

4. The system according to claim 1, characterized in that, The pre-trained classification model is a classifier based on multilayer perceptron or XGBoost, trained using image data from a large number of historical forensic cases and clinical medical examination cases, along with corresponding final cause-of-death identification conclusions. The auxiliary inference module also includes a confidence assessment unit, used to evaluate the credibility of the model's prediction results. Simultaneously, it generates a heatmap using Grad-CAM technology, highlighting key areas for judgment on the original CT images.

5. The system according to any one of claims 1-4, characterized in that, It also includes a result output module, which is connected in communication with the auxiliary inference module, for displaying and exporting the probability distribution of cause of death categories, heat map visualization annotation, and text-based auxiliary analysis report; the text-based auxiliary analysis report can be exported in either Word or PDF format, and includes suspected cause of death, key positive findings and corresponding imaging descriptions.

6. A method for forensic CT image-assisted cause-of-death deduction based on the system described in any one of claims 1-5, characterized in that, Includes the following steps: S1. Data Acquisition: The data acquisition module acquires DICOM format PMCT image data, case text data, and basic individual information such as age and gender of the target object. The case text includes unstructured case text and structured case text. S2. Image preprocessing: The image preprocessing module performs window width and window level adjustment, bone removal artifact processing, voxel normalization, and multi-scale contrast enhancement operations on the PMCT image data in sequence to generate enhanced three-dimensional CT image data. S3. Multimodal Feature Extraction: The enhanced 3D CT image data is encoded using the 3DResFormer model to extract a 1024-dimensional visual feature vector; the case text data is segmented and entity recognized using the BioMed-Forensic BERT model, and then extracted and mapped into a 1024-dimensional semantic feature vector; individual basic information is numerically processed and incorporated into the feature vector. S4. Multimodal feature fusion: Visual feature vectors and semantic feature vectors are deeply fused through a two-stage cross-attention fusion algorithm. First, enhanced visual feature vectors are obtained through text-guided image enhancement. Then, feature interaction is achieved through bidirectional collaborative cross-attention. Finally, a comprehensive feature vector is generated through multi-scale fusion and dimensionality compression. S5. Assisted Cause of Death Inference: Input the comprehensive feature vector into the pre-trained classification model, output the probability distribution of cause of death category, and generate a heat map of key areas through Grad-CAM technology. Combine the probability distribution and heat map results to automatically generate a text-based auxiliary analysis report. S6. Results Output: The results output module displays the ranking of causes of death probabilities, CT images with overlaid heat maps, and supports the export of auxiliary analysis reports.

7. The method according to claim 6, characterized in that, The classification model training method in step S5 includes the following steps: S1. Constructing a training dataset: Collect forensic CT image data from clinical medical examination cases and historical closed cases. Forensic and radiology experts annotate key injury areas in the images and associate them with the corresponding final cause of death identification conclusions to form a gold standard for annotation. S2. Data Augmentation: Randomly rotate, translate, scale, and flip CT images in the training dataset to increase the amount of data and prevent model overfitting; S3. Pre-training and fine-tuning: First, the 3DResFormer backbone network and BioMed-Forensic BERT language model are pre-trained using a large public medical image dataset. Then, the entire model is fine-tuned end-to-end using a forensic dataset to optimize the model parameters. S4. Model Validation: Test the trained model using an independent validation set to evaluate accuracy, recall, and AUC. Once the metrics meet the preset thresholds, the model training is complete, and a usable classification model is obtained.

8. The method according to claim 6, characterized in that, The method also includes a model iterative optimization step: as new forensic identification cases accumulate, the image data, case information and final identification conclusions of the new cases are added to the training dataset, and the classification model is incrementally trained regularly to continuously optimize the model's prediction accuracy.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a computer processor, implement the forensic CT image-assisted cause-of-death deduction method as described in any one of claims 6-8.