Three-dimensional reconstruction analysis method for forensic injury based on multi-modal image fusion

Abstract

The embodiment of the present application relates to a kind of forensic injury three-dimensional reconstruction analysis method based on multimodal image fusion, the method comprises: obtaining the internal tomographic image data of the person identified and body surface three-dimensional scanning data;The internal tomographic image data and the body surface three-dimensional scanning data are time-space unified registration, and unified time-space coordinate system is established;Based on the preset machine learning model, under the unified time-space coordinate system, damage related tissue region is automatically segmented;Based on the data after registration and segmentation, deformation interpolation is carried out between different time nodes, and four-dimensional reconstruction model showing the dynamic change of damage region morphology with time is generated;In the unified time-space coordinate system, at least one damage quantification index is measured, and the damage quantification index is matched with the preset damage identification standard knowledge base, and damage degree evaluation result is output.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of artificial intelligence image recognition technology, specifically to a method for three-dimensional reconstruction and analysis of forensic injuries based on multimodal image fusion. Background Technology

[0002] In forensic clinical identification practice, determining the degree of injury is one of the core tasks. The examiner needs to make a scientific and objective judgment on the morphology, timing, and mechanism of the injury, and issue an identification opinion accordingly. With the popularization of medical imaging technology, computed tomography (CT) and magnetic resonance imaging (MRI) have become routine examination methods for forensic injury identification.

[0003] Currently, forensic injury assessment is typically conducted manually through image review. Individuals rely on their experience to identify signs of injury such as fracture lines, soft tissue edema, and callus formation, and then combine this information with case details such as the time of injury and the weapon used. However, this process has significant limitations: First, the conclusions drawn from image review are highly dependent on the examiner's personal experience; different examiners may interpret the same image differently, leading to strong subjectivity and a lack of consistency in the expert opinion. Second, for cases requiring follow-up examinations at different time points to observe the dynamic evolution of the injury, examiners can only make subjective judgments by comparing image sequences from different time points, lacking an effective means of integrating images from discrete time points into a continuous dynamic process. Summary of the Invention

[0004] Based on the above-mentioned situation of the prior art, the purpose of the embodiments of the present invention is to provide a three-dimensional reconstruction and analysis method for forensic injuries based on multimodal image fusion, which can fuse, reconstruct and analyze multimodal and multi-temporal image data under a unified spatiotemporal framework, thereby realizing the objectivity and interpretability of forensic identification.

[0005] To achieve the above objectives, according to one aspect of the present invention, a method for three-dimensional reconstruction and analysis of forensic injuries based on multimodal image fusion is provided, the method comprising: Acquire internal tomographic images and three-dimensional surface scan data of the subject of the examination, wherein the internal tomographic images include data from at least two different time points; The internal tomographic image data and the three-dimensional scan data of the body surface are spatiotemporally registered to establish a unified spatiotemporal coordinate system. Based on a pre-set machine learning model, the damage-related tissue regions are automatically segmented in the unified spatiotemporal coordinate system. Based on the registered and segmented data, deformation interpolation is performed between the different time points to generate a four-dimensional reconstruction model that shows the dynamic changes in the morphology of the damaged area over time. Under the unified spatiotemporal coordinate system, at least one damage quantification index is measured, and the damage quantification index is matched with a preset damage identification standard knowledge base to output the damage degree assessment result.

[0006] Furthermore, spatiotemporal registration is performed on the internal tomographic image data and the three-dimensional surface scan data, including: The three-dimensional scan data of the body surface is coarsely aligned with the body surface contour reconstructed from the internal tomographic image data using principal component analysis. The iterative nearest point algorithm with normal constraints is used for iterative optimization to transform the coordinate system of the three-dimensional surface scan data to the coordinate system of the internal tomographic image data. The error function of the iterative nearest point algorithm with normal constraints introduces a penalty term for the angle between the point cloud normal vectors.

[0007] Furthermore, the spatiotemporal registration of the internal tomographic image data and the three-dimensional surface scan data also includes: For internal tomographic image data at different time points, alignment is performed in the time dimension using data from one of the specified time points as a fixed reference frame.

[0008] Furthermore, the preset machine learning model includes a multi-task 3D convolutional neural network; the multi-task 3D convolutional neural network includes a semantic segmentation branch, an edge morphology classification branch, and a healing stage regression branch.

[0009] Furthermore, the semantic segmentation branch is used to output the segmentation results of bones, soft tissues, fracture lines, callus, and edema areas; The edge morphology classification branch is used to output the probability that each voxel belongs to a fresh fracture edge or an old fracture edge; The healing phase regression branch is used to output callus thickness and density values ​​for each voxel.

[0010] Furthermore, a four-dimensional reconstruction model is generated that shows the dynamic changes in the morphology of the damaged area over time, including: The segmented data at different time points are reconstructed into three-dimensional surfaces to obtain three-dimensional mesh models for each time point. Using the three-dimensional mesh model of each time node as keyframes, a non-rigid registration algorithm is used to calculate the deformation field between adjacent keyframes; Based on the deformation field, a three-dimensional model is generated at any intermediate moment between the key frames to form a continuous time series model.

[0011] Furthermore, the method also includes stress field inversion: An initial finite element model in an undamaged state is constructed. Based on the epidermal damage morphology in the three-dimensional scan data of the body surface, a virtual pressure field is applied to the initial finite element model so that the deformation of the model surface is consistent with the three-dimensional scan data of the body surface. The direction and magnitude of the external force are iteratively solved, and the obtained stress values ​​are mapped into a heat map and superimposed on the four-dimensional reconstruction model. The elastic modulus of bones, soft tissues, and skin in the initial finite element model is based on the CT values ​​of the internal tomographic images.

[0012] Furthermore, the at least one damage quantification index includes fracture displacement distance, soft tissue damage volume, callus formation thickness, and fracture line length.

[0013] Furthermore, the damage quantification indicators are matched with a preset damage identification standard knowledge base, including: Map the qualitative description items in the damage assessment criteria to numerical judgment rules; Using damage quantification indicators as input, the system calls the numerical judgment rules and membership functions in the damage assessment standard knowledge base, performs fuzzy inference, and outputs the probability distribution and confidence level of each damage severity level.

[0014] Furthermore, the internal tomographic image data includes CT data, and the three-dimensional surface scan data includes point cloud data acquired by a light three-dimensional scanner.

[0015] In summary, the embodiments of the present invention provide a method for forensic injury three-dimensional reconstruction analysis based on multimodal image fusion. The method includes: acquiring internal tomographic image data and body surface three-dimensional scan data of the subject; performing spatiotemporal unified registration on the internal tomographic image data and the body surface three-dimensional scan data to establish a unified spatiotemporal coordinate system; automatically segmenting the injury-related tissue area under the unified spatiotemporal coordinate system based on a preset machine learning model; performing deformation interpolation between different time points based on the registered and segmented data to generate a four-dimensional reconstruction model showing the dynamic changes of the morphology of the injury area over time; measuring at least one injury quantification index under the unified spatiotemporal coordinate system, matching the injury quantification index with a preset injury identification standard knowledge base, and outputting the injury degree assessment result. The technical solution provided by the embodiments of the present invention fundamentally transforms forensic injury identification from experience-driven to data-driven approaches. It expands the object of image analysis from two-dimensional slices with a single modality and a single time point to four-dimensional image sequences that span multiple modalities and time points. The task of image recognition is upgraded from simple tissue structure identification to automatic reasoning of deep semantic attributes such as injury healing stage and direction of external force. This enables the automatic extraction of spatiotemporal evolution patterns and mechanical response characteristics from images, allowing the temporal attributes, degree level, and injury mode of the injury to be quantified and repeatedly verified by image algorithms, thereby significantly improving the objectivity and interpretability of forensic identification. Attached Figure Description

[0016] Figure 1 This is a flowchart of a forensic injury three-dimensional reconstruction analysis method based on multimodal image fusion provided by an embodiment of the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0018] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of the present invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in one or more embodiments of the present invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the element or object listed following the word and its equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0019] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings. The embodiments of the present invention provide a method for three-dimensional reconstruction and analysis of forensic injuries based on multimodal image fusion. Figure 1 The flowchart of the forensic injury three-dimensional reconstruction analysis method based on multimodal image fusion provided by the embodiments of the present invention is shown below. Figure 1 As shown, the method includes the following steps: S202. Acquire internal tomographic image data and three-dimensional surface scan data of the examinee, wherein the internal tomographic image data includes data from at least two different time points. In this embodiment of the invention, the internal tomographic image data is, for example, CT data. CT data is reconstructed using X-ray attenuation coefficients and has high contrast display capability for hard tissues with high density, such as bone. The internal tomographic image data includes data from at least two different time points. For example, the first time point is an emergency or initial examination completed within 24 hours after the injury, at which time the images can reflect the morphological characteristics of the acute phase of injury, such as the sharpness of the fracture ends, acute soft tissue edema, and hematoma; the second time point is a follow-up examination completed 30 to 45 days after the injury, at which time the callus begins to form and gradually fills the fracture gap, and the soft tissue edema tends to be absorbed, reflecting the typical histological changes during the injury repair period. The three-dimensional surface scan data includes point cloud data acquired by a three-dimensional scanner, with each frame of scan data appended with a corresponding timestamp label. A point cloud is a collection of three-dimensional coordinate points on the surface of an object. The optical 3D scanner projects coded structured light patterns onto the surface of an object and uses a camera to capture deformation images. Triangulation calculations then generate millions of spatial sampling points, forming point cloud data that describes the surface morphology. This point cloud data not only contains the three-dimensional spatial coordinates of each sampling point but also records the scanner's six-degree-of-freedom attitude (three-axis acceleration and three-axis angular velocity) in space at the moment of scanning using an inertial measurement unit fixed to the scanner body. This data is used to subsequently stitch together point cloud fragments from multiple scanning perspectives into a complete surface model.

[0020] S204. Perform spatiotemporal unified registration on the internal tomographic image data and the three-dimensional surface scan data to establish a unified spatiotemporal coordinate system. In this embodiment of the invention, before performing spatiotemporal unified registration, the internal tomographic image data and the three-dimensional surface scan data can be preprocessed, i.e., filtered, denoised, and interpolated, respectively, to obtain CT body data and three-dimensional point cloud data of the body surface. Spatiotemporal unified registration includes cross-modal registration and temporal dimension registration: S2041, Cross-modal registration.

[0021] S2041a. Principal component analysis is used to coarsely align the 3D surface scan data with the surface contour reconstructed from internal tomographic images. The principal component directions of the 3D spatial distribution are calculated for both the 3D point cloud data and the CT volume data. Specifically, the covariance matrices of the two sets of data are constructed, and eigenvalue decomposition is performed to obtain three eigenvectors sorted by eigenvalue, corresponding to the first, second, and third principal directions of the data distribution. The principal directions of the two sets of data are aligned by rotating and translating the 3D point cloud data so that its three principal directions coincide with or are opposite to the three principal directions of the CT volume data, while also ensuring that the centroids of the two sets of data coincide, thus achieving coarse alignment.

[0022] S2041b: Iterative optimization is performed using an iterative nearest-point algorithm with normal constraints to transform the coordinate system of the three-dimensional scan data of the body surface to the coordinate system of the internal tomographic image data. The error function of this iterative nearest-point algorithm with normal constraints incorporates a penalty term for the angle between the point cloud normal vectors. Iterative fine optimization is performed using the iterative nearest-point algorithm with normal constraints. In this embodiment of the invention, the error function of the iterative nearest point not only includes the sum of squared Euclidean distances from the source point cloud to the nearest point in the target point cloud, but also introduces a penalty term for the angle between the normal vectors. This term calculates the cosine of the angle between the normal vectors of the matched point pair. Its contribution is small when the angle is close to zero (i.e., the normal vector directions are consistent), and increases when the angle is close to 90 degrees or larger (i.e., the normal vector directions deviate significantly). The purpose of introducing the penalty term for the angle between the normal vectors is that, for a naturally continuous curved surface, the two correctly matched points should not only be geometrically close in spatial distance, but also in the direction of their local small planes. In facial depressions such as the mouth and eye sockets, without a penalty term for the included normal vector angle, a point in the 3D point cloud data might be closer in Euclidean distance to a point on the opposite sidewall of the depression in the CT volume data, even though their normal vectors are almost opposite, leading to mismatches. The penalty term for the included normal vector angle can suppress these mismatches.

[0023] S2042, Time Dimension Registration. For CT body data at different time points, alignment is performed in the time dimension using data from a specified time point as a fixed reference frame. For example, acute-phase CT data from 24 hours post-injury is selected as the fixed reference frame, and CT data from the recovery phase (30-45 days post-injury) is used as a floating frame for registration. In this embodiment, the floating and fixed frames are first registered using affine transformation (including twelve degrees of freedom: translation, rotation, scaling, and shearing) to correct for overall displacement, rotation, and slight anisotropic scaling caused by differences in scan positioning. The similarity metric for affine transformation registration uses normalized cross-correlation or mutual information, and the optimization method employs gradient descent or the L-BFGS algorithm. After completing global affine transformation registration, a free deformation model based on B-spline basis functions is introduced to handle local tissue deformation. The spatial domain of the data is divided into a grid composed of equally spaced control points. Each control point carries a three-dimensional displacement vector. The actual displacement of any voxel within the grid is obtained by weighted interpolation of the displacements of the surrounding control points using the B-spline basis functions that enclose it. In the B-spline basis functions, each control point only affects the displacement field within a finite neighborhood, rather than the entire volume data. This allows for flexible simulation of local deformations (such as the thickening of the callus region during fracture healing or the absorption and shrinkage of the edema region) without interfering with the aligned portions far from that region.

[0024] Through the above-mentioned unified spatiotemporal registration, all the three-dimensional point cloud data of the body surface and the CT volume data of each time node involved in the analysis are transformed into a unified spatiotemporal coordinate system with the CT volume data of the specified time node as the spatial reference. This enables the body surface injury morphology and the deep fracture at the corresponding location to be accurately located and jointly analyzed at any time.

[0025] S206. Based on a pre-set machine learning model, the damage-related tissue region is automatically segmented in a unified spatiotemporal coordinate system. In this embodiment of the invention, a pre-trained multi-task three-dimensional convolutional neural network is used as the machine learning model. This neural network can output semantic segmentation results of multiple types of tissues, morphological attributes of fracture edges, and quantitative regression values ​​of the healing stage from the input three-dimensional CT volume data at one time. The overall structure of the neural network is an improved 3D U-Net architecture. The encoder part uses the residual module of ResNet34 as the backbone extraction network, and performs three-dimensional convolution and pooling operations step by step to extract multi-scale image features from low-level texture to high-level semantics. The decoder part gradually restores the spatial resolution of the feature map through similar deconvolution or upsampling operations, and fuses the feature maps from the corresponding levels of the encoder through skip connections. At each fusion point of the skip connection, an attention gating module is introduced. This attention gating module learns a gating signal to weight the low-level features from the encoder, amplifying the feature responses related to the damage area (especially small but highly significant structures such as fracture lines and callus), and suppressing redundant information from normal background tissue. The neural network is divided into three parallel output branches at the end of the decoder. The first branch is the semantic segmentation branch, which outputs the segmentation results for bones, soft tissues, fracture lines, callus, and edema areas. This branch outputs a multi-channel 3D probability map, with each channel corresponding to a preset tissue category, including but not limited to bones, soft tissues, fracture lines, callus, and edema areas. For each voxel, the one with the highest probability in each channel is taken as its final category label, resulting in the semantic segmentation result. The second branch is the edge morphology classification branch, which outputs the probability that each voxel belongs to the edge of a fresh or old fracture. This branch outputs a two-channel 3D probability map, with the two channels corresponding to the classification probabilities of "irregular edges of fresh fractures" and "smooth edges of old fractures," respectively. The classification probability is based on the fact that early (fresh) fractures after injury have sharp fracture ends and irregular fracture surfaces, which appear on CT images as drastic changes in voxel grayscale and irregular geometric directions at the fracture line edges; while old fractures, after several weeks of repair, tend to have rounded and smooth fracture ends due to osteoclast absorption and callus remodeling. The edge morphology classification branch in this neural network can transform the aforementioned morphological patterns into discriminative features that the network can learn. During the training phase, the loss function of this edge morphology classification branch not only includes the cross-entropy between the predicted category and the manually labeled category, but also incorporates a penalty term for the voxel gradient features of the fracture edge, guiding the network to focus on changes in the sharpness of the fracture edge. The third branch is the healing stage regression branch, which outputs callus thickness and density values ​​for each voxel. This healing stage regression branch outputs scalar values ​​representing callus thickness and callus density (HU values ​​expressed as CT values).Bone callus thickness is the average value measured along the normal direction of the bone in the segmented callus region, and bone callus density is the average CT value of the voxels in the callus region. These two parameters are positively correlated with the time elapsed after injury; the callus gradually thickens and mineralizes as the healing process progresses, and the CT value increases accordingly. The loss function of the regression branch for this healing stage can use mean squared error loss or smoothed L1 loss to minimize the difference between the neural network prediction and the manually labeled true values. The multi-task three-dimensional convolutional neural network provided in this embodiment of the invention can use a CT dataset verified through forensic practice, labeled and measured, as the training set.

[0026] S208. Based on the registered and segmented data, deformation interpolation is performed between different time points to generate a four-dimensional reconstruction model that shows the dynamic changes in the morphology of the damaged area over time.

[0027] S2081. Perform 3D surface reconstruction on the segmented data at different time points to obtain the 3D mesh model for each time point. For the category label of each voxel output by the semantic segmentation branch, the moving cube algorithm is used to extract the triangular mesh surface of the segmentation target. The moving cube algorithm uses eight adjacent voxels in the volume data to form a cube unit. Based on the relative relationship between the gray value or label value at the eight vertices and the preset isosurface threshold, the topological configuration of the intersection of the cube unit and the isosurface is determined by looking up a table, and the precise coordinates of the intersection point are calculated by linear interpolation. After processing each unit, all intersection surfaces are connected to form a complete 3D mesh model of the internal organization. For the 3D point cloud data of the body surface, the Poisson surface reconstruction algorithm is used to generate a high-detail 3D mesh model of the skin. Poisson surface reconstruction transforms the surface reconstruction problem into solving the Poisson equation. Based on the input point cloud data and its normal vector, an implicit function indicating the internal and external space is constructed. The gradient field of this implicit function approximates the direction of the normal vector near the sampling point. Solving for the zero isosurface of this implicit function yields the reconstructed surface. The Poisson reconstruction algorithm has a certain tolerance for input noise and can automatically fill in small-scale missing holes in the point cloud to generate a watertight continuous surface.

[0028] S2082. Using the internal organization 3D mesh model at each time node as keyframes, a non-rigid registration algorithm is used to calculate the deformation field between adjacent keyframes. The internal organization 3D mesh models acquired and processed at different time nodes are used as keyframes, and a non-rigid registration algorithm is used to calculate the 3D deformation field between adjacent keyframes.

[0029] S2083. Based on the deformation field, a three-dimensional mesh model of the internal tissue at any intermediate moment between keyframes is generated, forming a continuous time series model. After obtaining the deformation field between adjacent keyframes, for any target moment located between the sampling times of two keyframes, the spatial position of each vertex at that intermediate moment is calculated by linear or geodesic interpolation along the streamlines of the deformation field, thereby generating a complete three-dimensional model of that target moment. By setting the interpolation frame rate (e.g., 10 frames per second), a dynamic model sequence that continuously evolves from the acute phase to the repair phase can be generated, i.e., the fourth dimension of information unfolding along time. This dynamic time series can intuitively show the complete healing process of the fracture gap gradually narrowing as the callus grows and the edema area gradually receding as the exudate is absorbed.

[0030] According to certain optional implementations, a stress field inversion step may also be included to infer the characteristics of the external force that caused the damage from the final morphology of the surface damage.

[0031] S2084. Construct an initial finite element model in an undamaged state. Based on the epidermal injury morphology in the three-dimensional scan data of the body surface, apply a virtual pressure field to this initial finite element model to ensure that the surface deformation of the model is consistent with the three-dimensional scan data of the body surface. The elastic moduli of the bones, soft tissues, and skin in this initial finite element model are based on the CT values ​​of the internal tomographic image data. Based on the bone, soft tissue, and skin meshes in the initial undeformed model, construct an initial finite element model containing the above three-layer structure in the finite element analysis software environment. If the subject has a healthy contralateral limb (e.g., a fracture of the right scapula, with the left side normal), generate a three-dimensional mesh model of the bones, soft tissues, and skin on the undamaged side through mirror transformation, as the initial undeformed model; if there is no contralateral healthy limb (e.g., damage to the spine or midline structure of the skull), select a template model matching age, gender, and body type from a pre-constructed standard digital human database, and adapt its anatomical features to the main skeletal contours of the subject through non-rigid registration to obtain the initial undeformed model. For each element in the initial finite element model, its corresponding voxel gray value (CT value) in the CT volume data is converted into the elastic modulus of the material using an empirical formula. Typically, there is an approximate exponential relationship between the elastic modulus of bone and the CT value; the elastic modulus of cortical bone can be taken in the range of 12 to 20 GPa, and that of cancellous bone in the range of 0.1 to 5 GPa. The elastic modulus of soft tissue is generally much smaller than that of bone, taking the level of hundreds of kPa; and that of skin in the level of tens of MPa. The above-mentioned three-dimensional mesh model of skin is used as the target deformation. An initial virtual pressure distribution is applied to the corresponding region of the skin layer in the initial finite element model, and finite element solution is performed to obtain the calculated deformation of the skin surface under the current pressure distribution. The calculated deformation is compared with the actual deformation in the three-dimensional scan data of the body surface. Based on the comparison results, the direction and magnitude of each point of the pressure field are adjusted, and the solution is re-executed. The above cycle of "applying pressure, calculating deformation, comparing deviation, and adjusting pressure" is iterated until the calculated deformation and the actual deformation are consistent within the preset tolerance range. At this point, the force field converges, and the applied pressure field distribution is considered as the approximate inverse solution of the external force causing the epidermal abrasion.

[0032] S2085. Iteratively solve for the direction and magnitude of the external force, map the obtained stress values ​​as a heat map, and overlay it onto the above four-dimensional reconstruction model. Map the final iteratively obtained surface contact stress values ​​as a heat map and overlay it onto the above four-dimensional reconstruction model.

[0033] S210. Under a unified spatiotemporal coordinate system, measure at least one quantitative injury indicator and match it with a pre-defined injury assessment standard knowledge base to output the injury severity assessment result. The quantitative injury indicators include fracture displacement distance, soft tissue injury volume, callus formation thickness, and fracture line length. These measurements are performed on a three-dimensional mesh model. The fracture displacement distance measurement uses a mirror image of the contralateral healthy bone as a reference template. For example, if the right tibia is fractured while the left tibia is intact, first, the CT data of the left healthy tibia is reconstructed and segmented in three dimensions. Then, the left model is flipped to the right spatial position through mirror transformation, making it correspond anatomically to the residual bone segment of the right fracture, serving as an anatomical alignment template. The shortest distance from each vertex of the right fracture remnant to the surface of the left mirror alignment template is calculated, and the maximum value or representative quantile is taken as the fracture displacement distance. When a usable contralateral healthy limb is unavailable, a matching reference bone model is retrieved from a pre-defined standard digital human database according to the gender, age, and height range of the person being assessed. This reference model is then scaled and registered in three dimensions and used as a substitute template. The measurement of soft tissue injury volume is based on a segmented edema region mask. The total number of voxels contained in this region is directly counted, and multiplied by the spatial volume of a single voxel (determined by the slice thickness and intra-slice resolution of the CT scan) to obtain the absolute volume of the edema region, in cubic millimeters. The measurement of callus formation thickness uses the segmented primary cortical surface as a reference. Rays are emitted along the external normal direction of the bone, and the straight-line distance from the primary cortical surface to the point where the ray intersects with the outer surface of the callus region is measured. After dense sampling near the fracture area, the maximum value or axial mean is taken as the callus formation thickness index. The measurement of fracture line length first involves three-dimensional skeletonization on the fracture line segmentation mask. By iteratively stripping boundary voxels, the connected centerline of the individual voxel width is finally retained. Then, the total spatial curve length of the fracture line is obtained by integrating along the voxel connection path of the skeleton curve. The above matching process may include: S2101. Map the qualitative description items in the injury assessment standards to numerical judgment rules. For example, the determination of the degree of "significant displacement of long bone fractures of the limbs" in the qualitative description item can be mapped to the numerical judgment rule "fracture displacement distance is greater than or equal to 3mm". Store the above numerical judgment standards in the preset injury assessment standard knowledge base.

[0034] S2102. Using damage quantification indicators as input, the numerical judgment rules and membership functions in the damage assessment standard knowledge base are invoked to perform fuzzy inference, outputting the probability distribution and confidence level of each damage severity level. During fuzzy inference, the input damage quantification indicators are converted into membership degrees of various fuzzy sets (e.g., small displacement, medium displacement, large displacement) through preset membership functions in the knowledge base; these membership degrees are matched with the judgment rules in the knowledge base, activating the corresponding rules and performing rule aggregation to obtain fuzzy output sets of each damage level (e.g., minor injury, slight injury, serious injury); using the centroid method or the maximum membership method, the fuzzy output sets are converted into clear probability distributions (probability values ​​for each damage level), and the confidence level (range 0-1) is calculated based on the rule activation strength and evidence consistency.

[0035] In summary, the embodiments of the present invention relate to a forensic injury three-dimensional reconstruction analysis method based on multimodal image fusion. The method includes: acquiring internal tomographic image data and body surface three-dimensional scan data of the subject; performing spatiotemporal unified registration on the internal tomographic image data and the body surface three-dimensional scan data to establish a unified spatiotemporal coordinate system; automatically segmenting the injury-related tissue area under the unified spatiotemporal coordinate system based on a preset machine learning model; performing deformation interpolation between different time points based on the registered and segmented data to generate a four-dimensional reconstruction model showing the dynamic changes of the morphology of the injury area over time; measuring at least one injury quantification index under the unified spatiotemporal coordinate system, matching the injury quantification index with a preset injury identification standard knowledge base, and outputting the injury degree assessment result. The technical solution provided by the embodiments of the present invention fundamentally transforms forensic injury identification from experience-driven to data-driven approaches. It expands the object of image analysis from two-dimensional slices with a single modality and a single time point to four-dimensional image sequences that span multiple modalities and time points. The task of image recognition is upgraded from simple tissue structure identification to automatic reasoning of deep semantic attributes such as injury healing stage and direction of external force. This enables the automatic extraction of spatiotemporal evolution patterns and mechanical response characteristics from images, allowing the temporal attributes, degree level, and injury mode of the injury to be quantified and repeatedly verified by image algorithms, thereby significantly improving the objectivity and interpretability of forensic identification.

[0036] It should be understood that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples; within the framework of the invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of the invention as described above, which are not provided in the details for the sake of brevity. The specific embodiments of the invention described above are merely illustrative or explanatory of the principles of the invention and do not constitute a limitation of the invention. Therefore, any modifications, equivalent substitutions, improvements, etc., made without departing from the spirit and scope of the invention should be included within the protection scope of the invention. Furthermore, the appended claims are intended to cover all variations and modifications falling within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A method for three-dimensional reconstruction and analysis of forensic injuries based on multimodal image fusion, characterized in that, The method includes: Acquire internal tomographic images and three-dimensional surface scan data of the subject of the examination, wherein the internal tomographic images include data from at least two different time points; The internal tomographic image data and the three-dimensional scan data of the body surface are spatiotemporally registered to establish a unified spatiotemporal coordinate system. Based on a pre-set machine learning model, the damage-related tissue regions are automatically segmented in the unified spatiotemporal coordinate system. Based on the registered and segmented data, deformation interpolation is performed between the different time points to generate a four-dimensional reconstruction model that shows the dynamic changes in the morphology of the damaged area over time. Under the unified spatiotemporal coordinate system, at least one damage quantification index is measured, and the damage quantification index is matched with a preset damage identification standard knowledge base to output the damage degree assessment result.

2. The method according to claim 1, characterized in that, Spatiotemporal registration of the internal tomographic image data and the three-dimensional surface scan data includes: The three-dimensional scan data of the body surface is coarsely aligned with the body surface contour reconstructed from the internal tomographic image data using principal component analysis. The iterative nearest point algorithm with normal constraints is used for iterative optimization to transform the coordinate system of the three-dimensional surface scan data to the coordinate system of the internal tomographic image data. The error function of the iterative nearest point algorithm with normal constraints introduces a penalty term for the angle between the point cloud normal vectors.

3. The method according to claim 2, characterized in that, The spatiotemporal registration of the internal tomographic image data and the three-dimensional surface scan data also includes: For internal tomographic image data at different time points, alignment is performed in the time dimension using data from one of the specified time points as a fixed reference frame.

4. The method according to any one of claims 1-3, characterized in that, The preset machine learning model includes a multi-task 3D convolutional neural network; the multi-task 3D convolutional neural network includes a semantic segmentation branch, an edge morphology classification branch, and a healing stage regression branch.

5. The method according to claim 4, characterized in that, The semantic segmentation branch is used to output the segmentation results of bones, soft tissues, fracture lines, callus, and edema areas; The edge morphology classification branch is used to output the probability that each voxel belongs to a fresh fracture edge or an old fracture edge; The healing phase regression branch is used to output callus thickness and density values ​​for each voxel.

6. The method according to claim 5, characterized in that, Generate a four-dimensional reconstruction model that shows the dynamic changes in the morphology of the damaged area over time, including: The segmented data at different time points are reconstructed into three-dimensional surfaces to obtain three-dimensional mesh models for each time point. Using the three-dimensional mesh model of each time node as keyframes, a non-rigid registration algorithm is used to calculate the deformation field between adjacent keyframes; Based on the deformation field, a three-dimensional model is generated at any intermediate moment between the key frames to form a continuous time series model.

7. The method according to claim 6, characterized in that, The method also includes stress field inversion: An initial finite element model in an undamaged state is constructed. Based on the epidermal damage morphology in the three-dimensional scan data of the body surface, a virtual pressure field is applied to the initial finite element model so that the deformation of the model surface is consistent with the three-dimensional scan data of the body surface. The direction and magnitude of the external force are iteratively solved, and the obtained stress values ​​are mapped into a heat map and superimposed on the four-dimensional reconstruction model. The elastic modulus of bones, soft tissues, and skin in the initial finite element model is based on the CT values ​​of the internal tomographic images.

8. The method according to claim 7, characterized in that, The at least one quantitative indicator of injury includes fracture displacement distance, soft tissue injury volume, callus formation thickness, and fracture line length.

9. The method according to claim 8, characterized in that, Matching the damage quantification indicators with a preset damage identification standard knowledge base includes: Map the qualitative description items in the damage assessment criteria to numerical judgment rules; Using damage quantification indicators as input, the system calls the numerical judgment rules and membership functions in the damage assessment standard knowledge base, performs fuzzy inference, and outputs the probability distribution and confidence level of each damage severity level.

10. The method according to claim 9, characterized in that, The internal tomographic image data includes CT data, and the three-dimensional surface scan data includes point cloud data acquired by a light three-dimensional scanner.

Images