A multi-modal image fusion-based intelligent bone defect boundary identification method

Abstract

The present application relates to the technical field of bone defect boundary recognition, and particularly relates to a kind of intelligent recognition method of bone defect boundary based on multi-modal image fusion.The present application obtains CT abnormal anchor point and MRI abnormal anchor point of constructing multiple sets of matching anchor point pairs according to the gray level features of CT image and MRI registration image, and obtains the image change consistency of each matching anchor point pair;According to the gradient features of each voxel in CT image and MRI registration image, obtain the fusion weight coefficient of each voxel in each image;According to the image change consistency of multiple sets of matching anchor point pairs in the local range of each voxel, the fusion weight coefficient in CT image and MRI registration image and the gray level features, obtain the fusion feature value of each voxel in CT image and MRI registration image, and the boundary of bone defect region is recognized.The present application improves the accuracy of bone defect boundary recognition by accurate fusion analysis after registration image.

Description

Technical Field

[0001] This invention relates to the field of bone defect boundary recognition technology, specifically to an intelligent bone defect boundary recognition method based on multimodal image fusion. Background Technology

[0002] Bone defects are structural losses of bone caused by common clinical reasons such as trauma, tumor resection, debridement of osteomyelitis, revision of artificial joints, or correction of congenital malformations. In orthopedics, oral and maxillofacial surgery, and neurosurgery, the precise repair of bone defects has always been a core challenge in clinical practice. The key to the success of repair surgery lies in the ability to accurately, objectively, and repeatedly identify and quantify the boundaries of the bone defect before surgery.

[0003] In existing technologies, global rigid alignment and static parameter fusion are performed on voxel features of CT and MRI. However, since CT and MRI are usually acquired at different times and on different devices, there are often differences in the patient's multi-dimensional positional parameters, resulting in inconsistent initial spatial positions of the two modal images. Consequently, the accuracy of image registration is reduced due to insufficient calibration of initial positional differences. Furthermore, different images have different sensitivities to different tissue components, and existing fusion methods cannot dynamically adjust the fusion weights according to local tissue characteristics, resulting in blurred or lost information boundaries of bone defects in the fused images, which affects the accuracy of intelligent identification of bone defect boundaries. Summary of the Invention

[0004] To address the technical problem that existing fusion methods struggle to dynamically adjust fusion weights based on local tissue characteristics, leading to blurred or lost information at bone defect boundaries in fused images and affecting the accuracy of intelligent bone defect boundary recognition, this invention aims to provide an intelligent bone defect boundary recognition method based on multimodal image fusion. The specific technical solution adopted is as follows: This invention proposes an intelligent identification method for bone defect boundaries based on multimodal image fusion, the method comprising: CT and MRI images of the target area are acquired, and multi-dimensional positional parameters of the corresponding images are obtained. The target area includes bone defect areas and normal bone tissue. Based on the differences in multidimensional positional parameters between CT and MRI images, MRI registration images with CT images as the reference are obtained; based on the grayscale characteristics of CT and MRI registration images, CT abnormal anchor points and MRI abnormal anchor points are obtained, and multiple sets of matching anchor point pairs are obtained; based on the grayscale characteristics of different voxels within the local range of the corresponding matching anchor points in different images, the consistency of image changes of each set of matching anchor point pairs is obtained. Based on the gradient characteristics of each voxel in the CT and MRI registered images, the fusion weight coefficient of each voxel in each image is obtained; based on the consistency of image changes of multiple sets of matching anchor point pairs in the local area of ​​each voxel, the corresponding fusion weight coefficients in the CT and MRI registered images, and the grayscale characteristics, the fusion feature value of each voxel in the CT and MRI registered images is obtained. Based on the distribution of fusion feature values ​​of different voxels in CT and MRI registered images, the boundaries of bone defect areas are identified.

[0005] Furthermore, the method for acquiring the MRI registration images includes: The cumulative difference values ​​of the same dimension positional parameters between CT and MRI images are obtained and normalized to represent the degree of positional shift. Using CT images as a reference, spatial transformation of MRI images is performed based on the degree of body positional shift to obtain MRI registration images.

[0006] Furthermore, the step of using CT images as a reference and spatially transforming MRI images based on the degree of body positional shift to obtain MRI registration images includes: Using CT images as a reference, if the degree of body position deviation is greater than a preset deviation threshold, based on the difference between the corresponding body position parameters of the same dimension between CT images and MRI images, after performing affine transformation on the MRI images, a feature matching algorithm is used to obtain MRI registration images. If the degree of body positional deviation is less than or equal to the preset deviation threshold, the feature matching algorithm is used to obtain the MRI registration image.

[0007] Furthermore, the method for obtaining the CT abnormal anchor points and MRI abnormal anchor points includes: Based on the grayscale characteristics of CT images and MRI registered images, the bone tissue region of CT images and the abnormal signal region of MRI registered images are obtained. For the bone tissue region, the curvature of each edge voxel is obtained, and the difference in curvature between each edge voxel and the previous edge voxel is multiplied by the relative distance between the corresponding edge voxels as the curvature change rate; if the curvature change rate is greater than a preset change threshold, the corresponding edge voxel is used as a CT abnormality anchor point. For abnormal signal regions, the gradient intensity of each voxel is obtained. If the gradient intensity of a voxel is greater than that of other voxels in the spatial range, the corresponding voxel is taken as the MRI abnormal anchor point. The extreme values ​​of gray values ​​among all voxels are obtained, and the voxels corresponding to the extreme values ​​are taken as the MRI abnormal anchor points.

[0008] Furthermore, the method for obtaining the bone tissue region of the CT image and the abnormal signal region of the MRI registration image includes: The gray-level histograms of voxels in CT images are obtained, and an adaptive threshold segmentation algorithm is used to obtain the segmentation threshold of the gray-level histograms. The range of voxels with gray levels greater than the segmentation threshold is taken as the bone tissue region. In T2-weighted MRI images, voxels with gray values ​​higher than a preset gray value threshold are used as seed points. If the gray values ​​of adjacent voxels are within a preset gray value similar range, the corresponding adjacent voxels are used as growth seed points. The range formed by the seed point and all its growth seed points is obtained as the abnormal signal region.

[0009] Furthermore, the method for obtaining the multiple sets of matching anchor point pairs includes: In CT images, the relative distance between each CT abnormal anchor point and different MRI abnormal anchor points is obtained. If the MRI abnormal anchor point corresponding to the minimum relative distance is within the neighborhood of the CT abnormal anchor point, the CT abnormal anchor point and the MRI abnormal anchor point are formed into a set of matching anchor point pairs.

[0010] Furthermore, the method for obtaining image change consistency includes: For any image in CT or MRI registration, obtain the difference between the mean gray value of all voxels within the local area corresponding to the matching anchor point in the image and the gray reference value of normal bone tissue. Based on the deviation characteristics between the difference results and the grayscale fluctuation characteristics of normal bone tissue, the degree of bone tissue difference at the corresponding matching anchor point in each image is obtained. The deviation characteristics are positively correlated with the degree of bone tissue difference. The differences in the degree of bone tissue difference between each pair of matched anchor points in CT and MRI images are obtained. Based on the sum of the degree of bone tissue difference, the differences in the degree of bone tissue difference are negatively correlated and normalized to serve as the consistency of image changes for each pair of matched anchor points.

[0011] Furthermore, the method for obtaining the fusion weight coefficients includes: Obtain the gradient intensity of each voxel in the CT and MRI registered images; The sum of the gradient intensities of each voxel in all images is obtained as the overall gradient intensity of each voxel. For any image, the ratio of the gradient intensity of each voxel in the image to the overall gradient intensity is obtained, and used as the fusion weight coefficient of each voxel in the corresponding image.

[0012] Furthermore, the method for obtaining the fused feature values ​​includes: The mean value of the image change consistency of all groups of matched anchor point pairs within the local range of each voxel is obtained as the local change consistency of each voxel. The fusion feature value of each voxel is obtained based on the fusion weight coefficient, gray value, and local variation consistency of each voxel in all images.

[0013] Furthermore, the method for obtaining the fused feature values ​​includes: For any image, the gray values ​​of different voxels in the image are normalized to obtain the product between the fusion weight coefficient of each voxel in each image and the normalized gray value, which is used as the weighted gray value. The product of the weighted grayscale sum and the consistency of local changes for each voxel in all images is calculated and used as the fusion feature value for each voxel. This invention also proposes an intelligent bone defect boundary recognition system based on multimodal image fusion. The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements any of the steps of the intelligent bone defect boundary recognition method based on multimodal image fusion described above.

[0014] The present invention has the following beneficial effects: This invention obtains MRI registration images based on CT images by leveraging the differences in multi-dimensional positional parameters between CT and MRI images. This overcomes the global mismatch problem caused by inconsistent patient scanning positions, facilitating precise delineation of the comprehensive boundaries of bone defects. Based on the grayscale characteristics of the CT and MRI registration images, it obtains CT and MRI abnormal anchor points and multiple sets of matching anchor point pairs, greatly improving the targeting and accuracy of local area alignment. Furthermore, based on the grayscale characteristics of different voxels within the local area corresponding to the matching anchor points in different images, it obtains the consistency of image changes for each set of matching anchor point pairs, reflecting the similarity of local signal changes between images. Based on the gradient characteristics of each voxel in the CT and MRI registered images, the fusion weight coefficient of each voxel in each image is obtained. The gradient intensity of the voxel is used to evaluate the edge sharpness of the point in different modalities, and the fusion weight is dynamically and adaptively allocated. Based on the consistency of image changes of multiple sets of matching anchor point pairs within the local area of ​​each voxel, the corresponding fusion weight coefficients in the CT and MRI registered images, and grayscale characteristics, the fusion feature value of each voxel in the CT and MRI registered images is obtained, realizing the complementary fusion of multimodal information. Based on the distribution of fusion feature values ​​of different voxels in the CT and MRI registered images, the boundary of the bone defect area is identified. This invention improves the accuracy of bone defect boundary identification by performing accurate fusion analysis after registration of images. Attached Figure Description

[0015] Figure 1 A flowchart illustrating an intelligent identification method for bone defect boundaries based on multimodal image fusion, provided in one embodiment of the present invention; Figure 2 This is a flowchart illustrating a method for obtaining CT and MRI abnormal anchor points according to an embodiment of the present invention. Detailed Implementation

[0016] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0017] The following description, in conjunction with the accompanying drawings, details a specific scheme for an intelligent identification method for bone defect boundaries based on multimodal image fusion provided by this invention.

[0018] Please see Figure 1 The diagram illustrates a flowchart of an intelligent bone defect boundary recognition method based on multimodal image fusion according to an embodiment of the present invention. The specific method includes: Step S1: Obtain CT and MRI images of the target area and acquire multi-dimensional positional parameters of the corresponding images. The target area includes bone defect areas and normal bone tissue.

[0019] In the embodiments of the present invention, considering that CT and MRI are acquired at different times and on different devices, the patient's positional parameters often differ, resulting in inconsistent initial spatial positions of the two modal images. This leads to a decrease in the accuracy of image registration due to insufficient calibration caused by the initial positional differences. Therefore, it is necessary to analyze the multi-dimensional positional parameters of the images during image fusion. First, CT images are obtained based on computed tomography to accurately characterize the hard anatomical boundaries of bone tissue, and MRI images are obtained based on magnetic resonance imaging to clearly display the lesion activity boundaries of soft tissue and bone marrow.

[0020] It should be noted that, in the embodiments of the present invention, in order to avoid interference from noise in the basic image, the acquired image is preprocessed, including motion artifact weakening based on the difference between neighboring pixels and image brightness correction based on histogram equalization; the specific means are well known to those skilled in the art and will not be described in detail here.

[0021] Accurate scanning of each imaging site was performed in accordance with clinical imaging examination standards. At the same time, multi-dimensional positional parameters of the patient during the imaging were recorded in a structured manner, including limb placement angle, trunk / limb offset, scanning projection orientation, joint flexion, scanning slice thickness and interslice spacing, etc., and the data were bound in real time with the corresponding modal image data to provide basic data support for subsequent elimination of registration errors caused by positional differences.

[0022] It should be noted that, in the embodiments of the present invention, after obtaining the multi-dimensional body position parameters, the pre-set extreme value range of each part of the human body in the clinical equipment scanning specifications is used as the upper and lower limits. Range normalization is used to map the original body position parameters to the dimensionless 0 to 1 interval to eliminate the influence of different dimensions. The specific means are technical means well known to those skilled in the art and will not be described in detail here.

[0023] Step S2: Based on the differences in multi-dimensional positional parameters between CT and MRI images, obtain MRI registration images with CT images as the reference; based on the grayscale characteristics of CT and MRI registration images, obtain CT abnormal anchor points and MRI abnormal anchor points, and obtain multiple sets of matching anchor point pairs; based on the grayscale characteristics of different voxels in the neighborhood of the corresponding matching anchor points in different images, obtain the consistency of image changes for each set of matching anchor point pairs.

[0024] Because CT and MRI images are acquired at different times and on different devices, there are often differences in the patient's multi-dimensional positional parameters, resulting in inconsistent initial spatial positions of the two modalities. Consequently, the accuracy of image registration is reduced due to insufficient calibration of the initial positional differences. The greater the degree of positional deviation, the greater the bone contour registration deviation, and the greater the possibility of mismatch in local areas. Therefore, it is more necessary to perform registration matching based on multi-dimensional positional parameters. Based on the differences in multi-dimensional positional parameters between CT and MRI images, MRI registration images with CT images as the reference are obtained.

[0025] Preferably, in one embodiment of the present invention, the method for acquiring MRI registration images includes: The first step is to obtain the cumulative difference values ​​of the same dimension of body position parameters between CT images and MRI images, and then normalize them as the degree of body position deviation. It should be noted that the difference represents the absolute value of the difference; in the embodiments of the present invention, the normalization acquisition method includes: based on the maximum value of the cumulative difference of body position parameters between images obtained in advance from a large amount of historical experimental data, if the calculated cumulative difference value is greater than the maximum value, the body position offset degree is set to 1; if the calculated cumulative difference value is less than or equal to the maximum value, the ratio of the cumulative difference value to the maximum value is calculated as the body position offset degree, that is, normalized to the range of 0-1; the specific means are well known in the art and are not limited or described here.

[0026] The second step involves using CT images as a reference and spatially transforming MRI images based on the degree of body positional shift to obtain MRI registration images.

[0027] It should be noted that, in one embodiment of the present invention, using CT images as a reference, spatial transformation is performed on MRI images based on the degree of body positional shift to obtain MRI registration images, including: The first step is to use CT images as a reference. If the positional deviation is greater than a preset deviation threshold, the MRI images are affinely transformed based on the difference between the corresponding positional parameters of the same dimension between the CT images and the MRI images, and then the MRI registration images are obtained by using a feature matching algorithm. It should be noted that, considering the different imaging principles of CT and MRI, and the lack of a direct correspondence in grayscale distribution, bone contour is the only stable and consistent rigid anatomical feature across modalities. Therefore, bone contour extraction needs to be performed on CT and MRI images before registration. In the embodiments of this invention, bone contour edges in CT images are extracted using the Canny edge detection operator; grayscale thresholding and edge enhancement processing are performed on MRI images to extract bone contour edges; the specific methods are well known to those skilled in the art and will not be elaborated here.

[0028] In an embodiment of the present invention, the affine transformation process is as follows: Based on the differences between corresponding positional parameters of the same dimension between CT and MRI images, the angular difference components are extracted as rotation parameters around coordinate axes. For example, the difference in limb placement angles is mapped to a rotation angle around the Z-axis, the difference in scanning projection azimuth tilt angles is mapped to a rotation angle around the Y-axis, and the difference in joint flexion is mapped to a rotation angle around the X-axis. These are all converted into sine and cosine values ​​and filled into the corresponding rotation positions of the affine matrix to form a rotation matrix. The displacement difference components are extracted as translation parameters, such as the torso... The trunk / limb offset is mapped to a translation matrix: the three-dimensional distance difference of the offset is directly extracted as a translation vector; the scan slice thickness and interslice spacing are mapped to a scaling matrix, and the ratio of the sum of the MRI slice thickness and interslice spacing to the sum of the CT slice thickness and interslice spacing is calculated as the Z-axis scaling factor; the translation matrix, rotation matrix and scaling matrix are sequentially multiplied and concatenated to obtain the affine transformation matrix used for pre-correction, and spatial geometric transformation and resampling interpolation are performed on the MRI bone contour; the specific methods are well known to those skilled in the art and will not be elaborated here.

[0029] The second step is to obtain MRI registration images if the degree of body position deviation is less than or equal to the preset deviation threshold, using a feature matching algorithm.

[0030] It should be noted that, in one embodiment of the present invention, the greater the degree of body position offset, the greater the difference in body position between images, and the greater the body position difference. In order to avoid the risk of mismatch due to excessive body position offset, the offset threshold is preset to 0.25 based on relevant historical experience. The feature matching algorithm adopts the SIFT-based feature matching algorithm. In other embodiments of the present invention, the preset offset threshold can be set according to specific circumstances, and is not limited or described here.

[0031] On CT images, the cortex of normal bone tissue appears as a continuous, dense, high-density image, appearing bright white. The trabeculae appear as a network of medium-high density, and the medullary cavity is low-density soft tissue. Bone defect areas appear as discontinuous cortical bone, presenting as distinct low-density dark areas. On MRI images, normal bone marrow fat appears as a medium-to-high signal on T2-weighted images. Inflammation, edema, tumors, and infections, due to their higher free water content, appear as bright white signals on T2-weighted images, allowing for the identification of anomalous anchor points with distinct characteristics. Based on the grayscale characteristics of the CT and MRI registered images, CT anomalous anchor points and MRI anomalous anchor points are obtained, and multiple sets of matching anchor point pairs are acquired.

[0032] Preferably, in one embodiment of the present invention, the method for obtaining CT abnormal anchor points and MRI abnormal anchor points is described in [reference needed]. Figure 2 It illustrates a flowchart of a method for obtaining CT and MRI abnormal anchor points, including: Step S201: Based on the grayscale characteristics of the CT images and the MRI registered images, obtain the bone tissue region of the CT images and the abnormal signal region of the MRI registered images.

[0033] It should be noted that, in one embodiment of the present invention, the gray-level histogram of voxels in CT images is obtained, and an adaptive threshold segmentation algorithm is used to obtain the segmentation threshold of the gray-level histogram; the range formed by voxels with gray values ​​greater than the segmentation threshold is taken as the bone tissue region, and bone fragments containing normal cortical bone, trabeculae, and the edge of bone defects are extracted; wherein, the adaptive threshold segmentation algorithm can be the Otsu method; in the T2-weighted image of MRI, voxels with gray values ​​higher than a preset gray value threshold are taken as seed points, and if the gray values ​​of adjacent voxels are within a preset gray value similar range, the corresponding adjacent voxels are taken as growth seed points; the range formed by the seed point and all its growth seed points is obtained as the abnormal signal region; the specific means are well known to those skilled in the art and will not be described in detail here.

[0034] It should be noted that, in one embodiment of the present invention, since T2-weighted images have high signal characteristics, the larger the gray value, the more it corresponds to a bright edema area. The method for obtaining the preset gray value threshold is as follows: sort the gray values ​​of all voxels from smallest to largest, and obtain the voxel located at the 95th percentile of the gray value as the preset gray value threshold. In order to analyze high signals with similar surrounding conditions together, the method for obtaining the preset gray similarity range is as follows: obtain the mean gray value of all seed points, obtain the standard deviation of the gray values ​​of all seed points; obtain the difference between the mean gray value and the standard deviation of the gray value of the preset multiple, obtain the sum of the mean gray value and the standard deviation of the gray value of the preset multiple, and take the range formed between the difference and the sum as the preset gray similarity range. The preset multiple is set to 2, that is, it covers most similar voxels for analysis. The specific means are well known to those skilled in the art and will not be described in detail here.

[0035] Step S202: For the bone tissue region, obtain the curvature of each edge voxel, and obtain the ratio of the difference in curvature between each edge voxel and its adjacent edge voxels to the relative distance between the corresponding edge voxels as the curvature change rate; if the curvature change rate is greater than the preset change threshold, the corresponding edge voxel is used as the CT abnormality anchor point.

[0036] It should be noted that the difference represents the absolute value of the calculated difference. In one embodiment of the present invention, for the bone tissue region, the first-order partial derivatives and second-order partial derivatives of the edge voxels in different coordinate dimensions, as well as the second-order cross partial derivatives in different spatial planes, are calculated respectively. The first-order partial derivatives, second-order partial derivatives, and second-order cross partial derivatives are substituted into the three-dimensional isosurface mean curvature formula, and the obtained value is used as the curvature of the edge voxels.

[0037] It should be noted that in some embodiments of the present invention, the relative distance is obtained by Euclidean distance or Manhattan distance calculation methods. The greater the curvature difference between adjacent edge voxels, the smaller the relative distance. That is, there is a large rate of curvature change in short distances, and feature analysis is more necessary. The greater the rate of curvature change, the less likely the edge is to be continuous. In order to select edge voxels with more credible interruptions, the preset change threshold is set to 0.5 based on relevant historical experience. In other embodiments of the present invention, the preset change threshold can be set according to specific circumstances, which will not be limited or elaborated here.

[0038] Step S203: For the abnormal signal region, obtain the gradient intensity of each voxel. If the gradient intensity of a voxel is greater than the gradient intensity of other voxels in the spatial range, the corresponding voxel is taken as the MRI abnormal anchor point. Obtain the extreme values ​​of gray values ​​among all voxels and take the voxel corresponding to the extreme value as the MRI abnormal anchor point.

[0039] It should be noted that in some embodiments of the present invention, the three-dimensional gradient intensity is calculated using the Sobel operator or the Scharr operator. The gradient intensity reflects the drastic change in the gray value of the image. The greater the gradient intensity, the more likely it is to be at the boundary. Extreme values ​​are extracted by analyzing the gray value distribution of voxels. That is, if the gray value of a voxel is greater than the gray values ​​of its left and right adjacent voxels, the gray value of the voxel is taken as the maximum value; if the gray value of a voxel is less than the gray values ​​of its left and right adjacent voxels, the gray value of the voxel is taken as the minimum value. The gray value extreme points help to reflect the core features inside the lesion, that is, the brightest center of edema or the darkest point of necrosis. The MRI abnormal anchor point simultaneously anchors the outer contour and internal structure of the defect area, improving the accuracy of matching with the CT abnormal anchor point. The specific means are well known to those skilled in the art and will not be described in detail here.

[0040] It should be noted that, in one embodiment of the present invention, the size of the spatial range is a range of 3×3×3 formed by selecting each voxel as a reference and adjacent voxels; in other embodiments of the present invention, the size of the spatial range can be set according to specific circumstances, and will not be limited or described in detail here.

[0041] Preferably, in one embodiment of the present invention, the method for obtaining multiple sets of matching anchor point pairs includes: In CT images, the relative distance between each CT abnormal anchor point and different MRI abnormal anchor points is obtained. If the MRI abnormal anchor point corresponding to the minimum relative distance is within the neighborhood of the CT abnormal anchor point, the CT abnormal anchor point and the MRI abnormal anchor point are formed into a set of matching anchor point pairs.

[0042] Based on this, the matching anchor point pair represents a set of spatially corresponding points determined in CT and MRI registered images for the same normal bone tissue anatomical location based on the image registration algorithm.

[0043] It should be noted that, in some embodiments of the present invention, the relative distance can be obtained by existing distance calculation methods such as Euclidean distance or Manhattan distance. The specific means are well known to those skilled in the art and will not be described in detail here. In one embodiment of the present invention, the neighborhood range of the CT abnormal anchor point is a range with a radius of 5 mm centered on the CT abnormal anchor point. In other embodiments of the present invention, the size of the neighborhood range can be set according to specific circumstances and will not be limited or described in detail here.

[0044] Since abnormal areas on CT and MRI should be spatially similar, the higher the similarity of image changes, the more consistent the image changes of each pair of matching anchor points are. Based on the grayscale characteristics of different voxels within the local range of the corresponding matching anchor points in different images, the consistency of image changes of each pair of matching anchor points is obtained.

[0045] Preferably, in one embodiment of the present invention, the method for obtaining image change consistency includes: The first step is to obtain the difference between the mean gray value of all voxels within the local area of ​​the corresponding matching anchor point in the image and the gray reference value of normal bone tissue for any image in CT and MRI registration images. Based on the deviation characteristics between the difference results and the gray fluctuation characteristics of normal bone tissue, the degree of bone tissue difference at the corresponding matching anchor point in each image is obtained. The deviation characteristics are positively correlated with the degree of bone tissue difference. It should be noted that, in one embodiment of the present invention, the size of the local area is an area of ​​11×11×11 formed by the matching anchor point and adjacent voxels; the grayscale reference value of normal bone tissue is the average grayscale value of all voxels in normal bone tissue; in other embodiments of the present invention, the size of the local area can be set according to specific circumstances, and will not be limited or described in detail here.

[0046] It should be noted that the difference represents the absolute value of the difference. The larger the difference, the greater the difference between the mean gray value of all voxels and the gray value reference value of normal bone tissue. The smaller the gray value fluctuation characteristics of normal bone tissue, the more dispersed the gray value distribution, the greater the deviation characteristics, and the greater the degree of bone tissue difference. The smaller the difference, the closer the mean gray value of all voxels is to the gray value reference value of normal bone tissue. The smaller the gray value fluctuation characteristics of normal bone tissue, the more uniform the gray value distribution, the smaller the deviation characteristics, and the smaller the degree of bone tissue difference. Therefore, the deviation characteristics are positively correlated with the degree of bone tissue difference.

[0047] It should be noted that, in one embodiment of the present invention, the gray-scale fluctuation characteristics are characterized by calculating the standard deviation. The larger the standard deviation, the larger the gray-scale fluctuation characteristics. The gray-scale fluctuation of normal bone tissue is relatively large, and the occurrence of gray-scale deviation is a normal physiological phenomenon. The smaller the degree of difference in bone tissue, the smaller the gray-scale fluctuation characteristics. The smaller the gray-scale fluctuation of normal bone tissue, the more obvious the degree of difference in bone tissue. In other embodiments of the present invention, the gray-scale fluctuation characteristics can also be characterized by calculating the variance. The specific means are well known to those skilled in the art and will not be described in detail here.

[0048] In one embodiment of the present invention, the method for obtaining the degree of bone tissue difference is as follows: considering that the gray-level fluctuation feature may be 0, a very small positive number with consistent dimensions is added to the denominator, the value of which can be specifically set according to the range of values ​​of the denominator; the difference result is calculated by dividing it by the sum of the gray-level fluctuation feature and the very small positive number of normal bone tissue, reflecting the multiple of the gray-level offset relative to the normal gray-level fluctuation, as the degree of bone tissue difference of the corresponding matching anchor point in each image; the formula is expressed as: ;in, Indicates the first in the image The degree of bone tissue difference at each matching anchor point; Indicates the first in the image The average gray value within a local range of each matching anchor point; This represents the grayscale reference value for normal bone tissue in the image; This indicates the grayscale fluctuation characteristics of normal bone tissue in an image; It represents a very small positive number.

[0049] The second step is to obtain the difference in the degree of bone tissue difference between each pair of matched anchor points in the CT images and the MRI registered images; based on the sum of the degree of bone tissue difference, the difference in the degree of bone tissue difference is negatively correlated and normalized to serve as the consistency of image changes for each pair of matched anchor points.

[0050] It should be noted that the difference represents the absolute value of the difference. The greater the difference in the degree of bone tissue difference, the more inconsistent the changes between images. The difference in the degree of bone tissue difference is negatively correlated with the consistency of image changes. In the embodiments of the present invention, if the degree of bone tissue difference is 0, the sum of the degree of bone tissue difference is 0, there is no difference feature, and the consistency of image changes is set to 1. If there is a non-zero value for the degree of bone tissue difference, the ratio of the difference in the degree of bone tissue difference to the sum of the degree of bone tissue difference is calculated, and the difference between the positive integer 1 and the ratio is calculated. That is, negative correlation normalization mapping is performed so that the value of the consistency of image changes is within the range of 0-1.

[0051] Step S3: Based on the gradient characteristics of each voxel in the CT image and the MRI registration image, obtain the fusion weight coefficient of each voxel in each image; based on the consistency of image changes of multiple sets of matching anchor point pairs in the local area of ​​each voxel, the corresponding fusion weight coefficient in the CT image and the MRI registration image, and the grayscale characteristics, obtain the fusion feature value of each voxel in the CT image and the MRI registration image.

[0052] Because the internal tissue composition of bone defect areas is complex, CT and MRI have different sensitivities to each tissue component. CT images are sensitive to calcification and ossification structures, and can clearly show the interruption of bone cortex and the distribution of bone fragments. MRI images are sensitive to changes in soft tissue, edema, hemorrhage and bone marrow signals, and can accurately define the boundary between the defect area and soft tissue. Therefore, gradient features help to reflect the intensity of characteristic changes of voxels and assign reasonable fusion weight coefficients. Based on the gradient features of each voxel in the CT and MRI registered images, the fusion weight coefficient of each voxel in each image is obtained.

[0053] Preferably, in one embodiment of the present invention, the method for obtaining the fusion weight coefficients includes: Obtain the gradient intensity of each voxel in the CT and MRI registered images; The sum of the gradient intensities of each voxel in all images is obtained as the overall gradient intensity of each voxel. For any image, the ratio of the gradient intensity of each voxel in the image to the overall gradient intensity is obtained, and used as the fusion weight coefficient of each voxel in the corresponding image.

[0054] It should be noted that in some embodiments of the present invention, the gradient intensity is obtained by calculating the three-dimensional gradient magnitude using the Sobel operator or the Scharr operator; to ensure that the fused value is always a stable linear combination of the two images, the sum of the fusion weight coefficients corresponding to the two images is 1; considering that when the overall gradient intensity may be 0, the voxel variation between images is less drastic and lacks significant boundary features, in order to preserve the basic background information of normal tissue and avoid information loss, the fusion weight coefficient is set to an equal value of 0.5; when the overall gradient intensity is not 0, the ratio of the gradient intensity of each voxel to the overall gradient intensity is calculated as the fusion weight coefficient; the specific means are well known to those skilled in the art and will not be described in detail here.

[0055] Based on this, the greater the gradient intensity, the more obvious the edge features, and the higher the fusion weight is dynamically allocated to ensure that the final fused image retains both the sharpness of the bone fracture ends in CT and the clarity of the soft tissue boundaries in MRI.

[0056] The fusion weight coefficient reflects the sensitivity of the image to tissue components. The larger the fusion weight coefficient, the greater the sensitivity and the larger the proportion that needs to be fused. The gray value reflects the basic anatomical information of the voxel. The two are combined by weighting through fusion weight to retain the modal features with the clearest local edges, which constitutes the basic value of the fusion feature value. The greater the consistency of local changes, the more abnormal the deviation between images, the more the basic gray value of the region is amplified, and the larger the fusion feature value is. Therefore, based on the consistency of image changes of multiple sets of matching anchor point pairs in the local range of each voxel, the corresponding fusion weight coefficient in CT images and MRI registration images, and gray value features, the fusion feature value of each voxel in CT images and MRI registration images is obtained.

[0057] Preferably, in one embodiment of the present invention, the method for obtaining fused feature values ​​includes: The first step is to obtain the mean value of the image change consistency of all groups of matched anchor point pairs within the local range of each voxel, which is used as the local change consistency of each voxel. It should be noted that, in one embodiment of the present invention, the size of the local area is an area of ​​11×11×11 formed by the matching anchor point and adjacent voxels; the grayscale reference value is the average grayscale value of all voxels in normal bone tissue; in other embodiments of the present invention, the size of the local area can be set according to specific circumstances, and will not be limited or described in detail here.

[0058] The second step is to obtain the fusion feature value of each voxel based on the fusion weight coefficient, gray value, and local change consistency of each voxel in all images. The fusion weight coefficient, gray value, and local change consistency are all positively correlated with the fusion feature value.

[0059] It should be noted that the larger the fusion weight coefficient, the greater the sensitivity to tissue components, the larger the proportion that needs to be fused, and the larger the fusion feature value. Gray values ​​reflect the basic anatomical information of voxels. The two are weighted and combined through fusion weight to retain the modal features with the clearest local edges, which constitute the basic value of the fusion feature value. The greater the consistency of local changes, the more abnormal the deviations between images are, the more the basic gray value of the region is amplified, and the larger the fusion feature value is. Therefore, the fusion weight coefficient, gray value, and consistency of local changes are all positively correlated with the fusion feature value.

[0060] In one embodiment of the present invention, for any image, the maximum and minimum gray values ​​of voxels in the image are selected, and the gray values ​​of different voxels in the image are normalized to the range of 0-1; the product between the fusion weight coefficient and the normalized gray value of each voxel in each image is obtained as the weighted gray value; the product between the weighted gray value of each voxel in all images and the consistency of local changes is calculated as the fusion feature value of each voxel; the formula for the fusion feature value is expressed as: ;in, Indicates the first Fusion eigenvalues ​​of individual elements; Indicates the first The mean of image variation consistency among all matched anchor point pairs within a local range of an individual element, i.e., local variation consistency; Indicates the first The fusion weighting coefficient of individual pixels in CT images; Indicates the first Fusion weighting coefficients of individual pixels in MRI images; Indicates the first The normalized grayscale value of a voxel in a CT image; Indicates the first The normalized grayscale value of an individual pixel in a registered MRI image.

[0061] Step S4: Based on the distribution of fusion feature values ​​of different voxels in CT images and MRI registered images, the boundary of the bone defect area is identified.

[0062] Fusion features combine features from multiple images, which helps to more accurately represent the features of defective areas in an image.

[0063] It should be noted that, in the embodiments of the present invention, an initial three-dimensional fused image is constructed based on the fusion feature values ​​of all voxels; based on the initial three-dimensional fused image, an adaptive segmentation threshold for the distribution of fusion feature values ​​is obtained using the Otsu method, and the range of voxels whose fusion feature values ​​are greater than the segmentation threshold is taken as the bone defect area; the three-dimensional Canny edge detection operator is used to extract the edge of the bone defect area at full scale to obtain the edge point set and identify the bone defect boundary; the specific means are well known to those skilled in the art and will not be described in detail here.

[0064] In summary, this invention obtains CT and MRI abnormal anchor points for constructing multiple sets of matching anchor point pairs based on the grayscale features of CT and MRI registered images, and obtains the image change consistency of each set of matching anchor point pairs; it obtains the fusion weight coefficient of each voxel in each image based on the gradient features of each voxel in the CT and MRI registered images; and it obtains the fusion feature value of each voxel in the CT and MRI registered images based on the image change consistency of multiple sets of matching anchor point pairs within the local area of ​​each voxel, the corresponding fusion weight coefficient in the CT and MRI registered images, and the grayscale features, thereby identifying the boundary of the bone defect area. This invention improves the accuracy of bone defect boundary identification by performing accurate fusion analysis after registering the images.

[0065] It should be noted that, in other embodiments of the present invention, based on the same application concept as the intelligent recognition method for bone defect boundaries based on multimodal image fusion provided in the embodiments of this application, an intelligent recognition system for bone defect boundaries based on multimodal image fusion is also proposed. The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of an intelligent recognition method for bone defect boundaries based on multimodal image fusion.

Claims

1. A method for intelligent recognition of bone defect boundaries based on multimodal image fusion, characterized in that, The method includes: CT and MRI images of the target area are acquired, and multi-dimensional positional parameters of the corresponding images are obtained. The target area includes bone defect areas and normal bone tissue. Based on the differences in multidimensional positional parameters between CT and MRI images, MRI registration images with CT images as the reference are obtained; based on the grayscale characteristics of CT and MRI registration images, CT abnormal anchor points and MRI abnormal anchor points are obtained, and multiple sets of matching anchor point pairs are obtained; based on the grayscale characteristics of different voxels within the local range of the corresponding matching anchor points in different images, the consistency of image changes of each set of matching anchor point pairs is obtained. Based on the gradient characteristics of each voxel in the CT and MRI registered images, the fusion weight coefficient of each voxel in each image is obtained; based on the consistency of image changes of multiple sets of matching anchor point pairs in the local area of ​​each voxel, the corresponding fusion weight coefficients in the CT and MRI registered images, and the grayscale characteristics, the fusion feature value of each voxel in the CT and MRI registered images is obtained. Based on the distribution of fusion feature values ​​of different voxels in CT and MRI registered images, the boundaries of bone defect areas are identified.

2. The intelligent identification method for bone defect boundaries based on multimodal image fusion according to claim 1, characterized in that, The method for acquiring the MRI registration images includes: The cumulative difference values ​​of the same dimension positional parameters between CT and MRI images are obtained and normalized to represent the degree of positional shift. Using CT images as a reference, spatial transformation of MRI images is performed based on the degree of body positional shift to obtain MRI registration images.

3. The intelligent identification method for bone defect boundaries based on multimodal image fusion according to claim 2, characterized in that, The process of using CT images as a reference and spatially transforming MRI images based on the degree of body positional shift to obtain MRI registration images includes: Using CT images as a reference, if the degree of body position deviation is greater than a preset deviation threshold, based on the difference between the corresponding body position parameters of the same dimension between CT images and MRI images, after performing affine transformation on the MRI images, a feature matching algorithm is used to obtain MRI registration images. If the degree of body positional deviation is less than or equal to the preset deviation threshold, the feature matching algorithm is used to obtain the MRI registration image.

4. The intelligent recognition method for bone defect boundaries based on multimodal image fusion according to claim 1, characterized in that, The methods for obtaining the CT abnormal anchor points and MRI abnormal anchor points include: Based on the grayscale characteristics of CT images and MRI registered images, the bone tissue region of CT images and the abnormal signal region of MRI registered images are obtained. For the bone tissue region, the curvature of each edge voxel is obtained, and the difference in curvature between each edge voxel and its adjacent edge voxels is divided by the relative distance between the corresponding edge voxels as the curvature change rate; if the curvature change rate is greater than a preset change threshold, the corresponding edge voxel is used as a CT abnormality anchor point. For abnormal signal regions, the gradient intensity of each voxel is obtained. If the gradient intensity of a voxel is greater than that of other voxels in the spatial range, the corresponding voxel is taken as the MRI abnormal anchor point. The extreme values ​​of gray values ​​among all voxels are obtained, and the voxels corresponding to the extreme values ​​are taken as the MRI abnormal anchor points.

5. The intelligent recognition method for bone defect boundaries based on multimodal image fusion according to claim 4, characterized in that, The methods for obtaining the bone tissue region of the CT images and the abnormal signal region of the MRI registration images include: The gray-level histograms of voxels in CT images are obtained, and an adaptive threshold segmentation algorithm is used to obtain the segmentation threshold of the gray-level histograms. The range of voxels with gray levels greater than the segmentation threshold is taken as the bone tissue region. In T2-weighted MRI images, voxels with gray values ​​higher than a preset gray value threshold are used as seed points. If the gray values ​​of adjacent voxels are within a preset gray value similar range, the corresponding adjacent voxels are used as growth seed points. The range formed by the seed point and all its growth seed points is obtained as the abnormal signal region.

6. The intelligent recognition method for bone defect boundaries based on multimodal image fusion according to claim 1, characterized in that, The methods for obtaining the multiple sets of matching anchor point pairs include: In CT images, the relative distance between each CT abnormal anchor point and different MRI abnormal anchor points is obtained. If the MRI abnormal anchor point corresponding to the minimum relative distance is within the neighborhood of the CT abnormal anchor point, the CT abnormal anchor point and the MRI abnormal anchor point are formed into a set of matching anchor point pairs.

7. The intelligent recognition method for bone defect boundaries based on multimodal image fusion according to claim 1, characterized in that, The method for obtaining the consistency of image changes includes: For any image in CT or MRI registration, obtain the difference between the mean gray value of all voxels within the local area of ​​the corresponding matching anchor point in the image and the gray reference value of normal bone tissue. Based on the deviation characteristics between the difference results and the grayscale fluctuation characteristics of normal bone tissue, the degree of bone tissue difference at the corresponding matching anchor point in each image is obtained. The deviation characteristics are positively correlated with the degree of bone tissue difference. The differences in the degree of bone tissue difference between each pair of matched anchor points in CT and MRI images are obtained. Based on the sum of the degree of bone tissue difference, the differences in the degree of bone tissue difference are negatively correlated and normalized to serve as the consistency of image changes for each pair of matched anchor points.

8. The intelligent recognition method for bone defect boundaries based on multimodal image fusion according to claim 1, characterized in that, The method for obtaining the fusion weight coefficients includes: Obtain the gradient intensity of each voxel in the CT and MRI registered images; The sum of the gradient intensities of each voxel in all images is obtained as the overall gradient intensity of each voxel. For any image, the ratio of the gradient intensity of each voxel in the image to the overall gradient intensity is obtained, and used as the fusion weight coefficient of each voxel in the corresponding image.

9. The intelligent recognition method for bone defect boundaries based on multimodal image fusion according to claim 1, characterized in that, The method for obtaining the fusion feature values ​​includes: The mean value of the image change consistency of all groups of matched anchor point pairs within the local range of each voxel is obtained as the local change consistency of each voxel. The fusion feature value of each voxel is obtained based on the fusion weight coefficient, gray value, and local variation consistency of each voxel in all images.

10. The intelligent recognition method for bone defect boundaries based on multimodal image fusion according to claim 9, characterized in that, The method for obtaining the fusion feature values ​​includes: For any image, the gray values ​​of different voxels in the image are normalized to obtain the product between the fusion weight coefficient of each voxel in each image and the normalized gray value, which is used as the weighted gray value. The product of the weighted gray-level summation of each voxel in all images and the consistency of local changes is calculated as the fusion feature value of each voxel.

Images