A method and system for evaluating spinal osteoporotic fractures based on deep learning

Abstract

The present invention provides a method and system for evaluating spinal osteoporotic fractures based on deep learning, which can improve the comprehensiveness and interpretability of the risk assessment of spinal fragility fractures. The present invention deeply integrates the spinal spatial geometric structure, imaging texture features and clinical data based on a hierarchical feature extraction module, breaking through the limitations of traditional single-modal analysis; innovatively adopts a dual-mode graph neural network structure combining rigid edges and flexible edges, where the rigid edges forcibly maintain the anatomical connection of adjacent vertebral bodies, and the flexible edges quantify the fracture risk cascade effect between inter-segmental vertebral bodies; further combines a multi-head graph attention network to adaptively capture the long-distance global interaction relationship between vertebral bodies, enabling the risk signals of high-risk vertebral bodies to spread to the associated regions along high-weight edges, and intuitively revealing the key risk vertebral bodies and their conduction paths through visual mapping (such as node colors representing risk levels and edge connection strengths reflecting the mutual influence coefficients between vertebral bodies).

Description

Technical Field

[0001] The present invention relates to the technical field of medical image analysis and artificial intelligence, and particularly relates to a method and system for evaluating spinal osteoporotic fractures based on deep learning. Background Art

[0002] There are significant technical bottlenecks in the existing clinical evaluation system: Traditional diagnostic methods mainly rely on Dual energy X-ray Absorptiometry (DXA) to evaluate Bone Mineral Density (BMD), and combine risk assessment tools such as FRAX to predict the risk of spinal fractures. However, it can only evaluate the overall risk of the spine and cannot locate high-risk vertebral bodies, resulting in a high misdiagnosis rate.

[0003] In recent years, artificial intelligence technology has tried to break through the above limitations, but the existing solutions still have the following technical defects:

[0004] 1) Ignoring the fracture cascade effect: Traditional machine learning methods extract features such as single vertebral body texture through radiomics, but do not consider the "fracture cascade effect" between vertebral bodies, that is, when an abnormal vertebral body appears, it will increase the fracture risk of the remaining vertebral bodies, with the greatest impact on adjacent vertebral bodies. Therefore, the dependence of the fracture risk of adjacent vertebral bodies on relevance is ignored, which can lead to a decrease in prediction accuracy;

[0005] 2) Insufficient global perception ability: Deep learning methods need to stack multiple layers of convolutions to achieve cross-vertebral interaction, resulting in a sharp increase in the number of model parameters and a decrease in computational efficiency. Moreover, the cubic receptive field of the standard three-dimensional convolution kernel (such as 3×3×3) is difficult to cover the long and narrow anatomical structure of the spine (with significant extension in the length direction), resulting in the failure of modeling the long-distance dependence relationship between vertebrae;

[0006] 3) Coarse-grained multi-modal fusion mechanism: Although existing methods integrate imaging and clinical data, they adopt a simple splicing strategy, ignoring the weight differences of different modal features between vertebral segments. At the same time, the black-box feature extraction process leads to insufficient decision interpretability, making it difficult for doctors to trace the decision basis.

[0007] Therefore, it is necessary to provide a new way to solve the above technical problems. Summary of the Invention

[0008] To achieve the above objects and other advantages of the present invention, the first object of the present invention is to provide a method for evaluating spinal osteoporotic fractures based on deep learning, including the following steps:

[0009] Obtain the spinal image and clinical data of the patient, and extract multi-modal features;

[0010] Stitch and fuse the multi-modal features, and construct a spine graph structure based on anatomical constraints according to the anatomical characteristics of the spine;

[0011] Use a multi-head graph attention neural network for global interaction of multi-modal features and spine fracture risk assessment.

[0012] Furthermore, the multi-modal features include geometric features, imaging features, and clinical features.

[0013] Furthermore, the step of extracting multi-modal features includes extracting geometric features:

[0014] Take the center point position of the minimum circumscribed rectangle of each vertebral body as the spatial position of the vertebral body ;

[0015] Calculate geometric features using the Cobb measurement method.

[0016] Furthermore, the step of calculating geometric features using the Cobb measurement method includes:

[0017] Take the normal intersection angle of the extension lines of the outer edges of adjacent vertebral bodies as the intervertebral angle, and the distance between the end points of the inner edge tangents as the intervertebral space;

[0018] The characteristic values of each vertebral body are the average of the calculation results of adjacent two vertebral bodies;

[0019] Fuse the spatial coordinates, intervertebral angle, and intervertebral space into the geometric features of each vertebral body to realize the quantitative characterization of the spatial morphology of the spine vertebral body.

[0020] Furthermore, the step of extracting multi-modal features includes extracting imaging features:

[0021] Take the minimum circumscribed cuboid of each vertebral body as the input size of the 3D convolutional neural network, and use zero-padding to uniformly complete the size of the remaining vertebral bodies;

[0022] Extract imaging features of each vertebral body in turn through the 3D convolutional neural network.

[0023] Furthermore, the 3D convolutional neural network includes three 3D convolutional layers and three global pooling layers. Each convolutional layer is equipped with batch normalization and activation functions. The convolutional layer is used to perform convolutional operations on the input image to extract its imaging features. The global pooling layer is used to reduce the dimension of the feature map obtained by convolution, and finally output the imaging features of each vertebral body.

[0024] Furthermore, the step of extracting multi-modal features includes extracting clinical features:

[0025] For the clinical data of the patient, use data preprocessing to achieve heterogeneous feature fusion;

[0026] The preprocessed data is concatenated into a multi-dimensional vector, which is used as the clinical feature of each vertebral body.

[0027] Furthermore, the steps of realizing heterogeneous feature fusion for the clinical data of the patient by data preprocessing include:

[0028] Standardize the continuous variables to eliminate the difference in measurement units;

[0029] Use one-hot encoding or label encoding for categorical variables to avoid misjudgment of the category order by the model;

[0030] The steps of concatenating the preprocessed data into a multi-dimensional vector include:

[0031] Concatenate the standardized continuous variables and the encoded categorical variables into a multi-dimensional vector.

[0032] Furthermore, the steps of concatenating and fusing the multi-modal features and constructing a spine graph structure based on anatomical constraints according to the anatomical characteristics of the spine include:

[0033] Define each vertebral body of the spine as a graph node, and the feature vector of each node is concatenated and fused by three types of modal data: imaging, geometry, and clinical;

[0034] Construct two types of edge connections according to the anatomical characteristics of the spine.

[0035] Furthermore, the steps of constructing two types of edge connections according to the anatomical characteristics of the spine include:

[0036] For adjacent vertebral bodies, use rigid edges to force connections and fix the weights to maintain the anatomical continuity of the spine sequence;

[0037] For cross-segment vertebral bodies, use flexible edges to connect, and calculate the connection weights according to the spatial Euclidean distance and bone density gradient Calculate the connection weights through the Gaussian kernel function Quantify the mechanical correlation strength between vertebral bodies. When > the threshold, establish a connection to screen out vertebral body pairs with significant biomechanical conduction effects, and realize the establishment of a cross-scale structure from a rigid anatomical framework to a flexible connection network.

[0038] Furthermore, the steps of using a multi-head graph attention neural network for global interaction of multi-modal features and spine fracture risk assessment include:

[0039] Map the node features to the hidden space through a learnable parameter matrix For each vertebral body node and its neighbors , calculate the normalized attention weights:

[0040] ;

[0041] Among them, is the attention vector;

[0042] According to iteratively update the node features, so that the multi-modal features of the high-risk vertebral body diffuse to the associated vertebral bodies through high-weight edges, and the high-weight edges are configured as a preset value, which is the anatomical connection strength × dynamic attention weight;

[0043] Output the fracture probability of each vertebral body through the Sigmoid function.

[0044] Furthermore, it further includes the steps of:

[0045] Generate a risk heat map through bicubic interpolation and superimpose it on the sagittal reconstruction view of the original image;

[0046] Represent the risk level with the node color, and the edge transparency is proportional to and display the connection between any vertebral body node and its associated vertebral body.

[0047] The second object of the present invention is to provide a deep learning-based spinal osteoporosis fracture assessment system, which applies the above method and includes a multi-modal feature extraction module, a spinal graph structure construction module, and a spinal fracture risk assessment module; among them,

[0048] The multi-modal feature extraction module is used to obtain the patient's spinal image and clinical data and extract multi-modal features;

[0049] The spinal graph structure construction module is used to splice and fuse the multi-modal features and construct a spinal graph structure based on anatomical constraints according to the spinal anatomy characteristics;

[0050] The spinal fracture risk assessment module is used to perform global interaction of multi-modal features and spinal fracture risk assessment by using a multi-head graph attention neural network.

[0051] Compared with the prior art, the beneficial effects of the present invention are:

[0052] The present invention constructs a method and system for evaluating spinal osteoporotic fractures based on anatomical constraints and graph neural networks, which can improve the comprehensiveness and interpretability of the risk assessment of spinal fragility fractures. Compared with the prior art, its core advantages are as follows: based on a hierarchical feature extraction module, it deeply integrates the spinal spatial geometric structure, imaging texture features, and clinical data, breaking through the limitations of traditional single-modal analysis; innovatively adopting a dual-mode graph neural network structure combining rigid edges and flexible edges, where the rigid edges force the anatomical connection of adjacent vertebrae to be maintained, and the flexible edges quantify the fracture risk cascade effect between inter-segmental vertebrae; further combining a multi-head graph attention network to adaptively capture the long-distance global interaction relationship between vertebrae, enabling the risk signal of high-risk vertebrae to spread to the associated area along high-weight edges, and intuitively revealing the key risk vertebrae and their conduction paths through visual mapping (such as node colors representing risk levels and edge connection strengths reflecting the mutual influence coefficients between vertebrae). This method reduces the computational complexity of traditional finite element analysis while avoiding the interpretability defects of black-box models, providing an intelligent solution that takes into account both anatomical constraints and clinical operability for the assessment of spinal fragility fractures.

[0053] The above description is only an overview of the technical solution of the present invention. In order to be able to understand the technical means of the present invention more clearly and to be implemented in accordance with the content of the specification, the following describes in detail with reference to the preferred embodiments of the present invention and the accompanying drawings. The specific implementation manner of the present invention is given in detail by the following embodiments and their accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The drawings described herein are used to provide a further understanding of the present invention, and form a part of this application. The illustrative embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0055] Figure 1 is the flowchart of the method for evaluating spinal osteoporotic fractures based on deep learning Figure 1 ;

[0056] Figure 2 is the flowchart of the method for evaluating spinal osteoporotic fractures based on deep learning Figure 2 ;

[0057] Figure 3 is the schematic diagram of the spinal graph structure with anatomical constraints;

[0058] Figure 4 is the schematic diagram of the first layer of the multi-head graph attention network;

[0059] Figure 5 is the flowchart of geometric feature extraction;

[0060] Figure 6 is the flowchart of calculating geometric features using the Cobb measurement method;

[0061] Figure 7 It is the flow chart of image feature extraction;

[0062] Figure 8 Flowcharts were extracted for clinical features;

[0063] Figure 9 Flowchart for clinical data preprocessing;

[0064] Figure 10 Construct a flow chart for the structure of the spine graph based on anatomical constraints;

[0065] Figure 11 To construct two types of edge connection flow graphs based on the anatomical characteristics of the spine;

[0066] Figure 12 This is a flowchart of the global interaction of multi-modal features of multi-head graph attention neural network and spinal fracture risk assessment;

[0067] Figure 13 Visualize a flow chart for the model;

[0068] Figure 14 Schematic diagram of the spinal osteoporotic fracture assessment system based on deep learning;

[0069] Figure 15 It is a schematic diagram of computer equipment;

[0070] Figure 16 A schematic diagram of a computer-readable storage medium. DETAILED DESCRIPTION

[0071] The present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. It should be noted that, under the premise of no conflict, the embodiments or technical features described below can be arbitrarily combined to form a new embodiment.

[0072] Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in the field without making any creative work shall fall within the scope of protection of the present invention.

[0073] The figure numbers in this application are only used to distinguish the various steps in the scheme, and are not used to limit the execution order of the various steps. The specific execution order is subject to the description in the specification.

[0074] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention. Example 1

[0075] A method for evaluating spinal osteoporotic fractures based on deep learning, as Figure 1 - Figure 2 shown, includes the following steps:

[0076] S100. Obtain the spinal image and clinical data of the patient, and extract multi-modal features;

[0077] In this embodiment, the spinal CT image and clinical data of the patient (such as age, bone density value, fracture history, etc.) are obtained, and multi-modal features are extracted through the following parallel processing flow.

[0078] Furthermore, the multi-modal features include geometric features, image features, and clinical features.

[0079] In some embodiments, as Figure 5 shown, the step of extracting multi-modal features includes extracting geometric features:

[0080] S101. Take the center point position of the minimum circumscribed rectangle of each vertebral body as the spatial position of the vertebral body ;

[0081] S102. Calculate geometric features using the Cobb measurement method.

[0082] Furthermore, as Figure 6 shown, the step of calculating geometric features using the Cobb measurement method includes:

[0083] S1021. Take the normal intersection angle of the extension lines of the outer edges of adjacent vertebral bodies as the intervertebral angle, and the distance between the end points of the inner tangent as the intervertebral space;

[0084] S1022. Take the average value of the calculation results of adjacent two vertebral bodies for each vertebral body feature value, and directly use the unilateral adjacent value for the first and last vertebral bodies;

[0085] S1023. Integrate the spatial coordinates, intervertebral angle and intervertebral space into the geometric features of each vertebral body to realize the quantitative characterization of the spatial morphology of the spinal vertebral body.

[0086] In some embodiments, as Figure 7 shown, the step of extracting multi-modal features includes extracting image features:

[0087] S103. Take the minimum circumscribed cuboid of each vertebral body as the input size of the 3D convolutional neural network, and use zero-padding to uniformly complete the size of the remaining vertebral bodies;

[0088] S104. Extract image features of each vertebral body in turn through the 3D convolutional neural network.

[0089] Further, the 3D convolutional neural network includes three 3D convolutional layers and three global pooling layers. Each convolutional layer is equipped with batch normalization (BN) and an activation function (ReLU). The convolutional layer is used to perform a convolutional operation on the input image to extract its image features, and the global pooling layer is used to reduce the dimension of the feature map obtained by convolution, and finally output the image features of each vertebra.

[0090] In some embodiments, as Figure 8 shown, the step of extracting multi-modal features includes extracting clinical features:

[0091] S105. For the clinical data of the patient, data preprocessing is used to achieve heterogeneous feature fusion;

[0092] Specifically, for clinical information such as the patient's bone density (continuous variable), age (continuous variable), gender (categorical variable), and fracture history (categorical variable), data preprocessing is used to achieve heterogeneous feature fusion.

[0093] Further, as Figure 9 shown, the step of using data preprocessing to achieve heterogeneous feature fusion for the patient's clinical data includes:

[0094] S1051. Standardize the continuous variables. For example, bone density uses Z-score standardization, and age uses Min-Max normalization to eliminate the difference in dimension;

[0095] S1052. For categorical variables (such as gender, fracture history, etc.), use one-hot encoding (such as converting gender to a binary vector) or label encoding (such as converting fracture history to 0 / 1) to avoid misjudgment of the category order by the model;

[0096] S106. Concatenate the preprocessed data into a multi-dimensional vector, and use this as the clinical feature of each vertebra.

[0097] Specifically, the step of concatenating the preprocessed data into a multi-dimensional vector includes:

[0098] Concatenate the standardized continuous variables and the encoded categorical variables into a multi-dimensional vector, and use this as the clinical feature of each vertebra.

[0099] S110. Concatenate and fuse the multi-modal features, and construct a spine graph structure based on anatomical constraints according to the anatomical characteristics of the spine;

[0100] In some embodiments, as Figure 10 shown, the step of concatenating and fusing the multi-modal features and constructing a spine graph structure based on anatomical constraints according to the anatomical characteristics of the spine includes:

[0101] S111. Define each vertebral body of the spine (T1-L5) as a graph node, and the feature vector of each node is spliced and fused by three types of modal data: imaging, geometry, and clinical, to ensure the efficient representation of multi-modal information.

[0102] S112. Construct two types of edge connections based on the anatomical characteristics of the spine.

[0103] Further, as Figure 11 shown, the step of constructing two types of edge connections based on the anatomical characteristics of the spine includes:

[0104] S1121. For adjacent vertebral bodies (such as T12-L1, L1-L2), use rigid edges to force the connection and fix the weight at 1.0 to maintain the anatomical continuity of the spinal column;

[0105] S1122. For cross-segment vertebral bodies, use flexible edges to connect, and calculate the connection weight according to the spatial Euclidean distance and bone density gradient Calculate the connection weight, and quantify the mechanical correlation strength between vertebral bodies through the Gaussian kernel function ( ). When > threshold (such as, ), establish the connection, so as to screen out the vertebral body pairs with significant biomechanical conduction effects, and realize the establishment of a cross-scale structure from a rigid anatomical framework to a flexible connection network, as Figure 3 shown.

[0106] S120. Use a multi-head graph attention neural network for global interaction of multi-modal features and spinal fracture risk assessment.

[0107] Optionally, use a 3-layer multi-head graph attention network (Multi-head GAT, number of heads = 4) for global interaction of multi-modal features and fracture risk assessment, Figure 4 is a schematic diagram of the first layer of the 3-layer multi-head graph attention network.

[0108] In some embodiments, as Figure 12 shown, the step of using a multi-head graph attention neural network for global interaction of multi-modal features and spinal fracture risk assessment includes:

[0109] S121. Map the node features to the hidden space through a learnable parameter matrix For each vertebral body node and its neighbors , calculate the normalized attention weight:

[0110] ;

[0111] Among them, is the attention vector;

[0112] S122. Press Iteratively update the node features, so that the multi-modal features of the high-risk vertebral body diffuse to the associated vertebral bodies through high-weight edges, and the high-weight edges are configured as a preset value, such as , which can be interpreted as anatomical connection strength × dynamic attention weight;

[0113] S123. Output the fracture probability of each vertebral body through the Sigmoid function.

[0114] In some embodiments, such as Figure 13 shown, it further includes the step:

[0115] S130. After mapping the final node features to risk probabilities (0-1) through the Sigmoid function, generate a risk heat map through bicubic interpolation and superimpose it on the sagittal reconstruction view of the original CT image;

[0116] S140. Represent the risk level with the node color, for example, red indicates a risk > 0.7, orange indicates a risk of 0.4-0.7, green indicates a risk < 0.4, and the edge transparency is proportional to and display the connection between any vertebral body node and its associated vertebral body ( ).

[0117] The present invention provides a method for evaluating spinal osteoporotic fractures based on anatomical constraints and graph neural networks. First, a multi-modal feature extraction module is used to parallelly obtain the multi-modal features of the spine: geometric features reflecting the complex structure of the spine, image features reflecting the vertebral body morphology and internal texture, and clinical features reflecting individual information. On this basis, a spinal vertebral body graph structure is constructed based on anatomical constraints, each vertebral body is defined as a graph node, the node features are generated by fusing multi-modal data, and two types of edge connections are established according to spinal anatomical constraints - rigid edges are used to forcefully connect adjacent vertebral bodies to maintain anatomical continuity, and flexible edge weights are calculated through spatial distance and bone density gradient between cross-segment vertebral bodies to realize the graph neural network structure modeling of spinal anatomical structure characteristics. Further, a multi-head graph attention network is used to capture the influence coefficients between vertebral bodies, realize the global interaction of multi-modal features between vertebral bodies, drive the risk signals of high-risk vertebral bodies to diffuse to associated vertebral bodies along high-weight edges through a learnable attention weight matrix, and realize the quantification of fracture cascade effects. Finally, a visualization method is used to map the node color to the fracture risk level (red - high risk, green - low risk), and the edge connection shows the attention connection between each vertebral body and the rest of the vertebral bodies, and the attention weight is used to intuitively reflect the mutual influence relationship between vertebral bodies. Embodiment 2

[0118] A spine osteoporotic fracture assessment system based on deep learning applies the above method. For a detailed description of the method, reference can be made to the corresponding description in the above method embodiments, which will not be elaborated here. As Figure 14 shown, the system 200 includes a multi-modal feature extraction module 210, a spine graph structure construction module 220, and a spine fracture risk assessment module 230; among them,

[0119] The multi-modal feature extraction module is used to obtain the patient's spine image and clinical data and extract multi-modal features;

[0120] The spine graph structure construction module is used to splice and fuse the multi-modal features and construct a spine graph structure based on anatomical constraints according to the spine anatomical characteristics;

[0121] The spine fracture risk assessment module is used to perform global interaction of multi-modal features and spine fracture risk assessment by using a multi-head graph attention neural network.

[0122] Based on the technical solution of the above embodiment, optionally, the multi-modal features include geometric features, image features, and clinical features.

[0123] Based on the technical solution of the above embodiment, optionally, the step of extracting multi-modal features includes extracting geometric features:

[0124] The center point position of the minimum circumscribed rectangle of each vertebral body is used as the spatial position of the vertebral body ;

[0125] The Cobb measurement method is used to calculate geometric features.

[0126] Based on the technical solution of the above embodiment, optionally, the step of calculating geometric features by using the Cobb measurement method includes:

[0127] The normal intersection angle of the extended outer edges of adjacent vertebral bodies is used as the intervertebral angle, and the distance between the end points of the inner edge tangents is used as the intervertebral space;

[0128] The characteristic value of each vertebral body takes the average value of the calculation results of adjacent two vertebral bodies;

[0129] The spatial coordinates, intervertebral angle, and intervertebral space are fused into the geometric features of each vertebral body to realize the quantitative characterization of the spatial morphology of the spine vertebral body.

[0130] Based on the technical solution of the above embodiment, optionally, the step of extracting multi-modal features includes extracting image features:

[0131] Take the minimum bounding cuboid of each vertebral body as the input size of the 3D convolutional neural network, and use zero-padding to uniformly complete the size of the remaining vertebral bodies.

[0132] Extract image features from each vertebral body in turn through the 3D convolutional neural network.

[0133] On the basis of the technical solution of the above embodiment, optionally, the 3D convolutional neural network includes three 3D convolutional layers and three global pooling layers. Each convolutional layer is equipped with batch normalization and an activation function. The convolutional layer is used to perform a convolutional operation on the input image to extract its image features. The global pooling layer is used to reduce the dimension of the feature map obtained by convolution, and finally output the image features of each vertebral body.

[0134] On the basis of the technical solution of the above embodiment, optionally, the step of extracting multi-modal features includes extracting clinical features:

[0135] For the clinical data of the patient, use data preprocessing to achieve heterogeneous feature fusion;

[0136] Concatenate the preprocessed data into a multi-dimensional vector, and use this as the clinical feature of each vertebral body.

[0137] On the basis of the technical solution of the above embodiment, optionally, the step of using data preprocessing to achieve heterogeneous feature fusion for the clinical data of the patient includes:

[0138] Standardize continuous variables to eliminate the difference in dimension;

[0139] Use one-hot encoding or label encoding for categorical variables to avoid misjudgment of the category order by the model;

[0140] The step of concatenating the preprocessed data into a multi-dimensional vector includes:

[0141] Concatenate the standardized continuous variables and the encoded categorical variables into a multi-dimensional vector.

[0142] On the basis of the technical solution of the above embodiment, optionally, the step of concatenating and fusing the multi-modal features and constructing a spine graph structure based on anatomical constraints according to the anatomical characteristics of the spine includes:

[0143] Define each vertebral body of the spine as a graph node, and the feature vector of each node is concatenated and fused by three types of modal data: image, geometry, and clinical;

[0144] Construct two types of edge connections according to the anatomical characteristics of the spine.

[0145] On the basis of the technical solution of the above embodiment, optionally, the step of constructing two types of edge connections according to the anatomical characteristics of the spine includes:

[0146] Adopt rigid edges to forcefully connect and fix the weights for adjacent vertebral bodies to maintain the anatomical continuity of the spinal column sequence;

[0147] For multi-segment vertebral bodies, use flexible edges to connect, and calculate the connection weights according to the spatial Euclidean distance and bone density gradient Calculate the connection weights, and quantify the mechanical correlation strength between vertebral bodies through the Gaussian kernel function When > the threshold, establish a connection to screen out vertebral body pairs with significant biomechanical conduction effects, and achieve the establishment of a cross-scale structure from a rigid anatomical framework to a flexible connection network.

[0148] Based on the technical solution of the above embodiment, optionally, the step of using a multi-head graph attention neural network for multi-modal feature global interaction and spinal fracture risk assessment includes:

[0149] Map the node features to the latent space through a learnable parameter matrix For each vertebral body node and its neighbors , calculate the normalized attention weights:

[0150] ;

[0151] Among them, is the attention vector;

[0152] According to Iteratively update the node features to spread the multi-modal features of high-risk vertebral bodies to associated vertebral bodies through high-weight edges, and the high-weight edges are configured as a preset value, which is the anatomical connection strength × dynamic attention weight;

[0153] Output the fracture probability of each vertebral body through the Sigmoid function.

[0154] Based on the technical solution of the above embodiment, optionally, it further includes the steps:

[0155] Generate a risk heat map through bicubic interpolation and superimpose it on the sagittal reconstruction view of the original image;

[0156] Use the node color to represent the risk level, and the edge transparency is proportional to and display the connection between any vertebral body node and its associated vertebral body.

[0157] The present invention provides a spinal osteoporotic fracture assessment system based on anatomical constraints and graph neural networks. First, a multi-modal feature extraction module is used to parallelly obtain multi-modal features of the spine: geometric features reflecting the complex structure of the spine, imaging features reflecting the vertebral body morphology and internal texture, and clinical features reflecting individual information. On this basis, a spinal vertebral graph structure is constructed based on anatomical constraints. Each vertebral body is defined as a graph node, and the node features are generated by fusing multi-modal data. Two types of edge connections are established according to spinal anatomical constraints - rigid edges are used to forcefully connect adjacent vertebral bodies to maintain anatomical continuity, and flexible edge weights are calculated based on spatial distance and bone density gradient between cross-segment vertebral bodies, realizing the graph neural network structure modeling of the spinal anatomical structure characteristics. Further, a multi-head graph attention network is used to capture the influence coefficients between vertebral bodies, realizing the global interaction of multi-modal features between vertebrae. The risk signal of high-risk vertebral bodies is driven to spread to associated vertebral bodies along high-weight edges through a learnable attention weight matrix, realizing the quantification of the fracture cascade effect. Finally, a visualization method is used to map the node colors to the fracture risk levels (red - high risk, green - low risk), and the edge connections show the attention connections of each vertebral body to the rest of the vertebral bodies, intuitively reflecting the mutual influence relationship between vertebral bodies using the attention weights. Embodiment 3

[0158] A computer device 300, as Figure 15 shown, includes a memory 310, a processor 320, and a computer program 330 stored on the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a method for assessing spinal osteoporotic fractures based on deep learning. For a detailed description of the method, reference can be made to the corresponding description in the above method embodiments, which will not be elaborated here. Embodiment 4

[0159] A computer-readable storage medium, as Figure 16 shown, stores a computer program thereon. When the computer program is executed by a processor, it implements the steps of a method for assessing spinal osteoporotic fractures based on deep learning. For a detailed description of the method, reference can be made to the corresponding description in the above method embodiments, which will not be elaborated here.

[0160] The number of devices and the scale of processing described here are used to simplify the description of the present invention. Applications, modifications, and variations of the present invention will be obvious to those skilled in the art.

[0161] Although the embodiments of the present invention have been disclosed above, it is not limited to the applications listed in the specification and embodiments. It can be fully applied to various fields suitable for the present invention. For those skilled in the art, additional modifications can be easily made. Therefore, without departing from the general concept defined by the claims and their equivalents, the present invention is not limited to the specific details and the illustrated and described examples here.

[0162] The device, computer device, non-volatile computer storage medium, and method provided in the embodiments of this specification are corresponding. Therefore, the device, computer device, and non-volatile computer storage medium also have beneficial technical effects similar to those of the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the corresponding device, computer device, and non-volatile computer storage medium will not be elaborated here.

[0163] Those skilled in the art also know that in addition to implementing the controller in the form of pure computer-readable program code, the method steps can be logically programmed to enable the controller to implement the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, embedded microcontrollers, etc. Therefore, such a controller can be regarded as a hardware component, and the devices included therein for implementing various functions can also be regarded as the structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software units for implementing the method and the structures within the hardware component.

[0164] The systems, devices, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described by dividing them into various units according to their functions. Of course, when implementing one or more embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0165] Those skilled in the art should understand that the embodiments of this specification can be provided as a method, system, or computer program product. Therefore, the embodiments of this specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of this specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0166] This specification is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementing the processes Figure 1 one process or multiple processes and / or blocks Figure 1means for the functions specified in one or more boxes.

[0167] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction device that implements the functions specified in one Figure 1 one or more processes and / or boxes Figure 1 means for the functions specified in one or more boxes.

[0168] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operational steps are performed on the computer or other programmable device to produce a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one Figure 1 one or more processes and / or boxes Figure 1 means for the functions specified in one or more boxes.

[0169] It should also be noted that the term "comprising", "including" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, commodity or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, commodity or device. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, commodity or device comprising the said element.

[0170] This specification can be described in the general context of computer-executable instructions executed by a computer, such as program units. Generally, program units include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The specification can also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program units can be located in local and remote computer storage media including storage devices.

[0171] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method embodiments.

[0172] The above is only for the embodiments of this specification and is not intended to limit one or more embodiments of this specification. For those skilled in the art, various changes and modifications can be made to one or more embodiments of this specification. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification shall be included within the scope of the claims of one or more embodiments of this specification.

Claims

1. A method for evaluating spinal osteoporotic fractures based on deep learning, characterized in that: The following steps are involved: Obtain patient spinal images and clinical data, and extract multimodal features; The multimodal features are spliced ​​and fused to construct a spinal graph structure based on anatomical constraints according to the anatomical characteristics of the spine; A multi-head graph attention neural network is used for global interaction of multimodal features and spinal fracture risk assessment; The step of splicing and fusing the multimodal features and constructing a spinal graph structure based on anatomical constraints according to the anatomical characteristics of the spine includes: Each vertebra of the spine is defined as a graph node, and the feature vector of each node is spliced ​​and fused by three types of modality data: imaging, geometry, and clinical. Two types of edge connections are constructed based on the anatomical characteristics of the spine; The steps of constructing two types of edge connections according to the anatomical characteristics of the spine include: Adjacent vertebrae are connected by rigid edges and weights are fixed to maintain the anatomical continuity of the spinal column sequence. For inter-segment vertebrae, flexible edge connections are used based on the spatial Euclidean distance. Bone density gradient Calculate the connection weights using the Gaussian kernel function Quantify the strength of mechanical connection between vertebrae. >When the threshold is reached, connections are established to screen out vertebral pairs with significant biomechanical conduction effects, thus achieving cross-scale structural establishment from a rigid anatomical framework to a flexible connection network; The steps of using a multi-head graph attention neural network to perform global interaction of multimodal features and spinal fracture risk assessment include: Through the learnable parameter matrix Map the node features to the latent space, for each vertebral node and its neighbors , calculate the normalized attention weight: ， in, is the attention vector; according to Iteratively update node features so that the multimodal features of high-risk vertebrae diffuse to associated vertebrae through high-weight edges, where the high-weight edges are configured as Preset value, is anatomical connection strength × dynamic attention weight; The probability of each vertebral fracture is output through the Sigmoid function.

2. The method for evaluating spinal osteoporotic fractures based on deep learning according to claim 1, characterized in that: The multimodal features include geometric features, imaging features, and clinical features.

3. A method for evaluating spinal osteoporotic fractures based on deep learning as claimed in claim 2, characterized in that: The step of extracting multimodal features includes extracting geometric features: The center point of the minimum circumscribed rectangle of each vertebra is taken as the spatial position of the vertebra ; The geometrical characteristics are calculated using the Cobb measurement method.

4. A method for evaluating spinal osteoporotic fractures based on deep learning as claimed in claim 3, characterized in that: The step of calculating geometric features using the Cobb measurement method comprises: The intersection angle of the normal lines of the extended lines of the adjacent vertebral outer edges As the intervertebral angle, the distance between the end points of the inner edge tangent as intervertebral space; The characteristic value of each vertebra is the average value of the calculation results of two adjacent vertebrae; The spatial coordinates, intervertebral angles and intervertebral spaces are integrated into the geometric features of each vertebra to achieve quantitative characterization of the spatial morphology of the spinal vertebrae.

5. A method for evaluating spinal osteoporotic fractures based on deep learning as claimed in claim 2, characterized in that: The step of extracting multimodal features includes extracting image features: The smallest circumscribed cuboid of each vertebra is used as the input size of the 3D convolutional neural network, and the size of the remaining vertebrae is uniformly filled by zero padding. The image features of each vertebra are extracted in turn through a 3D convolutional neural network.

6. A method for evaluating spinal osteoporotic fractures based on deep learning as claimed in claim 5, characterized in that: The 3D convolutional neural network includes three 3D convolutional layers and three global pooling layers. Each convolutional layer has batch normalization and activation functions. The convolutional layer is used to perform convolution operations on the input image to extract its image features. The global pooling layer is used to reduce the dimension of the feature map obtained by convolution, and finally output the image features of each vertebra.

7. A method for evaluating spinal osteoporotic fractures based on deep learning as claimed in claim 2, characterized in that: The step of extracting multimodal features includes extracting clinical features: For the patients’ clinical data, data preprocessing is used to achieve heterogeneous feature fusion; The preprocessed data were concatenated into multidimensional vectors, which were used as the clinical characteristics of each vertebra.

8. A method for evaluating spinal osteoporotic fractures based on deep learning as claimed in claim 7, characterized in that: The steps of implementing heterogeneous feature fusion by data preprocessing for the patient's clinical data include: Continuous variables were standardized to eliminate dimensional differences; Use one-hot encoding or label encoding for categorical variables to avoid the model misjudging the order of categories; The step of splicing the preprocessed data into a multidimensional vector comprises: The standardized continuous variables and the encoded categorical variables are concatenated into a multidimensional vector.

9. A method for evaluating spinal osteoporotic fractures based on deep learning as claimed in claim 1, characterized in that: Also includes the steps: The risk heat map was generated by bicubic interpolation and superimposed on the sagittal reconstructed view of the original image; The node color indicates the risk level, and the edge transparency and proportional to the vertebral body, and shows the connection between any vertebral body node and its associated vertebral body.

10. A spinal osteoporotic fracture assessment system based on deep learning, using the method according to any one of claims 1 to 9, characterized in that: It includes multimodal feature extraction module, spinal graph structure construction module, and spinal fracture risk assessment module; among them, The multimodal feature extraction module is used to obtain the patient's spinal images and clinical data and extract multimodal features; The spinal graph structure construction module is used to splice and fuse the multimodal features and construct a spinal graph structure based on anatomical constraints according to the anatomical characteristics of the spine; The spinal fracture risk assessment module is used to use a multi-head graph attention neural network to perform global interaction of multimodal features and spinal fracture risk assessment.

Images