Method and system for early screening of abnormal spinal curvature based on multi-modal data fusion

Abstract

The invention relates to an early screening method and system for abnormal spinal curvature based on multi-modal data fusion, and the method comprises the steps: carrying out the fusion of space-time sequence features based on obtained multi-modal data, and carrying out the three-dimensional biomechanical modeling, and obtaining a spine-pelvis-lower limb dynamic coupling network; carrying out weighted calculation on the key mechanical transmission path through a graph attention mechanism based on a dynamic coupling network to obtain a path strengthening feature vector; performing multi-modal risk quantitative modeling based on the path strengthening feature vector to obtain a multi-dimensional risk index; and based on multi-dimensional risk indexes, carrying out grading early warning decision making through threshold value comparison and dynamic risk layering so as to realize early screening of spinal curvature abnormity. By implementing the scheme, the early screening accuracy of the abnormal spinal curvature can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer-aided diagnosis, in particular to a scoliosis early screening method and system based on multi-modal data fusion. BACKGROUND

[0002] Scoliosis (such as adolescent idiopathic scoliosis, postural scoliosis, etc.) is a common musculoskeletal disease in the adolescent population. Early screening and intervention are of great clinical significance for preventing deformity progression and improving prognosis. Traditional screening methods have the following limitations: 1) strong subjectivity, such as Adam forward bending test, which relies on doctor's visual assessment of scoliosis and is easily influenced by observer's experience; 2) radiation exposure, as X-ray examination is the "gold standard" for diagnosing scoliosis, but frequent use may increase the risk of cancer in adolescents; 3) existing computer-aided screening systems are mostly based on single data source (such as image or posture data), which is difficult to fully capture the mechanical coupling relationship of spine-pelvis-lower extremities; 4) traditional methods focus on static morphological analysis, ignoring the abnormality of mechanical transmission in the movement process. Therefore, in order to solve the problems of strong subjectivity, high radiation exposure risk, strong limitations of single modal data, and insufficient identification of dynamic mechanical transmission abnormality in traditional screening methods, it is necessary to build an intelligent screening method and system based on multi-modal data fusion and three-dimensional biomechanical modeling, to realize a new paradigm of radiation-free, full-dimensional, early detection, and precise intervention for scoliosis screening through spatiotemporal sequence feature integration, key path analysis driven by graph attention mechanism, and dynamic risk stratification warning. SUMMARY

[0003] The technical problem to be solved by the present application is to provide a scoliosis early screening method and system based on multi-modal data fusion to solve the above-mentioned problems of the prior art.

[0004] The technical solution of the present application to solve the above-mentioned technical problem is as follows: a scoliosis early screening method based on multi-modal data fusion, the method comprising:

[0005] S1, based on the obtained multi-modal data, spatiotemporal sequence features are fused, and three-dimensional biomechanical modeling is performed to obtain a dynamic coupling network of spine-pelvis-lower extremities;

[0006] S2, based on the dynamic coupling network, weighted calculation of key mechanical transmission path is performed through graph attention mechanism to obtain a path reinforcement feature vector;

[0007] S3, based on the path reinforcement feature vector, multi-modal risk quantification modeling is performed to obtain a multi-dimensional risk index;

[0008] S4, based on the multi-dimensional risk index, through threshold comparison and dynamic risk stratification, a graded early warning decision is made to realize early screening of spinal curvature abnormalities.

[0009] Further, in step S1, the multi-modal data includes multi-channel surface electromyography data collected based on a back electrode array, foot region pressure data collected based on a foot pressure sensor, and spinal motion trajectory data collected based on an inertial measurement unit.

[0010] Further, in step S1, the multi-modal data includes multi-channel surface electromyography data collected based on a back electrode array, foot region pressure data collected based on a foot pressure sensor, and spinal motion trajectory data collected based on an inertial measurement unit.

[0011] S11, data preprocessing and spatio-temporal alignment are performed on the obtained multi-modal data to obtain a sEMG signal matrix after spatio-temporal calibration, a foot pressure distribution heat map, and a filtered spinal motion trajectory;

[0012] S12, muscle synergistic contraction parameters are extracted from the sEMG signal matrix, gait asymmetry parameters are extracted based on the foot pressure distribution heat map, and spinal motion harmonic components are extracted from the filtered spinal motion trajectory;

[0013] S13, based on the muscle synergistic contraction parameters, gait asymmetry parameters, and spinal motion harmonic components, a cross-modal spatio-temporal feature tensor and an anatomic-motion joint atlas are calculated through a multi-scale spatio-temporal graph convolution network;

[0014] S14, based on the cross-modal spatio-temporal feature tensor and the anatomic-motion joint atlas, three-dimensional biomechanical modeling and dynamic coupling network construction are performed to obtain a spinal-pelvic-lower limb dynamic coupling network.

[0015] Further, in step S12, the gait asymmetry parameters include a gait cycle asymmetry index, a bilateral lower limb load difference coefficient, and time-frequency features of a center of pressure trajectory, wherein:

[0016] The gait cycle asymmetry index is calculated by the following formula:

[0017] ;

[0018] Wherein, N represents the total number of gait cycles, represents the left and right foot ground contact phase difference, represents the complete gait cycle phase, represents the left foot pressure integral, represents the right foot pressure integral;

[0019] The double lower limb load difference coefficient is calculated by the following formula:

[0020] ;

[0021] Wherein, T represents the length of the time window, , represents the left foot pressure peak, represents the right foot pressure peak.

[0022] Further, in step S13, the cross-modal spatio-temporal feature tensor and the anatomic-motor joint atlas are calculated based on the muscle synergistic contraction parameter, gait asymmetry parameter and spinal motion harmonic component through a multi-scale spatio-temporal graph convolution network, comprising:

[0023] S131, different scale spatio-temporal features are extracted from the muscle synergistic contraction parameter, gait asymmetry parameter and spinal motion harmonic component through a multi-scale spatio-temporal graph convolution layer;

[0024] S132, the extracted different modal features are spliced and weighted summed based on a graph attention mechanism to obtain a fused cross-modal spatio-temporal feature tensor;

[0025] S133, the graph structure information of the anatomic structure-motor function correlation and the multi-modal data spatio-temporal dependence is determined;

[0026] S134, the graph structure information is input into the multi-scale spatio-temporal graph convolution network, and the graph structure is optimized based on the graph attention mechanism in the network and combined with anatomical knowledge to obtain the anatomic-motor joint atlas.

[0027] Further, in step S2, the path reinforcement feature vector is obtained by weighted calculation of the key mechanical transmission path based on the dynamic coupling network through the graph attention mechanism, comprising:

[0028] S21, the dynamic coupling network is encoded based on the graph attention mechanism to obtain an attention encoded dynamic coupling network;

[0029] S22, the key mechanical transmission path is extracted based on the attention encoded dynamic coupling network to obtain a key mechanical transmission path set, wherein each path contains a node feature sequence, an edge feature sequence and a time sequence feature sequence;

[0030] S23, multi-path feature fusion is performed based on the key mechanical transmission path set to obtain a path reinforcement feature vector.

[0031] Further, in step S22, the dynamic coupling network based on the attention encoding extracts the key mechanical transmission path to obtain a key mechanical transmission path set, including:

[0032] S221, the node and edge in the dynamic coupling network are calculated by using the graph attention network to generate the node-level attention coefficient and the edge-level attention coefficient;

[0033] S222, based on the node-level attention coefficient and the edge-level attention coefficient, a deep search algorithm is used to identify all potential mechanical transmission paths, and each mechanical transmission path is sorted according to the cumulative attention weight;

[0034] S223, combined with the anatomical prior knowledge, the sorted mechanical transmission path is screened and optimized to remove the path that does not conform to the biomechanical principle, and the key mechanical transmission path is reserved to form the key mechanical transmission path set.

[0035] Further, in step S3, the multi-dimensional index includes at least one of the biomechanical parameters covering the Cobb angle and the trunk rotation angle, the motor function parameters covering the gait coefficient of variation and the muscle activation, and the mechanical transmission path characteristic parameters covering the key path weight and the energy transmission efficiency.

[0036] Further, in step S4, based on the multi-dimensional risk index, threshold comparison and dynamic risk stratification are used for grading early warning decision to realize early screening of the spinal curvature abnormality, including:

[0037] S41, the multi-dimensional risk index is standardized and quantified, and compared with a preset medical threshold, and the abnormal index exceeding the threshold range is marked according to the comparison result;

[0038] S42, based on the number of abnormal indexes, the severity of index value, and the combination mode of indexes, a dynamic risk stratification model is constructed;

[0039] S43, based on the dynamic risk stratification model, a preset risk grading threshold and a differential intervention strategy are used for grading early warning decision to realize early screening of the spinal curvature abnormality.

[0040] In a second aspect, the present application discloses a spinal curvature abnormality early screening system based on multi-modal data fusion, which comprises a space-time feature fusion and three-dimensional biomechanical modeling module, a key mechanical transmission path weighting calculation module, a multi-modal risk quantification modeling module, and a grading early warning decision module, wherein:

[0041] The spatiotemporal feature fusion and three-dimensional biomechanical modeling module is used for fusing spatiotemporal sequence features and three-dimensional biomechanical modeling based on the obtained multi-modal data, and obtaining a spine-pelvis-lower limb dynamic coupling network.

[0042] The key mechanical transmission path weighting calculation module is used for weighting calculation of a key mechanical transmission path based on the dynamic coupling network through a graph attention mechanism, and obtaining a path reinforcement feature vector.

[0043] The multi-modal risk quantification modeling module is used for multi-modal risk quantification modeling based on the path reinforcement feature vector, and obtaining a multi-dimensional risk index.

[0044] The hierarchical early warning decision module is used for hierarchical early warning decision making based on the multi-dimensional risk index through threshold comparison and dynamic risk stratification, so as to realize early screening of spinal curvature abnormalities.

[0045] The present application has the advantages that a spine-pelvis-lower limb dynamic coupling network is constructed based on multi-modal data, a key mechanical transmission path is quantified by using a graph attention mechanism, three-dimensional biomechanical modeling and risk stratification decision making are combined, early screening of spinal curvature abnormalities from morphology to function is realized, and the diagnostic accuracy is improved. BRIEF DESCRIPTION OF DRAWINGS

[0046] Figure 1 A flowchart of a spinal curvature abnormality early screening method based on multi-modal data fusion disclosed by the present application is shown in the figure.

[0047] Figure 2 A structural diagram of a spinal curvature abnormality early screening system based on multi-modal data fusion disclosed by the present application is shown in the figure. DETAILED DESCRIPTION

[0048] The principles and characteristics of the present application are described below in combination with the drawings, and the examples are only used to explain the present application and not to limit the scope of the present application.

[0049] As shown in the figure, a spinal curvature abnormality early screening method based on multi-modal data fusion is characterized in that the method comprises: Figure 1

[0050] Step S1, fusing spatiotemporal sequence features and three-dimensional biomechanical modeling based on the obtained multi-modal data, and obtaining a spine-pelvis-lower limb dynamic coupling network.

[0051] Step S2, weighting calculation of a key mechanical transmission path based on the dynamic coupling network through a graph attention mechanism, and obtaining a path reinforcement feature vector.

[0052] ​Step S3, multi-modal risk quantification modeling based on the path reinforcement feature vector is performed to obtain a multi-dimensional risk index.

[0053] Step S4, based on the multi-dimensional risk index, threshold comparison and dynamic risk stratification are used for graded early warning decision-making to realize early screening of spinal curvature abnormalities.

[0054] As can be seen from the above, the spinal curvature abnormality early screening method based on multi-modal data fusion disclosed in the present application constructs a spine-pelvis-lower limb dynamic coupling network based on multi-modal data, quantifies key mechanical transmission paths using a graph attention mechanism, combines three-dimensional biomechanical modeling and risk stratification decision-making, realizes early screening of spinal curvature abnormalities from morphology to function, and improves diagnostic accuracy.

[0055] In one of the embodiments, in step S1, the multi-modal data includes multi-channel surface electromyography data collected based on a back electrode array, foot region pressure data collected based on a foot pressure sensor, and spinal motion trajectory data collected based on an inertial measurement unit.

[0056] In one of the embodiments, in step S1, the fusion of spatio-temporal sequence features based on the obtained multi-modal data and three-dimensional biomechanical modeling are performed to obtain a spine-pelvis-lower limb dynamic coupling network, which includes:

[0057] Step S11, data preprocessing and spatio-temporal alignment are performed on the obtained multi-modal data to obtain a sEMG signal matrix after spatio-temporal calibration, a foot pressure distribution heat map, and a filtered spinal motion trajectory.

[0058] Specifically, the data preprocessing here includes filtering and denoising, rectification and smoothing, signal segmentation, normalization processing, heat map generation, trajectory smoothing, and coordinate system unification, and the present application is not limited to the specific embodiments. The spatio-temporal alignment ensures the consistency of multi-modal data in time and space through time synchronization and space registration, and provides a basis for subsequent multi-modal data fusion and three-dimensional biomechanical modeling.

[0059] Step S12, muscle synergistic contraction parameters are extracted from the sEMG signal matrix, gait asymmetry parameters are extracted based on the foot pressure distribution heat map, and spinal motion harmonic components are extracted from the filtered spinal motion trajectory.

[0060] Specifically, the muscle synergistic contraction parameters are extracted from the sEMG signal matrix by a non-negative matrix factorization algorithm. The gait asymmetry parameters are quantified by synthesizing the gait cycle asymmetry index, the double lower limb load difference coefficient, and the center of pressure trajectory time-frequency characteristics. The gait cycle asymmetry index combines the phase difference and the pressure integral difference to evaluate the timing offset and load difference in the gait cycle. The double lower limb load difference coefficient is obtained by the pressure peak difference and the load change rate to reflect the static and dynamic asymmetry of the double lower limb load. The COP time-frequency characteristics are evaluated by wavelet packet decomposition to assess the balance control ability. The spine motion harmonic component is extracted by a Hilbert-Huang transform algorithm. The transform specifically adaptively decomposes the spine motion trajectory into multiple intrinsic mode functions by empirical mode decomposition, and then calculates the instantaneous frequency and amplitude by Hilbert transform. Finally, the main harmonic component is extracted and the periodicity of the spine motion is quantified.

[0061] In step S13, based on the muscle synergistic contraction parameters, the gait asymmetry parameters, and the spine motion harmonic component, a multi-scale spatio-temporal graph convolution network is used to calculate a cross-modal spatio-temporal feature tensor and an anatomic-motion joint atlas.

[0062] Specifically, the multi-scale spatio-temporal graph convolution network adopts a hierarchical architecture, captures the spatio-temporal features of muscle synergistic contraction, gait asymmetry, and spine motion by using multi-scale convolution kernels, and realizes cross-modal feature fusion and graph structure optimization by using a graph attention mechanism. The input layer of the network is responsible for integrating the muscle synergistic contraction parameters, the gait asymmetry parameters, and the spine motion harmonic component. After extracting local and global features by the multi-scale spatio-temporal graph convolution layer, the modal weights are dynamically allocated by the self-attention mechanism to perform feature splicing and weighted summation. Further, the present application also combines anatomical knowledge (such as Fryette mechanism, Gracovetsky spine engine theory, and joint kinematic coupling rules) to constrain and optimize the graph structure, and finally generates a cross-modal feature tensor and a hierarchical anatomic-motion joint atlas containing spatio-temporal dependence and anatomic-functional association.

[0063] In step S14, based on the cross-modal spatio-temporal feature tensor and the anatomic-motion joint atlas, a three-dimensional biomechanical modeling and dynamic coupling network are constructed to obtain a spine-pelvis-lower limb dynamic coupling network.

[0064] Specifically, combined with finite element analysis and multi-body dynamics, the trans-modal feature tensor is used to drive muscle force simulation and joint constraint definition, and the anatomic-motion combined atlas is fused to initialize the bone, ligament and muscle path parameters, and finally a three-dimensional biomechanical model is constructed through simulation simulation. Further need to explain is that the above-mentioned dynamic coupling network construction is mainly based on the nodes (such as vertebrae, pelvis, femur) and dynamic edges (such as muscle activation, joint contact force) predefined by the atlas, and the edge weight is updated by the trans-modal feature tensor to reflect the real-time coupling relationship; finally, Fryette mechanism, Gracovetsky spine engine theory and muscle fascia chain transmission model are integrated to form a dynamic coupling network of spine-pelvis-lower limb containing neuromuscular control and biomechanical constraints.

[0065] In one of the embodiments, in step S12, the gait asymmetry parameters include gait cycle asymmetry index, bilateral lower limb load difference coefficient, and time-frequency features of pressure center trajectory, wherein:

[0066] The gait cycle asymmetry index is calculated by the following formula:

[0067] ;

[0068] Wherein, N represents the total number of gait cycles, represents the left-right foot ground contact phase difference, represents the complete gait cycle phase, represents the left foot pressure integral, represents the right foot pressure integral.

[0069] The bilateral lower limb load difference coefficient is calculated by the following formula:

[0070] ;

[0071] Wherein, T represents the length of the time window, , represents the left foot pressure peak, represents the right foot pressure peak.

[0072] In one of the embodiments, in step S13, based on the muscle synergistic contraction parameters, gait asymmetry parameters, and spinal motion harmonic components, the trans-modal spatiotemporal feature tensor and the anatomic-motion combined atlas are calculated by the multi-scale spatiotemporal graph convolution network, including:

[0073] In step S131, different scales of spatiotemporal features are extracted from muscle synergistic contraction parameters, gait asymmetry parameters and spinal motion harmonic components by multi-scale spatiotemporal graph convolution layer.

[0074] Specifically, the multi-scale spatio-temporal graph convolution layer adopts three sets of parallel convolution kernels (3x3, 5x5, 7x7) to capture local muscle coordination details, periodic gait asymmetry patterns, and macroscopic spinal motion trends, respectively, and finally outputs a multi-scale spatio-temporal feature tensor containing spatio-temporal dependence and anatomical-functional correlation, including muscle coordination contraction features, gait asymmetry features, and spinal motion features.

[0075] In step S132, the extracted different modal features are subjected to feature splicing and weighted summation based on a graph attention mechanism to obtain a fused cross-modal spatio-temporal feature tensor.

[0076] Specifically, the present application dynamically calculates the weights of each modal feature through a self-attention mechanism, and after assigning each weight to each modal feature, the corresponding weighted feature is obtained. Then, splicing each weighted feature can obtain a cross-modal spatio-temporal feature tensor.

[0077] In step S133, the graph structure information of the anatomical-movement joint atlas is determined based on anatomical structure-movement function correlation and multi-modal data spatio-temporal dependence.

[0078] Specifically, the present application will construct an initial graph structure based on an anatomical knowledge graph and a movement function graph, combined with anatomical structure-movement function correlation. Then, the spatio-temporal dependence of the multi-modal data is fused into the initial graph structure through a conditional random field, and the edge weights are dynamically optimized based on the parameter learning and potential function optimization mechanism of the conditional random field, and combined with the penalty term of anatomical constraint, to finally output the graph structure information of the anatomical-movement joint atlas containing anatomical-functional correlation and spatio-temporal dependence.

[0079] In step S134, the graph structure information is input into the multi-scale spatio-temporal graph convolution network, and the graph structure is optimized based on the graph attention mechanism in the network and combined with anatomical knowledge to obtain the anatomical-movement joint atlas.

[0080] Specifically, the present application inputs the graph structure information into the multi-scale spatio-temporal graph convolution network, and captures local anatomical details, periodic motion patterns, and macroscopic biomechanical trends through three sets of parallel convolution kernels (3x3, 5x5, 7x7). At the same time, the edge weights are dynamically adjusted by using the graph attention network, and the graph structure is iteratively optimized combined with the penalty term of anatomical constraint. Finally, the multi-scale features are fused through a self-attention mechanism to output the anatomical-movement joint atlas containing fine anatomical structure, dynamic movement function, and spatio-temporal dependence.

[0081] In one embodiment, in step S2, the path reinforcement feature vector is obtained by the dynamic coupling network based on the graph attention mechanism for weighted calculation of the key mechanical transmission path, including:

[0082] Step S21, based on the graph attention mechanism, the dynamic coupling network is encoded in space-time features to obtain the attention encoded dynamic coupling network.

[0083] Specifically, the node features (such as bone position / speed) of the dynamic coupling network, the edge features (such as muscle force, joint stiffness), and the time sequence information are taken as input data, and the node features are mapped to the time attention space and the spatial attention space through the learnable parameter matrix (W t , W s ) respectively. Then, based on the multi-modal feature fusion and biomechanical constraints, the time dimension attention coefficient α t and the spatial dimension attention coefficient α s are calculated by the following formula:

[0084] α t = softmax(LeakyReLU(a t ^T [W t h i W t h j PE(t)]+ B t (θ joint )));

[0085] α s = softmax(LeakyReLU(a s ^T [W s h i W s h j C anatomy ]+ B s (F muscle )));

[0086] Wherein, PE(t) represents the time sequence phase encoding, T represents the gait cycle time length, h i and h j represent the feature vectors of node i and node j, a t , a s represent the learnable parameter vector, C anatomy represents the anatomical constraint matrix, B t (θ joint )= W bθ θ joint , B s (F muscle )= W bF Fmuscle , θ joint denotes the joint angle vector, F muscle denotes the muscle force vector, W bθ and W bF denote the learnable weight matrix.

[0087] Based on the above formula, it needs to be explained that the timing phase encoding PE(t) in the formula maps the timing information in the gait cycle to a high-dimensional space, enhancing the model's perception ability of periodic motion patterns. The introduction of the anatomical constraint matrix C anatomy ensures the physiological reasonableness of the calculation results and avoids the emergence of non-physiological mechanical transmission paths. In addition, the design of the dynamic bias term B t (θ joint ), B s (F muscle ) enables the model to adapt to the mechanical transmission characteristics under different joint angles and muscle force conditions, improving the model's generalization ability.

[0088] Finally, the time dimension attention coefficient α t and the spatial dimension attention coefficient α s are fused by element-by-element multiplication, that is, α t α s , to obtain the fused space-time attention coefficient α. Through the fused space-time attention coefficient α, feature weighting aggregation and residual update processing are performed to obtain the attention encoded dynamic coupling network. For each node i, the application will take the fused attention coefficient α as the weight, and based on the features of the neighbor node j, weighted summation is performed (for reference, the formula is as follows: , is a learnable feature transformation matrix), to obtain the aggregated features . The aggregated features are connected in residual connection with the original node features (residual connection design can alleviate gradient disappearance and preserve original dynamic features), and the LeakyReLU activation function is used for non-linear expression ability enhancement (for reference, the formula is as follows: ), to realize the update of the node i features.

[0089] Further, the application will also update the original edge weight based on the fused attention coefficient α, which can be specifically referred to in the following formula: (wherein, denotes the anatomical prior weight), wherein the anatomical prior weight The introduction of the attention mechanism further makes the model more inclined to select a mechanical transmission path that conforms to anatomical rules during optimization. Finally, based on the updated node features and edge weights a new network is constructed, i.e. a dynamic coupling network that considers both the periodicity of the gait in the time dimension and the mechanical transmission pattern in the spatial dimension, which provides a high-quality feature representation for subsequent key mechanical transmission path search.

[0090] Step S22, extracting a key mechanical transmission path based on the attention- encoded dynamic coupling network to obtain a key mechanical transmission path set, wherein each path includes a node feature sequence, an edge feature sequence, and a time sequence feature sequence.

[0091] Step S23, performing multi-path feature fusion based on the key mechanical transmission path set to obtain a path reinforcement feature vector.

[0092] Specifically, the present application fuses the features of each path in the key mechanical transmission path set through attention-weighted summation and anatomical constraint-guided graph aggregation operations. First, the attention mechanism is used to calculate the weight of each path, which reflects the importance of the path in the overall mechanical transmission. Second, anatomical prior knowledge is combined to guide and constrain the path features (i.e., the path features are weighted and modified by an anatomical constraint matrix, and the feature values are clipped by introducing a physiological range limit), to ensure that the fused features conform to the principles of biomechanics. Finally, through graph aggregation operations, the weighted path features are integrated into a unified path reinforcement feature vector, which not only retains the key information of each path but also reflects the overall mechanical transmission pattern.

[0093] In one embodiment, in step S22, the key mechanical transmission path is extracted based on the attention-encoded dynamic coupling network to obtain a key mechanical transmission path set, which includes:

[0094] Step S221, using a graph attention network to calculate the attention weights of the nodes and edges in the dynamic coupling network to generate node-level attention coefficients and edge-level attention coefficients.

[0095] Specifically, the graph attention network calculates the attention weights of the neighbor nodes (i.e., node-level attention coefficients) for each node and the attention weights of the transmitted forces (i.e., edge-level attention coefficients) for each edge through a self-attention mechanism.

[0096] Step S222, based on the node-level attention coefficient and the edge-level attention coefficient, a depth-first search algorithm is used to identify all potential mechanical transmission paths, and each mechanical transmission path is sorted according to the cumulative attention weight.

[0097] It should be noted that the cumulative attention weight refers to the product of all edge-level attention coefficients on the path, which quantitatively reflects the comprehensive importance of the path in the mechanical transmission network.

[0098] Step S223, in combination with anatomical prior knowledge, the sorted mechanical transmission paths are screened and optimized to remove paths that do not conform to the principles of biomechanics, and to retain key mechanical transmission paths to form a set of key mechanical transmission paths.

[0099] Specifically, in combination with anatomical prior knowledge, the sorted paths are screened in two stages. First, non-physiological paths (such as knee joint hyperextension paths) are filtered out through an anatomical constraint matrix. Second, a path validity evaluation function is used to evaluate the validity of each remaining path, and based on a validity threshold comparison, the optimal path is selected as the key mechanical transmission path to ensure that the final set obtained not only conforms to the principles of biomechanics, but also covers the main mechanical transmission modes.

[0100] In one embodiment, the path validity evaluation function is as follows:

[0101] ;

[0102] wherein, represents each edge on the path, represents the actual mechanical transmission amount on edge , represents the maximum mechanical transmission amount on edge , represents the energy consumption on edge , represents the maximum energy consumption allowed on edge , , represents the weight coefficient.

[0103] It should be noted that the evaluation function considers the mechanical transmission efficiency and energy consumption by weighted summation, so that the evaluation result can quantitatively reflect the comprehensive performance of the path in mechanical transmission, thereby guiding the algorithm to preferentially select efficient transmission paths.

[0104] Further, in step S3, the multi-dimensional index includes at least one of biomechanical parameters covering Cobb angle and torso rotation angle, motor function parameters covering gait coefficient of variation and muscle activation, and mechanical transmission path characteristic parameters covering key path weight and energy transmission efficiency.

[0105] Further, in step S4, based on the multi-dimensional risk index, threshold comparison and dynamic risk stratification are used for grading early warning decision-making to realize early screening of spinal curvature abnormalities, including:

[0106] In step S41, the multi-dimensional risk index is standardized and quantified, and compared with a preset medical threshold. Abnormal indexes exceeding the threshold range are marked according to the comparison result.

[0107] In step S42, a dynamic risk stratification model is constructed based on the number of abnormal indexes, the severity of index values, and the combination mode of indexes.

[0108] Wherein, the index value refers to the degree of deviation of the specific value of each abnormal index from the normal range, and the combination mode of indexes refers to the synergistic effect of different abnormal indexes in the aspects of biomechanics, space-time dynamics and pathological mechanism. For example, when Cobb angle increases and torso rotation angle increases, it may accelerate the progression of spinal deformity by exacerbating vertebral rotational collapse; when sedentary time is >8 hours / day and unilateral muscle activation is abnormal, it may form a vicious cycle through muscle fatigue and posture compensation, and exacerbate scoliosis.

[0109] Specifically, the dynamic risk stratification model comprehensively evaluates the number of abnormal indexes, the severity of each index value, and the combination mode of indexes, identifies complex nonlinear relationships through machine learning algorithms, and finally divides the risk into low, medium and high levels. Based on the core biomarkers of individual growth and development stage (such as bone age, height growth rate), and the relevance of the progression of spinal curvature abnormalities, the threshold is dynamically adjusted to adapt to the progress risk difference of individual growth and development stage.

[0110] In step S43, based on the dynamic risk stratification model, a grading early warning decision is made through a preset risk grading threshold and a differentiated intervention strategy to realize early screening of spinal curvature abnormalities.

[0111] Specifically, based on the dynamic risk stratification model, by setting low, medium and high risk classification thresholds and matching differentiated intervention strategies (such as low risk corresponding to follow-up every 6 months plus posture education; medium risk corresponding to follow-up every 3 months plus brace treatment and core muscle group training; high risk corresponding to multidisciplinary consultation and surgical evaluation), while dynamically adjusting the threshold according to the individual growth stage (such as reducing the high-risk threshold of Cobb angle from 25° to 20° during the growth peak of adolescence), the precise classification and early warning of spinal curvature abnormalities and early intervention are ultimately achieved.

[0112] Please refer to Figure 2 The application discloses a kind of based on multi-modal data fusion's spinal curvature abnormality early screening system, the system includes space-time feature fusion and three-dimensional biomechanical modeling module, key mechanics transmission path weighted calculation module, multi-modal risk quantization modeling module, and classification warning decision module, wherein:

[0113] The space-time feature fusion and three-dimensional biomechanical modeling module are used to fuse the space-time sequence features based on the obtained multi-modal data, and to model three-dimensional biomechanics, to obtain the dynamic coupling network of spine-pelvis-lower limb.

[0114] The key mechanics transmission path weighted calculation module is used to calculate the weighted calculation of key mechanics transmission path based on the dynamic coupling network through graph attention mechanism, to obtain path reinforcement feature vector.

[0115] The multi-modal risk quantization modeling module is used to perform multi-modal risk quantization modeling based on the path reinforcement feature vector, to obtain multi-dimensional risk indicators.

[0116] The classification warning decision module is used to perform classification warning decision based on the multi-dimensional risk indicators through threshold comparison and dynamic risk stratification, to realize the early screening of spinal curvature abnormalities.

[0117] In one embodiment, the above modules are also used to implement the method of early screening of spinal curvature abnormalities based on multi-modal data fusion as described in any of the preceding method embodiments, which is not limited by the present application.

[0118] As can be seen from the above, the application discloses a kind of based on multi-modal data fusion's spinal curvature abnormality early screening system, constructs the dynamic coupling network of spine-pelvis-lower limb based on multi-modal data, quantifies key mechanics transmission path using graph attention mechanism, realizes the early screening of spinal curvature abnormalities from morphology to function by combining three-dimensional biomechanical modeling and risk stratification decision, and improves the diagnostic accuracy.

[0119] The above merely describes preferred embodiments of the present application and is not used to limit the present application, and any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for early screening of spinal curvature abnormalities based on multimodal data fusion, characterized in that: The method comprises: S1. Based on the acquired multimodal data, the spatiotemporal sequence features are integrated and three-dimensional biomechanical modeling is performed to obtain a dynamic coupling network of the spine, pelvis and lower limbs. S2. Based on the dynamic coupling network, a graph attention mechanism is used to perform weighted calculation on key mechanical transmission paths to obtain a path reinforcement feature vector. S3. Perform multimodal risk quantification modeling based on the path reinforcement feature vector to obtain multidimensional risk indicators; S4. Based on the multi-dimensional risk indicators, threshold comparison and dynamic risk stratification are used to make graded warning decisions to achieve early screening of spinal curvature abnormalities.

2. The method according to claim 1, characterized in that In step S1, the multimodal data includes multi-channel surface electromyography data collected based on the back electrode array, pressure data of various areas of the sole of the foot collected based on the sole pressure sensor, and spinal motion trajectory data collected based on the inertial measurement unit.

3. The method according to claim 2, characterized in that In step S1, the spatiotemporal sequence feature fusion and three-dimensional biomechanical modeling based on the acquired multimodal data are performed to obtain a spine-pelvis-lower limb dynamic coupling network, including: S11, performing data preprocessing and spatiotemporal alignment on the acquired multimodal data to obtain a spatiotemporally calibrated sEMG signal matrix, a plantar pressure distribution heat map, and a filtered spinal motion trajectory; S12, extracting muscle co-contraction parameters from the sEMG signal matrix, extracting gait asymmetry parameters based on the plantar pressure distribution thermogram, and extracting spinal motion harmonic components from the filtered spinal motion trajectory; S13. Based on the muscle co-contraction parameters, gait asymmetry parameters, and spinal motion harmonic components, a cross-modal spatiotemporal feature tensor and an anatomical-motion joint atlas are calculated using a multi-scale spatiotemporal graph convolutional network; S14. Based on the cross-modal spatiotemporal feature tensor and the anatomical-motion joint atlas, three-dimensional biomechanical modeling and dynamic coupling network construction are performed to obtain a spine-pelvis-lower limb dynamic coupling network.

4. The method according to claim 3, characterized in that In step S12, the gait asymmetry parameters include the gait cycle asymmetry index, the load difference coefficient of both lower limbs, and the time-frequency characteristics of the pressure center trajectory, wherein: The gait cycle asymmetry index is calculated by the following formula: ； Where N represents the total number of gait cycles, Indicates the phase difference between the left and right feet touching the ground. represents the phase of the complete gait cycle, represents the left foot pressure integral, represents the right foot pressure integral; The lower limb load difference coefficient is calculated by the following formula: ； Where T represents the time window length, 、, Indicates the peak pressure of the left foot, Indicates peak right foot pressure.

5. The method according to claim 3, characterized in that In step S13, based on the muscle co-contraction parameters, gait asymmetry parameters, and spinal motion harmonic components, a cross-modal spatiotemporal feature tensor and an anatomical-motion joint atlas are calculated by a multi-scale spatiotemporal graph convolutional network, including: S131. Through the multi-scale spatiotemporal graph convolution layer, spatiotemporal features of different scales are extracted from muscle co-contraction parameters, gait asymmetry parameters and spinal motion harmonic components; S132. Perform feature concatenation and weighted summation based on the graph attention mechanism on the extracted features of different modalities to obtain a fused cross-modal spatiotemporal feature tensor; S133, determining graph structure information of the anatomical-motor joint atlas based on the anatomical structure-motor function association and the spatiotemporal dependency of the multimodal data; S134. Input the graph structure information into a multi-scale spatiotemporal graph convolutional network, optimize the graph structure based on the graph attention mechanism in the network and in combination with anatomical knowledge to obtain an anatomical-motion joint atlas.

6. The method according to claim 1, characterized in that In step S2, the weighted calculation of the key mechanical transmission paths based on the dynamic coupling network is performed through the graph attention mechanism to obtain the path reinforcement feature vector, including: S21. Encoding the spatiotemporal features of the dynamic coupling network based on a graph attention mechanism to obtain an attention-encoded dynamic coupling network; S22. Extracting key mechanical transmission paths based on the attention-encoded dynamic coupling network to obtain a set of key mechanical transmission paths, wherein each path includes a node feature sequence, an edge feature sequence, and a time series feature sequence; S23. Perform multi-path feature fusion based on the key mechanical transfer path set to obtain a path enhancement feature vector.

7. The method according to claim 6, characterized in that In step S22, extracting key mechanical transmission paths based on the attention-encoded dynamic coupling network to obtain a set of key mechanical transmission paths includes: S221. Calculate attention weights for nodes and edges in the dynamic coupling network using a graph attention network to generate node-level attention coefficients and edge-level attention coefficients. S222. Based on the node-level attention coefficient and the edge-level attention coefficient, a depth-first search algorithm is used to identify all potential mechanical transmission paths, and each mechanical transmission path is sorted according to the cumulative attention weight; S223. Combined with anatomical prior knowledge, the sorted mechanical transmission pathways are screened and optimized to remove pathways that do not conform to the principles of biomechanics, retain key mechanical transmission pathways, and form a set of key mechanical transmission pathways.

8. The method according to claim 1, characterized in that In step S3, the multidimensional indicators include at least one of biomechanical parameters covering Cobb angle and trunk rotation angle, motor function parameters covering gait variation coefficient and muscle activation, and mechanical transfer path characteristic parameters covering critical path weight and energy transfer efficiency.

9. The method according to claim 1, characterized in that In step S4, based on the multi-dimensional risk indicators, threshold comparison and dynamic risk stratification are used to make graded warning decisions to achieve early screening of spinal curvature abnormalities, including: S41, performing standardized quantification processing on the multi-dimensional risk indicators, comparing them with preset medical thresholds, and marking abnormal indicators that exceed the threshold range based on the comparison results; S42. Construct a dynamic risk stratification model based on the number of abnormal indicators, the severity of the indicator values, and the combination pattern between indicators; S43. Based on the dynamic risk stratification model, graded warning decisions are made through preset risk grading thresholds and differentiated intervention strategies to achieve early screening of spinal curvature abnormalities.

10. An early screening system for spinal curvature abnormality based on multimodal data fusion, characterized in that: The system includes a spatiotemporal feature fusion and three-dimensional biomechanical modeling module, a key mechanical transmission path weighted calculation module, a multimodal risk quantification modeling module, and a hierarchical early warning decision module, wherein: The spatiotemporal feature fusion and three-dimensional biomechanical modeling module is used to fuse spatiotemporal sequence features and perform three-dimensional biomechanical modeling based on the acquired multimodal data to obtain a dynamic coupling network of the spine, pelvis and lower limbs; The key mechanical transmission path weighted calculation module is used to perform weighted calculation of the key mechanical transmission path based on the dynamic coupling network through the graph attention mechanism to obtain a path reinforcement feature vector; The multimodal risk quantification modeling module is used to perform multimodal risk quantification modeling based on the path reinforcement feature vector to obtain a multi-dimensional risk indicator; The graded warning decision module is used to make graded warning decisions based on the multi-dimensional risk indicators through threshold comparison and dynamic risk stratification to achieve early screening of spinal curvature abnormalities.

Images