Adolescent Scoliosis Intelligent Monitoring System Based on Multimodal Sensing

Abstract

This invention relates to the field of health management technology, specifically to an intelligent monitoring system for adolescent scoliosis based on multimodal sensing. It includes: a multimodal data acquisition unit; a data preprocessing unit, which standardizes the multi-source raw data output by the multimodal data acquisition unit, eliminates environmental interference and motion artifacts based on an adaptive Kalman filter algorithm, and extracts quantifiable physical feature parameters; an intelligent analysis unit; and an early warning execution unit. This invention combines an optical image sensor and an inertial sensor through a multimodal data acquisition unit, and achieves precise matching and calibration of static morphological parameters and dynamic motion data through a spatiotemporal correlation algorithm in the data correlation module. This solves the problem of insufficient complementarity caused by the lack of spatiotemporal correlation in existing multimodal data, and can form a complementary dataset containing both static and dynamic features.

Description

Technical Field

[0001] This invention relates to the field of health management technology, and more specifically, to an intelligent monitoring system for adolescent scoliosis based on multimodal sensing. Background Technology

[0002] As a progressive three-dimensional spinal deformity, early and accurate monitoring is crucial to preventing the condition from worsening. Current monitoring technologies often rely on single types of data or lack specific optimization for spinal characteristics during data processing, leading to the neglect of the correlation between static morphology and dynamic movement characteristics, thus limiting the comprehensiveness and accuracy of monitoring.

[0003] In existing technologies, for example, Chinese patent CN202411881149.5 discloses a scoliosis monitoring and early warning system for children. This system acquires multimodal data through posture sensors, accelerometers, and image acquisition devices, extracts features using convolutional neural networks and long short-term memory networks, utilizes a hidden Markov model to identify behavioral contexts to adjust early warning thresholds, and generates personalized intervention strategies. Another example is Chinese patent CN202311661867.7, which discloses a scoliosis detection method and system based on muscle current signals. This method determines the presence of scoliosis by analyzing the deviation of muscle currents on both sides of the spine, achieving non-invasive monitoring.

[0004] While the aforementioned technical solutions possess corresponding design advantages, they also have significant limitations: CN202411881149.5, although employing multimodal data acquisition, lacks a precise spatiotemporal correlation mechanism between static morphological parameters and dynamic motion data. It fails to match and calibrate the spatial coordinates of optical images with the physical positions of inertial sensors for spinal anatomical locations, and also fails to align the acquisition times of the two types of data based on a unified time reference. This results in a blurred mapping relationship between static morphological features and dynamic motion features, hindering the full utilization of the complementarity of multi-source data and affecting the accuracy of feature extraction. CN202311661867.7 relies solely on a single data source—myomuscular electrical signals—and cannot capture the three-dimensional structural morphology of the spine in the coronal and sagittal planes, as well as the dynamic motion parameters of the vertebrae. Its data dimension is limited to neuromuscular electrical signals, making it difficult to comprehensively reflect the three-dimensional deformity characteristics and kinematic laws of scoliosis, and its ability to identify subtle morphological changes in early scoliosis is insufficient. Therefore, we propose a multimodal sensing-based intelligent monitoring system for adolescent scoliosis. Summary of the Invention

[0005] The purpose of this invention is to provide an intelligent monitoring system for adolescent scoliosis based on multimodal sensing, in order to solve the problems mentioned in the background art, such as insufficient complementarity due to the lack of accurate spatiotemporal correlation of multimodal data, the inability of noise suppression mechanisms to adapt to the time-varying characteristics of spinal motion, the lack of integration of spinal biomechanical features in the analysis model and the fixed weight allocation of key pathological features, and the difficulty of a single data source to comprehensively reflect the static morphology and dynamic motion characteristics of the spine.

[0006] To address the aforementioned technical problems, the present invention aims to provide an intelligent monitoring system for adolescent scoliosis based on multimodal sensing, comprising:

[0007] A multimodal data acquisition unit is used to capture multidimensional features of spinal structural abnormalities. By combining optical image sensors and inertial sensors, it synchronously acquires static morphological parameters and dynamic motion data of the spine, forming a complementary feature dataset composed of multi-source raw data.

[0008] The data preprocessing unit is used to standardize the multi-source raw data output by the multimodal data acquisition unit, eliminate environmental interference and motion artifacts based on the adaptive Kalman filter algorithm, and extract quantifiable physical feature parameters.

[0009] The intelligent analysis unit generates a scoliosis risk level based on the physical feature parameters output by the data preprocessing unit. It constructs a machine learning model that integrates spinal biomechanical features to perform cross-dimensional fusion analysis of the physical feature parameters and strengthens the weight ratio of key pathological features in the risk level calculation through an adaptive weight allocation mechanism.

[0010] The early warning execution unit is used to trigger an alarm mechanism when the risk level of scoliosis output by the intelligent analysis unit exceeds a preset threshold. It drives the audible and visual alarm device through an electrical signal and simultaneously pushes electronic data containing characteristic abnormal parameters.

[0011] The data storage unit is used to store multi-source raw data output by the multimodal data acquisition unit, physical characteristic parameters output by the data preprocessing unit, and scoliosis risk level output by the intelligent analysis unit. The data is stored in association using a general-purpose storage medium.

[0012] As a further improvement to this technical solution, the multimodal data acquisition unit includes an optical image sensing module and an inertial sensing module, wherein:

[0013] The optical image sensing module is a combination of a depth camera and a visible light camera. The depth camera is used to acquire three-dimensional structural information of the spine in the coronal and sagittal planes to extract static morphological parameters (such as scoliosis angle, pelvic tilt angle, scapular symmetry, and thoracic asymmetry index). The visible light camera is used to simultaneously acquire surface images of the corresponding area to assist in locating anatomical landmarks (such as spinous processes, iliac crests, and acromions). The depth camera and the visible light camera achieve coordinated alignment of acquisition timing through trigger signals.

[0014] The inertial sensing module consists of at least three inertial sensors that integrate triaxial acceleration and triaxial angular velocity detection functions. These sensors are respectively installed at the anatomical locations of the T2, T8, and L4 vertebrae on the adolescent's body surface. They are used to collect acceleration and angular velocity change data of the spine during dynamic activities (such as walking, bending over, turning, standing and turning, sitting and standing up). The data acquisition of each sensor is started synchronously through unified timing control.

[0015] As a further improvement to this technical solution, the multimodal data acquisition unit also includes a data association module. The data association module is connected to the optical image sensing module and the inertial sensing module, respectively, and is used to receive the static morphological parameters and corresponding body surface image data output by the optical image sensing module, as well as the acceleration change data and angular velocity change data output by the inertial sensing module. The data association module matches and calibrates the spatial coordinates of the static morphological parameters with the anatomical position of the inertial sensor through a preset spatiotemporal association algorithm. At the same time, it aligns the acquisition time of the two types of data based on a unified time reference, establishes a mapping relationship between static morphological features and dynamic motion features, and finally generates structured multi-source raw data containing time stamps, spatial coordinates and motion parameters, forming a complementary feature dataset.

[0016] As a further improvement to this technical solution, the data preprocessing unit includes a data receiving and classification module, a standardization processing module, a filtering processing module, and a feature extraction module, wherein:

[0017] The data receiving and classification module is connected to the output end of the multimodal data acquisition unit, and is used to receive raw data from multiple sources and classify and cache it according to sensor type to form a subset of optical image data and a subset of inertial motion data.

[0018] The standardization processing module is connected to the data receiving and classification module. It is used to integrate the optical image data subset and the inertial motion data subset into mixed data by aligning them according to the timestamp. It also performs scale normalization processing on the optical three-dimensional coordinate parameters in the mixed data and dimension unification processing on the acceleration and angular velocity parameters in the mixed data, so that the two types of parameters maintain the consistency of numerical range in the same mixed data structure.

[0019] The filtering module is connected to the standardization module and has a built-in adaptive Kalman filter algorithm for noise suppression of the standardized mixed data. It eliminates ambient light interference, sensor drift and motion artifacts by dynamically adjusting the filtering parameters.

[0020] The feature extraction module is connected to the filtering module and extracts quantifiable physical feature parameters (such as the rate of change of the physiological curvature of the spine, the deviation of the vertebral motion trajectory, and the motion symmetry index) from the filtered data based on preset feature engineering rules.

[0021] As a further improvement to this technical solution, the filtering module uses an adaptive Kalman filter algorithm to suppress noise in the standardized mixed data, including the following steps:

[0022] S230.1, Mixed Data Reception and Structure Parsing:

[0023] The system receives standardized mixed data output from the standardization processing module. The standardized mixed data includes a subset of optical parameters and a subset of inertial parameters. The subset of optical parameters consists of parameters from the subset of optical image data in the mixed data, including the scale-normalized three-dimensional coordinates of spinal feature points and the relative positions of anatomical landmarks. The subset of inertial parameters consists of parameters from the subset of inertial motion data in the mixed data, including the dimensionally unified acceleration and angular velocity of the corresponding positions of the T2, T8, and L4 vertebrae.

[0024] Furthermore, the two types of parameters are aligned according to the timestamp to form a frame data sequence. ,in For time frame indexing;

[0025] S230.2 System Status and Parameter Initialization:

[0026] The system state vector is defined based on a subset of parameters from mixed data. ,in: express transpose; express transpose; The transpose operation represents a matrix or vector;

[0027] This is a subset of optical parameters, containing normalized 3D coordinates of spinal feature points. ;in, For the first The normalized coordinates of each spinal feature point are mapped to [0, ... 1] Interval (based on the total length of the spine);

[0028] This is a subset of inertial parameters (corresponding to a subset of inertial parameters), containing normalized acceleration and angular velocity. ;in, For the first Normalized acceleration at vertebral position, For the first The normalized angular velocities of the vertebral body positions are all mapped to [-1]. 1] interval;

[0029] Initialize first frame state estimation State covariance matrix (The diagonal elements are the squared values ​​of the standardized errors of each parameter), where Represents the vector after integrating the multimodal data from the first frame; measurement noise covariance matrix. ,in Corresponding optical parameter measurement noise, Corresponding inertial parameter measurement noise; initial process noise covariance matrix (Based on noise statistical characteristics settings in static scenarios);

[0030] S230.3, State Prediction: For Time, based on Optimal estimation of state vector at time step The predicted state is calculated using the state equation, and the formula is:

[0031] ;

[0032] in: Let be the state transition matrix, and , For optical parameter submatrices (adapting to the changing characteristics of the optical parameter subset, using a rigid body motion model to represent the integrity of the spinal structure). For the inertial parameter submatrix (adapting to the changing characteristics of the inertial parameter subset, using kinematic differential relations to describe the time-varying laws of acceleration and angular velocity); For the control matrix, For associated sampling intervals; express Time based The predicted state vector at time estimate;

[0033] Simultaneous calculation of the prediction covariance matrix: ;in, express The covariance matrix of the predicted state at each time step represents the statistical characteristics of the prediction error. express The covariance matrix of the time-optimal estimate; Represents the state transition matrix The transpose of the matrix; express The process noise covariance matrix at time step;

[0034] S230.4, Residual Calculation and Noise Adjustment:

[0035] Pick Moment-mixed data frames As a measurement vector ,in These are measured values ​​of optical parameters. These are measured values ​​of inertial parameters;

[0036] Calculate residuals As evidence of measurement; among them, Represents the measurement matrix, and , This represents a sub-block of the optical parameter measurement matrix, which maps optical state parameters to optical measurement values. This represents a sub-block of the inertial parameter measurement matrix, which maps inertial state parameters to inertial measurement values.

[0037] Based on residual modulus Adjustment process noise covariance matrix ,in For adaptive adjustment coefficients; Indicates the adaptive adjustment coefficient;

[0038] S230.5, Status Update:

[0039] Calculate Kalman gain Update the optimal estimate Correcting covariance ;in, This represents the measurement noise covariance matrix, characterizing the inherent noise of the sensor. Represents the matrix inversion operation; Represents the identity matrix, with dimensions matching the state vector;

[0040] S230.6, Output of filtering results: Time-optimal estimation The denoised mixed data is output to the feature extraction module, and steps S230.3 to S230.5 are repeated until all frame data processing is completed.

[0041] As a further improvement to this technical solution, the intelligent analysis unit includes a feature selection module, a model inference module, and a level calibration module, wherein:

[0042] The feature selection module is connected to the feature extraction module of the data preprocessing unit. It is used to screen core pathological features from physical feature parameters. By calculating the biomechanical correlation between each feature and the pathological changes of scoliosis (such as the influence weight of the feature on spinal imbalance), features with significant correlation are retained to form an input feature set.

[0043] The model inference module incorporates a machine learning model that integrates spinal biomechanical features. It adopts a "feature layer fusion + decision layer fusion" architecture, which specifically includes: the feature layer couples static morphological features with dynamic motion features; the decision layer outputs a preliminary risk level based on the integrated model.

[0044] The level calibration module is based on the principle of spinal biomechanics to construct a calibration mechanism, which corrects the initial risk level and ensures that it is consistent with the pathological progression of scoliosis.

[0045] As a further improvement to this technical solution, the machine learning model of the model inference module includes a biomechanical constraint layer and a cross-dimensional fusion layer, wherein:

[0046] The biomechanical constraint layer is embedded in the spinal kinematic coupling equation: ;in, The angle of vertebral rotation. For the scoliosis angle, The coupling coefficient (determined based on spinal anatomical features). This is the error term;

[0047] The cross-dimensional fusion layer achieves the fusion of static and dynamic features through a feature mapping formula: ;in, To fuse feature vectors, This is an optical static feature vector. For inertial dynamic feature vectors, , These are the fusion weight matrices for the two types of features (determined through feature importance analysis).

[0048] As a further improvement to this technical solution, the intelligent analysis unit also includes a weight initialization module and a dynamic adjustment module, wherein:

[0049] The weight initialization module is connected to the feature selection module. Based on the biomechanical pathological effects of the core features (such as the degree of influence of the scoliosis angle on spinal stability), it assigns initial weights to each feature. The weight values ​​are positively correlated with the pathological correlation of the features.

[0050] The dynamic adjustment module is connected to the model inference module and adjusts the weights by analyzing the actual correlation changes between features and risk levels (such as the consistency between feature value fluctuations and risk level changes).

[0051] Increase the weight of a feature as the correlation between it and the risk level strengthens with the accumulation of data.

[0052] When the correlation weakens, its weight is reduced in order to dynamically strengthen the weight of key pathological features.

[0053] As a further improvement to this technical solution, the early warning execution unit includes a threshold judgment module, an audio-visual driving module, and a data push module, wherein:

[0054] The threshold judgment module is connected to the output end of the intelligent analysis unit. It is used to receive scoliosis risk level data and compare the scoliosis risk level data with the risk level threshold (set according to the pathological grading standard of scoliosis) built into the system. When the risk level exceeds the threshold, a trigger signal is output.

[0055] The sound and light driving module is connected to the threshold judgment module. After receiving the trigger signal, it is converted into a driving signal through an electrical signal amplification circuit to control the sound and light alarm device (including LED warning lights and buzzers) to work in a preset mode (such as the light flashing frequency and buzzer interval corresponding to the risk level).

[0056] The data push module is connected to the threshold judgment module and the data storage unit respectively. After receiving the trigger signal, it extracts the characteristic abnormal parameters (such as excessive side bending angle and abnormal values ​​of motion symmetry) within the corresponding detection period from the data storage unit, pushes them to the preset terminal (such as the guardian's mobile phone or the medical institution management system) through an encrypted communication protocol (such as HTTPS), and generates push logs and stores them in the data storage unit.

[0057] As a further improvement to this technical solution, the data storage unit includes a data receiving module, a hierarchical storage module, an association index module, and an access control module, wherein:

[0058] The data receiving module is used to receive multi-source raw data (including a subset of optical image data and a subset of inertial motion data), physical characteristic parameters (including the rate of change of the physiological curvature of the spine, etc.) and the risk level of scoliosis, and to attach a unique identifier (including the detection object ID, timestamp, and data type label) to each type of data.

[0059] The hierarchical storage module is connected to the data receiving module and implements hierarchical storage based on data characteristics;

[0060] Furthermore, the tiered storage is specifically as follows:

[0061] Multi-source raw data (large volume, unstructured) is stored on magnetic storage media (such as hard disk arrays).

[0062] Physical characteristics (medium capacity, structured) are stored in solid-state storage media (such as SSDs).

[0063] Scoliosis risk levels (small capacity, high access frequency) are stored in an in-memory database to achieve a balance between data storage efficiency and access speed.

[0064] The associated index module is connected to the hierarchical storage module. It establishes the association relationship between the three types of data through preset index rules (with the detection object ID as the core key and the timestamp as the auxiliary key), and generates an index table containing data storage path and association weight (the association weight between the original data and the feature parameters in the same detection period is set to the highest), which supports fast source tracing and querying of cross-type data.

[0065] The access control module is connected to the associated index module and sets access permissions based on the data sensitivity level.

[0066] Furthermore, setting access permissions based on data sensitivity levels specifically includes:

[0067] Multi-source raw data and physical characteristic parameters can only be accessed by authorized medical institutions through an encrypted interface;

[0068] The risk level of scoliosis can be queried by the subject of the test and their guardian after identity verification;

[0069] All data access operations generate logs (including access time, operation type, and access subject) and are synchronously stored in the hierarchical storage module.

[0070] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0071] 1. This invention combines an optical image sensor and an inertial sensor through a multimodal data acquisition unit, and achieves accurate matching and calibration of static morphological parameters and dynamic motion data through the spatiotemporal correlation algorithm of the data correlation module. This solves the problem of insufficient complementarity caused by the lack of spatiotemporal correlation in multimodal data in the prior art, and can form a complementary dataset containing static and dynamic features, so as to capture the multidimensional features of spinal structural abnormalities more comprehensively.

[0072] 2. The data preprocessing unit of this invention adopts an adaptive Kalman filter algorithm to dynamically adjust the filter parameters for the standardized mixed data to eliminate environmental interference and motion artifacts. This solves the problem that the noise suppression mechanism in the prior art cannot adapt to the time-varying characteristics of spinal motion, and helps to improve the reliability of the processed physical feature parameters.

[0073] 3. The intelligent analysis unit of the present invention constructs a machine learning model that integrates the biomechanical features of the spine, embeds the coupling equation between the vertebral rotation angle and the scoliosis angle, and dynamically adjusts the weight ratio of key pathological features through an adaptive weight allocation mechanism. This solves the problem that the analysis model in the prior art does not integrate biomechanical features and the weight allocation is fixed, and enables the risk level assessment to better fit the pathological progression pattern of scoliosis.

[0074] 4. Compared with monitoring technologies that rely on a single data source, this invention collects three-dimensional structural information of the spine through an optical image sensor and dynamic motion data of the vertebrae through an inertial sensor. It covers multi-dimensional features of the static morphology and dynamic motion of the spine, solves the problem that a single data source cannot fully reflect the characteristics of scoliosis, and provides richer feature evidence for the monitoring of scoliosis. Attached Figure Description

[0075] Figure 1 This is a schematic diagram of the system framework of the present invention;

[0076] The meanings of the labels in the diagram are as follows:

[0077] 100. Multimodal data acquisition unit; 110. Optical image sensing module; 120. Inertial sensing module; 130. Data association module;

[0078] 200. Data preprocessing unit; 210. Data receiving and classification module; 220. Standardization module; 230. Filtering module; 240. Feature extraction module;

[0079] 300. Intelligent Analysis Unit; 310. Feature Selection Module; 320. Model Inference Module; 330. Level Calibration Module; 340. Weight Initialization Module; 350. Dynamic Adjustment Module;

[0080] 400. Early warning execution unit; 410. Threshold judgment module; 420. Sound and light driving module; 430. Data push module;

[0081] 500. Data storage unit; 510. Data receiving module; 520. Hierarchical storage module; 530. Association index module; 540. Access control module. Detailed Implementation

[0082] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0083] like Figure 1 As shown, this embodiment provides an intelligent monitoring system for adolescent scoliosis based on multimodal sensing, including:

[0084] The multimodal data acquisition unit 100 is used to capture multidimensional features of spinal structural abnormalities. By combining optical image sensors and inertial sensors, it synchronously acquires static morphological parameters and dynamic motion data of the spine to form a complementary feature dataset composed of multi-source raw data.

[0085] In this embodiment, the multimodal data acquisition unit 100 includes an optical image sensing module 110 and an inertial sensing module 120, wherein:

[0086] The optical image sensing module 110 is a combination of a depth camera and a visible light camera. The depth camera is used to acquire three-dimensional structural information of the spine in the coronal and sagittal planes to extract static morphological parameters (such as scoliosis angle, pelvic tilt angle, scapular symmetry, and thoracic asymmetry index). The visible light camera is used to simultaneously acquire surface images of the corresponding area to assist in locating anatomical landmarks (such as spinous processes, iliac crests, and acromions). The depth camera and the visible light camera achieve coordinated alignment of acquisition timing through trigger signals.

[0087] As a further explanation of this embodiment, the depth camera and the visible light camera of the optical image sensing module 110 in this embodiment are mounted on the same adjustable frame. The frame can move in the vertical and horizontal directions to adapt to the spinal acquisition needs of teenagers of different heights and ensure that the center of the lens is directly facing the spinal region (from below the cervical spine to above the pelvis).

[0088] As a further explanation of this embodiment, the depth camera in this embodiment is used to capture the three-dimensional structure of the spine. Its acquisition range covers the coronal plane (left-right direction) and sagittal plane (front-back direction) of the spine. It acquires the depth information of the region by emitting a detection signal of a specific wavelength (such as near-infrared light), and then generates three-dimensional point cloud data, from which static morphological parameters such as scoliosis angle and pelvic tilt angle are extracted. The visible light camera is installed side by side with the depth camera, and the lens is oriented in the same direction as the depth camera. It synchronously acquires color surface images of the spinal region. The images can clearly show the outlines of anatomical landmarks such as the spinous processes, iliac crests, and acromions, providing visual reference for subsequent localization.

[0089] Furthermore, in this embodiment, the timing coordination of the depth camera and the visible light camera acquisition is achieved through the built-in synchronization circuit of the module: the synchronization circuit sends a trigger signal every fixed time interval (e.g., 20ms) to simultaneously activate the acquisition units of the depth camera and the visible light camera, ensuring that the three-dimensional structural information of the spine and the surface image are recorded synchronously at the same time, avoiding feature misalignment caused by acquisition time difference.

[0090] The inertial sensing module 120 consists of at least three inertial sensors that integrate triaxial acceleration and triaxial angular velocity detection functions. These sensors are respectively installed at the anatomical locations of the T2, T8, and L4 vertebrae on the adolescent's body surface. They are used to collect acceleration and angular velocity change data of the spine during dynamic activities (such as walking, bending over, turning, standing and turning, sitting and getting up). The data acquisition of each sensor is started synchronously through unified timing control.

[0091] As a further explanation of this embodiment, each inertial sensor in the inertial sensing module 120 is an independently packaged unit. Its shell is made of lightweight, flexible material, which can conform to the skin of adolescents and reduce discomfort during activities. The deployment position of the sensor is determined as follows: the T2 vertebral body is located about one finger width above the midpoint of the line connecting the inner edges of the two scapulae; the T8 vertebral body is located at the intersection of the horizontal line connecting the midpoint of the two armpits and the midline of the spine; and the L4 vertebral body is located at the intersection of the line connecting the highest points of the two iliac crests and the midline of the spine. During deployment, a guardian or professional can use palpation to assist in marking the points to ensure that the center of the sensor is aligned with the marked points.

[0092] In this embodiment, the multimodal data acquisition unit 100 further includes a data association module 130, which is connected to the optical image sensing module 110 and the inertial sensing module 120 respectively. The data association module 130 is used to receive the static morphological parameters and corresponding body surface image data output by the optical image sensing module 110, and the acceleration change data and angular velocity change data output by the inertial sensing module 120. The data association module 130 matches and calibrates the spatial coordinates of the static morphological parameters with the anatomical position of the inertial sensor through a preset spatiotemporal association algorithm. At the same time, it aligns the acquisition time of the two types of data based on a unified time reference, establishes a mapping relationship between static morphological features and dynamic motion features, and finally generates structured multi-source raw data containing time stamps, spatial coordinates and motion parameters, forming a complementary feature dataset.

[0093] As a further explanation of this embodiment, the data association module 130 in this embodiment realizes spatiotemporal association in the following way:

[0094] Spatial matching calibration: Based on the surface images acquired by the visible light camera, anatomical landmarks such as the spinous processes and iliac crests of the spine are identified in the images, and their two-dimensional coordinates in the images are converted into three-dimensional spatial coordinates (combined with the depth information of the depth camera); at the same time, according to the deployment location of the inertial sensor (corresponding points of T2, T8, and L4 vertebrae), the spatial position of the sensor is marked in the same three-dimensional spatial coordinate system, and the spatial coordinates of the static morphological parameters are associated with the sensor position through a coordinate mapping algorithm to ensure that the two correspond in space;

[0095] Time reference alignment: The data association module 130 adds time stamps to all input data. The time stamps are generated based on the clock submodule built into the data association module 130, ensuring that the time records of optical image data and inertial motion data originate from the same reference. For data with slight differences in acquisition time (such as millisecond-level deviations caused by sensor response speed), time interpolation is used to map the static morphological parameters to the dynamic motion data at the corresponding time.

[0096] Finally, the data association module 130 integrates the processed information into structured data. Each data entry contains a time stamp (such as "hour-minute-second-millisecond"), three-dimensional coordinate parameters of the static morphology of the spine, and acceleration and angular velocity parameters of the corresponding vertebral position, forming a complete complementary feature dataset, which is then output to the data preprocessing unit 200.

[0097] It should be added that the multi-module collaborative logic of the multimodal data acquisition unit 100 in this embodiment is as follows: the optical image sensing module 110 and the inertial sensing module 120 synchronously start data acquisition based on unified timing control. The optical image sensing module 110 acquires static morphological parameters of the spine and corresponding surface image data through the collaborative alignment of the depth camera and the visible light camera. The inertial sensing module 120 acquires acceleration and angular velocity data during dynamic motion through sensors deployed at specific vertebral positions. After receiving the above two types of data, the data association module 130 matches and calibrates the spatial coordinates of the static morphological parameters with the anatomical position of the inertial sensor through a spatiotemporal association algorithm, and aligns the acquisition time based on a unified time reference to establish a mapping relationship between static and dynamic features, and finally generates structured multi-source raw data to form a complementary feature dataset.

[0098] The data preprocessing unit 200 is used to standardize the multi-source raw data output by the multimodal data acquisition unit 100, eliminate environmental interference and motion artifacts based on the adaptive Kalman filter algorithm, and extract quantifiable physical feature parameters.

[0099] In this embodiment, the data preprocessing unit 200 includes a data receiving and classification module 210, a standardization processing module 220, a filtering processing module 230, and a feature extraction module 240, wherein:

[0100] The data receiving and classification module 210 is connected to the output end of the multimodal data acquisition unit 100, and is used to receive raw data from multiple sources and classify and buffer them according to sensor type to form a subset of optical image data and a subset of inertial motion data.

[0101] As a further explanation of this embodiment, the data receiving and classification module 210 in this embodiment establishes a connection with the output end of the multimodal data acquisition unit 100 through a wired communication interface (such as USB 3.0) to receive the multi-source raw data transmitted by the unit in real time. The module has a built-in 1GB cache, which uses a dual-partition structure to store two types of data subsets, as detailed below:

[0102] The optical image data subset is cached in a structured format of "timestamp (accurate to millisecond) + 3D point cloud data of the spine + RGB image of the body surface + static morphological parameters (lateral curvature angle, pelvic tilt angle, etc.)". Each data frame is appended with a frame number for integrity verification.

[0103] The inertial motion data subset is cached in the format of "timestamp + T2 vertebral sensor data + T8 vertebral sensor data + L4 vertebral sensor data", where each sensor data contains the raw measurements of triaxial acceleration and triaxial angular velocity.

[0104] During the caching process, the data receiving and classification module 210 performs an integrity check every 100 frames of data received, and determines whether there is packet loss by comparing the continuity of the frame numbers. If packet loss is found, the multimodal data acquisition unit 100 is triggered to retransmit the corresponding frame data through a feedback signal to ensure that the time series of the two subsets are uninterrupted.

[0105] The standardization processing module 220 is connected to the data receiving and classification module 210. It is used to integrate the optical image data subset and the inertial motion data subset into mixed data by aligning them according to the timestamp, and to perform scale normalization processing on the optical three-dimensional coordinate parameters in the mixed data, and to perform dimension unification processing on the acceleration and angular velocity parameters in the mixed data, so that the two types of parameters maintain the consistency of the numerical range in the same mixed data structure.

[0106] As a further explanation of this embodiment, after the standardization processing module 220 reads the two subsets of data from the data receiving and classification module 210, it executes the following process:

[0107] First, perform timestamp alignment:

[0108] The timestamps of optical and inertial data are iterated, and a sliding window matching method is used (the window width is set to twice the sampling period, such as 20ms). Optical frames with a time difference ≤ 10ms are associated with inertial frames as a single frame of mixed data; frames with a time difference > 50ms are marked as invalid data and discarded to avoid timing misalignment affecting subsequent processing.

[0109] Then, the optical three-dimensional coordinate parameters and inertial parameters are standardized:

[0110] Scale normalization processing is applied to optical 3D coordinate parameters: using the entire length of the spine (the straight-line distance from the C7 spinous process to the S1 spinous process) as a reference, the 3D coordinates of all spinal feature points are normalized. Converting to a relative value, the calculation formula is:

[0111] ;

[0112] in, , , The original coordinates of the feature point. For the entire length of the spine, , , The normalized coordinates are mapped to [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 16, 17, 18, 19 ... 1] interval.

[0113] Dimensional unification processing for inertial parameters:

[0114] The acceleration parameters are based on the acceleration range conversion of adolescents' daily activities, and the formula is: ;

[0115] Angular velocity parameters are based on the conversion of angular velocity ranges during spinal dynamic motion, and the formula is: ;

[0116] in, , These are the original inertial parameters. , The normalized parameters are mapped to the interval [-1, 1].

[0117] Finally, after processing, the mixed data is sorted by "timestamp + "Formatted storage ensures that the numerical ranges of the two types of parameters are consistent in the same frame of data."

[0118] The filtering module 230 is connected to the standardization module 220 and has a built-in adaptive Kalman filter algorithm for noise suppression of the standardized mixed data. It eliminates ambient light interference, sensor drift and motion artifacts by dynamically adjusting the filtering parameters.

[0119] The feature extraction module 240 is connected to the filtering module 230. Based on preset feature engineering rules, it extracts quantifiable physical feature parameters (such as the rate of change of the physiological curvature of the spine, the deviation of the vertebral motion trajectory, the motion symmetry index, etc.) from the filtered data.

[0120] As a further explanation of this embodiment, the process by which the feature extraction module 240 extracts quantifiable physical feature parameters based on preset feature engineering rules includes the following steps:

[0121] First, the data source and preprocessing are clarified: the feature extraction module 240 receives the denoised mixed data output by the filtering module 230, which contains a subset of optical parameters (such as the normalized three-dimensional coordinates of spinal feature points). , , , where is the feature point index) and a subset of inertial parameters (such as the normalized acceleration of vertebrae T2, T8, and L4). and angular velocity ,in (Indicates the vertebral body position); All parameters have been filtered to eliminate environmental interference and motion artifacts.

[0122] Subsequently, the rate of change of the physiological curvature of the spine was extracted:

[0123] The first step is to determine the sagittal plane of the spine: using the anterior-posterior direction of the human body as a reference, select... flat( (Using normalized axial coordinates along the midline of the spine) as the sagittal plane;

[0124] The second step is to fit the spinal curve: This involves matching the coordinates of spinal feature points (such as T2, T8, and L4 spinous processes) from the filtered optical parameters. according to The values ​​are arranged in ascending order, and linear interpolation is used to complete adjacent points to form a continuous sagittal curve;

[0125] The third step is to calculate the curvature: take three adjacent points on the curve. The curvature of the curve segment is calculated using the three-point circle method (the curvature value is positively correlated with the degree of bending).

[0126] Step 4, calculate the rate of change: for the first... Frame and the The curvature values ​​at the same location in each frame are differen and divided by the time interval between the two frames. (Matching the data acquisition frequency), the curvature change rate is obtained, and the formula is: ,in For the first Frame curvature, For the first Frame curvature;

[0127] Then, extract the deviation of the vertebral motion trajectory:

[0128] The first step is to reconstruct the actual trajectory: normalized acceleration from a subset of inertial parameters. and angular velocity By performing inverse normalization (restoring to the actual physical quantities), the displacement changes of vertebrae T2, T8, and L4 are calculated through quadratic integration. The initial coordinates from the optical parameters are then superimposed to obtain the actual motion trajectory. ,in For time;

[0129] The third step is to calculate the deviation: For each time t, the deviation between the actual trajectory and the reference trajectory is calculated using the Euclidean distance formula, i.e.:

[0130] ;

[0131] in, express Time of the first Deviation in the motion trajectory of each vertebra; express Time of the first The actual normalization of each vertebral body coordinate; express Time of the first Reference trajectory normalization of individual vertebrae coordinate; express Time of the first The actual normalization of each vertebral body coordinate; express Time of the first Reference trajectory normalization of individual vertebrae coordinate; express Time of the first The actual normalization of each vertebral body coordinate; express Time of the first Reference trajectory normalization of individual vertebrae coordinate;

[0132] Next, extract the motion symmetry index:

[0133] The first step is to pair symmetrical landmarks: select bilateral symmetrical anatomical points from the subset of optical parameters, including the left acromion. with the right acromion Left iliac crest With the right iliac crest ;

[0134] The second step is to calculate the static symmetry deviation: using the spinal midline (a straight line fitted through the coordinates of the C7, T8, and S1 spinous processes) as a reference, calculate the distance difference from the bilateral points to the midline, such as the acromion symmetry deviation. ,in This is a distance calculation function;

[0135] The third step is to calculate the dynamic symmetry deviation: For the acceleration and angular velocity of the T2 and T8 vertebrae in the inertial parameters, the contralateral virtual point parameters are obtained through symmetrical mapping along the spinal midline. The mean absolute value of the time series difference between the two sides is calculated, such as the acceleration symmetry deviation. ,in For frame number, For both sides Frame acceleration;

[0136] The fourth step is to integrate the comprehensive index: take the arithmetic mean of the static and dynamic symmetry deviations to obtain the comprehensive index of motion symmetry;

[0137] Finally, the feature extraction module 240 integrates the extracted physical feature parameters such as the rate of change of spinal physiological curvature, vertebral motion trajectory deviation, and motion symmetry index into structured data and outputs them to the intelligent analysis unit 300 for subsequent risk level assessment.

[0138] In this embodiment, the filtering module 230 performs noise suppression on the standardized mixed data using an adaptive Kalman filter algorithm, including the following steps:

[0139] S230.1, Mixed Data Reception and Structure Parsing:

[0140] The system receives standardized mixed data output from the standardization processing module 220. The standardized mixed data includes a subset of optical parameters and a subset of inertial parameters. The subset of optical parameters consists of parameters from the subset of optical image data in the mixed data, including the scale-normalized three-dimensional coordinates of spinal feature points and the relative positions of anatomical landmarks. The subset of inertial parameters consists of parameters from the subset of inertial motion data in the mixed data, including the dimensionally unified acceleration and angular velocity of the corresponding positions of the T2, T8, and L4 vertebrae.

[0141] Furthermore, the two types of parameters are aligned according to the timestamp to form a frame data sequence. ,in For time frame indexing;

[0142] S230.2 System Status and Parameter Initialization:

[0143] The system state vector is defined based on a subset of parameters from mixed data. ,in: express transpose; express transpose; The transpose operation represents a matrix or vector;

[0144] This is a subset of optical parameters, containing normalized 3D coordinates of spinal feature points. ;in, For the first The normalized coordinates of each spinal feature point are mapped to [0, ... 1] Interval (based on the total length of the spine);

[0145] This is a subset of inertial parameters (corresponding to a subset of inertial parameters), containing normalized acceleration and angular velocity. ;in, For the first Normalized acceleration at vertebral position, For the first The normalized angular velocities of the vertebral body positions are all mapped to [-1]. 1] interval;

[0146] Initialize first frame state estimation State covariance matrix (The diagonal elements are the squared values ​​of the standardized errors of each parameter), where Represents the vector after integrating the multimodal data from the first frame; measurement noise covariance matrix. ,in Corresponding optical parameter measurement noise, Corresponding inertial parameter measurement noise; initial process noise covariance matrix (Based on noise statistical characteristics settings in static scenarios);

[0147] S230.3, State Prediction: For Time, based on Optimal estimation of state vector at time step The predicted state is calculated using the state equation, and the formula is:

[0148] ;

[0149] in: Let be the state transition matrix, and , For optical parameter submatrices (adapting to the changing characteristics of the optical parameter subset, using a rigid body motion model to represent the integrity of the spinal structure). For the inertial parameter submatrix (adapting to the changing characteristics of the inertial parameter subset, using kinematic differential relations to describe the time-varying laws of acceleration and angular velocity); For the control matrix, For associated sampling intervals; express Time based The predicted state vector at time estimate;

[0150] Simultaneous calculation of the prediction covariance matrix: ;in, express The covariance matrix of the predicted state at each time step represents the statistical characteristics of the prediction error. express The covariance matrix of the time-optimal estimate; Represents the state transition matrix The transpose of the matrix; express The process noise covariance matrix at time step;

[0151] S230.4, Residual Calculation and Noise Adjustment:

[0152] Pick Moment-mixed data frames As a measurement vector ,in These are measured values ​​of optical parameters. These are measured values ​​of inertial parameters;

[0153] Calculate residuals As evidence of measurement; among them, Represents the measurement matrix, and , This represents a sub-block of the optical parameter measurement matrix, which maps optical state parameters to optical measurement values. This represents a sub-block of the inertial parameter measurement matrix, which maps inertial state parameters to inertial measurement values.

[0154] Based on residual modulus Adjustment process noise covariance matrix ,in For adaptive adjustment coefficients; Indicates the adaptive adjustment coefficient;

[0155] In this step, the adaptive adjustment coefficient Based on the dynamic calculation of the residual vector magnitude, the formula is:

[0156] ;

[0157] in, For residuals, For the observed values, For the predicted state; The initial noise standard deviation (after 10 sets of static calibration, optical coordinate deviation) Inertial angular velocity noise );

[0158] Furthermore, the measurement matrix Dimensions (6-dimensional observation: coordinate deviation of 3 optical marker points, 3 inertial angular velocity deviations; 12-dimensional state: coordinates of 6 cone nodes, 6 inertial parameters).

[0159] S230.5, Status Update:

[0160] Calculate Kalman gain Update the optimal estimate Correcting covariance ;in, This represents the measurement noise covariance matrix, characterizing the inherent noise of the sensor. Represents the matrix inversion operation; Represents the identity matrix, with dimensions matching the state vector;

[0161] S230.6, Output of filtering results: Time-optimal estimation The denoised mixed data is output to the feature extraction module 240, and steps S230.3 to S230.5 are repeated until all frame data processing is completed.

[0162] It should be added that the multi-module collaborative logic of the data preprocessing unit 200 in this embodiment is as follows: the data receiving and classification module 210 receives the multi-source raw data output by the multimodal data acquisition unit 100, and classifies and caches it into optical image data subsets and inertial motion data subsets according to sensor type; the standardization processing module 220 integrates the two types of data subsets into mixed data by aligning them according to timestamps, and performs scale normalization and dimension unification processing; the filtering processing module 230 uses an adaptive Kalman filter algorithm to suppress noise in the standardized mixed data and eliminate environmental interference and motion artifacts; the feature extraction module 240 extracts quantifiable physical feature parameters from the filtered data based on preset feature engineering rules, and finally outputs them to the intelligent analysis unit 300.

[0163] The intelligent analysis unit 300 generates the risk level of scoliosis based on the physical feature parameters output by the data preprocessing unit 200. By constructing a machine learning model that integrates the biomechanical features of the spine, it performs cross-dimensional fusion analysis on the physical feature parameters and strengthens the weight ratio of key pathological features in the risk level calculation through an adaptive weight allocation mechanism.

[0164] In this embodiment, the intelligent analysis unit 300 includes a feature selection module 310, a model inference module 320, and a level calibration module 330, wherein:

[0165] The feature selection module 310 is connected to the feature extraction module 240 of the data preprocessing unit 200. It is used to screen core pathological features from physical feature parameters. By calculating the biomechanical correlation between each feature and the pathological changes of scoliosis (such as the influence weight of the feature on spinal imbalance), features with significant correlation are retained to form an input feature set.

[0166] As a further explanation of this embodiment, the feature selection module 310 of this embodiment filters core pathological features through the following quantification steps:

[0167] Biomechanical correlation calculation: The correlation is quantified using the characteristic contribution formula. ;in, Score the correlation of the i-th physical feature. These are physical characteristic parameters (such as scoliosis angle, motion trajectory deviation, etc.). The biomechanical balance index of the spine (calculated based on the vertebral load distribution model). This represents the gradient of the influence of the feature on the balance index. For variance calculation;

[0168] Core Feature Filtering: Set Filtering Threshold ,in Total number of features; retain Features form the input feature set ;in, This represents a single element in the set, corresponding to the 1st, 2nd, ... 1st element. One core physical characteristic parameter, Indicates the number of core features. This indicates the total number of physical features initially extracted by the feature extraction module 240.

[0169] The model inference module 320 incorporates a machine learning model that integrates spinal biomechanical features. It adopts a "feature layer fusion + decision layer fusion" architecture, which specifically includes: the feature layer couples static morphological features with dynamic motion features; the decision layer outputs a preliminary risk level based on the integrated model.

[0170] As a further explanation of this embodiment, the decision-level fusion of the model inference module 320 in this embodiment includes the following steps:

[0171] First, fuse the feature vectors Input at least two basic machine learning models (such as random forest and gradient boosting tree);

[0172] Subsequently, each model outputs an independent risk score. ( (Model number);

[0173] Then, based on the accuracy of each model in historical data... Assign model weights (The total weight is 1);

[0174] Finally, the preliminary risk level is calculated using a weighted average. .

[0175] The 330 grade calibration module constructs a calibration mechanism based on the principles of spinal biomechanics to correct the initial risk level and ensure consistency with the pathological progression of scoliosis.

[0176] As a further explanation of this embodiment, the risk level correction of the level calibration module 330 in this embodiment includes the following steps:

[0177] First, the initial risk level will be determined. Comparison with the pathological progression pattern of scoliosis;

[0178] Subsequently, through coupling equations Verify biomechanical consistency and calculate deviation values. ;

[0179] Then, if (If coupling is abnormal), then... Upgrade by 1 level, if the dynamic stability index is normal and If it is too high, then lower it by 1 level;

[0180] Finally, output the calibrated risk level. .

[0181] In this embodiment, the machine learning model of the model inference module 320 includes a biomechanical constraint layer and a cross-dimensional fusion layer, wherein:

[0182] Biomechanical constraint layer embedded in spinal kinematic coupling equations: ;in, The angle of vertebral rotation. For the scoliosis angle, The coupling coefficient (determined based on spinal anatomical features). This is the error term;

[0183] The cross-dimensional fusion layer achieves the fusion of static and dynamic features through feature mapping formulas: ;in, To fuse feature vectors, This is an optical static feature vector. For inertial dynamic feature vectors, , These are the fusion weight matrices for the two types of features (determined through feature importance analysis).

[0184] In this embodiment, the intelligent analysis unit 300 further includes a weight initialization module 340 and a dynamic adjustment module 350, wherein:

[0185] The weight initialization module 340 is connected to the feature selection module 310. Based on the biomechanical pathological effects of the core features (such as the degree of influence of the scoliosis angle on spinal stability), it assigns initial weights to each feature. The weight values ​​are positively correlated with the pathological correlation of the features.

[0186] As a further explanation of this embodiment, the initial weight allocation of the weight initialization module 340 in this embodiment includes the following steps:

[0187] First, obtain the core features and their correlations output by the feature selection module. ;

[0188] Then, the initial weights for each feature are calculated. ;in For the first The initial weights of the features are summed as follows:

[0189] Then, features directly related to spinal stability (such as scoliosis angle) are assigned higher weights (e.g., 0.2-0.3), while auxiliary features are assigned lower weights (e.g., 0.05-0.1).

[0190] Finally, the initial weight matrix is ​​output.

[0191] The dynamic adjustment module 350 is connected to the model inference module 320, and adjusts the weights by analyzing the actual correlation changes between features and risk levels (such as the consistency between feature value fluctuations and risk level changes).

[0192] Increase the weight of a feature as the correlation between it and the risk level strengthens with the accumulation of data.

[0193] When the correlation weakens, its weight is reduced in order to dynamically strengthen the weight of key pathological features.

[0194] As a further explanation of this embodiment, the dynamic adjustment of weights by the dynamic adjustment module 350 in this embodiment includes the following steps:

[0195] First, a sliding window (e.g., 100 consecutive data sets) is used to calculate the risk level for each feature after calibration. Pearson correlation coefficient ;in, For covariance, For variance;

[0196] Subsequently, if (If the relevance is enhanced), then the weight is increased; if If the correlation weakens, then reduce the weight.

[0197] Then, update the weights using the following formula: ,in For the updated weights, It is a symbolic function;

[0198] Finally, the updated weights are normalized (ensuring the sum is 1) and constrained to be in the range of 0.02-0.4.

[0199] The early warning execution unit 400 is used to trigger an alarm mechanism when the risk level of scoliosis output by the intelligent analysis unit 300 exceeds a preset threshold. It drives the audible and visual alarm device through an electrical signal and simultaneously pushes electronic data containing characteristic abnormal parameters.

[0200] In this embodiment, the early warning execution unit 400 includes a threshold judgment module 410, an audio-visual driving module 420, and a data push module 430, wherein:

[0201] The threshold judgment module 410 is connected to the output end of the intelligent analysis unit 300. It is used to receive scoliosis risk level data and compare the scoliosis risk level data with the risk level threshold built into the system (set according to the pathological grading standard of scoliosis). When the risk level exceeds the threshold, a trigger signal is output.

[0202] As a further explanation of this embodiment, the specific content of the risk level threshold (set according to the pathological grading standard for scoliosis) built into the system in this embodiment is as follows:

[0203] The system's built-in risk level thresholds are set based on clinically recognized scoliosis pathological grading standards (with Cobb angle as the core indicator), and are correlated with the risk levels (levels 1-5) output by the intelligent analysis unit 300:

[0204] The pathological grading criteria for scoliosis are: mild scoliosis (Cobb angle <20°), moderate scoliosis (Cobb angle 20°-40°), and severe scoliosis (Cobb angle >40°).

[0205] The correspondence between systemic risk level and pathological grade is as follows: risk level 1-2 corresponds to mild scoliosis, level 3 corresponds to moderate scoliosis, and level 4-5 corresponds to severe scoliosis.

[0206] Therefore, the system's built-in risk level threshold is set to level 3. That is, when the risk level output by the intelligent analysis unit 300 is ≥ level 3, it is determined that the threshold has been exceeded and the early warning mechanism is triggered.

[0207] Understandably, this threshold can be adjusted through the system management interface (for example, medical institutions can lower the threshold to level 2 according to clinical needs to strengthen the early warning of mild scoliosis). The adjustment range is limited to levels 1-5 to ensure that the correlation with the pathological grading standard is not disrupted, and it has clinical adaptability and operational feasibility.

[0208] Furthermore, the specific implementation logic of the threshold determination module 410 in this embodiment is as follows:

[0209] Threshold judgment module 410 receives the calibrated risk level output by intelligent analysis unit 300. (Values ​​range from 1 to 5, corresponding to risks from low to high);

[0210] Call the system's built-in risk level threshold (Based on the pathological grading standards for scoliosis, for example, when the clinical definition is "moderate or higher risk requiring warning") );

[0211] By comparing the rule "if" If the signal is triggered, a trigger signal will be output to determine whether to initiate the early warning process.

[0212] The sound and light driving module 420 is connected to the threshold judgment module 410. After receiving the trigger signal, it converts it into a driving signal through the electrical signal amplification circuit to control the sound and light alarm device (including LED warning light and buzzer) to work in a preset mode (such as the light flashing frequency and buzzer interval corresponding to the risk level).

[0213] As a further explanation of this embodiment, after the acoustic-optical driving module 420 receives the trigger signal:

[0214] First, the digital trigger signal is converted into an analog signal sufficient to drive the audio-visual equipment through an electrical signal amplification circuit (such as a power amplifier circuit built with an operational amplifier).

[0215] Subsequently, according to The specific level controls the audible and visual alarm devices to respond according to preset modes, for example:

[0216] when (Moderate risk): LED warning lights flash at a frequency of 2 seconds per time, and buzzers sound at intervals of 3 seconds per time;

[0217] when (Severe Risk): The LED warning light flashes rapidly at a frequency of 0.5 seconds per second, and the buzzer sounds continuously;

[0218] Understandably, the above response mode can be adjusted through the system configuration interface to adapt to the warning needs of different scenarios.

[0219] The data push module 430 is connected to the threshold judgment module 410 and the data storage unit 500 respectively. After receiving the trigger signal, it extracts the characteristic abnormal parameters (such as excessive side bending angle, abnormal value of motion symmetry) within the corresponding detection period from the data storage unit 500, pushes them to the preset terminal (such as the guardian's mobile phone, medical institution management system) through the encrypted communication protocol (such as HTTPS), and generates push logs and stores them in the data storage unit 500.

[0220] As a further explanation of this embodiment, when the data push module 430 of this embodiment responds to the trigger signal:

[0221] First, abnormal parameter extraction: retrieve abnormal physical features marked by the feature extraction module 240 within the current detection period from the data storage unit 500 (such as "excessive scoliosis angle" and "excessive motion symmetry deviation", i.e. parameters that deviate from the normal range in the feature extraction process).

[0222] Subsequently, encrypted push: using the HTTPS encryption protocol, the abnormal parameters are encapsulated into structured data (such as JSON format, containing abnormal feature name, value, and timestamp) and pushed to preset terminals (such as the guardian's mobile phone, medical institution management system, and the terminal address is pre-configured through the system backend).

[0223] Finally, log recording: generate an early warning push log, including push time, risk level, and summary of abnormal parameters, and store it in the "early warning log" partition of data storage unit 500 for easy subsequent traceability.

[0224] It should be added that the multi-module collaborative logic of the early warning execution unit 400 in this embodiment is as follows:

[0225] First, the threshold judgment module 410 receives the scoliosis risk level data output by the intelligent analysis unit 300 and compares it with the risk level threshold (set according to the pathological grading standard) built into the system. When the risk level exceeds the threshold, a trigger signal is output. Then, the sound and light drive module 420 receives the trigger signal and converts it into a drive signal through an electrical signal amplification circuit to control the sound and light alarm device to work according to a preset mode (such as the light flashing frequency and the beep interval corresponding to the risk level). At the same time, the data push module 430 receives the trigger signal, extracts the characteristic abnormal parameters (such as the excessive scoliosis angle and abnormal values ​​of motion symmetry) corresponding to the detection period from the data storage unit 500, pushes them to preset terminals (such as the guardian's mobile phone and the medical institution management system) through the HTTPS encryption protocol, and generates a push log and stores it in the data storage unit 500.

[0226] The data storage unit 500 is used to store the multi-source raw data output by the multimodal data acquisition unit 100, the physical characteristic parameters output by the data preprocessing unit 200, and the scoliosis risk level output by the intelligent analysis unit 300. The data is stored in association using a general-purpose storage medium.

[0227] In this embodiment, the data storage unit 500 includes a data receiving module 510, a hierarchical storage module 520, an association index module 530, and an access control module 540, wherein:

[0228] The data receiving module 510 is used to receive multi-source raw data (including a subset of optical image data and a subset of inertial motion data), physical characteristic parameters (including the rate of change of the physiological curvature of the spine, etc.) and the risk level of scoliosis, and to attach a unique identifier (including the detection object ID, timestamp, and data type label) to each type of data.

[0229] As a further explanation of this embodiment, the data receiving process of the data receiving module 510 in this embodiment includes the following steps: First, the data output by the multimodal data acquisition unit 100, the data preprocessing unit 200, and the intelligent analysis unit 300 is received through the data interface unit; then, the identifier generation unit generates a unique identifier in the format of "detection object ID-timestamp-data type label" and attaches it to the corresponding data; then, the integrity verification unit verifies the data, and if it passes, proceeds to the next step, and if it fails, it triggers retransmission; finally, the complete data with the unique identifier is transmitted to the hierarchical storage module 520.

[0230] The hierarchical storage module 520 is connected to the data receiving module 510 and implements hierarchical storage based on data characteristics;

[0231] As a further explanation of this embodiment, the hierarchical storage of this embodiment is as follows:

[0232] Multi-source raw data (large volume, unstructured) is stored on magnetic storage media (such as hard disk arrays).

[0233] Physical characteristics (medium capacity, structured) are stored in solid-state storage media (such as SSDs).

[0234] Scoliosis risk levels (small capacity, high access frequency) are stored in an in-memory database to achieve a balance between data storage efficiency and access speed.

[0235] As a further explanation of this embodiment, the hierarchical storage process of the hierarchical storage module 520 in this embodiment includes the following steps: First, receiving the identified data output by the data receiving module 510; then, allocating it to the corresponding storage medium according to the data type (raw data / feature parameters / risk level); then, completing the data writing according to the preset storage structure (folder hierarchy / table / key-value pair); finally, sending the storage completion signal and data storage path to the associated index module 530.

[0236] The association index module 530 is connected to the hierarchical storage module 520. It establishes the association relationship between the three types of data through preset index rules (with the detection object ID as the core key and the timestamp as the auxiliary key), and generates an index table containing data storage path and association weight (the association weight between the original data and feature parameters in the same detection period is set to the highest), which supports fast source tracing and querying of cross-type data.

[0237] As a further explanation of this embodiment, the indexing rules and index table structure of the associated index module 530 in this embodiment specifically include:

[0238] The preset indexing rules use the object ID as the core key (to ensure data association for the same object) and the timestamp as the secondary key (sorted by time).

[0239] The index table is a structured data table (stored on SSD), with fields including: core key (detection object ID), auxiliary key (time stamp), data type, storage path, association weight (value 0-1, the association weight between raw data and feature parameters in the same detection period is set to 0.9, and the association weight between cross-period data is set to 0.3), and index update time.

[0240] As a further explanation of this embodiment, the association index creation process of the association index module 530 in this embodiment includes the following steps: First, receive the storage completion signal and data storage path sent by the hierarchical storage module 520; then, extract the detection object ID and timestamp from the unique identifier of the data as the core key and auxiliary key; then, fill in the information according to the fields of the index table and set the association weight according to the data association; finally, update the index table and synchronize it to the memory database to ensure that it can be quickly called during query.

[0241] Access control module 540 is connected to associated index module 530 and sets access permissions based on data sensitivity level.

[0242] As a further explanation of this embodiment, the specific methods for setting access permissions based on data sensitivity levels include:

[0243] Multi-source raw data and physical characteristic parameters can only be accessed by authorized medical institutions through an encrypted interface;

[0244] The risk level of scoliosis can be queried by the subject of the test and their guardian after identity verification;

[0245] All data access operations generate logs (including access time, operation type, and access subject) and are synchronously stored in the hierarchical storage module 520.

[0246] As a further explanation of this embodiment, the access control process of the access control module 540 in this embodiment includes the following steps: First, receiving an external access request (including access subject information and request data identifier); then, verifying the identity and permission level of the access subject, and rejecting and recording if the permissions do not match; then, after the permissions are granted, querying the associated index module 530 to obtain the storage path according to the request data identifier, and retrieving the data from the corresponding storage medium; next, transmitting the data through an encrypted interface (external access) or directly (internal system access); finally, generating an access log and storing it in the hierarchical storage module 520.

[0247] It should be added that the multi-module collaborative logic of the data storage unit 500 in this embodiment is as follows:

[0248] First, the data receiving module 510 receives multi-source raw data, physical characteristic parameters, and scoliosis risk levels, and adds a unique identifier containing the detection object ID, timestamp, and data type tag to each type of data. Then, the hierarchical storage module 520 receives the identified data and implements hierarchical storage based on data characteristics (multi-source raw data is stored on magnetic storage media, physical characteristic parameters on solid-state storage media, and risk levels in an in-memory database). Next, the association index module 530, based on the hierarchical storage results, establishes the association relationship between the three types of data using the detection object ID as the core key and the timestamp as the auxiliary key, generating an index table containing storage paths and association weights. Finally, the access control module 540 sets access permissions based on data sensitivity levels, verifies permissions for data access operations, and generates access logs that are stored in the hierarchical storage module 520.

[0249] Those skilled in the art will understand that the process of implementing all or part of the steps of the above embodiments can be carried out by hardware or by a program instructing the relevant hardware.

[0250] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multi-modal sensor based intelligent monitoring system for scoliosis in adolescents, characterized in that, The application relates to a spinal scoliosis risk assessment system, which comprises the following: a multi-modal data acquisition unit (100) for capturing multi-dimensional features of spinal structure abnormalities, combining an optical image sensor with an inertial sensor to synchronously acquire static morphological parameters and dynamic motion data of a spine, and forming a complementary feature data set composed of multi-source original data; a data preprocessing unit (200) for standardizing multi-source original data output by the multi-modal data acquisition unit (100), eliminating environmental interference and motion artifacts based on an adaptive Kalman filtering algorithm, and extracting quantifiable physical feature parameters; an intelligent analysis unit (300) for generating a spinal scoliosis risk level based on the physical feature parameters output by the data preprocessing unit (200), performing cross-dimensional fusion analysis on the physical feature parameters by constructing a machine learning model integrating spinal biomechanical features, and strengthening the weight proportion of key pathological features in the risk level calculation through an adaptive weight distribution mechanism; the intelligent analysis unit (300) comprises a feature selection module (310), a model inference module (320) and a level calibration module (330), wherein: the feature selection module (310) is connected with a feature extraction module (240) of the data preprocessing unit (200) and is used for screening core pathological features from the physical feature parameters, retaining features with significant biomechanical correlation with spinal scoliosis pathological changes to form an input feature set by calculating the biomechanical correlation of each feature with the spinal scoliosis pathological changes; the model inference module (320) internally has a machine learning model integrating spinal biomechanical features, adopts a "feature layer fusion + decision layer fusion" architecture, and specifically comprises: a feature layer for coupling static morphological features and dynamic motion features; a decision layer for outputting a preliminary risk level based on an integrated model; the machine learning model of the model inference module (320) comprises a biomechanical constraint layer and a cross-dimensional fusion layer, wherein: The biomechanical constraint layer is embedded in a spinal kinematics coupling equation: ; wherein, is a vertebral rotation angle, is a spinal scoliosis angle, is a coupling coefficient, is an error term; The cross-dimension fusion layer realizes fusion of static and dynamic features through a feature mapping formula: ; wherein, is a fusion feature vector, is an optical static feature vector, is an inertial dynamic feature vector, , are fusion weight matrices of the two types of features respectively; the level calibration module (330) constructs a calibration mechanism based on spinal biomechanical principles, corrects the preliminary risk level, and ensures consistency with the spinal scoliosis pathological progression rule; the intelligent analysis unit (300) further comprises a weight initialization module (340) and a dynamic adjustment module (350), wherein: the weight initialization module (340) is connected with the feature selection module (310) and assigns initial weights to each feature based on the biomechanical pathological influence of core features, and the weight value is positively correlated with the pathological correlation of the feature; the dynamic adjustment module (350) is connected with the model inference module (320) and adjusts the weight by analyzing the actual correlation change between the feature and the risk level: when the correlation between the feature and the risk level is enhanced with data accumulation, the weight of the feature is increased; when the correlation is weakened, the weight of the feature is reduced, so as to dynamically strengthen the weight proportion of key pathological features. The early warning execution unit (400) is configured to trigger an alarm mechanism when the scoliosis risk level output by the intelligent analysis unit (300) exceeds a preset threshold, drive an audible and visual alarm device through an electrical signal, and synchronously push electronic data containing characteristic abnormal parameters; The data storage unit (500) is configured to store the multi-source raw data output by the multi-modal data acquisition unit (100), the physical characteristic parameters output by the data preprocessing unit (200), and the scoliosis risk level output by the intelligent analysis unit (300), and realize the associated storage of data by using a general storage medium.

2. The multi-modal sensor based intelligent monitoring system for scoliosis in adolescents as claimed in claim 1 wherein, The multi-modal data acquisition unit (100) comprises an optical image sensing module (110) and an inertial sensing module (120), wherein: The optical image sensing module (110) adopts a combined device composed of a depth camera and a visible light camera, the depth camera is configured to collect three-dimensional structural information of the coronal plane and the sagittal plane of the spine to extract static morphological parameters, the visible light camera is configured to synchronously collect the body surface image of the corresponding region to assist in positioning the anatomical landmark points, and the depth camera and the visible light camera are cooperatively aligned in the collection time sequence through a trigger signal; The inertial sensing module (120) is composed of at least three inertial sensors integrating three-axis acceleration and three-axis angular velocity detection functions, and is respectively arranged at the anatomical positions of the T2, T8 and L4 vertebral bodies of the adolescent body surface, and is configured to collect acceleration change data and angular velocity change data of the spine in dynamic activities, and each sensor realizes synchronous starting of data collection through unified time sequence control.

3. The multi-modal sensor based intelligent monitoring system for scoliosis in adolescents as claimed in claim 2 wherein, The multi-modal data acquisition unit (100) further comprises a data correlation module (130) connected with the optical image sensing module (110) and the inertial sensing module (120), configured to receive the static morphological parameters and the corresponding body surface image data output by the optical image sensing module (110), and the acceleration change data and the angular velocity change data output by the inertial sensing module (120); the data correlation module (130) matches and calibrates the spatial coordinates of the static morphological parameters with the anatomical positions of the inertial sensors through a preset space-time correlation algorithm, simultaneously aligns the collection time of the two types of data based on a unified time reference, establishes a mapping relationship between the static morphological features and the dynamic motion features, and finally generates structured multi-source raw data containing time identifiers, spatial coordinates and motion parameters, and forms a complementary feature data set.

4. The multi-modal sensor based intelligent monitoring system for scoliosis in adolescents as claimed in claim 3 wherein, The data preprocessing unit (200) comprises a data receiving and classification module (210), a standardization processing module (220), a filtering processing module (230), and a feature extraction module (240), wherein: The data receiving and classification module (210) is connected with the output end of the multi-modal data acquisition unit (100), configured to receive the multi-source raw data and classify and buffer them according to the sensor types, to form an optical image data subset and an inertial motion data subset; The standardization processing module (220) is connected with the data receiving and classification module (210), and is used for aligning and integrating the optical image data subset and the inertial motion data subset into mixed data according to timestamps, performing scale normalization processing on optical three-dimensional coordinate parameters in the mixed data, and performing dimension unification processing on acceleration and angular velocity parameters in the mixed data, so that the two types of parameters maintain numerical interval consistency in the same mixed data structure; The filtering processing module (230) is connected with the standardization processing module (220), and is internally provided with an adaptive Kalman filtering algorithm, and is used for noise suppression on the mixed data after standardization, and eliminating environmental light interference, sensor drift and motion artifacts by dynamically adjusting filtering parameters; The feature extraction module (240) is connected with the filtering processing module (230), and is used for extracting quantifiable physical feature parameters from the filtered data based on preset feature engineering rules.

5. The multi-modal sensor based intelligent monitoring system for scoliosis in adolescents as claimed in claim 4 wherein, The early warning execution unit (400) includes a threshold judgment module (410), an audible and light driving module (420) and a data pushing module (430), wherein: The threshold judgment module (410) is connected with the output end of the intelligent analysis unit (300), is used for receiving the scoliosis risk level data, and comparing the scoliosis risk level data with the risk level threshold value built-in the system, and outputting a trigger signal when the risk level exceeds the threshold value; The audible and light driving module (420) is connected with the threshold judgment module (410), and after receiving the trigger signal, converts the trigger signal into a driving signal through an electric signal amplification circuit, and controls the audible and light alarm device to work in a preset mode; The data pushing module (430) is connected with the threshold judgment module (410) and the data storage unit (500) respectively, extracts the feature abnormal parameters in the corresponding detection period from the data storage unit (500) after receiving the trigger signal, pushes the feature abnormal parameters to a preset terminal through an encryption communication protocol, and generates a pushing log and stores the pushing log in the data storage unit (500).

6. The multi-modal sensor based intelligent monitoring system for scoliosis in adolescents as claimed in claim 5 wherein, The data storage unit (500) includes a data receiving module (510), a hierarchical storage module (520), an association index module (530) and an access control module (540), wherein: The data receiving module (510) is used for receiving multi-source original data, physical feature parameters and scoliosis risk levels, and attaching a unique identifier to each type of data; The hierarchical storage module (520) is connected with the data receiving module (510), and implements hierarchical storage based on data characteristics; The association index module (530) is connected with the hierarchical storage module (520), establishes an association relationship of the three types of data through a preset index rule, generates an index table containing a data storage path and an association weight, and supports fast traceability query of cross-type data; The access control module (540) is connected with the association index module (530), and sets access permissions based on data sensitivity levels.

Images