Multi-mode synchronous acquisition spine dynamic evaluation system

Abstract

The invention discloses a multi-modal synchronous acquisition spine dynamic evaluation system, which relates to the field of biomedical engineering, and comprehensively considers multi-source information such as back depth image data, pressure offset and gravity center, back electromyographic signals and the like according to the multi-modal synchronous acquisition spine dynamic evaluation system. Through fusion of an iToF depth camera, a pressure induction balancing instrument and a high-sampling-rate surface electromyography instrument, synchronous acquisition and intelligent judgment of multi-modal data are realized, and a multi-modal fusion judgment mechanism based on a multi-modal weight adaptive model is designed to assist scoliosis screening, offset positioning and motion risk judgment. The problems that an existing scoliosis evaluation method is low in efficiency, high in subjectivity and the like are solved, and the accuracy and real-time performance of posture analysis are effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of biomedical engineering, in particular to a multi-modal synchronous acquisition dynamic evaluation system for spine. BACKGROUND

[0002] In recent years, the incidence of adolescent scoliosis is increasing year by year, and idiopathic scoliosis accounts for more than 80%. Scoliosis has become a disease that seriously affects the physical and mental health of adolescents. If it is not discovered and corrected early, long-term scoliosis may develop into structural deformity, and severe cases may be complicated with heart and lung dysfunction, and even lead to paralysis or death. At present, the clinical screening of scoliosis mainly relies on manual physical examination by doctors and X-ray diagnosis, which not only has a cumbersome process and low efficiency, but also has the risk of radiation exposure, and it is difficult to meet the needs of large-scale adolescent scoliosis screening. Considering the multi-source information such as back image and depth data, foot pressure offset and center of gravity change, and back surface electromyographic response, the accuracy and efficiency of scoliosis screening and evaluation can be improved, therefore, it is of urgent practical significance to develop a multi-modal synchronous acquisition dynamic evaluation system for spine. SUMMARY

[0003] In view of the problems that the traditional single image recognition or pressure sensing method often has recognition lag, data deviation or lack of muscle activity information when dealing with complex posture changes, the purpose of the present application is a multi-modal synchronous acquisition dynamic evaluation system for spine, which aims to accurately capture the posture offset and muscle response difference generated by individuals during dynamic actions such as bending over and twisting the torso.

[0004] In order to achieve the above technical purpose, the present application provides a multi-modal synchronous acquisition dynamic evaluation system for spine, comprising:

[0005] a back image and depth data acquisition and analysis module for acquiring the back image of the subject and analyzing the spatial offset trajectory of the spinal midline in the image to generate a first result;

[0006] a double-foot pressure offset and center of gravity monitoring module for acquiring the double-foot pressure distribution data of the subject, calculating the center of gravity offset amount during the posture change of the individual, judging the stability and potential unbalanced behavior, and generating a second result;

[0007] a back surface electromyographic response acquisition module for acquiring the bilateral muscle group activation signal of the subject when the individual bends over or rapidly changes posture, extracting the electromyographic waveform characteristic value, identifying the compensation or synergistic muscle group participation, and generating a third result;

[0008] The multi-modal fusion judgment evaluation module is configured to construct a multi-modal weight adaptive model according to the correlation among the first result, the second result and the third result, and comprehensively judge the posture state of the subject.

[0009] Preferably, the back image and depth data acquisition and analysis module is further configured to use an iToF depth camera based on the principle of structured light to image the back of the subject, acquire dynamic depth vector images of the subject in real time, and automatically calibrate the midline of the spine and analyze the spatial offset trajectory of the midline of the spine through an image processing algorithm.

[0010] Preferably, the image processing algorithm comprises target detection and bounding box extraction, RGB and depth map coordinate transformation, ROI extraction and average depth calculation, and depth difference and ATR angle calculation.

[0011] Preferably, the double-foot pressure offset and center of gravity monitoring module is further configured to acquire double-foot pressure data in real time, analyze the center of gravity offset trajectory, and evaluate the balance ability of the patient in different postures.

[0012] Preferably, the double-foot pressure offset and center of gravity monitoring module is further configured to acquire plantar pressure data through a multi-point pressure sensor array, and draw a center of gravity trajectory graph in real time by using a center of gravity calculation algorithm to determine the offset direction and amplitude.

[0013] Preferably, the back surface electromyographic response acquisition module is further configured to monitor the center of gravity change and muscle activity intensity of the individual in a standing and dynamic state in real time according to the back surface electromyographic signal and the double-foot pressure distribution data, wherein the pressure center is calculated according to the double-foot pressure distribution change, and the real-time center of gravity offset trajectory is estimated in combination with visual information.

[0014] Preferably, the multi-modal fusion judgment evaluation module is further configured to use the ATR angle sequence and the posture offset amount corresponding to the images, the threshold response state and the feedback phase time sequence corresponding to the electromyography, and the center of gravity trajectory length, the offset rate and the symmetry index corresponding to the pressure as input features of the multi-modal weight adaptive fusion model.

[0015] Preferably, the multi-modal fusion judgment evaluation module is further configured to initially use equal-weight average fusion for the weight factor of the multi-modal weight adaptive fusion model, and subsequently train the weight factor according to clinical data or expert labels, and use a multi-layer perception regression module to establish a dynamic judgment system.

[0016] The present application discloses the following technical effects:

[0017] The system of the present application realizes low latency and multi-modal real-time fusion of data acquisition while ensuring high precision, making posture recognition more comprehensive and reliable. Compared with the prior art, the following remarkable effects are achieved:

[0018] The key physiological parameters can be accurately captured at the moment of individual dynamic behavior, and the timeliness of action trigger recognition is improved. The automatic weight adjustment mechanism avoids misjudgment caused by single-mode data distortion, and makes the overall recognition more robust. The integrated module design improves system usability and can be widely used in rehabilitation medicine, intelligent assessment, and old age fall risk screening. The muscle compensation action caused by posture imbalance is effectively identified, and real-time and quantitative data support is provided for muscle functional assessment and intervention. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0020] Figure 1 The figure is a schematic diagram of the system structure described in the present application. DETAILED DESCRIPTION

[0021] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0022] As Figure 1 shown, the present application provides a multi-modal synchronous acquisition spine dynamic evaluation system, including a spine dynamic evaluation system body, a back image and depth data acquisition and analysis module, a double foot pressure offset and gravity center monitoring module, a back surface electromyographic response acquisition module and a multi-modal fusion judgment and evaluation module:

[0023] The multi-modal synchronous acquisition spine dynamic evaluation system acquires the back image and dynamic depth vector image of the subject through the back image and depth data acquisition and analysis module, acquires the double-foot pressure distribution data in real time and calculates the center of gravity offset in the posture change process of the subject in real time through the double-foot pressure offset and center of gravity monitoring module, acquires the 16-channel surface electromyogram of the bilateral muscle groups of the back of the subject when the posture of the subject changes through the back surface electromyogram response acquisition module, dynamically adjusts the analysis weight of each type of data through the multi-modal fusion determination and evaluation module, and assists in spine offset positioning and motion risk evaluation.

[0024] (1) Back image and depth data acquisition and analysis module:

[0025] An iToF depth camera using the principle of structured light images the back of the subject and acquires dynamic depth vector images in real time. The spine midline is automatically calibrated through an image processing algorithm, and the spatial offset trajectory is analyzed.

[0026] (2) Double-foot pressure offset and center of gravity monitoring module:

[0027] Eight piezoelectric sensors are arranged at the bottom of the balance instrument to acquire double-foot pressure distribution data with a 0.01s delay precision. The center of gravity offset in the posture change process of the individual is calculated in real time to judge the stability and potential unbalanced behavior.

[0028] (3) Back surface electromyogram response acquisition module:

[0029] The surface electromyograph is configured with 16 channels, and high-frequency sampling (>2048Hz) is started at the moment when the individual bends over or the body posture changes rapidly. The activation signals of the bilateral muscle groups of the back are recorded synchronously, and the electromyogram waveform characteristic values are extracted for identifying the compensation or synergistic muscle group participation.

[0030] (4) Multi-modal fusion determination and evaluation module:

[0031] A "multi-modal weight adaptive model" is proposed, which dynamically adjusts the analysis weight of each type of data according to the actual acquisition situation. When the balance instrument detects that the center of gravity offset exceeds 5%, the weight of the electromyogram data is automatically increased to deal with the blur problem caused by unstable posture. If the image analysis identifies that the spine height difference is greater than 10mm, the electromyogram channel is activated for further analysis of the muscle compensation degree, which assists in spine offset positioning and motion risk determination. When the electromyogram signal shows that the muscle on one side is continuously over-activated (the electromyogram characteristic value exceeds the threshold value), the weight of the pressure distribution and spine midline offset analysis on the side is automatically increased to identify the potential compensatory posture.

[0032] 1. Back image and depth data acquisition and analysis module:

[0033] An iToF depth camera using the principle of structured light is used to image the back of the subject, and its dynamic depth vector image is collected in real time. The spinal cord midline is automatically calibrated by image processing algorithm, and its spatial offset trajectory is analyzed.

[0034] The image processing algorithm includes:

[0035] Target detection and bounding box extraction: given an RGB image I RGB , the YOLOv8 model is used to detect the human body, and the ROI region is output: Where c (i) = "person"; where B is the total number of detection categories, c (i) represents the detection subcategory.

[0036] RGB and depth map coordinate transformation: let the size of the RGB image be (H rgb , W rgb ), and the size of the depth map be (H d , W d ), then the coordinate scaling factor is:

[0037]

[0038] ROI region mapping to depth map:

[0039]

[0040] Where, respectively represent the left upper corner horizontal coordinate, the left upper corner vertical coordinate, the right upper corner horizontal coordinate and the right lower corner vertical coordinate of the depth map mapping point, s x and s y respectively represent the horizontal coordinate scaling factor and the vertical coordinate scaling factor.

[0041] ROI extraction and average depth calculation:

[0042] Let the center row of the depth image be:

[0043]

[0044] Where r c is the center row of the depth image and H is the height of the depth image.

[0045] Up / down offset o pixels, ROI with height h is intercepted:

[0046]

[0047] Where R top and R bot respectively represent the ROI region and the lower ROI region on the depth image.

[0048] Average depth per column:

[0049]

[0050] where Z top (x) and Z bot (x) represent the average depth of the upper and lower regions on the depth image, respectively.

[0051] Depth difference and ATR angle calculation:

[0052] Depth difference is:

[0053] ΔZ(x) = Z top (x) - Z bot (x) ;

[0054] Average depth is:

[0055]

[0056] where ΔZ(x) represents the depth difference, Z avg (x) represents the average depth difference, Z top (x) and Z bot (x) represent the average depth of the upper and lower regions on the depth image, respectively.

[0057] Unit pixel physical length is:

[0058]

[0059] where scale(x) represents the unit pixel physical length, Z avg (x) represents the average depth difference, f x is the camera focal length.

[0060] ROI vertical pixel spacing is:

[0061] d px = 2o + h

[0062] d mm (x) = d px · scale(x)

[0063] where d px is the distance between the centers of the upper and lower ROI regions, o and h represent the offset pixels and the height of the cut, respectively; d mm (x) represents the actual distance between the upper and lower ROI regions, and scale(x) represents the unit pixel physical length.

[0064] Final ATR tilt angle is:

[0065]

[0066] Where, θ(x) is the ATR angle (unit: radian), ΔZ(x) and d mm (x) represent the depth difference and the distance between the centers of the upper and lower ROI regions, respectively; θ deg (x) represents the ATR angle (unit: degree).

[0067] 2. Foot pressure deviation and center of gravity monitoring module:

[0068] This module is used to collect real-time foot pressure data, analyze the center of gravity deviation trajectory, and evaluate the balance ability of the patient in different postures. The system combines a multi-point pressure sensor array and a high-precision force plate to realize dynamic center of gravity monitoring.

[0069] (1) System composition and structure explanation

[0070] 1) Standing position detection device

[0071] Size: 500mm x 400mm x 100mm

[0072] Sensor distribution: FA+FB (left lower limb), FC+FD (right lower limb)

[0073] Purpose: Collect foot pressure distribution, evaluate standing stability and center of gravity deviation

[0074] (2) Pressure acquisition and center of gravity calculation

[0075] The system collects plantar pressure data through a multi-point pressure sensor array. Using the center of gravity calculation algorithm, the center of gravity trajectory is drawn in real time to determine the deviation direction and amplitude.

[0076] (3) Data processing and feedback mechanism

[0077] The collected data is transmitted to the host computer after being processed by the amplifier box for real-time analysis. The display unit simultaneously displays the pressure distribution graph and the center of gravity trajectory. It supports training feedback mode and assists patients in adjusting their posture in combination with the holding device.

[0078] (4) Technical parameter matching table

[0079] Item Technical requirements Test range and accuracy Left foot, right foot 100 kg, error ≤ ± 5% Maximum amplitude in frontal plane Erect: 11.0 cm, error ≤ ± 2% Maximum amplitude in sagittal plane Erect: 8.0 cm, error ≤ ± 2% Zero point verification Error ≤ ± 0.5 mm Offset accuracy Error ≤ ± 5% Trajectory length accuracy Error ≤ ± 5% Reaction consistency The display unit should accurately reflect the force plate data Force plate consistency Each force plate error ≤ 1% Center of gravity trajectory display Real-time display of center of gravity trajectory Test area identification Clear standing position identification on force plate

[0080] 3. Back surface electromyographic response acquisition module:

[0081] This module aims to monitor the center of gravity change and muscle activity intensity of individuals in standing and dynamic states through back surface electromyographic signals and foot pressure distribution data.

[0082] (1) System composition and structure:

[0083] Acquisition unit: Network ball camera: for posture monitoring and auxiliary center of gravity estimation. Multi-channel wireless surface electromyography acquisition module: collect back muscle group electrical signals.

[0084] Signal processing unit: Amplification box: primary signal amplification. Data receiving box: wireless reception and transfer of electromyography and pressure signals. Isolation power supply: ensure system safety. Desktop computer and display: control program operation and visual display.

[0085] Display and feedback unit: Display interface includes three-state feedback progress bar: contraction state: blue; relaxation state: green; stimulation state: red and flashing indicator light.

[0086] (2) Technical parameters and performance indicators

[0087] Biofeedback instrument performance:

[0088] Item Index Feedback threshold accuracy Center frequency 100 Hz, 5 μV ~ 1000 μV, error ≤ ± 10% Power frequency suppression capability Feedback remains unchanged when 50 Hz sine signal (100 μV) is superimposed Indication accuracy 0.05 μV ~ 5000 μV, error ≤ ± 10% or ± 2 μV (the larger one) Resolution ≤ 2 μV System noise ≤ 1 μV Passband ≥ 15 Hz ~ 1000 Hz Differential mode input impedance ≥ 5 MΩ Common mode rejection ratio ≥ 100 dB Power frequency trap Residual ≤ 5 μV (peak-to-valley value) after trapping Scanning speed 0.1 s / D ~ 20 s / D, error ≤ ± 10%

[0089] (3) Module workflow:

[0090] Signal acquisition: Obtain back surface electromyography and bilateral foot pressure data.

[0091] Data processing: Amplification box processing and filtering power frequency interference through wave trap; data receiving module wireless forwarding to main control.

[0092] State judgment: According to threshold value, display relaxation or contraction or stimulation state.

[0093] Center of gravity shift analysis: Calculate pressure center according to bilateral foot pressure distribution changes; estimate real-time center of gravity shift trajectory combined with visual information.

[0094] Result feedback: Display visual progress bar state and center of gravity trajectory.

[0095] 4. Multimodal fusion determination and evaluation module:

[0096] (1) Model objective

[0097] Based on the correlation between image features, electromyography response, and pressure trajectory, construct a multimodal weight adaptive fusion model to realize comprehensive determination of posture state.

[0098] (2) Feature fusion method

[0099] 1) Input features:

[0100] Image module: ATR angle sequence, posture offset

[0101] Electromyography module: Threshold value response state, feedback phase timing

[0102] Pressure module: barycenter trajectory length, offset rate, symmetry index

[0103] 2) Weight factor: W img , W emg , W prs , initially set as the average, and then dynamically adjusted by training data

[0104] 3) Decision function: S = W img * F img + W emg * F emg + W prs * F prs ; wherein S represents the decision function, W img , W emg , W prs respectively represent the image module weight factor, the electromyography module weight factor and the pressure module weight factor, F img , F emg , F prs respectively represent the image module output result, the electromyography module output result and the balance instrument module output result.

[0105] (3) Model training suggestion: initially use equal weight average fusion, and then train the weight factor according to clinical data or expert label, and a multi-layer perception regression module can be used to establish a dynamic decision system.

[0106] The spine dynamic evaluation system for multi-modal synchronous acquisition provided by the application comprehensively considers multi-source information such as back depth image data, pressure offset and barycenter and back electromyography signals, realizes synchronous acquisition and intelligent judgment of multiple modal data by fusing an iToF depth camera, a pressure sensing balance instrument and a high sampling rate surface electromyography instrument, and designs a multi-modal fusion decision mechanism based on a multi-modal weight self-adaptive model, assists in scoliosis screening, offset positioning and motion risk judgment, solves the problems of low efficiency and strong subjectivity of the current existing scoliosis evaluation method, and effectively improves the accuracy and real-time performance of posture analysis.

[0107] The application is described with reference to flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The device that implements the functions specified in one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0108] In the description of the application, it needs to be understood that the terms "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the application, the meaning of "multiple" is two or more, unless otherwise specifically limited.

[0109] Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application also intends to include these modifications and variations.

Claims

1. A multi-modal simultaneous acquisition dynamic spine assessment system, comprising: Comprise: a back image and depth data acquisition and analysis module for acquiring the back image of a subject and analyzing the spatial offset trajectory of the spinal midline in the image to generate a first result; a double-foot pressure offset and center of gravity monitoring module for acquiring the double-foot pressure distribution data of the subject, calculating the center of gravity offset amount in the individual posture change process, and judging the stability and potential imbalance behavior to generate a second result; a back surface electromyographic response acquisition module for acquiring the bilateral muscle group activation signals of the subject when the individual bends over or the body rapidly changes posture, extracting the electromyographic waveform characteristic value, and identifying the compensation or synergistic muscle group participation to generate a third result; a multi-modal fusion judgment and evaluation module for constructing a multi-modal weight adaptive model according to the correlation between the first result, the second result, and the third result, and comprehensively judging the posture state of the subject.

2. The spinal column dynamic evaluation system of claim 1, wherein: the back image and depth data acquisition and analysis module is further configured to use an iToF depth camera based on the principle of structured light to image the back of the subject, acquire real-time dynamic depth vector images, automatically calibrate the spinal midline through image processing algorithms, and analyze the spatial offset trajectory.

3. The spinal column dynamic evaluation system of claim 2, wherein: the image processing algorithm comprises target detection and bounding box extraction, RGB and depth map coordinate transformation, ROI extraction and average depth calculation, and depth difference and ATR angle calculation.

4. The spinal column dynamic evaluation system of claim 3, wherein: the double-foot pressure offset and center of gravity monitoring module is further configured to acquire real-time double-foot pressure data, analyze the center of gravity offset trajectory, and evaluate the balance ability of the patient in different postures.

5. The spinal column dynamic evaluation system of claim 4, wherein: the double-foot pressure offset and center of gravity monitoring module is further configured to acquire plantar pressure data through a multi-point pressure sensor array, use a center of gravity calculation algorithm to draw a center of gravity trajectory graph in real time, and judge the offset direction and amplitude.

6. The spinal column dynamic evaluation system of claim 5, wherein: the back surface electromyographic response acquisition module is further configured to monitor the center of gravity change and muscle activity intensity of the individual in the standing and dynamic state in real time according to the back surface electromyographic signal and the double-foot pressure distribution data, wherein the pressure center is calculated according to the double-foot pressure distribution change, and the real-time center of gravity offset trajectory is estimated in combination with the visual information.

7. The spinal column dynamic evaluation system of claim 6, wherein: the multi-modal fusion judgment and evaluation module is further configured to use the ATR angle sequence and posture offset amount corresponding to the image, the threshold response state and feedback phase time sequence corresponding to the electromyography, and the center of gravity trajectory length, offset rate, and symmetry index corresponding to the pressure as input features of the multi-modal weight adaptive fusion model.

8. The multi-modal synchronous acquisition and dynamic evaluation system of a spine according to claim 7, characterized in that: The multi-modal fusion determination and evaluation module is further configured to initially adopt equal-weight average fusion for the weight factors of the multi-modal weight adaptive fusion model, and subsequently train the weight factors according to clinical data or expert labels, and establish a dynamic determination system using a multi-layer perception regression module.

Images