Image recognition-based scoliosis three-dimensional posture training feedback method and device

Abstract

This invention provides a three-dimensional posture training feedback method and device for scoliosis based on image recognition, belonging to the fields of medical rehabilitation and image recognition technology. The method includes the following steps: acquiring depth and color images of the patient, reconstructing a three-dimensional skeletal model and comparing it with a standard posture model, inputting the data into a trained deep learning model and outputting a personalized training plan; extracting key spinal feature points, combining them with the patient's physiological parameter data, and calculating real-time movement evaluation values ​​using a weighted algorithm; providing visual / auditory feedback through smart glasses, superimposing standard virtual images; updating the training plan when the movement evaluation value is not lower than a threshold within a continuous period, thus achieving dynamic personalized rehabilitation. This invention effectively improves the accuracy of training movements through precise real-time feedback and personalized guidance, prevents secondary injuries caused by improper movements, significantly enhances rehabilitation effects, and ultimately controls and reverses the progression of scoliosis.

Description

Technical Field

[0001] This invention relates to the fields of medical rehabilitation and image recognition technology, specifically to a three-dimensional posture training feedback method and device for scoliosis based on image recognition. Background Technology

[0002] Scoliosis is a common three-dimensional spinal deformity in adolescents, with an incidence rate of approximately 1% to 3%. Idiopathic scoliosis in adolescents is the most common type, accounting for about 80%. If this condition is not detected and intervened in its early stages, it can develop into a severe structural deformity, leading to multiple consequences such as limited cardiopulmonary function, nerve compression, physical deformities, and psychological problems. Therefore, early screening and accurate identification are of crucial clinical significance for preventing the condition from worsening and developing personalized intervention strategies. Currently, patients often perform training at home on their own, lacking real-time guidance and feedback from professional therapists. This can easily lead to inaccurate movements, affecting the treatment's effectiveness and even causing secondary injuries.

[0003] Current mainstream rehabilitation treatments for scoliosis include bracing, manual therapy, and specific exercise training. Among these, specific exercise training, unlike general symmetrical exercise training, often targets three-dimensional spinal abnormalities. The movements are more complex and require one-on-one supervision and guidance from a therapist. This treatment method suffers from high personalization requirements, high costs, and limited applicability in home settings. Therefore, developing a low-cost, non-contact, easy-to-use training device that provides accurate real-time feedback has significant clinical importance and market demand. The invention patent with publication number CN118866243A provides a scoliosis correction exercise monitoring system and method based on visual feedback. The system includes: a standard movement storage unit for storing standard scoliosis exercise movements and tutorials; an appearance capture and posture monitoring device, including but not limited to motion sensors and infrared sensors, for real-time monitoring of the scoliosis patient's posture and appearance information; a data processing device for receiving data from the appearance capture and posture monitoring device, processing and analyzing it, generating real-time exercise parameters for the patient, and simultaneously generating visual feedback signals; and a visualization feedback device for receiving signals from the data processing device, visually displaying the patient's real-time posture information and projected images, and guiding the patient in targeted exercises. However, this system requires physicians to set treatment goals, and the entire system is bulky, making it unsuitable for patients to perform regular home treatment.

[0004] In view of this, it is necessary to study a new type of compact, lightweight, safe, simple, and widely applicable three-dimensional posture training real-time feedback device and training real-time feedback method for scoliosis, so as to save labor costs and ensure the standardization and effectiveness of exercise therapy, in order to solve the above-mentioned technical problems. Summary of the Invention

[0005] In view of the technical problem of existing scoliosis three-dimensional posture training lacking a monitoring feedback device, and based on meeting the needs of scoliosis patients for efficient home training, this invention provides a scoliosis three-dimensional posture training feedback method and device based on image recognition. This method can effectively help patients improve training efficiency, thereby controlling and reversing the progression of scoliosis. This training feedback device does not require any wearable sensors. It captures the patient's posture using a regular camera and a depth camera, and reconstructs the three-dimensional joints (three-dimensional skeletal model) of the human body using image recognition algorithms. By comparing the patient's real-time posture with a standard treatment posture model (standard posture model), it provides the patient with intuitive, real-time visual and auditory feedback through VR glasses. Combined with deep learning to provide the patient with a training plan, it guides them to perform accurate rehabilitation training, effectively improving the efficacy of home training.

[0006] In a first aspect, the present invention provides a three-dimensional posture training feedback method for scoliosis based on image recognition, comprising the following steps: S1, acquires the patient's depth and color image information; S2 reconstructs the patient's three-dimensional skeletal model using image recognition algorithms, compares and analyzes it with a standard posture model, and inputs the analysis results into a trained deep learning model to output a personalized training plan for the patient's current condition. S3, extract key skeletal feature points of the patient's spine, calculate the differences between the key skeletal feature points and the standard posture model in terms of angle, displacement, and rotation, and consider the patient's physiological parameter data to calculate the patient's real-time motion assessment value through a weighted algorithm; S4. Through smart glasses worn by the patient, visual and / or auditory feedback of a personalized training plan is provided to the patient; and the standard posture model and standard rehabilitation movement path are superimposed on the patient's real field of vision in the form of virtual images; S5, in each personalized training session within a continuous cycle, when all real-time action evaluation values ​​are not lower than the preset threshold, it is determined that the patient has mastered each action in the initial personalized training plan. As a result, the patient's three-dimensional skeletal model is rebuilt, and an updated personalized training plan is output.

[0007] As a further improvement of the present invention, in step S2, the process of establishing and training the deep learning model includes the following steps: S21, the collected historical patient data is used as a dataset. The historical patient data includes the angle difference, rotation difference, displacement difference and Cobb angle value of the patient and the standard posture, as well as the corresponding training plan, and is normalized. The training plan is represented by numerical code. S22, input the normalized data into the input layer, and divide the data into features X=[x1,x2,x3] and corresponding labels Y=[y1,y2]; Where x1 represents the angle difference between the patient and the standard posture; x2 represents the difference in rotation between the patient and the standard posture; x3 represents the displacement difference between the patient and the standard posture; y1 represents the training plan; y2 represents the patient's Cobb angle value; S23 uses multiple hidden layers to extract features from the input data; S24, outputs the patient's training plan and Cobb angle value. The output layer maps the features of the hidden layer onto the patient's training plan and Cobb angle value, using the following formula: (Equation 1) in, This indicates the anticipated training plan for the patient. This indicates the estimated Cobb angle value for the patient; Represents the ReLU function; W1', W2' These represent the weights of the output layer; b1' and b2' represent the biases of the output layer, respectively; S25, Design a loss function to optimize the model so that the output value of the deep learning model is close to the actual training plan of the patient, as shown below: (Equation 2) in, These represent the patient's training plan and the true label corresponding to the Cobb angle value, respectively. These represent the patient's training plan and Cobb angle value output by the model, respectively. α and β represent the weights of the loss function, respectively; N Indicates the number of times the patient has received training; S26. The Adam optimizer is used to update the model's weights and biases to minimize the loss function and obtain a trained deep learning model.

[0008] As a further improvement of the present invention, the weighting algorithm in step S3 is as follows: (Equation 3) in, score This represents the real-time motion evaluation value; Represents the physiological optimization function; , , , These represent the differences between the patient and the standard posture model during training in terms of angle, displacement, rotation, and Cobb angle, respectively. , , , These represent the fractions deducted for each unit of deviation in angle, displacement, rotation, and Cobb angle, respectively.

[0009] As a further improvement of the present invention, the physiological parameter data includes the patient's age, height, and gender; the physiological optimization function is as follows: (Equation 4) Where k_age(x) represents the age stiffness coefficient, defined as: Where x represents age; Where s_height(Z) represents the height scaling factor, defined as: s_height(Z) = 1 + 0.05 × Z; Z represents the Z-score standardized value of height, calculated by the formula Z = (H-μ) / σ, where μ represents the average height of the same age group, σ represents the standard deviation, and H represents the patient's height. Where w_gender(s) represents the gender weighting coefficient, which is defined as: s represents the patient's gender, s=1 represents female, and s=0 represents male; m_cobb(y) The patient's Cobb angle weighting coefficient is defined as follows: ; y represents the patient's Cobb angle.

[0010] As a further improvement of the present invention, a three-dimensional convolutional neural network is used to establish a three-dimensional skeletal model of the patient.

[0011] As a further improvement of the present invention, the method further includes step S6, as follows: repeating the training process from step S3 to step S5 until the reconstructed three-dimensional skeletal model of the patient approaches the standard posture model.

[0012] Secondly, the present invention also provides a three-dimensional posture training feedback device for scoliosis based on image recognition, which uses the above-mentioned training feedback method for training feedback, including: The image acquisition module includes a depth camera and a regular camera, which are used to acquire the patient's depth image information and color image information, respectively; The data processing and control module is electrically connected to the image acquisition module and is used to receive image information, reconstruct the patient's three-dimensional skeletal model, and output a personalized training plan based on the patient's current condition based on a deep learning model. The intelligent scoring module is electrically connected to the data processing and control module. It is used to extract key skeletal feature points of the patient's spine, calculate the differences between the key skeletal feature points and the standard posture model in terms of angle, displacement, and rotation parameters, and calculate the patient's real-time action evaluation value considering the patient's physiological parameters. The smart glasses module is electrically connected to the data processing and control module and the intelligent scoring module. It is worn by the patient to provide the patient with visual and / or auditory feedback for a personalized training plan, and to overlay standard posture models and standard rehabilitation movement paths in the form of virtual images into the patient's real field of vision. The plan update module is electrically connected to the data processing and control module and the intelligent scoring module, respectively. It is used to determine that the patient has mastered each action in the personalized training plan in each training session within a continuous cycle of the personalized training plan, when all real-time action evaluation values ​​in the intelligent scoring module are not lower than a preset threshold. The module then rebuilds the patient's three-dimensional skeletal model and outputs the updated personalized training plan.

[0013] As a further improvement of the present invention, the data processing and control module includes a reconstruction unit and a deep learning unit; the reconstruction unit is used to receive image information, reconstruct a three-dimensional skeletal model of the patient, and compare and analyze it with a standard posture model, and then input the comparison and analysis results into the deep learning model trained by the deep learning unit; the deep learning unit outputs a personalized training plan for the patient's current condition.

[0014] As a further improvement of the present invention, the intelligent scoring module includes a feature extraction unit and a motion evaluation unit; the feature extraction unit is used to extract key skeletal feature points of the patient's spine; the motion evaluation unit calculates the differences between the key skeletal feature points and the standard posture model in terms of angle, displacement, and rotation parameters, and takes into account the patient's age, height, and gender physiological parameters, and calculates the real-time motion evaluation value through a weighted algorithm.

[0015] Thirdly, the present invention also provides the application of the above-mentioned image recognition-based three-dimensional posture training feedback device for scoliosis in three-dimensional posture training.

[0016] Compared with the prior art, the present invention has the following advantages and beneficial effects: Non-contact monitoring: Patients do not need to wear any sensors; their posture can be captured by a camera, improving the convenience and comfort of use.

[0017] Real-time 3D reconstruction: The patient's 3D skeletal model is reconstructed in real time using image recognition algorithms, providing a technical basis for accurate posture assessment.

[0018] Intuitive visual feedback: By overlaying standard posture models with real-time postures through smart glasses, patients can intuitively see movement deviations, as if a professional therapist were guiding them.

[0019] Intelligent scoring system: Generates a comprehensive score through multi-dimensional difference analysis to provide patients with a quantitative assessment of training effectiveness.

[0020] Personalized training program: Based on a deep learning model, the training plan is dynamically adjusted according to the patient's specific condition and training progress to achieve truly personalized rehabilitation.

[0021] Suitable for home and clinical use: The device has a simple structure and is easy to operate. It can be used for hospital rehabilitation treatment as well as for daily home training, which greatly reduces rehabilitation costs.

[0022] Improve training effectiveness: Through precise real-time feedback and personalized guidance, the accuracy of training movements can be effectively improved, secondary injuries caused by improper movements can be prevented, and rehabilitation results can be significantly enhanced.

[0023] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0024] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the present invention will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0025] Figure 1 This is a flowchart of the steps of the image recognition-based three-dimensional posture training feedback method for scoliosis provided in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the structure of the image recognition-based three-dimensional posture training feedback device for scoliosis provided in Embodiment 2 of the present invention. Detailed Implementation

[0026] The embodiments of the technical solution of the present invention will now be described in detail with reference to the accompanying drawings. These embodiments are merely illustrative of the technical solution of the present invention and are therefore intended to limit the scope of protection of the present invention.

[0027] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the invention, are intended to cover non-exclusive inclusion.

[0028] In the description of the embodiments of this invention, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this invention, "multiple" means two or more, unless otherwise explicitly defined.

[0029] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0030] In the description of the embodiments of this invention, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0031] In the description of the embodiments of the present invention, the term "multiple" refers to two or more (including two), similarly, "multiple groups" refers to two or more (including two groups), and "multiple pieces" refers to two or more (including two pieces).

[0032] In the description of the embodiments of the present invention, the technical terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of the present invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the embodiments of the present invention.

[0033] In the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the technical terms such as "installation," "connection," "joining," and "fixing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of the present invention according to the specific circumstances.

[0034] To address the technical problem of the lack of monitoring and feedback devices in existing scoliosis three-dimensional posture training, and based on meeting the needs of scoliosis patients for efficient home training, this invention provides a scoliosis three-dimensional posture training feedback method and device based on image recognition. This method can effectively help patients improve training efficiency, thereby controlling and reversing the progression of scoliosis. This training feedback device does not require any wearable sensors. It captures the patient's posture using a regular camera and a depth camera, and reconstructs the three-dimensional joints (three-dimensional skeletal model) of the human body using image recognition algorithms. By comparing the patient's real-time posture with a standard treatment posture model (standard posture model), it provides the patient with intuitive, real-time visual and auditory feedback, guiding them to perform accurate rehabilitation training and effectively improving the efficacy of home training.

[0035] Example 1 Please see Figure 1 As shown, the image recognition-based three-dimensional posture training feedback method for scoliosis provided by this invention includes the following steps: S1, acquires the patient's depth and color image information. The color image is taken by a regular camera, and the depth image is taken by a depth camera and contains depth information (three-dimensional information).

[0036] S2 reconstructs a three-dimensional skeletal model of the patient using an image recognition algorithm and a three-dimensional convolutional neural network, and compares and analyzes it with a standard posture model (standard three-dimensional spinal model); the analysis results are input into a trained deep learning model, and finally output a personalized training plan for the current patient condition.

[0037] Step 2, the process of building and training the deep learning model, includes the following steps: S21, the collected historical patient data is used as a dataset. The historical patient data includes the angle difference, rotation difference, displacement difference and Cobb angle value of the patient and the standard posture, as well as the corresponding training plan, which is normalized. The training plan is represented by numerical code. For example, each historical patient data set includes the corresponding training plan for the patient. Identical training plans are categorized, and they are sorted according to the proportion of different training plans. The ranking order is used as the numerical code for the training plan.

[0038] S22, input the normalized data into the input layer, and divide the data into features X=[x1,x2,x3] and corresponding labels Y=[y1,y2]; Where x1 represents the angle difference between the patient and the standard posture; x2 represents the difference in rotation between the patient and the standard posture; x3 represents the displacement difference between the patient and the standard posture; y1 represents the training plan; y2 represents the patient's Cobb angle value.

[0039] S23 uses multiple hidden layers to extract features from the input data.

[0040] S24 outputs the patient's training plan and Cobb angle value. The output layer maps the features of the hidden layer onto the patient's training plan and Cobb angle value, as shown in the formula: (Equation 1) in, This indicates the anticipated training plan for the patient. This indicates the estimated Cobb angle value for the patient; Represents the ReLU function; W1' and W1' represent the weights of the output layer; b1' and b2' represent the bias of the output layer; S25, Design a loss function to optimize the model so that its output value closely approximates the actual patient's condition when using the training plan, as shown below: (Equation 2) in, This represents the true labels corresponding to the patient's training plan and Cobb angle value; This represents the patient's training plan and Cobb angle value as output by the model; α and β represent the weights of the loss function; N This indicates the number of times the patient has received training.

[0041] Introducing the Cobb angle as a key evaluation metric into the loss function aims to significantly improve the accuracy and clinical suitability of the model's output training plan.

[0042] Specifically, by quantitatively comparing the Cobb angle predicted by the model with the Cobb angle measured in actual clinical settings, and using the deviation between the two as an important component of the loss function, clear guidance and constraints can be applied to the model optimization process.

[0043] S26 uses the Adam optimizer to update the model's weights and biases to minimize the loss function.

[0044] S3 extracts key skeletal feature points of the patient's spine, calculates the differences between them and the standard posture model in terms of angle, displacement, and rotation, and calculates real-time motion evaluation values ​​by weighting the patient's age, height, and gender.

[0045] When extracting spinal feature points and comparing them with standard posture models, physiological parameters such as patient age, height, and gender must be considered. Age affects skeletal development, degeneration, and postural control; standard postures differ across age groups, and spinal posture changes with age. Height determines spinal length and mechanical load; proportional differences need to be normalized, otherwise, the accuracy of assessments of angles, displacements, and other indicators will be directly affected. Gender brings differences in skeletal structure, pelvic shape, spinal curvature, and muscle strength, and women are significantly affected by hormones, resulting in significant postural changes at different physiological stages. Therefore, the weighted algorithm dynamically adjusts accordingly to make the comprehensive real-time motion assessment value more accurate.

[0046] Therefore, based on the patient's physiological parameter data and comparative data, the weighted algorithm in step S3 is calculated as follows: (Equation 3) Wherein, score represents the real-time motion evaluation value; Represents the physiological optimization function; , , , These represent the differences between the patient and the standard posture model during training in terms of angle, displacement, rotation, and Cobb angle, respectively. , , , These represent the fractions deducted for each unit of deviation in angle, displacement, rotation, and Cobb angle, respectively.

[0047] For example, 5 points are deducted for every degree of angular deviation from the standard posture model. The value is 5; for every centimeter of deviation from the standard posture model in displacement, 3 points are deducted. The value is 3; for every degree of deviation from the standard pose model in rotation, 4 points are deducted, so at this time... The value is 4; for every degree of deviation from the standard pose model in the Cobb angle, 2 points are deducted, and at this time... The value is 2. It should be noted that the above is just an example; in actual training, the deduction score can be flexibly set according to clinical needs.

[0048] The physiological optimization function is as follows: (Equation 4) Where k_age(x) represents the age stiffness coefficient, which is defined as: ; x represents age; the age stiffness coefficient indicates that the younger the age, the more severe the deduction should be, to ensure that more rigorous correction can be achieved, and to strive to achieve complete correction of scoliosis at a young age.

[0049] Where s_height(Z) represents the height scaling factor, which is defined as: s_height(Z) = 1 + 0.05 × Z, Where Z represents the Z-score standardized value of height, and the calculation formula is Z = (H - μ) / σ, where μ represents the average height of the same age group and σ represents the standard deviation. The height scaling factor indicates that the taller the patient, the more stringent the correction should be, because the pressure on the spine increases with height, making correction more difficult compared to shorter patients.

[0050] Where w_gender(s) represents the gender weighting coefficient, which is defined as: s represents its gender, s=1 represents female, s=0 represents male; m_cobb(y) The patient's Cobb angle weighting coefficient is defined as follows: ; y represents the patient's Cobb angle.

[0051] Epidemiological surveys show that the incidence rate is significantly higher in women than in men, especially in moderate to severe scoliosis, where the male-to-female ratio can reach 5.4:1. This difference is related to factors such as women's poorer spinal biomechanical stability, higher ligament laxity, and hormonal fluctuations during puberty. In algorithm design, female patients should be given a higher gender weighting coefficient to enable them to more strictly adhere to the training plan.

[0052] S4 provides patients with personalized training plans through smart glasses worn on their bodies, providing visual and / or auditory feedback; and overlays standard 3D spinal models and standard rehabilitation movement paths as virtual images onto the patient's real field of vision.

[0053] S5, in each training session within a continuous cycle, when the evaluation value of all actions is not lower than the preset threshold, it is determined that the patient has mastered all the actions in the current training plan, the patient's three-dimensional skeletal model is rebuilt and the updated personalized training plan is output.

[0054] The system generates a comprehensive score from 0 to 100 based on real-time motion assessment values ​​and displays it in a corner of the smart glasses. When the patient's training reaches the set conditions, the system automatically updates the training plan. As the patient continues training, their scoliosis will constantly change. Once the patient has mastered the current plan, the training plan is updated based on their current scoliosis condition, achieving dynamic updates to the personalized training program.

[0055] The training plan is updated when the score of all training movements is not lower than a preset threshold during a consecutive week or two of training. The threshold can be set according to actual needs, such as 85 points. This method ensures that the device system will not easily adjust the plan because of a high score in a single training session, effectively avoiding the risk of blindly advancing due to accidental good performance, and ensuring the stability and safety of the patient's rehabilitation process.

[0056] In some specific implementations, step S6 is also included, as follows: repeat the training process from step S3 to step S5 until the reconstructed three-dimensional skeletal model of the patient approaches the standard posture model and reaches the recovery level.

[0057] Example 2 Please see Figure 2 As shown, Embodiment 2 of the present invention also provides a three-dimensional posture training feedback device for scoliosis based on image recognition. It uses the above-mentioned training feedback method to perform three-dimensional posture training feedback for scoliosis and mainly includes: an image acquisition module, a data processing and control module, a smart glasses module, an intelligent scoring module, and a plan update module.

[0058] The image acquisition module includes a depth camera and a regular camera, used to acquire depth and color image information of the patient, respectively.

[0059] The data processing and control module, electrically connected to the image acquisition module, receives image information, reconstructs a three-dimensional skeletal model of the patient using an image recognition algorithm, and compares and analyzes it with a standard posture model. The analysis results are then input into a trained deep learning model, which outputs a personalized training plan tailored to the patient's current condition. Specifically, the data processing and control module includes a reconstruction unit and a deep learning unit. The reconstruction unit receives image information, reconstructs a three-dimensional skeletal model of the patient, compares and analyzes it with a standard posture model, and then inputs the comparison analysis results into the deep learning model trained by the deep learning unit. The deep learning unit outputs a personalized training plan tailored to the patient's current condition.

[0060] The intelligent scoring module is electrically connected to the data processing and control module and the smart glasses module, respectively. It is used to extract key skeletal feature points of the patient's spine, including important anatomical landmarks such as vertebrae, acromion, and iliac crest, and calculate the differences between these features and the standard posture model in terms of angle, displacement, and rotation. A weighted algorithm is then used to calculate a real-time motion evaluation value. Specifically, the intelligent scoring module includes a feature extraction unit and a motion evaluation unit. The feature extraction unit extracts key skeletal feature points of the patient's spine. The motion evaluation unit calculates the differences between these key skeletal feature points and the standard posture model in terms of angle, displacement, and rotation parameters, taking into account the patient's age, height, and gender physiological parameters, and calculates a real-time motion evaluation value using a weighted algorithm.

[0061] The smart glasses module is electrically connected to the data processing and control module and is worn by the patient to provide the patient with visual and / or auditory feedback for a personalized training plan; and to overlay a standard three-dimensional model of the spine (standard posture model) and a standard rehabilitation movement path in the form of virtual images onto the patient's real field of vision.

[0062] The smart glasses are AR (Augmented Reality) glasses, connecting to the data processing and control module via wireless communication methods (such as Wi-Fi and Bluetooth). The smart glasses display can show a standard 3D model of the spine (standard posture model), rehabilitation movement paths, real-time scoring information, etc.

[0063] The plan update module is electrically connected to the data processing and control module and the intelligent scoring module, respectively. It is used to determine that the patient has mastered all the actions in the current training plan in each training session within a continuous cycle, when the evaluation value of all actions is not lower than the preset threshold. The module then rebuilds the patient's three-dimensional skeletal model and outputs the updated personalized training plan.

[0064] During training, the image acquisition module captures the patient's posture images in real time. The data processing and control module reconstructs the patient's 3D skeletal model using image recognition algorithms and outputs a personalized training plan. The smart glasses overlay the standard posture model as a virtual image onto the patient's real field of vision, simultaneously displaying the patient's real-time posture contour line. Two different colors are used to represent the contour line of the standard posture model and the patient's real-time posture contour line, for example, green for the standard posture model and blue for the patient's real-time posture contour line, for real-time overlay comparison. Furthermore, areas of deviation are highlighted with a striking color (such as red) and indicated by animated arrows in the direction requiring adjustment.

[0065] Meanwhile, the audio feedback unit of the smart glasses module provides voice prompts and guidance, playing concise voice commands such as "tilt your body 5 degrees to the left" to guide the patient in making fine adjustments. When the patient's posture falls within the allowable error range of the standard posture, the system provides positive feedback, such as displaying a green "correct" indicator and playing encouraging sound effects, thereby helping the patient form muscle memory and complete high-quality training. This intuitive visual feedback allows the patient to immediately perceive movement deviations, as if a rehabilitation therapist were providing visual guidance, improving the patient's training efficiency.

[0066] The intelligent scoring module generates a comprehensive score through multi-dimensional difference analysis, providing patients with a quantitative assessment of training effectiveness. The plan update module, based on real-time feedback from the intelligent scoring module, reconstructs the patient's three-dimensional skeletal model and dynamically adjusts the personalized training plan according to the patient's specific condition and training progress, achieving truly personalized rehabilitation.

[0067] The image recognition-based three-dimensional posture training feedback device for scoliosis provided in this embodiment can provide precise, convenient, and personalized rehabilitation training guidance for scoliosis patients, effectively improve training results, and has significant clinical application value and market prospects.

[0068] Example 3 The present invention also provides the application of the above-mentioned image recognition-based three-dimensional posture training feedback device for scoliosis in three-dimensional posture training. The above-mentioned image recognition-based three-dimensional posture training feedback method for scoliosis is used for training and real-time feedback, thereby realizing the application of rehabilitation training.

[0069] In summary, this invention provides a three-dimensional posture training feedback method and device for scoliosis based on image recognition, belonging to the fields of medical rehabilitation and image recognition technology. The method includes the following steps: acquiring depth and color images of the patient; reconstructing a three-dimensional skeletal model and comparing it with a standard posture model; inputting the data into a trained deep learning model and outputting a personalized training plan; extracting spinal feature points; combining the patient's age, height, and gender; and calculating real-time movement evaluation values ​​using a weighted algorithm; providing visual / auditory feedback through smart glasses, overlaying standard virtual images; and updating the training plan when the movement evaluation value is not lower than a threshold within a continuous period, thus achieving dynamic personalized rehabilitation. This invention can effectively help patients improve training efficiency, thereby controlling and reversing the progression of scoliosis. Through precise real-time feedback and personalized guidance, it effectively improves the accuracy of training movements, prevents secondary injuries caused by improper movements, and significantly enhances rehabilitation outcomes.

[0070] It should be noted that the present invention is not limited to the above-described embodiments. The above embodiments are merely examples, and any embodiments that have the same structure and perform the same effects as the technical concept within the scope of the present invention are included within the scope of the present invention. Furthermore, various modifications that can be conceived by those skilled in the art to the embodiments, and other ways of constructing by combining some of the constituent elements of the embodiments, without departing from the spirit of the present invention, are also included within the scope of the present invention.

Claims

1. A three-dimensional posture training feedback method for scoliosis based on image recognition, characterized in that... By recognizing patient posture information from images, patients are guided to conduct self-rehabilitation training, which includes the following steps: S1, acquires the patient's depth and color image information; S2 reconstructs the patient's three-dimensional skeletal model through image recognition algorithms, compares and analyzes it with a standard posture model, inputs the analysis results into a trained deep learning model, and outputs a personalized training plan for the patient's current condition. S3, extract key skeletal feature points of the patient's spine, calculate the differences between the key skeletal feature points and the standard posture model in terms of angle, displacement, and rotation, and consider the patient's physiological parameter data to calculate the patient's real-time motion assessment value through a weighted algorithm; S4. Through smart glasses worn by the patient, visual and / or auditory feedback of a personalized training plan is provided to the patient; and the standard posture model and standard rehabilitation movement path are superimposed on the patient's real field of vision in the form of virtual images; S5, in each personalized training session within a continuous cycle, when all real-time action evaluation values ​​are not lower than the preset threshold, it is determined that the patient has mastered each action in the personalized training plan. Therefore, the patient's depth image information and color image information are re-acquired, a three-dimensional skeletal model of the patient is established, and an updated personalized training plan is output.

2. The image recognition-based three-dimensional posture training feedback method for scoliosis according to claim 1, characterized in that, Step S2, the process of establishing and training the deep learning model, includes the following steps: S21, the collected historical patient data is used as a dataset. The historical patient data includes the angle difference, rotation difference, displacement difference and Cobb angle value of the patient and the standard posture, as well as the corresponding training plan, and is normalized. The training plan is represented by numerical code. S22, input the normalized data into the input layer, and divide the data into features X=[x1,x2,x3] and corresponding labels Y=[y1,y2]; Where x1 represents the angle difference between the patient and the standard posture; x2 represents the difference in rotation between the patient and the standard posture; x3 represents the displacement difference between the patient and the standard posture; y1 represents the training plan; y2 represents the patient's Cobb angle value; S23 uses multiple hidden layers to extract features from the input data; S24, outputs the patient's training plan and Cobb angle value. The output layer maps the features of the hidden layer onto the patient's training plan and Cobb angle value, using the following formula: (Equation 1) in, This indicates the anticipated training plan for the patient. This indicates the estimated Cobb angle value for the patient. When the Cobb angle value is >10°, interventional training is required. Represents the ReLU function; W1', W2' These represent the weights of the output layer; b1' and b2' represent the biases of the output layer, respectively; S25, Design a loss function to optimize the model so that the output value of the deep learning model is close to the actual training plan of the patient, as shown below: (Equation 2) in, These represent the patient's training plan and the true label corresponding to the Cobb angle value, respectively. These represent the patient's training plan and Cobb angle value output by the model, respectively. α and β represent the weights of the loss function, respectively; N Indicates the number of times the patient has received training; S26. The Adam optimizer is used to update the model's weights and biases to minimize the loss function and obtain a trained deep learning model.

3. The image recognition-based three-dimensional posture training feedback method for scoliosis according to claim 1, characterized in that, In step S3, the weighting algorithm is as follows: ； (Equation 3) in, score This represents the real-time motion evaluation value; Represents the physiological optimization function; , , , These represent the differences between the patient and the standard posture model during training in terms of angle, displacement, rotation, and Cobb angle, respectively. , , , These represent the fractions deducted for each unit of deviation in angle, displacement, rotation, and Cobb angle, respectively.

4. The image recognition-based three-dimensional posture training feedback method for scoliosis according to claim 3, characterized in that, The physiological parameter data includes the patient's age, height, and gender; the physiological optimization function is shown below: (Equation 4) Where k_age(x) represents the age stiffness coefficient, defined as: Where x represents age; Where s_height(Z) represents the height scaling factor, defined as: s_height(Z) = 1 + 0.05 × Z; Z represents the Z-score standardized value of height, calculated by the formula Z = (H-μ) / σ, where μ represents the average height of the same age group, σ represents the standard deviation, and H represents the patient's height. Where w_gender(s) represents the gender weighting coefficient, which is defined as: s represents the patient's gender, s=1 represents female, and s=0 represents male; m_cobb(y) The patient's Cobb angle weighting coefficient is defined as follows: ; y represents the patient's Cobb angle.

5. The image recognition-based three-dimensional posture training feedback method for scoliosis according to claim 1 uses a three-dimensional convolutional neural network to establish a three-dimensional skeletal model of the patient, and repeats the training process from step S3 to step S5 until the reconstructed three-dimensional skeletal model of the patient approaches the standard posture model.

6. A three-dimensional posture training feedback device for scoliosis based on image recognition, characterized in that, Training feedback is performed using the image recognition-based three-dimensional posture training feedback method for scoliosis as described in any one of claims 1 to 5, including: The image acquisition module includes a depth camera and a regular camera, which are used to acquire the patient's depth image information and color image information, respectively; The data processing and control module is electrically connected to the image acquisition module and is used to receive image information, reconstruct the patient's three-dimensional skeletal model, and output a personalized training plan based on the patient's current condition based on a deep learning model. The intelligent scoring module is electrically connected to the data processing and control module. It is used to extract key skeletal feature points of the patient's spine, calculate the differences between the key skeletal feature points and the standard posture model in terms of angle, displacement, and rotation parameters, and calculate the patient's real-time action evaluation value considering the patient's physiological parameters. The smart glasses module is electrically connected to the data processing and control module and the intelligent scoring module. It is worn by the patient to provide the patient with visual and / or auditory feedback for a personalized training plan, and to overlay standard posture models and standard rehabilitation movement paths in the form of virtual images into the patient's real field of vision. The plan update module is electrically connected to the data processing and control module and the intelligent scoring module, respectively. It is used to determine that the patient has mastered each action in the personalized training plan in each training session within a continuous cycle of the personalized training plan, when all real-time action evaluation values ​​in the intelligent scoring module are not lower than a preset threshold. The module then rebuilds the patient's three-dimensional skeletal model and outputs the updated personalized training plan.

7. The image recognition-based three-dimensional posture training feedback device for scoliosis according to claim 6, characterized in that, The data processing and control module includes a reconstruction unit and a deep learning unit. The reconstruction unit receives image information, reconstructs a three-dimensional skeletal model of the patient, compares and analyzes it with a standard pose model, and then inputs the comparison and analysis results into the deep learning model trained by the deep learning unit. The deep learning unit outputs a personalized training plan for the patient's current condition.

8. The image recognition-based three-dimensional posture training feedback device for scoliosis according to claim 6, characterized in that, The intelligent scoring module includes a feature extraction unit and a motion evaluation unit. The feature extraction unit is used to extract key skeletal feature points of the patient's spine. The motion evaluation unit calculates the differences between the key skeletal feature points and the standard posture model in terms of angle, displacement, and rotation parameters, and takes into account the patient's age, height, and gender physiological parameters, and calculates the real-time motion evaluation value through a weighted algorithm.

9. The application of the image recognition-based three-dimensional posture training feedback device for scoliosis as described in any one of claims 6 to 8, characterized in that, The application of the image recognition-based three-dimensional posture training feedback device for scoliosis in three-dimensional posture training.

Images