Balance evaluation measurement sensing system based on FTIR (Fourier transform infrared spectroscopy) and multi-modal AI (artificial intelligence) analysis

Abstract

The invention discloses a balance evaluation measurement sensing system based on FTIR and multi-modal AI analysis, and relates to the technical field of sensing equipment, the system comprises a platform body, an FTIR balance measurement unit, a double-path mirror image folding optical acquisition module and a calculation analysis unit; the sensing system is compact in structure and good in stability, supports multi-posture (standing / sitting) evaluation and can synchronously obtain high-resolution pressure distribution images of the two feet, the sensing system analyzes plantar pressure images based on an FTIR optical source by utilizing a calculation and analysis unit in combination with multi-modal artificial intelligence, and the sensing precision is improved. Precise identification and quantitative evaluation of scoliosis related somatosensory defects are achieved, and the defects that an existing balance sensor is large in size, insufficient in safety and the like can be overcome.

Description

Technical Field

[0001] This application relates to the field of sensing device technology, and in particular to a balance evaluation measurement sensing system based on FTIR and multimodal AI analysis. Background Technology

[0002] Patients with scoliosis (AIS) often have proprioceptive deficits and balance disorders. Current clinical assessments rely heavily on physician experience, X-rays (which involve radiation), or expensive motion capture systems, making it difficult to conduct rapid, low-cost, and repetitive screening and monitoring of treatment outcomes in community settings, homes, or general clinics. While balance devices based on pressure plates exist, their accuracy and resolution are limited, as are their assessment capabilities. Summary of the Invention

[0003] The main objective of this application is to propose a balance assessment measurement sensing system based on FTIR and multimodal AI analysis to improve the measurement resolution and assessment capability of somatosensory defects related to scoliosis.

[0004] To achieve the above objectives, this application proposes a balance evaluation measurement sensing system based on FTIR and multimodal AI analysis, including: a platform body, an FTIR balance measurement unit, a dual-path mirror folding optical acquisition module, and a calculation and analysis unit; The FTIR balance measurement unit includes two optical waveguide plates. Two standing windows are opened on the top of the platform body. An optical waveguide plate is independently installed below each standing window. The optical waveguide plate is used to support the subject standing on one foot and serves as the carrier for FTIR generation. The two optical waveguide plates are isolated from each other and are used to independently collect the plantar pressure distribution of the left and right feet. The dual-path mirror folding optical acquisition module includes two optical acquisition sub-units, each corresponding to one of the optical waveguide plates; each optical acquisition sub-unit is used to acquire the plantar pressure image of the subject. The computational analysis unit is used to identify and quantify the subject's somatosensory deficits related to scoliosis based on the plantar pressure images.

[0005] In some embodiments, each of the optical acquisition subunits includes an LED light source array, a binocular camera, and a planar reflector; The LED light source array is disposed on the side edge of the optical waveguide plate, and is used to couple light into the optical waveguide plate at an angle greater than the critical angle, so that total internal reflection occurs inside the optical waveguide plate; The optical axis of the binocular camera is arranged at a 45-degree angle to the normal direction of the optical waveguide plate, and is used to acquire the plantar pressure image of the subject. The planar reflector is fixedly installed at a 45-degree angle below the optical waveguide plate to fold the reflected light path from the lower surface of the optical waveguide plate by 90 degrees and guide it to the binocular camera.

[0006] In some embodiments, the computational analysis unit includes: An image receiving unit is used to dynamically receive video streams sent by the two optical acquisition subunits, each video stream including multiple plantar pressure images; The trajectory calculation unit is used to continuously calculate and output the trajectory of the center of plantar pressure and the trajectory of the center of gravity of the human body based on the plantar pressure image by pixel brightness weighted averaging. The assessment and quantification unit is used to identify and quantify the subject's somatosensory defects related to scoliosis using AI algorithms based on the plantar pressure center trajectory and the body's center of gravity trajectory.

[0007] In some embodiments, the trajectory calculation unit includes: The first calculation unit is used to calculate the trajectory of the plantar pressure center using the following formula: ; ; in, COP It is the trajectory of the center of plantar pressure. It is the pressure value at a certain point on the sole of the foot. The denominator represents the total pressure, and the numerator represents the weighted sum of pressures along the x or y axis. The second calculation unit is used to calculate the trajectory of the human body's center of gravity using the following formula: ; ; in, COG It is the trajectory of the human body's center of gravity. It is the height of the subject. It is the angle of the subject's swing in the forward and backward direction; It is the swing angle of the subject in the left-right direction.

[0008] In some embodiments, the evaluation quantification unit includes: AI algorithms are used to extract the time-domain and frequency-domain features of the plantar pressure center trajectory and the human body's center of gravity trajectory; An unsupervised clustering algorithm is used to perform cluster analysis on the extracted time-domain features and frequency-domain features, and the subjects are divided into corresponding balance ability categories. The balance ability category was compared with clinical standards to quantitatively assess the subjects' somatosensory deficits related to scoliosis.

[0009] The embodiments of this application include at least the following beneficial effects: This application provides a balance assessment and measurement sensing system based on FTIR and multimodal AI analysis. The system of this application includes a platform body, an FTIR balance measurement unit, a dual-channel mirror folding optical acquisition module, and a calculation and analysis unit. This application provides a sensing system with a compact structure, good stability, support for multi-posture (standing / sitting) assessment, and the ability to simultaneously acquire high-resolution pressure distribution images of both feet. This sensing system is based on an FTIR optical source and uses the calculation and analysis unit combined with multimodal artificial intelligence to analyze plantar pressure images, so as to achieve accurate identification and quantitative assessment of somatosensory defects related to scoliosis. Moreover, it can overcome the shortcomings of existing balance sensors such as large size and insufficient safety. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A schematic diagram of the structure of a balance evaluation measurement sensing system based on FTIR and multimodal AI analysis provided in an embodiment of this application; Figure 2 An example structural diagram of a balance evaluation measurement sensing system based on FTIR and multimodal AI analysis provided in an embodiment of this application. Detailed Implementation

[0012] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0013] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0014] Reference Figure 1This application provides a balance evaluation measurement sensing system based on FTIR and multimodal AI analysis, including: a platform body, an FTIR balance measurement unit, a dual-path mirror folding optical acquisition module, and a calculation and analysis unit; The FTIR balance measurement unit includes two optical waveguide plates. Two standing windows are opened on the top of the platform body. An optical waveguide plate is independently installed below each standing window. The optical waveguide plate is used to support the subject standing on one foot and serves as the carrier for FTIR generation. The two optical waveguide plates are isolated from each other and are used to independently collect the plantar pressure distribution of the left and right feet. The dual-path mirror folding optical acquisition module includes two optical acquisition sub-units, each corresponding to one of the optical waveguide plates; each optical acquisition sub-unit is used to acquire the plantar pressure image of the subject. The computational analysis unit is used to identify and quantify the subject's somatosensory deficits related to scoliosis based on the plantar pressure images.

[0015] Optionally, each of the optical acquisition subunits includes an LED light source array, a binocular camera, and a planar reflector; The LED light source array is disposed on the side edge of the optical waveguide plate, and is used to couple light into the optical waveguide plate at an angle greater than the critical angle, so that total internal reflection occurs inside the optical waveguide plate; The optical axis of the binocular camera is arranged at a 45-degree angle to the normal direction of the optical waveguide plate, and is used to acquire the plantar pressure image of the subject. The planar reflector is fixedly installed at a 45-degree angle below the optical waveguide plate to fold the reflected light path from the lower surface of the optical waveguide plate by 90 degrees and guide it to the binocular camera.

[0016] Optionally, the calculation and analysis unit includes: An image receiving unit is used to dynamically receive video streams sent by the two optical acquisition subunits, each video stream including multiple plantar pressure images; The trajectory calculation unit is used to continuously calculate and output the trajectory of the center of plantar pressure and the trajectory of the center of gravity of the human body based on the plantar pressure image by pixel brightness weighted averaging. The assessment and quantification unit is used to identify and quantify the subject's somatosensory defects related to scoliosis using AI algorithms based on the plantar pressure center trajectory and the body's center of gravity trajectory.

[0017] Optionally, the trajectory calculation unit includes: The first calculation unit is used to calculate the trajectory of the plantar pressure center using the following formula: ; ; in, COP It is the trajectory of the center of plantar pressure. It is the pressure value at a certain point on the sole of the foot. The denominator represents the total pressure, and the numerator represents the weighted sum of pressures along the x or y axis. The second calculation unit is used to calculate the trajectory of the human body's center of gravity using the following formula: ; ; in, COG It is the trajectory of the human body's center of gravity. It is the height of the subject. It is the angle of the subject's swing in the forward and backward direction; It is the swing angle of the subject in the left-right direction.

[0018] Optionally, the evaluation quantification unit includes: AI algorithms are used to extract the time-domain and frequency-domain features of the plantar pressure center trajectory and the human body's center of gravity trajectory; An unsupervised clustering algorithm is used to perform cluster analysis on the extracted time-domain features and frequency-domain features, and the subjects are divided into corresponding balance ability categories. The balance ability category was compared with clinical standards to quantitatively assess the subjects' somatosensory deficits related to scoliosis.

[0019] The following sections will provide a detailed description and explanation of some optional embodiments of this application, using specific application examples.

[0020] Figure 2 This is an example structural diagram of a balance evaluation measurement sensing system based on FTIR and multimodal AI analysis. Figure 2 In the diagram, (1) and (2) are glass plates, (3) and (4) are 45-degree reflectors, (5) and (6) are binocular cameras, and (7) is a computing and analysis unit.

[0021] This embodiment provides a compact plantar pressure balance sensing system. The system can assess fall risk by analyzing the short-term plantar pressure center trajectory of a user when standing naturally. It includes: an FTIR (Frustrated Total Internal Reflection) balance measurement unit, a dual-channel mirror folding optical acquisition module, and a calculation and analysis unit.

[0022] 1. FTIR Balanced Measurement Unit: The platform of this sensing system has two standing windows on its top. Below each window is an independently mounted transparent glass waveguide plate, used to support the subject standing on one foot and serving as the carrier for FTIR generation. The two waveguide plates are isolated from each other and are used to independently acquire the plantar pressure distribution of the left and right feet.

[0023] 2. Dual-channel mirror-folded optical acquisition module: The module consists of two identical optical acquisition sub-units, each corresponding to an optical waveguide plate.

[0024] Each optical acquisition subunit includes: (1) An LED light source array is set on the side edge of the optical waveguide plate to couple light into the optical waveguide plate at an angle greater than the critical angle, so that total internal reflection occurs inside the plate.

[0025] (2) A high-resolution binocular camera, whose optical axis is arranged at a 45-degree angle to the normal direction of the optical waveguide plate.

[0026] (3) A plane mirror, fixed at a 45-degree angle below the optical waveguide plate, is used to fold the reflected light from the lower surface of the optical waveguide plate by 90 degrees and guide it to the camera, such as... Figure 2 As shown.

[0027] By introducing a reflector, the traditional vertical optical path is converted into a horizontal one, allowing the camera to be positioned laterally, thereby significantly reducing the overall thickness (Z-axis height) of the entire sensing system.

[0028] The principle of FTIR (Suppressed Total Internal Reflection): When an LED light source shines into a glass plate from the edge, because the refractive index of glass is greater than that of air, theoretically all the light will be reflected back into the glass plate if no object touches the glass surface. The camera under the glass plate will not be able to capture any light. However, when a sample touches the glass surface, the condition for total internal reflection is broken at this point of contact. Diffuse reflection will occur in these surface areas instead of total internal reflection.

[0029] 3. Calculation and Analysis Unit: It communicates with two high-resolution cameras (WIFI) and receives and displays live video streams from both cameras.

[0030] Center of Pressure Calculation: Based on real-time pressure images, the trajectory of the plantar center of pressure (COP) and the center of gravity (COG) is continuously calculated and output through pixel brightness-weighted averaging. The COP reflects the control strategy employed by the human body to maintain balance, while the COG reflects the actual swaying of the human body.

[0031] The COP calculation formula is as follows: ; ; in It is the pressure value at a certain point on the sole of the foot. The denominator represents the total pressure, and the numerator represents the weighted sum of the pressures on x or y.

[0032] The COG calculation formula is as follows: ; ; in It refers to human height. It is the angle of the human body's swing in the forward and backward direction; It refers to the angle of the human body's swing in the left and right directions.

[0033] Next, AI algorithms were used to extract temporal features (total trajectory length, swing area, velocity) and frequency domain features (energy percentage in specific frequency bands) from the COP / COG trajectory. An unsupervised clustering algorithm (k-means, preset k=2) was employed to perform cluster analysis on the extracted multidimensional feature vector set, dividing the subject group into different balance ability categories. The results were then compared with the clinical gold standard (Cobb angle, fall history, lower limb muscle strength grade) using the Fowlkes-Mallows index (FMI). When FMI > 0.6, it was confirmed that the system's assessment results were significantly consistent with the clinical severity of scoliosis or somatosensory deficits, and the group classified into specific clusters had a higher risk of falls.

[0034] In summary, this embodiment includes the following key technical solutions: 1) Calculate COG and COP by using the principle of suppressed total internal reflection and combining visual imaging.

[0035] 2) A reflector is placed on each side inside the balance measurement system at a 45-degree angle. This special design reduces the overall frame thickness of the balance measurement system significantly.

[0036] Beneficial effects: 1) Calculate COG and COP by using the principle of suppressed total internal reflection and combining visual imaging.

[0037] 2) A reflector is placed on each side inside the balance measurement system at a 45-degree angle. This special design reduces the overall frame thickness of the balance measurement system significantly.

[0038] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0039] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A balance evaluation measurement sensing system based on FTIR and multimodal AI analysis, characterized in that, The system includes: a platform body, an FTIR balanced measurement unit, a dual-channel mirror folding optical acquisition module, and a calculation and analysis unit; The FTIR balance measurement unit includes two optical waveguide plates. Two standing windows are opened on the top of the platform body. An optical waveguide plate is independently installed below each standing window. The optical waveguide plate is used to support the subject standing on one foot and serves as the carrier for FTIR generation. The two optical waveguide plates are isolated from each other and are used to independently collect the plantar pressure distribution of the left and right feet. The dual-path mirror folding optical acquisition module includes two optical acquisition sub-units, each corresponding to one of the optical waveguide plates; each optical acquisition sub-unit is used to acquire the plantar pressure image of the subject. The computational analysis unit is used to identify and quantify the subject's somatosensory deficits related to scoliosis based on the plantar pressure images.

2. The balance evaluation measurement sensing system based on FTIR and multimodal AI analysis according to claim 1, characterized in that, Each of the optical acquisition subunits includes an LED light source array, a binocular camera, and a planar reflector; The LED light source array is disposed on the side edge of the optical waveguide plate, and is used to couple light into the optical waveguide plate at an angle greater than the critical angle, so that total internal reflection occurs inside the optical waveguide plate; The optical axis of the binocular camera is arranged at a 45-degree angle to the normal direction of the optical waveguide plate, and is used to acquire the plantar pressure image of the subject. The planar reflector is fixedly installed at a 45-degree angle below the optical waveguide plate to fold the reflected light path from the lower surface of the optical waveguide plate by 90 degrees and guide it to the binocular camera.

3. The balance evaluation measurement sensing system based on FTIR and multimodal AI analysis according to claim 1, characterized in that, A balance assessment measurement sensing system based on FTIR and multimodal AI analysis, characterized in that the calculation and analysis unit includes: An image receiving unit is used to dynamically receive video streams sent by the two optical acquisition subunits, each video stream including multiple plantar pressure images; The trajectory calculation unit is used to continuously calculate and output the trajectory of the center of plantar pressure and the trajectory of the center of gravity of the human body based on the plantar pressure image by pixel brightness weighted averaging. The assessment and quantification unit is used to identify and quantify the subject's somatosensory defects related to scoliosis using AI algorithms based on the plantar pressure center trajectory and the body's center of gravity trajectory.

4. The balance evaluation measurement sensing system based on FTIR and multimodal AI analysis according to claim 3, characterized in that, The trajectory calculation unit includes: The first calculation unit is used to calculate the trajectory of the plantar pressure center using the following formula: ； ； in, COP It is the trajectory of the center of plantar pressure. It is the pressure value at a certain point on the sole of the foot. The denominator represents the total pressure, and the numerator represents the weighted sum of pressures along the x or y axis. The second calculation unit is used to calculate the trajectory of the human body's center of gravity using the following formula: ； ； in, COG It is the trajectory of the human body's center of gravity. It is the height of the subject. It is the angle of the subject's swing in the forward and backward direction; It is the swing angle of the subject in the left-right direction.

5. The balance evaluation measurement sensing system based on FTIR and multimodal AI analysis according to claim 3, characterized in that, The evaluation quantification unit includes: AI algorithms are used to extract the time-domain and frequency-domain features of the plantar pressure center trajectory and the human body's center of gravity trajectory; An unsupervised clustering algorithm is used to perform cluster analysis on the extracted time-domain features and frequency-domain features, and the subjects are divided into corresponding balance ability categories. The balance ability category was compared with clinical standards to quantitatively assess the subjects' somatosensory deficits related to scoliosis.

Images