Intelligent scoliosis detection system and method based on multi-mode artificial intelligence

Abstract

The invention provides a scoliosis intelligent detection system and method based on multi-modal artificial intelligence, and the system comprises a multi-modal input layer which is used for receiving a double-posture photo and electronic questionnaire data uploaded by a user; the intelligent processing layer is connected with the multi-modal input layer, and comprises an image analysis module used for carrying out portrait segmentation, spine positioning, feature extraction and corner detection on the double-posture picture so as to calculate a spine curvature parameter; and the questionnaire processing module is used for carrying out OCR (Optical Character Recognition), scale scoring and psychological risk prediction on the electronic questionnaire so as to generate psychological assessment parameters. The problems that in existing spine detection design, traditional radiology is high in accumulated harm to the human body; a computer vision detection system is single in detection dimension and serious in data splitting; the problem of psychological influence on the patient in the detection process is solved.

Description

Technical Field

[0001] This invention relates to the field of spinal detection technology, and more specifically, to an intelligent scoliosis detection system and method based on multimodal artificial intelligence. Background Technology

[0002] There are currently two main methods for detecting scoliosis: (1) Technical principle of traditional radiological detection system: Based on X-ray imaging technology, the two-dimensional projection of the spine is obtained through anteroposterior / lateral radiographs, and the Cobb angle (international gold standard) is calculated after manually marking the upper and lower endplates of the vertebral bodies. Derivative technologies include: EOS low-dose three-dimensional imaging: radiation dose is reduced by 50%, but the equipment cost exceeds 3 million yuan; ultrasound detection method: no radiation but limited accuracy (error ±8°), suitable for infant screening; Typical procedure: Patient positioning → X-ray imaging → film digitization → doctor manual annotation → angle calculation, taking approximately 25 minutes per case, requiring radiologist involvement; Key drawbacks include: cumulative radiation risk (limited to ≤2 times per year for adolescents); strong equipment dependence (only available in level 2 and above hospitals); static assessment (unable to capture dynamic posture compensation); Computer vision detection system: The application of computer vision (CV) in scoliosis detection has evolved from traditional methods to deep learning. Early CV technology relied primarily on traditional image processing algorithms, such as edge detection, feature point extraction, and morphological analysis. However, these methods have high requirements for image quality and pose, are difficult to automate, require manual annotation by doctors, are inefficient, and are easily affected by subjective factors. With the rise of deep learning, models based on convolutional neural networks (CNN) have been used to automatically detect and segment the spine, identify vertebrae, and perform precise coordinate localization, thereby achieving automated calculation of the Cobb angle, significantly improving detection efficiency and objectivity.

[0003] In recent years, multimodal technology has experienced rapid development, its core being the integration and understanding of data from different modalities, such as text, images, audio, and video. With the rapid advancement of Large Language Models (LLMs), multimodal technology has also seen new breakthroughs. The power of these models lies in their ability not only to process single-type data but also to map information from different modalities into a single semantic space through a unified architecture (such as the Transformer). This enables the models to perform cross-modal reasoning and generation. Multimodal technology is gradually moving from academic research to practical applications, becoming a key direction for the development of next-generation artificial intelligence.

[0004] In existing technologies, traditional radiology causes significant cumulative harm to the human body during spinal detection; computer vision detection systems suffer from limited detection dimensions and severe data fragmentation; and there are psychological impacts on patients during the detection process. Therefore, we propose an intelligent scoliosis detection system and method based on multimodal artificial intelligence to improve these aspects. Summary of the Invention

[0005] The purpose of this invention is to address the following issues with current spinal detection designs: traditional radiology causes significant cumulative harm to the human body; computer vision detection systems suffer from limited detection dimensions and severe data fragmentation; and there are also issues with the psychological impact on patients during the detection process.

[0006] To achieve the above-mentioned objectives, the present invention provides the following technical solution: A multimodal artificial intelligence-based intelligent scoliosis detection system and method are proposed to improve the above-mentioned problems.

[0007] The application is as follows: A multimodal artificial intelligence-based intelligent scoliosis detection system includes: A multimodal input layer is used to receive user-uploaded dual-pose photos and electronic questionnaire data; The intelligent processing layer, connected to the multimodal input layer, includes: The image analysis module is used to perform portrait segmentation, spine localization, feature extraction, and corner detection on dual-pose photos in order to calculate spinal curvature parameters; The questionnaire processing module is used to perform OCR recognition, scale scoring, and psychological risk prediction on electronic questionnaires to generate psychological assessment parameters. The data fusion layer is used to store the output data of the image analysis module and the questionnaire processing module, and to realize cross-modal association between image data and questionnaire data through user ID; The fusion decision module is configured to use a multi-input neural network to nonlinearly fuse spinal curvature parameters with psychological assessment parameters to generate a comprehensive assessment result. The report generation module is used to generate a comprehensive report that includes spinal parameters and psychological assessments based on the comprehensive evaluation results.

[0008] As a preferred technical solution of this application, the multimodal input layer includes a client and a Web API layer. The client supports the shooting of dual-pose photos and the input of electronic questionnaires, while the Web API layer provides asynchronous interfaces for data uploading and interaction.

[0009] As a preferred technical solution of this application, the image analysis module uses the improved Harris corner detection method to locate key points of the spine.

[0010] As a preferred technical solution of this application, the questionnaire processing module uses the SRS questionnaire and classifies it by anxiety index and risk level.

[0011] As a preferred technical solution in this application, the data fusion layer interacts with the intelligent processing layer through a RESTful API, and the data is encapsulated in JSON format.

[0012] As a preferred technical solution of this application, the multi-input neural network in the fusion decision module includes: The data input layer is used to receive standardized spinal curvature parameters and psychological assessment parameters; The fusion layer uses a fully connected layer to learn weights and model interactions between data from different modalities. The decision output layer is used to output a risk vector that includes spinal deformity risk, mental health risk, and comprehensive management recommendations.

[0013] As a preferred technical solution of this application, the system supports lightweight deployment, the core algorithm is compressed to less than 15MB, and it can achieve real-time processing on mobile devices with a frame rate of no less than 25fps.

[0014] A detection method for a multimodal artificial intelligence-based intelligent scoliosis detection system includes the following steps: Step S1: Receive the user's uploaded dual-pose photos and electronic questionnaire; Step S2: Parallel processing of dual-pose photographs and electronic questionnaires, including: image analysis of dual-pose photographs to calculate spinal curvature parameters; questionnaire analysis of electronic questionnaires to generate psychological assessment parameters; Step S3: Use a multi-input neural network to fuse the spinal curvature parameters and psychological assessment parameters to generate a comprehensive assessment result; Step S4: Generate a comprehensive report based on the comprehensive assessment results, including spinal parameters and psychological assessment.

[0015] As a preferred technical solution of this application, the image analysis includes portrait segmentation, spine localization, feature extraction and improved Harris corner detection; the questionnaire analysis includes OCR recognition, scale scoring based on SRS questionnaire and psychological risk prediction; the dual-posture photos include standing posture and forward bending posture; the psychological assessment parameters include anxiety index and risk level; and the comprehensive report includes Cobb angle, ATR angle and psychological risk level information.

[0016] As a preferred technical solution of this application, in the fusion decision step, the multi-input neural network receives standardized data through the data input layer, performs nonlinear weighted fusion in the fusion layer, and outputs a multidimensional risk vector in the decision output layer. The method is executed on a mobile terminal, and the core algorithm is no larger than 15MB after compression, with a processing frame rate of ≥25fps.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: In the scheme of this application: Through innovative multimodal fusion algorithms, the weights of different features can be automatically adjusted according to specific contexts, making the assessment results more accurate and personalized, and possessing the potential to predict the risk of disease progression. Meanwhile, the system is universally applicable in real life. Through multimodal fusion, the measurement error of spinal curvature (such as Cobb angle) is controlled within 3°. Compared with single surface image analysis (error 5°-8°), the accuracy is improved by 40%, reaching a clinically usable standard close to that of X-ray films, while the misdiagnosis rate is reduced by 37%. In addition, the entire testing process has been shortened from 30 minutes in the traditional method to less than 2 minutes, improving screening efficiency by 15 times and reducing equipment costs by 90%. It can be completed with just an ordinary camera, eliminating the reliance on expensive and radiation-emitting X-ray equipment. By using algorithm lightweighting technology, the core model is compressed to 15MB, enabling real-time computation at ≥25fps on mobile devices. A lighting invariance enhancement module has also been developed, which can maintain detection stability under various lighting conditions and has strong environmental robustness. Attached Figure Description

[0018] Figure 1 The system architecture diagram of a scoliosis intelligent detection system and method based on multimodal artificial intelligence provided in this application; Figure 2 The flowchart of the detection method of the intelligent scoliosis detection system and method based on multimodal artificial intelligence provided in this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0020] Therefore, the following detailed description of the embodiments of the present invention is not intended to limit the scope of the claimed invention, but merely to illustrate some embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention. It should be noted that, unless otherwise specified, the embodiments, features, and technical solutions in the embodiments of the present invention can be combined with each other.

[0021] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0022] like Figures 1-2 As shown, this embodiment proposes an intelligent scoliosis detection system based on multimodal artificial intelligence, including: A multimodal input layer is used to receive user-uploaded dual-pose photos and electronic questionnaire data; The intelligent processing layer, connected to the multimodal input layer, includes: The image analysis module is used to perform portrait segmentation, spine localization, feature extraction, and improved Harris corner detection on dual-pose photos to calculate spinal curvature parameters. The questionnaire processing module is used to perform OCR recognition, scale scoring, and psychological risk prediction on electronic questionnaires to generate psychological assessment parameters. The data fusion layer is used to store the output data of the image analysis module and the questionnaire processing module, and to realize cross-modal association between image data and questionnaire data through user ID; The fusion decision module is configured to use a multi-input neural network to nonlinearly fuse spinal curvature parameters with psychological assessment parameters to generate a comprehensive assessment result. The report generation module is used to generate a comprehensive report that includes spinal parameters and psychological assessments based on the comprehensive evaluation results.

[0023] The multimodal input layer includes a client and a Web API layer. The client supports taking photos in two poses and inputting electronic questionnaires, while the Web API layer provides asynchronous interfaces for data uploading and interaction.

[0024] The image analysis module uses an improved Harris corner detection method to locate key points on the spine.

[0025] The questionnaire processing module uses scales including the SRS questionnaire and classifies them by anxiety index and risk level.

[0026] The data fusion layer interacts with the intelligent processing layer via a RESTful API, and the data is encapsulated in JSON format.

[0027] Image and questionnaire data are linked across modalities using user IDs.

[0028] The multi-input neural network in the fusion decision module includes: The data input layer is used to receive standardized spinal curvature parameters and psychological assessment parameters; The fusion layer uses a fully connected layer to learn weights and model interactions between data from different modalities. The decision output layer is used to output a risk vector that includes spinal deformity risk, mental health risk, and comprehensive management recommendations.

[0029] The system supports lightweight deployment, with core algorithms compressed to below 15MB, enabling real-time processing on mobile devices at a frame rate of no less than 25fps.

[0030] Environmental robustness: The Light Invariance Enhancement Module (LIEM) was developed to maintain detection stability in the illuminance range of 200-1000 lux, resulting in: a 15-fold increase in screening efficiency (2 minutes / case); a 90% reduction in equipment cost (requiring only a regular camera); and a 37% decrease in misdiagnosis rate (clinical validation data).

[0031] Overall advantages: Improved accuracy: Multimodal fusion reduces curvature measurement error to <3° (40% improvement over single-image analysis); Efficiency optimization: The entire process takes less than 2 minutes (the traditional method takes 30 minutes); Reduced costs: Only a regular camera and server are needed (no X-ray equipment required); Comprehensive assessment: For the first time, a combined assessment of physiological indicators and psychological state has been achieved.

[0032] A detection method for a multimodal artificial intelligence-based intelligent scoliosis detection system includes the following steps: Step S1: Receive the user's uploaded dual-pose photos and electronic questionnaire; Step S2: Parallel processing of dual-pose photographs and electronic questionnaires, including: image analysis of dual-pose photographs to calculate spinal curvature parameters; questionnaire analysis of electronic questionnaires to generate psychological assessment parameters; Step S3: Use a multi-input neural network to fuse the spinal curvature parameters and psychological assessment parameters to generate a comprehensive assessment result; Step S4: Generate a comprehensive report based on the comprehensive assessment results, including spinal parameters and psychological assessment.

[0033] Image analysis includes human face segmentation, spine localization, feature extraction, and improved Harris corner detection; questionnaire analysis includes OCR recognition, SRS-based scale scoring, and psychological risk prediction; dual-posture photos include standing and forward bending postures; psychological assessment parameters include anxiety index and risk level; and the comprehensive report includes Cobb angle, ATR angle, and psychological risk level information. The structured data output by the image analysis module (such as Cobb angle and ATR angle values) serves as the first input; the risk level classification output by the questionnaire analysis module (such as psychological stress scores of 0-10) serves as the second input. These data undergo standardization before entering the fusion model to eliminate dimensional differences between different modalities.

[0034] In the fusion decision-making step, the multi-input neural network receives standardized data through the data input layer, performs nonlinear weighted fusion in the fusion layer, and outputs a multidimensional risk vector in the decision output layer. The method is executed on a mobile terminal, and the core algorithm is no larger than 15MB after compression, with a processing frame rate of ≥25fps.

[0035] Traditional scoliosis assessment methods rely on a single modality, whose decision function can be expressed as: Y_monomodal = f(Xi), where Y_monomodal is the final assessment result, f(Xi) is the analysis function for the single modality, and Xi represents data from a specific modality, such as X-rays or surface photographs. This method has limitations in diagnostic accuracy; for example, assessing the Cobb angle solely based on surface data may not achieve an accuracy rate exceeding 85%. To overcome this limitation, we adopted a multimodal fusion decision algorithm, the core of which is to deeply integrate features from different modalities. Its decision function can be expressed as: Y 融合 =F(X image X psych ), where X image Represents image feature vectors (such as the coordinates of key points on the spine), X psych F() represents the numerical features of psychological assessment (such as anxiety scores), and is a non-linear fusion function, typically implemented by a multi-input neural network.

[0036] The key to this algorithm lies in feature weights and interactions. In traditional linear fusion, the formula might be: However, our deep learning model achieves more complex nonlinear fusion. Through training, the model automatically learns the optimal interaction between weights and features. For example, when a user has a Cobb angle of 15° (usually considered mild) but a psychological score as high as 9, the fusion model can dynamically increase the weights of high psychological features, thereby predicting their overall risk level as "high" rather than "low".

[0037] This fusion mechanism transforms evaluation from a simple weighted summation into an intelligent decision-making process capable of capturing potential correlations between different modalities.

[0038] Fusion Layer: This is the core of the algorithm, using fully connected layers to receive input from different data sources. It learns and assigns different weights to understand the potential correlation between image analysis results and psychological assessment results. For example, the model might discover that although a Cobb angle is small, if it is accompanied by a high psychological stress score, the patient is at risk of further deterioration or requiring psychological intervention. Decision Output Layer: The information processed by the fusion layer is finally sent to the output layer, which outputs one or more comprehensive evaluation indicators. For example, it can output a risk vector containing multiple dimensions, such as "spinal deformity risk", "mental health risk", and "comprehensive management recommendations".

[0039] One specific implementation is applied to a large-scale screening scenario in schools: it aims to achieve efficient, contactless, and low-cost initial screening of scoliosis for a large number of students in primary and secondary school settings; (1) System hardware configuration Data acquisition terminal: A mobile terminal integrated into the campus physical examination vehicle, equipped with a standard RGB camera (1080P resolution, 60Hz sampling frequency). Computing unit: Edge computing box equipped with a lightweight AI model (CPU: Intel i5-1135G7, integrated graphics card); Network environment: A stable connection is established with the cloud central server through 5G CPE devices.

[0040] (2) Software parameters and algorithm configuration ①Image Analysis Module: Human face segmentation: The lightweight model MODNet is used, and the background removal threshold is set to 0.85; Spine localization: A cropped YOLOv5s model was used, with an input resolution of 640×640 pixels and a confidence threshold of 0.7. Key point detection: An improved Harris corner detection algorithm is used, with a neighborhood size of 5×5.

[0041] ② Questionnaire processing module: Questionnaire type: Electronic SRS-22 questionnaire (quality of life scale); OCR engine: PaddleOCR Lightweight Edition, used for quickly recognizing checkmark information in paper questionnaires.

[0042] ③ Fusion Decision Module: Model: A lightweight multi-input neural network deployed using TensorFlow Lite, with a model size of 14.8MB; Input features: Cobb angle estimate, ATR angle, SRS questionnaire total score (0-100 points); Output: Overall risk level (low / medium / high).

[0043] Specific implementation steps: ① Data Collection: Under guidance, students complete back images in two postures—standing and forward bending—in a designated area, and then fill out the SRS-22 electronic questionnaire using a tablet. The entire process takes approximately 90 seconds per person. ② Data processing: Image and questionnaire data are asynchronously uploaded to the edge computing box via Web API; ③ Parallel analysis: The image analysis process is executed automatically, outputting spinal curvature parameters; the questionnaire system automatically scores and calculates scores for psychological and quality of life dimensions; ④ Fusion Decision: The lightweight fusion model integrates and analyzes physiological and psychological data. For example, when the system detects that a student's Cobb angle is 12° (mild), but their scores in the "pain" and "self-image" dimensions of the SRS questionnaire are extremely low, the model will increase their overall risk level. ⑤ Result Generation and Push: The system automatically generates a brief report and pushes the results to the school health room management system and the parent's mini-program via an encrypted link.

[0044] (4) The part of the central code used is: json { "user_id": "STU20241025001", "spine_status": { "Cobb_angle": 12, "ATR_angle": 5, "postural_asymmetry": "Mild" }, "psychological_assessment": { "SRS_total_score": 68, "anxiety_index": 0.34, "risk_level": "Medium" / / Due to a low quality of life score, the overall risk level has been upgraded from "Low" to "Medium". }, Recommendation: "A follow-up examination is recommended after 6 months, and attention should be paid to daily posture management and mental well-being." } ``` The implementation effect of this embodiment: In a stress test simulating 10,000 students, the system's average processing time was 105 seconds per case, representing a 15-fold improvement in screening efficiency compared to traditional outpatient procedures. Compared to X-ray diagnostic results, this embodiment achieved a 92.5% accuracy rate in detecting positive cases with a Cobb angle >10°, while reducing the false diagnosis rate by 37%.

[0045] Another embodiment is applied to the clinical auxiliary diagnosis scenario in hospitals, mainly targeting hospital orthopedic outpatient clinics, to provide doctors with accurate and quantitative clinical decision support, and to realize patient disease tracking and personalized management; (1) System hardware configuration: ① Acquisition terminal: a trinocular stereo camera for clinic use (1080P resolution, frame rate ≥30fps), equipped with a standard attitude calibration board; ② Computing unit: Local server deployed within the institute (GPU: NVIDIA T4, 16GB video memory); ③ Storage system: It interfaces with the hospital's PACS system to enable data archiving and historical comparison.

[0046] (2) Software parameters and algorithm configuration ①Image Analysis Module: Feature extraction backbone network: ResNet34, outputting a 1024-dimensional spinal morphology feature vector; Key point detection accuracy: sub-pixel level, repeatability error < 0.5 pixels.

[0047] ② Questionnaire processing module: Questionnaire type: Full version of SRS-22 and SAS Anxiety Scale; Integration method: Interact with the hospital's HIS system to automatically obtain basic patient information.

[0048] ③ Fusion Decision Module: Model: A deep multi-input neural network built on PyTorch; Dynamic weight adjustment mechanism: The weight of psychological assessment in the fusion decision is automatically adjusted based on the user's age. For example, for adolescent patients aged 12-16, the weight of psychological assessment is increased by 30% because their physical appearance is more closely related to their mental state; Output: A multi-dimensional report containing quantified risk probabilities.

[0049] (3) Specific implementation steps ① High-precision data acquisition: Under the guidance of a physician, patients complete dual-pose image acquisition in a standard lighting environment to ensure proper posture; ② Data Access: The system accesses patients' historical image data through the hospital's intranet for on-screen comparison; ③ In-depth analysis: The image analysis module generates 12 morphological parameters, including high-precision Cobb angle, vertebral rotation angle, and scapular asymmetry. Questionnaire data was analyzed together with clinical diagnostic records to assess patients’ pain levels, mental health and quality of life; ④ Clinical Fusion Decision Making: The deep fusion model comprehensively analyzes current physical signs, historical data, and psychological state. For example, for a patient with a Cobb angle of 18°, if the model detects that the vertebral rotation has increased by 3° within six months and the patient has a high SAS anxiety score, the report will prominently indicate "high risk of progression" and recommend enhanced follow-up and psychological intervention. ⑤ Report Integration: The generated comprehensive report can be directly written into the hospital's EMR system and supports one-click generation of clinical diagnosis and treatment suggestions.

[0050] (5) The part of the central code used is: json { "patient_id": "HOSP20241120005", "examination_date": "2025-11-20", "comparison_with_previous": { "Cobb_angle_change": "+2° (6 months)", "trend": "Slow Progression" }, "detailed_spine_metrics": { "Cobb_angle_primary": 18, "Cobb_angle_secondary": 8, "vertebral_rotation": 7, "kyphosis_angle": 35, "lumbar_shift": "4mm to the right" }, "psychological_profile": { "SAS_score": 58, "SRS_pain_domain": 2.8, "risk_category": "High" / / Due to the integration of psychological factors into the perspective of progress.}, "clinical_decision_support": { "recommendation": "It is recommended to have a follow-up examination within 3 months, consider custom-made orthotic braces, and recommend a psychological counseling clinic." "ICD10_codes": ["M41.9", "F41.1"] } } ``` (6) Implementation effect of this embodiment: During a six-month clinical trial at a top-tier hospital in the city, the system assisted in completing 3,850 assessments. Results showed a 40% improvement in physician diagnostic efficiency and a 25% increase in patient follow-up compliance. Compared with the measurement results of the EOS system (low-dose 3D imaging), the Cobb angle measurement error in this embodiment was controlled within ±2.1°, meeting the needs for precise clinical assessment.

[0051] The above embodiments are only used to illustrate the present invention and are not intended to limit the technical solutions described herein. Although the present invention has been described in detail with reference to the above embodiments, the present invention is not limited to the specific embodiments described above. Therefore, any modifications or equivalent substitutions to the present invention, as well as all technical solutions and improvements that do not depart from the spirit and scope of the invention, are covered within the scope of the claims of the present invention.

Claims

1. A scoliosis intelligent detection system and method based on multimodal artificial intelligence, characterized in that, include: A multimodal input layer is used to receive user-uploaded dual-pose photos and electronic questionnaire data; The intelligent processing layer, connected to the multimodal input layer, includes: The image analysis module is used to perform portrait segmentation, spine localization, feature extraction, and corner detection on the dual-pose photos in order to calculate the spinal curvature parameters. The questionnaire processing module is used to perform OCR recognition, scale scoring, and psychological risk prediction on the electronic questionnaire to generate psychological assessment parameters. The data fusion layer is used to store the output data of the image analysis module and the questionnaire processing module, and to realize cross-modal association between image data and questionnaire data through user ID; The fusion decision module is configured to use a multi-input neural network to nonlinearly fuse the spinal curvature parameters with psychological assessment parameters to generate a comprehensive assessment result. The report generation module is used to generate a comprehensive report that includes spinal parameters and psychological assessment based on the comprehensive assessment results.

2. The intelligent scoliosis detection system based on multimodal artificial intelligence according to claim 1, characterized in that, The multimodal input layer includes a client and a Web API layer. The client supports taking dual-pose photos and inputting electronic questionnaires, while the Web API layer provides asynchronous interfaces for data uploading and interaction.

3. The intelligent scoliosis detection system based on multimodal artificial intelligence according to claim 1, characterized in that, The image analysis module uses an improved Harris corner detection method to locate key points on the spine.

4. The intelligent scoliosis detection system based on multimodal artificial intelligence according to claim 1, characterized in that, The questionnaire processing module uses scales including the SRS questionnaire and classifies anxiety index and risk level.

5. The intelligent scoliosis detection system based on multimodal artificial intelligence according to claim 1, characterized in that, The data fusion layer interacts with the intelligent processing layer via a RESTful API, and the data is encapsulated in JSON format.

6. The intelligent scoliosis detection system based on multimodal artificial intelligence according to claim 1, characterized in that, The multi-input neural network in the fusion decision module includes: The data input layer is used to receive standardized spinal curvature parameters and psychological assessment parameters; The fusion layer uses a fully connected layer to learn weights and model interactions between data from different modalities. The decision output layer is used to output a risk vector that includes spinal deformity risk, mental health risk, and comprehensive management recommendations.

7. The intelligent scoliosis detection system based on multimodal artificial intelligence according to claim 1, characterized in that, The system supports lightweight deployment, with the core algorithm compressed to less than 15MB, enabling real-time processing on mobile devices at a frame rate of no less than 25fps.

8. A detection method for a multimodal artificial intelligence-based intelligent scoliosis detection system, characterized in that, Includes the following steps: Step S1: Receive the user's uploaded dual-pose photos and electronic questionnaire; Step S2: Parallel processing of the dual-posture photographs and electronic questionnaires, including: image analysis of the dual-posture photographs to calculate spinal curvature parameters; and questionnaire analysis of the electronic questionnaires to generate psychological assessment parameters. Step S3: Use a multi-input neural network to fuse the spinal curvature parameters and psychological assessment parameters to generate a comprehensive assessment result; Step S4: Generate a comprehensive report containing spinal parameters and psychological assessment based on the comprehensive assessment results.

9. The detection method of a multimodal artificial intelligence-based intelligent scoliosis detection system according to claim 8, characterized in that, The image analysis includes portrait segmentation, spine localization, feature extraction, and improved Harris corner detection; the questionnaire analysis includes OCR recognition, SRS-based scale scoring, and psychological risk prediction; the dual-posture photos include standing and forward-bending postures; the psychological assessment parameters include anxiety index and risk level; and the comprehensive report includes Cobb angle, ATR angle, and psychological risk level information.

10. The detection method of a multimodal artificial intelligence-based intelligent scoliosis detection system according to claim 9, characterized in that, In the fusion decision-making step, the multi-input neural network receives standardized data through the data input layer, performs nonlinear weighted fusion in the fusion layer, and outputs a multidimensional risk vector in the decision output layer. The method is executed on a mobile terminal, and the core algorithm is no larger than 15MB after compression, with a processing frame rate of ≥25fps.

Images