AI-driven complete cycle scoliosis management system

Abstract

The invention discloses an AI-driven complete-cycle scoliosis management system, which belongs to the field of scoliosis management systems and comprises a data fusion module, a feature extraction module, a text processing module, a diagnosis inference module and an output presentation module. Wherein the data fusion module is used for performing deep fusion on sensor data of the wearable equipment and X-ray / CT medical image data to generate comprehensive fusion prediction data; the feature extraction module uses a pre-trained fusion encoder to perform feature extraction and encoding on the fusion prediction data to generate potential feature representation; the text processing module adopts a pre-trained text encoder to encode free text diagnosis information provided by a doctor and a patient and extract semantic features; and the diagnosis inference module is used for fusing the fusion features and the text features. According to the invention, the sensing data of the wearable device and the medical image data can be effectively integrated to give full play to the complementary advantages thereof, so that the diagnosis accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the field of scoliosis management systems, and specifically to an AI-driven full-cycle scoliosis management system. Background Art

[0002] Scoliosis is a common disease that harms adolescents and children, and its incidence is relatively high worldwide. In order to effectively control the incidence of the disease and improve the health level of the public, full-cycle scoliosis management is particularly important. Through systematic prevention, screening, assessment, intervention and rehabilitation measures, scoliosis can be detected and corrected in time, the disease can be avoided from worsening, the cost of treatment can be reduced, and the quality of life of patients can be improved.

[0003] Full-cycle scoliosis management is a comprehensive health management strategy that aims to manage scoliosis in all aspects, from prevention, screening, assessment, intervention to rehabilitation. With the rapid development of wearable devices and medical imaging technology, using these multimodal data for intelligent medical diagnosis has become an important research direction. Artificial intelligence technologies such as deep learning have shown strong advantages in feature representation learning, multi-source information fusion, and interpretable modeling, providing strong support for the realization of intelligent diagnosis.

[0004] However, the existing full-cycle scoliosis management system has the following problems: it is unable to effectively integrate wearable device sensor data and medical imaging data to give full play to their complementary advantages, resulting in low diagnostic accuracy. Therefore, those skilled in the art provide an AI-driven full-cycle scoliosis management system to solve the problems raised in the above background technology. Summary of the invention

[0005] The purpose of the present invention is to provide an AI-driven full-cycle scoliosis management system that can effectively integrate wearable device sensor data and medical imaging data to give full play to their complementary advantages, thereby improving diagnostic accuracy and solving the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] AI-driven full-cycle scoliosis management system, including data fusion module, feature extraction module, text processing module, diagnosis inference module and output presentation module;

[0008] Among them, the data fusion module is used to deeply fuse the sensor data of the wearable device and the X-ray / CT medical image data to generate comprehensive fusion prediction data; the feature extraction module uses a pre-trained fusion encoder to extract and encode the fusion prediction data to generate potential feature representation; the text processing module uses a pre-trained text encoder to encode the free text diagnostic information provided by doctors and patients and extract semantic features; the diagnosis inference module is used to fuse the fusion features and text features, and generate text-based diagnosis and treatment recommendations through a mask decoder; the output presentation module outputs the diagnosis results in the form of probability distribution to enhance interpretability and user trust.

[0009] As a further solution of the present invention: the data fusion module includes a signal extraction module, a multimodal fusion module and a bending prediction module, wherein the signal extraction module extracts multimodal data from multiple sensors of the wearable device and performs preprocessing and feature extraction on the multimodal data; the multimodal fusion module is used to deeply fuse the features of different modalities to obtain a set of comprehensive feature representations; the bending prediction module performs sequence modeling and final bending prediction on the data of the wearable device based on the fused feature vector.

[0010] As a further solution of the present invention: the specific process of the signal extraction module preprocessing and feature extraction of multimodal data is as follows:

[0011] Receive sensor data: Collect multi-channel sensor data from wearable devices, including accelerometers, gyroscopes, and pressure sensors, to form raw time-domain signal inputs;

[0012] Background removal preprocessing: eliminate noise interference and background signals, and improve signal quality through filtering or noise removal algorithms;

[0013] FFT transform: Use fast Fourier transform to convert time domain signals into frequency domain signals to capture frequency characteristics;

[0014] Time domain signal and frequency domain signal acquisition: Time domain signals and frequency domain signals are respectively regarded as feature inputs to capture different signal performances;

[0015] Convolutional layer processing: Convolutional neural network extracts features from time domain signals and frequency domain signals;

[0016] Self-attention layer enhances features: The self-attention mechanism is introduced to weight the extracted features to improve the system's ability to identify key features;

[0017] The feature extraction layer generates feature vectors: The extracted features are further processed through a fully connected layer or dimensionality reduction technology to generate time domain feature vector 1 and frequency domain feature vector 2.

[0018] As a further solution of the present invention: the specific process of the multimodal fusion module deeply fusing the features of different modalities is as follows:

[0019] Multilayer perceptron preliminary feature processing: Multilayer perceptron 1 and multilayer perceptron 2 perform preliminary processing on time domain feature vector 1 and frequency domain feature vector 2 respectively as the first step of feature conversion;

[0020] Enhancement of residual connection and cross connection: Residual connection and cross connection are used to maintain the integrity of feature information and enhance feature transfer capability;

[0021] Combination of convolution and max pooling layers: The combination of multiple convolution layers and max pooling layers is used to gradually compress and extract features, filtering out key information in multimodal features;

[0022] Multimodal fusion of cross-connection architecture: The complex cross-connection structure further realizes the fusion of different modal features;

[0023] Fusion feature vector generation: The first feature vector and the second feature vector after the fusion of the time domain signal and the frequency domain signal are finally integrated to obtain a fused feature vector.

[0024] As a further solution of the present invention: the specific process of the bending prediction module performing sequence modeling and final bending prediction on the data of the wearable device based on the fused feature vector is as follows:

[0025] Long Short-Term Memory Network Sequence Modeling: Use LSTM to model sequence features;

[0026] Further processing by multi-layer perceptron: The output of LSTM is processed by multi-layer perceptron 3 to further extract and transform features, providing more layers of feature support for prediction;

[0027] Integration of sensor prediction data: Integrate the sensor data predicted by the model to ensure that features from different sources and different modalities are fully represented in the prediction.

[0028] Non-linear transformation of activation function: Before the output layer, the ReLU activation function is applied to perform non-linear transformation on the data to obtain the prediction result.

[0029] Output prediction results: The final output prediction results represent the bending prediction value of the wearable device signal, completing the process from data collection, feature extraction to final prediction.

[0030] As a further solution of the present invention: the feature extraction module is a framework for multimodal medical data processing and prediction, which aims to extract information from static images and dynamic sensor data at different time points to perform comprehensive medical predictions, wherein the framework of the feature extraction module includes an input data layer, a potential diffusion stage, a potential diffusion model and a data fusion stage; the input data layer provides multimodal input data for the framework, and normalizes the information on the timeline for analysis in subsequent stages; the potential diffusion stage encodes and extracts features from data of different modalities through an encoder to generate a potential representation that is consistent in time; the potential diffusion model integrates multimodal data features and noise to capture the intrinsic relationship between multimodal data; the data fusion stage comprehensively analyzes the scoliosis state of the patient through predictors and fusion functions based on the generated multimodal features.

[0031] As a further solution of the present invention: the text processing module includes a concept generation module and a feature fusion and prediction module, wherein the concept generation module generates a core concept embedding vector for scoliosis diagnosis by combining medical images with text descriptions, providing support for subsequent feature fusion and prediction; the feature fusion and prediction module uses the fusion features of image and text concepts to predict the type of scoliosis and provide the final diagnosis result.

[0032] As a further solution of the present invention: the specific process of the feature fusion and prediction module predicting the type of scoliosis is:

[0033] S1: Data preprocessing;

[0034] S101: receiving fusion prediction data, the system combines the X-ray and CT images with the text embedding vector and inputs them into the prediction model as multimodal fusion data;

[0035] S102: Processing by training a fusion encoder, that is, the system jointly processes the image and text data through the trained fusion encoder to extract features meaningful for scoliosis diagnosis;

[0036] S2: Feature extraction;

[0037] S201: generating a feature map, i.e., the fusion encoder converts the input data into a feature map, which represents the diagnostic information of different regions;

[0038] S202: Multi-layer feature representation, i.e., the feature extraction process generates multi-level feature representation, so that the model can capture features of different resolutions and scales in the image to cover the details of various parts of the spine;

[0039] S3: feature matching and attention calculation;

[0040] S301: Matching image features with text embedding vectors, that is, matching the extracted image features with the text embedding vectors in the concept generation module, thereby establishing an association between the image data and the diagnosis concept;

[0041] S302: Generate an attention heat map of the concept, that is, calculate the saliency of each concept in the image through the attention mechanism and generate an attention heat map;

[0042] S303: Displaying the importance weight of the region. The brightness or color depth of the heat map indicates the importance weight of different regions.

[0043] S4: Feature fusion and prediction;

[0044] S401: Generate a concept score vector, that is, the system generates a score vector for each concept, indicating the possibility of each scoliosis type;

[0045] S402: filtering out small probability values ​​through threshold screening processing according to the threshold screening results, so as to improve the reliability of the diagnosis result;

[0046] S403: finally outputting the concept label, that is, the filtered concept score vector is binarized and the final concept label is output.

[0047] As a further solution of the present invention: the diagnosis inference module is a multimodal diagnosis and treatment suggestion framework, which combines text, medical image and spatial information processing to achieve flexible diagnosis and treatment plan generation, wherein the specific process of the diagnosis inference module to achieve diagnosis and treatment plan generation is:

[0048] Import input sources. The input sources of the system include information from doctors and patients, so as to comprehensively collect the required data and provide a basis for accurate diagnosis and personalized treatment recommendations;

[0049] Text processing, the text processing pipeline is responsible for converting free text into embedding vectors to support subsequent analysis;

[0050] Image processing: The image processing process normalizes and extracts features from X-ray and CT images to generate image features for diagnosis;

[0051] Spatial information processing, that is, processing the spatial information of different anatomical structures to provide assistance for diagnosis;

[0052] By prompting the encoder to integrate data from multiple modalities into one, a comprehensive feature representation is formed;

[0053] Generate specific diagnosis and treatment recommendations based on comprehensive features through a mask decoder;

[0054] The output results present the diagnosis and treatment recommendations generated by the system in a hierarchical manner to meet the needs of different granularities.

[0055] As a further solution of the present invention: the specific process of the image processing is:

[0056] Normalization network: The image input is processed through the normalization network to eliminate the differences between different devices and different imaging conditions, so that the image reaches a unified quality standard;

[0057] Image normalization alignment function: Use the alignment function to align the anatomical structures in the image to ensure the consistency and comparability of images in different directions, and finally obtain a normalized image;

[0058] Pre-trained fusion encoder: Use the pre-trained model to process the normalized image and extract the required feature vector to generate the image embedding vector;

[0059] Generate image feature vector: The image feature vector contains key visual information that can be used for diagnosis and is fused with text features for comprehensive analysis.

[0060] As a further solution of the present invention: the present application also includes a computer-readable storage medium, on which a computer program is stored. When the computer program runs on a computer, the computer runs an AI-driven full-cycle scoliosis management system, wherein one or more programs also include instructions for the following operations: using a mobile device through a preset app or applet, calling the camera and level of the mobile device, taking an image of the patient's back in a standing position and uploading it to a server, using an intelligent detection model in the background for analysis, and pushing the results to a user end, wherein the user end includes a personal mobile end, an enterprise mobile end, and a management background.

[0061] As a further solution of the present invention: the personal mobile terminal includes a first personal center unit, a first detection center unit, a rehabilitation center unit, a protective gear display and consultation unit, an education and popular science unit and an AI consultation unit, wherein:

[0062] The first personal center unit is used to provide user registration and login, user basic information management, historical detection record viewing, intelligent reminder, and message notification viewing functions;

[0063] The first detection center unit is used to guide the user to take or upload a back photo and a walking video, and obtain the detection results in real time or at a fixed time;

[0064] The rehabilitation center unit is used to display the location and contact information of nearby rehabilitation clinics based on positioning;

[0065] The protective gear display and consultation unit is used to display pictures and descriptions of various protective gears and provide purchase / replacement channels or consultation methods;

[0066] The educational science unit is used to popularize scientific knowledge, provide popular science articles, videos and animations about spinal health, and enhance parents' health awareness;

[0067] The AI ​​consulting unit is used to provide AI intelligent consulting functions.

[0068] As a further solution of the present invention: the enterprise mobile terminal includes a second person center unit and a second detection center unit, wherein:

[0069] The second personal center unit is used to provide enterprise user registration and login functions;

[0070] The second detection center unit is used to select schools, grades, classes, and students, guide users to take and upload back photos and walking videos, and finally obtain detection results in real time or at a scheduled time.

[0071] As a further solution of the present invention: the management backend includes a personnel and authority management unit, an enterprise management unit, a data center unit, a consulting center unit, a cooperative merchant and rehabilitation center management unit, and an agent management and distribution unit, wherein:

[0072] The personnel and authority management unit is used to perform authority classification and assign different authorities to different roles to ensure system security;

[0073] The enterprise management unit is used to maintain the basic information of the enterprise and manage the import and export of data;

[0074] The data center unit is used to collect statistics on all data to be analyzed, analyzed, and abnormal, generate a school-level spinal health report, analyze the incidence and trend of scoliosis, and finally export and share the data;

[0075] The consultation center unit is used to display all consultation contents to be responded to and can customize the responses;

[0076] The cooperative merchant and rehabilitation center management unit is used to manage the cooperative merchants and rehabilitation centers;

[0077] The agent management and distribution unit is used to manage agents and distributors.

[0078] Compared with the prior art, the present invention has the following beneficial effects:

[0079] The full-cycle scoliosis management system of this application can make full use of the complementary advantages of wearable devices and medical imaging data, extract rich feature representations through deep learning technology, and integrate the professional knowledge of doctors and patients to provide accurate and explainable diagnostic suggestions. Compared with traditional methods, this system can greatly improve diagnostic efficiency and accuracy, facilitate doctors to manage the disease throughout the cycle, enhance communication and trust between doctors and patients, and bring significant value to clinical diagnosis and treatment practice. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 This is a structural diagram of the AI-driven full-cycle scoliosis management system;

[0081] Figure 2 This is a framework diagram of the data fusion module in the AI-driven full-cycle scoliosis management system;

[0082] Figure 3 This is a framework diagram of the feature extraction module in the AI-driven full-cycle scoliosis management system;

[0083] Figure 4 This is a framework diagram of the text processing module in the AI-driven full-cycle scoliosis management system;

[0084] Figure 5 This is a framework diagram of the diagnostic inference module in the AI-driven full-cycle scoliosis management system. DETAILED DESCRIPTION

[0085] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0086] As mentioned in the background technology of this application, research has found that the existing full-cycle scoliosis management system cannot effectively integrate wearable device sensor data and medical imaging data to give full play to their complementary advantages, which results in low diagnostic accuracy and certain defects.

[0087] In order to solve the above-mentioned defects, the present application discloses an AI-driven full-cycle scoliosis management system that can effectively integrate wearable device sensor data and medical imaging data to give full play to their complementary advantages, thereby improving diagnostic accuracy.

[0088] The following will describe in detail how the solution of the present application solves the above technical problems in conjunction with the accompanying drawings.

[0089] See also Figure 1In an embodiment of the present invention, an AI-driven full-cycle scoliosis management system includes a data fusion module, a feature extraction module, a text processing module, a diagnosis inference module and an output presentation module; wherein the data fusion module is used to deeply fuse the sensor data of the wearable device and the X-ray / CT medical image data to generate comprehensive fusion prediction data; the feature extraction module uses a pre-trained fusion encoder to extract and encode the fusion prediction data to generate a potential feature representation; the text processing module uses a pre-trained text encoder to encode the free text diagnosis information provided by doctors and patients to extract semantic features; the diagnosis inference module is used to fuse the fusion features and the text features to generate text-based diagnosis and treatment recommendations through a mask decoder; the output presentation module outputs the diagnosis results in the form of probability distribution to enhance interpretability and user trust.

[0090] In this embodiment, if Figure 2 As shown, the data fusion module includes a signal extraction module, a multimodal fusion module and a bending prediction module. The core of the signal extraction module is to extract multimodal data from multiple sensors of the wearable device and perform preprocessing and feature extraction on it; the purpose of the multimodal fusion module is to deeply fuse the features of different modalities to obtain a set of comprehensive feature representations; the bending prediction module performs sequence modeling and final bending prediction on the data of the wearable device based on the fused feature vector.

[0091] In this embodiment, the specific process of the signal extraction module preprocessing and feature extraction of multimodal data is as follows:

[0092] Receive sensor data: Collect multi-channel sensor data from wearable devices, including accelerometers, gyroscopes, and pressure sensors, to form raw time-domain signal inputs;

[0093] Background removal preprocessing: eliminate noise interference and background signals, and improve the quality of the signal through filtering or noise removal algorithms (such as low-pass filtering and smoothing filtering); this step ensures the accuracy of subsequent feature extraction;

[0094] FFT transform (time domain to frequency domain conversion): Use Fast Fourier Transform (FFT) to convert time domain signals into frequency domain signals to capture frequency characteristics; frequency domain signals provide details that time domain information cannot reflect, allowing the system to analyze signal characteristics from different perspectives;

[0095] Time domain signal and frequency domain signal acquisition: Time domain signals and frequency domain signals are respectively regarded as feature inputs to capture different signal performances; time domain features mainly reflect the changes of signals over time, while frequency domain features capture periodic or frequency-related information;

[0096] Convolutional layer processing: Convolutional neural network (CNN) extracts features from time domain and frequency domain signals; different convolution kernels can capture the local pattern, periodicity and time-frequency domain characteristics of the signal;

[0097] Self-attention layer enhances features: The self-attention mechanism is introduced to weight the extracted features to improve the system's ability to identify key features. Through the self-attention layer, the model can learn the important parts of the signal, so that the key signal features are highlighted in subsequent processing steps.

[0098] The feature extraction layer generates feature vectors: The extracted features are further processed through a fully connected layer or dimensionality reduction technology to generate time domain feature vectors 1 and frequency domain feature vectors 2; these feature vectors represent the characteristic patterns of signals in different domains and provide a basis for the multimodal fusion module.

[0099] In this embodiment, the specific process of the multimodal fusion module to deeply fuse the features of different modalities is as follows: (1) Preliminary feature processing of multilayer perceptron (MLP): Multilayer perceptron 1 and multilayer perceptron 2 perform preliminary processing on time domain feature vector 1 and frequency domain feature vector 2 respectively as the first step of feature conversion; multilayer perceptron can perform nonlinear transformation on the original features and map different feature spaces to the same dimension for subsequent fusion processing. (2) Enhancement of residual connection and cross-connection: Residual connection (ResidualConnection) and cross-connection (Cross-Connection) are used to maintain the integrity of feature information and enhance feature transmission capability; residual connection helps to prevent the gradient vanishing problem of deep networks, while cross-connection can strengthen the interaction between different features, effectively retaining and enriching the expression ability of multimodal signals. (3) Combination of convolution and maximum pooling layers: The combination of multiple groups of convolution layers and maximum pooling layers is used to gradually compress and extract features and filter out key information in multimodal features; the maximum pooling layer reduces the amount of calculation while retaining the main features, thereby improving the efficiency of the model. (4) Multimodal fusion of cross-connection architecture: The complex cross-connection structure further realizes the fusion of different modal features; through this design, the features of different modalities are intertwined and interacted with each other, thereby enhancing the system's expressiveness and making the final fused feature vector more representative. (5) Generation of fused feature vector: The first feature vector and the second feature vector after the fusion of the time domain signal and the frequency domain signal are finally integrated to obtain a fused feature vector. The time domain and frequency domain refer to the time domain signal and the frequency domain signal. The time domain is the actual signal of the sensor that changes over time after the background is removed, and the frequency domain is the signal after the time domain signal is processed by the fast Fourier transform (FFT). The sensor signals at multiple locations are connected after the fast Fourier transform (FFT). After the convolution layer, the self-attention layer and the feature extraction layer, the scoliosis feature vectors of the two projection planes are obtained. They can be understood as feature vectors along the xz axis and the yz axis of the human body, and these two feature vectors are the first feature vector and the second feature vector. This fused feature vector is a highly abstract representation of the overall characteristics of the signal by the system, and contains multi-level details of multimodal information.

[0100] In this embodiment, the bending prediction module performs sequence modeling and final bending prediction on the data of the wearable device based on the fusion feature vector as follows: (1) Long short-term memory network (LSTM) sequence modeling: LSTM is used to model the sequence features; LSTM can remember the feature patterns for a long time, is suitable for processing time series data, and effectively captures the dynamic information in the features. It has a strong modeling ability for time series related features and is suitable for analyzing the time series laws in the wearable device signals. (2) Further processing by multilayer perceptron: The output of LSTM is processed by multilayer perceptron 3 to further extract and transform features, providing more levels of feature support for prediction; the MLP structure increases the nonlinear modeling ability of the system, which helps to fine-tune complex signal features. (3) Integration of sensor prediction data: The sensor data predicted by the model is integrated to ensure that the features of different sources and different modes are fully expressed in the prediction; this makes the final prediction result more comprehensive and accurate. (4) Nonlinear transformation of activation function: Before the output layer, the ReLU activation function is applied to perform nonlinear transformation on the data to obtain the prediction result; the introduction of the activation function improves the expression ability of the model and makes the output more consistent with the distribution of actual data. (5) Output prediction results: The final output prediction results represent the bending prediction value of the wearable device signal, completing the process from data collection, feature extraction to final prediction.

[0101] The data fusion module achieves deep fusion of time domain and frequency domain features through structures such as residual and cross connection, so that different modal features can complement each other and improve the system's comprehensive understanding of signals. Considering the time domain and frequency domain features at the same time not only makes the system more sensitive to signal changes, but also can capture deep information such as the periodicity and spectral characteristics of the signal. The residual connection alleviates the gradient vanishing problem of the deep model, while the cross connection enhances the interaction between features and helps to capture the subtle differences of multimodal signals. The use of LSTM to capture time series features and combined with deep feature fusion significantly improves the modeling and prediction accuracy of time series signals. In general, the framework of the data fusion module uses a combination of multi-level feature extraction, complex feature fusion, and sequence modeling to provide an efficient and accurate solution for signal processing and prediction of wearable devices. This complex structural design can better cope with the processing needs of multimodal signals, making it more robust and flexible in practical applications.

[0102] In this embodiment, if Figure 3 As shown in the figure, the feature extraction module is a framework for multimodal medical data processing and prediction. It aims to extract information from static images and dynamic sensor data at different time points to make comprehensive medical predictions. The following is a detailed interpretation of each part of the framework:

[0103] 1. Input data layer

[0104] Objective: Provide multimodal input data to the framework and normalize the information on the timeline for analysis in subsequent stages.

[0105] X-ray and CT data: static image data obtained at two different time points, t0 and t1, usually contain information about the patient's current and previous lesion conditions. These image data can help observe the progression of scoliosis, such as the degree of scoliosis, changes in bone density, etc.

[0106] Wearable device sensor data: Dynamic data stream from t0 to t1, which records the patient's continuous posture information during this time period, such as acceleration, gyroscope, pressure, etc. These data can reflect the changes in scoliosis status in real time and provide information for real-time tracking of disease progression.

[0107] Timeline arrangement: Arrange the data in chronological order so that the model can analyze the evolution of the patient's condition at each point in time and make predictions about future health conditions.

[0108] 2. Potential spread stage

[0109] Objective: Encode and extract features from data of different modalities through an encoder to generate temporally consistent latent representations.

[0110] 2.1 Encoder processing:

[0111] Medical image coding: Encode X-ray and CT image data at t0 and t1 using models such as deep convolutional networks (such as ResNet or U-Net). These encoders can extract high-level features from images, such as tissue structure, density changes, etc.

[0112] Irregular sequence encoding: For sensor data, an encoder based on irregular time series (such as a time series model based on LSTM or Transformer) is used to process it. This encoder can process irregularly sampled time series data and generate feature representations within the period.

[0113] 2.2 Potential Representation:

[0114] Latent representation at time t0: The encoded image and sensor data representation at time t0 can be regarded as the scoliosis state of the patient at t0.

[0115] Potential representation at time t1: Similar to t0, it represents the scoliosis status of the patient at t1.

[0116] Coded representation of the time period t0-t1: The coded representation generated based on the sensor data reflects the dynamic physiological state of the patient during this time period, which is convenient for fusion in subsequent stages.

[0117] 2.3 Noise processing:

[0118] Adding noise: Adding noise to the encoded representation can improve the generalization ability of the model by simulating potential interference or uncertainty in the data.

[0119] Noise representation generation: Generate a noise representation on the encoded representation from t0 to t1 to build the robustness of the model in different modal data.

[0120] 2.4 Comparison mechanism:

[0121] Representation comparison: Comparing the latent representations at time t0 and t1 allows us to assess changes in the patient’s health status over time.

[0122] Cross-attention fusion: The cross-attention mechanism is used to correlate and integrate the temporal relationships between X-ray, CT and sensor data to provide a unified representation for data of different modalities.

[0123] 3. Potential Diffusion Model

[0124] Objective: To integrate multimodal data features and noise through latent diffusion model to capture the intrinsic relationship between multimodal data.

[0125] Integrating representations and noise: Integrating latent representations of different modalities with noisy data makes the model more tolerant to interference factors in the data.

[0126] Normal distribution noise: Introducing standard normally distributed noise to simulate the uncertainty and randomness in the data, thereby helping the model to be more robust to abnormal fluctuations in the data.

[0127] Cross-attention mechanism: Through cross-attention, the information of X-ray, CT images and sensor time series data is effectively combined to produce more representative comprehensive features.

[0128] Multimodal feature fusion: Features from different sources are deeply integrated to obtain a comprehensive feature representation. This fusion enhances the model's ability to understand multimodal information and provides a reliable data foundation for the final prediction.

[0129] 4. Data fusion stage

[0130] Objective: Based on the generated multimodal features, the predictor and fusion function are used to comprehensively analyze the scoliosis status of the patient.

[0131] 4.1 Three predictors:

[0132] Predictor 1: It is specialized in processing X-ray and CT data at time t0, and mainly analyzes static images of patients.

[0133] Predictor 2: It specifically processes the time series data generated by the sensor and analyzes the dynamic lateral bending state from t0 to t1.

[0134] Predictor 3: processes the updated latent representation, including features integrated from image and sensor data, to provide a comprehensive analysis of multimodal information.

[0135] 4.2 Fusion function:

[0136] The outputs of the three predictors are combined to generate the final prediction result through a fusion function. The fusion function can be a weighted average, a voting mechanism, or a deep neural network fusion module to maximize the information provided by each predictor.

[0137] The final fusion data generated can fully reflect the patient's scoliosis condition and its future trend.

[0138] 4.3 Model Optimization:

[0139] Use loss function 1 and loss function 2 to optimize the fusion results of different predictors or models respectively. You can use multi-objective loss functions, such as cross entropy, mean square error, etc., to improve the accuracy of different prediction targets.

[0140] Generate a new training data set 2 to further optimize the model and improve its application effect in actual medical scenarios.

[0141] The main features of the feature extraction module framework are as follows:

[0142] Multi-time point data processing capability: By encoding and modeling the potential representations of static and dynamic data at different time points, the framework can effectively capture the changes in the patient's scoliosis condition over time.

[0143] Multimodal data fusion: Combines X-ray, CT and other images with time series data generated by sensors to obtain comprehensive patient scoliosis information and enhance diagnostic accuracy.

[0144] Innovative latent diffusion model: Through the latent diffusion model and noise processing, the framework has strong robustness in dealing with uncertainties in the data.

[0145] Complex noise processing mechanism: Through noise injection and processing mechanism, the model can better adapt to random fluctuations in the data and improve generalization ability.

[0146] Multi-layer prediction and fusion strategy: Multimodal features are predicted and integrated hierarchically to fully utilize the advantages of each predictor and provide multi-dimensional support for medical decision-making.

[0147] This framework design can integrate the advantages of imaging and sensor data at multiple time points, dynamically track the condition of scoliosis, and provide more accurate diagnostic references for clinicians. Through the innovative application of noise processing and potential diffusion mechanisms, the framework has a stronger ability to cope with complex medical data, and is expected to improve the accuracy and reliability of diagnostic results.

[0148] In this embodiment, the text processing module includes a concept generation module and a feature fusion and prediction module, and the corresponding framework diagram is shown in FIG. Figure 4 As shown, the following is a detailed explanation of each step:

[0149] Upper part - concept generation module

[0150] Objective: To generate the core concept embedding vector of scoliosis diagnosis by combining medical images with text descriptions, so as to provide support for subsequent feature fusion and prediction.

[0151] 1. Input Data

[0152] Peking University-Tsinghua full-cycle scoliosis large model: The system relies on the comprehensive scoliosis data set provided by Peking University-Tsinghua Data, including data from patients at different stages of the disease, thus forming a complete cross-cycle database with diversity and high quality.

[0153] Classification information of scoliosis types: This includes different types of scoliosis classifications, such as thoracic curvature, lumbar curvature, double curvature, etc., providing basic classification categories for the system.

[0154] Corresponding X-ray and CT image samples: Provide real X-ray and CT images to help the system train and understand the correspondence between image features and scoliosis classification.

[0155] 2. Processing Flow

[0156] Generate concept sets: Based on the input medical imaging data and scoliosis classification information, the system generates multiple concept sets. These concepts define the symptoms, characteristics, and related data of various types of scoliosis, providing basic knowledge for the model.

[0157] Training text encoder: The system processes the concept set through the trained text encoder, converting each concept into a quantifiable text embedding vector. The task of the text encoder is to understand the connection between medical concepts and actual images and generate corresponding concept embeddings.

[0158] Generate text embedding vectors: The processed text encoder generates multiple text embedding vectors (such as ), these vectors represent different category concepts of scoliosis and are used for matching and analysis with image features in subsequent modules.

[0159] Lower part - feature fusion and prediction module

[0160] Objective: To predict the type of scoliosis by using the fusion features of image and text concepts and provide the final diagnosis results.

[0161] 1. Data preprocessing

[0162] Receiving fused prediction data: The system combines X-ray and CT images with text embedding vectors and inputs them into the prediction model as multimodal fusion data.

[0163] Processing through trained fusion encoder: The system jointly processes image and text data through the trained fusion encoder to extract features that are meaningful for scoliosis diagnosis.

[0164] 2. Feature extraction

[0165] Generating feature maps: The fusion encoder converts the input data into feature maps, which represent the diagnostic information of different regions.

[0166] Multi-layer feature representation: The feature extraction process generates multi-level feature representation, which enables the model to capture features of different resolutions and scales in the image to cover the details of various parts of the spine.

[0167] 3. Feature matching and attention calculation

[0168] Matching image features with text embedding vectors: The extracted image features are matched with the text embedding vectors in the concept generation module to establish the association between the image data and the diagnostic concepts.

[0169] Generate concept attention heatmap: The attention mechanism calculates the saliency of each concept in the image and generates an attention heatmap. This heatmap can intuitively show the spinal regions that are important in diagnosis.

[0170] Displaying regional importance weights: The brightness or color depth of the heat map indicates the importance weights of different regions, helping doctors focus on areas where lesions may exist.

[0171] 4. Feature fusion and prediction

[0172] Generate concept score vectors: The system generates a score vector for each concept (such as probability values ​​0.82, 0.13, 0.61, etc.), indicating the possibility of each scoliosis type.

[0173] Through screening threshold processing: filter the results according to the threshold and filter out small probability values ​​to improve the reliability of the diagnosis results.

[0174] Final output concept label: The filtered concept score vector is binarized and the final concept label is output (such as 0 for no lesion and 1 for lesion) to provide doctors with clear diagnostic suggestions.

[0175] The main innovations of the text processing module are as follows:

[0176] 1. Integration of medical images and text concepts: The system can integrate medical image data with text concept information to improve the accuracy of diagnosis.

[0177] 2. Multimodal fusion technology: The multimodal data fusion technology of images and texts is used to enable the model to make more accurate judgments using information from different data sources.

[0178] 3. Attention mechanism: The attention mechanism is introduced to calculate the attention weight of the image area, so that the system can focus on the area with a higher possibility of lesions.

[0179] 4. Probabilistic prediction output: The prediction results are displayed in the form of probability, providing doctors with more valuable diagnostic results.

[0180] The framework features of the text processing module are as follows:

[0181] 1. End-to-end diagnostic process: The system automates the entire process from data input to final output of diagnostic results.

[0182] 2. Combining deep learning with medical knowledge: The system combines medical images and clinical knowledge during training to achieve more accurate diagnosis.

[0183] 3. Strong interpretability: The attention heat map displays the focus area to help doctors intuitively understand the decision-making basis of the model.

[0184] 4. The prediction results are expressed probabilistically: The diagnosis results not only show the type, but also reflect the credibility of the diagnosis results through probability scores.

[0185] The framework of the text processing module is specially designed for scoliosis diagnosis and can achieve the following results:

[0186] Automatic analysis of medical images: The system automatically analyzes input X-ray and CT images and identifies the type of scoliosis.

[0187] Provide diagnostic advice: Output diagnostic results and suggestions to assist doctors in making decisions.

[0188] Displaying areas of interest: Using attention heatmaps to display areas of possible lesions makes the diagnosis process more transparent.

[0189] Provide credibility score: The diagnosis results are output in the form of probability to help doctors evaluate the credibility of the results.

[0190] This is of great help to doctors' clinical diagnosis. It can not only improve the accuracy of diagnosis, but also improve the efficiency of diagnosis and provide patients with more timely treatment guidance.

[0191] In this embodiment, if Figure 5 As shown in the figure, the diagnosis inference module is a multimodal diagnosis and treatment recommendation framework that combines text, medical imaging, and spatial information processing to achieve flexible diagnosis and treatment plan generation. The following is a detailed explanation of each component:

[0192] 1. Input Source

[0193] The system's input sources include information from doctors and patients to comprehensively collect the required data to provide a basis for accurate diagnosis and personalized treatment recommendations:

[0194] Doctor-side system auxiliary information: Data collected through the doctor-side information system may include diagnostic points, patient background, laboratory test results, etc. This type of information helps the system understand the doctor's observations and preliminary diagnostic direction.

[0195] Patient APP auxiliary information: The information collected by the patient-side application may include medical history, symptom self-report, medication records, etc., to supplement the doctor-side information and help the system understand the patient's condition and needs more comprehensively.

[0196] Free text prompt information: The descriptions of doctors and patients are usually expressed in natural language, which are converted into free text format understandable by the system through preprocessing for further analysis.

[0197] X-ray or CT images in any direction: The system can accept X-ray or CT images in different directions and angles to achieve more comprehensive image analysis, thereby improving the accuracy and comprehensiveness of diagnosis.

[0198] 2. Text processing flow

[0199] The text processing pipeline is responsible for converting free text into embedding vectors to support subsequent analysis:

[0200] Pre-trained text encoder: This encoder converts free text information into embedding vectors so that natural language descriptions can be expressed in vector form. This process can use pre-trained models such as BERT and GPT.

[0201] Generate text embedding vectors: The embedding vectors generated by the text encoder can capture the semantic information in the text, supporting the system to understand patient descriptions or doctor observations.

[0202] Intent encoder: The intent encoder further processes the text embedding vector to generate an intent embedding vector, identifying the specific needs in the text information (such as diagnostic intent, treatment intent, etc.), laying the foundation for subsequent comprehensive analysis.

[0203] 3. Image processing flow

[0204] The image processing pipeline normalizes and extracts features from X-ray and CT images to generate image features for diagnosis:

[0205] Normalization network: The image input is processed through the normalization network to eliminate the differences between different devices and different imaging conditions, so that the image reaches a unified quality standard.

[0206] Image normalization alignment function: The alignment function is used to align the anatomical structures in the image to ensure the consistency and comparability of images in different directions, and finally obtain a normalized image.

[0207] Pre-trained fusion encoder: Use a pre-trained model (such as ResNet or CNN) to process the normalized image and extract the required feature vector to generate an image embedding vector.

[0208] Generate image feature vector: The image feature vector contains key visual information that can be used for diagnosis and is fused with text features for comprehensive analysis.

[0209] 4. Spatial Information Processing

[0210] The spatial information processing module processes the spatial information of different anatomical structures to provide assistance for diagnosis:

[0211] Spatial encoder: processes spatial information including point and block information. Point information may refer to a specific area of ​​interest, such as a lesion point; block information refers to a larger anatomical area or tissue block.

[0212] Extracting spatial features: The spatial encoder extracts the features of these spatial information to facilitate understanding of the spatial location and extent of lesions or tissues.

[0213] Fusion with text and image features: Spatial features are fused with the embedding vectors of text and images to achieve multi-dimensional analysis of complex cases.

[0214] 5. Tip the encoder

[0215] The hint encoder is the core of the system, which integrates data from multiple modalities into a comprehensive feature representation:

[0216] Integrate information from different modalities: The prompt encoder receives features from text, image, and spatial encoders and performs weighted fusion to generate a comprehensive information representation.

[0217] Generate a comprehensive feature representation: This representation combines text descriptions, imaging features, and spatial information to ensure the system's comprehensive understanding of the diagnostic object and provides a basis for the final diagnosis and recommendation generation.

[0218] 6. Mask Decoder

[0219] The mask decoder generates specific diagnosis and treatment recommendations based on the comprehensive features:

[0220] Receive all processed features: The decoder takes all fused feature vectors from the prompt encoder to generate an output that meets specific diagnostic or treatment needs.

[0221] Generate final diagnosis and treatment recommendations: The decoder converts comprehensive features into language-based recommendation outputs, providing doctors and patients with reference diagnosis and treatment plans.

[0222] 7. Output results

[0223] The output results present the diagnosis and treatment recommendations generated by the system in a hierarchical manner to meet the needs of different granularity:

[0224] Text-based diagnosis and treatment recommendations: This layer outputs holistic diagnosis and recommendation text, suitable for descriptive diagnostic reports.

[0225] Diagnosis and treatment recommendations based on a certain point: Diagnostic recommendations for specific locations, such as specific recommendations for lesion points, make it easier for doctors to make accurate diagnosis and treatment.

[0226] Block-based diagnosis and treatment recommendations: Treatment recommendations for larger areas or anatomical regions to support broader treatment decisions.

[0227] The main features of the diagnostic inference module framework are as follows:

[0228] 1. Multimodal input: By combining text information, medical images and spatial features, the system can process multi-dimensional data sources and improve the comprehensiveness of diagnosis.

[0229] 2. Multi-level processing: Processing at the text, image, and spatial levels ensures that the system understands the patient's condition from different angles, facilitating comprehensive judgment.

[0230] 3. Flexible output: Through output levels of different granularity, the system can adapt to different diagnosis and treatment needs.

[0231] 4. Integration advantage: The system not only combines the professional judgment of doctors, but also incorporates the feedback information of patients and comprehensive imaging data to achieve more accurate and personalized medical support.

[0232] The innovations of the diagnostic inference module framework are as follows:

[0233] 1. Comprehensive information integration capabilities: The system can integrate multimodal data to achieve a comprehensive patient portrait.

[0234] 2. Multi-level diagnostic recommendations: The system can provide recommendations of different granularities to meet a variety of clinical needs.

[0235] 3. Flexible adaptability: It can process medical images in any direction, improving the practicality of the system.

[0236] 4. Consider the needs of doctors and patients: The system can assist doctors in diagnosis while also providing clear feedback to patients.

[0237] Expected outcomes of the diagnostic inference module framework

[0238] 1. Assist doctors to make more accurate diagnoses: The multimodal information provided by the system can be used as a reference for doctors’ diagnoses, helping to improve the accuracy of diagnoses.

[0239] 2. Provide personalized treatment plans: Based on the patient’s background and medical images, the system can generate treatment recommendations suitable for the individual.

[0240] 3. Improve medical efficiency: Automated information processing and suggestion generation can save doctors’ time and improve efficiency.

[0241] 4. Enhance doctor-patient communication: The suggestions generated by the system are easy for patients to understand and help them better participate in treatment decisions.

[0242] In this embodiment, the application also includes a computer-readable storage medium, on which a computer program is stored. When the computer program runs on a computer, the computer runs an AI-driven full-cycle scoliosis management system, wherein one or more programs also include instructions for the following operations: using a mobile device through a preset app or applet, calling the camera and level of the mobile device, taking an image of the patient's back in a standing position and uploading it to a server, using an intelligent detection model in the background for analysis, and pushing the results to the user end, which includes a personal mobile end, an enterprise mobile end, and a management background.

[0243] In this embodiment, the personal mobile terminal includes a first personal center unit, a first detection center unit, a rehabilitation center unit, a protective gear display and consultation unit, an education and science unit and an AI consultation unit, wherein the first personal center unit is used to provide user registration and login, user basic information management, historical detection record viewing, intelligent reminders, and message notification viewing functions; the first detection center unit is used to guide users to take or upload back photos and walking videos, and obtain test results in real time or at a fixed time; the rehabilitation center unit is used to display the location and contact information of nearby rehabilitation clinics based on positioning; the protective gear display and consultation unit is used to display pictures and descriptions of various protective gear, and provide purchase / replacement channels or consultation methods; the education and science unit is used to popularize scientific knowledge, provide popular science articles, videos and animations about spinal health, and enhance parents' health awareness; the AI ​​consultation unit is used to provide AI intelligent consultation functions.

[0244] In this embodiment, the enterprise mobile terminal includes a second personal center unit and a second detection center unit, wherein the second personal center unit is used to provide enterprise user registration and login functions; the second detection center unit is used to select schools, grades, classes, and students, and guide users to take and upload back photos and walking videos, and finally obtain detection results in real time or at a scheduled time.

[0245] In this embodiment, the management background includes a personnel and authority management unit, an enterprise management unit, a data center unit, a consulting center unit, a cooperative merchant and rehabilitation center management unit, and an agent management and distribution unit, wherein the personnel and authority management unit is used to perform authority classification, assign different authorities to different roles, and ensure system security; the enterprise management unit is used to maintain the basic information of the enterprise and manage the import and export of data; the data center unit is used to count all data to be analyzed, analyzed, and abnormal, generate a school-level spinal health report, analyze the incidence and trend of scoliosis, and finally export and share data; the consulting center unit is used to display all consultation content to be responded to, and can customize the reply; the cooperative merchant and rehabilitation center management unit is used to manage the cooperative merchants and rehabilitation centers; the agent management and distribution unit is used to manage agents and distributors.

[0246] The full-cycle scoliosis management system of the present invention can make full use of the complementary advantages of wearable devices and medical imaging data, extract rich feature representations through deep learning technology, and integrate the professional knowledge of doctors and patients to provide accurate and explainable diagnostic suggestions. Compared with traditional methods, this system can greatly improve diagnostic efficiency and accuracy, facilitate doctors to manage the disease throughout the cycle, enhance communication and trust between doctors and patients, and bring significant value to clinical diagnosis and treatment practice.

[0247] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0248] What is described above is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. AI-driven full-cycle scoliosis management system, characterized by: It includes a data fusion module, a feature extraction module, a text processing module, a diagnosis inference module and an output presentation module; Among them, the data fusion module is used to deeply fuse the sensor data of the wearable device and the X-ray / CT medical image data to generate comprehensive fusion prediction data; the feature extraction module uses a pre-trained fusion encoder to extract and encode the fusion prediction data to generate potential feature representation; the text processing module uses a pre-trained text encoder to encode the free text diagnostic information provided by doctors and patients and extract semantic features; the diagnosis inference module is used to fuse the fusion features and text features, and generate text-based diagnosis and treatment recommendations through a mask decoder; the output presentation module outputs the diagnosis results in the form of probability distribution to enhance interpretability and user trust.

2. The AI-driven full-cycle scoliosis management system according to claim 1, characterized in that: The data fusion module includes a signal extraction module, a multimodal fusion module and a bending prediction module, wherein the signal extraction module extracts multimodal data from multiple sensors of the wearable device and performs preprocessing and feature extraction on the multimodal data; the multimodal fusion module is used to deeply fuse the features of different modalities to obtain a set of comprehensive feature representations; the bending prediction module performs sequence modeling and final bending prediction on the data of the wearable device based on the fused feature vector.

3. The AI-driven full-cycle scoliosis management system according to claim 2, characterized in that: The specific process of the signal extraction module preprocessing and feature extraction of multimodal data is as follows: Receive sensor data: Collect multi-channel sensor data from wearable devices, including accelerometers, gyroscopes, and pressure sensors, to form raw time-domain signal inputs; Background removal preprocessing: eliminate noise interference and background signals, and improve signal quality through filtering or noise removal algorithms; FFT transform: Use fast Fourier transform to convert time domain signals into frequency domain signals to capture frequency characteristics; Time domain signal and frequency domain signal acquisition: Time domain signals and frequency domain signals are respectively regarded as feature inputs to capture different signal performances; Convolutional layer processing: Convolutional neural network extracts features from time domain signals and frequency domain signals; Self-attention layer enhances features: The self-attention mechanism is introduced to weight the extracted features to improve the system's ability to identify key features; The feature extraction layer generates feature vectors: The extracted features are further processed through a fully connected layer or dimensionality reduction technology to generate time domain feature vector 1 and frequency domain feature vector 2.

4. The AI-driven full-cycle scoliosis management system according to claim 3, characterized in that: The specific process of the multimodal fusion module to deeply fuse the features of different modalities is as follows: Multilayer perceptron preliminary feature processing: Multilayer perceptron 1 and multilayer perceptron 2 perform preliminary processing on time domain feature vector 1 and frequency domain feature vector 2 respectively as the first step of feature conversion; Enhancement of residual connection and cross connection: Residual connection and cross connection are used to maintain the integrity of feature information and enhance feature transfer capability; Combination of convolution and max pooling layers: The combination of multiple convolution layers and max pooling layers is used to gradually compress and extract features, filtering out key information in multimodal features; Multimodal fusion of cross-connection architecture: The complex cross-connection structure further realizes the fusion of different modal features; Fusion feature vector generation: The first feature vector and the second feature vector after the fusion of the time domain signal and the frequency domain signal are finally integrated to obtain a fused feature vector.

5. The AI-driven full-cycle scoliosis management system according to claim 4, characterized in that: The specific process of the bending prediction module performing sequence modeling and final bending prediction on the data of the wearable device based on the fusion feature vector is as follows: Long Short-Term Memory Network Sequence Modeling: Use LSTM to model sequence features; Further processing by multi-layer perceptron: The output of LSTM is processed by multi-layer perceptron 3 to further extract and transform features, providing more layers of feature support for prediction; Integration of sensor prediction data: Integrate the sensor data predicted by the model to ensure that features from different sources and different modalities are fully represented in the prediction; Nonlinear transformation of activation function: Before the output layer, the ReLU activation function is applied to perform nonlinear transformation on the data to obtain the prediction result; Output prediction results: The final output prediction results represent the bending prediction value of the wearable device signal, completing the process from data collection, feature extraction to final prediction.

6. The AI-driven full-cycle scoliosis management system according to claim 5, characterized in that: The feature extraction module is a framework for multimodal medical data processing and prediction, which aims to extract information from static images and dynamic sensor data at different time points to make comprehensive medical predictions. The framework of the feature extraction module includes an input data layer, a potential diffusion stage, a potential diffusion model and a data fusion stage; the input data layer provides multimodal input data for the framework, and normalizes the information on the timeline for analysis in subsequent stages; the potential diffusion stage encodes and extracts features from data of different modalities through an encoder to generate a potential representation that is consistent in time; the potential diffusion model integrates multimodal data features and noise to capture the intrinsic relationship between multimodal data; the data fusion stage comprehensively analyzes the scoliosis status of the patient through predictors and fusion functions based on the generated multimodal features.

7. The AI-driven full-cycle scoliosis management system according to claim 6, characterized in that: The text processing module includes a concept generation module and a feature fusion and prediction module, wherein the concept generation module generates a core concept embedding vector for scoliosis diagnosis by combining medical images with text descriptions, providing support for subsequent feature fusion and prediction; the feature fusion and prediction module uses the fusion features of image and text concepts to predict the type of scoliosis and provide the final diagnosis result.

8. The AI-driven full-cycle scoliosis management system according to claim 7, characterized in that: The specific process of the feature fusion and prediction module predicting the type of scoliosis is as follows: S1: Data preprocessing; S101: receiving fusion prediction data, the system combines the X-ray and CT images with the text embedding vector and inputs them into the prediction model as multimodal fusion data; S102: Processing by training a fusion encoder, that is, the system jointly processes the image and text data through the trained fusion encoder to extract features meaningful for scoliosis diagnosis; S2: Feature extraction; S201: generating a feature map, i.e., the fusion encoder converts the input data into a feature map, which represents the diagnostic information of different regions; S202: Multi-layer feature representation, i.e., the feature extraction process generates multi-level feature representation, so that the model can capture features of different resolutions and scales in the image to cover the details of various parts of the spine; S3: Feature matching and attention calculation; S301: Matching image features with text embedding vectors, that is, matching the extracted image features with the text embedding vectors in the concept generation module, thereby establishing an association between the image data and the diagnosis concept; S302: Generate an attention heat map of the concept, that is, calculate the saliency of each concept in the image through the attention mechanism and generate an attention heat map; S303: Displaying the importance weight of the region. The brightness or color depth of the heat map indicates the importance weight of different regions. S4: Feature fusion and prediction; S401: Generate a concept score vector, that is, the system generates a score vector for each concept, indicating the possibility of each scoliosis type; S402: filtering out small probability values ​​through threshold screening processing according to the threshold screening results, so as to improve the reliability of the diagnosis result; S403: finally outputting the concept label, that is, the filtered concept score vector is binarized and the final concept label is output.

9. The AI-driven full-cycle scoliosis management system according to claim 8, characterized in that: The diagnosis inference module is a multimodal diagnosis and treatment suggestion framework that combines text, medical images and spatial information processing to achieve flexible diagnosis and treatment plan generation. The specific process of the diagnosis inference module to achieve diagnosis and treatment plan generation is as follows: Import input sources. The input sources of the system include information from doctors and patients, so as to comprehensively collect the required data and provide a basis for accurate diagnosis and personalized treatment recommendations; Text processing, the text processing pipeline is responsible for converting free text into embedding vectors to support subsequent analysis; Image processing: The image processing process normalizes and extracts features from X-ray and CT images to generate image features for diagnosis; Spatial information processing, that is, processing the spatial information of different anatomical structures to provide assistance for diagnosis; By prompting the encoder to integrate data from multiple modalities into one, a comprehensive feature representation is formed; Generate specific diagnosis and treatment recommendations based on comprehensive features through a mask decoder; The output results present the diagnosis and treatment recommendations generated by the system in a hierarchical manner to meet the needs of different granularities.

10. The AI-driven full-cycle scoliosis management system according to claim 9, characterized in that: It also includes a computer-readable storage medium, on which a computer program is stored. When the computer program runs on a computer, the computer runs the system as described in any one of claims 1 to 9, wherein one or more programs also include instructions for the following operations: using a mobile device through a preset app or applet to call the camera and level of the mobile device, take an image of the patient's back in a standing position and upload it to a server, analyze it using an intelligent detection model in the background, and push the results to a user end, wherein the user end includes a personal mobile end, an enterprise mobile end, and a management background.

11. The AI-driven full-cycle scoliosis management system according to claim 10, characterized in that: The personal mobile terminal includes a first personal center unit, a first detection center unit, a rehabilitation center unit, a protective gear display and consultation unit, an education and science unit, and an AI consultation unit, wherein: The first personal center unit is used to provide user registration and login, user basic information management, historical detection record viewing, intelligent reminder, and message notification viewing functions; The first detection center unit is used to guide the user to take or upload a back photo and a walking video, and obtain the detection results in real time or at a fixed time; The rehabilitation center unit is used to display the location and contact information of nearby rehabilitation clinics based on positioning; The protective gear display and consultation unit is used to display pictures and descriptions of various protective gears and provide purchase / replacement channels or consultation methods; The educational science unit is used to popularize scientific knowledge, provide popular science articles, videos and animations about spinal health, and enhance parents' health awareness; The AI ​​consulting unit is used to provide AI intelligent consulting functions.

12. The AI-driven full-cycle scoliosis management system according to claim 10, characterized in that: The enterprise mobile terminal includes a second person center unit and a second detection center unit, wherein: The second personal center unit is used to provide enterprise user registration and login functions; The second detection center unit is used to select schools, grades, classes, and students, guide users to take and upload back photos and walking videos, and finally obtain detection results in real time or at a scheduled time.

13. The AI-driven full-cycle scoliosis management system according to claim 10, characterized in that: The management backend includes a personnel and authority management unit, an enterprise management unit, a data center unit, a consulting center unit, a cooperative merchant and rehabilitation center management unit, and an agent management and distribution unit. The personnel and authority management unit is used to perform authority classification and assign different authorities to different roles to ensure system security; The enterprise management unit is used to maintain the basic information of the enterprise and manage the import and export of data; The data center unit is used to collect statistics on all data to be analyzed, analyzed, and abnormal, generate a school-level spinal health report, analyze the incidence and trend of scoliosis, and finally export and share the data; The consultation center unit is used to display all consultation contents to be responded to and can customize the responses; The cooperative merchant and rehabilitation center management unit is used to manage the cooperative merchants and rehabilitation centers; The agent management and distribution unit is used to manage agents and distributors.

Images