Full-dimensional multi-modal quantitative assessment method for patients with knee osteoarthritis

Abstract

The application relates to a full-dimension multi-modal quantitative evaluation method for knee osteoarthritis patients, which is applied to the technical field of data evaluation, and the method comprises the following steps: deploying an evaluation environment and time-synchronizing multi-modal sensors; based on the synchronized multi-modal sensors and augmented reality, multi-dimension data of a test personnel performing a preset evaluation task is collected; the multi-dimension data is quality graded to determine the quality grades of various data channels in the multi-dimension data; the objective evidence weight and the test personnel self-evaluation weight are dynamically adjusted based on the quality grades and the multi-dimension data; the objective mapping score calculated based on the multi-dimension data is fused with the test personnel self-evaluation score to generate a fusion score of each evaluation item, and a processed evaluation result and an auditable evaluation report are generated. The application has the effect of improving the accuracy of the test personnel evaluation of the combined cognitive impairment.

Description

Technical Field

[0001] This application relates to the technical field of data evaluation, and in particular to a multi-dimensional, multimodal quantitative evaluation method for patients with knee osteoarthritis. Background Technology

[0002] Total knee arthroplasty (TKA) is the standard treatment for end-stage knee osteoarthritis (KOA), and postoperative efficacy assessment directly impacts rehabilitation and clinical decisions. The currently internationally recognized assessment standard is the NKSS 2.0 scale, which integrates five dimensions: objective indicators, symptoms, satisfaction, expectations, and functional activities, enabling systematic pre- and post-operative evaluation. However, this scale belongs to the participant-reported outcome (PRO) paradigm, highly dependent on the participant's cognitive function and verbal expression ability. Clinical data shows that elderly orthopedic participants have a higher incidence of postoperative cognitive impairment. Due to the decline in higher central nervous system functions, these participants, although able to perceive pain, cannot accurately understand and express scale items, leading to an inability to self-report on core dimensions such as symptoms and satisfaction, resulting in a significant decrease in the reliability and validity of the assessment results.

[0003] Traditional assessments rely on imaging and subjective judgment, failing to fully reflect the true state of rehabilitation. Behavioral observation scales such as PAINAD are highly subjective and have low quantitative accuracy. Multi-source data lacks unified clock synchronization and quality control, resulting in untraceable and non-compliant results. Biased pain assessments can lead to inappropriate use of analgesics, while missing functional assessments result in unreasonable rehabilitation plans, prolonged hospital stays, and increased medical costs. Furthermore, existing systems lack a closed-loop comparison mechanism for pre- and post-operative data, making it difficult to objectively quantify surgical efficacy.

[0004] The existing assessment system is not suitable for test subjects with cognitive impairment. The multimodal data is not synchronized, the quality is uncontrollable, and the assessment of pain and function relies on subjective descriptions, which leads to assessment failure and decision-making bias. Summary of the Invention

[0005] To improve the accuracy of assessments by testers with comorbid cognitive impairment, this application provides a multi-dimensional, multimodal quantitative assessment method for patients with knee osteoarthritis.

[0006] Firstly, this application provides a comprehensive, multimodal quantitative assessment method for patients with knee osteoarthritis, employing the following technical solution: A comprehensive, multimodal quantitative assessment method for patients with knee osteoarthritis includes: An evaluation environment is deployed and the multimodal sensors are time-synchronized to obtain a synchronized multimodal sensor, wherein the multimodal sensor includes a motion sensing module, a physiological signal acquisition module, and an augmented reality interaction module; Based on the aforementioned synchronous multimodal sensor and augmented reality, multidimensional data is collected when testers perform preset evaluation tasks. The multidimensional data includes kinematic data, electrophysiological data, eye-tracking data, acoustic data, facial expression data, and plantar pressure data. The quality of the multi-dimensional data is graded to determine the quality level of each data channel in the multi-dimensional data. Based on the quality level and the pain-related physiological signals in the multi-dimensional data, the weight of objective evidence and the weight of the tester's self-assessment are dynamically adjusted. Based on the adjusted objective evidence weights and the adjusted tester self-evaluation weights, the objective mapping score calculated based on the multi-dimensional data is fused with the tester self-evaluation score obtained through the augmented reality interaction module to generate a fusion score for each evaluation item. Anomaly detection and missing data processing are performed on the fusion score to generate a processing evaluation result; Summarize the processing and evaluation results to generate an auditable evaluation report.

[0007] By employing the above technical solutions, time synchronization of multimodal sensors ensures that multi-dimensional data, including kinematics, electrophysiology, eye movement, acoustics, facial expressions, and plantar pressure, are collected under a unified time reference. This solves the problem of insufficient information from a single modality. The collected data is graded by quality, enabling the identification of data reliability from each channel and preventing low-quality data from interfering with the assessment results. Dynamically adjusting the weights of objective evidence and test subject self-assessment based on quality levels and pain-related physiological signals effectively corrects for underreporting or overreporting of pain that may occur in test subjects with cognitive impairment due to difficulties in verbal expression. This allows for a reasonable fusion of objective physiological signals and subjective self-assessment. Through anomaly detection and missing data handling, the integrity and reliability of the assessment data are guaranteed, thereby significantly improving the accuracy of assessments for test subjects with combined cognitive impairment.

[0008] Optionally, the preset assessment exercise includes symptom assessment tasks, satisfaction assessment tasks, and functional activity assessment tasks; the collection of multi-dimensional data by the tester performing the preset assessment tasks based on the synchronous multimodal sensor and augmented reality includes: For the symptom assessment task, the following data were collected: knee joint movement trajectory, skin conductance peak, pain avoidance index, pupillary pain response, confidence level of pain keyword recognition, and probability of micro-expression action units when the test subjects performed virtual walkway walking and virtual step climbing. For the aforementioned satisfaction assessment task, the non-specific skin conductance response frequency, skin conductance level drift, fixation microtremor power, proportion of aversion expression frames, motion-induced pain, latency of first viewing of the armrest, speech syllable rate, saccade entropy, and knee angular velocity decrease rate were collected from the test subjects when performing sitting, lying down, getting out of bed, simulating housework, and leisure activities. For the aforementioned functional activity assessment task, gait parameters, knee flexion angle, turning angle, step-up and step-down angles, time taken to stand up, getting on and off a vehicle, lateral movement distance, step-up and step-down angles, squatting and kneeling angles, running cadence, and knee trajectory variation coefficients were collected from the test subjects when performing walking and standing, standard activities, advanced activities, and free activities.

[0009] Optionally, the data channels include a motion sensor channel, an electrophysiological channel, an eye-tracking channel, an acoustic channel, and a facial expression channel; the quality grading of the multi-dimensional data, determining the quality level of each data channel in the multi-dimensional data, includes: The data frame loss rate of the motion sensing channel, the baseline drift of the electrophysiological channel, the pupil loss rate of the eye-tracking channel, the keyword recognition confidence of the acoustic channel, and the facial occlusion rate of the facial expression channel are calculated respectively. The calculation results of each data channel are compared with multiple preset quality level threshold ranges to determine the quality level of each data channel; When the quality level of any of the data channels is the lowest level, the uncertainty of the data channel is multiplied by a preset penalty coefficient to obtain the processing uncertainty; The processing uncertainties of each data channel are weighted and squared to generate objective uncertainties.

[0010] Optionally, the dynamic adjustment of the objective evidence weight and the tester's self-rating weight based on the quality level and the pain-related physiological signals in the multi-dimensional data includes: Acquire the skin electrophysiological signals and skin conductivity peak values ​​monitored by the electrophysiological channels; When the quality level of the electrophysiological channel is optimal and the peak value of the skin conductance response exceeds a preset threshold, a weighting adjustment mechanism is triggered. The weight of the skin electrophysiological signal is increased to a first preset ratio within a preset time period, and the remaining weight is allocated to the objective evidence weight and the self-evaluation weight of the tester according to the original ratio and then normalized. When the quality level of any of the data channels is the lowest level, a quality degradation process is triggered. The weight of the data channel is reduced by a second preset ratio, and the weights of the other data channels are proportionally amplified and then normalized again. When the deviation between the objective mapping score corresponding to the objective evidence weight and the tester's self-score exceeds a preset score threshold, multimodal conflict handling is triggered. Increase the weight of the objective evidence and decrease the weight of the self-assessment.

[0011] Optionally, the step of fusing the objective mapping score calculated based on the multi-dimensional data with the tester's self-assessment score obtained through the augmented reality interaction module, according to the adjusted objective evidence weight and the adjusted tester self-assessment weight, to generate a fusion score for each evaluation item includes: Obtain the item type of the evaluation item and the key parameters in the multi-dimensional data; Based on the entry type, select the corresponding formula from the preset mapping formulas as the formula to be used; Calculate the objective mapping score based on the formula to be used and the key parameters; A visual scoring interface is displayed to the testers using augmented reality devices, and the testers' selection data is collected as their self-scoring. The fusion score is calculated based on the adjusted objective evidence weights, objective mapping scores, self-assessment weights, and tester self-assessment scores. The item uncertainty of the fusion score is calculated based on the adjusted objective evidence weights, the objective uncertainty, and the preset uncertainty of the tester's self-assessment.

[0012] Optionally, the step of performing anomaly detection and missing value processing on the fusion score to generate a processing evaluation result includes: Obtain the raw data corresponding to the fusion score; Based on the raw data, determine whether there are any preset abnormal conditions; If a preset abnormal condition exists, the corresponding evaluation item will be invalidated and its weight will be reset to zero. When the number of evaluation entries marked as invalid in the same dimension is one, the interpolation algorithm is triggered; The interpolated value is calculated based on the scores of other valid entries in the current dimension, and the uncertainty of the interpolated value is amplified by a preset factor. When the number of entries marked as invalid is greater than one, the current dimension will be marked as pending review and left blank for clinical supplementation. When the assessment type is postoperative assessment, retrieve the preoperative assessment total score, postoperative assessment total score, and expected improvement value of the same test subject; The score difference is calculated based on the combined preoperative and postoperative assessment scores; Based on the ratio of the score difference to the preoperative expected improvement value, the expected satisfaction level is generated. Based on the results of anomaly detection and missing data handling, a processing evaluation result is generated.

[0013] Optionally, the step of summarizing the processing evaluation results and generating an auditable evaluation report includes: The fusion scores and uncertainties of each of the data dimensions are summarized to obtain the total fusion evaluation score and total uncertainty. The scores, uncertainties, total fusion evaluation scores, total uncertainties, and expected satisfaction levels of each data dimension are assembled into an auditable evaluation report according to a preset format.

[0014] Secondly, this application provides a multi-dimensional, multimodal quantitative assessment system for patients with knee osteoarthritis, employing the following technical solution: A comprehensive, multimodal quantitative assessment system for patients with knee osteoarthritis includes: An environment deployment synchronization module is used to deploy and evaluate the environment and synchronize the time of the multimodal sensors to obtain synchronized multimodal sensors, wherein the multimodal sensors include a motion sensing module, a physiological signal acquisition module, and an augmented reality interaction module; The dimensional data acquisition module is used to acquire multidimensional data when testers perform preset evaluation tasks based on the synchronous multimodal sensor and augmented reality. The multidimensional data includes kinematic data, electrophysiological data, eye-tracking data, acoustic data, facial expression data, and plantar pressure data. The quality level determination module is used to classify the quality of the multi-dimensional data and determine the quality level of each data channel in the multi-dimensional data. The weighted data adjustment module is used to dynamically adjust the weight of objective evidence and the weight of the tester's self-assessment based on the quality level and the pain-related physiological signals in the multi-dimensional data. The fusion score generation module is used to fuse the objective mapping score calculated based on the multi-dimensional data with the tester's self-evaluation score obtained through the augmented reality interaction module, based on the adjusted objective evidence weight and the adjusted tester's self-evaluation weight, to generate a fusion score for each evaluation item. The evaluation result generation module is used to perform anomaly detection and missing value processing on the fusion score and generate the processed evaluation result. The evaluation report generation module is used to summarize the evaluation results and generate an auditable evaluation report.

[0015] By employing the above technical solutions, time synchronization of multimodal sensors ensures that multi-dimensional data, including kinematics, electrophysiology, eye movement, acoustics, facial expressions, and plantar pressure, are collected under a unified time reference. This solves the problem of insufficient information from a single modality. The collected data is graded by quality, enabling the identification of data reliability from each channel and preventing low-quality data from interfering with the assessment results. Dynamically adjusting the weights of objective evidence and test subject self-assessment based on quality levels and pain-related physiological signals effectively corrects for underreporting or overreporting of pain that may occur in test subjects with cognitive impairment due to difficulties in verbal expression. This allows for a reasonable fusion of objective physiological signals and subjective self-assessment. Through anomaly detection and missing data handling, the integrity and reliability of the assessment data are guaranteed, thereby significantly improving the accuracy of assessments for test subjects with combined cognitive impairment.

[0016] Thirdly, this application provides an electronic device that adopts the following technical solution: An electronic device includes a processor coupled to a memory; The processor is configured to execute a computer program stored in the memory, such that the electronic device executes the computer program of the full-dimensional multimodal quantitative assessment method for patients with knee osteoarthritis as described in any of the first aspects.

[0017] Fourthly, this application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium storing a computer program capable of being loaded by a processor and executing the full-dimensional multimodal quantitative assessment method for patients with knee osteoarthritis as described in any of the first aspects.

[0018] In summary, this application includes at least one of the following beneficial technical effects: Multimodal sensor time synchronization ensures that kinematic, electrophysiological, eye-tracking, acoustic, facial expression, and plantar pressure data are collected under a unified time reference, addressing the issue of insufficient information from a single modality. Data quality grading identifies the reliability of data from each channel, preventing low-quality data from interfering with assessment results. Dynamically adjusting the weights of objective evidence and test subject self-reporting based on quality grades and pain-related physiological signals effectively corrects for underreporting or overreporting of pain that may occur in test subjects with cognitive impairment due to difficulties in verbal expression. This allows for a reasonable fusion of objective physiological signals and subjective self-reporting. Anomaly detection and missing data handling ensure the integrity and reliability of assessment data, significantly improving the accuracy of assessments for test subjects with combined cognitive impairment. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a multi-dimensional, multimodal quantitative assessment method for patients with knee osteoarthritis provided in this application embodiment.

[0020] Figure 2 This is a structural block diagram of a multi-dimensional, multimodal quantitative assessment system for patients with knee osteoarthritis provided in this application embodiment.

[0021] Figure 3 This is a structural block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0022] The present application will be further described in detail below with reference to the accompanying drawings.

[0023] This application provides a method for comprehensive multimodal quantitative assessment of patients with knee osteoarthritis. This method can be executed by an electronic device, which can be a server or a terminal device. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The terminal device can be a smartphone, tablet, desktop computer, etc., but is not limited to these.

[0024] Figure 1 This is a flowchart illustrating a multi-dimensional, multimodal quantitative assessment method for patients with knee osteoarthritis, provided in an embodiment of this application.

[0025] like Figure 1 As shown, the main process of this method is described below (steps S101 to S107): Step S101: Deploy the evaluation environment and synchronize the multimodal sensors in time to obtain synchronized multimodal sensors, wherein the multimodal sensors include a motion sensing module, a physiological signal acquisition module, and an augmented reality interaction module.

[0026] In this embodiment, the deployment evaluation environment is a complete test scenario that includes physical space calibration, sensor array configuration, and software architecture initialization. The motion sensing module specifically includes a 60GHz millimeter-wave radar and an inertial measurement unit (IMU) for real-time capture of knee joint movement trajectory, angle, and gait parameters. The physiological signal acquisition module includes a skin conductance response wristband, a PVDF piezoelectric film throat vibration patch, and a pressure-sensing insole for collecting skin conductance responses, speech acoustic features, and plantar pressure distribution. The augmented reality interaction module includes eye-tracking glasses with a see-through optical module for presenting a virtual evaluation scenario and collecting eye movement data and facial expressions. Through the coordinated deployment and nanosecond-level time synchronization of the motion sensing module, physiological signal acquisition module, and augmented reality interaction module, the temporal consistency of subsequent multi-dimensional data can be ensured, providing a reliable spatiotemporal benchmark for multimodal fusion analysis.

[0027] Step S102: Based on synchronous multimodal sensors and augmented reality, collect multidimensional data when testers perform preset evaluation tasks. The multidimensional data includes kinematic data, electrophysiological data, eye-tracking data, acoustic data, facial expression data, and plantar pressure data.

[0028] For step S102, for the symptom assessment task, data were collected on the knee joint movement trajectory, peak skin conductance response, pain avoidance index, pupillary pain response, confidence level of pain keyword recognition, and probability of micro-expression action units when the test subjects performed virtual walkway walking and virtual stair climbing. For the satisfaction assessment task, data were collected on the non-specific skin conductance response frequency, skin conductance level drift, fixation micro-tremor power, proportion of aversion expression frames, motion-induced pain, latency of first looking at the handrail, speech syllable rate, saccade entropy, and knee angular velocity decrease rate when the test subjects performed sitting, lying down, getting out of bed, simulating housework, and leisure activities. For the functional activity assessment task, data were collected on the gait parameters, knee flexion angle, turning angle, stair climbing angle, time to stand up, getting on and off a vehicle, lateral movement distance, stair climbing angle, squatting and kneeling angle, running cadence, and knee trajectory variation coefficient when the test subjects performed walking and standing, standard activities, advanced activities, and free activities.

[0029] In this embodiment, the preset assessment movements are based on the three core assessment dimensions of the NKSS2.0 scale structure. Differentiated scenario deployments address the problem that patients with cognitive impairment cannot accurately describe their symptoms verbally. The symptom assessment task mainly focuses on the patient's pain perception and avoidance behavior under dynamic load; the satisfaction assessment task focuses on the patient's subjective comfort and emotional response in resting or daily life simulation scenarios; and the functional activity assessment task covers the complete functional spectrum from basic walking to higher-order movements.

[0030] Virtual walkways and virtual stair climbing are visualized guided paths projected onto the floor of the assessment room using augmented reality devices, corresponding to two typical pain-inducing scenarios: weight-bearing on flat ground and stair climbing, respectively. Knee joint movement trajectory is calculated using millimeter-wave radar to track the coordinates of key points on the affected knee in real time, determining the spatial path and flexion angle changes, quantifying the limitation of joint mobility during movement. Skin conductance peak is the instantaneous maximum conductivity caused by sympathetic nerve excitation, monitored by a skin conductance wristband, directly reflecting the intensity of physiological stress triggered by pain. The pain avoidance index is the percentage of time spent staring at the healthy knee or handrail, calculated based on eye-tracking data; a higher value indicates a stronger avoidance response to pain. Pupillary pain response is the involuntary dilation of the pupil diameter under pain stimulation, serving as an objective indicator of limbic system pain processing. Pain keyword recognition confidence is the probability value of identifying specific words such as "pain" and "tired" after acquiring speech signals through a laryngeal vibration patch and decoding them using an acoustic model. Micro-expression action unit probability is the probability of specific muscle movements such as frown lines or expressions of disgust detected using a facial expression analysis model.

[0031] Sitting, lying down, getting out of bed, simulating housework, and leisure activities are five types of life-oriented assessment scenarios constructed through AR scenes, each corresponding to different sub-items of satisfaction in the NKSS2.0 scale; Non-specific skin conductance response frequency is the number of spontaneous skin conductance fluctuations per unit time in the sitting state, reflecting the baseline anxiety or discomfort level; Skin conductance level drift is the slope of the skin conductance baseline change over time in the lying state, used to assess nocturnal pain or persistent dull pain; Fixation microtremor power is the frequency domain energy of micro-tremors of the eyeballs when the patient fixates on a low-stimulation visual anchor point, and an abnormally high power suggests potential pain interference or inattention; Disgust expression frame proportion is the proportion of frames in the video stream in which disgust micro-expressions are detected out of the total number of frames; Motor-induced pain is the peak increment of skin conductance recorded at the moment of performing actions such as getting out of bed or doing housework. The initial latency to the handrail is the time interval from the start of the movement to when the gaze first falls on the handrail or other support, reflecting the degree of need for stability; speech syllable rate is the number of effective syllables emitted per unit time, and a slower speech rate is often associated with pain or fatigue; saccadic entropy is an information entropy value that measures the degree of disorder in the eye movement scanning path, and a high entropy value indicates disordered visual search strategies, commonly seen when cognitive load is too high or pain distracts attention; knee angular velocity decay rate is the proportion of knee joint angular velocity attenuation relative to baseline during simulated household chores, reflecting a decline in muscle control.

[0032] Walking and standing, standard activity, advanced activity, and free activity are four functional sub-dimensions categorized according to progressive movement complexity. Gait parameters include basic kinematic indices such as stride length, cadence, and gait period symmetry; knee flexion angle, turn angle, step-up / step-down angle, ladder-up / step-down angle, and squatting / kneeling angle all refer to the maximum or minimum angle values ​​reached by the knee joint in a specific movement phase, used to assess joint range of motion; standing up time is the time required to go from a seated position to a fully upright position, reflecting lower limb explosive power; getting on and off vehicles and lateral movement distance are spatial indicators for assessing complex coordination and balance; running cadence is the number of steps taken per unit time; the knee trajectory coefficient of variation is a statistic that measures the consistency of knee joint movement trajectory in multiple repetitive movements, with a larger coefficient of variation indicating less stable motor control.

[0033] Step S103: Perform quality grading on the multi-dimensional data to determine the quality level of each data channel in the multi-dimensional data.

[0034] For step S103, the frame loss rate of the motion sensing channel, the baseline drift of the electrophysiological channel, the pupil loss rate of the eye-tracking channel, the keyword recognition confidence of the acoustic channel, and the facial occlusion rate of the facial expression channel are calculated respectively. The calculation results of each data channel are compared with multiple preset quality level threshold ranges to determine the quality level of each data channel. When the quality level of any data channel is the lowest level, the uncertainty of the data channel is multiplied by a preset penalty coefficient to obtain the processing uncertainty. The processing uncertainty of each data channel is weighted and squared to generate the objective uncertainty.

[0035] In this embodiment, the data channels specifically include a motion sensor channel, an electrophysiological channel, an eye-tracking channel, an acoustic channel, and a facial expression channel. Each channel corresponds to a specific physical sensor and a dedicated quality quantification index. The motion sensor channel's frame loss rate is the proportion of valid data frames that failed to be successfully received or parsed within a preset time window, directly reflecting the continuity of motion trajectory data. The electrophysiological channel, derived from a skin conductance wristband, measures baseline drift as the non-specific, slow change in skin conductance over time at rest, characterizing electrode contact stability and the degree of interference from sweating. The eye-tracking channel, derived from the eye-tracking module of augmented reality glasses, measures pupil loss rate as the proportion of frames in the sampling period where the pupil center coordinates could not be locked by the algorithm due to blinking, occlusion, or sudden changes in illumination. The acoustic channel, derived from a throat vibration patch or microphone array, measures keyword recognition confidence as the probability score when the speech recognition model outputs specific pain keywords, ranging from 0 to 1. The facial expression channel, derived from a video stream captured by a camera, measures facial occlusion rate as the proportion of the total facial area where key facial features are obscured by hands, medical devices, or environmental objects.

[0036] Quality levels are typically categorized into best, intermediate, and lowest levels, each corresponding to a specific numerical threshold range. These ranges are determined through maximum likelihood estimation based on extensive preclinical experimental data. Different channels correspond to different numerical ranges, and the specific numerical ranges need to be set according to the actual situation; no specific limitations are made here.

[0037] The lowest grade indicates poor data quality in that channel, posing a high risk of measurement error. Uncertainty is a statistic characterizing the dispersion of measurement results for a single data channel; initial uncertainty is typically obtained based on the inherent accuracy calibration of the sensor. The preset penalty coefficient is a constant greater than 1, preferably 1.5 in this embodiment. When the QualGrad quality grading algorithm determines a channel's quality grade as C, the system automatically triggers a penalty mechanism, multiplying the channel's current initial uncertainty by 1.5 to obtain the amplified processed uncertainty. Objective uncertainty is a comprehensive index characterizing the overall data reliability of the entire multimodal evaluation system at the current moment. Using the root sum of squares method, the processed uncertainties of all five data channels after the above processing are obtained. Then, the processed uncertainties of each channel are weighted, the weighted sum of squares is calculated, and finally, the square root of the sum of squares is taken.

[0038] Step S104: Based on the quality level and pain-related physiological signals in the multi-dimensional data, dynamically adjust the weights of objective evidence and tester self-rating. For step S104, the skin electrophysiological signals and skin conductivity response peaks monitored by the electrophysiological channels are acquired. When the quality level of the electrophysiological channel is optimal and the skin conductivity response peak exceeds a preset threshold, a weighting mechanism is triggered. The weight of the skin electrophysiological signal is increased to a first preset ratio within a preset time period, and the remaining weight is allocated to the objective evidence weight and the tester's self-evaluation weight according to the original ratio and then normalized. When the quality level of any data channel is the lowest, a quality degradation process is triggered. The weight of the data channel is decreased to a second preset ratio, and the weights of other data channels are proportionally increased and then normalized. When the deviation between the objective mapping score corresponding to the objective evidence weight and the tester's self-evaluation score exceeds a preset score threshold, a multimodal conflict processing is triggered. The objective evidence weight is increased and the self-evaluation weight is decreased.

[0039] In this embodiment, the skin electrophysiological signal is continuous time-series data acquired by a skin conductance response sensor, reflecting the activation level of the test subject's sympathetic nervous system. The peak value of the skin conductance response is the maximum transient amplitude extracted after preprocessing the original skin electrophysiological signal using zero-phase bandpass filtering, measured in micro-Siemens. A dynamic weighting service thread monitors the data stream of the electrophysiological channel in real time, calculates the baseline drift within a sliding window to determine the channel quality level, and simultaneously detects the signal peak value. By extracting this key feature in real time, a highly accurate physiological basis can be provided for subsequent dynamic weighting adjustments, ensuring that the system can capture severe pain responses that patients with cognitive impairment may not be able to express verbally.

[0040] The optimal level is defined by the data quality grading algorithm, indicating that the baseline drift of the electrophysiological channel is less than a preset lower limit and the signal integrity meets the optimal level standard. The preset threshold is a pain judgment threshold set based on the maximum likelihood estimation of the preoperative cognitive impairment population. Its function is to identify significant pain physiological events, thereby initiating a correction procedure for pain underreporting, provided that the data is confirmed to be highly reliable. The first preset ratio is the temporary target proportion of skin electrophysiological signals in the fusion calculation, and the preset duration is the time window for maintaining the weight in an upward state, which can be set according to the duration of the evaluation item. By changing the coefficient distribution in the multimodal fusion formula, the contribution of physiological signals to the final score is significantly improved. The lowest level is defined by the data quality grading algorithm, indicating that the data frame loss rate, occlusion rate, or drift of a certain channel exceeds the maximum tolerance limit. The second preset ratio is the reduction in weight for low-quality channels, which can be dynamically calculated based on the quality level difference. By reducing the confidence of unreliable data and redistributing the released weight share to high-quality channels, adaptive balance of multimodal data is achieved.

[0041] The objective mapping score is a quantitative score calculated based on multi-dimensional physiological data, while the test subject's self-rating is subjective selection data obtained through an augmented reality interaction module. A preset score threshold is a critical value used to identify and handle inconsistencies between subjective and objective scores caused by patient emotional fluctuations, cognitive biases, or misunderstandings. The absolute value of the difference between the two is calculated in real time; once it exceeds the threshold, it is considered a multimodal conflict. Up and down adjustments are automatic adjustments made by the system to correct the fusion strategy in multimodal conflict scenarios, increasing the weight of objective physiological data in the final score while suppressing the influence of potentially biased subjective self-ratings. In practice, the system increases the weight coefficient of objective evidence and correspondingly decreases the weight coefficient of self-ratings, then recalculates the fusion score.

[0042] Step S105: Based on the adjusted objective evidence weights and the adjusted tester self-evaluation weights, the objective mapping score calculated based on multi-dimensional data is fused with the tester self-evaluation score obtained through the augmented reality interaction module to generate a fusion score for each evaluation item.

[0043] For step S105, obtain the item type and key parameters from the multi-dimensional data of the evaluation items; based on the item type, select the corresponding formula from the preset mapping formula as the formula to be used; calculate the objective mapping score based on the formula to be used and the key parameters; display a visual scoring interface to the testers using an augmented reality device, and collect the testers' selection data as the testers' self-score; calculate the fusion score based on the adjusted objective evidence weight, objective mapping score, self-score weight, and testers' self-score; calculate the item uncertainty of the fusion score based on the adjusted objective evidence weight, objective uncertainty, and the preset uncertainty of the testers' self-score.

[0044] In this embodiment, the item type is the classification identifier of a specific sub-dimension of the currently executed NKSS2.0 scale, used to distinguish different task scenarios such as symptom assessment, satisfaction assessment, or functional activity assessment. Key parameters are a set of feature values ​​extracted from the synchronous multimodal sensor that are directly related to the clinical significance of the current item. Their source depends on the data stream that has undergone quality grading and weight adjustment. These parameters are passed from the acquisition layer to the fusion service thread via a Redis cache queue, serving as input variables for subsequent formula matching.

[0045] The preset mapping formula is a library of mathematical models customized for different item types, pre-stored in the SQLite database of the edge computing box. Its function is to transform the objective evidence dynamically adjusted according to the above embodiments into mapping scores that conform to clinical dimensions. The selection process is achieved by querying the item ID index. The system automatically calls the matching weighted summation algorithm or nonlinear transformation function according to the determined item type. The objective mapping is divided into quantitative values ​​reflecting the patient's objective physiological state obtained by substituting dimensionless or multidimensional key parameters into the formula to be used and then normalizing them. The value range is strictly limited to the full score interval of the corresponding item. The calculation process is to substitute the real-time key parameter values ​​obtained above into the selected formula coefficients for linear combination or threshold truncation operations, thereby realizing the mathematical transformation from raw sensor signals to clinically interpretable scores, eliminating the fusion barrier caused by the difference in dimensions of different modal data, and providing a unified benchmark for subsequent weighted fusion with self-assessment data. The visual scoring interface is an interactive graphical user interface rendered in the patient's field of vision using augmented reality devices such as Microsoft HoloLens 2, requiring no upper limb manipulation. Its forms include gaze-locked pain thermometer scales, emoji expression rails, or moon texture anchors. The tester's self-score is data on subjective choices made by the patient under AR guidance through eye-tracking gaze or minimal voice commands. Its purpose is to provide a low-cognitive-load subjective feedback channel for patients with cognitive impairment who are unable to provide complex verbal descriptions.

[0046] The fusion score is a weighted average of the objective physiological signal mapping score and the patient's subjective self-rating score. Its calculation is based on the dynamically adjusted objective evidence weight and self-rating weight as described in the above embodiment. The dynamic nature of the weights is reflected in the following: if high-quality and significant pain physiological signals are detected in the preceding steps, the objective evidence weight is automatically increased to correct for potential underreporting of pain in patients with cognitive impairment; conversely, if physiological signals are missing or of low quality, the self-rating weight is appropriately increased. The item uncertainty is a quantitative indicator characterizing the confidence interval of the fusion score, used to meet the regulatory requirements for data traceability and error quantification in software medical devices. Its calculation is based on the law of error propagation, synthesizing the uncertainty of objective measurements and the preset uncertainty of subjective ratings according to the adjusted weights.

[0047] Step S106: Perform anomaly detection and missing value processing on the fusion score to generate processing evaluation results.

[0048] For step S106, obtain the original data corresponding to the fusion score; determine whether there are preset abnormal conditions based on the original data; if preset abnormal conditions exist, mark the corresponding evaluation item as invalid and perform weight reset to zero; when the number of evaluation items marked as invalid in the same dimension is one, trigger the imputation algorithm; calculate the imputation value based on the scores of other valid items in the current dimension, and amplify the uncertainty of the imputation value by a preset factor; when the number of items marked as invalid is greater than one, mark the current dimension as pending review and leave it blank for clinical supplementation; when the evaluation type is postoperative evaluation, retrieve the preoperative evaluation total score, postoperative evaluation total score, and preoperative expected improvement value of the same tester; calculate the score difference based on the preoperative evaluation combined with the postoperative and postoperative evaluation total scores; generate the expected satisfaction level based on the ratio of the score difference to the preoperative expected improvement value; generate the processing evaluation result based on the results of abnormality detection and missing value processing.

[0049] In this embodiment, the raw data consists of the fusion scores and corresponding uncertainties of each evaluation item generated above, as well as the raw signals from the multimodal sensors supporting the calculation of the fusion scores. Specifically, raw physiological signals aligned with the current fusion score timestamp are extracted from the InfluxDB time-series database or Redis cache in the data layer, including peak values ​​of skin conductance response, pupillary pain response, pain avoidance index, microexpression movement unit probability, and confidence level of laryngeal vibration keyword recognition. Predefined anomaly conditions are a predefined set of logical rules used to identify invalid evaluations or contradictory physiological signals. An anomaly handling service compares the extracted raw data with preset thresholds to identify anomalies such as painless artifacts, signal loss, or physiological-behavioral conflicts.

[0050] The weight reset to zero process forces the contribution of evaluation items marked as invalid to zero in subsequent total score calculations, eliminating the interference of outlier data on the overall evaluation results. Once an item is determined to meet the preset outlier conditions, its status flag is immediately updated to invalid, and its objective evidence weight and self-assessment weight in the fusion calculation are simultaneously adjusted to 0. The imputation algorithm is a method to estimate the missing value by utilizing the statistical characteristics of other valid items in the same dimension when a single item is missing or invalid. The system first counts the validity status of all sub-items under the current evaluation dimension. If only one sub-item is marked as invalid, the imputation mechanism based on the intra-class correlation coefficient is automatically activated. The imputed value is a substitute score calculated based on the mean of valid items in the same dimension or a regression model, while uncertainty amplification is used to reflect the reduced reliability of the imputed data relative to the measured data. In specific calculations, a two-way random effects model is used, with the mean score of other valid items in the current dimension as the basis for the imputed value. At the same time, to quantify the additional error brought by imputation, the uncertainty of the imputed value is set to a preset multiple of the original average uncertainty.

[0051] The "pending review" status represents a conservative approach taken when multiple data anomalies occur within the same dimension, causing statistical imputation to lose its reliable basis. If two or more invalid entries are detected within the same dimension, the overall data quality for that dimension is deemed insufficient to support automatic algorithmic repair. Therefore, the total score for that dimension is left blank, and the dimension is marked as "pending review" in red on the doctor's interface. At this time, no imputation values ​​are generated; instead, a clinical data entry instruction is generated, prompting the doctor to manually reassess or supplement the data through other clinical means.

[0052] The preoperative and postoperative assessment scores are the NKSS2.0 full-dimensional quantitative scores obtained by the patient before and after surgery using this method. The preoperative expected improvement value is the expected rehabilitation target value entered by the doctor or selected by the patient through the AR interface during the preoperative stage. After recognizing that the current assessment task is a postoperative assessment, the system automatically connects to the built-in database and queries the data table. The score difference is the difference between the postoperative assessment score and the preoperative assessment score, used to quantify the actual functional improvement brought about by the surgery. This value reflects the net gain of the patient in the three dimensions of symptoms, satisfaction, and functional activity after treatment and rehabilitation. Expectation satisfaction is a quantitative indicator that measures whether the surgical effect has met the patient's preoperative expectations. It is calculated by dividing the score difference calculated in the previous steps by the preoperative expected improvement value and converting it into a percentage. This indicator links objective functional improvement with subjective expected goals, and can intuitively reflect the cost-effectiveness of treatment or the achievement of the patient's psychological expectations. The processed assessment result is a final structured data object that integrates information such as abnormality markings, imputation data, review status, and expectation satisfaction. This results package not only includes quantified scores but also carries metadata on data quality to ensure the auditability and clinical credibility of the assessment results.

[0053] Step S107: Summarize and process the evaluation results to generate an auditable evaluation report.

[0054] For step S107, the fusion scores and uncertainties of each data dimension are summarized to obtain the total fusion evaluation score and total uncertainty; the scores, uncertainties, total fusion evaluation score, total uncertainty, and expected satisfaction of each data dimension are assembled into an auditable evaluation report according to a preset format.

[0055] In this embodiment, a comprehensive quantitative index is constructed based on the evaluation results of each dimension generated in the preceding steps. The data dimensions specifically cover the symptom dimensions in the NKSS 2.0 scale. The processor reads the fusion score of each item after anomaly detection and missing value processing, along with its corresponding item uncertainty. The fusion evaluation total score is obtained through summation, and the total uncertainty is synthesized through square root calculation. Specifically, the fusion evaluation total score is the sum of the scores of each valid dimension, while the total uncertainty follows the error propagation law in the measurement uncertainty representation guidelines, calculated as the square root of the sum of the squares of the uncertainties of each independent component. The dispersed quantitative data is then transformed into standardized deliverables that meet the requirements. The preset format includes structured data files and single-line auditable strings. The expected improvement value before surgery and the postoperative fusion evaluation total score are extracted from the database. The expected satisfaction level is calculated by combining the preoperative total score. Subsequently, the scores, uncertainties, total score, total uncertainty, and expected satisfaction level of each dimension, along with data quality status identifiers and interpolation identifiers, are assembled into a single-line string according to specific delimiters. This compact and information-dense format encapsulation can meet the requirements for the integrity, consistency and traceability of electronic records, enabling clinicians to grasp the reliability of assessment results and the achievement of patient expectations within seconds.

[0056] Figure 2 The structural block diagram of a multi-dimensional, multimodal quantitative assessment system 200 for patients with knee osteoarthritis provided in the application embodiment is shown below.

[0057] like Figure 2 As shown, the comprehensive multimodal quantitative assessment system 200 for patients with knee osteoarthritis mainly includes: The environment deployment synchronization module 201 is used to deploy the evaluation environment and synchronize the time of the multimodal sensors to obtain synchronized multimodal sensors, wherein the multimodal sensors include a motion sensing module, a physiological signal acquisition module and an augmented reality interaction module; The dimensional data acquisition module 202 is used to acquire multidimensional data when testers perform preset evaluation tasks based on synchronous multimodal sensors and augmented reality. The multidimensional data includes kinematic data, electrophysiological data, eye-tracking data, acoustic data, facial expression data, and plantar pressure data. The quality level determination module 203 is used to classify the quality of multi-dimensional data and determine the quality level of each data channel in the multi-dimensional data. The weighted data adjustment module 204 is used to dynamically adjust the weights of objective evidence and testers’ self-evaluation based on the quality level and pain-related physiological signals in multi-dimensional data. The fusion score generation module 205 is used to fuse the objective mapping score calculated based on multi-dimensional data with the tester's self-evaluation score obtained through the augmented reality interaction module, based on the adjusted objective evidence weight and the adjusted tester's self-evaluation weight, to generate a fusion score for each evaluation item. The evaluation result generation module 206 is used to perform anomaly detection and missing data processing on the fusion score and generate the processed evaluation result. The assessment report generation module 207 is used to summarize and process the assessment results and generate an auditable assessment report.

[0058] As an optional implementation of this embodiment, the dimensional data acquisition module 202 is specifically used for: For symptom assessment tasks, collecting data on knee joint movement trajectory, peak skin conductance response, pain avoidance index, pupillary pain response, confidence level of pain keyword recognition, and probability of micro-expression action units when test subjects perform virtual walkway walking and virtual stair climbing; For satisfaction assessment tasks, collecting data on non-specific skin conductance response frequency, skin conductance level drift, fixation micro-tremor power, proportion of aversion expression frames, movement-induced pain, latency of first-time handrail viewing, speech syllable rate, saccade entropy, and knee angular velocity decrease rate when test subjects perform sitting, lying down, getting out of bed, simulating housework, and leisure activities; For functional activity assessment tasks, collecting data on gait parameters, knee flexion angle, turning angle, stair climbing angle, time to stand up, getting on and off vehicles, lateral movement distance, stair climbing angle, squatting and kneeling angle, running cadence, and knee trajectory variation coefficient when test subjects perform walking and standing, standard activities, advanced activities, and free activities.

[0059] As an optional implementation of this embodiment, the quality level determination module 203 is specifically used to calculate the data frame loss rate of the motion sensing channel, the baseline drift of the electrophysiological channel, the pupil loss rate of the eye-tracking channel, the keyword recognition confidence of the acoustic channel, and the facial occlusion rate of the facial expression channel; compare the calculation results of each data channel with multiple preset quality level threshold intervals to determine the quality level of each data channel; when the quality level of any data channel is the lowest level, multiply the uncertainty of the data channel by a preset penalty coefficient to obtain the processing uncertainty; and perform a weighted square root operation on the processing uncertainty of each data channel to generate the objective uncertainty.

[0060] As an optional implementation of this embodiment, the weight data adjustment module 204 is specifically used to acquire the skin electrophysiological signal and skin conductivity response peak value monitored by the electrophysiological channel. When the quality level of the electrophysiological channel is the optimal level and the skin conductivity response peak value exceeds a preset threshold, a weight adjustment mechanism is triggered; the weight of the skin electrophysiological signal is increased to a first preset ratio within a preset time period, and the remaining weight is allocated to the objective evidence weight and the tester's self-evaluation weight according to the original ratio and then normalized; when the quality level of any data channel is the lowest level, a quality degradation process is triggered; the weight of the data channel is decreased by a second preset ratio, and the weights of other data channels are proportionally increased and then normalized; when the deviation between the objective mapping score corresponding to the objective evidence weight and the tester's self-evaluation score exceeds a preset score threshold, a multimodal conflict processing is triggered; the objective evidence weight is increased and the self-evaluation weight is decreased.

[0061] As an optional implementation of this embodiment, the fusion score generation module 205 is specifically used to obtain the item type of the evaluation item and the key parameters in the multi-dimensional data; based on the item type, select the corresponding formula from the preset mapping formula as the formula to be used; calculate the objective mapping score based on the formula to be used and the key parameters; display a visual scoring interface to the testers based on the augmented reality device, and collect the testers' selection data as the testers' self-score; calculate the fusion score based on the adjusted objective evidence weight, objective mapping score, self-score weight and testers' self-score; and calculate the item uncertainty of the fusion score based on the adjusted objective evidence weight, objective uncertainty and the preset uncertainty of the testers' self-score.

[0062] As an optional implementation of this embodiment, the evaluation result generation module 206 is specifically used to obtain the original data corresponding to the fusion score; determine whether there are preset abnormal conditions based on the original data. If preset abnormal conditions exist, the corresponding evaluation item is invalidated and the weight is reset to zero; when the number of evaluation items marked as invalid in the same dimension is one, the interpolation algorithm is triggered; the interpolation value is calculated based on the scores of other valid items in the current dimension, and the uncertainty of the interpolation value is amplified by a preset factor; when the number of items marked as invalid is greater than one, the current dimension is marked as pending review and left blank for clinical supplementation; when the evaluation type is postoperative evaluation, the preoperative evaluation total score, postoperative evaluation total score, and preoperative expected improvement value of the same tester are retrieved; the score difference is calculated based on the preoperative evaluation combined with the postoperative and postoperative evaluation total scores; the expected satisfaction is generated based on the ratio of the score difference to the preoperative expected improvement value; and the processed evaluation result is generated based on the results of abnormality detection and missing value processing.

[0063] As an optional implementation of this embodiment, the evaluation report generation module 207 is specifically used to summarize the fusion scores and uncertainties of each data dimension to obtain the total fusion evaluation score and total uncertainty; and to assemble the scores, uncertainties, total fusion evaluation score, total uncertainty, and expected satisfaction of each data dimension into an auditable evaluation report according to a preset format.

[0064] In one example, the module in any of the above devices may be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0065] For example, when modules in a device can be implemented via a processing element scheduler, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor capable of calling programs. Alternatively, these modules can be integrated together as a system-on-a-chip (SOC).

[0066] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the above-described device and module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0067] Figure 3 This is a structural block diagram of the electronic device 300 provided in an embodiment of this application.

[0068] like Figure 3 As shown, the electronic device 300 includes a processor 301 and a memory 302, and may further include one or more of an information input / output (I / O) interface 303, a communication component 304, and a communication bus 305.

[0069] The processor 301 controls the overall operation of the electronic device 300 to complete all or part of the steps of the aforementioned multi-dimensional quantitative assessment method for patients with knee osteoarthritis. The memory 302 stores various types of data to support the operation of the electronic device 300. This data may include, for example, instructions for any application or method operating on the electronic device 300, as well as application-related data. The memory 302 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as one or more of Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0070] I / O interface 303 provides an interface between processor 301 and other interface modules, such as keyboards, mice, and buttons. These buttons can be virtual or physical. Communication component 304 is used for wired or wireless communication between electronic device 300 and other devices. Wireless communication includes Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G, or 4G, or a combination thereof. Therefore, the corresponding communication component 304 may include a Wi-Fi component, a Bluetooth component, and an NFC component.

[0071] The electronic device 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the multi-dimensional quantitative assessment method for patients with knee osteoarthritis given in the above embodiments.

[0072] The communication bus 305 may include a path for transmitting information between the aforementioned components. The communication bus 305 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 305 can be divided into an address bus, a data bus, a control bus, etc.

[0073] Electronic device 300 may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), and in-vehicle terminals (such as in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers, and may also be servers.

[0074] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the above-described method for comprehensive multimodal quantitative assessment of patients with knee osteoarthritis.

[0075] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0076] The terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0077] The above description is merely a preferred embodiment of this application and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this application is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the foregoing application concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features with similar functions claimed in this application.

Claims

1. A comprehensive, multimodal quantitative assessment method for patients with knee osteoarthritis, characterized in that, include: An evaluation environment is deployed and the multimodal sensors are time-synchronized to obtain a synchronized multimodal sensor, wherein the multimodal sensor includes a motion sensing module, a physiological signal acquisition module, and an augmented reality interaction module; Based on the aforementioned synchronous multimodal sensor and augmented reality, multidimensional data is collected when testers perform preset evaluation tasks. The multidimensional data includes kinematic data, electrophysiological data, eye-tracking data, acoustic data, facial expression data, and plantar pressure data. The quality of the multi-dimensional data is graded to determine the quality level of each data channel in the multi-dimensional data. Based on the quality level and the pain-related physiological signals in the multi-dimensional data, the weight of objective evidence and the weight of the tester's self-assessment are dynamically adjusted. Based on the adjusted objective evidence weights and the adjusted tester self-evaluation weights, the objective mapping score calculated based on the multi-dimensional data is fused with the tester self-evaluation score obtained through the augmented reality interaction module to generate a fusion score for each evaluation item. Anomaly detection and missing data processing are performed on the fusion score to generate a processing evaluation result; Summarize the processing and evaluation results to generate an auditable evaluation report.

2. The method according to claim 1, characterized in that, The preset assessment exercise includes symptom assessment tasks, satisfaction assessment tasks, and functional activity assessment tasks; the multi-dimensional data collected by the tester when performing the preset assessment tasks based on the synchronous multimodal sensor and augmented reality includes: For the symptom assessment task, the following data were collected: knee joint movement trajectory, skin conductance peak, pain avoidance index, pupillary pain response, confidence level of pain keyword recognition, and probability of micro-expression action units when the test subjects performed virtual walkway walking and virtual step climbing. For the aforementioned satisfaction assessment task, the non-specific skin conductance response frequency, skin conductance level drift, fixation microtremor power, proportion of aversion expression frames, motion-induced pain, latency of first viewing of the armrest, speech syllable rate, saccade entropy, and knee angular velocity decrease rate were collected from the test subjects when performing sitting, lying down, getting out of bed, simulating housework, and leisure activities. For the aforementioned functional activity assessment task, gait parameters, knee flexion angle, turning angle, step-up and step-down angles, time taken to stand up, getting on and off a vehicle, lateral movement distance, step-up and step-down angles, squatting and kneeling angles, running cadence, and knee trajectory variation coefficients were collected from the test subjects when performing walking and standing, standard activities, advanced activities, and free activities.

3. The method according to claim 1, characterized in that, The data channels include motion sensor channels, electrophysiological channels, eye-tracking channels, acoustic channels, and facial expression channels; the quality grading of the multi-dimensional data, determining the quality level of each data channel in the multi-dimensional data, includes: The data frame loss rate of the motion sensing channel, the baseline drift of the electrophysiological channel, the pupil loss rate of the eye-tracking channel, the keyword recognition confidence of the acoustic channel, and the facial occlusion rate of the facial expression channel are calculated respectively. The calculation results of each data channel are compared with multiple preset quality level threshold ranges to determine the quality level of each data channel; When the quality level of any of the data channels is the lowest level, the uncertainty of the data channel is multiplied by a preset penalty coefficient to obtain the processing uncertainty; The processing uncertainties of each data channel are weighted and squared to generate objective uncertainties.

4. The method according to claim 3, characterized in that, The dynamic adjustment of the weights of objective evidence and the weights of the test subject's self-assessment based on the quality level and the pain-related physiological signals in the multi-dimensional data includes: Acquire the skin electrophysiological signals and skin conductivity peak values ​​monitored by the electrophysiological channels; When the quality level of the electrophysiological channel is optimal and the peak value of the skin conductance response exceeds a preset threshold, a weighting adjustment mechanism is triggered. The weight of the skin electrophysiological signal is increased to a first preset ratio within a preset time period, and the remaining weight is allocated to the objective evidence weight and the self-evaluation weight of the tester according to the original ratio and then normalized. When the quality level of any of the data channels is the lowest level, a quality degradation process is triggered. The weight of the data channel is reduced by a second preset ratio, and the weights of the other data channels are proportionally amplified and then normalized again. When the deviation between the objective mapping score corresponding to the objective evidence weight and the tester's self-score exceeds a preset score threshold, multimodal conflict handling is triggered. Increase the weight of the objective evidence and decrease the weight of the self-assessment.

5. The method according to claim 4, characterized in that, The objective mapping score calculated based on the multi-dimensional data is fused with the tester's self-assessment score obtained through the augmented reality interaction module, according to the adjusted objective evidence weight and the adjusted tester self-assessment weight, to generate a fusion score for each evaluation item, including: Obtain the item type of the evaluation item and the key parameters in the multi-dimensional data; Based on the entry type, select the corresponding formula from the preset mapping formulas as the formula to be used; Calculate the objective mapping score based on the formula to be used and the key parameters; A visual scoring interface is displayed to the testers using augmented reality devices, and the testers' selection data is collected as their self-scoring. The fusion score is calculated based on the adjusted objective evidence weights, objective mapping scores, self-assessment weights, and tester self-assessment scores. The item uncertainty of the fusion score is calculated based on the adjusted objective evidence weights, the objective uncertainty, and the preset uncertainty of the tester's self-assessment.

6. The method according to claim 5, characterized in that, The process of performing anomaly detection and missing value processing on the fusion score to generate a processing evaluation result includes: Obtain the raw data corresponding to the fusion score; Based on the raw data, determine whether there are any preset abnormal conditions; If a preset abnormal condition exists, the corresponding evaluation item will be invalidated and its weight will be reset to zero. When the number of evaluation entries marked as invalid in the same dimension is one, the interpolation algorithm is triggered; The interpolated value is calculated based on the scores of other valid entries in the current dimension, and the uncertainty of the interpolated value is amplified by a preset factor. When the number of entries marked as invalid is greater than one, the current dimension as a whole is marked as pending review and left blank for clinical supplementation. When the assessment type is postoperative assessment, retrieve the preoperative assessment total score, postoperative assessment total score, and expected improvement value of the same test subject; The score difference is calculated based on the combined preoperative and postoperative assessment scores; Based on the ratio of the score difference to the preoperative expected improvement value, the expected satisfaction level is generated. Based on the results of anomaly detection and missing data handling, a processing evaluation result is generated.

7. The method according to claim 6, characterized in that, The process of summarizing the evaluation results and generating an auditable evaluation report includes: The fusion scores and uncertainties of each of the data dimensions are summarized to obtain the total fusion evaluation score and total uncertainty. The scores, uncertainties, total fusion evaluation scores, total uncertainties, and expected satisfaction levels of each data dimension are assembled into an auditable evaluation report according to a preset format.

8. A comprehensive, multimodal quantitative assessment system for patients with knee osteoarthritis, characterized in that, include: An environment deployment synchronization module is used to deploy and evaluate the environment and synchronize the time of the multimodal sensors to obtain synchronized multimodal sensors, wherein the multimodal sensors include a motion sensing module, a physiological signal acquisition module, and an augmented reality interaction module; The dimensional data acquisition module is used to acquire multidimensional data when testers perform preset evaluation tasks based on the synchronous multimodal sensor and augmented reality. The multidimensional data includes kinematic data, electrophysiological data, eye-tracking data, acoustic data, facial expression data, and plantar pressure data. The quality level determination module is used to classify the quality of the multi-dimensional data and determine the quality level of each data channel in the multi-dimensional data. The weighted data adjustment module is used to dynamically adjust the weight of objective evidence and the weight of the tester's self-assessment based on the quality level and the pain-related physiological signals in the multi-dimensional data. The fusion score generation module is used to fuse the objective mapping score calculated based on the multi-dimensional data with the tester's self-evaluation score obtained through the augmented reality interaction module, based on the adjusted objective evidence weight and the adjusted tester's self-evaluation weight, to generate a fusion score for each evaluation item. The evaluation result generation module is used to perform anomaly detection and missing value processing on the fusion score and generate the processed evaluation result. The evaluation report generation module is used to summarize the evaluation results and generate an auditable evaluation report.

9. An electronic device, characterized in that, Includes a processor, which is coupled to a memory; The processor is configured to execute a computer program stored in the memory, causing the electronic device to perform the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Includes a computer program or instructions that, when run on a computer, cause the computer to perform the method as described in any one of claims 1 to 7.

Images