The invention relates to the technical field of
artificial intelligence and human-computer interaction, and discloses a
rehabilitation training adjustment method and
system based on multi-
modal feature fusion and a storage medium, and the method comprises the steps: obtaining voice, face and physiological signals, constructing a multi-
modal data stream, extracting features, and outputting a speech error classification identifier and a physiological
signal quality index; splitting the facial action net displacement into healthy and affected sides, correcting affected side features based on healthy side features, and constructing visual representation; fusing each feature to output a comprehensive
state vector; updating a dynamic baseline in a task gap, mapping a
state vector, and screening to obtain an
effective action set; calculating a composite reward and storing the composite reward in an experience playback
pool to update the strategy network; and based on the action set, re-weighting the
large model output probability, and generating a target interaction corpus. According to the method, the state sensing precision is improved by correcting the deviation of the affected side through the uninjured side, and a
closed loop of self-adaptive adjustment of the
rehabilitation difficulty and safe interaction
text generation is realized by combining the dynamic baseline and penalty mechanism optimization
reinforcement learning.