Big model driving-based body cognitive motion cooperative training method and device

Abstract

The invention discloses a method and a device for cooperative training of body cognitive movement based on large model driving, and relates to the technical field of rehabilitation training. Constructing a user portrait according to the information and a preset format template; the user portrait is input into a preset large language model, a rehabilitation training task matched with the user portrait is output firstly, then a subtask is generated according to the rehabilitation training task and output, and the subtask is a task logic chain used for completing the rehabilitation training task; inputting the rehabilitation training task and the subtask into a preset visual language model, outputting a task scene, and visualizing the task scene; receiving an operation instruction input by a user, and operating the material image in the current subtask according to the operation instruction; and judging whether the current subtask operation is correct or not according to the current subtask completion result and a preset standard. The collaborative rehabilitation efficiency is improved; personalized adaptation is enhanced; and the training scheme updating response time is shortened.

Description

Technical Field

[0001] The present invention relates to the field of rehabilitation training technology, and in particular to an embodied cognitive-motor collaborative training method and device driven by a large model. Background Art

[0002] Traditional rehabilitation training often relies on paper-and-pencil assessments, physical teaching aids, video demonstrations, or one-on-one guidance from professionals. Furthermore, traditional rehabilitation training separates motor rehabilitation for muscle and nerve function from cognitive impairment recovery training. These training methods suffer from a single interactive mode, lagging feedback mechanisms, a lack of personalized adjustments to address patient differences, and low engagement, making it difficult to fully stimulate patients' active participation. This is especially true in the early stages of rehabilitation, the golden period for rehabilitation, where proper rehabilitation guidance is crucial. Failure to fully mobilize patient motivation will result in missing this golden opportunity for intervention.

[0003] Therefore, a large model-driven embodied cognitive-motor collaborative training method and device were developed to solve the above problems. Summary of the Invention

[0004] The present invention proposes an embodied cognitive-motor collaborative training method and device driven by a large model to solve the problems of existing rehabilitation training methods such as single interaction mode, lagging feedback mechanism, low personalization and low patient participation.

[0005] The present invention achieves the above-mentioned purpose through the following technical solutions: The present invention provides a large-scale model-driven embodied intelligent movement collaborative rehabilitation training method, comprising: Acquiring information, including basic information of the user and cognitive impairment rehabilitation information; Constructing a user profile based on the information and a preset format template; Input the user profile into a preset large language model, first output a rehabilitation training task that matches the user profile, then generate and output subtasks based on the rehabilitation training task, where the subtasks are a task logic chain for completing the rehabilitation training task; Inputting the rehabilitation training task and subtasks into a preset visual language model, outputting a task scene, and visualizing the task scene, wherein the task scene includes a background image matching the rehabilitation training task and a material image matching the subtask; Receive an operation instruction input by a user, and operate the material image in the current subtask according to the operation instruction; Determine whether the current subtask operation is correct based on the current subtask completion result and the preset standards. If correct, proceed to the next subtask. If incorrect, issue a prompt to re-execute the task. Repeat this step until the completion results of all subtasks are correct.

[0006] Furthermore, a task operation reminder matching the rehabilitation training task and subtask is issued simultaneously with or after outputting the task scenario, and the task reminder includes one or more of a text reminder, a voice reminder or an action reminder.

[0007] Furthermore, the material image includes matching material and interference material. When it is detected that the user operates on the interference material and / or a preset area near the interference material, an alarm prompt is issued. The alarm prompt includes the name of the interference material and / or one or more of a voice prompt, picture prompt or text prompt to remind the user of a task error.

[0008] Furthermore, it also includes setting task parameters, and the setting task parameters includes: Obtain the user's historical rehabilitation training data; Inputting the historical rehabilitation training movement data into the preset large language model, and outputting task parameters of the subtask, wherein the task parameters include rehabilitation mode, rehabilitation intensity, speech speed of voice reminder, task difficulty level, number of distracting materials, and number of task reminders; A setting instruction is issued to the rehabilitation robot according to the task parameters.

[0009] Furthermore, it also includes a rehabilitation mode and intensity feedback adjustment. The rehabilitation mode includes applying assistance or resistance to the user, and the intensity is the size of the applied force.

[0010] Furthermore, task feedback adjustment is also included, and the task feedback adjustment includes: Real-time acquisition of rehabilitation training data during the user's operation of the current subtask; The completion result of the current subtask and the real-time rehabilitation training movement data are input into the preset large language model, and the adjusted task parameters are output.

[0011] Furthermore, obtaining the user's rehabilitation training exercise data includes obtaining one or more of the following: Obtain the user's hand flexion and extension finger range of motion, flexion and extension speed, flexion and extension strength, and flexion and extension stability; Obtain the range of motion, speed, force, and stability of the user's upper limbs; Obtain the user's lower limb motion range, lower limb motion speed, motion force, and motion stability.

[0012] Furthermore, the operation instruction includes one of an instruction to select a material, an instruction to move a material, an instruction to shake a material, and an instruction to place a material on a visual interface.

[0013] Furthermore, an update mode is included, and the update mode includes: After completing a rehabilitation training task, obtaining the completion result of the final rehabilitation training task and the rehabilitation training movement data during the completion of the rehabilitation training task; Update the user's current cognitive impairment rehabilitation information with the completion result of the final rehabilitation training task; The current cognitive impairment rehabilitation information and rehabilitation training movement data during the completion of the rehabilitation training task are input into the preset large language model, and the updated task parameters and user portrait are output and saved. The updated task parameters and user portrait are used for the user's next training.

[0014] The present invention also provides an embodied intelligent movement collaborative rehabilitation training device driven by a large model, comprising: An acquisition module, the acquisition module is used to acquire information, the information including basic information of the user and cognitive impairment rehabilitation information; A construction module, the construction module is used to construct a user portrait based on the information and a preset format template; A task module, which is used to input the user profile into a preset large language model, first output a rehabilitation training task that matches the user profile, and then generate and output subtasks based on the rehabilitation training task. The subtasks are a task logic chain used to complete the rehabilitation training task; A material module, which is used to input the rehabilitation training task and subtasks into a preset visual language model, output a task scene, and visualize the task scene, wherein the task scene includes a background image matching the rehabilitation training task and a material image matching the subtask; An operation module, the operation module is used to receive an operation instruction input by a user and operate the material image in the current subtask according to the operation instruction; The judgment module is used to judge whether the current subtask operation is correct based on the current subtask completion result and preset standards. If it is correct, it will enter the next subtask. If it is not correct, it will issue a prompt to re-execute the task and repeat this step until the completion results of all subtasks are correct.

[0015] The beneficial effects of the present invention are: The embodied cognitive-motor collaborative training method and device based on large model driving proposed in the present invention improves the efficiency of collaborative rehabilitation and shortens the time to complete cognitive tasks; personalized adaptation is enhanced, and scene generation is highly matched with users; and the response time for training program updates is shortened.

[0016] The present invention utilizes the knowledge reasoning and task planning capabilities of a large model to generate multi-stage, multi-difficulty embodied tasks based on the patient's abilities, training history, and scenario goals. It conducts interactive guidance through a graphical interface and natural language, and adjusts the task content in real time based on the patient's operational behavior and semantic expression. The present invention focuses on dynamic training task generation, individual behavior feedback analysis, and adaptive task iteration for patients with cognitive impairment, and constructs a patient-centered intelligent rehabilitation closed loop. The present invention is based on an embodied training model based on semantic understanding and task evolution, which effectively breaks through the static content and linear process limitations of traditional rehabilitation systems, and realizes a paradigm shift in cognitive rehabilitation from "content display" to "interactive generation." BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a method flow chart of the embodied cognitive-motor collaborative training method and device driven by a large model in this application. DETAILED DESCRIPTION

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the embodiments of the present invention. It should be understood that the described embodiments are only a portion of the embodiments of the present invention, not all of them. Generally, the components of the embodiments of the present invention described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

[0019] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0020] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0021] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0022] like Figure 1 As shown, a method and device for embodied cognitive-motor collaborative training based on a large model drive includes: S1: Acquire information, including basic information of the user and cognitive impairment rehabilitation information; The basic information includes age, hobbies, occupation, daily living habits and other information.

[0023] Cognitive impairment rehabilitation information includes the rehabilitation site, rehabilitation degree, and duration of rehabilitation.

[0024] Information is primarily acquired through AI agents. For example, a large language model can be used to conduct opening interactions based on multiple rounds of dialogue. Users are asked multiple-choice questions related to their occupation, daily activities, and habits, guiding them through interactive choices. Example questions include, but are not limited to, "Are you still working?" and "Which of the following best describes your professional background?" The options cover common occupations and lifestyles, such as retired teachers, corporate employees, and housewives, ensuring that the system can easily perform structured analysis of user profiles. S2: Build a user profile based on the information and a preset format template; Output formatted user profile text in a unified manner, with a clear user profile structure for subsequent model invocation and task generation logic. This ensures a clear profile structure and facilitates unified parsing across large models. This template serves as the core context for prompts in the generative model and is embedded in the subsequent rehabilitation task generation process, achieving semantic information binding between user profile and task generation.

[0025] S3: Inputting the user profile into a preset large language model, first outputting a rehabilitation training task matching the user profile, then generating and outputting subtasks based on the rehabilitation training task, wherein the subtasks are a task logic chain for completing the rehabilitation training task; For example, a cooking task might have a kitchen setting, while a painting task might have a studio setting. Subtasks for a cooking task include selecting ingredients, washing vegetables, cutting vegetables, cooking, and plating. Subtasks can involve selecting the right answer, such as choosing the right ingredients for scrambled eggs with tomatoes. Subtasks can also involve performing a rehabilitation action after selecting the right answer. For example, in an upper limb rehabilitation robot scenario, selecting a knife and performing a back-and-forth motion to complete the task can be done. Subtasks can be expressed in text and voice.

[0026] S4: Inputting the rehabilitation training task and subtasks into a preset visual language model, outputting a task scene, and visualizing the task scene, wherein the task scene includes a background image matching the rehabilitation training task and a material image matching the subtask; S5: receiving an operation instruction input by the user, and operating the material image in the current subtask according to the operation instruction; the user can operate the material in the subtask by operating the rehabilitation robot.

[0027] S6: Determine whether the current subtask operation is correct based on the current subtask completion result and the preset standard. If correct, proceed to the next subtask. If incorrect, issue a prompt to re-execute the task. Repeat this step until the completion results of all subtasks are correct.

[0028] To ensure the medical rationality and technical consistency of generated tasks, the system introduces a verification model to conduct multiple rounds of review of generated content. The verification model is divided into the following three categories: ① Semantic consistency detection: Verifying whether the task text and image semantically correspond to each other, and whether there are any "image-text mismatches" or interaction errors; ② Interaction logic verification: Verifying whether the drag path and option logic are feasible to ensure smooth interaction and accurate feedback; ③ Rehabilitation rationality review: Embedded rehabilitation standard knowledge graph determines whether the current generated task meets medical adaptation requirements such as user age, disease stage, and ability level. If the verification fails, the system will automatically roll back and regenerate the material until the generation standard is met.

[0029] The large language model can adopt the large language model GPT-4.5, and the visual language model can adopt the BLIP model.

[0030] In one embodiment, a task operation reminder matching the rehabilitation training task and subtask is issued simultaneously with or after outputting the task scenario, and the task reminder includes one or more of a text reminder, a voice reminder, or an action reminder.

[0031] For example, if the user chooses "wash vegetables", the system will generate the next task instructions in the kitchen sink scene, realizing a smooth and natural interactive closed loop. In addition, different semantic roles can be dynamically switched to build an immersive rehabilitation scene. For example, when performing life-related cognitive training tasks, the large language model can be used to play roles such as "daughter", "neighbor" or "shopping guide", and guide the user into the task state through language, situational interpretation and other methods. Each role behavior is based on the results generated by the large model, and can automatically adjust the tone, questioning method and interaction rhythm according to the user portrait and the current task logic.

[0032] In one embodiment, the material image includes matching material and interference material. When it is detected that the user operates on the interference material and / or a preset area near the interference material, an alarm prompt is issued. The alarm prompt includes the name of the interference material and / or one or more of a voice prompt, a picture prompt or a text prompt to remind the user of a task error.

[0033] For example, each subtask may contain 4-5 materials, including correct answer materials and incorrect answer materials. The correct answer materials serve as matching materials, and the incorrect answer materials serve as interference materials. For example, in the rehabilitation training subtask of cooking, some sports-related materials can be generated, such as basketball, baseball, etc., to help users conduct cognitive impairment rehabilitation training; each material has pictures, text descriptions, and voice matching the text descriptions, which is convenient for explaining the wrong answer after the user answers incorrectly.

[0034] For example, for the task "washing tomatoes," the system automatically generates an image scene containing elements such as "tomatoes," "sink," and "hand tools," and annotates the interactive areas in the image. Image generation adheres to the three principles of semantic consistency, natural scene, and interactive operability.

[0035] For example, based on real-time detection results (such as incorrect dragging paths or incorrect option selections), personalized guidance statements or prompt animations are generated. This prompt strategy follows a cognitive behavioral intervention mechanism of "guidance first, correction second, and encouragement third" to improve training engagement and accuracy. For example, "Try putting tomatoes in the sink" or "You selected a water cup, but water cups are not suitable for holding vegetables."

[0036] In one embodiment, the task parameters are further set, and the task parameters are set including: Obtain the user's historical rehabilitation training data; Inputting the historical rehabilitation training movement data into the preset large language model, and outputting task parameters of the subtask, wherein the task parameters include rehabilitation mode, rehabilitation intensity, speech speed of voice reminder, task difficulty level, number of distracting materials, and number of task reminders; A setting instruction is issued to the rehabilitation robot according to the task parameters.

[0037] This ensures that users can complete rehabilitation tasks within an acceptable range of challenge, improving training effectiveness.

[0038] In one embodiment, a rehabilitation mode and intensity feedback adjustment are further included, wherein the rehabilitation mode includes applying assistance or resistance to the user, and the intensity is the magnitude of the applied force.

[0039] In one embodiment, task feedback adjustment is further included, and the task feedback adjustment includes: Real-time acquisition of rehabilitation training data during the user's operation of the current subtask; The completion result of the current subtask and the real-time rehabilitation training movement data are input into the preset large language model, and the adjusted task parameters are output.

[0040] In one embodiment, obtaining the user's rehabilitation training motion data includes obtaining one or more of the following: Obtain the user's hand flexion and extension finger range of motion, flexion and extension speed, flexion and extension strength, and flexion and extension stability; Obtain the range of motion, speed, force, and stability of the user's upper limbs; Obtain the user's lower limb motion range, lower limb motion speed, motion force, and motion stability.

[0041] In one embodiment, the operation instruction includes one of an instruction to select a material, an instruction to move a material, an instruction to shake a material, and an instruction to place a material on a visual interface.

[0042] Human-computer interaction is achieved through operational instructions. For example, in a drag interaction, the user is required to drag a target object (e.g., a tomato) to a designated location (e.g., a sink) using a mouse or touch. Coordinate matching and path tracking are used to determine whether the interaction meets the requirements. In a single-choice interaction, the system presents multiple image or text options, and the user clicks to select the one that best meets the task objective. For example, in the "Select a container for food" task, options such as "cup," "plate," and "pan" are provided to guide the user in making the correct selection.

[0043] In one embodiment, an update mode is further included, and the update mode includes: After completing a rehabilitation training task, obtaining the completion result of the final rehabilitation training task and the rehabilitation training movement data during the completion of the rehabilitation training task; Update the user's current cognitive impairment rehabilitation information with the completion result of the final rehabilitation training task; The current cognitive impairment rehabilitation information and rehabilitation training movement data during the completion of the rehabilitation training task are input into the preset large language model, and the updated task parameters and user portrait are output and saved. The updated task parameters and user portrait are used for the user's next training.

[0044] By setting up dynamic task evaluation mechanisms (such as RL-based reward functions and task evaluation based on large model confidence), we determine whether the current task exceeds or falls below the user's cognitive load. If a task difficulty mismatch is detected, the system will use the model to automatically adjust the complexity of subsequent subtasks. It will also correct the user's behavioral path through "step-by-step prompts" (such as step-by-step guidance, error correction guidance, and key clue reinforcement), helping them evolve towards the correct rehabilitation movements or strategies, thereby completing a complete training process that meets rehabilitation standards.

[0045] In one embodiment, a large model-driven embodied intelligent motion collaborative rehabilitation training device is provided, comprising: An acquisition module, the acquisition module is used to acquire information, the information including basic information of the user and cognitive impairment rehabilitation information; A construction module, the construction module is used to construct a user portrait based on the information and a preset format template; A task module, which is used to input the user profile into a preset large language model, first output a rehabilitation training task that matches the user profile, and then generate and output subtasks based on the rehabilitation training task. The subtasks are a task logic chain used to complete the rehabilitation training task; a material module, which is used to input the rehabilitation training task and subtasks into a preset visual language model, output a task scene, and visualize the task scene, wherein the task scene includes a background image matching the rehabilitation training task and a material image matching the subtask; An operation module, the operation module is used to receive an operation instruction input by a user and operate the material image in the current subtask according to the operation instruction; The judgment module is used to judge whether the current subtask operation is correct based on the current subtask completion result and preset standards. If it is correct, it will enter the next subtask. If it is not correct, it will issue a prompt to re-execute the task and repeat this step until the completion results of all subtasks are correct.

[0046] The present invention also provides a rehabilitation system, which includes three subsystems: Subsystem 1: Rehabilitation robot system.

[0047] The rehabilitation robot is the operating end in this system. Users perform sports rehabilitation training through the rehabilitation robot. The rehabilitation training robot can provide power-assisted training. This mode is mainly used in the early stage of patient rehabilitation. It can also provide resistance for users to perform strength training. This mode is mainly used in the middle stage of patient rehabilitation.

[0048] The rehabilitation robot in this patent can be a hand rehabilitation robot, an upper limb rehabilitation robot or a lower limb rehabilitation robot.

[0049] The rehabilitation robot is equipped with motion sensors that can obtain the user's corresponding motion information, such as: The hand rehabilitation robot performs rehabilitation training through rehabilitation gloves, which can obtain motion information of the user's hand in dimensions such as the range of motion of the finger flexion and extension, flexion and extension speed, flexion and extension force (scalar), and flexion and extension stability (jitter during flexion and extension). The upper limb rehabilitation robot performs training by moving a two-dimensional handle. It can obtain motion information in dimensions such as the range of motion of the user's upper limbs, upper limb movement speed, movement force (scalar), and movement stability (shake during the movement of the handle). The lower limb rehabilitation robot is trained by moving a specific mechanical device tied to the knee joint. It can obtain motion information in dimensions such as the user's lower limb range of motion, lower limb movement speed, movement force (scalar), and movement stability (shake during lower limb movement). The activities of the rehabilitation robot can be fed back to subsystem 2, and operations such as selection and cursor movement can be performed through the rehabilitation robot.

[0050] Subsystem 2: Host computer program-display interface.

[0051] The host computer program runs independently on mobile devices such as tablets and mobile phones, and is used to interact with subsystems 1 and 3, and generate multimodal interactive information such as pictures, voice and text for cognitive impairment rehabilitation training. At the same time, the host computer can also collect users' personal information. The user interacts with Subsystem 1 via Bluetooth. Subsystem 1 allows users to select answers, select targets, and move objects. The rehabilitation robot can be thought of as a "mouse" that requires force to operate. Meanwhile, Subsystem 2 acquires motion data from Subsystem 1.

[0052] It interacts with subsystem 3 through the network, uploads basic user information and motion information, and receives multimodal information returned by subsystem 3. Detailed introduction is given in subsystem 3.

[0053] Subsystem 3: Embodied Intelligence Model.

[0054] The subsystem 3 is embedded with the embodied cognitive-motor collaborative training method and device based on large model driving according to the present invention.

[0055] The workflow of the rehabilitation system is as follows: Step 1. User Profile Construction. Through Subsystem 1, the user selects basic information matching the user in Subsystem 2, such as age, hobbies, and duration of rehabilitation. During this process, Subsystem 2 acquires and stores the user's exercise data. Once the user's basic information is entered, it is uploaded along with the exercise data to Subsystem 3 for processing in Step 2.

[0056] Step 2. Embodied intelligence multimodal task generation. Subsystem 3 performs inference based on the input from step 1, obtains multimodal rehabilitation training tasks and materials, and displays them on subsystem 2: Background image of the rehabilitation training task scene; The current subtask, that is, what kind of operation the user needs to complete; For materials related to the current subtask, just display the pictures first; At the same time, subsystem 2 sets the exercise intensity and mode suitable for the current user inferred by subsystem 3 to subsystem 1, and the user can enter step 3 to operate.

[0057] Step 3. User Execution Training. Based on the task described by Subsystem 2, the user selects an answer or moves through Subsystem 1, collecting motion data during the process. If the user makes an error during the selection process, text and audio of the corresponding material are displayed to inform the user of the error and help with cognitive recovery. A timeout is also set for the subtask, starting at the point when the subtask is generated and completed. When the timeout expires or the user successfully completes the task, the motion data and completion status are uploaded to Subsystem 3, and the process proceeds to Step 4.

[0058] Step 4. Model Optimization and Next Scenario Generation. After Step 3 is completed, Subsystem 2 uploads the motion data to Subsystem 3. If all five subtasks have been completed, the process proceeds to Step 5. Otherwise, Subsystem 3 determines the difficulty of the current subtask based on the real-time rehabilitation training motion data and results. It then infers the appropriate materials for the user's next scenario and the rehabilitation model and strength of Subsystem 1. If the data returned indicates that the current task is too easy, the difficulty is increased; if it is too difficult, the difficulty is reduced. After the scenario and tasks are generated, the process continues with Step 3.

[0059] Step 5: Model Enhancement. After a task is completed, Subsystem 3 infers the user's training results and motion data again, updates the user profile, and saves it. The next time the user trains, it generates new scenarios and tasks with a difficulty level more appropriate to the user's current stage of recovery.

[0060] The present invention is further described below by taking the patient's upper limb rehabilitation as an example: The user inputs their preferences (cooking preference) through the rehabilitation robot. This represents the mid-stage of upper limb rehabilitation. The rehabilitation robot captures the motion data from the input phase and uploads this basic information to the model to generate a "kitchen" scene.

[0061] Model reasoning can generate the task of "cooking" and requires adding a little impedance to the user, so a kitchen scene is generated, and the rehabilitation robot is controlled to generate 5N of resistance during the "cooking" process.

[0062] Material generation: Given an ingredient and a sink, the rest of the materials are irrelevant materials such as books. The ingredient is located in the upper left corner of the interface, the sink is located in the lower right corner of the interface, and the rest of the materials are randomly located.

[0063] Difficulty confirmation: The model sends back data, sets the upper limb rehabilitation robot mode to impedance mode, and the force to 5N.

[0064] Multimodal task generation: Generates voice and text to remind users that they need to clean the food. First, they should operate the rehabilitation robot to select the food, then move it to the sink, wait for the food to be immersed in water, and then shake the rehabilitation robot to complete the food cleaning.

[0065] The user operates the rehabilitation robot to complete the task. During the task, there is obvious shaking in the motion trajectory and the completion time is long. The rehabilitation robot uploads the motion data and motion results to the model.

[0066] After receiving the motion data from the previous task, the model analyzes that the user's cognitive ability is strong, the ingredients are correctly selected, and the movement trend is towards the sink. However, the current difficulty is too high, so the force is adjusted to 2N and the next task is generated: chopping the ingredients. Simultaneously, it generates interference elements similar to the ingredients, such as a knife for cutting vegetables, and a ruler. Once generated, the user is prompted to start the task.

[0067] Through multiple task trainings and data feedback, the model analyzed that the user's motor ability is medium, the impedance force is set at 3.5N, and the training time is more appropriate at 15 minutes. The cognitive level is relatively high. The next training will use this parameter as a benchmark to set the scene and tasks.

[0068] Examples of prompt words for a large language model are as follows: You are a language model agent tasked with creating rehabilitation tasks for Alzheimer's patients. The tasks should be safe, simple, and helpful for memory recall. Interactions will primarily be conducted through a graphical interface with hardware limitations, with both text and speech support required.

[0069] Here are the steps: 1. Initial task generation: Develop a series of tasks related to uploaded user preferences or commonly used rehabilitation activities such as sports, art, cooking, etc.

[0070] Five subtasks were designed to match the user's current motion data, with the goal of completing this rehabilitation training task line while ensuring ease of interaction.

[0071] Generate rehabilitation exercise intensity and mode (resistance or assistance, how many Newtons of force) that matches the user's motion data.

[0072] 2. Generate and adjust each subtask: After the user completes the subtask, get user feedback containing pictures.

[0073] Verify that the images are placed correctly and are relevant (for example, a knife moving onto a dish represents cutting, a dish moving onto a sink represents washing, a dish moving onto a pot represents cooking, etc.).

[0074] Adjust the difficulty of subsequent tasks based on feedback.

[0075] The generated materials need to have interference items. For example, during the cutting task, materials unrelated to the cutting task can appear to increase the difficulty.

[0076] 3. User interaction and framework updates: Only the current subtask is provided after each step is completed.

[0077] Reflect the user's progress through task updates.

[0078] 4. Feedback processing Feedback images were analyzed to assess task completion and image accuracy.

[0079] Adjust the difficulty of future tasks based on the returned motion data and provide descriptive information.

[0080] Adjust the intensity and pattern of the rehabilitation exercise for the next task, specify whether it is resistance or assistance, and the specific force in Newtons.

[0081] The present invention proposes an embodied cognitive-motor collaborative training method and device driven by a large model. The present invention utilizes the knowledge reasoning and task planning capabilities of the large model to generate multi-stage, multi-difficulty embodied tasks according to the patient's ability, training history and scenario goals, and interactively guides through a graphical interface and natural language, and adjusts the task content in real time based on the patient's operational behavior and semantic expression. The present invention focuses on the dynamic training task generation, individual behavior feedback analysis and adaptive task iteration of patients with cognitive impairment, and constructs a patient-centered intelligent rehabilitation closed loop. The present invention is based on an embodied training model of semantic understanding and task evolution, which effectively breaks through the static content and linear process limitations of traditional rehabilitation systems, and realizes the paradigm shift of cognitive rehabilitation from "content display" to "interactive generation".

[0082] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A large-scale model-driven embodied cognitive-motor collaborative training method, characterized in that: include: Acquiring information, including basic information of the user and cognitive impairment rehabilitation information; Constructing a user profile based on the information and a preset format template; Input the user profile into a preset large language model, first output a rehabilitation training task that matches the user profile, then generate and output subtasks based on the rehabilitation training task, where the subtasks are a task logic chain for completing the rehabilitation training task; Inputting the rehabilitation training task and subtasks into a preset visual language model, outputting a task scene, and visualizing the task scene, wherein the task scene includes a background image matching the rehabilitation training task and a material image matching the subtask; Receive an operation instruction input by a user, and operate the material image in the current subtask according to the operation instruction; Determine whether the current subtask operation is correct based on the current subtask completion result and the preset standards. If correct, proceed to the next subtask. If incorrect, issue a prompt to re-execute the task. Repeat this step until the completion results of all subtasks are correct.

2. The method of embodied cognitive-motor collaborative training based on large model driving according to claim 1, characterized in that: At the same time as or after outputting the task scenario, a task operation reminder matching the rehabilitation training task and subtask is issued, wherein the task reminder includes one or more of a text reminder, a voice reminder or an action reminder.

3. A large model-driven embodied cognitive-motor collaborative training method according to claim 1 or 2, characterized in that: The material image includes matching material and interference material. When it is detected that the user operates on the interference material and / or a preset area near the interference material, an alarm prompt is issued. The alarm prompt includes the name of the interference material and / or one or more of a voice prompt, a picture prompt or a text prompt to remind the user of a task error.

4. The method of embodied cognitive-motor collaborative training based on large model drive according to claim 2, characterized in that: The task parameters are set up, including: Obtain the user's historical rehabilitation training data; Inputting the historical rehabilitation training movement data into the preset large language model, and outputting task parameters of the subtask, wherein the task parameters include rehabilitation mode, rehabilitation intensity, speech speed of voice reminder, task difficulty level, number of distracting materials, and number of task reminders; A setting instruction is issued to the rehabilitation robot according to the task parameters.

5. The method of embodied cognitive-motor collaborative training based on large model drive according to claim 4, characterized in that: It also includes a rehabilitation mode and intensity feedback adjustment. The rehabilitation mode includes applying assistance or resistance to the user, and the intensity is the size of the applied force.

6. The method of embodied cognitive-motor collaborative training based on large model driving according to claim 4, characterized in that: It also includes task feedback adjustment, which includes: Real-time acquisition of rehabilitation training data during the user's operation of the current subtask; The completion result of the current subtask and the real-time rehabilitation training movement data are input into the preset large language model, and the adjusted task parameters are output.

7. The method of embodied cognitive-motor collaborative training based on large model drive according to claim 6, characterized in that: Obtaining the user's rehabilitation training data includes obtaining one or more of the following: Obtain the user's hand flexion and extension finger range of motion, flexion and extension speed, flexion and extension strength, and flexion and extension stability; Obtain the range of motion, speed, force, and stability of the user's upper limbs; Obtain the user's lower limb motion range, lower limb motion speed, motion force, and motion stability.

8. The method of embodied cognitive-motor collaborative training based on large model driving according to claim 1, characterized in that: The operation instruction includes one of an instruction to select a material on a visual interface, an instruction to move a material, an instruction to shake a material, and an instruction to place a material.

9. The method of embodied cognitive-motor collaborative training based on large model driving according to claim 1, characterized in that: Also included is an update mode, which includes: After completing a rehabilitation training task, obtaining the completion result of the final rehabilitation training task and the rehabilitation training movement data during the completion of the rehabilitation training task; Update the user's current cognitive impairment rehabilitation information with the completion result of the final rehabilitation training task; The current cognitive impairment rehabilitation information and rehabilitation training movement data during the completion of the rehabilitation training task are input into the preset large language model, and the updated task parameters and user portrait are output and saved. The updated task parameters and user portrait are used for the user's next training.

10. A large model-driven embodied cognitive-motor collaborative training device, characterized in that: include: An acquisition module, the acquisition module is used to acquire information, the information including basic information of the user and cognitive impairment rehabilitation information; A construction module, the construction module is used to construct a user portrait based on the information and a preset format template; A task module, which is used to input the user profile into a preset large language model, first output a rehabilitation training task that matches the user profile, and then generate and output subtasks based on the rehabilitation training task. The subtasks are a task logic chain used to complete the rehabilitation training task; a material module, which is used to input the rehabilitation training task and subtasks into a preset visual language model, output a task scene, and visualize the task scene, wherein the task scene includes a background image matching the rehabilitation training task and a material image matching the subtask; An operation module, the operation module is used to receive an operation instruction input by a user and operate the material image in the current subtask according to the operation instruction; The judgment module is used to judge whether the current subtask operation is correct based on the current subtask completion result and preset standards. If it is correct, it will enter the next subtask. If it is not correct, it will issue a prompt to re-execute the task and repeat this step until the completion results of all subtasks are correct.

Images