Generation method and device of cognitive rehabilitation training content and cognitive rehabilitation training device

Abstract

The invention discloses a cognitive rehabilitation training content generation method and device and a cognitive rehabilitation training device. The cognitive rehabilitation training content generation method comprises the following steps: collecting content materials related to a patient and relatives and friends of the patient; according to the content materials, generating story scripts corresponding to each story scene of the story by utilizing a large language model, and generating image generation cues and story description texts corresponding to each story scene according to the story scripts; generating video clips and voice audios corresponding to each story scene according to the story script, the image generation prompt word, the story description text and the relative and friend voice audios of the patient; and generating a cognitive rehabilitation training video according to the video clip and the voice audio.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method and device for generating cognitive rehabilitation training content, and a cognitive rehabilitation training device. Background Art

[0002] Alzheimer's disease is a challenging neurodegenerative disease that severely impairs cognitive function. Its neuropathological mechanisms lead to functional degeneration in key brain regions, such as the hippocampus and frontal lobe, leading to memory loss, attention deficit, and language and thinking impairments. These impairments significantly reduce patients' ability to care for themselves and place a heavy burden on their families. With the global prevalence of Alzheimer's disease increasing, effective responses are urgently needed.

[0003] Currently, traditional cognitive rehabilitation training often uses generic templates for content design, failing to consider patients' life experiences, interests, and cognitive levels. This leads to a disconnect between training content and patient realities, resulting in poor engagement and effectiveness. Methodologically, traditional cognitive rehabilitation training focuses on single cognitive skill training, with little consideration for patients' emotional and psychological changes. This lack of effective emotional support and interaction fails to motivate patients, limiting the potential for rehabilitation and the fulfillment of their needs.

[0004] With the development of artificial intelligence technology, personalized cognitive training programs are gradually becoming possible. Generative AI can generate personalized images and story content from materials provided by patients' relatives and friends. In terms of audio generation, it can simulate the voices of relatives and friends, enhancing the appeal and emotional resonance of training. In addition, memory models and cognitive psychology principles play an important guiding role in cognitive rehabilitation training. Memory models emphasize activating patients' long-term memory storage and retrieval mechanisms through specific stimulation and training strategies, while cognitive psychology focuses on understanding patients' cognitive processing, including characteristics of attention, perception, and thinking, so that more targeted training tasks can be designed.

[0005] In summary, existing cognitive training methods have limitations in activating long-term memory neural circuits and simultaneously improving cognitive function and meeting emotional needs. Therefore, developing an effective, personalized, and emotionally supportive cognitive rehabilitation program is urgently needed and has significant significance in overcoming the challenges of cognitive rehabilitation in Alzheimer's disease. Summary of the Invention

[0006] The embodiments of the present disclosure provide a method and device for generating cognitive rehabilitation training content, and a cognitive rehabilitation training device, so as to provide an efficient, personalized, and emotionally supportive cognitive rehabilitation program.

[0007] According to one aspect of an embodiment of the present disclosure, a method for generating cognitive rehabilitation training content is provided, comprising: collecting content materials related to a patient and the patient's relatives and friends, the content materials comprising a story narrative text, a first character description text, and photos related to the patient and the patient's relatives and friends, and collecting voice audio of the patient's relatives and friends, wherein the story narrative text is used to describe a story experienced by the patient and the patient's relatives and friends, and the first character description text is used to describe the personality of the patient and the patient's relatives and friends; based on the content materials, generating a story script corresponding to each story scene of the story using a large language model, and generating image generation prompt words and story narrative text corresponding to each story scene based on the story script, wherein the story narrative text is a narrative text that describes each story scene from the perspective of the patient's relatives and friends; generating video clips and voice audio corresponding to each story scene based on the story script, the image generation prompt words, the story narrative text, and the patient's relatives and friends voice audio; and generating a cognitive rehabilitation training video based on the video clips and voice audio.

[0008] According to another aspect of the embodiments of the present disclosure, a storage medium is further provided. The storage medium includes a stored program, wherein the above method is executed by a processor when the program is running.

[0009] According to another aspect of the embodiment of the present disclosure, a cognitive rehabilitation training device is also provided, including: a cognitive rehabilitation training video generation module, a cognitive rehabilitation training video playback module, an expression acquisition module, an expression and concentration recognition module, a consultation module, and an emotional state recording and analysis module. Among them, the cognitive rehabilitation training video generation module is used to generate cognitive rehabilitation training videos according to the above method; the cognitive rehabilitation training video playback module is used to play cognitive rehabilitation training videos; the expression acquisition module is used to use a camera to collect patient facial video data during cognitive rehabilitation training; the expression and concentration recognition module is used to identify the patient's facial expression and determine the patient's concentration score based on the frequency of changes in the patient's facial expression and emotional fluctuations; the consultation module is used to construct an interactive question-and-answer scene based on the concentration score and emotional state, combined with the story script and story scene image generated in the process of generating the cognitive rehabilitation training video, and interact with the patient based on the constructed interactive question-and-answer scene; and the emotional state recording and analysis module is used to monitor the patient's emotional changes throughout the process and generate an emotional analysis report.

[0010] According to another aspect of the embodiment of the present disclosure, a device for generating cognitive rehabilitation training content is also provided, including: a content material collection module for collecting content materials related to patients and their relatives and friends, the content materials including story narrative text, first character description text and photos related to patients and their relatives and friends, and collecting voice audio of patients and their relatives and friends, wherein the story narrative text is used to describe the story experienced by the patient and their relatives and friends, and the first character description text is used to describe the personality of the patient and their relatives and friends; a prompt word and story narrative text generation module for generating a story script corresponding to each story scene of the story based on the content material using a large language model, and generating prompt words and story narrative text corresponding to each story scene based on the story script, wherein the story narrative text is a narrative text that describes each story scene from the perspective of the patient and their relatives and friends; an audio and video generation module for generating video clips and voice audio corresponding to each story scene based on the story script, image generation prompt words, story narrative text and voice audio of patients and their relatives and friends; and generating a cognitive rehabilitation training video based on the video clips and voice audio.

[0011] According to another aspect of an embodiment of the present disclosure, a device for generating cognitive rehabilitation training content is also provided, including: a processor; and a memory, connected to the processor, for providing the processor with instructions for processing the following processing steps: collecting content materials related to the patient and the patient's relatives and friends, the content materials including story narrative text, first character description text and photos related to the patient and the patient's relatives and friends, and collecting voice audio of the patient's relatives and friends, wherein the story narrative text is used to describe the story experienced by the patient and the patient's relatives and friends, and the first character description text is used to describe the personality of the patient and the patient's relatives and friends; based on the content materials, using a large language model to generate story scripts corresponding to each story scene of the story, and generating image generation prompt words and story narrative text corresponding to each story scene based on the story script, wherein the story narrative text is a narrative text that describes each story scene from the perspective of the patient's relatives and friends; generating video clips and voice audio corresponding to each story scene based on the story script, image generation prompt words, story narrative text and patient's relatives and friends voice audio; and generating cognitive rehabilitation training videos based on the video clips and voice audio.

[0012] The present invention thus achieves the following beneficial effects: By integrating a training content framework based on memory models and cognitive psychology principles, combined with generative AI, multimodal large language models, and sentiment analysis technologies, it comprehensively optimizes cognitive rehabilitation training. By utilizing materials provided by patients' relatives and friends to generate personalized audio and video content, the training content is highly aligned with the patient's needs, significantly improving the relevance and effectiveness of training and effectively stimulating patient participation.

[0013] Throughout the training process, we incorporate elements of emotional awareness and companionship, utilizing the synthesis of voices from friends and family, materials based on shared experiences, and real-time responses to emotional changes to create a warm and familiar training environment. This not only helps alleviate patients' anxiety but also strengthens their compliance and persistence in training, thereby promoting the progress of cognitive rehabilitation.

[0014] Furthermore, the present invention can dynamically optimize training strategies based on the patient's training status and emotional changes, making the interactive training process more flexible and efficient, further enhancing training effectiveness. Through these approaches, the present invention can significantly improve patients' cognitive abilities, help them restore their social skills, reduce the burden on their families and society, and significantly enhance their quality of life, helping them better cope with the challenges of the disease and gradually move towards a relatively normal life trajectory. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of this application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings: Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to embodiment 1 of the present disclosure; Figure 2 is a flowchart of a method for generating cognitive rehabilitation training content according to the first aspect of Example 1 of the present disclosure; Figure 3 is a more specific flowchart of the method for generating cognitive rehabilitation training content according to the first aspect of Example 1 of the present disclosure; Figure 4 is a schematic diagram of the cognitive rehabilitation training device according to the second aspect of Example 1 of the present disclosure; Figure 5 is a schematic diagram of a device for generating cognitive rehabilitation training content according to embodiment 2 of the present disclosure; and Figure 6 3 is a schematic diagram of a device for generating cognitive rehabilitation training content according to Example 3 of the present disclosure. DETAILED DESCRIPTION

[0016] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present disclosure.

[0017] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0018] Example 1 The method embodiment provided in this embodiment can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Figure 1 FIG1 shows a hardware structure block diagram of a computing device for implementing the method of the present invention. Figure 1 As shown, a computing device may include one or more processors (the processor may include, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA) or other processing device), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, transmission device, and input / output interface are connected to the processor via a bus. In addition, it may also include: a display, a keyboard, and a cursor control device connected to the input / output interface. Those skilled in the art will understand that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0019] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be fully or partially integrated into any of the other components of the computing device. As discussed in the embodiments of the present disclosure, the data processing circuitry functions as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0020] The memory can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method of the present invention in the embodiment of the present disclosure. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the method of the present invention for implementing the above-mentioned application. The memory may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely located relative to the processor, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0021] The transmission device is used to receive or send data via a network. Specific examples of the aforementioned network may include a wireless network provided by a communications provider of the computing device. In one embodiment, the transmission device includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, the transmission device may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0022] The display may be, for example, a touch screen liquid crystal display (LCD) that enables a user to interact with a user interface of the computing device.

[0023] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing devices described above.

[0024] In the above operating environment, according to the first aspect of this embodiment, a method for generating cognitive rehabilitation training content is provided. Figure 2 A schematic diagram of the process is shown in FIG. Figure 2 As shown, the method includes: S202: Collecting content materials related to the patient and the patient's relatives and friends, including story narrative text, first character description text, and photos related to the patient and the patient's relatives and friends, and collecting voice and audio of the patient's relatives and friends, wherein the story narrative text is used to describe the story experienced by the patient and the patient's relatives and friends, and the first character description text is used to describe the personality of the patient and the patient's relatives and friends; S204: Based on the content material, a large language model is used to generate a story script corresponding to each story scene, and image generation prompt words and story narrative text corresponding to each story scene are generated based on the story script, wherein the story narrative text is a narrative text that describes each story scene from the perspective of the patient's relatives and friends; S206: Generate video clips and audio clips corresponding to each story scene based on the story script, image-generated prompt words, story narration text, and voice and audio of the patient's relatives and friends; and S208: Generate a cognitive rehabilitation training video based on the video clip and the voice audio clip.

[0025] Specifically, according to this embodiment, in order to generate cognitive rehabilitation training content for Alzheimer's patients, story materials can be collected through a computing device. P 1 and relatives and friends of the patient P 2 related content materials, including P 1 and relatives and friends of the patient P 2 Related story narrative text T story , character role and personality description (i.e. first character description text) T person ={ T person1 , T person2}, including patients P 1 and relatives and friends of the patient P 2 photos I person ={ I 1, I 2} and relatives and friends of patients P 2 Voice audio of patients' relatives and friends A audio The story text T story Used to describe patients P 1 and relatives and friends of the patient P 2 stories of shared experiences, first character description text T person Used to describe patients P 1 and relatives and friends of the patient P 2's character. Thus the above materials serve as the basic materials for subsequent content generation. Among them, T person Only a simple description is needed, such as "Character 1: patient, male, serious; Character 2: patient's wife, female, gentle." T storyThe time, place, characters and main plot of the event should be covered, as well as photos. I person The face of the person should be clearly identifiable and the audio data A audio The sound quality should be clear and without obvious noise interference. This collection process is recorded as: (S202).

[0026] Then, the computing device can be used to interact with the large language model based on the content material through prompt information to generate story scripts corresponding to each story scene. , and according to the story script T script Generate image-generating prompt words corresponding to each story scene and storytelling text , in which the story narrative text is in line with the patient's relatives and friends P 2. Narrative text describing each story scene from a perspective (S204). The specific process will be described in detail below.

[0027] Then, you can use the computing device to T script , image generation prompt words T prompt , storytelling text T narration And the voice audio of the patient's relatives and friends A audio , generating video clips corresponding to each story scene and audio clips of voice (S206). The specific process will be described in detail later.

[0028] Finally, a computing device can be used to V full and voice audio clips , generate cognitive rehabilitation training videos V final As cognitive rehabilitation training content (S208), the specific process will be described in detail later.

[0029] As mentioned in the background, traditional cognitive rehabilitation training often uses generic templates for content design, failing to consider the patient's life experiences, interests, and cognitive level. This results in a disconnect between the training content and the patient's actual situation, resulting in poor engagement and effectiveness. Therefore, the present invention provides a method for generating effective cognitive rehabilitation training content. Furthermore, it utilizes generative AI technology to generate personalized audio and video content based on material provided by the patient and their family and friends, to better meet the patient's cognitive needs.

[0030] Optionally, based on the content material, the operation of using the large language model to generate a story script corresponding to each story scene includes: T person , using a large language model to generate P 1 and relatives and friends of the patient P 2 related second character description text T character ={ T character1 , T character2}, where the second character description text is used to describe the patient P 1 and relatives and friends of the patient P 2's personality traits and abilities; and based on the story text T story and the second character description text T character Generate story script T script .

[0031] Specifically, it is possible to interact with a large language model through a computing device. For example, let the GPT4o-mini model act as a story writer and use the story characters to improve the prompt words. P character , according to the patient P 1 and relatives and friends of the patient P 2. First character description text , form the corresponding prompt information and input it into the large language model GPT4o-mini, so that the patient's P 1 and relatives and friends of the patient P 2's personality traits and abilities (second character description text) , recorded as: , .

[0032] Among them, the story character improvement prompt words P character An example is: "You are now a professional novel character creation expert. According to the user's character requirements < >, please fill in the basic information of this character in the following format: Name: ; Species: ; Gender: ; Personality Preference: ; Personality Ability: ".

[0033] An example would be: "Thomas, male, serious and insightful."

[0034] generate An example is as follows: "Name: Thomas; Species: Human; Gender: Male; Personality Preferences: Likes to be alone, likes to read, and likes a quiet environment; Personality Abilities: Although he has a serious demeanor, he has a strong insight into the emotions of others."

[0035] The story script can then be used by a computing device to generate prompt words P script , according to the second character description text and storytelling text T story Construct the corresponding prompt information and input it into the large language model GPT4o-mini, so that the large language model GPT4o-mini can be used according to the predetermined number of scenarios n Generate multi-scene story scripts , recorded as: .

[0036] Among them, the story script generates prompt words P script , an example is: "You are the plot writer of the novel. Write a short story based on the tone and characters of the story input by the user. Please divide the entire story into ten separate scenes. Each scene should be a short sentence. Each scene should be described independently, including the scene layout. The final output should follow the following format: Scene #1: ; Scene #2: Scene 3: ; Scene #4: ; Scene 5: ; Scene #6: ; Scene #7: ; Scene #8: ; Scene #9: ; Scene #10: . You should keep the story as simple as possible. When writing the story, you need to mention two characters in each scene and write in the third person. The writing style should be natural. The description of character 1 is , the description of role 2 is , the story is T story >".

[0037] T story An example is: "The couple had dinner together, went to the park, and fed the ducks."

[0038] Generated Used to describe character actions and scene details in detail. For example, the generated An example would be: "In the kitchen, a man and a woman worked together to prepare dinner, and the air was filled with the aroma of food."

[0039] Optionally, based on the story script T script Generate image-generating prompt words corresponding to each story scene Operations include: using a large language model, according to the story script T script Generate image and generate prompt words T prompt . And according to the story script T script Generate story narrative text Operations include: using a large language model, according to the story script T script Generate story narrative text T narration .

[0040] That is to say, in this embodiment, the GPT4o-mini model can be used as a prompt information assistant of a text graph model, and the computing device can generate prompt information based on the prompt words. P prompt ,according to Construct corresponding prompt information, and then input the large language model GPT4o-mini to generate prompt words for image generation , recorded as: .

[0041] Prompt words used to generate prompt information P prompt, an example is: "You will act as an artistic StableDiffusion prompt assistant. ## Background introduction: Stable Diffusion is a text-based graph model that uses deep learning. It supports generating new images by using prompts to describe the elements to be included. ## Prompt concept: prompt is used to describe images. It is composed of common words and uses English half-width "," as delimiters. Each word or phrase separated by "," is called a tag. So the prompt is composed of a series of tags separated by ",". ## Format requirements: You will accept a complete story input by the user. Each story is divided into different Scences, and generate corresponding prompts based on the described scenes. Steps to generate prompts: ① Add "a couple" to the beginning of the prompt, separated by ","; ② The actions or interactions of the characters, accurate and concise. ③ Main body of the picture: Accurate, complete and concise English description of the main body of the picture, such as A girl in a garden, and summarize the details of the main body (the main body can be people, things, objects, and scenery) and the core content of the picture. This part is generated based on the theme I give you each time. You can add more reasonable details related to the theme. ④ Scene description and character layout: You need to complete the background of each Scence (there must be a description of the location) and the description of the character layout according to the context of the entire story, such as "a man standing by the river", "standing under a tree", "sitting at home", etc. ## Restrictions on prompts: ① The tag content is described in English words or phrases, and is not limited to the words I give you. Note that it can only contain keywords or phrases. ② The number of tags is limited to 20, and the number of words is limited to 30. ③ Tags should not be enclosed in quotation marks (""). ④ Tags are arranged in order of importance from high to low. # Task: The user will tell you the story content of the prompt to be generated in natural language. Your task is to imagine a complete picture based on this content, and then convert it into a detailed, high-quality prompt so that Stable Diffusion can generate high-quality images. The final output should follow this format: #Scene 1: settings of scene 1;#Scene 2: settings of scene 2; #Scene3: settings of scene 3;and so on. The given story script is < T script >." Contains iThe subject of the scene, the subject's actions and events, and the layout of the scene and characters. For example, the generated An example is: "a couple, preparing dinner in the kitchen, in the kitchen at home, the food is fragrant, intimate and warm atmosphere."

[0042] In addition, the GPT4o-mini model can be used as a storyteller, and the prompt words generated by the computational model based on the storytelling words P narration , based on the generated story script content T script Construct corresponding prompt information, and then input it into the large language model GPT4o-mini to generate story narrative text that matches the perspective and tone of relatives and friends . It is necessary to accurately describe the core events within the specified word count to provide appropriate text materials for subsequent speech generation, which can be recorded as: .

[0043] Among them, the prompt word P narration , for example: "You are a narrator of a story. Based on a story input by the user, select character2< T character2 >Rewrite the story as the protagonist. You must use "you" to refer to character 1 in the story. T character1 >. Each scene should be a simple sentence describing only the main events. Each scene should be narrated on a separate line. The final output should follow this format: Scene #1: Short narration of Act 1; Scene #2: Short narration of Act 2; Scene 3: Short narration of Act 3; and so on... You should keep the story as simple as possible. The given story script is < T script >." Generated An example is: "In the cozy kitchen, the aroma of food filled the air as we prepared dinner together, our movements coordinated perfectly."

[0044] Optionally, video clips corresponding to each story scene are generated. Operations include: I person Processing is performed to obtain the character image after removing the backgroundI processed ={ I processed1 , I processed2}; According to the story script T script and character images I processed , using images to generate prompt words, and generate story scene images corresponding to each story scene ; and according to the story scene image I scene and story scripts T script , generate video clips .

[0045] Specifically, the computing device first uses the RMBG-1.4 model and a precise image segmentation algorithm to segment the person's photo. I person Perform background removal to obtain a pure human image I processed ={ I processed1 , I processed2}, recorded as: .

[0046] The algorithm accurately distinguishes the main body of the person and the background by analyzing and identifying image pixels, ensuring that the background pixels are completely removed and that subsequent image synthesis steps are not affected by the background pixel content.

[0047] The computing device then uses the StoryMaker model to generate prompt words based on the image. , and the character image after removing the background and , generating a series of starting frames that are consistent with the story scenes with the reference characters as the main body. Specifically, the computing device generates prompt words for the images corresponding to each story scene Through the text encoder Encode to get text feature vector , the patient P 1 and relatives and friends of the patient P 2 photos I person ={ I 1, I 2} Using Image Encoder and feature extraction encoder for face detection , get the image feature vector , , and the facial feature vector , Then, the attention mechanism is used to fuse the text feature vector and image features into the diffusion generation process of StoryMaker to generate a patient P 1 and relatives and friends of the patient P 2 story scene images corresponding to each story scene . It is recorded as: .

[0048] The computing device then uses KLING AI’s graph-based video model to transform the story scene images into I scene and story scripts T script Input the model to generate the corresponding starting video clip . It is recorded as: .

[0049] Then, based on the generated starting video clip, the computing device uses KLING AI's video continuation model to continue reasonable plot reasoning based on the storyline and continue to generate video content, obtaining video clips corresponding to each story scene: . It is recorded as: .

[0050] Among them, the video continuation model analyzes the existing video content and story scripts, predicts the development trends of subsequent plots and character actions, and thus generates coherent and reasonable video content.

[0051] Further optionally, generate a speech audio segment Operations include: Narrating text based on the story T narration And refer to the voice audio of the patient's relatives and friends A audio , generating narrative texts related to each story T narration Corresponding audio clip A narration .

[0052] Specifically, in the process of generating the voice audio segment, the computing device can generate the voice audio segment according to the story narration text T narration , and refer to the audio of patients' relatives and friends A audio , using the GPT-SoVITS model to convert the story narrative text T narration Converted into audio clips of speech similar to the speech of the patient's relatives and friends , among which The generation process of a speech audio segment is recorded as: .

[0053] This allows you to convert voice audio clips A narration Serves as the voiceover for the video clips of the corresponding story scenes.

[0054] Further optionally, generate a cognitive rehabilitation training video V final The operation includes: adjusting the speech speed of the voice audio segment according to the video duration of the corresponding video segment to obtain the adjusted voice audio segment A adjusted ; and voice audio A audio Synthesize with video clips to generate cognitive rehabilitation training videos V final . Optionally, the voice audio A audio Synthesize with video clips to generate cognitive rehabilitation training videos V final The operation also includes: narrating the text according to the story T narration Generate corresponding subtitle files Sub final ; and voice audio A audio , video clips and subtitle files Sub final Synthesize and generate cognitive rehabilitation training videos.

[0055] Specifically, in this embodiment, the computing device can also use FFmpag to achieve story voice, video and subtitle synthesis and overall video production. Specifically: First, according to each video clip Duration , for speech audio clips The speech speed is adjusted to match the rhythm of the video. .

[0056] Then, use the storytelling text As the text of the subtitles, adjust the font size and length of the subtitles to ensure that the subtitles are clear and easy to read in the video screen. The subtitle file obtained by processing is .

[0057] Finally, the video , audio and subtitles Synthesize the final cognitive rehabilitation training video .

[0058] Thus, through the above methods, the present invention provides a method for generating effective cognitive rehabilitation training content. At the same time, using generative AI technology, personalized audio and video content is generated based on the materials provided by the patient and their relatives and friends to better meet the patient's cognitive needs.

[0059] also, Figure 3 A specific flow chart showing the method for generating effective cognitive rehabilitation training content according to the invention is shown: S302: Collect content materials related to the patient and his / her relatives and friends, and record them as: ; S304: Based on the patient P 1 and relatives and friends of the patient P 2. First character description text , generate the second character description text T character ={ T character1 , T character2}, recorded as: , ; S306: Describing the text according to the second person T characte and storytelling text T story Generate story scripts for multiple scenes , recorded as: ; S308: According to the story script T script Generate image-generating prompt words corresponding to each story scene , recorded as: ; S310: According to the story script T script Generate story narrative text corresponding to each story scene , recorded as: ; S312: Photos I person Processing is performed to obtain the character image after removing the background I processed ={ I processed1 , I processed2}, recorded as: ; S314: Generate prompt words based on the image T prompt and character images I processed , generate story scene images corresponding to each story scene , recorded as: ; S316: Based on story scene images I scene and story script T script Generate the corresponding starting video clip , recorded as: ; S318: Generate video clips corresponding to each story scene based on the starting video clip and the story script: , recorded as: ; S320: Generate voice audio segments corresponding to each story scene , recorded as: ; S322: Adjusting the voice audio segment to match the duration of the video segment , and process the story narration file to obtain subtitle files corresponding to each story scene ; S324: Video clip , audio clip and subtitles Synthesize the final cognitive rehabilitation training video .

[0060] The design of the cognitive rehabilitation training content provided by the present invention combines the memory model and the cognitive psychology model, and its characteristics are: 1) Training content design based on scenario review: Using the scenario review training paradigm, audio and video materials are used to activate the patient's brain memory consolidation neural network and enhance memory and cognitive abilities.

[0061] 2) Multimodal training stimulation content design: By combining subtitles, sound, vision and other multi-channel stimulation, we can activate the coordinated work of multiple brain regions, thereby optimizing cognitive training effects.

[0062] 3) Design training content that integrates emotional support. Create audio and video stories based on experiences shared by patients and their families and friends, simulating the voices of those close to them. By awakening and resonating with emotional memories, this approach can alleviate patients' emotions, enhance cognitive processing, and strengthen their acceptance of training.

[0063] In addition, reference Figure 1 As shown, according to a second aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0064] In addition, according to another aspect of this embodiment, a cognitive rehabilitation training device is provided. Figure 4 A schematic diagram of the cognitive rehabilitation training device 100 is shown. Figure 4 As shown, the cognitive rehabilitation training device 100 includes: a cognitive rehabilitation training video generation module 102, a cognitive rehabilitation training video playback module 104, an expression collection module 106, an expression and concentration recognition module 108, a consultation module 110 and an emotional state recording and analysis module 112.

[0065] The cognitive rehabilitation training video generation module 102 is configured to generate a cognitive rehabilitation training video according to any of the above methods.

[0066] The cognitive rehabilitation training video playing module 104 is used to play cognitive rehabilitation training videos to perform cognitive rehabilitation training on patients.

[0067] The expression acquisition module 106 is used to collect patient facial video data using a camera during cognitive rehabilitation training. The captured video data has clear visual content and can capture subtle changes in the patient's facial expressions.

[0068] The expression and focus recognition module 108 is used to accurately identify the patient's facial emotional fluctuations using the Convnextv2 model , Including happiness, sadness, surprise, fear, anger, disgust and contempt. Then, through the frequency of changes in facial expressions and mood swings Determine the patient's concentration and obtain a concentration score .

[0069] The consultation module 110 has built-in professional cognitive rehabilitation consultation logic, and combines the established story background to conduct personalized dialogue guidance. The language style is gentle and professional, focusing on creating a safe and pressure-free communication environment, and timely repeating and summarizing the patient's words to ensure understanding and attention to the patient's expression. The AI ​​consultation is based on the patient's concentration. and mood swings , combined with the generated personalized story script content With visual content , build an interactive question-and-answer scenario. The specific steps are as follows: First, during the interactive Q&A session for each scenario, the patient is shown the corresponding image. , through intuitive visual presentation, help patients better integrate into the story context. Then, according to the concentration level and emotional label, different question and answer guidance is carried out. When the concentration is low, there is no obvious emotion, or the emotion is doubtful, the optimization work of deepening memory is prioritized. At this time, the memory guidance prompt word is sent to GPT-4o-mini , the format is "Based on the situation: < T script >, generate 1-3 key memory questions (identifying the characters / time / place / event). Example: "Where did the protagonist go? What did they do?" Later in the conversation, understand the patient's answers. If the patient has difficulty understanding a question or answers inaccurately, patiently provide explanations and guidance to help the patient better understand the question and answer it smoothly. Example guidance: "The story mentions where the protagonist departed. Do you remember? / They might have gone to a place related to walking." After generating the questions, engage in a Q&A session based on the context. This approach guides the patient to focus on the key information in the story and strengthens their memory.

[0070] When the patient's response shows that the memory is successful, the emotional empathy mode guidance phase based on the patient's emotional expression is entered. At this time, the emotional empathy prompt word is sent to GPT-4o-mini , an example is: "Analysis situation: < T script >, guiding the user into a conversation. Subsequent questions may include: 1. Emotion recognition layer. Example: "Are you feeling happy / sad / surprised / afraid right now?" 2. Experience association layer. Example: "Have you ever encountered a similar situation before? / Have you thought of anything else that made you feel <happy / sad / surprised / afraid>?" 3. Value guidance layer. Example: "If you had a choice, what would you do? / What else do you want to do next?" During the conversation, the user is required to be guided to express their emotions and to provide positive responses and support. This approach allows for more targeted interaction with patients, helping them better understand the story and strengthen their memory during cognitive rehabilitation, while also promoting emotional expression and psychological adjustment.

[0071] The emotional state recording and analysis module 112 is used to monitor the patient's emotional changes throughout the process and generate an emotional analysis report. The emotion analysis report contains information such as emotion changes, frequency and duration of expression changes in each time period, which is used to adjust the cognitive training program.

[0072] The present invention thus achieves the following beneficial effects: By integrating a training content framework based on memory models and cognitive psychology principles, combined with generative AI, multimodal large language models, and sentiment analysis technologies, it comprehensively optimizes cognitive rehabilitation training. By utilizing materials provided by patients' relatives and friends to generate personalized audio and video content, the training content is highly aligned with the patient's needs, significantly improving the relevance and effectiveness of training and effectively stimulating patient participation.

[0073] Throughout the training process, we incorporate elements of emotional awareness and companionship, utilizing the synthesis of voices from friends and family, materials based on shared experiences, and real-time responses to emotional changes to create a warm and familiar training environment. This not only helps alleviate patients' anxiety but also strengthens their compliance and persistence in training, thereby promoting the progress of cognitive rehabilitation.

[0074] Furthermore, the present invention can dynamically optimize training strategies based on the patient's training status and emotional changes, making the interactive training process more flexible and efficient, further enhancing training effectiveness. Through these approaches, the present invention can significantly improve patients' cognitive abilities, help them restore their social skills, reduce the burden on their families and society, and significantly enhance their quality of life, helping them better cope with the challenges of the disease and gradually move towards a relatively normal life trajectory.

[0075] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0076] Through the description of the above embodiments, those skilled in the art will clearly understand that the methods according to the above embodiments can be implemented using software plus the necessary general-purpose hardware platform. Of course, hardware can also be used, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, or optical disk) and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0077] Example 2 Figure 5 FIG2 shows a device 500 for generating cognitive rehabilitation training content according to this embodiment, which corresponds to the method described in the first aspect of embodiment 1. Figure 5 As shown, the device 500 includes: a content material collection module 510, which is used to collect content materials related to patients and their relatives and friends, the content materials including story narrative text, first character description text and photos related to the patients and their relatives and friends, and collects voice and audio of the patients and their relatives and friends, wherein the story narrative text is used to describe the story experienced by the patients and their relatives and friends, and the first character description text is used to describe the personality of the patients and their relatives and friends; a prompt word and story narrative text generation module 520, which is used to generate a story script corresponding to each story scene of the story based on the content material using a large language model, and generate prompt words and story narrative text corresponding to each story scene based on the story script, wherein the story narrative text is a narrative text that describes each story scene from the perspective of the patients and their relatives and friends; an audio and video generation module 530, which is used to generate prompt words, story narrative text and voice and audio of the patients and their relatives and friends based on the story script, the image, and generate video clips and voice audio corresponding to each story scene; and a cognitive rehabilitation training video generation module 540, which is used to generate cognitive rehabilitation training videos based on the video clips and voice audio.

[0078] Optionally, the prompt word and story narration text generation module 520 includes: a second character description text generation sub-module, which is used to generate a second character description text related to the patient and the patient's relatives and friends based on the first character description text using a large language model, wherein the second character description text is used to describe the personality traits and ability expertise settings of the patient and the patient's relatives and friends; and a story script generation sub-module, which is used to generate a story script based on the story narration text and the second character description text.

[0079] Optionally, the prompt word and story narrative text generation module 520 includes an image-generated prompt word generation submodule for generating image-generated prompt words based on the story script using a large language model. Furthermore, the prompt word and story narrative text generation module 520 includes a story narrative text generation submodule for generating story narrative text based on the story script. The operation of generating story narrative text based on the story script includes generating story narrative text based on the story script using the large language model.

[0080] Optionally, the audio and video generation module 530 includes: a photo processing submodule, which is used to process photos to obtain character images after removing the background; a story scene image generation submodule, which is used to generate prompt words using images based on the story script and character images, and generate story scene images corresponding to each story scene; and a video clip generation submodule, which is used to generate video clips based on the story scene images and the story script.

[0081] Optionally, the audio and video generation module 530 includes: a voice and audio generation submodule, which is used to generate voice and audio corresponding to each story narrative text based on the story narrative text and with reference to the voice and audio of the patient's relatives and friends.

[0082] Optionally, the cognitive rehabilitation training video generation module 540 includes: a speech and audio adjustment submodule, which is used to adjust the speaking speed of the speech and audio according to the video length of the corresponding video clip to obtain the adjusted speech and audio; and a cognitive rehabilitation training video generation submodule, which is used to synthesize the speech and audio and the video clip to generate a cognitive rehabilitation training video.

[0083] Optionally, the cognitive rehabilitation training video generation submodule further includes: a subtitle file unit for generating a corresponding subtitle file based on the story narrative text; and a video synthesis unit for synthesizing the voice audio, video clips and subtitle file to generate a cognitive rehabilitation training video.

[0084] Therefore, the design of the cognitive rehabilitation training content provided by the present invention combines the memory model and the cognitive psychology model, and its characteristics are: 1) Training content design based on scenario review: Using the scenario review training paradigm, audio and video materials are used to activate the patient's brain memory consolidation neural network and enhance memory and cognitive abilities.

[0085] 2) Multimodal training stimulation content design: By combining subtitles, sound, vision and other multi-channel stimulation, we can activate the coordinated work of multiple brain regions, thereby optimizing cognitive training effects.

[0086] 3) Design training content that integrates emotional support. Create audio and video stories based on experiences shared by patients and their families and friends, simulating the voices of those close to them. By awakening and resonating with emotional memories, this approach can alleviate patients' emotions, enhance cognitive processing, and strengthen their acceptance of training.

[0087] Therefore, the present invention can achieve the following beneficial effects: through the training content architecture based on memory models and cognitive psychology principles, combined with the synergy of generative AI, multimodal large language models and sentiment analysis technology, cognitive rehabilitation training is comprehensively optimized.

[0088] Example 3 Figure 6 FIG2 shows a device 600 for generating cognitive rehabilitation training content according to this embodiment, which corresponds to the method described in the first aspect of embodiment 1. Figure 6As shown, the device 600 includes: a processor 610; and a memory 620, which is connected to the processor 610 and is used to provide the processor with instructions for processing the following processing steps: collecting content materials related to the patient and the patient's relatives and friends, the content materials including story narrative text, first character description text and photos related to the patient and the patient's relatives and friends, and collecting patient's relatives and friends' voice audio, wherein the story narrative text is used to describe the story experienced by the patient and the patient's relatives and friends, and the first character description text is used to describe the personality of the patient and the patient's relatives and friends; based on the content materials, using a large language model to generate story scripts corresponding to each story scene of the story, and generating image generation prompt words and story narrative text corresponding to each story scene based on the story script, wherein the story narrative text is a narrative text that describes each story scene from the perspective of the patient's relatives and friends; based on the story script, image generation prompt words, story narrative text and patient's relatives and friends' voice audio, generate video clips and voice audio corresponding to each story scene; and generate cognitive rehabilitation training videos based on the video clips and voice audio.

[0089] Optionally, the operation of using a large language model to generate a story script corresponding to each story scene of the story based on the content material includes: using a large language model to generate a second character description text related to the patient and the patient's relatives and friends based on the first character description text, wherein the second character description text is used to describe the personality traits and ability expertise settings of the patient and the patient's relatives and friends; and generating a story script based on the story narrative text and the second character description text.

[0090] Optionally, the operation of generating image generation prompt words corresponding to each story scene according to the story script includes: using a large language model to generate image generation prompt words according to the story script, and the operation of generating story narrative text according to the story script includes: using a large language model to generate story narrative text according to the story script.

[0091] Optionally, the operation of generating video clips corresponding to each story scene includes: processing photos to obtain character images after removing the background; generating prompt words using images according to the story script and the character images, and generating story scene images corresponding to each story scene; and generating video clips according to the story scene images and the story script.

[0092] Optionally, the operation of generating voice audio includes: generating voice audio corresponding to each story narration text based on the story narration text and referring to the voice audio of the patient's relatives and friends.

[0093] Optionally, the operation of generating a cognitive rehabilitation training video includes: adjusting the speech speed of the voice audio according to the video length of the corresponding video clip to obtain adjusted voice audio; and synthesizing the voice audio and the video clip to generate a cognitive rehabilitation training video.

[0094] Optionally, the operation of synthesizing the speech audio and video clips to generate a cognitive rehabilitation training video also includes: generating a corresponding subtitle file based on the story narrative text; and synthesizing the speech audio, video clips and subtitle file to generate a cognitive rehabilitation training video.

[0095] Therefore, the design of the cognitive rehabilitation training content provided by the present invention combines the memory model and the cognitive psychology model, and its characteristics are: 1) Training content design based on scenario review: Using the scenario review training paradigm, audio and video materials are used to activate the patient's brain memory consolidation neural network and enhance memory and cognitive abilities.

[0096] 2) Multimodal training stimulation content design: By combining subtitles, sound, vision and other multi-channel stimulation, we can activate the coordinated work of multiple brain regions, thereby optimizing cognitive training effects.

[0097] 3) Design training content that integrates emotional support. Create audio and video stories based on experiences shared by patients and their families and friends, simulating the voices of those close to them. By awakening and resonating with emotional memories, this approach can alleviate patients' emotions, enhance cognitive processing, and strengthen their acceptance of training.

[0098] Therefore, the present invention can achieve the following beneficial effects: through the training content architecture based on memory models and cognitive psychology principles, combined with the synergy of generative AI, multimodal large language models and sentiment analysis technology, cognitive rehabilitation training is comprehensively optimized.

[0099] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0100] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0101] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0102] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0103] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0104] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), a mobile hard drive, a magnetic disk, or an optical disk.

[0105] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A method for generating cognitive rehabilitation training content, characterized in that: include: Collecting content materials related to the patient and their relatives and friends, including story narrative text, first character description text, and photos related to the patient and their relatives and friends, and collecting voice audio of the patient and their relatives and friends, wherein the story narrative text is used to describe the story experienced by the patient and their relatives and friends, and the first character description text is used to describe the personality of the patient and their relatives and friends; Based on the content material, a large language model is used to generate a story script corresponding to each story scene of the story, and image generation prompt words and story narrative text corresponding to each story scene are generated based on the story script, wherein the story narrative text is a narrative text that describes each story scene from the perspective of the patient's relatives and friends; Generate video clips and voice audio corresponding to each story scene based on the story script, the image generation prompt words, the story narration text, and the voice audio of the patient's relatives and friends; as well as A cognitive rehabilitation training video is generated based on the video clip and the voice audio.

2. The method according to claim 1, characterized in that The operation of generating a story script corresponding to each story scene of the story using a large language model based on the content material includes: Based on the first character description text, using a large language model to generate a second character description text related to the patient and the patient's relatives and friends, wherein the second character description text is used to describe the personality traits and ability and expertise settings of the patient and the patient's relatives and friends; and The story script is generated according to the story narration text and the second character description text.

3. The method according to claim 2, characterized in that The operation of generating image generation prompt words corresponding to each story scene according to the story script includes: using a large language model to generate the image generation prompt words according to the story script, and The operation of generating the story narration text according to the story script includes: using a large language model to generate the story narration text according to the story script.

4. The method according to claim 3, characterized in that The operation of generating video clips corresponding to the respective story scenes includes: Processing the photo to obtain a person image after removing the background; According to the story script and the character images, using the images to generate prompt words, and generating story scene images corresponding to each story scene; and The video clip is generated according to the story scene image and the story script.

5. The method according to claim 4, characterized in that The operation of generating the voice audio includes: The voice audio corresponding to each story narration text is generated based on the story narration text and with reference to the voice audio of the patient's relatives and friends.

6. The method according to claim 5, characterized in that The operations for generating cognitive rehabilitation training videos include: Adjusting the speech speed of the voice audio according to the video duration of the corresponding video clip to obtain adjusted voice audio; and The speech audio and the video clip are synthesized to generate the cognitive rehabilitation training video, and wherein, The operation of synthesizing the voice audio and the video clip to generate the cognitive rehabilitation training video also includes: Generate a corresponding subtitle file according to the story narrative text; and The speech audio, the video clip and the subtitle file are synthesized to generate the cognitive rehabilitation training video.

7. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the processor executes the method according to any one of claims 1 to 6.

8. A cognitive rehabilitation training device, characterized in that: include: A cognitive rehabilitation training video generation module (102), a cognitive rehabilitation training video playback module (104), an expression acquisition module (106), an expression and concentration recognition module (108), a consultation module (110), and an emotional state recording and analysis module (112), wherein: The cognitive rehabilitation training video generation module (102) is used to generate a cognitive rehabilitation training video according to the method according to any one of claims 1 to 6; The cognitive rehabilitation training video playing module (104) is used to play the cognitive rehabilitation training video; The expression acquisition module (106) is used to collect patient facial video data using a camera during cognitive rehabilitation training; The expression and concentration recognition module (108) is used to recognize the patient's facial expression and determine the patient's concentration score based on the frequency of changes in the patient's facial expression and emotional fluctuations; The consultation module (110) is used to construct an interactive question-and-answer scene based on the concentration score and the emotional state, in combination with the story script and story scene image generated in the process of generating the cognitive rehabilitation training video, and interact with the patient based on the constructed interactive question-and-answer scene; and The emotional state recording and analysis module (112) is used to monitor the patient's emotional changes throughout the entire process and generate an emotional analysis report.

9. A device for generating cognitive rehabilitation training content, characterized in that: include: A content material collection module is used to collect content materials related to patients and their relatives and friends, wherein the content materials include story narrative text, first character description text, and photos related to the patients and their relatives and friends, and collect voice audio of the patients and their relatives and friends, wherein the story narrative text is used to describe the stories experienced by the patients and their relatives and friends, and the first character description text is used to describe the personalities of the patients and their relatives and friends; a prompt word and story narrative text generation module, configured to generate, based on the content material, a story script corresponding to each story scene of the story using a large language model, and generate, based on the story script, image generation prompt words and story narrative text corresponding to each story scene, wherein the story narrative text is a narrative text describing each story scene from the perspective of the patient's relatives and friends; An audio and video generation module is used to generate video clips and voice audio corresponding to each story scene based on the story script, the image generation prompt words, the story narrative text, and the voice audio of the patient's relatives and friends; as well as The cognitive rehabilitation training video generation module is used to generate a cognitive rehabilitation training video based on the video clips and the voice audio.

10. A device for generating cognitive rehabilitation training content, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Collecting content materials related to the patient and their relatives and friends, including story narrative text, first character description text, and photos related to the patient and their relatives and friends, and collecting voice audio of the patient and their relatives and friends, wherein the story narrative text is used to describe the story experienced by the patient and their relatives and friends, and the first character description text is used to describe the personality of the patient and their relatives and friends; Based on the content material, a large language model is used to generate a story script corresponding to each story scene of the story, and image generation prompt words and story narrative text corresponding to each story scene are generated based on the story script, wherein the story narrative text is a narrative text that describes each story scene from the perspective of the patient's relatives and friends; Generate video clips and voice audio corresponding to each story scene based on the story script, the image generation prompt words, the story narration text, and the voice audio of the patient's relatives and friends; as well as A cognitive rehabilitation training video is generated based on the video clip and the voice audio.

Images