Role playing data set construction method based on image enhancement

Abstract

The invention discloses a role playing data set construction method based on image enhancement. Relates to the technical field of digital human interaction. Comprising the following steps: acquiring a multi-scene image, and processing by using a multi-modal large language model to generate an image text description; based on the image text description, adopting a retrieval enhancement generation method to generate a figure description of the image main body; generating a memory story between the image content and the person settings by using a DeepSeek large language model based on the image text description and the person settings description; performing multiple rounds of interaction according to the memory story by adopting a double-role playing mechanism, and generating a dialogue data set according to an interaction result; and performing supervised fine tuning on the multi-modal large language model by using the generated dialogue data set. According to the method, a high-quality, personalized and emotion-rich Chinese memory therapy dialogue data set can be constructed, and the method has an important application value for improving artificial intelligence and assisting sentiment and cognitive health of old people.

Description

Technical Field

[0001] This invention relates to the field of digital human interaction technology, and more specifically to a method for constructing a role-playing dataset based on image enhancement. Background Technology

[0002] Among numerous non-pharmacological interventions, recall therapy has shown great potential. This therapy, through external stimuli such as photos and videos, guides older adults to recall and share personal experiences, and has been proven to effectively improve mood, enhance communication, and promote social participation.

[0003] However, traditional memory therapy relies heavily on professionally trained personnel for guidance, making it resource-intensive and difficult to scale up. Furthermore, many older adults experience communication difficulties, hindering sustained and in-depth conversations, which significantly diminishes the therapy's potential to evoke deep emotional resonance.

[0004] To overcome these challenges, existing technologies have proposed several automated solutions that attempt to combine visual question-answering generation technology with dialogue systems for image-based memory therapy. The typical workflow of these systems involves first generating a series of questions based on the image content, and then passing the user's answers to an independent dialogue module for further interaction. While these systems achieve a degree of interaction, their inherent technical limitations are also significant. First, the interaction process is rigid, lacking the flexibility and personalization to adjust based on real-time user feedback, making it difficult to establish genuine emotional connections with elderly users. More importantly, a critical gap exists in the current technological field: a severe lack of high-quality, multi-turn dialogue datasets specifically designed for memory therapy scenarios. This gap limits the development and evaluation of intelligent dialogue systems capable of guiding elderly users through emotional and immersive memory experiences.

[0005] Therefore, how to construct a dataset that can fit the life experience of the elderly, support multi-turn natural interaction, and take into account both emotional support and therapeutic effect assessment, so as to train a dialogue model with greater empathy and intelligence, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] In view of this, the present invention provides a method for constructing a role-playing dataset based on image enhancement. Through a scenario-driven interactive approach, the present invention can construct a high-quality, personalized, and emotionally rich Chinese memory therapy dialogue dataset, which significantly improves the empathy, scenario adaptability, and emotional support capabilities of the model's generated responses. This has important application value for improving the emotional and cognitive health of the elderly assisted by artificial intelligence.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: a method for constructing a role-playing dataset based on image enhancement, comprising: Acquire images from multiple scenes and use a multimodal large language model to generate image text descriptions; Based on image text description, a retrieval-enhanced generation method is used to generate a character description of the main subject of the image; Based on image text descriptions and character descriptions, DeepSeek large language model is used to generate memory stories between image content and character descriptions; A dual-role-playing mechanism is adopted, which involves multiple rounds of interaction based on the recalled story, and a dialogue dataset is generated based on the interaction results. Supervised fine-tuning of a multimodal large language model is performed using the generated dialogue dataset.

[0008] Preferably, the image text description includes the appearance and emotional characteristics of the people in the image, the inferred location and context, and a comprehensive explanation of the entire image.

[0009] Preferably, the specific process for generating the character description includes: Initial search: Retrieve relevant character elements from publicly available character datasets based on image and text descriptions; Iterative improvement: When character elements are missing, a complete character design is formed by supplementing the search results; File integration: Integrate all character elements into a character description, which includes name, gender, age, occupation, and hobbies.

[0010] Preferably, the DeepSeek large language model generates narrative memory stories based on image text descriptions and character descriptions; the memory stories include image scenes, emotional reactions, and character interactions, and are personalized according to different character descriptions.

[0011] Preferably, the process of generating a dialogue dataset related to the memory story includes: The DeepSeek R1 model simulates caregivers, while the DeepSeek R1-Qwen 32B model simulates the elderly. The DeepSeek R1-Qwen 32B model demonstrates differences in memory clarity and emotional responses based on the content of images in different scenarios. The DeepSeek R1 model guides the DeepSeek R1-Qwen 32B model to recall based on the content of the scene image, and adjusts the questioning strategy according to the clarity of the DeepSeek R1-Qwen 32B model's answer; By using different questioning strategies, multi-turn dialogue results are obtained, and a multi-turn dialogue dataset for Chinese image recall therapy is generated based on the dialogue results.

[0012] Preferably, the supervised fine-tuning process includes: Base model selection: A pre-trained Chinese multimodal large language model with instruction-following capabilities was selected as the base model; Supervised fine-tuning: The base model is fine-tuned using the multi-turn dialogue dataset. During training, the current image, character description, and multi-turn dialogue history are combined and input into the base model. The training objective is to enable the base model to generate standard assistant responses in the dialogue sequence. Enhanced Thinking Chains: During the training or reasoning phase, the thinking chain prompting method is used to guide the pedestal model to simulate the questioning and reasoning of caregivers, and to understand the story behind the image through step-by-step analysis and reasoning.

[0013] As can be seen from the above technical solution, compared with the prior art, this invention discloses a method for constructing a role-playing dataset based on image enhancement. This method constructs a context-driven framework for character information generation and role-playing, using scenario images related to the life experiences of the elderly to guide participants in multi-round dialogues around the images, forming a standard dataset for training emotion-supportive dialogue models. The beneficial effects of this invention are: It solves the data acquisition challenge: by constructing a framework using synthetic data, it effectively avoids the privacy, cost, and resource limitations encountered when collecting elderly people's dialogue data in real-world scenarios, and achieves scalable data production.

[0014] Improved data quality and authenticity: Through a systematic process of scenario-driven, persona-building, story generation, and role-playing, multi-turn dialogues with personalization, contextualization, and emotional depth were generated, resulting in high-quality data that is closer to real therapeutic interactions.

[0015] Enhanced model performance: Dialogue models trained on datasets constructed using this method outperform those with empathy, contextual adaptability, and emotional support in generating responses, enabling more therapeutic and memory-evoking interactions.

[0016] It has promoted the application of technology: It provides a key data foundation for developing a more natural and therapeutic intelligent care dialogue system for the elderly, and is of great value for promoting the application of artificial intelligence in the field of smart elderly care. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0018] Figure 1 The method flowchart provided by the present invention.

[0019] Figure 2 The flowchart for scene-driven character information generation and role-playing synthetic data construction provided by this invention is shown.

[0020] Figure 3 This is a distribution map of the most common occupations in the dataset provided by this invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] This invention aims to address the difficulty in obtaining authentic therapeutic data in existing memory therapy methods due to factors such as privacy and cost. It proposes a scenario-driven method for constructing memory therapy datasets. This method, through a systematic synthetic data generation process, organically combines image descriptions, character personas, personalized stories, and multi-turn dialogues. It can construct emotionally resonant and authentic multi-turn dialogue data without relying on real caregivers and patients, and use this data to train a dialogue model with greater empathy and intelligence.

[0023] like Figure 1-2 As shown, this embodiment of the invention discloses a method for constructing a role-playing dataset based on image enhancement, including: Acquire images from multiple scenes and use a multimodal large language model to generate image text descriptions; Based on image text description, a retrieval-enhanced generation method is used to generate a character description of the main subject of the image; Based on image text descriptions and character descriptions, DeepSeek large language model is used to generate memory stories between image content and character descriptions; A dual-role-playing mechanism is adopted, which involves multiple rounds of interaction based on the recalled story, and a dialogue dataset is generated based on the interaction results. Supervised fine-tuning of a multimodal large language model is performed using the generated dialogue dataset.

[0024] Preferably, the image text description includes the appearance and emotional characteristics of the people in the image, the inferred location and context, and a comprehensive explanation of the entire image.

[0025] Specifically, the process of generating the character description includes: Initial search: Retrieve relevant character elements from publicly available character datasets based on image and text descriptions; Iterative improvement: When character elements are missing, a complete character design is formed by supplementing the search results; File integration: Integrate all character elements into a character description, which includes name, gender, age, occupation, and hobbies.

[0026] In one specific embodiment of the present invention, PersonaChat and PersonaHub are both open-source datasets. PersonaChat is available as open source on ParlAI2, and PersonaHub is available as open source on Hugging Face; both can be downloaded to obtain publicly available persona datasets.

[0027] Specifically, the DeepSeek large language model generates narrative memory stories based on image text descriptions and character descriptions; the memory stories include image scenes, emotional reactions, and character interactions, and are personalized according to different character descriptions.

[0028] Specifically, the process of generating a dialogue dataset related to the memory story includes: The DeepSeek R1 model simulates caregivers, while the DeepSeek R1-Qwen 32B model simulates the elderly. The DeepSeek R1-Qwen 32B model demonstrates differences in memory clarity and emotional responses based on the content of images in different scenarios. The DeepSeek R1 model guides the DeepSeek R1-Qwen 32B model to recall based on the content of the scene image, and adjusts the questioning strategy according to the clarity of the DeepSeek R1-Qwen 32B model's answer; By using different questioning strategies, multi-turn dialogue results are obtained, and a multi-turn dialogue dataset for Chinese image recall therapy is generated based on the dialogue results.

[0029] Specifically, the supervised fine-tuning process includes: Base model selection: A pre-trained Chinese multimodal large language model with instruction-following capabilities was selected as the base model; Supervised fine-tuning: The base model is fine-tuned using the multi-turn dialogue dataset. During training, the current image, character description, and multi-turn dialogue history are combined and input into the base model. The training objective is to enable the base model to generate standard assistant responses in the dialogue sequence. Enhanced Thinking Chains: During the training or reasoning phase, the thinking chain prompting method is used to guide the pedestal model to simulate the questioning and reasoning of caregivers, and to understand the story behind the image through step-by-step analysis and reasoning.

[0030] In a specific embodiment of the present invention, a method for constructing synthetic data based on scene-driven character information generation and role-playing is described in detail. This method creates realistic and emotionally resonant multi-turn dialogue data by systematically generating image descriptions, personalized photo personas, memory-related stories, and coherent dialogues.

[0031] To construct a photo corpus suitable for recollection therapy in the elderly, the publicly available VQGcoco-5000 image dataset was selected as input. The multimodal large language model Qwen2.5-VL Max was used to process the images, generating detailed descriptions that highlight the physical appearance and emotional characteristics of the people in the image, the inferred location and context, and a comprehensive explanation of the entire image. The resulting descriptions are stored in JSON format for easy subsequent use and expansion. The generated image description prompts and examples are shown below: Prompt words Please describe the content of the image in detail and fill in the results in the following JSON template. {"persons": [{ "name": "", / / Person's name, such as: little girl, adult woman "gender": "", / / Gender, male for female "description": "" / / Character description }], / / People appearing in the photo (excluding pedestrians and passersby). If no one is present, enter null. "location": ["", "", ""], / / Predict the location where the photo was taken "description": "" / / Detailed description of the image} Example: {"persons": [{ "name": "Little Boy", "gender": "male", "description": "A little boy wearing a dark T-shirt and blue shorts is playing on the grass outdoors. He appears to be throwing a frisbee, his expression focused, his arms outstretched, ready to catch or throw the frisbee." }], "location": ["backyard", "residential area", "outdoor"], "description": "This photo shows a little boy playing on a grassy field outdoors. In the background is a white storage shed, next to a small green boat. The boy, dressed casually, is throwing a frisbee, appearing very absorbed and happy." Based on the above image description, Retrieval Enhancement Generation (RAG) technology is used to generate a complete character profile for the subject in the photo. The specific steps are as follows: (1) Initial search: The search module retrieves relevant character elements from the publicly available character dataset based on the image description; (2) Iterative improvement: When character elements are missing, complete character settings are formed by supplementing the search. (3) File integration: The generation module integrates all elements into a detailed and coherent personal file, including name, gender, age, occupation, hobbies, etc.

[0032] The generative model is Qwen2.5-32B-Instruct, and the inference model is DeepSeek R1-Qwen 32B. If there are no obvious people in the image, the photographer is assumed to be an elderly subject. The prompts and examples for the inference model's judgment are as follows: [Judgment Principles] Did you obtain information about your profession, hobbies, and family / friend relationships from the database responses? If you haven't received a certain piece of information, and you haven't asked about it yet, please continue to ask questions. If you have already asked a question, but the database does not find the relevant information, then you can end the questioning process. Please summarize the information you have found, and summarize the target persona according to three dimensions: profession, hobbies, and family and friends relationship. Appropriately create and fabricate the target persona and fill it into the following JSON template.

[0033] Example: {"name": "Li Ming", "gender": "male", "personal_description": "Li Ming is a lively and adorable little boy who loves playing outdoors, especially throwing frisbees. He has a warm family, and his parents often accompany him to various family activities." "profession": "no fixed occupation" "hobby": "Flying disc throwing, exploring new places, family gatherings", "relationship": "Close relationship with parents, parents may work as legal assistants or social workers, warm family environment" Furthermore, using the DeepSeek R1-Qwen 32B model, narrative memory stories are generated based on photos and character profiles. The stories include scene backgrounds, emotional responses, and character interactions, and are personalized according to different character profiles to ensure realism and emotional resonance. The prompts and examples for generating memory stories are as follows: You are an imaginative writer.

[0034] Here is a photo for you. The details of the photo are as follows: {image_info} Now, please combine the photo information and {person_name}'s personal information to write the story behind this photo, focusing on {person_name}. You need to ensure that your story fully reflects {person_name}'s personal information, including their profession, hobbies, family and friends, personality traits, and life experiences.

[0035] If there are other people in the photo besides {person_name}, then you must fully consider her personal information and relationships with relatives and friends, reasonably infer the identities of these people, and incorporate them into the story.

[0036] Example: The sunlight gently bathed the grass in Li Ming's backyard. It was an ordinary weekend afternoon, a rare opportunity for Li Ming's parents to spend time with him. Seven-year-old Li Ming was at an active and energetic age, and his favorite activity was playing frisbee with his father in the backyard. Li Ming wore a dark T-shirt and blue shorts, his pockets stuffed with his favorite frisbee stickers. He squatted on the ground, clutching a red frisbee tightly, intently observing his surroundings, preparing to throw it. Behind him was a white storage shed, and next to it sat a small green boat, his father's old boat, which his father occasionally used to take Li Ming to the nearby lake. The sunlight was particularly bright today, casting long shadows on the grass. Suddenly, Li Ming stood up, his eyes sparkling with excitement… Finally, a multi-turn dialogue generation framework based on dual role-playing is proposed. In this framework, DeepSeekR1 simulates the caregiver, and DeepSeek R1-Qwen 32B simulates the elderly person. Dialogue generation follows these rules: The responses of older adults were based on personalized stories, showing differences in memory clarity (clear, possibly blurry or forgotten) and emotional responses; Caregivers cannot directly access the background story and need to guide the elderly to recall it more deeply based on image information. They should also adjust their questioning strategies according to the clarity of the elderly’s answers, while leaving room for the elderly to express their emotions in order to enhance empathy and companionship in the conversation.

[0037] The multi-turn dialogue data generated using this method encompasses diverse contextual and emotional features, effectively enhancing the model's practicality in memory therapy applications. A sample of the multi-turn dialogue data set is shown below: Example: {"role": "assistant", "content": "Grandpa Li, look how energetic you look in this photo, wearing a dark T-shirt and blue shorts! Did you throw the frisbee that far? Your posture looks so professional; do you often have frisbee games with your parents in the backyard?"} {"role": "user", "content": "Xiao Wang, this photo really brings back so many years of memories. I was little then, living with my parents in the backyard of our old house. I remember it was a summer day, the sun was shining brightly, and I was wearing this outfit, throwing frisbees with my dad in the backyard. I really often played with my parents in the backyard; they were so busy with work, so they always cherished the time they could spend with me."} {"role": "assistant", "content": "That white storage shed you mentioned looks quite interesting, with a small green boat next to it. Is it where you keep your toys at home? Was Mom cheering you and Dad on from the side and taking this picture?"} {"role": "user", "content": "That little boat behind the white storage shed... I think my dad bought it when he was young, and he'd occasionally take me to the lake to play. My mom took the photo; she didn't participate much in throwing the frisbee, but she was always watching me, a smile on her face. Those were simple days, but very warm."} {"role": "assistant", "content": "That little green boat in your backyard makes me want to row it! When Dad takes you to the lake, will he teach you how to skip stones? I bet you're playing frisbee while picking up flat stones to use next time, right?"} {"role": "user", "content": "That little green boat was bought by my dad when he was young. He would occasionally take me to the lake. When we were rowing, I would always sit at the bow, watching the ripples on the water, and I was so happy. As for skipping stones, I don't quite remember if my dad taught me, but I definitely loved collecting stones by the lake. Now, thinking back, I can almost hear my mom laughing and telling me to be careful from the shore."} Furthermore, this invention utilizes a context-driven framework for character information generation and role-playing to ultimately generate a large-scale and rich dataset of multi-turn dialogues for Chinese image-based memory therapy. This dataset contains 4983 images, each accompanied by a detailed description, a story based on personalized memories, and background information about the photo's owner, as shown in Table 2. Each image corresponds to a 7-turn dialogue, ensuring sufficient dialogue depth for therapeutic interaction. The dataset covers 1646 unique professions, showcasing a wide range of professional backgrounds and life experiences. On average, nurses' prompts are 69.84 Chinese characters long, concise and targeted; their reasoning responses average 697.96 Chinese characters, reflecting thoughtful emotional support and authentic dialogue flow; and the accompanying story narratives average 592.53 Chinese characters, providing rich context for each dialogue and enhancing its relevance and coherence.

[0038] Table 1 Dataset Table

[0039] Figure 3 The dataset showcases the nine most common professions, including unique roles such as wildlife photographer and ski instructor, demonstrating diversity that transcends traditional occupations. This diversity not only enriches the content of memory therapy but also enhances the dataset's generalization ability, enabling models trained on this dataset to better communicate and interact with older adults from diverse backgrounds.

[0040] In another specific embodiment of the present invention, the definition and training objective of the dialogue model is to use the dataset generated in the aforementioned stages to train an intelligent assistant model that can generate empathetic and supportive dialogue responses for elderly users in a memory therapy scenario.

[0041] Task Definition: The task in this embodiment can be formally described as: given an image A task or character setting prompt Current dialogue input for elderly users And the complete history of the dialogue up to the current round. The goal is to generate the next round of responses from the intelligent assistant. .in, This represents the user's input in the i-th round. Representative assistant at the The dialogue begins with the assistant's first round of responses. Initiate. Received reply. It must remain coherent with the context of the conversation and demonstrate emotional sensitivity and therapeutic relevance.

[0042] Model training and fine-tuning: To achieve the above-mentioned task objectives, this embodiment uses supervised fine-tuning technology to adapt the base model.

[0043] Base model selection: A pre-trained Chinese multimodal large language model with instruction-following capabilities will be selected as the base model. Possible models include, but are not limited to, the Qwen2.5-VL-7B-Instruct model or the MiniCPM-Llama3-V-2.5 model.

[0044] Supervised fine-tuning: Fine-tuning the base model using the generated dialogue dataset. Shaping the general model into an efficient, empathetic caregiver agent.

[0045] Enhanced thought chain techniques: During the training or inference phases, thought chain cues can be incorporated. This technique guides the model to simulate the reasoning process of a caregiver's agent, understanding the story behind the image through step-by-step analysis and reasoning. As the model acquires more contextual information in the dialogue, it will be guided to ask more specific and emotionally relevant questions, thereby achieving deeper and more meaningful memory-awakening dialogues.

[0046] Comparative Experimental Design and Results Analysis: Three sets of experiments were designed for comparison: (1) Original Model: The zero-shot method was used to directly input text instructions into the baseline model; (2) "+SFT" Model: The baseline model was supervisedly fine-tuned using only the dataset generated in the fourth step of this invention; (3) "+COT_SFT" Model: The baseline model was trained by combining the chain of thought (CoT) with supervised fine-tuning. The memory therapy dialogues generated by each set of models were scored using the large model automatic evaluation method.

[0047] Experimental results, as shown in Table 2, indicate that while the original baseline model possessed strong instruction compliance ability, it performed poorly on the recall therapy task relying solely on cue engineering, with an average score of only 54.83. After fine-tuning with "+SFT", the model showed improvements in instruction compliance, caregiver image building, and recall guidance. The model trained using the "+COT_SFT" method achieved significant improvements across all evaluation metrics, with the average score increasing to 90.27. Specifically, the Qwen2.5-VL-7B-Instruct model improved by 41.57, 32.75, and 34.98 points in instruction compliance, recall guidance, and thinking quality metrics, respectively; the MiniCPM-Llama3-V-2.5 model improved by 43.19 and 39.41 points in caregiver image building and conversational ability, respectively.

[0048] Table 2 Experimental Results

[0049] Conclusion: The experimental results fully demonstrate that the dataset constructed in this invention is of high quality and effectiveness. Using the "+COT_SFT" training method described in this invention, models with smaller parameter counts (7B and 8B levels) can achieve performance levels comparable to ultra-large-scale models used to generate data (such as DeepSeek R1) on the specific task of recall therapy (achieving approximately 95% performance in the evaluation), validating the advanced nature and practical value of the method described in this invention.

[0050] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0051] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for constructing a role-playing dataset based on image enhancement, characterized in that, include: Acquire images from multiple scenes and use a multimodal large language model to generate image text descriptions; Based on image text description, a retrieval-enhanced generation method is used to generate a character description of the main subject of the image; Based on image text descriptions and character descriptions, DeepSeek large language model is used to generate memory stories between image content and character descriptions; A dual-role-playing mechanism is adopted, which involves multiple rounds of interaction based on the recalled story, and a dialogue dataset is generated based on the interaction results. Supervised fine-tuning of a multimodal large language model is performed using the generated dialogue dataset.

2. The method for constructing a role-playing dataset based on image enhancement according to claim 1, characterized in that, The image description includes the appearance and emotional characteristics of the people in the image, the inferred location and context, and a comprehensive explanation of the entire image.

3. The method for constructing a role-playing dataset based on image enhancement according to claim 2, characterized in that, The specific process of generating the character description includes: Initial search: Retrieve relevant character elements from publicly available character datasets based on image and text descriptions; Iterative improvement: When character elements are missing, a complete character design is formed by supplementing the search results; File integration: Integrate all character elements into a character description, which includes name, gender, age, occupation, and hobbies.

4. The method for constructing a role-playing dataset based on image enhancement according to claim 3, characterized in that, The DeepSeek large language model generates narrative memory stories based on image text descriptions and character descriptions; the memory stories include image scenes, emotional reactions, and character interactions, and are personalized according to different character descriptions.

5. The method for constructing a role-playing dataset based on image enhancement according to claim 4, characterized in that, The process of generating a dialogue dataset related to the memory story includes: The DeepSeek R1 model simulates caregivers, while the DeepSeek R1-Qwen 32B model simulates the elderly. The DeepSeek R1-Qwen 32B model demonstrates differences in memory clarity and emotional responses based on the content of images in different scenarios. The DeepSeek R1 model guides the DeepSeek R1-Qwen 32B model to recall based on the content of the scene image, and adjusts the questioning strategy according to the clarity of the DeepSeek R1-Qwen 32B model's answer; By using different questioning strategies, multi-turn dialogue results are obtained, and a multi-turn dialogue dataset for Chinese image recall therapy is generated based on the dialogue results.

6. The method for constructing a role-playing dataset based on image enhancement according to claim 5, characterized in that, The supervised fine-tuning process includes: Base model selection: A pre-trained Chinese multimodal large language model with instruction-following capabilities was selected as the base model; Supervised fine-tuning: The pedestal model is fine-tuned using the multi-turn dialogue dataset. During training, the current image, character description, and multi-turn dialogue history are combined and input into the pedestal model. The training objective is to enable the pedestal model to generate standard assistant responses in the dialogue sequence. Enhanced Thinking Chains: During the training or reasoning phase, the thinking chain prompting method is used to guide the pedestal model to simulate the questioning and reasoning of caregivers, and to understand the story behind the image through step-by-step analysis and reasoning.

Images