A Remote Interaction Method for Elderly Emotional Comfort Robots Based on Scene Data Collection

Abstract

The application discloses a kind of old person emotional comfort robot remote interaction methods based on scene collection, it is related to human-computer interaction technical field.The method comprises the following steps: obtaining memory scene data and real-time scene data, form scene data set;Collect initial action information and initial voice information of old user and parse, construct behavior feature vector and emotional state vector;Decision function is constructed, generates scene scheduling decision result and interaction rhythm control parameter, controls scene collection terminal to carry out mobile path adjustment and collection angle adjustment, and the execution delay and prompt frequency of control instruction are adjusted;Virtual scene is generated, forms immersive presentation process;Collect feedback behavior data, generate emotional feedback content, and carry out emotional response and guide to old user.The application realizes adaptive immersive emotional interaction for old user, significantly improves emotional comfort effect and interaction experience quality.

Description

Technical Field

[0001] This application relates to the field of human-computer interaction technology, and in particular to a remote interaction method for an emotional comfort robot for the elderly based on scene acquisition. Background Technology

[0002] Due to declining physical function, limited mobility, and shrinking social circles, many elderly people find it difficult to go out or visit places of emotional significance, leading to feelings of loneliness and emotional deprivation. To alleviate these issues, existing technologies have gradually developed remote interaction and emotional support solutions based on video calls, virtual reality, and companion robots. Meanwhile, with the development of multimodal perception, intelligent decision-making, and human-computer interaction technologies, how to integrate multi-source data for elderly emotional support and build more intelligent, personalized, and immersive remote interaction systems has become an important direction for current technological development.

[0003] In related technologies, existing systems mostly use fixed or general scene content, lacking the ability to model the historical experiences and emotional memories of elderly users. This results in scenes lacking personalization and emotional connection, making it difficult to evoke genuine emotional resonance. Secondly, the functional modules are usually independent of each other, lacking a unified decision-making mechanism, making it difficult to achieve coordinated optimization between emotional state, scene selection, and interaction control. In addition, existing systems generally lack the ability to continuously perceive and dynamically adjust to real-time user feedback. The interaction process is one-way or has weak feedback, making it difficult to form a continuous and effective emotional interaction loop, thus requiring improvement. Summary of the Invention

[0004] The purpose of this invention is to provide a remote interaction method for an emotional comfort robot for the elderly based on scene data collection, so as to solve the problems mentioned in the background art.

[0005] This application provides a remote interaction method for an emotional comfort robot for the elderly based on scene data acquisition, which adopts the following technical solution:

[0006] Historical information data of elderly users is obtained to construct a memory scene model, generate memory scene data, collect real-time scene data of the target scene through a scene acquisition terminal, and perform unified encoding processing on the memory scene data and the real-time scene data to form a scene data set;

[0007] Collect and parse the initial action and voice information of elderly users, extract operation response time parameters and interaction frequency parameters, and construct behavioral feature vectors.

[0008] The initial voice information and behavioral feature vector are fused and analyzed to extract emotion category parameters, emotion intensity parameters, and emotion change trend parameters, and to construct an emotional state vector.

[0009] Based on the emotional state vector, scene data set, and behavioral feature vector, a decision function is constructed to generate scene scheduling decision results and interaction rhythm control parameters.

[0010] Based on the scene scheduling decision results and interaction rhythm control parameters, the scene acquisition terminal is controlled to adjust its movement path and acquisition angle, and the execution delay and prompt frequency of the control commands are adjusted to generate target scene data.

[0011] The target scene data is fused and processed to generate a virtual scene, which is then output through a presentation device to form an immersive presentation process.

[0012] During the immersive presentation process, feedback action information and feedback voice information of elderly users are collected to obtain feedback behavior data. The emotional state vector is updated, and emotional feedback content is generated based on the updated emotional state vector. The emotional feedback content is output through the emotional comfort robot to provide emotional response and guidance to the elderly users.

[0013] Preferably, the steps of acquiring historical information data of elderly users to construct a memory scene model, generating memory scene data, collecting real-time scene data of the target scene through a scene acquisition terminal, and uniformly encoding the memory scene data and the real-time scene data to form a scene data set are as follows:

[0014] Obtain a historical information dataset of elderly users, which includes historical residential address data, social relationship-related location data, and location identification data corresponding to historical image data;

[0015] The historical information dataset is subjected to structured parsing to extract scene elements, which include spatial location elements, environmental object elements, and temporal correlation elements.

[0016] A memory scene model is constructed based on the scene elements, a three-dimensional spatial structure is generated based on the spatial location elements, entity objects in the scene are supplemented based on the environmental object elements, and the scene state is calibrated with time features based on the time association elements.

[0017] The memory scene model is subjected to consistency verification. When there are missing scene elements, the missing parts are deduced and completed to generate memory scene data.

[0018] The target scene is collected by the scene acquisition terminal to obtain real-time scene data, which includes panoramic image data, environmental audio data and spatial structure data.

[0019] The real-time scene data is preprocessed, including data denoising, time synchronization, and unified spatial coordinate processing.

[0020] The memory scene data and real-time scene data are uniformly encoded to construct a unified data representation structure, resulting in a scene data set.

[0021] Preferably, the steps of collecting and parsing the initial action and voice information of elderly users, extracting operation response time parameters and interaction frequency parameters, and constructing behavioral feature vectors are as follows:

[0022] The system collects initial motion information and initial voice information of elderly users. The initial motion information includes head posture change information, limb movement information and position change information. The initial voice information includes voice signal data and voice interaction command data.

[0023] The initial motion information is preprocessed, including data denoising, time alignment, and motion segmentation, to obtain standardized motion data;

[0024] The initial speech information is preprocessed, including speech denoising, speech framing and speech recognition processing, to obtain speech parsing data;

[0025] Based on the standardized action data and voice parsing data, the sequence of user interaction events is determined, and the trigger time and response time of each interaction event are recorded.

[0026] Based on the trigger time and response time of each interactive event in the interactive event sequence, the operation response time parameter is calculated to characterize the reaction time of elderly users to system feedback.

[0027] The interaction frequency parameter is calculated based on the number of interaction events occurring per unit time in the interaction event sequence, which is used to characterize the interaction activity level of elderly users.

[0028] The operation response time parameter and interaction frequency parameter are normalized, and a behavior feature vector is constructed.

[0029] Preferably, the step of fusing and analyzing the initial voice information and behavioral feature vectors to extract emotion category parameters, emotion intensity parameters, and emotion change trend parameters, and constructing an emotional state vector, specifically includes:

[0030] Speech features are extracted from the initial speech information to obtain speech feature parameters, which include fundamental frequency parameters, speech rate parameters, volume parameters, and pitch variation parameters.

[0031] Based on the behavioral feature vector, behavioral feature parameters are extracted, including operation response time parameters and interaction frequency parameters.

[0032] The speech feature parameters and behavioral feature parameters are time-aligned to construct a multimodal feature sequence. The multimodal feature sequence is then processed using a preset emotion recognition model to obtain the emotion discrimination result.

[0033] Based on the emotion discrimination results, an emotion category parameter is determined to characterize the current emotion type of elderly users;

[0034] Based on the aforementioned speech feature parameters and behavioral feature parameters, an emotion intensity parameter is calculated to characterize the magnitude of emotion changes.

[0035] Based on the changes in the emotion intensity parameter within a continuous time window, an emotion change trend parameter is calculated to characterize the direction and rate of emotion change.

[0036] The emotion category parameter, emotion intensity parameter, and emotion change trend parameter are combined to construct an emotion state vector.

[0037] Preferably, the step of constructing a decision function and generating scene scheduling decision results and interaction rhythm control parameters based on the emotional state vector, scene data set, and behavioral feature vector specifically includes:

[0038] The scene data set is labeled with features to generate scene feature labels, which include scene familiarity parameters, emotional relief parameters, social attribute parameters, and environmental stimulus intensity parameters.

[0039] A scene matching model is constructed based on the emotional state vector and scene feature labels, and the matching score of each candidate scene is calculated.

[0040] The candidate scenarios are sorted according to their matching scores, and the scenarios whose matching scores meet the preset conditions are selected as the target scenarios, generating scenario scheduling decision results.

[0041] A rhythm control model is constructed based on the behavioral feature vector, and interactive rhythm control parameters are calculated. The interactive rhythm control parameters include control command execution delay parameters and interactive prompt frequency parameters.

[0042] Preferably, the steps of controlling the scene acquisition terminal to adjust its movement path and acquisition angle based on the scene scheduling decision results and interaction rhythm control parameters, and adjusting the execution delay and prompt frequency of control commands to generate target scene data, are as follows:

[0043] Extract and parse the scene scheduling decision results to obtain the target scene identification information and the corresponding path planning parameters and view control parameters;

[0044] Based on the path planning parameters, a movement path control sequence for the scene acquisition terminal is generated, and the scene acquisition terminal is controlled to move according to the movement path control sequence.

[0045] Based on the aforementioned viewpoint control parameters, a data acquisition angle control command is generated, and the scene acquisition terminal is controlled to adjust its acquisition direction and viewpoint according to the data acquisition angle control command.

[0046] During the movement of the scene acquisition terminal, panoramic image data, environmental audio data, and spatial structure data of the target scene are continuously acquired to form a real-time scene data stream;

[0047] Based on the interaction rhythm control parameters, control command execution delay parameters and interaction prompt frequency parameters are extracted. The execution time of the movement path control sequence and acquisition angle control command is adjusted according to the control command execution delay parameters to match the operation response characteristics of elderly users.

[0048] Based on the interactive prompt frequency parameter, a prompt control command is generated, and interactive guidance information is output at a preset frequency;

[0049] The real-time scene data stream is synchronized and integrated to generate target scene data.

[0050] Preferably, the steps of fusing the target scene data to generate a virtual scene, and outputting the virtual scene through a presentation device to form an immersive presentation process, specifically include:

[0051] Based on the target scene data, extract the panoramic image data, environmental audio data, and spatial structure data;

[0052] The panoramic image data is processed, including image denoising, distortion correction and multi-view image stitching, to generate a panoramic image frame sequence;

[0053] The environmental audio data is processed, including noise suppression, sound source localization, and spatial sound effect reconstruction, to generate spatial audio data.

[0054] The spatial structure data is processed by spatial modeling to construct a three-dimensional spatial model of the scene, and the panoramic image frame sequence is spatially mapped.

[0055] The panoramic image frame sequence, spatial audio data, and scene 3D spatial model are subjected to time-series synchronization processing to generate unified multimodal fusion scene data;

[0056] Virtual scene rendering is performed based on the multimodal fused scene data to generate a virtual scene;

[0057] The virtual scene is output through a presentation device, allowing elderly users to receive visual and auditory information corresponding to the virtual scene, thus creating an immersive presentation process.

[0058] Preferably, during the immersive presentation process, the following steps are taken: collecting feedback action information and feedback voice information from elderly users to obtain feedback behavior data; updating the emotional state vector; generating emotional feedback content based on the updated emotional state vector; and outputting the emotional feedback content through an emotional comfort robot to provide emotional response and guidance to elderly users.

[0059] During the immersive presentation process, feedback action information and feedback voice information of elderly users are collected and preprocessed to obtain real-time action processing data and real-time voice processing data;

[0060] Based on the real-time motion processing data and real-time speech processing data, a feedback behavior sequence is constructed, and feedback behavior feature parameters are extracted. The feedback behavior feature parameters include motion change amplitude parameters, speech emotion feature parameters, and interaction response change parameters to obtain feedback behavior data.

[0061] The original emotional state vector is corrected based on the feedback behavior data, the emotional intensity parameter is adjusted according to the feedback behavior feature parameters, and the emotional change trend parameter is updated according to the feedback behavior changes within a continuous time window to obtain the updated emotional state vector.

[0062] The updated emotional state vector is input into a preset emotional response model to generate emotional feedback content, which includes voice response content, interactive guidance content, and scene adjustment suggestions.

[0063] Based on the emotional feedback content, corresponding output control commands are generated. The emotional comfort robot executes the output control commands to output voice information and interactive prompts to the elderly user, providing emotional response and guidance.

[0064] In summary, this application includes at least one of the following beneficial technical effects:

[0065] 1. By constructing a scene data set that integrates memory scene data and real-time scene data, a unified expression of real-world scenes and historical emotional scenes is achieved. This allows the system to overlay personalized memory elements on top of the real environment, significantly enhancing the emotional relevance and individual adaptability of the immersive experience. By analyzing the initial action and voice information of elderly users, operation response time parameters and interaction frequency parameters are extracted to construct behavioral feature vectors. This enables quantitative modeling of user interaction capabilities and behavioral habits, allowing the system to adaptively adjust the interaction rhythm based on the elderly user's reaction speed and operation characteristics, thereby reducing the difficulty of use and improving the age-friendliness of the interaction. By fusing initial voice information and behavioral feature vectors, an emotional state vector is constructed, enabling multi-dimensional identification and quantitative expression of elderly users' emotional categories, intensity, and trends. This improves the accuracy and stability of emotion recognition, giving the system a stronger emotional perception capability. By constructing a unified decision function, the emotional state vector, scene data set, and behavioral feature vector are fused and processed to generate scene scheduling decision results and interaction rhythm control parameters. This achieves collaborative decision-making between scene selection and interaction control, avoiding the fragmentation between functional modules and improving the overall coordination and intelligence level of the system. Based on scene scheduling decision results and interaction rhythm control parameters, the movement path and acquisition angle of the scene acquisition terminal are controlled, and the execution delay and prompt frequency of control commands are adjusted. This effectively transforms decision results into actual execution, ensuring the system's accuracy in scene acquisition while also considering the operational adaptability of elderly users, thereby improving the stability and usability of the execution process. By fusing and processing target scene data to generate a virtual scene, and outputting it through a presentation device, an immersive presentation process is constructed. This allows elderly users to obtain a visual and auditory experience close to a real environment, enhancing their sense of immersion and participation, thus effectively improving the emotional comfort effect. During the immersive presentation process, by collecting feedback action and voice information from elderly users, the emotional state vector is dynamically updated, and emotional feedback content is generated based on the updated emotional state vector, forming a closed-loop emotional interaction mechanism. This allows the system to continuously perceive changes in user emotions and respond adaptively, realizing a shift from static interaction to dynamic adjustment, significantly improving the continuity and intelligence level of emotional companionship.

[0066] 2. By coordinating the movement path and acquisition angle of the scene acquisition terminal based on scene scheduling decision results and interaction rhythm control parameters, and dynamically adjusting the execution delay and prompt frequency of control commands, efficient mapping of decision results to the actual execution process is achieved. By parsing the scene scheduling decision results, target scene identification information, path planning parameters, and viewpoint control parameters are obtained, giving the scene acquisition process clear target orientation and execution accuracy, thereby improving the targeting and completeness of target scene acquisition. Simultaneously, by generating a movement path control sequence and controlling the scene acquisition terminal to move according to the sequence, and adjusting the acquisition direction and viewpoint based on viewpoint control parameters, comprehensive coverage of the target scene's spatial range and key viewpoints is achieved, improving the effectiveness and accuracy of scene data acquisition. During the acquisition process, by continuously acquiring panoramic image data, environmental audio data, and spatial structure data, a real-time scene data stream is formed, ensuring the continuity and multimodal integrity of scene information. By introducing interaction rhythm control parameters to adjust the execution delay of control commands, the system's response speed is matched to the operational responsiveness of elderly users, avoiding discomfort caused by responses that are too fast or too slow. Simultaneously, by controlling the frequency of interactive prompts, the system provides interactive guidance information at an appropriate frequency, ensuring the effectiveness of guidance while avoiding information overload, thereby improving the system's age-friendly interactive experience. Through time synchronization and data integration processing of real-time scene data streams, target scene data with a unified structure and consistent timing is generated, providing high-quality input data support for subsequent immersive presentations. This not only improves the accuracy and completeness of scene acquisition but also significantly enhances the system's adaptability and stability during execution, effectively improving the overall interactive experience and emotional comfort.

[0067] 3. During the immersive experience, feedback action and voice information from elderly users are continuously collected, preprocessed, and fused for analysis to obtain feedback behavior data, enabling dynamic perception of the user's real-time interaction status. By introducing feedback behavior sequences and multi-dimensional feedback behavior feature parameters, the system can comprehensively depict the behavioral changes and emotional expressions of elderly users during the immersive experience, improving the accuracy and sensitivity of recognizing the user's immediate state. Based on the feedback behavior data, the original emotional state vector is dynamically corrected. By adjusting the emotional intensity parameter and updating the emotional change trend parameter, the emotional state is transformed from a static result into a continuous evolution process, significantly improving the real-time performance and dynamic adaptability of emotional modeling. This allows the system to adjust promptly according to emotional changes during the user experience, avoiding response bias caused by emotional state lag. By inputting the updated emotional state vector into a preset emotional response model, emotional feedback content including voice replies, interactive guidance, and scene adjustment suggestions is generated. This allows the system to output differentiated response strategies for different emotional states, thereby enhancing the relevance and emotional fit of the interaction. Furthermore, by transforming emotional feedback into output control commands, which are then executed by the emotional comfort robot, the system enables voice output and interactive prompts for elderly users. This gives the system a complete capability from "emotional perception" to "emotional response," which not only enhances the system's real-time adaptability to changes in the emotions of elderly users but also significantly improves the continuity, intelligence, and humanized experience of emotional interaction, thereby effectively improving the emotional comfort effect and user satisfaction. Attached Figure Description

[0068] Figure 1 This is a schematic diagram illustrating the specific steps of an embodiment of the remote interaction method for an emotional comfort robot for the elderly based on scene acquisition according to the present invention. Detailed Implementation

[0069] The following examples and... Figure 1 The present invention will be described in further detail, but the embodiments of the present invention are not limited thereto.

[0070] This invention discloses a remote interaction method for an emotional comfort robot for the elderly based on scene data acquisition, which specifically includes the following steps:

[0071] Step S1: Obtain historical information data of elderly users to construct a memory scene model, generate memory scene data, collect real-time scene data of the target scene through a scene acquisition terminal, and perform unified encoding processing on the memory scene data and the real-time scene data to form a scene data set;

[0072] Step S2: Collect and parse the initial action information and initial voice information of elderly users, extract operation response time parameters and interaction frequency parameters, and construct behavioral feature vectors;

[0073] Step S3: Perform fusion analysis on the initial voice information and behavioral feature vector to extract emotion category parameters, emotion intensity parameters, and emotion change trend parameters, and construct an emotional state vector;

[0074] Step S4: Based on the emotional state vector, scene data set, and behavioral feature vector, construct a decision function to generate scene scheduling decision results and interaction rhythm control parameters;

[0075] Step S5: Based on the scene scheduling decision results and interaction rhythm control parameters, control the scene acquisition terminal to adjust the movement path and acquisition angle, and adjust the execution delay and prompt frequency of the control commands to generate target scene data.

[0076] Step S6: The target scene data is fused to generate a virtual scene, and the virtual scene is output through a presentation device to form an immersive presentation process;

[0077] Step S7: During the immersive presentation process, feedback action information and feedback voice information of elderly users are collected to obtain feedback behavior data. The emotional state vector is updated, and emotional feedback content is generated based on the updated emotional state vector. The emotional feedback content is output through the emotional comfort robot to provide emotional response and guidance to the elderly users.

[0078] In practical applications, a memory scene model is constructed by acquiring historical information data from elderly users. This model is then combined with real-time scene data obtained from scene acquisition terminals to form a unified scene data set. This achieves dual-source fusion modeling of real-world and memory scenes, not only recreating the real environment but also supplementing it with emotionally valuable historical scenes. This gives the system personalized and emotionally resonant capabilities, significantly enhancing the realism and emotional connection of the immersive experience. By collecting initial action and voice information from elderly users and extracting operation response time and interaction frequency parameters, a behavioral feature vector is constructed. Introducing parameters such as operation response time and interaction frequency objectively reflects the reaction speed and interactive activity level of elderly users, providing a basis for subsequent interaction rhythm control. This allows the system to adapt to the behavioral characteristics of elderly users, reducing the difficulty of use and improving interactivity. By fusing and analyzing initial voice information and behavioral feature vectors, emotion category, emotion intensity, and emotion change trend parameters are extracted to construct an emotional state vector. By fusing multi-source information, the accuracy and stability of emotion recognition are improved, enabling the system to more comprehensively understand the psychological state of elderly users and achieve a shift from passive response to active perception. A decision function is constructed based on emotional state vectors, scene data sets, and behavioral feature vectors to generate scene scheduling decision results and interaction rhythm control parameters. Through this unified decision function, emotional states, environmental resources, and user behavior characteristics are comprehensively calculated to achieve collaborative decision-making on which scene to select and how to interact, thus avoiding the fragmentation caused by independent operation of each module and giving the system integrity and intelligence. Based on the scene scheduling decision results and interaction rhythm control parameters, the movement path, acquisition angle, and control command execution rhythm of the scene acquisition terminal are adjusted. Through path and perspective adjustments, accurate acquisition of the target scene is achieved, and rhythm control ensures that the interaction process is suitable for elderly users' operational abilities, guaranteeing that the system execution process is both accurate and age-friendly. The target scene data is fused and processed to generate a virtual scene, which is then output through a presentation device, creating an immersive presentation process. Through the fusion of images, audio, and spatial information, elderly users can obtain a sensory experience close to a real environment, enhancing their sense of immersion and participation, creating a basic environment for emotional interaction, and improving the overall experience quality. During the immersive presentation process, feedback action information and feedback voice information of elderly users are collected, the emotional state vector is updated, and a dynamic closed-loop adjustment mechanism of the system is constructed. This enables the system to continuously perceive changes in user emotions and adjust response strategies accordingly, thereby achieving a continuous and adaptive emotional interaction process. This upgrades the system from static response to dynamic adjustment, significantly improving the emotional comfort effect and user satisfaction.

[0079] The steps of acquiring historical information data of elderly users to construct a memory scene model, generating memory scene data, collecting real-time scene data of the target scene through a scene acquisition terminal, and uniformly encoding the memory scene data and the real-time scene data to form a scene data set are as follows:

[0080] Step S11: Obtain the historical information dataset of elderly users. The historical information dataset includes historical residential address data, social relationship associated location data, and location identification data corresponding to historical image data.

[0081] Step S12: Perform structured parsing on the historical information dataset to extract scene elements, which include spatial location elements, environmental object elements, and time-related elements.

[0082] Step S13: Construct a memory scene model based on the scene elements, generate a three-dimensional spatial structure based on the spatial location elements, supplement the entity objects in the scene based on the environmental object elements, and perform time feature calibration on the scene state based on the time association elements.

[0083] Step S14: Perform consistency verification on the memory scene model. When there are missing scene elements, perform deduction and completion on the missing parts to generate memory scene data.

[0084] Step S15: Collect the target scene through the scene acquisition terminal to obtain real-time scene data, which includes panoramic image data, environmental audio data and spatial structure data.

[0085] Step S16: Preprocess the real-time scene data, including data denoising, time synchronization, and unified spatial coordinate processing.

[0086] Step S17: Perform unified encoding processing on the memory scene data and the real-time scene data to construct a unified data representation structure and obtain a scene data set.

[0087] In practical applications, by structurally analyzing historical information datasets and extracting spatial location elements, environmental object elements, and temporal correlation elements, the originally scattered and unstructured historical data is transformed into computable scene elements, thus providing a foundation for the construction of memory scene models. Based on this, through the generation of 3D spatial structures, the supplementation of entity objects, and the calibration of temporal features, the system achieves digital reconstruction of important past life scenes for elderly users, enabling it to express individual memories. Simultaneously, through consistency verification and missing information deduction and completion, the completeness and rationality of memory scene data are further improved, enhancing the continuity and credibility of virtual scenes. Real-time acquisition of target scenes is achieved through scene acquisition terminals, and the acquired panoramic image data, environmental audio data, and spatial structure data are preprocessed to achieve high-quality data acquisition and standardized processing of the real environment, providing a reliable data foundation for subsequent fusion. By uniformly encoding memory scene data and real-time scene data, a unified data expression structure is constructed, allowing data from different sources and of different types to be organized and represented within the same framework, thus providing a consistent data input interface for subsequent sentiment analysis, decision generation, and immersive presentation.

[0088] The steps for collecting and parsing initial action and voice information of elderly users, extracting operation response time and interaction frequency parameters, and constructing behavioral feature vectors are as follows:

[0089] Step S21: Collect the initial movement information and initial voice information of the elderly user. The initial movement information includes head posture change information, limb movement information and position change information. The initial voice information includes voice signal data and voice interaction command data.

[0090] Step S22: Preprocess the initial motion information, including data denoising, time alignment and motion segmentation, to obtain standardized motion data;

[0091] Step S23: Preprocess the initial speech information, including speech denoising, speech framing and speech recognition processing, to obtain speech parsing data;

[0092] Step S24: Based on the standardized action data and voice parsing data, determine the user interaction event sequence and record the trigger time and response time of each interaction event;

[0093] Step S25: Calculate the operation response time parameter based on the trigger time and response time of each interactive event in the interactive event sequence, which is used to characterize the reaction time of elderly users to system feedback;

[0094] Step S26: Calculate the interaction frequency parameter based on the number of interaction events occurring per unit time in the interaction event sequence, which is used to characterize the interaction activity level of elderly users.

[0095] Step S27: Normalize the operation response time parameter and interaction frequency parameter, and construct a behavior feature vector.

[0096] In practical applications, by collecting and preprocessing initial action and speech information, the raw multi-source sensory data is transformed into structured, standardized action and speech parsing data, thereby improving data usability and analytical accuracy. Based on this, by constructing user interaction event sequences and recording trigger and response times, the system can characterize the temporal behavioral features of users during interactions. Calculating the operation response time parameter accurately reflects the elderly user's reaction speed to system feedback, thus characterizing their operational ability and cognitive burden. Calculating the interaction frequency parameter quantifies the user's interactive activity level per unit time, reflecting their participation and usage habits. By normalizing these parameters and constructing behavioral feature vectors, data of different dimensions can be expressed on a unified scale, achieving an effective transformation from raw sensory data to high-level behavioral characteristics. This enables the system to accurately understand the interaction characteristics of elderly users, providing crucial information for subsequent emotional state recognition, interaction rhythm control, and intelligent decision-making, thereby enhancing the system's age-friendliness and interactive intelligence.

[0097] The steps for fusing and analyzing the initial speech information and behavioral feature vectors to extract emotion category parameters, emotion intensity parameters, and emotion change trend parameters, and constructing an emotional state vector, are as follows:

[0098] Step S31: Extract speech features from the initial speech information to obtain speech feature parameters, including fundamental frequency parameters, speech rate parameters, volume parameters, and pitch variation parameters.

[0099] Step S32: Based on the behavior feature vector, extract behavior feature parameters, including operation response time parameters and interaction frequency parameters;

[0100] Step S33: The speech feature parameters and behavioral feature parameters are time-aligned to construct a multimodal feature sequence. The multimodal feature sequence is then processed using a preset emotion recognition model to obtain the emotion discrimination result.

[0101] Step S34: Determine the emotion category parameter based on the emotion discrimination result, which is used to characterize the current emotion type of the elderly user;

[0102] Step S35: Calculate the emotion intensity parameter based on the speech feature parameters and behavioral feature parameters to characterize the magnitude of emotion changes;

[0103] Step S36: Based on the changes in the emotion intensity parameter within a continuous time window, calculate the emotion change trend parameter to characterize the direction and rate of emotion change.

[0104] Step S37: Combine the emotion category parameter, emotion intensity parameter, and emotion change trend parameter to construct an emotion state vector.

[0105] In practical applications, by extracting speech features from initial speech information, the system obtains speech feature parameters such as fundamental frequency, speech rate, volume, and intonation changes, enabling it to capture the user's emotional expression characteristics at the acoustic level. Simultaneously, based on behavioral feature vectors, it extracts operation response time parameters and interaction frequency parameters, incorporating the user's behavioral response characteristics into the analysis scope, thus overcoming the limitations of single speech analysis in emotion recognition. By performing time alignment processing on speech and behavioral feature parameters and constructing a multimodal feature sequence, a unified expression of different modal information in the time dimension is achieved. A pre-set emotion recognition model processes the multimodal feature sequence to obtain emotion discrimination results, and further extracts emotion category parameters, enabling the system to identify the current emotion type of elderly users. By combining speech and behavioral features to calculate emotion intensity parameters, a quantitative expression of the magnitude of emotion changes is achieved. By analyzing the changes in emotion intensity within a continuous time window, emotion change trend parameters are calculated, reflecting the direction and rate of emotion evolution, thus giving emotional states a dynamic descriptive capability. By combining emotion category parameters, emotion intensity parameters, and emotion change trend parameters, an emotion state vector is constructed, transforming the originally complex and variable emotional information into a structured and computable unified representation. This not only improves the accuracy and robustness of emotion recognition but also provides a core basis for subsequent scene scheduling decisions and emotion feedback generation, thereby enhancing the system's emotion understanding ability and intelligent interaction level.

[0106] The steps for constructing a decision function and generating scene scheduling decision results and interaction rhythm control parameters based on the aforementioned emotional state vector, scene data set, and behavioral feature vector are as follows:

[0107] Step S41: Perform feature annotation on the scene data set to generate scene feature labels. The scene feature labels include scene familiarity parameters, emotional relief parameters, social attribute parameters, and environmental stimulus intensity parameters.

[0108] Step S42: Construct a scene matching model based on the emotional state vector and scene feature labels, and calculate the matching score for each candidate scene;

[0109] Step S43: Sort the candidate scenarios according to their matching scores, select the scenarios whose matching scores meet the preset conditions as the target scenarios, and generate the scenario scheduling decision results.

[0110] Step S44: Construct a rhythm control model based on the behavior feature vector, and calculate the interaction rhythm control parameters, which include control command execution delay parameters and interaction prompt frequency parameters.

[0111] In practical applications, feature annotation is performed on scene datasets to generate scene feature labels containing parameters such as scene familiarity, emotional relief, social attributes, and environmental stimulus intensity. This imbues the original scene data with calculable semantic attributes, providing a quantitative basis for scene selection. Based on this, a scene matching model is constructed using emotional state vectors. Matching scores are calculated for each candidate scene, establishing a mapping between emotional state and scene features. This allows the system to select the target scene most suitable for the current emotional state, improving the targeting and emotional fit of scene scheduling. By ranking the matching scores and filtering target scenes based on preset conditions, the system achieves optimal decision output from multiple candidate scenes, ensuring the rationality and stability of the scene scheduling results. A rhythm control model is constructed based on behavioral feature vectors. By calculating the delay parameter of control command execution and the frequency parameter of interactive prompts, the system can dynamically adjust the interaction rhythm according to the elderly user's operation response ability and interaction activity level. This avoids the maladaptation problem caused by the interaction process being too fast or too slow, and realizes the collaborative decision-making of scene selection and interaction control. This not only improves the overall intelligence level of the system operation, but also significantly enhances the system's comprehensive capabilities in terms of emotional adaptability and age-friendly interaction.

[0112] Based on the scene scheduling decision results and interaction rhythm control parameters, the steps for controlling the scene acquisition terminal to adjust its movement path and acquisition angle, and adjusting the execution delay and prompt frequency of control commands to generate target scene data are as follows:

[0113] Step S51: Extract and parse the scene scheduling decision results to obtain the target scene identification information and the corresponding path planning parameters and view control parameters;

[0114] Step S52: Generate a movement path control sequence for the scene acquisition terminal based on the path planning parameters, and control the scene acquisition terminal to move according to the movement path control sequence;

[0115] Step S53: Generate a collection angle control command based on the viewpoint control parameters, and control the scene collection terminal to adjust the collection direction and viewpoint according to the collection angle control command;

[0116] Step S54: During the movement of the scene acquisition terminal, panoramic image data, environmental audio data and spatial structure data of the target scene are continuously acquired to form a real-time scene data stream;

[0117] Step S55: Based on the interaction rhythm control parameters, extract the control command execution delay parameters and interaction prompt frequency parameters, and adjust the execution time of the movement path control sequence and the acquisition angle control command according to the control command execution delay parameters to match the operation response characteristics of elderly users.

[0118] Step S56: Generate a prompt control command based on the interactive prompt frequency parameter, and output interactive guidance information at a preset frequency;

[0119] Step S57: Perform time synchronization and data integration processing on the real-time scene data stream to generate target scene data.

[0120] In practical applications, by analyzing the scene scheduling decision results, target scene identification information, path planning parameters, and viewpoint control parameters are obtained. This enables the system to clearly define the execution objectives of "where to collect data" and "how to collect data," providing a precise basis for subsequent control. By generating a movement path control sequence and driving the scene acquisition terminal to move sequentially, the system achieves comprehensive acquisition of the target scene's spatial range. Simultaneously, the terminal's viewpoint is dynamically adjusted via acquisition angle control commands, ensuring that the acquired content better aligns with the user's focus, thus improving the targeting and completeness of scene acquisition. During the acquisition process, panoramic image data, environmental audio data, and spatial structure data are continuously acquired to form a real-time scene data stream, enabling multimodal dynamic perception of the target scene. Furthermore, an interaction rhythm control parameter is introduced to adjust the execution delay of control commands, ensuring that the system's response speed matches the operational reaction ability of elderly users, avoiding discomfort caused by overly fast or slow responses, thereby improving the system's age-friendly experience. In addition, the interaction prompt frequency parameter controls the output rhythm of prompt information, allowing the system to provide guidance at an appropriate frequency, avoiding information overload while ensuring necessary interactive assistance. By synchronizing and integrating real-time scene data streams, target scene data with a unified structure and consistent timing is generated, providing high-quality input for subsequent immersive presentation. This achieves a key transformation from "decision output" to "physical execution and data generation," ensuring not only the accuracy and completeness of scene acquisition but also significantly improving the comfort and adaptability of the interaction process through a rhythm control mechanism.

[0121] The steps of fusing the target scene data to generate a virtual scene, and then outputting the virtual scene through a presentation device to form an immersive presentation process are as follows:

[0122] Step S61: Based on the target scene data, extract the panoramic image data, environmental audio data, and spatial structure data;

[0123] Step S62: Perform image processing on the panoramic image data, including image denoising, distortion correction and multi-view image stitching, to generate a panoramic image frame sequence.

[0124] Step S63: Perform audio processing on the environmental audio data, including noise suppression, sound source localization, and spatial sound effect reconstruction, to generate spatial audio data;

[0125] Step S64: Perform spatial modeling processing on the spatial structure data to construct a three-dimensional spatial model of the scene, and perform spatial mapping on the panoramic image frame sequence;

[0126] Step S65: Perform time-series synchronization processing on the panoramic image frame sequence, spatial audio data, and scene 3D spatial model to generate unified multimodal fusion scene data;

[0127] Step S66: Perform virtual scene rendering processing based on the multimodal fusion scene data to generate a virtual scene;

[0128] Step S67: The virtual scene is output through a presentation device, so that elderly users can receive the visual and auditory information corresponding to the virtual scene, forming an immersive presentation process.

[0129] In practical applications, panoramic image data, environmental audio data, and spatial structure data are extracted from target scene data to achieve comprehensive acquisition of visual, auditory, and spatial information. By denoising, correcting distortion, and stitching multi-viewpoints on the panoramic image data, a continuous and stable sequence of panoramic image frames is generated, thereby improving the clarity and completeness of the visual presentation. Noise suppression, sound source localization, and spatial sound effect reconstruction of the environmental audio data imbue the audio information with spatial directionality and layering, enhancing auditory realism. Furthermore, by modeling the spatial structure data and constructing a 3D spatial model of the scene, the scene acquires spatial geometric information, and the panoramic image frame sequence is mapped into 3D space, achieving a unification of visual information and spatial structure. By performing temporal synchronization processing on the panoramic image frame sequence, spatial audio data, and 3D spatial model, unified multimodal fusion scene data is generated, ensuring the consistency of different modal information in time and space, thus avoiding perceptual misalignment problems. Based on this, virtual scene rendering processing is performed on multimodal fusion scene data to generate a presentable virtual scene, which is then output through a presentation device, allowing elderly users to receive visual and auditory information simultaneously, thus forming an immersive presentation process. This achieves a key transformation from raw scene data to an immersive virtual experience, which not only enhances the realism and continuity of the scene presentation but also strengthens the user's sense of immersion and participation, providing a high-quality perceptual environment for subsequent emotional interaction. It is an important foundational step in achieving emotional comfort.

[0130] During the immersive presentation process, feedback action and voice information of elderly users are collected to obtain feedback behavior data. The emotional state vector is then updated. Based on the updated emotional state vector, emotional feedback content is generated and output through an emotional comfort robot to provide emotional response and guidance to elderly users. The specific steps are as follows:

[0131] Step S71: During the immersive presentation process, the feedback action information and feedback voice information of the elderly user are collected and preprocessed to obtain real-time action processing data and real-time voice processing data.

[0132] Step S72: Based on the real-time action processing data and real-time speech processing data, construct a feedback behavior sequence, extract feedback behavior feature parameters, including action change amplitude parameters, speech emotion feature parameters and interaction response change parameters, to obtain feedback behavior data;

[0133] Step S73: Correct the original emotional state vector based on the feedback behavior data, adjust the emotion intensity parameter according to the feedback behavior feature parameter, and update the emotion change trend parameter according to the feedback behavior change within a continuous time window to obtain the updated emotional state vector.

[0134] Step S74: Input the updated emotional state vector into the preset emotional response model to generate emotional feedback content, which includes voice reply content, interactive guidance content, and scene adjustment suggestions;

[0135] Step S75: Generate corresponding output control instructions based on the emotional feedback content, and execute the output control instructions through the emotional comfort robot to output voice information and interactive prompts to the elderly user, thereby providing emotional response and guidance to the elderly user.

[0136] In practical applications, by continuously collecting feedback action and voice information from elderly users during the immersive experience and preprocessing it, real-time action and voice processing data are obtained. This allows the system to capture behavioral changes and voice expressions in real time during the user experience, reflecting the user's immediate response to the current virtual scene. By constructing feedback behavior sequences and extracting parameters such as action change amplitude, voice emotion features, and interaction response changes, a multi-dimensional characterization of user feedback behavior is achieved, enabling the system to identify subtle differences in user emotional changes and interaction states. Furthermore, by modifying the original emotional state vector based on feedback behavior data, dynamically adjusting the emotion intensity parameter, and updating the emotion change trend parameter based on changes within a continuous time window, the emotional state is transformed from a static description into a dynamic evolution process, thereby improving the real-time performance and accuracy of emotion modeling. The updated emotional state vector is input into a preset emotional response model to generate emotional feedback content, including voice replies, interactive guidance, and scenario adjustment suggestions. This enables the system to respond specifically to the current emotional state, providing voice feedback and interactive guidance to elderly users. The system can not only "perceive emotions" but also "respond to emotions," thus forming a complete emotional interaction loop. This achieves real-time tracking and dynamic adjustment of the user's emotional state, giving the system continuous perception and adaptive response capabilities. It effectively enhances the continuity, accuracy, and humanization of emotional interaction, thereby significantly improving the emotional comfort and user experience for elderly users.

[0137] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A remote interaction method for an emotional comfort robot for the elderly based on scene data collection, characterized in that, Includes the following steps: Historical information data of elderly users is obtained to construct a memory scene model, generate memory scene data, collect real-time scene data of the target scene through a scene acquisition terminal, and perform unified encoding processing on the memory scene data and real-time scene data to form a scene data set; Collect and parse the initial action and voice information of elderly users, extract operation response time parameters and interaction frequency parameters, and construct behavioral feature vectors. The initial voice information and behavioral feature vector are fused and analyzed to extract emotion category parameters, emotion intensity parameters, and emotion change trend parameters, and to construct an emotional state vector. Based on the emotional state vector, scene data set, and behavioral feature vector, a decision function is constructed to generate scene scheduling decision results and interaction rhythm control parameters. Based on the scene scheduling decision results and interaction rhythm control parameters, the scene acquisition terminal is controlled to adjust its movement path and acquisition angle, and the execution delay and prompt frequency of the control commands are adjusted to generate target scene data. The target scene data is fused and processed to generate a virtual scene, which is then output through a presentation device to form an immersive presentation process. During the immersive presentation process, feedback action information and feedback voice information of elderly users are collected to obtain feedback behavior data. The emotional state vector is updated, and emotional feedback content is generated based on the updated emotional state vector. The emotional feedback content is output through the emotional comfort robot to provide emotional response and guidance to the elderly users.

2. The remote interaction method for an elderly emotional comfort robot based on scene acquisition according to claim 1, characterized in that, The steps of acquiring historical information data of elderly users to construct a memory scene model, generating memory scene data, collecting real-time scene data of the target scene through a scene acquisition terminal, and uniformly encoding the memory scene data and the real-time scene data to form a scene data set are as follows: Obtain a historical information dataset of elderly users, which includes historical residential address data, social relationship-related location data, and location identification data corresponding to historical image data; The historical information dataset is subjected to structured parsing to extract scene elements, which include spatial location elements, environmental object elements, and temporal correlation elements. A memory scene model is constructed based on the scene elements, a three-dimensional spatial structure is generated based on the spatial location elements, entity objects in the scene are supplemented based on the environmental object elements, and the scene state is calibrated with time features based on the time association elements. The memory scene model is subjected to consistency verification. When there are missing scene elements, the missing parts are deduced and completed to generate memory scene data. The target scene is collected by the scene acquisition terminal to obtain real-time scene data, which includes panoramic image data, environmental audio data and spatial structure data. The real-time scene data is preprocessed, including data denoising, time synchronization, and unified spatial coordinate processing. The memory scene data and real-time scene data are uniformly encoded to construct a unified data representation structure, resulting in a scene data set.

3. The remote interaction method for an elderly emotional comfort robot based on scene acquisition according to claim 1, characterized in that, The steps of collecting and parsing the initial action and voice information of elderly users, extracting operation response time parameters and interaction frequency parameters, and constructing behavioral feature vectors are as follows: The system collects initial motion information and initial voice information of elderly users. The initial motion information includes head posture change information, limb movement information and position change information. The initial voice information includes voice signal data and voice interaction command data. The initial motion information is preprocessed, including data denoising, time alignment, and motion segmentation, to obtain standardized motion data; The initial speech information is preprocessed, including speech denoising, speech framing and speech recognition processing, to obtain speech parsing data; Based on the standardized action data and voice parsing data, the sequence of user interaction events is determined, and the trigger time and response time of each interaction event are recorded. Based on the trigger time and response time of each interactive event in the interactive event sequence, the operation response time parameter is calculated to characterize the reaction time of elderly users to system feedback. The interaction frequency parameter is calculated based on the number of interaction events occurring per unit time in the interaction event sequence, which is used to characterize the interaction activity level of elderly users. The operation response time parameter and interaction frequency parameter are normalized, and a behavior feature vector is constructed.

4. The remote interaction method for an elderly emotional comfort robot based on scene acquisition according to claim 3, characterized in that, The step of fusing and analyzing the initial speech information and behavioral feature vectors to extract emotion category parameters, emotion intensity parameters, and emotion change trend parameters, and constructing an emotional state vector, specifically includes: Speech features are extracted from the initial speech information to obtain speech feature parameters, which include fundamental frequency parameters, speech rate parameters, volume parameters, and pitch variation parameters. Based on the behavioral feature vector, behavioral feature parameters are extracted, including operation response time parameters and interaction frequency parameters. The speech feature parameters and behavioral feature parameters are time-aligned to construct a multimodal feature sequence. The multimodal feature sequence is then processed using a preset emotion recognition model to obtain the emotion discrimination result. Based on the emotion discrimination results, an emotion category parameter is determined to characterize the current emotion type of elderly users; Based on the aforementioned speech feature parameters and behavioral feature parameters, an emotion intensity parameter is calculated to characterize the magnitude of emotion changes. Based on the changes in the emotion intensity parameter within a continuous time window, an emotion change trend parameter is calculated to characterize the direction and rate of emotion change. The emotion category parameter, emotion intensity parameter, and emotion change trend parameter are combined to construct an emotion state vector.

5. The remote interaction method for an elderly emotional comfort robot based on scene acquisition according to claim 4, characterized in that, The step of constructing a decision function and generating scene scheduling decision results and interaction rhythm control parameters based on the emotional state vector, scene data set, and behavioral feature vector is as follows: The scene data set is labeled with features to generate scene feature labels, which include scene familiarity parameters, emotional relief parameters, social attribute parameters, and environmental stimulus intensity parameters. A scene matching model is constructed based on the emotional state vector and scene feature labels, and the matching score of each candidate scene is calculated. The candidate scenarios are sorted according to their matching scores, and the scenarios whose matching scores meet the preset conditions are selected as the target scenarios, generating scenario scheduling decision results. A rhythm control model is constructed based on the behavioral feature vector, and interactive rhythm control parameters are calculated. The interactive rhythm control parameters include control command execution delay parameters and interactive prompt frequency parameters.

6. The remote interaction method for an elderly emotional comfort robot based on scene acquisition according to claim 5, characterized in that, The steps of controlling the scene acquisition terminal to adjust its movement path and acquisition angle based on the scene scheduling decision results and interaction rhythm control parameters, and adjusting the execution delay and prompt frequency of control commands to generate target scene data, are as follows: Extract and parse the scene scheduling decision results to obtain the target scene identification information and the corresponding path planning parameters and view control parameters; Based on the path planning parameters, a movement path control sequence for the scene acquisition terminal is generated, and the scene acquisition terminal is controlled to move according to the movement path control sequence. Based on the aforementioned viewpoint control parameters, a data acquisition angle control command is generated, and the scene acquisition terminal is controlled to adjust its acquisition direction and viewpoint according to the data acquisition angle control command. During the movement of the scene acquisition terminal, panoramic image data, environmental audio data, and spatial structure data of the target scene are continuously acquired to form a real-time scene data stream; Based on the interaction rhythm control parameters, control command execution delay parameters and interaction prompt frequency parameters are extracted. The execution time of the movement path control sequence and acquisition angle control command is adjusted according to the control command execution delay parameters to match the operation response characteristics of elderly users. Based on the interactive prompt frequency parameter, a prompt control command is generated, and interactive guidance information is output at a preset frequency; The real-time scene data stream is synchronized and integrated to generate target scene data.

7. A remote interaction method for an elderly emotional comfort robot based on scene acquisition according to claim 6, characterized in that, The steps of fusing the target scene data to generate a virtual scene, and then outputting the virtual scene through a presentation device to form an immersive presentation process are as follows: Based on the target scene data, extract the panoramic image data, environmental audio data, and spatial structure data; The panoramic image data is processed, including image denoising, distortion correction and multi-view image stitching, to generate a panoramic image frame sequence; The environmental audio data is processed, including noise suppression, sound source localization, and spatial sound effect reconstruction, to generate spatial audio data. The spatial structure data is processed by spatial modeling to construct a three-dimensional spatial model of the scene, and the panoramic image frame sequence is spatially mapped. The panoramic image frame sequence, spatial audio data, and scene 3D spatial model are subjected to time-series synchronization processing to generate unified multimodal fusion scene data; Virtual scene rendering is performed based on the multimodal fused scene data to generate a virtual scene; The virtual scene is output through a presentation device, allowing elderly users to receive visual and auditory information corresponding to the virtual scene, thus creating an immersive presentation process.

8. The remote interaction method for an elderly emotional comfort robot based on scene acquisition according to claim 1, characterized in that, The steps involved in collecting feedback action and voice information from elderly users during the immersive presentation process to obtain feedback behavior data, updating the emotional state vector, generating emotional feedback content based on the updated emotional state vector, and outputting the emotional feedback content through an emotional comfort robot to provide emotional response and guidance to elderly users are as follows: During the immersive presentation process, feedback action information and feedback voice information of elderly users are collected and preprocessed to obtain real-time action processing data and real-time voice processing data; Based on the real-time motion processing data and real-time speech processing data, a feedback behavior sequence is constructed, and feedback behavior feature parameters are extracted. The feedback behavior feature parameters include motion change amplitude parameters, speech emotion feature parameters, and interaction response change parameters to obtain feedback behavior data. The original emotional state vector is corrected based on the feedback behavior data, the emotional intensity parameter is adjusted according to the feedback behavior feature parameters, and the emotional change trend parameter is updated according to the feedback behavior changes within a continuous time window to obtain the updated emotional state vector. The updated emotional state vector is input into a preset emotional response model to generate emotional feedback content, which includes voice response content, interactive guidance content, and scene adjustment suggestions. Based on the emotional feedback content, corresponding output control commands are generated. The emotional comfort robot executes the output control commands to output voice information and interactive prompts to the elderly user, providing emotional response and guidance.

Images